bert-base-chinese预训练模型实战从数据准备到模型推理完整流程1. 模型简介与环境准备1.1 bert-base-chinese模型概述bert-base-chinese是Google发布的经典中文预训练语言模型基于Transformer架构专门针对中文文本优化。该模型在中文NLP任务中表现出色具有以下核心特点双向上下文理解通过Transformer编码器同时学习文本左右两侧的上下文信息预训练微调范式在大规模中文语料上预训练后可快速适配各类下游任务768维隐藏层每个中文字符可转换为768维的语义向量表示12层Transformer深度网络结构能捕捉复杂的语言特征1.2 环境配置与快速部署本镜像已预装所有依赖环境用户可通过简单命令快速启动# 进入模型目录 cd /root/bert-base-chinese # 运行演示脚本包含3个示例任务 python test.py镜像包含的关键组件模型文件pytorch_model.bin权重、config.json配置、vocab.txt词表演示脚本支持完型填空、语义相似度计算、特征提取三种功能依赖环境Python 3.8、PyTorch、HuggingFace Transformers库2. 数据准备与预处理2.1 训练数据格式要求典型的文本分类任务数据应包含两列feature文本内容如用户评论、新闻标题等label分类标签如正面/负面示例data.csvfeature,label 这款手机拍照效果很棒,正面 系统经常卡顿需要重启,负面2.2 数据预处理完整流程import pandas as pd from sklearn.preprocessing import LabelEncoder from sklearn.model_selection import train_test_split # 读取原始数据 data pd.read_csv(./data/data.csv, encodingutf-8) X data[feature] # 文本特征 y data[label].values # 标签 # 标签编码将文字标签转为数字 label_encoder LabelEncoder() y_encoded label_encoder.fit_transform(y) print(f共{len(label_encoder.classes_)}个分类) # 划分训练集和验证集9:1比例 X_train, X_val, y_train, y_val train_test_split( X, y_encoded, test_size0.1, random_state42)2.3 文本转换为BERT输入格式from transformers import BertTokenizer import torch tokenizer BertTokenizer.from_pretrained(/root/bert-base-chinese) def preprocess_for_bert(texts, labels): input_ids [] attention_masks [] for text in texts: encoded tokenizer.encode_plus( text, add_special_tokensTrue, max_length256, paddingmax_length, truncationTrue, return_attention_maskTrue ) input_ids.append(encoded[input_ids]) attention_masks.append(encoded[attention_mask]) return ( torch.tensor(input_ids), torch.tensor(attention_masks), torch.tensor(labels) ) # 处理训练集和验证集 train_inputs, train_masks, train_labels preprocess_for_bert(X_train, y_train) val_inputs, val_masks, val_labels preprocess_for_bert(X_val, y_val)3. 模型训练与微调3.1 构建训练数据加载器from torch.utils.data import TensorDataset, DataLoader, RandomSampler # 创建TensorDataset train_data TensorDataset(train_inputs, train_masks, train_labels) train_sampler RandomSampler(train_data) train_dataloader DataLoader(train_data, samplertrain_sampler, batch_size8) # 验证集同理 val_data TensorDataset(val_inputs, val_masks, val_labels) val_dataloader DataLoader(val_data, batch_size8)3.2 初始化模型与优化器from transformers import BertForSequenceClassification from transformers import AdamW, get_linear_schedule_with_warmup # 加载预训练模型 model BertForSequenceClassification.from_pretrained( /root/bert-base-chinese, num_labelslen(label_encoder.classes_), ignore_mismatched_sizesTrue ) # 设置优化器 optimizer AdamW(model.parameters(), lr2e-5, eps1e-8) # 学习率调度器 total_steps len(train_dataloader) * 5 # 假设训练5个epoch scheduler get_linear_schedule_with_warmup( optimizer, num_warmup_steps0, num_training_stepstotal_steps )3.3 训练循环实现import numpy as np def flat_accuracy(preds, labels): pred_flat np.argmax(preds, axis1).flatten() labels_flat labels.flatten() return np.sum(pred_flat labels_flat) / len(labels_flat) best_val_accuracy 0 for epoch in range(5): # 训练5轮 model.train() total_loss 0 for batch in train_dataloader: batch tuple(t.to(cuda) for t in batch) inputs, masks, labels batch model.zero_grad() outputs model(inputs, attention_maskmasks, labelslabels) loss outputs.loss total_loss loss.item() loss.backward() torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0) optimizer.step() scheduler.step() # 验证集评估 model.eval() val_accuracy 0 for batch in val_dataloader: batch tuple(t.to(cuda) for t in batch) inputs, masks, labels batch with torch.no_grad(): outputs model(inputs, attention_maskmasks) logits outputs.logits.detach().cpu().numpy() label_ids labels.to(cpu).numpy() val_accuracy flat_accuracy(logits, label_ids) avg_val_accuracy val_accuracy / len(val_dataloader) print(fEpoch {epoch1} | Val Acc: {avg_val_accuracy:.4f}) # 保存最佳模型 if avg_val_accuracy best_val_accuracy: best_val_accuracy avg_val_accuracy model.save_pretrained(./best_model)4. 模型推理与应用4.1 单条文本预测def predict(text, model, tokenizer, label_encoder): inputs tokenizer(text, return_tensorspt, paddingTrue, truncationTrue, max_length256) inputs {k:v.to(cuda) for k,v in inputs.items()} with torch.no_grad(): outputs model(**inputs) probs torch.nn.functional.softmax(outputs.logits, dim1) pred torch.argmax(probs, dim1) confidence torch.max(probs).item() return { label: label_encoder.inverse_transform([pred.item()])[0], confidence: confidence } # 示例预测 result predict(这个产品性价比很高, model, tokenizer, label_encoder) print(result) # 输出{label: 正面, confidence: 0.95}4.2 批量预测与结果保存import pandas as pd def batch_predict(input_file, output_file): df pd.read_csv(input_file) predictions [] confidences [] for text in df[feature]: res predict(text, model, tokenizer, label_encoder) predictions.append(res[label]) confidences.append(res[confidence]) df[pred_label] predictions df[confidence] confidences df.to_csv(output_file, indexFalse) # 批量预测示例 batch_predict(./data/new_data.csv, ./results/predictions.csv)5. 进阶应用与优化建议5.1 处理类别不平衡问题当某些类别样本过少时可采用以下策略# 过采样少数类示例 from sklearn.utils import resample minority_class data[data[label] 稀有类别] majority_class data[data[label] ! 稀有类别] minority_upsampled resample( minority_class, replaceTrue, n_sampleslen(majority_class), random_state42 ) balanced_data pd.concat([majority_class, minority_upsampled])5.2 学习率预热策略# 改进的优化器配置 optimizer AdamW(model.parameters(), lr5e-5, weight_decay0.01) scheduler get_linear_schedule_with_warmup( optimizer, num_warmup_steps100, # 前100步逐步提高学习率 num_training_stepstotal_steps )5.3 模型性能监控建议记录以下指标训练损失曲线验证准确率变化混淆矩阵分析各类别的精确率/召回率from sklearn.metrics import classification_report # 生成分类报告示例 y_true [...] # 真实标签 y_pred [...] # 预测标签 print(classification_report(y_true, y_pred))获取更多AI镜像想探索更多AI镜像和应用场景访问 CSDN星图镜像广场提供丰富的预置镜像覆盖大模型推理、图像生成、视频生成、模型微调等多个领域支持一键部署。