CNN、RNN、Transformer 三大模型实战：从MNIST到IMDb的3个代码示例

CNN、RNN、Transformer 三大模型实战从MNIST到IMDb的3个代码示例1. 深度学习模型实战入门当你第一次接触深度学习时可能会被各种术语和概念搞得晕头转向。但事实上深度学习最吸引人的地方在于它的实践性——通过几行代码你就能让计算机学会识别手写数字、分析电影评论情感甚至预测股票走势。本文将带你用PyTorch实现三个经典模型分别处理图像和文本数据。深度学习模型的核心在于自动提取特征。与传统机器学习需要手动设计特征不同CNN通过卷积核自动学习图像特征RNN擅长处理序列数据而Transformer则通过自注意力机制捕捉长距离依赖关系。这三种架构构成了现代深度学习的基础。为什么选择PyTorch动态计算图更符合Python编程习惯丰富的预训练模型和工具库调试方便错误信息友好工业界和学术界都广泛采用在开始编码前确保安装以下环境pip install torch torchvision numpy pandas pip install torchtext0.6.0 # 确保IMDb数据集兼容性2. CNN实战MNIST手写数字识别2.1 数据准备与预处理MNIST包含70,000张28x28的灰度手写数字图像。我们使用torchvision自动下载并预处理import torch from torchvision import datasets, transforms transform transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ]) train_data datasets.MNIST(../data, trainTrue, downloadTrue, transformtransform) test_data datasets.MNIST(../data, trainFalse, transformtransform) train_loader torch.utils.data.DataLoader(train_data, batch_size64, shuffleTrue) test_loader torch.utils.data.DataLoader(test_data, batch_size1000, shuffleTrue)2.2 CNN模型构建下面是一个经典的LeNet-5变种import torch.nn as nn import torch.nn.functional as F class CNN(nn.Module): def __init__(self): super(CNN, self).__init__() self.conv1 nn.Conv2d(1, 32, 3, 1) self.conv2 nn.Conv2d(32, 64, 3, 1) self.dropout1 nn.Dropout2d(0.25) self.dropout2 nn.Dropout2d(0.5) self.fc1 nn.Linear(9216, 128) self.fc2 nn.Linear(128, 10) def forward(self, x): x self.conv1(x) x F.relu(x) x self.conv2(x) x F.relu(x) x F.max_pool2d(x, 2) x self.dropout1(x) x torch.flatten(x, 1) x self.fc1(x) x F.relu(x) x self.dropout2(x) x self.fc2(x) return F.log_softmax(x, dim1)2.3 训练与评估训练循环是深度学习的核心流程def train(model, device, train_loader, optimizer, epoch): model.train() for batch_idx, (data, target) in enumerate(train_loader): data, target data.to(device), target.to(device) optimizer.zero_grad() output model(data) loss F.nll_loss(output, target) loss.backward() optimizer.step() def test(model, device, test_loader): model.eval() test_loss 0 correct 0 with torch.no_grad(): for data, target in test_loader: data, target data.to(device), target.to(device) output model(data) test_loss F.nll_loss(output, target, reductionsum).item() pred output.argmax(dim1, keepdimTrue) correct pred.eq(target.view_as(pred)).sum().item() test_loss / len(test_loader.dataset) print(fTest set: Average loss: {test_loss:.4f}, Accuracy: {correct}/{len(test_loader.dataset)} ({100. * correct / len(test_loader.dataset):.0f}%)) device torch.device(cuda if torch.cuda.is_available() else cpu) model CNN().to(device) optimizer torch.optim.Adam(model.parameters()) for epoch in range(1, 11): train(model, device, train_loader, optimizer, epoch) test(model, device, test_loader)提示在Colab或Kaggle Notebook中运行时记得开启GPU加速。完整训练10个epoch后准确率通常能达到99%以上。3. RNN实战IMDb情感分析3.1 文本数据处理IMDb数据集包含50,000条电影评论标记为正面或负面评价。我们需要特殊处理文本数据import torchtext from torchtext.data import Field, LabelField, BucketIterator TEXT Field(tokenizespacy, lowerTrue, include_lengthsTrue) LABEL LabelField(dtypetorch.float) train_data, test_data torchtext.datasets.IMDB.splits(TEXT, LABEL) TEXT.build_vocab(train_data, max_size25000, vectorsglove.6B.100d, unk_inittorch.Tensor.normal_) LABEL.build_vocab(train_data) train_loader, test_loader BucketIterator.splits( (train_data, test_data), batch_size64, sort_within_batchTrue, sort_keylambda x: len(x.text), devicedevice )3.2 LSTM模型设计使用双向LSTM捕捉上下文信息class RNN(nn.Module): def __init__(self, vocab_size, embedding_dim, hidden_dim, output_dim, n_layers, bidirectional, dropout): super().__init__() self.embedding nn.Embedding(vocab_size, embedding_dim) self.rnn nn.LSTM(embedding_dim, hidden_dim, num_layersn_layers, bidirectionalbidirectional, dropoutdropout) self.fc nn.Linear(hidden_dim*2 if bidirectional else hidden_dim, output_dim) self.dropout nn.Dropout(dropout) def forward(self, text, text_lengths): embedded self.dropout(self.embedding(text)) packed_embedded nn.utils.rnn.pack_padded_sequence(embedded, text_lengths.to(cpu)) packed_output, (hidden, cell) self.rnn(packed_embedded) hidden self.dropout(torch.cat((hidden[-2,:,:], hidden[-1,:,:]), dim1)) return self.fc(hidden)3.3 训练技巧文本分类需要特别注意梯度裁剪model RNN(len(TEXT.vocab), 100, 256, 1, 2, True, 0.5) model.embedding.weight.data.copy_(TEXT.vocab.vectors) optimizer torch.optim.Adam(model.parameters()) criterion nn.BCEWithLogitsLoss() def train(model, iterator, optimizer, criterion): model.train() for batch in iterator: text, text_lengths batch.text optimizer.zero_grad() predictions model(text, text_lengths).squeeze(1) loss criterion(predictions, batch.label) loss.backward() torch.nn.utils.clip_grad_norm_(model.parameters(), 1) optimizer.step() def evaluate(model, iterator, criterion): model.eval() with torch.no_grad(): for batch in iterator: text, text_lengths batch.text predictions model(text, text_lengths).squeeze(1) loss criterion(predictions, batch.label)4. Transformer实战序列预测4.1 简易Transformer实现下面实现一个简化版Transformer用于序列预测class TransformerModel(nn.Module): def __init__(self, ntoken, ninp, nhead, nhid, nlayers, dropout0.5): super(TransformerModel, self).__init__() self.pos_encoder PositionalEncoding(ninp, dropout) encoder_layers nn.TransformerEncoderLayer(ninp, nhead, nhid, dropout) self.transformer_encoder nn.TransformerEncoder(encoder_layers, nlayers) self.encoder nn.Embedding(ntoken, ninp) self.ninp ninp self.decoder nn.Linear(ninp, ntoken) def forward(self, src, src_maskNone): src self.encoder(src) * math.sqrt(self.ninp) src self.pos_encoder(src) output self.transformer_encoder(src, src_mask) output self.decoder(output) return output class PositionalEncoding(nn.Module): def __init__(self, d_model, dropout0.1, max_len5000): super(PositionalEncoding, self).__init__() self.dropout nn.Dropout(pdropout) pe torch.zeros(max_len, d_model) position torch.arange(0, max_len, dtypetorch.float).unsqueeze(1) div_term torch.exp(torch.arange(0, d_model, 2).float() * (-math.log(10000.0) / d_model)) pe[:, 0::2] torch.sin(position * div_term) pe[:, 1::2] torch.cos(position * div_term) pe pe.unsqueeze(0).transpose(0, 1) self.register_buffer(pe, pe) def forward(self, x): x x self.pe[:x.size(0), :] return self.dropout(x)4.2 训练策略Transformer需要特殊的学习率调度ntokens len(TEXT.vocab) model TransformerModel(ntokens, 200, 2, 200, 2).to(device) criterion nn.CrossEntropyLoss() optimizer torch.optim.Adam(model.parameters(), lr0.001) scheduler torch.optim.lr_scheduler.StepLR(optimizer, 1.0, gamma0.95) def train(): model.train() total_loss 0 for batch, i in enumerate(range(0, train_data.size(0)-1, bptt)): data, targets get_batch(train_data, i) optimizer.zero_grad() output model(data) loss criterion(output.view(-1, ntokens), targets) loss.backward() torch.nn.utils.clip_grad_norm_(model.parameters(), 0.5) optimizer.step() total_loss loss.item() return total_loss / len(train_data)5. 模型对比与选择指南模型类型优势劣势适用场景CNN局部特征提取能力强参数共享减少计算量难以处理序列数据图像分类、物体检测RNN能处理变长序列记忆历史信息训练困难梯度消失/爆炸时间序列预测、文本生成Transformer并行计算高效长距离依赖建模内存消耗大需要大量数据机器翻译、语音识别选择建议图像数据优先考虑CNN变种ResNet、EfficientNet短文本或时间序列可以尝试LSTM或GRU长文本或需要捕捉全局关系时使用Transformer计算资源有限时可以考虑模型蒸馏或量化实际项目中我们常常组合多种架构。比如在视频分析中使用CNN提取帧特征再用LSTM处理时序关系或者在文档分类中用Transformer提取文本特征再接全连接层分类。