用Python从零实现CRFBIO标注中的发射与转移矩阵实战解析在自然语言处理领域命名实体识别(NER)是信息抽取的基础任务之一。当我们第一次接触条件随机场(CRF)时那些复杂的公式和抽象的概率图模型常常让人望而生畏。本文将通过一个完整的Python实现案例带您直观理解CRF中最核心的两个概念发射矩阵(emission matrix)和转移矩阵(transition matrix)。1. 环境准备与数据构建首先确保已安装必要的Python库import numpy as np import torch from torch import nn import matplotlib.pyplot as plt我们构建一个极简的中文NER标注示例采用BIO标注体系# 样本数据句子和对应的BIO标签 sentences [[吃, 米饭], [喝, 汤]] labels [[O, B], [O, B]]BIO标注规则简单明了B实体开头(Begin)I实体内部(Inside)O非实体(Outside)2. CRF核心组件实现2.1 标签与特征映射首先建立标签与索引的双向映射tag2idx {B:0, I:1, O:2, START:3, END:4} idx2tag {v:k for k,v in tag2idx.items()} num_tags len(tag2idx)2.2 初始化转移矩阵转移矩阵定义了标签之间的转换概率# 随机初始化转移矩阵 transitions torch.randn(num_tags, num_tags, requires_gradTrue) # 添加约束B不能直接转B constraint_matrix torch.ones_like(transitions) constraint_matrix[tag2idx[B], tag2idx[B]] 0 # B→B禁止 constrained_transitions transitions * constraint_matrix2.3 构建发射矩阵发射矩阵表示从输入特征到标签的映射概率# 简单示例基于字符的one-hot编码 def char_to_vec(char): return torch.tensor([1 if c char else 0 for c in [吃,米,饭,喝,汤]], dtypetorch.float) # 随机初始化发射参数 emission_params torch.randn(5, num_tags, requires_gradTrue)3. 前向计算与损失函数3.1 序列得分计算定义计算序列得分的函数def sequence_score(emissions, tags, transitions): score torch.zeros(1) tags [tag2idx[START]] [tag2idx[t] for t in tags] [tag2idx[END]] for i in range(len(emissions)): # 发射得分 score emissions[i, tags[i1]] # 转移得分 score transitions[tags[i], tags[i1]] return score3.2 计算所有可能路径得分def total_score(emissions, transitions): # 使用动态规划高效计算 alpha torch.zeros(num_tags) alpha transitions[tag2idx[START]] emissions[0] for emission in emissions[1:]: alpha torch.logsumexp(alpha.unsqueeze(1) transitions emission, dim0) return torch.logsumexp(alpha transitions[:, tag2idx[END]], dim0)3.3 定义CRF损失def crf_loss(emissions, tags, transitions): gold_score sequence_score(emissions, tags, transitions) total total_score(emissions, transitions) return total - gold_score4. 训练与可视化4.1 训练过程optimizer torch.optim.SGD([transitions, emission_params], lr0.01) for epoch in range(100): total_loss 0 for sentence, tag_seq in zip(sentences, labels): # 准备发射分数 emissions torch.stack([emission_params char_to_vec(c) for c in sentence]) # 计算损失 loss crf_loss(emissions, tag_seq, constrained_transitions) total_loss loss.item() # 反向传播 optimizer.zero_grad() loss.backward() optimizer.step() print(fEpoch {epoch}, Loss: {total_loss/len(sentences)})4.2 矩阵可视化训练完成后我们可以可视化学习到的转移矩阵def plot_matrix(matrix, title): fig, ax plt.subplots() cax ax.matshow(matrix.detach().numpy()) fig.colorbar(cax) ax.set_xticks(range(num_tags)) ax.set_yticks(range(num_tags)) ax.set_xticklabels([idx2tag[i] for i in range(num_tags)]) ax.set_yticklabels([idx2tag[i] for i in range(num_tags)]) plt.title(title) plt.show() plot_matrix(constrained_transitions, Learned Transition Matrix)5. 解码与预测5.1 维特比解码实现def viterbi_decode(emissions, transitions): backpointers [] # 初始化 viterbi transitions[tag2idx[START]] emissions[0] backpointers.append(torch.argmax(viterbi, dim1)) # 递推 for emission in emissions[1:]: viterbi, backpointer torch.max(viterbi.unsqueeze(1) transitions emission, dim0) backpointers.append(backpointer) # 终止 best_score, best_tag torch.max(viterbi transitions[:, tag2idx[END]], dim0) # 回溯 best_path [best_tag.item()] for backpointer in reversed(backpointers): best_tag backpointer[best_tag] best_path.append(best_tag.item()) return list(reversed(best_path))[1:]5.2 预测示例test_sentence [喝, 可乐] emissions torch.stack([emission_params char_to_vec(c) for c in test_sentence]) best_path viterbi_decode(emissions, constrained_transitions) print(预测标签序列:, [idx2tag[idx] for idx in best_path])6. 工程实践中的优化技巧在实际项目中我们还需要考虑以下优化点特征工程除了字符本身可以加入词性、上下文窗口等特征批量处理实现批量化计算提升训练效率正则化添加L2正则防止过拟合学习率调度使用学习率衰减策略早停机制基于验证集性能提前终止训练# 示例添加L2正则化 def regularized_loss(emissions, tags, transitions, l2_lambda0.01): base_loss crf_loss(emissions, tags, transitions) l2_reg torch.norm(transitions, p2) torch.norm(emission_params, p2) return base_loss l2_lambda * l2_reg7. 扩展与进阶理解基础CRF实现后可以进一步探索BiLSTM-CRF结合神经网络自动学习特征表示BERT-CRF利用预训练语言模型提升性能半监督学习利用未标注数据提升模型泛化能力领域适应将通用NER模型迁移到特定领域# BiLSTM-CRF架构示意 class BiLSTM_CRF(nn.Module): def __init__(self, vocab_size, tag2idx): super().__init__() self.embedding nn.Embedding(vocab_size, 64) self.lstm nn.LSTM(64, 64//2, bidirectionalTrue) self.hidden2tag nn.Linear(64, len(tag2idx)) self.crf CRF(len(tag2idx)) def forward(self, x): embeds self.embedding(x) lstm_out, _ self.lstm(embeds.view(len(x), 1, -1)) emissions self.hidden2tag(lstm_out.view(len(x), -1)) return emissions
别再死记硬背CRF公式了！用Python手写一个BIO命名实体识别Demo，带你直观理解发射与转移矩阵
发布时间：2026/5/24 1:27:12
用Python从零实现CRFBIO标注中的发射与转移矩阵实战解析在自然语言处理领域命名实体识别(NER)是信息抽取的基础任务之一。当我们第一次接触条件随机场(CRF)时那些复杂的公式和抽象的概率图模型常常让人望而生畏。本文将通过一个完整的Python实现案例带您直观理解CRF中最核心的两个概念发射矩阵(emission matrix)和转移矩阵(transition matrix)。1. 环境准备与数据构建首先确保已安装必要的Python库import numpy as np import torch from torch import nn import matplotlib.pyplot as plt我们构建一个极简的中文NER标注示例采用BIO标注体系# 样本数据句子和对应的BIO标签 sentences [[吃, 米饭], [喝, 汤]] labels [[O, B], [O, B]]BIO标注规则简单明了B实体开头(Begin)I实体内部(Inside)O非实体(Outside)2. CRF核心组件实现2.1 标签与特征映射首先建立标签与索引的双向映射tag2idx {B:0, I:1, O:2, START:3, END:4} idx2tag {v:k for k,v in tag2idx.items()} num_tags len(tag2idx)2.2 初始化转移矩阵转移矩阵定义了标签之间的转换概率# 随机初始化转移矩阵 transitions torch.randn(num_tags, num_tags, requires_gradTrue) # 添加约束B不能直接转B constraint_matrix torch.ones_like(transitions) constraint_matrix[tag2idx[B], tag2idx[B]] 0 # B→B禁止 constrained_transitions transitions * constraint_matrix2.3 构建发射矩阵发射矩阵表示从输入特征到标签的映射概率# 简单示例基于字符的one-hot编码 def char_to_vec(char): return torch.tensor([1 if c char else 0 for c in [吃,米,饭,喝,汤]], dtypetorch.float) # 随机初始化发射参数 emission_params torch.randn(5, num_tags, requires_gradTrue)3. 前向计算与损失函数3.1 序列得分计算定义计算序列得分的函数def sequence_score(emissions, tags, transitions): score torch.zeros(1) tags [tag2idx[START]] [tag2idx[t] for t in tags] [tag2idx[END]] for i in range(len(emissions)): # 发射得分 score emissions[i, tags[i1]] # 转移得分 score transitions[tags[i], tags[i1]] return score3.2 计算所有可能路径得分def total_score(emissions, transitions): # 使用动态规划高效计算 alpha torch.zeros(num_tags) alpha transitions[tag2idx[START]] emissions[0] for emission in emissions[1:]: alpha torch.logsumexp(alpha.unsqueeze(1) transitions emission, dim0) return torch.logsumexp(alpha transitions[:, tag2idx[END]], dim0)3.3 定义CRF损失def crf_loss(emissions, tags, transitions): gold_score sequence_score(emissions, tags, transitions) total total_score(emissions, transitions) return total - gold_score4. 训练与可视化4.1 训练过程optimizer torch.optim.SGD([transitions, emission_params], lr0.01) for epoch in range(100): total_loss 0 for sentence, tag_seq in zip(sentences, labels): # 准备发射分数 emissions torch.stack([emission_params char_to_vec(c) for c in sentence]) # 计算损失 loss crf_loss(emissions, tag_seq, constrained_transitions) total_loss loss.item() # 反向传播 optimizer.zero_grad() loss.backward() optimizer.step() print(fEpoch {epoch}, Loss: {total_loss/len(sentences)})4.2 矩阵可视化训练完成后我们可以可视化学习到的转移矩阵def plot_matrix(matrix, title): fig, ax plt.subplots() cax ax.matshow(matrix.detach().numpy()) fig.colorbar(cax) ax.set_xticks(range(num_tags)) ax.set_yticks(range(num_tags)) ax.set_xticklabels([idx2tag[i] for i in range(num_tags)]) ax.set_yticklabels([idx2tag[i] for i in range(num_tags)]) plt.title(title) plt.show() plot_matrix(constrained_transitions, Learned Transition Matrix)5. 解码与预测5.1 维特比解码实现def viterbi_decode(emissions, transitions): backpointers [] # 初始化 viterbi transitions[tag2idx[START]] emissions[0] backpointers.append(torch.argmax(viterbi, dim1)) # 递推 for emission in emissions[1:]: viterbi, backpointer torch.max(viterbi.unsqueeze(1) transitions emission, dim0) backpointers.append(backpointer) # 终止 best_score, best_tag torch.max(viterbi transitions[:, tag2idx[END]], dim0) # 回溯 best_path [best_tag.item()] for backpointer in reversed(backpointers): best_tag backpointer[best_tag] best_path.append(best_tag.item()) return list(reversed(best_path))[1:]5.2 预测示例test_sentence [喝, 可乐] emissions torch.stack([emission_params char_to_vec(c) for c in test_sentence]) best_path viterbi_decode(emissions, constrained_transitions) print(预测标签序列:, [idx2tag[idx] for idx in best_path])6. 工程实践中的优化技巧在实际项目中我们还需要考虑以下优化点特征工程除了字符本身可以加入词性、上下文窗口等特征批量处理实现批量化计算提升训练效率正则化添加L2正则防止过拟合学习率调度使用学习率衰减策略早停机制基于验证集性能提前终止训练# 示例添加L2正则化 def regularized_loss(emissions, tags, transitions, l2_lambda0.01): base_loss crf_loss(emissions, tags, transitions) l2_reg torch.norm(transitions, p2) torch.norm(emission_params, p2) return base_loss l2_lambda * l2_reg7. 扩展与进阶理解基础CRF实现后可以进一步探索BiLSTM-CRF结合神经网络自动学习特征表示BERT-CRF利用预训练语言模型提升性能半监督学习利用未标注数据提升模型泛化能力领域适应将通用NER模型迁移到特定领域# BiLSTM-CRF架构示意 class BiLSTM_CRF(nn.Module): def __init__(self, vocab_size, tag2idx): super().__init__() self.embedding nn.Embedding(vocab_size, 64) self.lstm nn.LSTM(64, 64//2, bidirectionalTrue) self.hidden2tag nn.Linear(64, len(tag2idx)) self.crf CRF(len(tag2idx)) def forward(self, x): embeds self.embedding(x) lstm_out, _ self.lstm(embeds.view(len(x), 1, -1)) emissions self.hidden2tag(lstm_out.view(len(x), -1)) return emissions
