模型监控实战：性能追踪/数据漂移/自动告警

模型监控实战性能追踪/数据漂移/自动告警1. 监控指标模型监控指标 ├── 性能指标 │ ├── 准确率/精确率/召回率 │ ├── 延迟P50/P95/P99 │ └── 吞吐量QPS ├── 数据质量 │ ├── 缺失率 │ ├── 异常值比例 │ └── 特征分布变化 ├── 数据漂移 │ ├── 特征漂移Feature Drift │ ├── 标签漂移Label Drift │ └── 概念漂移Concept Drift └── 系统指标 ├── CPU/GPU 使用率 ├── 内存使用 └── 错误率2. 数据漂移检测fromscipy.statsimportks_2sampimportnumpyasnpdefdetect_drift(reference,current,threshold0.05):drift_report{}forcolinreference.columns:ifreference[col].dtypein[float64,int64]:stat,p_valueks_2samp(reference[col].dropna(),current[col].dropna())else:contingencypd.crosstab(reference[col],current[col])stat,p_value,_,_chi2_contingency(contingency)drift_report[col]{statistic:stat,p_value:p_value,drift_detected:p_valuethreshold,}returndrift_report3. 性能追踪importtimeimportnumpyasnpfromcollectionsimportdequeclassModelMonitor:def__init__(self,window_size1000):self.predictionsdeque(maxlenwindow_size)self.latenciesdeque(maxlenwindow_size)self.ground_truthdeque(maxlenwindow_size)deflog_prediction(self,prediction,latency,ground_truthNone):self.predictions.append(prediction)self.latencies.append(latency)ifground_truthisnotNone:self.ground_truth.append(ground_truth)defget_metrics(self):metrics{avg_latency:np.mean(self.latencies),p95_latency:np.percentile(self.latencies,95),prediction_count:len(self.predictions),}ifself.ground_truth:accuracynp.mean([ptforp,tinzip(self.predictions,self.ground_truth)])metrics[accuracy]accuracyreturnmetrics总结监控维度指标工具性能延迟/QPSPrometheus漂移KS/PSIEvidently质量缺失率/异常Great Expectations