保姆级教程：用K8s原生方式部署Prometheus监控全家桶（附完整YAML）

Kubernetes原生部署Prometheus监控栈的深度实践指南在云原生技术栈中监控系统的构建一直是保障业务稳定性的关键环节。不同于简单的Helm一键安装本文将从Kubernetes原生资源的角度深入剖析如何通过DaemonSet、ConfigMap和ServiceAccount等核心API对象搭建一套生产级可用的Prometheus监控体系。这种部署方式不仅能帮助您全面掌握各组件的运行机制还能根据实际需求进行灵活定制。1. 环境准备与架构设计1.1 集群基础要求在开始部署前请确保您的Kubernetes集群满足以下条件Kubernetes版本 ≥ 1.18推荐1.22每个工作节点至少2核CPU和4GB内存已配置默认StorageClass如需持久化存储集群内DNS服务正常运作可以通过以下命令快速验证集群状态kubectl get nodes -o wide kubectl get cs1.2 监控架构设计我们采用的监控栈包含以下核心组件组件类型功能部署方式node-exporterDaemonSet采集节点级指标每个节点部署1个Podkube-state-metricsDeployment采集K8s资源状态集群范围部署prometheus-serverStatefulSet指标存储与告警支持水平扩展alertmanagerDeployment告警通知路由高可用部署提示生产环境建议为prometheus-server配置持久化存储避免历史数据丢失2. Node-Exporter的DaemonSet部署2.1 创建监控专用命名空间首先为监控组件创建独立的命名空间kubectl create ns monitoring2.2 精细化DaemonSet配置node-exporter需要访问主机系统信息因此需要特殊权限配置。以下是经过生产验证的DaemonSet模板apiVersion: apps/v1 kind: DaemonSet metadata: name: node-exporter namespace: monitoring labels: k8s-app: node-exporter spec: selector: matchLabels: k8s-app: node-exporter template: metadata: labels: k8s-app: node-exporter spec: hostNetwork: true hostPID: true hostIPC: true tolerations: - key: node-role.kubernetes.io/master operator: Exists effect: NoSchedule containers: - name: node-exporter image: prom/node-exporter:v1.8.0 args: - --path.procfs/host/proc - --path.sysfs/host/sys - --collector.filesystem.ignored-mount-points^/(sys|proc|dev|host|etc)($|/) ports: - containerPort: 9100 hostPort: 9100 volumeMounts: - name: proc mountPath: /host/proc - name: sys mountPath: /host/sys volumes: - name: proc hostPath: path: /proc - name: sys hostPath: path: /sys关键配置解析hostNetwork: 直接使用主机网络避免网络性能损耗hostPID/hostIPC: 访问主机进程和IPC命名空间tolerations: 允许在master节点运行volumeMounts: 挂载主机系统目录部署后验证kubectl -n monitoring get pods -l k8s-appnode-exporter curl http://node-ip:9100/metrics3. Prometheus Server的核心配置3.1 RBAC权限配置Prometheus需要访问Kubernetes API因此需要创建合适的ServiceAccount和ClusterRoleapiVersion: v1 kind: ServiceAccount metadata: name: prometheus namespace: monitoring --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: prometheus rules: - apiGroups: [] resources: - nodes - services - endpoints - pods verbs: [get, list, watch] - apiGroups: [extensions, networking.k8s.io] resources: - ingresses verbs: [get, list, watch] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: prometheus roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: prometheus subjects: - kind: ServiceAccount name: prometheus namespace: monitoring3.2 智能抓取配置管理通过ConfigMap管理Prometheus的抓取配置可以实现动态更新apiVersion: v1 kind: ConfigMap metadata: name: prometheus-config namespace: monitoring data: prometheus.yml: | global: scrape_interval: 30s evaluation_interval: 30s scrape_configs: - job_name: kubernetes-nodes kubernetes_sd_configs: - role: node relabel_configs: - source_labels: [__address__] regex: (.*):10250 replacement: ${1}:9100 target_label: __address__ - job_name: kubernetes-service-endpoints kubernetes_sd_configs: - role: endpoints relabel_configs: - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape] action: keep regex: true - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path] action: replace target_label: __metrics_path__ regex: (.)关键relabel配置说明将kubelet端口从10250重定向到9100通过service注解自动发现监控目标支持自定义metrics路径配置3.3 StatefulSet部署方案对于生产环境推荐使用StatefulSet部署PrometheusapiVersion: apps/v1 kind: StatefulSet metadata: name: prometheus namespace: monitoring spec: serviceName: prometheus replicas: 2 selector: matchLabels: app: prometheus template: metadata: labels: app: prometheus spec: serviceAccountName: prometheus containers: - name: prometheus image: prom/prometheus:v2.51.1 args: - --config.file/etc/prometheus/prometheus.yml - --storage.tsdb.path/data - --web.enable-lifecycle ports: - containerPort: 9090 volumeMounts: - name: config mountPath: /etc/prometheus - name: data mountPath: /data volumes: - name: config configMap: name: prometheus-config volumeClaimTemplates: - metadata: name: data spec: accessModes: [ ReadWriteOnce ] resources: requests: storage: 100Gi高可用设计要点使用volumeClaimTemplates实现持久化存储多个副本共享相同的配置通过--web.enable-lifecycle支持配置热加载4. 服务暴露与访问控制4.1 安全的Service暴露方式推荐使用Ingress配合Basic Auth暴露服务apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: prometheus namespace: monitoring annotations: nginx.ingress.kubernetes.io/auth-type: basic nginx.ingress.kubernetes.io/auth-secret: prometheus-auth spec: rules: - host: prometheus.example.com http: paths: - path: / pathType: Prefix backend: service: name: prometheus port: number: 9090创建认证secrethtpasswd -c auth admin kubectl -n monitoring create secret generic prometheus-auth --from-fileauth4.2 网络策略限制只允许特定命名空间访问PrometheusapiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-prometheus-access namespace: monitoring spec: podSelector: matchLabels: app: prometheus ingress: - from: - namespaceSelector: matchLabels: name: grafana ports: - protocol: TCP port: 90905. 高级配置与优化技巧5.1 长期存方案与Thanos或VictoriaMetrics集成实现长期存储# Thanos Sidecar配置示例 - name: thanos-sidecar image: thanosio/thanos:v0.35.0 args: - sidecar - --prometheus.urlhttp://localhost:9090 - --tsdb.path/data ports: - containerPort: 10901 volumeMounts: - name: data mountPath: /data5.2 资源限制与调优建议的资源限制配置组件CPU请求内存请求CPU限制内存限制node-exporter100m50Mi200m100Miprometheus24Gi48Gialertmanager1256Mi2512Miresources: requests: cpu: 2 memory: 4Gi limits: cpu: 4 memory: 8Gi5.3 定期配置检查清单为确保监控系统持续健康运行建议每月检查存储空间使用情况df -h /data抓取目标状态up指标告警规则有效性组件版本更新资源使用率监控在大型集群中我们曾通过优化relabel配置将Prometheus内存使用降低40%。关键是将不必要