kubenetes从入门到上天系列第二十三篇：Kubernetes灰度发布

一灰度发布概念灰度发布也叫金丝雀发布。nginx.ingress.kubernetes.io/canary必须设置该 Annotation 值为true否则其它规则将不会生效。设置为true表示启用 canary 功能。设置为false表示不启用 canary 功能。nginx.ingress.kubernetes.io/canary-by-headernginx.ingress.kubernetes.io/canary-by-header表示基于请求头的名称进行灰度发布。请求头名称的特殊取值always无论什么情况下流量均会进入灰度服务。never无论什么情况下流量均不会进入灰度服务。若没有指定请求头名称的值则只要该头存在都会进行流量转发。nginx.ingress.kubernetes.io/canary-by-header-value表示基于请求头的值进行灰度发布。需要与canary-by-header头配合使用。nginx.ingress.kubernetes.io/canary-weight表示基于权重进行灰度发布。取值范围0~ 权重总值。若未设定总值默认总值为 100。nginx.ingress.kubernetes.io/canary-weight-total表示设定的权重总值。若未设定总值默认总值为 100。二根据服务权重的流量切分1需求如下假设当前线上环境您已经有一套服务 Service V1 对外提供 7 层服务此时上线了一些新的特性需要发布上线一个新的版本 Service V2。希望将百分之 10 的客户端请求转发到 Service V2 服务中。待运行一段时间稳定后可将所有的流量从 Service V1 切换到 Service V2 服务中再平滑地将 Service V1 服务下线。2实现思路如下在集群中部署两套系统一套是 v1 版本old-nginx一套是 v2 版本new-nginx两个版本都有自己的 service。定义两个 ingress 配置一个正常提供服务一个增加 canary 的 annotation。待 v2 版本无误后将其切换成 v2 版本并且将旧的版本下线流量全部接入新的 v2 版本。3创建旧的服务# old-nginx Service 配置 apiVersion: v1 kind: Service metadata: name: old-nginx namespace: default spec: selector: app: old-nginx ports: - name: http targetPort: 80 port: 80 --- # old-nginx Deployment 配置 apiVersion: apps/v1 kind: Deployment metadata: name: old-nginx namespace: default spec: replicas: 2 selector: matchLabels: app: old-nginx template: metadata: labels: app: old-nginx spec: containers: - name: nginx image: registry.cn-hangzhou.aliyuncs.com/acs-sample/old-nginx ports: - name: http containerPort: 80 --- # 主 Ingress 配置指向 old-nginx apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-old namespace: default spec: ingressClassName: nginx rules: - host: nginx.zyf.com http: paths: - path: / pathType: Prefix backend: service: name: old-nginx port: number: 804上线新的版本# new-nginx Service 配置 apiVersion: v1 kind: Service metadata: name: new-nginx namespace: default spec: selector: app: new-nginx ports: - name: http targetPort: 80 port: 80 --- # new-nginx Deployment 配置 apiVersion: apps/v1 kind: Deployment metadata: name: new-nginx namespace: default spec: replicas: 2 selector: matchLabels: app: new-nginx template: metadata: labels: app: new-nginx spec: containers: - name: nginx image: registry.cn-hangzhou.aliyuncs.com/acs-sample/new-nginx ports: - name: http containerPort: 80 --- # new-nginx 灰度 Ingress 配置10% 流量 apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-new namespace: default annotations: # 启用 Canary 灰度功能必须配置 nginx.ingress.kubernetes.io/canary: true # 配置 10% 流量转发到新版本 nginx.ingress.kubernetes.io/canary-weight: 10 nginx.ingress.kubernetes.io/canary-weight-total: 100 spec: ingressClassName: nginx rules: - host: nginx.zyf.com http: paths: - path: / pathType: Prefix backend: service: name: new-nginx port: number: 805查看一下ingress[rootk8s-node1 ingress]# kubectl get ingress NAME CLASS HOSTS ADDRESS PORTS AGE ingress-myapp-tls nginx tomcat.zyf.com 80, 443 48m ingress-new nginx nginx.zyf.com 80 4s ingress-old nginx nginx.zyf.com 80 9m [rootk8s-node1 ingress]#6查看效果C:\Users\JavaCfor /l %i in (1,1,10) do curl http://nginx.zyf.com C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old7将new的流量值设置为100此举是为了彻底切换到new流量上来。我们修改了新的权重之后apply之后流量变化如下C:\Users\JavaCfor /l %i in (1,1,1000) do curl http://nginx.zyf.com C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaCcurl http://nginx.zyf.com old ------------------------------------------------------- C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com new修改之后如下# new-nginx 灰度 Ingress 配置10% 流量 apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-new namespace: default annotations: # 启用 Canary 灰度功能必须配置 nginx.ingress.kubernetes.io/canary: true # 配置 10% 流量转发到新版本 nginx.ingress.kubernetes.io/canary-weight: 100 nginx.ingress.kubernetes.io/canary-weight-total: 100 spec: ingressClassName: nginx rules: - host: nginx.zyf.com http: paths: - path: / pathType: Prefix backend: service: name: new-nginx port: number: 80三基于客户端请求头的流量拆分1需求如下假设线上已运行了一套对外提供的七层 demo 应用此时开发了一些新的功能需要上线新版本 demo 应用但是又不想直接替换成新版本 demo 应用而是希望将请求头包含userkubesre的客户端请求转发到新版本 demo 应用中进行验证测试新版本 demo 应用等试验证通过并稳定后可将所有流量从老版本 demo 应用切换到新版本 demo 应用中再平滑地将老版本 demo 应用下线。2创建旧的服务# old-nginx Service 配置 apiVersion: v1 kind: Service metadata: name: old-nginx namespace: default spec: selector: app: old-nginx ports: - name: http targetPort: 80 port: 80 --- # old-nginx Deployment 配置 apiVersion: apps/v1 kind: Deployment metadata: name: old-nginx namespace: default spec: replicas: 2 selector: matchLabels: app: old-nginx template: metadata: labels: app: old-nginx spec: containers: - name: nginx image: registry.cn-hangzhou.aliyuncs.com/acs-sample/old-nginx ports: - name: http containerPort: 80 --- # 主 Ingress 配置指向 old-nginx apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-old namespace: default spec: ingressClassName: nginx rules: - host: nginx.zyf.com http: paths: - path: / pathType: Prefix backend: service: name: old-nginx port: number: 803上线新的版本# new-nginx Service 配置 apiVersion: v1 kind: Service metadata: name: new-nginx namespace: default spec: selector: app: new-nginx ports: - name: http targetPort: 80 port: 80 --- # new-nginx Deployment 配置 apiVersion: apps/v1 kind: Deployment metadata: name: new-nginx namespace: default spec: replicas: 2 selector: matchLabels: app: new-nginx template: metadata: labels: app: new-nginx spec: containers: - name: nginx image: registry.cn-hangzhou.aliyuncs.com/acs-sample/new-nginx ports: - name: http containerPort: 80 --- # new-nginx 灰度 Ingress 配置10% 流量 apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-new namespace: default annotations: nginx.ingress.kubernetes.io/canary: true nginx.ingress.kubernetes.io/canary-by-header: user nginx.ingress.kubernetes.io/canary-by-header-value: kubesre spec: ingressClassName: nginx rules: - host: nginx.zyf.com http: paths: - path: / pathType: Prefix backend: service: name: new-nginx port: number: 804查看效果C:\Users\JavaCcurl -H user: kubesre http://nginx.zyf.com new C:\Users\JavaCcurl http://nginx.zyf.com old C:\Users\JavaC四基于客户端来源ip进行流量切分1需求如下假设线上已运行了一套对外提供的七层 demo 应用此时开发了一些新的功能需要上线新版本 demo 应用又不想直接替换成新版本 demo 应用而是只希望公司内部人员能访问到新版本 demo 应用中进行测试验证新版本 demo 应用非公司内部人员访问还是访问到老版本应用中。等公司内部人员测试验证通过并稳定后可将所有流量从老版本 demo 应用切换到新版本 demo 应用中再平滑地将老版本 demo 应用下线。2实现原理也是通过请求头实现的。annotations: nginx.ingress.kubernetes.io/canary: true nginx.ingress.kubernetes.io/canary-by-header: X-Forwarded-For nginx.ingress.kubernetes.io/canary-by-header-value: 10.10.3.152这组注解实现了基于客户端 IP 的灰度发布只有来源 IP 为10.10.3.152的请求通过X-Forwarded-For请求头识别才会被转发到新版本 demo 应用其他 IP 仍访问老版本。