Kubernetes 简介一、什么是KubernetesKubernetes常简称为K8s是开源的生产级容器编排管理平台由Google基于内部Borg容器管理系统的十余年生产运维经验开源推出现已成为云原生领域的行业事实标准。其名称源自希腊语意为“舵手、领航员”缩写K8s是用数字8替代中间“ubernete”8个字母得来简洁易记。K8s核心定位是自动化部署、调度、扩缩容和运维容器化应用解决单机容器无法应对的集群化、规模化、高可用运维难题让复杂的分布式应用部署和管理更加简单、高效、稳定可适配本地服务器、私有云、公有云、混合云等各类基础设施环境具备极强的可移植性和扩展性。二、诞生背景与核心价值随着容器技术Docker的普及开发者可将应用及依赖打包为标准化容器实现环境一致性部署。但大规模落地后传统人工运维面临诸多痛点多服务器容器调度混乱、服务故障无法自动恢复、流量负载难以均衡、资源利用率低下、应用升级回滚风险高、无法适配业务弹性波动。Kubernetes的诞生彻底解决了以上问题摒弃人工手动运维模式采用声明式管理理念——用户只需定义应用的理想运行状态副本数、资源配额、网络策略等K8s会持续自动巡检、调优、修复确保集群运行状态始终贴合预期实现容器应用的全生命周期自动化管理。三、核心架构组成K8s集群采用经典的控制平面 工作节点二分架构模块化拆分职责保障集群高可用、高可靠运行整体架构清晰、扩展性强。1. 控制平面Master节点作为集群的“大脑”负责全局调度、资源管控、策略决策不运行业务容器核心组件包含kube-apiserver集群唯一API入口所有操作指令、资源配置均通过该组件交互支持鉴权、限流、校验是集群的核心通信枢纽。etcd分布式键值存储系统负责存储集群所有资源数据、配置信息和状态快照保证数据高可用、强一致性。kube-scheduler调度器根据节点资源使用率、调度策略、亲和性规则自动为新建容器分配最优工作节点。kube-controller-manager控制器合集持续监控集群资源状态处理节点故障、副本缺失、服务异常等场景维持集群稳态。cloud-controller-manager对接云厂商API实现云资源负载均衡、存储卷等与K8s集群的联动管理。2. 工作平面Node节点集群的“执行终端”负责实际运行业务容器每个节点均可部署业务负载核心组件包含kubelet节点代理接收控制平面指令管理本节点容器的创建、启动、停止、健康检查实时上报节点和容器状态。kube-proxy网络代理负责节点网络规则配置实现集群内部服务发现、负载均衡、流量转发。容器运行时负责容器的实际运行如Docker、containerd等是容器执行的底层依赖。四、核心资源与核心特性1. 核心基础资源K8s通过标准化资源对象定义业务和集群配置核心常用资源包括PodK8s最小调度单元一个Pod包含一个或多个紧密耦合的容器共享网络、存储资源是业务运行的基本载体。Deployment无状态应用管理资源用于部署微服务、Web服务等支持副本管理、滚动更新、版本回滚。StatefulSet有状态应用管理资源适配数据库、消息队列等需要稳定网络标识、持久化数据的业务。Service服务抽象为一组Pod提供统一访问入口实现集群内部负载均衡和服务发现屏蔽Pod动态变化的特性。PV/PVC存储抽象PV为集群持久化存储资源PVC为业务存储申请实现存储与业务解耦保障容器数据持久化。ConfigMap/Secret配置和密钥管理分别用于存储普通配置、密码、证书等敏感信息实现配置与镜像解耦。2. 核心能力特性自愈能力自动检测容器、节点故障故障容器自动重启、重建故障节点负载自动迁移杜绝单点故障。弹性扩缩容支持手动、自动扩缩容根据CPU、内存、自定义指标动态调整业务副本数适配业务流量峰值和低谷。滚动更新与回滚支持业务无感知滚动升级逐步替换旧版本Pod升级异常可一键回滚保障业务迭代零停机。资源精细化管理可为容器设置资源请求和上限合理分配集群算力、内存资源提升整体资源利用率避免资源抢占。多环境兼容与可移植一次打包、随处运行不受底层基础设施限制完美适配本地、公有云、私有云、混合云环境。丰富的生态扩展支持CRD自定义资源、插件扩展可对接监控、日志、告警、CI/CD、服务网格等各类云原生工具。五、主要应用场景Kubernetes广泛应用于互联网、金融、政务、物联网等各类行业核心场景如下微服务架构落地适配微服务拆分后的多服务、多实例部署实现服务统一编排、治理和运维。业务弹性伸缩场景电商大促、直播、活动营销等流量波动大的业务实现资源自动扩容缩容降本增效。高可用业务部署核心业务、核心系统的7×24小时稳定运行保障通过自愈、多副本机制杜绝业务中断。云原生持续交付对接CI/CD流水线实现代码提交、镜像构建、自动部署、版本迭代的全流程自动化。混合云与多云管理统一管控多平台基础设施屏蔽底层环境差异实现业务跨云迁移、容灾备份K8S集群环境搭建1 k8s 环境部署说明K8S中文官网 https://kubernetes.io/zh-cn/主机名ip角色reg.timinglee.org172.25.254.250harbor仓库k8s-master172.25.254.100masterk8s集群控制节点k8s-node1172.25.254.10workerk8s集群工作节点k8s-node2172.25.254.20workerk8s集群工作节点所有节点禁用selinux和防火墙,所有节点同步时间和解析,所有节点安装docker-ce,所有节点禁用swap注意注释掉/etc/fstab文件中的定义1.构建harbor镜像仓库[rootharbor ~]# cat /etc/yum.repos.d/docker.repo EOF [docker] name docker baseurl https://mirrors.aliyun.com/docker-ce/linux/rhel/9.8/x86_64/stable/ gpgcheck 0 EOF [rootharbor ~]# dnf install httpd createrepo-y [rootharbor ~]# mkdir /var/www/html/docker/ -p [rootharbor ~]# vim /etc/httpd/conf/httpd.conf Listen 4444 [rootharbor ~]# systemctl enable --now httpd [rootharbor ~]# dnf install docker-ce -downloadonly --destdir /mnt/ -y [rootharbor ~]# mv /mnt/*.rpm /var/www/html/docker/ [rootharbor ~]# createrepo -v /var/www/html/docker/ [rootharbor ~]# cat /etc/yum.repos.d/docker.repo EOF [docker] name docker baseurl http://172.25.254.250:4444/docker gpgcheck 0 EOF2.安装docker在harbor仓库节点[rootharbor ~]# dnf install docker-ce -y [rootharbor ~]# echo br_netfilter /etc/modules-load.d/docker_mod.conf [rootharbor ~]# modprobe -a br_netfilter [rootharbor ~]# vim /etc/sysctl.d/docker.conf net.bridge.bridge-nf-call-iptables 1 net.bridge.bridge-nf-call-ip6tables 1 net.ipv4.ip_forward 1 [rootharbor ~]# systemctl restart systemd-modules-load.service [rootharbor ~]# sysctl --system [rootharbor ~]# vim /lib/systemd/system/docker.service ExecStart/usr/bin/dockerd -H fd:// --containerd/run/containerd/containerd.sock --iptablestrue [rootharbor ~]# systemctl daemon-reload [rootharbor ~]# systemctl enable --now docker3.生成证书key[rootharbor ~]# mkdir /data/certs -p [rootharbor ~]# openssl req -newkey rsa:4096 \ -nodes -sha256 -keyout /data/certs/timinglee.org.key \ -addext subjectAltName DNS:reg.timinglee.org \ -x509 -days 365 -out /data/certs/timinglee.org.crt You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter ., the field will be left blank. ----- Country Name (2 letter code) [XX]:CN State or Province Name (full name) []:Shannxi Locality Name (eg, city) [Default City]:Xian Organization Name (eg, company) [Default Company Ltd]:kubernetes Organizational Unit Name (eg, section) []:harbor Common Name (eg, your name or your servers hostname) []:reg.timinglee.org Email Address []:admintiminglee.org4.编辑harbor配置文件[rootharbor ~]# tar zxf harbor-offline-installer-v2.5.4.tgz -C /opt/ [rootharbor ~]# cd /opt/harbor/ [rootharbor harbor]# ls common.sh harbor.v2.5.4.tar.gz harbor.yml.tmpl install.sh LICENSE prepare [rootharbor harbor]# cp harbor.yml.tmpl harbor.yml [rootharbor harbor]# vim harbor.yml hostname: reg.timinglee.org certificate: /data/certs/timinglee.org.crt private_key: /data/certs/timinglee.org.key harbor_admin_password: lee [rootharbor harbor]# ./install.sh --with-chartmuseum #编写启动脚本 [rootharbor ~]# vim /lib/systemd/system/harbor.service [Unit] Descriptionharbor with Docker Compose Documentationhttps://reg.timinglee.com/compose/ Afterdocker.service network-online.target Requiresdocker.service [Service] Typeoneshot WorkingDirectory/opt/harbor ExecStart/usr/bin/docker compose up -d ExecStop/usr/bin/docker compose down RemainAfterExityes [Install] WantedBymulti-user.target [rootharbor harbor]# docker compose down [rootharbor ~]# systemctl enable --now harbor Created symlink /etc/systemd/system/multi-user.target.wants/harbor.service → /usr/lib/systemd/system/harbor.service. [rootharbor harbor]# docker compose up -d [rootharbor harbor]# docker compose ps5.启动并验证[rootharbor harbor]# mkdir /etc/docker/certs.d/reg.timinglee.org/ -p [rootharbor harbor]# cp /data/certs/timinglee.org.crt /etc/docker/certs.d/reg.timinglee.org/ca.crt [rootharbor harbor]# vim /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 172.25.254.200 harbor reg.timinglee.org [rootharbor harbor]# systemctl restart docker [rootharbor harbor]# docker compose up -d [rootharbor harbor]# docker login reg.timinglee.org -u admin Password: WARNING! Your credentials are stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/go/credential-store/ Login Succeeded构建部署kubernetes所需其他主机1.所有主机配置关闭swapsystemctl disable --now swap.target systemctl mask swap.target sed /swap/s/^/#/g -i /etc/fstab2.配置可以使用harbor仓库安装docker所有主机mkdir /etc/docker/certs.d/reg.timinglee.org/ -p #在harbor主机中分发证书到所有主机 [rootharbor ~]# for i in 100 10 20 do scp /data/certs/timinglee.org.crt root172.25.250.$i:/etc/docker/certs.d/reg.timinglee.org/ca.crt done systemctl enable docker systemctl restart docker所有主机配置docker加速器cat /etc/docker/daemon.json EOF { registry-mirrors:[https://reg.timinglee.org] } EOF systemctl restart docker docker info所有主机彼此建立解析vim /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 172.25.254.100 master 172.25.254.10 node1 172.25.254.20 node2 172.25.254.250 reg.timinglee.org所有主机配置kubernetes安装源vim /etc/yum.repos.d/kubernetes.repo [kubernetes] name kubernetes baseurl https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.35/rpm/ gpgcheck 0 #检测 dnf list kubeletkubernetes的部署1.安装cri-dockerd#下载二进制包 [rootdocker-node1 ~]# wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.4.4/cri-dockerd-0.4.4.amd64.tgz #解压二进制压缩包 [rootdocker-node1 ~]# tar zxf cri-dockerd-0.4.4.amd64.tgz [rootdocker-node1 ~]# cd cri-dockerd/ [rootdocker-node1 cri-dockerd]# ls cri-dockerd cri-docker.service cri-docker.socket #安装cri-docker命令到系统 [rootdocker-node1 cri-dockerd]# install -o root -g root -m 0755 cri-dockerd /usr/local/bin/cri-dockerd [rootdocker-node1 cri-dockerd]# ls -l /usr/local/bin/cri-dockerd -rwxr-xr-x 1 root root 51638434 Aug 16 19:11 /usr/local/bin/cri-dockerd #生成启动文件 [rootdocker-node1 cri-dockerd]# cp cri-docker.s* /lib/systemd/system cp: overwrite /lib/systemd/system/cri-docker.service? y [rootdocker-node1 cri-dockerd]# chmod x /lib/systemd/system/cri-docker.s* [rootdocker-node1 cri-dockerd]# vim /lib/systemd/system/cri-docker.service [Service] Typenotify ExecStart/usr/local/bin/cri-dockerd --network-plugincni fd:// --pod-infra-container-imagereg.timinglee.org/k8s/pause:3.10.1 --container-runtime-endpoint ExecReload/bin/kill -s HUP $MAINPID TimeoutSec0 RestartSec2 Restartalways [rootdocker-node1 cri-dockerd]# systemctl daemon-reload #启动服务 [rootdocker-node1 cri-dockerd]# systemctl enable --now cri-docker.service #所有节点安装构建kubernetes 集群所需软 dnf install kubelet kubeadm kubectl -y systemctl enable --now kubelet.service #master节点中 kubectl 和kubeadm 补齐 [rootmaster ~]# echo source (kubectl completion bash) ~/.bashrc2.下载kubernetes集群所需镜[rootk8s-master ~]# kubeadm config images list I0820 09:44:45.514598 81700 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.35 registry.k8s.io/kube-apiserver:v1.35.7 registry.k8s.io/kube-controller-manager:v1.35.7 registry.k8s.io/kube-scheduler:v1.35.7 registry.k8s.io/kube-proxy:v1.35.7 registry.k8s.io/coredns/coredns:v1.13.1 registry.k8s.io/pause:3.10.1 registry.k8s.io/etcd:3.6.6-0 [rootmaster ~]# kubeadm config images pull \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.35.7 \ --cri-socketunix:///var/run/cri-dockerd.sock [config/images] Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.35.3 [config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.35.3 [config/images] Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.35.3 [config/images] Pulled registry.aliyuncs.com/google_containers/kube-proxy:v1.35.3 [config/images] Pulled registry.aliyuncs.com/google_containers/coredns:v1.13.1 [config/images] Pulled registry.aliyuncs.com/google_containers/pause:3.10.1 [config/images] Pulled registry.aliyuncs.com/google_containers/etcd:3.6.6-0 #上传镜像到本地harbor [rootmaster ~]# docker login reg.timinglee.org -u admin Password: WARNING! Your credentials are stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/go/credential-store/ Login Succeeded [rootmaster ~]# docker images --format {{.Repository}}:{{.Tag}} | awk -F / /google/{system(docker tag $0 reg.timinglee.org/k8s/$3)} [rootmaster ~]# docker images --format {{.Repository}}:{{.Tag}} | awk -F / /timinglee/{system(docker push $0)}3.在master中完成集群初始化[rootmaster ~]# kubeadm init --pod-network-cidr10.244.0.0/16 \ --image-repository reg.timinglee.org/k8s \ --kubernetes-version v1.35.7 \ --cri-socketunix:///var/run/cri-dockerd.sock 。。。 #每个人不一样其他主机加入本集群的凭证 kubeadm join 172.25.254.100:6443 --token tdwjoc.8d1yw3wl4r4tm6c4 \ --discovery-token-ca-cert-hash sha256:6b5950ef2cdba85d6dfdb564ee90d4187fa3d341767dc9852cbdd5c9dee4f927 #如果忘记 [rootmaster ~]# kubeadm token create --print-join-command kubeadm join 172.25.254.100:6443 --token jl4ztx.cax3iysvu7onsh5s --discovery-token-ca-cert-hash sha256:6b5950ef2cdba85d6dfdb564ee90d4187fa3d341767dc9852cbdd5c9dee4f927 #如果初始化出问题 [rootmaster ~]# kubeadm reset --cri-socketunix:///var/run/cri-dockerd.sock #可以重置集群设定 #添加kubernets环境变量到本机 [rootmaster ~]# echo export KUBECONFIG/etc/kubernetes/admin.conf ~/.bash_profile [rootmaster ~]# source ~/.bash_profile [rootmaster ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION master NotReady control-plane 102s v1.35.3 #添加node节点到本集群 [rootnode1 ~]# kubeadm join 172.25.254.100:6443 --token jl4ztx.cax3iysvu7onsh5s --discovery-token-ca-cert-hash sha256:6b5950ef2cdba85d6dfdb564ee90d4187fa3d341767dc9852cbdd5c9dee4f927 --cri-socketunix:///var/run/cri-dockerd.sock #测试 [rootmaster ~]# kubectl get nodes #可以看到集群中主机但是因为网络插件问题状态是NotReady NAME STATUS ROLES AGE VERSION master NotReady control-plane 5m16s v1.35.3 node1 NotReady none 66s v1.35.3 node2 NotReady none 58s v1.35.34.安装网络插件[rootk8s-master ~]# tar zxf kube-flannel-v0.28.9.tar.gz [rootk8s-master ~]# ls flannel kube-flannel-v0.28.9.tar.gz [rootk8s-master ~]# cd flannel/ [rootk8s-master flannel]# docker load -i kube-flannel-v0.28.9.tar Loaded image: ghcr.io/flannel-io/flannel-cni-plugin:v1.9.1-flannel3 Loaded image: ghcr.io/flannel-io/flannel:v0.28.9 [rootmaster ~]# docker tag ghcr.io/flannel-io/flannel-cni-plugin:v1.9.0-flannel1 reg.timinglee.org/flannel-io/flannel-cni-plugin:v1.9.0-flannel1 [rootmaster ~]# docker push reg.timinglee.org/flannel-io/flannel-cni-plugin:v1.9.0-flannel1 The push refers to repository [reg.timinglee.org/flannel-io/flannel-cni-plugin] 2c8aa52d4746: Pushed 5aa68bbbc67e: Pushed v1.9.0-flannel1: digest: sha256:b3d30c221113b30fea3e8a7fccb145e929b097d0319b9eeb6b5a591b10b5c671 size: 739 [rootmaster ~]# docker tag ghcr.io/flannel-io/flannel:v0.28.1 reg.timinglee.org/flannel-io/flannel:v0.28.1 [rootmaster ~]# docker push reg.timinglee.org/flannel-io/flannel:v0.28.1 The push refers to repository [reg.timinglee.org/flannel-io/flannel] 5668da16a30b: Pushed 5f70bf18a086: Pushed 00012e17b6cc: Pushed 9738bb9596cf: Pushed b6233bc105d7: Pushed 03767449b95b: Pushed 9a7d8cf9ff51: Pushed 13a60c10faeb: Pushed 70dc5e033175: Pushed 256f393e029f: Pushed v0.28.1: digest: sha256:e671adbc267460164555159210066d3304a43e3b5dd85cc0b5b6ad62e83aab52 size: 2414 [rootmaster ~]# vim kube-flannel.yml image: flannel-io/flannel:v0.28.1 image: flannel-io/flannel-cni-plugin:v1.9.0-flannel1 image: flannel-io/flannel:v0.28.1 [rootk8s-master flannel]# grep -n image: kube-flannel.yml #要改内容所在行的位置 148: image: flannel-io/flannel:v0.28.9 175: image: flannel-io/flannel-cni-plugin:v1.9.1-flannel3 186: image: flannel-io/flannel:v0.28.9 [rootmaster ~]# kubectl apply -f kube-flannel.yml namespace/kube-flannel created serviceaccount/flannel created clusterrole.rbac.authorization.k8s.io/flannel created clusterrolebinding.rbac.authorization.k8s.io/flannel created configmap/kube-flannel-cfg created daemonset.apps/kube-flannel-ds created #测试 [rootmaster ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION master Ready control-plane 13m v1.35.3 node1 Ready none 9m44s v1.35.3 node2 Ready none 9m36s v1.35.3 Kubernetes 简介及部署方法
Kubernetes 简介一、什么是KubernetesKubernetes常简称为K8s是开源的生产级容器编排管理平台由Google基于内部Borg容器管理系统的十余年生产运维经验开源推出现已成为云原生领域的行业事实标准。其名称源自希腊语意为“舵手、领航员”缩写K8s是用数字8替代中间“ubernete”8个字母得来简洁易记。K8s核心定位是自动化部署、调度、扩缩容和运维容器化应用解决单机容器无法应对的集群化、规模化、高可用运维难题让复杂的分布式应用部署和管理更加简单、高效、稳定可适配本地服务器、私有云、公有云、混合云等各类基础设施环境具备极强的可移植性和扩展性。二、诞生背景与核心价值随着容器技术Docker的普及开发者可将应用及依赖打包为标准化容器实现环境一致性部署。但大规模落地后传统人工运维面临诸多痛点多服务器容器调度混乱、服务故障无法自动恢复、流量负载难以均衡、资源利用率低下、应用升级回滚风险高、无法适配业务弹性波动。Kubernetes的诞生彻底解决了以上问题摒弃人工手动运维模式采用声明式管理理念——用户只需定义应用的理想运行状态副本数、资源配额、网络策略等K8s会持续自动巡检、调优、修复确保集群运行状态始终贴合预期实现容器应用的全生命周期自动化管理。三、核心架构组成K8s集群采用经典的控制平面 工作节点二分架构模块化拆分职责保障集群高可用、高可靠运行整体架构清晰、扩展性强。1. 控制平面Master节点作为集群的“大脑”负责全局调度、资源管控、策略决策不运行业务容器核心组件包含kube-apiserver集群唯一API入口所有操作指令、资源配置均通过该组件交互支持鉴权、限流、校验是集群的核心通信枢纽。etcd分布式键值存储系统负责存储集群所有资源数据、配置信息和状态快照保证数据高可用、强一致性。kube-scheduler调度器根据节点资源使用率、调度策略、亲和性规则自动为新建容器分配最优工作节点。kube-controller-manager控制器合集持续监控集群资源状态处理节点故障、副本缺失、服务异常等场景维持集群稳态。cloud-controller-manager对接云厂商API实现云资源负载均衡、存储卷等与K8s集群的联动管理。2. 工作平面Node节点集群的“执行终端”负责实际运行业务容器每个节点均可部署业务负载核心组件包含kubelet节点代理接收控制平面指令管理本节点容器的创建、启动、停止、健康检查实时上报节点和容器状态。kube-proxy网络代理负责节点网络规则配置实现集群内部服务发现、负载均衡、流量转发。容器运行时负责容器的实际运行如Docker、containerd等是容器执行的底层依赖。四、核心资源与核心特性1. 核心基础资源K8s通过标准化资源对象定义业务和集群配置核心常用资源包括PodK8s最小调度单元一个Pod包含一个或多个紧密耦合的容器共享网络、存储资源是业务运行的基本载体。Deployment无状态应用管理资源用于部署微服务、Web服务等支持副本管理、滚动更新、版本回滚。StatefulSet有状态应用管理资源适配数据库、消息队列等需要稳定网络标识、持久化数据的业务。Service服务抽象为一组Pod提供统一访问入口实现集群内部负载均衡和服务发现屏蔽Pod动态变化的特性。PV/PVC存储抽象PV为集群持久化存储资源PVC为业务存储申请实现存储与业务解耦保障容器数据持久化。ConfigMap/Secret配置和密钥管理分别用于存储普通配置、密码、证书等敏感信息实现配置与镜像解耦。2. 核心能力特性自愈能力自动检测容器、节点故障故障容器自动重启、重建故障节点负载自动迁移杜绝单点故障。弹性扩缩容支持手动、自动扩缩容根据CPU、内存、自定义指标动态调整业务副本数适配业务流量峰值和低谷。滚动更新与回滚支持业务无感知滚动升级逐步替换旧版本Pod升级异常可一键回滚保障业务迭代零停机。资源精细化管理可为容器设置资源请求和上限合理分配集群算力、内存资源提升整体资源利用率避免资源抢占。多环境兼容与可移植一次打包、随处运行不受底层基础设施限制完美适配本地、公有云、私有云、混合云环境。丰富的生态扩展支持CRD自定义资源、插件扩展可对接监控、日志、告警、CI/CD、服务网格等各类云原生工具。五、主要应用场景Kubernetes广泛应用于互联网、金融、政务、物联网等各类行业核心场景如下微服务架构落地适配微服务拆分后的多服务、多实例部署实现服务统一编排、治理和运维。业务弹性伸缩场景电商大促、直播、活动营销等流量波动大的业务实现资源自动扩容缩容降本增效。高可用业务部署核心业务、核心系统的7×24小时稳定运行保障通过自愈、多副本机制杜绝业务中断。云原生持续交付对接CI/CD流水线实现代码提交、镜像构建、自动部署、版本迭代的全流程自动化。混合云与多云管理统一管控多平台基础设施屏蔽底层环境差异实现业务跨云迁移、容灾备份K8S集群环境搭建1 k8s 环境部署说明K8S中文官网 https://kubernetes.io/zh-cn/主机名ip角色reg.timinglee.org172.25.254.250harbor仓库k8s-master172.25.254.100masterk8s集群控制节点k8s-node1172.25.254.10workerk8s集群工作节点k8s-node2172.25.254.20workerk8s集群工作节点所有节点禁用selinux和防火墙,所有节点同步时间和解析,所有节点安装docker-ce,所有节点禁用swap注意注释掉/etc/fstab文件中的定义1.构建harbor镜像仓库[rootharbor ~]# cat /etc/yum.repos.d/docker.repo EOF [docker] name docker baseurl https://mirrors.aliyun.com/docker-ce/linux/rhel/9.8/x86_64/stable/ gpgcheck 0 EOF [rootharbor ~]# dnf install httpd createrepo-y [rootharbor ~]# mkdir /var/www/html/docker/ -p [rootharbor ~]# vim /etc/httpd/conf/httpd.conf Listen 4444 [rootharbor ~]# systemctl enable --now httpd [rootharbor ~]# dnf install docker-ce -downloadonly --destdir /mnt/ -y [rootharbor ~]# mv /mnt/*.rpm /var/www/html/docker/ [rootharbor ~]# createrepo -v /var/www/html/docker/ [rootharbor ~]# cat /etc/yum.repos.d/docker.repo EOF [docker] name docker baseurl http://172.25.254.250:4444/docker gpgcheck 0 EOF2.安装docker在harbor仓库节点[rootharbor ~]# dnf install docker-ce -y [rootharbor ~]# echo br_netfilter /etc/modules-load.d/docker_mod.conf [rootharbor ~]# modprobe -a br_netfilter [rootharbor ~]# vim /etc/sysctl.d/docker.conf net.bridge.bridge-nf-call-iptables 1 net.bridge.bridge-nf-call-ip6tables 1 net.ipv4.ip_forward 1 [rootharbor ~]# systemctl restart systemd-modules-load.service [rootharbor ~]# sysctl --system [rootharbor ~]# vim /lib/systemd/system/docker.service ExecStart/usr/bin/dockerd -H fd:// --containerd/run/containerd/containerd.sock --iptablestrue [rootharbor ~]# systemctl daemon-reload [rootharbor ~]# systemctl enable --now docker3.生成证书key[rootharbor ~]# mkdir /data/certs -p [rootharbor ~]# openssl req -newkey rsa:4096 \ -nodes -sha256 -keyout /data/certs/timinglee.org.key \ -addext subjectAltName DNS:reg.timinglee.org \ -x509 -days 365 -out /data/certs/timinglee.org.crt You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter ., the field will be left blank. ----- Country Name (2 letter code) [XX]:CN State or Province Name (full name) []:Shannxi Locality Name (eg, city) [Default City]:Xian Organization Name (eg, company) [Default Company Ltd]:kubernetes Organizational Unit Name (eg, section) []:harbor Common Name (eg, your name or your servers hostname) []:reg.timinglee.org Email Address []:admintiminglee.org4.编辑harbor配置文件[rootharbor ~]# tar zxf harbor-offline-installer-v2.5.4.tgz -C /opt/ [rootharbor ~]# cd /opt/harbor/ [rootharbor harbor]# ls common.sh harbor.v2.5.4.tar.gz harbor.yml.tmpl install.sh LICENSE prepare [rootharbor harbor]# cp harbor.yml.tmpl harbor.yml [rootharbor harbor]# vim harbor.yml hostname: reg.timinglee.org certificate: /data/certs/timinglee.org.crt private_key: /data/certs/timinglee.org.key harbor_admin_password: lee [rootharbor harbor]# ./install.sh --with-chartmuseum #编写启动脚本 [rootharbor ~]# vim /lib/systemd/system/harbor.service [Unit] Descriptionharbor with Docker Compose Documentationhttps://reg.timinglee.com/compose/ Afterdocker.service network-online.target Requiresdocker.service [Service] Typeoneshot WorkingDirectory/opt/harbor ExecStart/usr/bin/docker compose up -d ExecStop/usr/bin/docker compose down RemainAfterExityes [Install] WantedBymulti-user.target [rootharbor harbor]# docker compose down [rootharbor ~]# systemctl enable --now harbor Created symlink /etc/systemd/system/multi-user.target.wants/harbor.service → /usr/lib/systemd/system/harbor.service. [rootharbor harbor]# docker compose up -d [rootharbor harbor]# docker compose ps5.启动并验证[rootharbor harbor]# mkdir /etc/docker/certs.d/reg.timinglee.org/ -p [rootharbor harbor]# cp /data/certs/timinglee.org.crt /etc/docker/certs.d/reg.timinglee.org/ca.crt [rootharbor harbor]# vim /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 172.25.254.200 harbor reg.timinglee.org [rootharbor harbor]# systemctl restart docker [rootharbor harbor]# docker compose up -d [rootharbor harbor]# docker login reg.timinglee.org -u admin Password: WARNING! Your credentials are stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/go/credential-store/ Login Succeeded构建部署kubernetes所需其他主机1.所有主机配置关闭swapsystemctl disable --now swap.target systemctl mask swap.target sed /swap/s/^/#/g -i /etc/fstab2.配置可以使用harbor仓库安装docker所有主机mkdir /etc/docker/certs.d/reg.timinglee.org/ -p #在harbor主机中分发证书到所有主机 [rootharbor ~]# for i in 100 10 20 do scp /data/certs/timinglee.org.crt root172.25.250.$i:/etc/docker/certs.d/reg.timinglee.org/ca.crt done systemctl enable docker systemctl restart docker所有主机配置docker加速器cat /etc/docker/daemon.json EOF { registry-mirrors:[https://reg.timinglee.org] } EOF systemctl restart docker docker info所有主机彼此建立解析vim /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 172.25.254.100 master 172.25.254.10 node1 172.25.254.20 node2 172.25.254.250 reg.timinglee.org所有主机配置kubernetes安装源vim /etc/yum.repos.d/kubernetes.repo [kubernetes] name kubernetes baseurl https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.35/rpm/ gpgcheck 0 #检测 dnf list kubeletkubernetes的部署1.安装cri-dockerd#下载二进制包 [rootdocker-node1 ~]# wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.4.4/cri-dockerd-0.4.4.amd64.tgz #解压二进制压缩包 [rootdocker-node1 ~]# tar zxf cri-dockerd-0.4.4.amd64.tgz [rootdocker-node1 ~]# cd cri-dockerd/ [rootdocker-node1 cri-dockerd]# ls cri-dockerd cri-docker.service cri-docker.socket #安装cri-docker命令到系统 [rootdocker-node1 cri-dockerd]# install -o root -g root -m 0755 cri-dockerd /usr/local/bin/cri-dockerd [rootdocker-node1 cri-dockerd]# ls -l /usr/local/bin/cri-dockerd -rwxr-xr-x 1 root root 51638434 Aug 16 19:11 /usr/local/bin/cri-dockerd #生成启动文件 [rootdocker-node1 cri-dockerd]# cp cri-docker.s* /lib/systemd/system cp: overwrite /lib/systemd/system/cri-docker.service? y [rootdocker-node1 cri-dockerd]# chmod x /lib/systemd/system/cri-docker.s* [rootdocker-node1 cri-dockerd]# vim /lib/systemd/system/cri-docker.service [Service] Typenotify ExecStart/usr/local/bin/cri-dockerd --network-plugincni fd:// --pod-infra-container-imagereg.timinglee.org/k8s/pause:3.10.1 --container-runtime-endpoint ExecReload/bin/kill -s HUP $MAINPID TimeoutSec0 RestartSec2 Restartalways [rootdocker-node1 cri-dockerd]# systemctl daemon-reload #启动服务 [rootdocker-node1 cri-dockerd]# systemctl enable --now cri-docker.service #所有节点安装构建kubernetes 集群所需软 dnf install kubelet kubeadm kubectl -y systemctl enable --now kubelet.service #master节点中 kubectl 和kubeadm 补齐 [rootmaster ~]# echo source (kubectl completion bash) ~/.bashrc2.下载kubernetes集群所需镜[rootk8s-master ~]# kubeadm config images list I0820 09:44:45.514598 81700 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.35 registry.k8s.io/kube-apiserver:v1.35.7 registry.k8s.io/kube-controller-manager:v1.35.7 registry.k8s.io/kube-scheduler:v1.35.7 registry.k8s.io/kube-proxy:v1.35.7 registry.k8s.io/coredns/coredns:v1.13.1 registry.k8s.io/pause:3.10.1 registry.k8s.io/etcd:3.6.6-0 [rootmaster ~]# kubeadm config images pull \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.35.7 \ --cri-socketunix:///var/run/cri-dockerd.sock [config/images] Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.35.3 [config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.35.3 [config/images] Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.35.3 [config/images] Pulled registry.aliyuncs.com/google_containers/kube-proxy:v1.35.3 [config/images] Pulled registry.aliyuncs.com/google_containers/coredns:v1.13.1 [config/images] Pulled registry.aliyuncs.com/google_containers/pause:3.10.1 [config/images] Pulled registry.aliyuncs.com/google_containers/etcd:3.6.6-0 #上传镜像到本地harbor [rootmaster ~]# docker login reg.timinglee.org -u admin Password: WARNING! Your credentials are stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/go/credential-store/ Login Succeeded [rootmaster ~]# docker images --format {{.Repository}}:{{.Tag}} | awk -F / /google/{system(docker tag $0 reg.timinglee.org/k8s/$3)} [rootmaster ~]# docker images --format {{.Repository}}:{{.Tag}} | awk -F / /timinglee/{system(docker push $0)}3.在master中完成集群初始化[rootmaster ~]# kubeadm init --pod-network-cidr10.244.0.0/16 \ --image-repository reg.timinglee.org/k8s \ --kubernetes-version v1.35.7 \ --cri-socketunix:///var/run/cri-dockerd.sock 。。。 #每个人不一样其他主机加入本集群的凭证 kubeadm join 172.25.254.100:6443 --token tdwjoc.8d1yw3wl4r4tm6c4 \ --discovery-token-ca-cert-hash sha256:6b5950ef2cdba85d6dfdb564ee90d4187fa3d341767dc9852cbdd5c9dee4f927 #如果忘记 [rootmaster ~]# kubeadm token create --print-join-command kubeadm join 172.25.254.100:6443 --token jl4ztx.cax3iysvu7onsh5s --discovery-token-ca-cert-hash sha256:6b5950ef2cdba85d6dfdb564ee90d4187fa3d341767dc9852cbdd5c9dee4f927 #如果初始化出问题 [rootmaster ~]# kubeadm reset --cri-socketunix:///var/run/cri-dockerd.sock #可以重置集群设定 #添加kubernets环境变量到本机 [rootmaster ~]# echo export KUBECONFIG/etc/kubernetes/admin.conf ~/.bash_profile [rootmaster ~]# source ~/.bash_profile [rootmaster ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION master NotReady control-plane 102s v1.35.3 #添加node节点到本集群 [rootnode1 ~]# kubeadm join 172.25.254.100:6443 --token jl4ztx.cax3iysvu7onsh5s --discovery-token-ca-cert-hash sha256:6b5950ef2cdba85d6dfdb564ee90d4187fa3d341767dc9852cbdd5c9dee4f927 --cri-socketunix:///var/run/cri-dockerd.sock #测试 [rootmaster ~]# kubectl get nodes #可以看到集群中主机但是因为网络插件问题状态是NotReady NAME STATUS ROLES AGE VERSION master NotReady control-plane 5m16s v1.35.3 node1 NotReady none 66s v1.35.3 node2 NotReady none 58s v1.35.34.安装网络插件[rootk8s-master ~]# tar zxf kube-flannel-v0.28.9.tar.gz [rootk8s-master ~]# ls flannel kube-flannel-v0.28.9.tar.gz [rootk8s-master ~]# cd flannel/ [rootk8s-master flannel]# docker load -i kube-flannel-v0.28.9.tar Loaded image: ghcr.io/flannel-io/flannel-cni-plugin:v1.9.1-flannel3 Loaded image: ghcr.io/flannel-io/flannel:v0.28.9 [rootmaster ~]# docker tag ghcr.io/flannel-io/flannel-cni-plugin:v1.9.0-flannel1 reg.timinglee.org/flannel-io/flannel-cni-plugin:v1.9.0-flannel1 [rootmaster ~]# docker push reg.timinglee.org/flannel-io/flannel-cni-plugin:v1.9.0-flannel1 The push refers to repository [reg.timinglee.org/flannel-io/flannel-cni-plugin] 2c8aa52d4746: Pushed 5aa68bbbc67e: Pushed v1.9.0-flannel1: digest: sha256:b3d30c221113b30fea3e8a7fccb145e929b097d0319b9eeb6b5a591b10b5c671 size: 739 [rootmaster ~]# docker tag ghcr.io/flannel-io/flannel:v0.28.1 reg.timinglee.org/flannel-io/flannel:v0.28.1 [rootmaster ~]# docker push reg.timinglee.org/flannel-io/flannel:v0.28.1 The push refers to repository [reg.timinglee.org/flannel-io/flannel] 5668da16a30b: Pushed 5f70bf18a086: Pushed 00012e17b6cc: Pushed 9738bb9596cf: Pushed b6233bc105d7: Pushed 03767449b95b: Pushed 9a7d8cf9ff51: Pushed 13a60c10faeb: Pushed 70dc5e033175: Pushed 256f393e029f: Pushed v0.28.1: digest: sha256:e671adbc267460164555159210066d3304a43e3b5dd85cc0b5b6ad62e83aab52 size: 2414 [rootmaster ~]# vim kube-flannel.yml image: flannel-io/flannel:v0.28.1 image: flannel-io/flannel-cni-plugin:v1.9.0-flannel1 image: flannel-io/flannel:v0.28.1 [rootk8s-master flannel]# grep -n image: kube-flannel.yml #要改内容所在行的位置 148: image: flannel-io/flannel:v0.28.9 175: image: flannel-io/flannel-cni-plugin:v1.9.1-flannel3 186: image: flannel-io/flannel:v0.28.9 [rootmaster ~]# kubectl apply -f kube-flannel.yml namespace/kube-flannel created serviceaccount/flannel created clusterrole.rbac.authorization.k8s.io/flannel created clusterrolebinding.rbac.authorization.k8s.io/flannel created configmap/kube-flannel-cfg created daemonset.apps/kube-flannel-ds created #测试 [rootmaster ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION master Ready control-plane 13m v1.35.3 node1 Ready none 9m44s v1.35.3 node2 Ready none 9m36s v1.35.3