JavaWeb centos web开发 SQLMAP multithreading perl dynamic notifications transactions arm vue绑定事件 ps视频教程全集完整版 js获取月份 pythonset python命令大全 java获取当前年份 java中的队列 java中的数据结构 java得到当前时间 java日期格式 linux系统简介 华为交换机学习指南 ip切换软件 字符串分割成数组 python数组赋值 脚本编程 极限防守图 backtrack4 jsp源码 mmap文件怎么打开 抖音APP下载 贪吃蛇c语言代码 top命令详解 易语言tv su模型交错 ps怎么复制粘贴 下载网络电视 寒冰之王 在线转码 罗技鼠标怎么调灵敏度
当前位置: 首页 > 学习教程  > 编程语言

Chiron在K8S中的实践

2020/8/31 15:33:03 文章标签:

K8S容器编排

文章目录

    • 1:k8s集群的安装
      • 1.1 k8s的架构
      • 1.2:修改所有节点IP地址、主机名和host解析
      • 1.3:master节点安装etcd
      • 1.4:master节点安装kubernetes
      • 1.5:所有node节点安装kubernetes
      • 6:所有节点配置flannel网络
      • 7:配置master为镜像仓库
    • 2:什么是k8s,k8s有什么功能?
      • 2.1 k8s的核心功能
      • 2.2 k8s的历史
      • 2.3 k8s的安装方式
      • 2.4 k8s的应用场景
    • 3:k8s常用的资源
      • 3.1 创建pod资源
      • 3.2 ReplicationController资源
      • 3.3 service资源
      • 3.4 deployment资源
      • 3.5 Daemon Set资源
      • 3.6 Pet Set资源
      • 3.7 Job资源
      • 3.8 tomcat+mysql练习
    • 4:k8s的附加组件
      • 4.1 dns服务
      • 4.2 namespace命令空间
      • 4.3 健康检查
        • 4.3.1 探针的种类
        • 4.3.2 探针的检测方法
        • 4.3.3 liveness探针的exec使用:
        • 4.3.4 liveness探针的httpGet使用:
        • 4.3.5 liveness探针的tcpSocket使用:
        • 4.3.6 readiness探针的httpGet使用:
      • 4.4 dashboard服务
      • 4.5 通过apiservicer反向代理访问service
    • 5: k8s弹性伸缩
      • 5.1 安装heapster监控
      • 5.2 弹性伸缩
    • 6:持久化存储
      • 数据持久化类型:
      • 6.1 emptyDir:
      • 6.2 HostPath:
      • 6.3 nfs:
        • Kubernetes使用NFS共享存储有两种方式:
      • 6.4 pvc:
        • 6.4.1:创建pv和pvc
        • 6.4.2:创建mysql-rc,pod模板里使用volume:PVC
        • 6.4.3: 验证持久化
      • 6.5: 分布式存储glusterfs
      • 6.6 k8s 对接glusterfs存储
          • 扩展(不做演示):
    • 7:与jenkins集成实现cd
      • 7. 1 :将本地代码上传到码云Gitee的仓库中
      • 7.2 :安装jenkins,并自动构建docker镜像
        • 1:安装jenkins
        • 2:访问jenkins
        • 3:编写Dockerfile,进行本地测试,然后推送到Git仓库中
        • 4:配置jenkins拉取Git仓库代码凭据
        • 5:Jenkins创建自由风格的工程“yiliao”;拉取代码测试
        • 6:更改Jenkins配置中执行的shell命令;使其创建Docker镜像,并推送到私有仓库
        • 7:Jenkins使用参数化构建,自动构建docker镜像并上传到私有仓库
      • 7.3 手动部署实现“yiliao”的发布,升级,回滚;方面理解后面通过 Jenkins自动化部署应用
      • 7.4 jenkins自动部署应用到k8s
    • 8:部署Prometheus+Grafana
      • Node节点安装cadvisor和node-exporter
      • Master节点部署Prometheus+Grafana
  • 回滚到指定版本
  • Jenkins使用参数化构建回滚到指定版本;执行Shell语句为:(参数化构建回滚到指定版本)
  • 构建工程“yiliao_undo”;不填写构建参数使其回滚到上一版本
  • Web浏览器访问验证:标题为“ChironW-2医疗”
  • 再次构建工程“yiliao_undo”;填写构建参数指定回滚到“REVISION=3”版本
  • Web浏览器访问验证:标题为“ChironW医疗”
      • Master节点部署Prometheus+Grafana

1:k8s集群的安装

1.1 k8s的架构

在这里插入图片描述

除了核心组件,还有一些推荐的Add-ons:

组件名称 说明
kube-dns 负责为整个集群提供DNS服务
Ingress Controller 为服务提供外网入口
Heapster 提供资源监控
Dashboard 提供GUI
Federation 提供跨可用区的集群
Fluentd-elasticsearch 提供集群日志采集、存储与查询

1.2:修改所有节点IP地址、主机名和host解析

# 修改所有节点主机名
[root@chiron ~]# hostnamectl set-hostname k8s-master
[root@chiron ~]# hostnamectl set-hostname k8s-node-1
[root@chiron ~]# hostnamectl set-hostname k8s-node-2

# 所有节点需要做hosts解析
[root@chiron ~]# cat <<EOF>>/etc/hosts
10.0.0.11  k8s-master
10.0.0.12  k8s-node-1
10.0.0.13  k8s-node-2
EOF

1.3:master节点安装etcd

# yum方式安装Etcd
[root@k8s-master ~]# yum install etcd -y

# 备份Etcd主配置文件 
[root@k8s-master ~]# cp /etc/etcd/etcd.conf{,.bak}

# 修改Etcd主配置文件
[root@k8s-master ~]# vim /etc/etcd/etcd.conf
6行:ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
21行:ETCD_ADVERTISE_CLIENT_URLS="http://10.0.0.11:2379"

或者

[root@k8s-master ~]# grep -Ev '#|^$' /etc/etcd/etcd.conf 
[root@k8s-master ~]# sed -i 's#ETCD_LISTEN_CLIENT_URLS="http://localhost:2379"#ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"#g' /etc/etcd/etcd.conf
[root@k8s-master ~]# sed -i 's#ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379"#ETCD_ADVERTISE_CLIENT_URLS="http://10.0.0.11:2379"#g' /etc/etcd/etcd.conf
[root@k8s-master ~]# grep -Ev '#|^$' /etc/etcd/etcd.conf 

# 启动Etcd服务,并设置开机自启动
[root@k8s-master ~]# systemctl start etcd.service && systemctl enable etcd.service

# 创建一个Key并查看
[root@k8s-master ~]# etcdctl set testdir/testkey0 0 && etcdctl get testdir/testkey0

# 检查Etcd节点是否健康
[root@k8s-master ~]# etcdctl -C http://10.0.0.11:2379 cluster-health

etcd原生支持做集群,

作业1:安装部署etcd集群,要求三个节点

1.4:master节点安装kubernetes

# Master节点Yum方式安装K8S
[root@k8s-master ~]# yum install kubernetes-master.x86_64 -y

# 备份apiserver配置文件 
[root@k8s-master ~]# cp /etc/kubernetes/apiserver{,.bak}

# 修改apiserver配置文件 
[root@k8s-master ~]# vim /etc/kubernetes/apiserver 
8行:  KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
11行:KUBE_API_PORT="--port=8080"
14行: KUBELET_PORT="--kubelet-port=10250"
17行:KUBE_ETCD_SERVERS="--etcd-servers=http://10.0.0.11:2379"
23行:KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"

或者

[root@k8s-master ~]# grep -Ev '#|^$' /etc/kubernetes/apiserver
[root@k8s-master ~]# sed -i 's#KUBE_API_ADDRESS="--insecure-bind-address=127.0.0.1"#KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"#g' /etc/kubernetes/apiserver
[root@k8s-master ~]# sed -i 's/^# KUBE_API_PORT="--port=8080"/KUBE_API_PORT="--port=8080/g' /etc/kubernetes/apiserver
[root@k8s-master ~]# sed -i 's/^# KUBELET_PORT="--kubelet-port=10250"/KUBELET_PORT="--kubelet-port=10250"/g' /etc/kubernetes/apiserver
[root@k8s-master ~]# sed -i 's#KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:2379"#KUBE_ETCD_SERVERS="--etcd-servers=http://10.0.0.11:2379"#g' /etc/kubernetes/apiserver
[root@k8s-master ~]# sed -i 's#KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"#KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"#g' /etc/kubernetes/apiserver
[root@k8s-master ~]# grep -Ev '#|^$' /etc/kubernetes/apiserver

# 备份config配置文件 
[root@k8s-master ~]# cp /etc/kubernetes/config{,.bak}

# 修改config配置文件 
[root@k8s-master ~]# vim /etc/kubernetes/config
22行:KUBE_MASTER="--master=http://10.0.0.11:8080"

或者

[root@k8s-master ~]# grep -Ev '#|^$' /etc/kubernetes/config
[root@k8s-master ~]# sed -i 's#KUBE_MASTER="--master=http://127.0.0.1:8080"#KUBE_MASTER="--master=http://10.0.0.11:8080"#g' /etc/kubernetes/config
[root@k8s-master ~]# grep -Ev '#|^$' /etc/kubernetes/config

# 启动 apiserver、controller-manager、scheduler 服务并加入系统服务
[root@k8s-master ~]# systemctl enable kube-apiserver.service kube-controller-manager.service kube-scheduler.service
[root@k8s-master ~]# systemctl restart kube-apiserver.service kube-controller-manager.service kube-scheduler.service
[root@k8s-master ~]# systemctl status kube-apiserver.service kube-controller-manager.service kube-scheduler.service

检查服务是否安装正常

# 检查集群组件间运行状态是否正常
[root@k8s-master ~]# kubectl get componentstatus
NAME                 STATUS    MESSAGE             ERROR
etcd-0               Healthy   {"health":"true"}   
scheduler            Healthy   ok                  
controller-manager   Healthy   ok 

1.5:所有node节点安装kubernetes

# 所有Node节点Yum方式安装K8S
[root@k8s-node ~]# yum install kubernetes-node.x86_64 -y

# 备份config配置文件 
[root@k8s-node ~]# cp /etc/kubernetes/config{,.bak}

# 修改config配置文件
[root@k8s-node ~]# vim /etc/kubernetes/config 
22行:KUBE_MASTER="--master=http://10.0.0.11:8080"

或者

[root@k8s-node ~]# grep -Ev '^$|#' /etc/kubernetes/config
[root@k8s-node ~]# sed -i 's#KUBE_MASTER="--master=http://127.0.0.1:8080"#KUBE_MASTER="--master=http://10.0.0.11:8080"#g' /etc/kubernetes/config
[root@k8s-node ~]# grep -Ev '^$|#' /etc/kubernetes/config

# 备份kubelet配置文件 
[root@k8s-node ~]# cp /etc/kubernetes/kubelet{,.bak}

# 修改kubelet配置文件 
[root@k8s-node ~]# vim /etc/kubernetes/kubelet
5行:KUBELET_ADDRESS="--address=0.0.0.0"
8行:KUBELET_PORT="--port=10250"
11行:KUBELET_HOSTNAME="--hostname-override=10.0.0.12"
# 注意:此处的"hostname-override=10.0.0.x",要根据当前node节点的IP或主机名填写
14行:KUBELET_API_SERVER="--api-servers=http://10.0.0.11:8080"

或者

[root@k8s-node ~]# grep -Ev '^$|#' /etc/kubernetes/kubelet
[root@k8s-node ~]# sed -i 's#KUBELET_ADDRESS="--address=127.0.0.1"#KUBELET_ADDRESS="--address=0.0.0.0"#g' /etc/kubernetes/kubelet
[root@k8s-node ~]# sed -i 's/^# KUBELET_PORT="--port=10250"/KUBELET_PORT="--port=10250"/g' /etc/kubernetes/kubelet
[root@k8s-node ~]# sed -i 's#KUBELET_API_SERVER="--api-servers=http://127.0.0.1:8080"#KUBELET_API_SERVER="--api-servers=http://10.0.0.11:8080"#g' /etc/kubernetes/kubelet

# 注意此处"hostname-override=10.0.0.x",要根据当前node节点的IP或主机名填写
[root@k8s-node-1 ~]# sed -i 's#KUBELET_HOSTNAME="--hostname-override=127.0.0.1"#KUBELET_HOSTNAME="--hostname-override=k8s-node-1#g' /etc/kubernetes/kubelet
[root@k8s-node-2 ~]# sed -i 's#KUBELET_HOSTNAME="--hostname-override=127.0.0.1"#KUBELET_HOSTNAME="--hostname-override=k8s-node-2"#g' /etc/kubernetes/kubelet

[root@k8s-node ~]# grep -Ev '^$|#' /etc/kubernetes/kubelet

# 启动 kubelet.service、kube-proxy.service 服务并加入系统服务
[root@k8s-node ~]# systemctl enable kubelet.service kube-proxy.service
[root@k8s-node ~]# systemctl restart kubelet.service kube-proxy.service
[root@k8s-node ~]# systemctl status kubelet.service kube-proxy.service

在master节点检查Nodes节点状态是否正常

[root@k8s-master ~]# kubectl get nodes
NAME         STATUS    AGE
k8s-node-1   Ready     13s
k8s-node-2   Ready     16s

6:所有节点配置flannel网络

yum install flannel -y
sed -i 's#http://127.0.0.1:2379#http://10.0.0.11:2379#g' /etc/sysconfig/flanneld

##master节点:
# 向Etcd中创建一个值,为flannel网络指定网段;否则 falannel 网络插件服务启动失败
[root@k8s-master ~]# etcdctl mk /atomic.io/network/config '{ "Network": "172.18.0.0/16" }'

# 为Master主机安装Docker,方便后面将Master主机作为私有镜像仓库“registry”
[root@k8s-master ~]# yum install docker -y

# 将flanneld.service docker服务设置开机自启动,并重启master主机所有K8S相关服务
[root@k8s-master ~]# systemctl enable flanneld.service docker && systemctl restart flanneld.service docker
[root@k8s-master ~]# systemctl restart kube-apiserver.service kube-controller-manager.service kube-scheduler.service
[root@k8s-master ~]# systemctl status kube-apiserver.service kube-controller-manager.service kube-scheduler.service flanneld.service docker

##所有Node节点:
# 将flanneld.service docker服务启动;并添加到系统开机自启
systemctl enable flanneld.service && systemctl restart flanneld.service

# 查看Node主机"docker0"网卡IP网段;默认为"172.17.0.1/16"网段
[root@k8s-node-1 ~]# ip addr |grep docker0
    inet 172.17.0.1/16 scope global docker0

# 重启Docker等Node主机所有K8S相关服务;重启Docker目的在于查看"docker0"网卡网段的变化
systemctl  restart  docker kubelet.service kube-proxy.service
systemctl status flanneld.service ocker kubelet.service kube-proxy.service

# 再次查看Node主机"docker0"网卡IP网段;已经变为flannel指定网段
[root@k8s-node-1 ~]# ip addr |grep docker0
    inet 172.18.37.1/24 scope global docker0

##验证两台Node间Pod是否能够ping通
# 在两台Node主机中下载"busybox"镜像,各自启动一个Pod
docker pull busybox
# 在Node1中启动一个Pod,查看它的IP地址
[root@k8s-node-1 ~]# docker run -it docker.io/busybox:latest
/ # ip addr |grep eth0
    inet 172.18.37.2/24 scope global eth0
/ # 
# 在Node2启的Pod中ping Node1的Pod地址;发现网络 ping 不通,而 ping 其的网管确是正常的?
[root@k8s-node-2 ~]# docker run -it docker.io/busybox:latest
/ # ping 172.18.37.2
	PING 172.18.37.2 (172.18.37.2): 56 data bytes
	--- 172.18.37.2 ping statistics ---
	2 packets transmitted, 0 packets received, 100% packet loss
/ # ping 172.18.37.1
	PING 172.18.37.1 (172.18.37.1): 56 data bytes
	64 bytes from 172.18.37.1: seq=0 ttl=61 time=1.700 ms
# 这是因为当Docker服务启动时,会拉起 iptables 防火墙规则,其中"DROP"了"FORWARD"规则链;需要修改"docker.service"文件,使其更改为"ACCEPT";需要在所有节点(包括Master)操作。
# 备份"docker.service"文件
cp /usr/lib/systemd/system/docker.service{,.bak}
# 修改"docker.service"文件
vim /usr/lib/systemd/system/docker.service
#在[Service]区域下增加一行
ExecStartPost=/usr/sbin/iptables -P FORWARD ACCEPT
或者
sed -i 'N;12a\ExecStartPost=/usr/sbin/iptables -P FORWARD ACCEPT' /usr/lib/systemd/system/docker.service
# 参数解释:
# 在某行(指具体行号)前或后加一行内容
# sed -i 'N;4a\ddpdf' a.txt
# sed -i 'N;4i\eepdf' a.txt
# N;4a 代表在第四行之后加入"ddpdf";即"a"代表在之后添加一行数据
# N;4i 代表在第四行之后加入"eepdf";即"i"代表在之前添加一行数据

# 修改之后,需要重新加载"docker.service"文件,并进行重启
systemctl daemon-reload && systemctl restart docker
# 再次在Node2中Pod,ping Node中的Pod,网络正常
[root@k8s-node-2 ~]# docker attach 576a192fcc4b
/ # ping 172.18.37.2
	PING 172.18.37.2 (172.18.37.2): 56 data bytes
	64 bytes from 172.18.37.2: seq=0 ttl=60 time=1.549 ms

7:配置master为镜像仓库

# 因为我们不是使用HTTPS方式的访问;所以在"/etc/docker/daemon.json"文件中添加"insecure-registries"的意义是信任指定的私有仓库;否则会报如下错误:
# The push refers to a repository [10.0.0.11:5000/centos:6.10]
# Get https://10.0.0.11:5000/v1/_ping: http: server gave HTTP response to HTTPS client

# 所有节点
vi /etc/docker/daemon.json
{
"registry-mirrors": ["https://registry.docker-cn.com"],
"insecure-registries": ["10.0.0.11:5000"]
}
或者
[root@k8s-master ~]# cat >/etc/docker/daemon.json<<EOF
> {
> "registry-mirrors": ["https://registry.docker-cn.com"],
> "insecure-registries": ["10.0.0.11:5000"]
> }
> EOF

systemctl restart docker

# master节点部署registry私有镜像仓库(不带registry认证)
docker run -d -p 5000:5000 --restart=always --name registry -v /opt/myregistry:/var/lib/registry  registry

# master节点部署registry私有镜像仓库(带registry认证);此处不建议使用,否则后面"kubectl create"创建容器时,会pull不到镜像,除非事先pull到Nodes节点中;
# 比较推荐带认证的私有仓库为:Harbor仓库,且拥有Web管控台。
yum install httpd-tools -y
mkdir /opt/registry-var/auth/ -p
htpasswd -Bbn chiron 123456 >> /opt/registry-var/auth/htpasswd
docker run -d -p 5000:5000 --restart=always --name registry -v /opt/registry-var/auth/:/auth/ -v /opt/myregistry:/var/lib/registry -e "REGISTRY_AUTH=htpasswd" -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd registry
# 提示:上传下载镜像都需要登陆哦
docker login 10.0.0.11:5000
chiron
123456

docker logout 10.0.0.11:5000登出

# 镜像推送与拉取验证:
# 向私有仓库"registry"上传本地镜像tar包;存放到"/opt/images"目录下
[root@k8s-node-1 ~]# mkdir /opt/images && cd /opt/images
[root@k8s-node-1 ~]# wget http://192.168.2.254:8888/docker_nginx1.13.tar.gz
[root@k8s-node-1 ~]# docker load -i docker_nginx1.13.tar.gz 
[root@k8s-node-1 ~]# docker tag docker.io/nginx:1.13 10.0.0.11:5000/nginx:1.13
[root@k8s-node-1 ~]# docker images
[root@k8s-node-1 ~]# docker push 10.0.0.11:5000/nginx:1.13

# 前边我们pull过一个"busybox:latest"的镜像,这里一并打个标签上传到"registry"私有仓库,作为后续使用
[root@k8s-node-1 ~]# docker tag docker.io/busybox:latest 10.0.0.11:5000/busybox:latest
[root@k8s-node-1 ~]# docker push 10.0.0.11:5000/busybox:latest

# Registry私有镜像仓库查看镜像是否存在
[root@k8s-master nginx]# ls /opt/myregistry/docker/registry/v2/repositories/nginx/_manifests/tags/

# Web页面访问验证URL地址:
http://10.0.0.11:5000/v2/nginx/tags/list

# 拉取镜像
[root@k8s-node-2 ~]# docker images
[root@k8s-node-2 ~]# docker pull 10.0.0.11:5000/nginx:1.13
[root@k8s-node-2 ~]# docker images

2:什么是k8s,k8s有什么功能?

k8s是一个docker集群的管理工具

k8s是容器的编排工具

2.1 k8s的核心功能

自愈: 重新启动失败的容器,在节点不可用时,替换和重新调度节点上的容器,对用户定义的健康检查不响应的容器会被中止,并且在容器准备好服务之前不会把其向客户端广播。

弹性伸缩: 通过监控容器的cpu的负载值,如果这个平均高于80%,增加容器的数量,如果这个平均低于10%,减少容器的数量

服务的自动发现和负载均衡: 不需要修改您的应用程序来使用不熟悉的服务发现机制,Kubernetes 为容器提供了自己的 IP 地址和一组容器的单个 DNS 名称,并可以在它们之间进行负载均衡。

滚动升级和一键回滚: Kubernetes 逐渐部署对应用程序或其配置的更改,同时监视应用程序运行状况,以确保它不会同时终止所有实例。 如果出现问题,Kubernetes会为您恢复更改,利用日益增长的部署解决方案的生态系统。

私密配置文件管理. web容器里面,数据库的账户密码

2.2 k8s的历史

2014年 docker容器编排工具,立项

2015年7月 发布kubernetes 1.0, 加入cncf基金会 孵化

2016年,kubernetes干掉两个对手,docker swarm,mesos marathon 1.2版

2017年 1.5 -1.9

2018年 k8s 从cncf基金会 毕业项目

2019年: 1.13, 1.14 ,1.15,1.16

cncf :cloud native compute foundation

kubernetes (k8s): 希腊语 舵手,领航 容器编排领域,

谷歌15年容器使用经验,borg容器管理平台,使用golang重构borg,kubernetes

2.3 k8s的安装方式

yum安装 1.5 最容易安装成功,最适合学习的

源码编译安装—难度最大 可以安装最新版

二进制安装—步骤繁琐 可以安装最新版 shell,ansible,saltstack

kubeadm 安装最容易, 网络 可以安装最新版

minikube 适合开发人员体验k8s, 网络

2.4 k8s的应用场景

k8s最适合跑微服务项目!
微服务的好处:
提供更高的并发,可用性更强,发布周期更短
微服务的缺点:
管理难度大,自动化代码上线,Ansible,saltstack;
docker,docker管理平台K8S,弹性伸缩

拓展:
MVC:软件开发架构,同一域名,不同功能,放在不同目录下
微服务:软件开发架构,不同域名,不同功能,放在独立的站点

3:k8s常用的资源

3.1 创建pod资源

pod是最小资源单位.

任何的一个k8s资源都可以由yml清单文件来定义
pod资源:至少由两个容器组成,pod基础容器和业务容器组成(最多1+4)
yaml文件声明,至少启动一个容器,创建一个Pod资源
Kubectl调用docker启动了两个容器
nginx:13 业务容器;提供访问
Pod 容器 实现K8S的高级功能

k8s中yaml的主要组成:

apiVersion: v1  api版本
kind: pod   资源类型
metadata:   属性
spec:       详细

k8s_pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app: web
spec:
  containers:
    - name: nginx
      image: 10.0.0.11:5000/nginx:1.13
      ports:
        - containerPort: 80

master节点创建后面存放K8S各种资源Yaml文件的存放路径

[root@k8s-master ~]# mkdir -p /root/k8s_yaml/pod && cd /root/k8s_yaml/pod/
[root@k8s-master pod]# vim k8s_pod.yaml
[root@k8s-master pod]# kubectl create -f k8s_pod.yaml
[root@k8s-master pod]# kubectl get pod
# 注意:此处查看到的容器状态非“Running”

[root@k8s-master pod]# kubectl describe pod nginx
# 注意:此处查看到的Pod详情下方会报错如下:
# Error syncing pod, skipping: failed to "StartContainer" for "POD" with ErrImagePull: "image pull failed for registry.access.redhat.com/rhel7/pod-infrastructure:latest, this may be because there are no credentials on this request.  details: (open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory)"
# 解决方法见下方修改Node节点“/etc/kubernetes/kubelet”文件(此处是为了踩坑)
# 解决完上方踩坑问题,删除旧的pod“nginx”,重新创建一次,则运行正常
[root@k8s-master pod]# kubectl delete pods nginx
[root@k8s-master pod]# kubectl create -f k8s_pod.yaml
[root@k8s-master pod]# kubectl get pod

# 可以查看到Pod更多信息
[root@k8s-master pod]# kubectl get pod -o wide
[root@k8s-master pod]# kubectl describe pod nginx

# 查看到pod的IP后,CURL访问验证是否能访问到nginx首页
[root@k8s-master pod]# curl 172.18.95.2

# 查看为什么启动了两个容器,存在什么不同与关联
[root@k8s-node-2 ~]# docker ps -a
CONTAINER ID        IMAGE                                            COMMAND                  CREATED             STATUS                    PORTS               NAMES
fd8681f40db0        10.0.0.11:5000/nginx:1.13                        "nginx -g 'daemon ..."   5 minutes ago       Up 5 minutes                                  k8s_nginx.91390390_nginx_default_c69aed1e-1012-11ea-bb8d-000c29c6d194_48670ad4
b9aff18feced        10.0.0.11:5000/rhel7/pod-infrastructure:latest   "/pod"                   5 minutes ago       Up 5 minutes                                  k8s_POD.e5ea03c1_nginx_default_c69aed1e-1012-11ea-bb8d-000c29c6d194_20a42fd7
# 查看业务容器“nginx:1.13”的详细信息
[root@k8s-node-2 ~]# docker inspect fd8681f40db0 |tail -20
            "SandboxID": "",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": null,
            "SandboxKey": "",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {}
        }
    }
]
# 查看基础容器的信息
[root@k8s-node-2 ~]# docker inspect b9aff18feced |tail -20
            "MacAddress": "02:42:ac:12:5f:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "d8db8b8692bd9f06bf1e041c869191cd304d9337bcfad041b7354650359203ba",
                    "EndpointID": "568288f7bb2bbba16ce0d4ca903b7f6174d74a620380bcf2b46146f1210b0fe9",
                    "Gateway": "172.18.95.1",
                    "IPAddress": "172.18.95.2",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:12:5f:02"
                }
            }
        }
    }
]
# 此处看到 nginxPod 的网卡使用的基础容器:b9aff18feced 的网络资源
[root@k8s-node-2 ~]# docker inspect fd8681f40db0 |grep -i network
            "NetworkMode": "container:b9aff18feced757e44b5b4d5e58686523777d4779349c63da7d9cd654731137a",
        "NetworkSettings": {
            "Networks": {}

node节点:

# 向私有仓库"registry"上传本地镜像tar包;存放到"/opt/images"目录下
# 导入本地上传基础镜像
[root@k8s-node-1 images]# wget http://192.168.2.254:8888/pod-infrastructure-latest.tar.gz 
[root@k8s-node-1 images]# docker load -i pod-infrastructure-latest.tar.gz 
[root@k8s-node-1 images]# docker tag docker.io/tianyebj/pod-infrastructure:latest 10.0.0.11:5000/rhel7/pod-infrastructure:latest
[root@k8s-node-1 images]# docker push 10.0.0.11:5000/rhel7/pod-infrastructure:latest

# 所有Node节点修改配置文件;使其启动Pod时,从本地私有仓库"registry"拉取基础镜像
vim /etc/kubernetes/kubelet
	KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=10.0.0.11:5000/rhel7/pod-infrastructure:latest"

或者

grep -Ev '^$|#' /etc/kubernetes/kubelet
sed -i 's#KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"#KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=10.0.0.11:5000/rhel7/pod-infrastructure:latest"#g' /etc/kubernetes/kubelet
grep -Ev '^$|#' /etc/kubernetes/kubelet

# 重启kubelet服务,使配置生效
systemctl restart kubelet.service

pod配置文件2:k8s_pod2.yaml;目的是演示让一个基础Pod,带起两个业务容器(共同使用基础Pod的网络地址)

apiVersion: v1
kind: Pod
metadata:
  name: test
  labels:
    app: web
spec:
  containers:
    - name: nginx
      image: 10.0.0.11:5000/nginx:1.13
      ports:
        - containerPort: 80
    - name: busybox
      image: 10.0.0.11:5000/busybox:latest
      command: ["sleep","10000"]
# Master创建一个基础Pod,两个业务Pod的示例
[root@k8s-master pod]# kubectl create -f k8s_pod2.yaml
[root@k8s-master pod]# kubectl get pod -o wide
[root@k8s-master pod]# kubectl describe pod test

# Node节点查看容器;进行验证
[root@k8s-node-1 images]# docker ps -a

pod是k8s最小的资源单位

3.2 ReplicationController资源

rc:保证指定数量的pod始终存活,rc通过标签选择器来关联pod

k8s资源的常见操作:

# 根据yaml创建资源
kubectl   create  -f   xxx.yaml
# 查看资源状态信息:-o wide 显示更多 --show-labels 查看标签
kubectl   get  (pod / rc / nodes) -o wide --show-labels
# 查看资源从创建到异常的详细信息
kubectl  describe  pod  nginx
# 删除指定资源
kubectl  delete   pod  nginx   或者kubectl delete  -f  xxx.yaml
# 编辑指定资源的配置信息
kubectl  edit  pod  nginx

创建一个rc:nginx-rc.yaml

# 创建属于RC相关的yaml文件的存放目录
[root@k8s-master pod]# mkdir /root/k8s_yaml/rc && cd /root/k8s_yaml/rc
[root@k8s-master rc]# vim nginx-rc.yaml
apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx
spec:
  replicas: 5  #副本5
  selector:
    app: myweb
  template:
    metadata:
      labels:
        app: myweb
    spec:
      containers:
      - name: myweb
        image: 10.0.0.11:5000/nginx:1.13
        ports:
        - containerPort: 80

创建RC并查看RC副本数量状态

[root@k8s-master rc]# kubectl create -f nginx-rc.yaml 
	replicationcontroller "nginx" created
[root@k8s-master rc]# kubectl get rc
	NAME      DESIRED   CURRENT   READY     AGE
	nginx     5         5         5         43s
[root@k8s-master rc]# kubectl get pod

导入并推送“10.0.0.11:5000/nginx:1.15”镜像;方便下面进行升级与回滚演示

# 向私有仓库"registry"上传本地镜像tar包;存放到"/opt/images"目录下
# 导入本地上传基础镜像
rz docker_nginx1.15.tar.gz
[root@k8s-node-1 images]# docker load -i docker_nginx1.15.tar.gz
[root@k8s-node-1 images]# docker tag docker.io/nginx:latest 10.0.0.11:5000/nginx:1.15
[root@k8s-node-1 images]# docker push 10.0.0.11:5000/nginx:1.15

rc的滚动升级
新建一个nginx-rc1.15.yaml

apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx2
spec:
  replicas: 5  #副本5
  selector:
    app: myweb2
  template:
    metadata:
      labels:
        app: myweb2
    spec:
      containers:
      - name: myweb2
        image: 10.0.0.11:5000/nginx:1.15
        ports:
        - containerPort: 80

在这里插入图片描述

创建RC – nginx1.13版

[root@k8s-master rc]# kubectl create -f nginx-rc.yaml 
[root@k8s-master rc]# kubectl get pod
[root@k8s-master rc]# kubectl get rc

升级

# 滚动升级
[root@k8s-master rc]# kubectl rolling-update nginx -f nginx-rc1.15.yaml --update-period=10s

# 升级过程中可以观察Pod的变化
[root@k8s-master rc]# kubectl get pod -w

回滚

滚动回滚
[root@k8s-master rc]# kubectl rolling-update nginx2 -f nginx-rc.yaml --update-period=1s

参数解释与验证版本方法

参数解释:
	--update-period=10s  指定间隔时间,不指定则默认为一分钟
验证:
	方法一:访问启动的Nginx容器  curl -I IP:Port 查看返回信息中版本号
	方法二:在Node节点查看容器启动时使用的镜像  docker ps -a
	方法三:查看标签与RC的Name名称  kubectl get rc -o wide  或者  kubectl get pod -o wide --show-labels

删除其中某个Pod,验证RC是否会始终保证指定数量Pod存活

[root@k8s-master rc]# kubectl get pod
[root@k8s-master rc]# kubectl delete pod nginx-1r7pg
	pod "nginx-1r7pg" deleted
[root@k8s-master rc]# kubectl get pod

注意:
	当某个Pod被删除时,RC会立即启动一个新的Pod;但是被删除的Pod已经找不到;可以从Pod的名称和运行时间中查看验证到。
	当Node2节点服务挂掉,RC会将原本在Node2节点的Pod数量,重新在正常的节点运行相应数量的Pod;但是,当Node2节点恢复正常后,RC并不会重新将之前的Pod运行到Node2中。

删除其中某个Node节点,验证RC是否会始终保证指定数量Pod存活

[root@k8s-master rc]# kubectl get pod -o wide
[root@k8s-master rc]# kubectl get node
[root@k8s-master rc]# kubectl delete node k8s-node-2
[root@k8s-master rc]# kubectl get pod -o wide

[root@k8s-node-2 ~]# systemctl restart kubelet.service

[root@k8s-master rc]# kubectl get node
[root@k8s-master rc]# kubectl get pod -o wide

注意:
	当Node2节点服务挂掉,RC会将原本在Node2节点的Pod数量,重新在正常的节点运行相应数量的Pod;但是,当Node2节点恢复正常后,RC并不会重新将之前的Pod运行到Node2中。

修改我们之前创建的Pod – nginx 的标签属性为"myweb";验证RC是否会删除运行时间最小的Pod数量

[root@k8s-master rc]# kubectl get pod -o wide --show-labels 
    NAME          READY     STATUS    RESTARTS   AGE       IP            NODE         LABELS
    nginx         1/1       Running   0          2m        172.18.95.2   k8s-node-2   app=web
    nginx-ctvmk   1/1       Running   0          59m       172.18.37.6   k8s-node-1   app=myweb
    nginx-gpg4g   1/1       Running   0          1h        172.18.37.4   k8s-node-1   app=myweb
    nginx-hr791   1/1       Running   0          59m       172.18.37.7   k8s-node-1   app=myweb
    nginx-m5pwh   1/1       Running   0          2s        172.18.95.4   k8s-node-2   app=myweb
    nginx-q1rf7   1/1       Running   0          59m       172.18.37.3   k8s-node-1   app=myweb
[root@k8s-master rc]# kubectl edit pod nginx
	apiVersion: v1
	kind: Pod
	metadata:
	  creationTimestamp: 2019-11-26T09:49:10Z
      labels:
        app: myweb   #此处原本是:web;更改为:myweb 然后保存
      name: nginx
      namespace: default
[root@k8s-master rc]# kubectl get pod -o wide --show-labels 
    NAME          READY     STATUS    RESTARTS   AGE       IP            NODE         LABELS
    nginx         1/1       Running   0          2m        172.18.95.2   k8s-node-2   app=myweb
    nginx-ctvmk   1/1       Running   0          1h        172.18.37.6   k8s-node-1   app=myweb
    nginx-gpg4g   1/1       Running   0          1h        172.18.37.4   k8s-node-1   app=myweb
    nginx-hr791   1/1       Running   0          1h        172.18.37.7   k8s-node-1   app=myweb
    nginx-q1rf7   1/1       Running   0          1h        172.18.37.3   k8s-node-1   app=myweb
    
# 这里可以看到运行时间最小的Pod -- "nginx-m5pwh" 被删除了;
# 说明 rc通过标签选择器来关联pod;如果有别的Pod混进来了,RC会默认删除运行时间最小的Pod

3.3 service资源

service帮助pod暴露端口和做负载均衡

创建一个service:k8s_svc.yaml

# 查看现有Pod;此处只保留了RC创建的资源
[root@k8s-master rc]# kubectl get pod -o wide --show-labels 
    NAME          READY     STATUS    RESTARTS   AGE       IP            NODE         LABELS
    nginx-2wrjn   1/1       Running   0          3m        172.18.95.3   k8s-node-2   app=myweb
    nginx-9hfzl   1/1       Running   0          3m        172.18.37.2   k8s-node-1   app=myweb
    nginx-bshlt   1/1       Running   0          3m        172.18.37.3   k8s-node-1   app=myweb
    nginx-ltktb   1/1       Running   0          3m        172.18.95.2   k8s-node-2   app=myweb
    nginx-zngfd   1/1       Running   0          3m        172.18.95.4   k8s-node-2   app=myweb

# 创建属于Service相关的yaml文件的存放目录
[root@k8s-master rc]# mkdir /root/k8s_yaml/svc && cd /root/k8s_yaml/svc
[root@k8s-master svc]# vim k8s_svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: myweb
spec:
  type: NodePort  #ClusterIP
  ports:
    - port: 80          #clusterIP
      nodePort: 30000   #node port
      targetPort: 80    #pod port
  selector:
    app: myweb

创建service,查看SVC与RC的标签是否一致,并在外部浏览器进行访问验证

[root@k8s-master svc]# kubectl create -f k8s_svc.yaml 
    service "myweb" created
[root@k8s-master svc]# kubectl get svc -o wide
    NAME         CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE       SELECTOR
    kubernetes   10.254.0.1      <none>        443/TCP        1d        <none>
    myweb        10.254.218.48   <nodes>       80:30000/TCP   1m        app=myweb
[root@k8s-master svc]# kubectl get rc -o wide
    NAME      DESIRED   CURRENT   READY     AGE       CONTAINER(S)   IMAGE(S)                    SELECTOR
    nginx     5         5         5         19m       myweb          10.0.0.11:5000/nginx:1.13   app=myweb

# 浏览器访问:http://node_IP:30000;访问正常,出现Nginx的首页;
# 验证SVC实现负载的效果详情见下方。

#扩容或缩容 Deployment、ReplicaSet、Replication Controller或 Job 中Pod数量。

# 查看RC:nginx的Pod数量
[root@k8s-master svc]# kubectl get pod -o wide --show-labels 
    NAME          READY     STATUS    RESTARTS   AGE       IP            NODE         LABELS
    nginx-2wrjn   1/1       Running   0          3m        172.18.95.3   k8s-node-2   app=myweb
    nginx-9hfzl   1/1       Running   0          3m        172.18.37.2   k8s-node-1   app=myweb
    nginx-bshlt   1/1       Running   0          3m        172.18.37.3   k8s-node-1   app=myweb
    nginx-ltktb   1/1       Running   0          3m        172.18.95.2   k8s-node-2   app=myweb
    nginx-zngfd   1/1       Running   0          3m        172.18.95.4   k8s-node-2   app=myweb

# 修改rc:nginx的pod数量
[root@k8s-master svc]# kubectl scale rc nginx --replicas=2
	replicationcontroller "nginx" scaled
或者
[root@k8s-master svc]# kubectl edit rc nginx
    apiVersion: v1
    kind: ReplicationController
    metadata:
      creationTimestamp: 2019-11-23T06:51:57Z
      generation: 7
      labels:
        app: myweb
      name: nginx
      namespace: default
      resourceVersion: "84460"
      selfLink: /api/v1/namespaces/default/replicationcontrollers/nginx
      uid: be236c49-0dbd-11ea-9d96-000c29c6d194
    spec:
      replicas: 2   # 此处原本数值为“5”;现修改为“2”;然后观察RC创建的Pod数量
      selector:
        app: myweb

# 再次观察RC:nginx的Pod数量
[root@k8s-master svc]# kubectl get pod -o wide --show-labels 
    NAME          READY     STATUS    RESTARTS   AGE       IP            NODE         LABELS
    nginx-9hfzl   1/1       Running   0          28m       172.18.37.2   k8s-node-1   app=myweb
    nginx-bshlt   1/1       Running   0          28m       172.18.37.3   k8s-node-1   app=myweb
# 编辑现有svc:myweb的标签等配置信息
[root@k8s-master svc]# kubectl edit svc myweb
    apiVersion: v1
    kind: Service
    metadata:
      creationTimestamp: 2019-11-23T07:09:09Z
      name: myweb
      namespace: default
      resourceVersion: "84421"
      selfLink: /api/v1/namespaces/default/services/myweb
      uid: 24d4ec0a-0dc0-11ea-9d96-000c29c6d194
    spec:
      clusterIP: 10.254.198.29
      ports:
      - nodePort: 30000
        port: 80
        protocol: TCP
        targetPort: 80
      selector:
        app: myweb
      sessionAffinity: None
      type: NodePort
    status:
      loadBalancer: {}
    "/tmp/kubectl-edit-fplln.yaml" 26L, 656C written
    service "myweb" edited
# 修改剩余两个Pod的首页index.html;方便下面验证负载均衡
# Service默认只支持RR(轮询模式)
# Round-Robin(RR),轮询调度,通信中信道调度的一种策略,该调度策略使用户轮流使用共享资源,不会考虑瞬时信道条件。
[root@k8s-master svc]# kubectl  exec  -it nginx-9hfzl /bin/bash
root@nginx-9hfzl:/# echo Chiron-web01 > /usr/share/nginx/html/index.html 
root@nginx-9hfzl:/# exit
exit
[root@k8s-master svc]# kubectl  exec  -it nginx-bshlt /bin/bash
root@nginx-bshlt:/# echo Chiron-web02 > /usr/share/nginx/html/index.html 
root@nginx-bshlt:/# exit
exit

# 验证
[root@k8s-master svc]# curl http://10.0.0.12:30000/
	Chiron-web02
[root@k8s-master svc]# curl http://10.0.0.12:30000/
	Chiron-web01

# 查看指定SVC(service)的详细信息;如VIP、Pod、Port信息等
[root@k8s-master svc]# kubectl describe svc myweb
    Name:                   myweb
    Namespace:              default
    Labels:                 <none>
    Selector:               app=myweb
    Type:                   NodePort
    IP:                     10.254.218.48
    Port:                   <unset> 80/TCP
    NodePort:               <unset> 30000/TCP
    Endpoints:              172.18.37.2:80,172.18.37.3:80
    Session Affinity:       None
    No events.

扩展nodePort范围;默认支持的范围为"30000-32767",在大型网络架构中,port端口不够分配可以修改此值

[root@k8s-master svc]# vim  /etc/kubernetes/apiserver
KUBE_API_ARGS="--service-node-port-range=3000-50000"
或者
[root@k8s-master svc]# grep -Ev '^$|#' /etc/kubernetes/apiserver
[root@k8s-master svc]# sed -i 's/KUBE_API_ARGS=""/KUBE_API_ARGS="--service-node-port-range=3000-50000"/g' /etc/kubernetes/apiserver
[root@k8s-master svc]# grep -Ev '^$|#' /etc/kubernetes/apiserver
[root@k8s-master svc]# systemctl restart kube-apiserver.service 

命令行创建service资源暴露随机端口方法:

kubectl expose rc nginx --type=NodePort --port=80
kubectl get service

service默认使用iptables来实现负载均衡, k8s 1.8以后的新版本中推荐使用lvs(四层负载均衡 传输层tcp,udp)

3.4 deployment资源

有rc在滚动升级之后,会造成服务访问中断,于是k8s引入了deployment资源;
造成RC在滚动升级之后访问中断的原因是:
RC滚动升级时,要保证RC定义的名称和标签等值不能一致;
这样就会造成滚动升级之后,原有的SVC与新的RC之间标签不一致;
这样SVC和Pod之间就没有办法进行关联;用户访问自然就中断了。
Deployment资源相比RC资源的好处:
升级服务不中断;
不依赖于Yaml文件;
通过改配置文件直接进行升级;
RC标签选择器,不支持模糊匹配;
Deployment标签选择器,支持模糊匹配

创建一个Deployment:k8s_svc.yaml

# 创建属于Deployment相关的yaml文件的存放目录
[root@k8s-master svc]# mkdir /root/k8s_yaml/deploy && cd /root/k8s_yaml/deploy
[root@k8s-master deploy]# vim k8s_deploy.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: 10.0.0.11:5000/nginx:1.13
        ports:
        - containerPort: 80
        resources:  
          limits:
            cpu: 100m
          requests:
            cpu: 100m
# 创建Deploy资源
[root@k8s-master deploy]# kubectl create -f k8s_deploy.yaml 
	deployment "nginx-deployment" created

Deployment资源关联Service资源实现负载均衡

# 查看所有资源
[root@k8s-master deploy]# kubectl get all
    NAME                      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/nginx-deployment   3         3         3            3           6m

    NAME       DESIRED   CURRENT   READY     AGE
    rc/nginx   5         5         5         41m

    NAME             CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
    svc/kubernetes   10.254.0.1      <none>        443/TCP        1d
    svc/myweb        10.254.218.48   <nodes>       80:30000/TCP   3h

    NAME                             DESIRED   CURRENT   READY     AGE
rs/nginx-deployment-2807576163   3         3         3         6m

    NAME                                   READY     STATUS    RESTARTS   AGE
    po/nginx-3xb2r                         1/1       Running   0          41m
    po/nginx-c8g2g                         1/1       Running   0          41m
po/nginx-deployment-2807576163-2np1q   1/1       Running   0          6m
po/nginx-deployment-2807576163-5r55r   1/1       Running   0          6m
po/nginx-deployment-2807576163-nsntt   1/1       Running   0          6m
    po/nginx-tq9j1                         1/1       Running   0          41m
    po/nginx-w3d06                         1/1       Running   0          41m
    po/nginx-xln8n                         1/1       Running   0          41m

# 为Deployment资源:nginx-deployment;创建service资源并暴露随机端口
[root@k8s-master deploy]# kubectl expose deployment nginx-deployment --port=80 --target-port=80 --type=NodePort
	service "nginx-deployment" exposed

# 再次查看所有资源
[root@k8s-master deploy]# kubectl get all
    NAME                      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/nginx-deployment   3         3         3            3           15m

    NAME       DESIRED   CURRENT   READY     AGE
    rc/nginx   5         5         5         50m

    NAME                   CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
    svc/kubernetes         10.254.0.1       <none>        443/TCP        1d
    svc/myweb              10.254.218.48    <nodes>       80:30000/TCP   4h
svc/nginx-deployment   10.254.133.177   <nodes>       80:31469/TCP   3s

    NAME                             DESIRED   CURRENT   READY     AGE
rs/nginx-deployment-2807576163   3         3         3         15m

    NAME                                   READY     STATUS    RESTARTS   AGE
    po/nginx-3xb2r                         1/1       Running   0          50m
    po/nginx-c8g2g                         1/1       Running   0          50m
po/nginx-deployment-2807576163-2np1q   1/1       Running   0          15m
po/nginx-deployment-2807576163-5r55r   1/1       Running   0          15m
po/nginx-deployment-2807576163-nsntt   1/1       Running   0          15m
    po/nginx-tq9j1                         1/1       Running   0          50m
    po/nginx-w3d06                         1/1       Running   0          50m
    po/nginx-xln8n                         1/1       Running   0          50m

# 验证访问正常;如果想验证是否负载均衡了,可以依据SVC中方法进行验证
[root@k8s-master deploy]# curl -I 10.0.0.12:31469
	HTTP/1.1 200 OK
	Server: nginx/1.13.12
[root@k8s-master deploy]# curl -I 10.0.0.13:31469
	HTTP/1.1 200 OK
	Server: nginx/1.13.12
注意:
	Deployment资源在升级回滚时,不再像RC一样需要依据Yaml配置文件;
	可通过直接更改配置文件,既能达到立即升级的效果。
[root@k8s-master deploy]# kubectl edit deployment nginx-deployment 
    spec:
      containers:
      - image: 10.0.0.11:5000/nginx:1.15   #将原有"1.13"的镜像更改为"1.15"
        imagePullPolicy: IfNotPresent
        name: nginx

# 查看资源的变化
[root@k8s-master deploy]# kubectl get all
    NAME                      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/nginx-deployment   3         3         3            3           30m

    NAME       DESIRED   CURRENT   READY     AGE
    rc/nginx   5         5         5         1h

    NAME                   CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
    svc/kubernetes         10.254.0.1       <none>        443/TCP        1d
    svc/myweb              10.254.218.48    <nodes>       80:30000/TCP   4h
svc/nginx-deployment   10.254.133.177   <nodes>       80:31469/TCP   15m

    NAME                             DESIRED   CURRENT   READY     AGE
rs/nginx-deployment-2807576163   0         0         0         30m
rs/nginx-deployment-3014407781   3         3         3         5m

    NAME                                   READY     STATUS    RESTARTS   AGE
    po/nginx-3xb2r                         1/1       Running   0          1h
    po/nginx-c8g2g                         1/1       Running   0          1h
po/nginx-deployment-3014407781-k5crz   1/1       Running   0          5m
po/nginx-deployment-3014407781-mh9hq   1/1       Running   0          5m
po/nginx-deployment-3014407781-wqdc7   1/1       Running   0          5m
    po/nginx-tq9j1                         1/1       Running   0          1h
    po/nginx-w3d06                         1/1       Running   0          1h
    po/nginx-xln8n                         1/1       Running   0          1h

# 验证访问到的最新版本号;并且服务未中断
[root@k8s-master deploy]# curl -I 10.0.0.12:31469
    HTTP/1.1 200 OK
    Server: nginx/1.15.5
[root@k8s-master deploy]# curl -I 10.0.0.13:31469
    HTTP/1.1 200 OK
    Server: nginx/1.15.5

# 服务不中断的原因,在于Deployment资源启动的Pod,拥有两个标签,升级回滚过程中变化的只会是另一个标签,而标签“app=nginx”是始终不会变化的,此处也证明了Deployment资源是支持模糊匹配的。
[root@k8s-master deploy]# kubectl get pod --show-labels 
    NAME                                READY     STATUS    RESTARTS   AGE       LABELS
    nginx-3xb2r                         1/1       Running   0          1h        app=myweb
    nginx-c8g2g                         1/1       Running   0          1h        app=myweb
nginx-deployment-3014407781-k5crz   1/1       Running   0          18m       app=nginx,pod-template-hash=3014407781
nginx-deployment-3014407781-mh9hq   1/1       Running   0          18m       app=nginx,pod-template-hash=3014407781
nginx-deployment-3014407781-wqdc7   1/1       Running   0          18m       app=nginx,pod-template-hash=3014407781
    nginx-tq9j1                         1/1       Running   0          1h        app=myweb
    nginx-w3d06                         1/1       Running   0          1h        app=myweb
    nginx-xln8n                         1/1       Running   0          1h        app=myweb

deployment升级和回滚命令总结:
命令行创建deployment:
kubectl run nginx --image=10.0.0.11:5000/nginx:1.13 --replicas=3 --record
命令行升级版本:
kubectl set image deploy nginx nginx=10.0.0.11:5000/nginx:1.15
查看deployment所有历史版本:
kubectl rollout history deployment nginx
deployment回滚到上一个版本:
kubectl rollout undo deployment nginx
deployment回滚到指定版本:
kubectl rollout undo deployment nginx --to-revision=2

命令行创建deployment:

# 查看我们之前将Deployment资源从"1.13"升级到"1.15"的历史版本记录;是没有做记录的改变;后面我们将通过下述方法进行有历史版本记录的演示
[root@k8s-master deploy]# kubectl rollout history deployment nginx-deployment 
    deployments "nginx-deployment"
    REVISION        CHANGE-CAUSE
    1               <none>
    2               <none>
# 新建一个Deployment资源:nginx;并记录命令与版本
[root@k8s-master deploy]# kubectl run nginx --image=10.0.0.11:5000/nginx:1.13 --replicas=3 --record
	deployment "nginx" created
# 参数解释:
	--image:指定镜像版本
	--replicas:指定副本数
	--record:记录命令,方便查看每次版本的变化;在资源注释中记录当前的kubectl命令。
			 如果设置为false,则不记录命令。
			 如果设置为true,则记录命令。
			 如果未设置,则默认为仅在现有注释值存在时更新该值。
# 再次查看Deployment资源历史版本操作,与上面对比,发现有完整的历史命令操作;同事也方便后期回滚到指定的版本
[root@k8s-master deploy]# kubectl rollout history deployment nginx
    deployments "nginx"
    REVISION        CHANGE-CAUSE
    1               kubectl run nginx --image=10.0.0.11:5000/nginx:1.13 --replicas=3 --record

# 弊端;为Deployment资源进行版本升级时,不能再通过修改配置文件“kubectl edit deployment nginx”进行升级了;否则记录的历史版本操作命令会如下,不方便我们查看做了哪些操作
[root@k8s-master deploy]# kubectl edit deployment nginx
[root@k8s-master deploy]# kubectl edit deployment nginx
# 修改镜像版本为“1.15”与“1.17”后,查看内历史版本只会记录命令,而不清楚执行了哪些操作
[root@k8s-master deploy]# kubectl rollout history deployment nginx
    deployments "nginx"
    REVISION        CHANGE-CAUSE
    1               kubectl run nginx --image=10.0.0.11:5000/nginx:1.13 --replicas=3 --record
    2               kubectl edit deployment nginx
    3               kubectl edit deployment nginx

# 注意:
    kubectl run 命令:在K8S中默认启动一个Deployment资源
    docker run 命令:在Docker中默认启动一个Pod容器

命令行升级版本;并且能通过查看历史版本做了哪些操作

# Node节点上传"nginx:1.17"的镜像到仓库中
[root@k8s-node-1 images]# wget http://192.168.2.254:8888/docker_nginx.tar.gz
[root@k8s-node-1 images]# docker load -i docker_nginx.tar.gz
[root@k8s-node-1 images]# docker tag nginx:latest 10.0.0.11:5000/nginx:1.17
[root@k8s-node-1 images]# docker push 10.0.0.11:5000/nginx:1.17

# 首先清除Deployment资源“nginx”然后新建
[root@k8s-master deploy]# kubectl delete deployment nginx
	deployment "nginx" deleted
[root@k8s-master deploy]# kubectl run nginx --image=10.0.0.11:5000/nginx:1.13 --replicas=3 --record
	deployment "nginx" created
# 升级版本为“1.15”
[root@k8s-master deploy]# kubectl set image deploy nginx nginx=10.0.0.11:5000/nginx:1.15
	deployment "nginx" image updated
# 升级版本为“1.17”
[root@k8s-master deploy]# kubectl set image deploy nginx nginx=10.0.0.11:5000/nginx:1.17
	deployment "nginx" image updated
# 查看Pod,找到其中一个Pod的IP地址;验证版本“1.17”
[root@k8s-master deploy]# kubectl get pod -o wide
[root@k8s-master deploy]# curl -I 172.18.37.4
    HTTP/1.1 200 OK
    Server: nginx/1.17.5

# 通过过滤RS也能查看到每次版本的不同;因为Deployment资源升级时,会新生成RS资源,而旧的不会被删除
[root@k8s-master deploy]# kubectl get rs -o wide
    NAME                          DESIRED   CURRENT   READY     AGE       CONTAINER(S)   IMAGE(S)                    SELECTOR
nginx-1147444844              3         3         3         2m        nginx          10.0.0.11:5000/nginx:1.17   pod-template-hash=1147444844,run=nginx
nginx-847814248               0         0         0         2m        nginx          10.0.0.11:5000/nginx:1.13   pod-template-hash=847814248,run=nginx
nginx-997629546               0         0         0         2m        nginx          10.0.0.11:5000/nginx:1.15   pod-template-hash=997629546,run=nginx
    nginx-deployment-2807576163   0         0         0         1h        nginx          10.0.0.11:5000/nginx:1.13   app=nginx,pod-template-hash=2807576163
    nginx-deployment-3014407781   3         3         3         1h        nginx          10.0.0.11:5000/nginx:1.15   app=nginx,pod-template-hash=3014407781

# 查看带有升级详情的历史记录
[root@k8s-master deploy]# kubectl rollout history deployment nginx
    deployments "nginx"
    REVISION        CHANGE-CAUSE
    1               kubectl run nginx --image=10.0.0.11:5000/nginx:1.13 --replicas=3 --record
    2               kubectl set image deploy nginx nginx=10.0.0.11:5000/nginx:1.15
    3               kubectl set image deploy nginx nginx=10.0.0.11:5000/nginx:1.17

查看deployment所有历史版本:
kubectl rollout history deployment nginx

deployment回滚到上一个版本;查看历史记录有什么变化?

[root@k8s-master deploy]# kubectl rollout undo deployment nginx
    deployment "nginx" rolled back
[root@k8s-master deploy]# kubectl rollout history deployment nginx
    deployments "nginx"
    REVISION        CHANGE-CAUSE
    1               kubectl run nginx --image=10.0.0.11:5000/nginx:1.13 --replicas=3 --record
    3               kubectl set image deploy nginx nginx=10.0.0.11:5000/nginx:1.17
    4               kubectl set image deploy nginx nginx=10.0.0.11:5000/nginx:1.15
    
# 结论:
	不指定版本号回滚,默认回滚到上一个版本“1.15”,
	并且“1.15”原本版本号“2”变成了“4”;即版本“4”相当于是拿版本“2”的命令重新执行了一次;
	且更新了版本号,由最大版本值向下进行迭代。

deployment回滚到指定版本;查看历史记录有什么变化?

[root@k8s-master deploy]# kubectl rollout undo deployment nginx --to-revision=1
	deployment "nginx" rolled back
[root@k8s-master deploy]# kubectl rollout history deployment nginx
    deployments "nginx"
    REVISION        CHANGE-CAUSE
    3               kubectl set image deploy nginx nginx=10.0.0.11:5000/nginx:1.17
    4               kubectl set image deploy nginx nginx=10.0.0.11:5000/nginx:1.15
    5               kubectl run nginx --image=10.0.0.11:5000/nginx:1.13 --replicas=3 --record

# 参数解释:
	--to-revision:指定回滚的版本号
# 结论如上所属

3.5 Daemon Set资源

DaemonSet确保集群中每个(部分)node运行一份pod副本,当node加入集群时创建pod,当node离开集群时回收pod;如果删除DaemonSet,其创建的所有pod也被删除;
DaemonSet中的pod覆盖整个集群;
当需要在集群内每个node运行同一个类型pod,使用DaemonSet是有价值的,以下是典型使用场景:
运行集群存储守护进程,如glusterd、ceph。
运行集群日志收集守护进程,如fluentd、logstash、filebeat。
运行节点监控守护进程,如Prometheus Node Exporter, collectd, Datadog agent, New Relic agent, or Ganglia gmond, cadvisor, node-exporter。

创建一个DaemonSet:k8s_daemon_set.yaml

# 创建属于DaemonSet相关的yaml文件的存放目录
[root@k8s-master deploy]# mkdir /root/k8s_yaml/daemon && cd /root/k8s_yaml/daemon
[root@k8s-master daemon]# vim k8s_daemon_set.yaml
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: daemon
spec:
  template:
    metadata:
      labels:
        app: daemon
    spec:
      containers:
      - name: daemon
        image: 10.0.0.11:5000/nginx:1.13
        ports:
        - containerPort: 80
# 创建DaemonSet资源
[root@k8s-master daemon]# kubectl create -f k8s_daemon_set.yaml 
	daemonset "daemon" created
# 查看DaemonSet创建的Pod;并且是一个节点分配一个
[root@k8s-master daemon]# kubectl get pod -o wide|grep daemon
daemon-0bpjx      1/1       Running   0          2s        172.18.95.4   k8s-node-2
daemon-8wrb8      1/1       Running   0          2s        172.18.37.8   k8s-node-1
# 查看DaemonSet状态
[root@k8s-master daemon]# kubectl get daemonset 
    NAME      DESIRED   CURRENT   READY     NODE-SELECTOR   AGE
    daemon    2         2         2         <none>          1m

3.6 Pet Set资源

	在Kubernetes中,大多数的Pod管理都是基于无状态、一次性的理念。例如Replication Controller、Deployment,它只是简单的保证可提供服务的Pod数量。 
		无数据的容器,无状态的容器
	PetSet,是由一组有状态的Pod组成,每个Pod有自己特殊且不可改变的ID,且每个Pod中都有自己独一无二、不能删除的数据。 
		有数据的容器,有状态的容器
	当应用有以下任意要求时,Pet Set的价值就体现出来了。
   ● 稳定的、唯一的网络标识。
   ● 稳定的、持久化的存储。
   ● 有序的、优雅的部署和扩展。
   ● 有序的、优雅的删除和停止。
   一个有序的index(比如PetSet的名字叫mysql,那么第一个启起来的Pet就叫mysql-0,第二个叫mysql-1,如此下去。当一个Pet down掉后,新创建的Pet会被赋予跟原来Pet一样的名字,通过这个名字就能匹配到原来的存储,实现状态保存。)

当应用有以下任意要求时,StatefulSet的价值就体现出来了。
   ● 稳定的、唯一的网络标识。
   ● 稳定的、持久化的存储。
   ● 有序的、优雅的部署和扩展。
   ● 有序的、优雅的删除和停止。

3.7 Job资源

	对于ReplicaSet、ReplicationController等类型的控制器而言,它希望pod保持预期数目、持久运行下去,除非用户明确删除,否则这些对象一直存在,它们针对的是耐久性任务,如web服务等。
	对于非耐久性任务,比如压缩文件,任务完成后,pod需要结束运行,不需要pod继续保持在系统中,这个时候就要用到Job。因此说Job是对ReplicaSet、ReplicationController等持久性控制器的补充。
	通常执行一次性任务,执行结束就删除。还可以设置其定期运行。

3.8 tomcat+mysql练习

在k8s中容器之间相互访问,通过VIP地址!

在这里插入图片描述

Yaml文件:

::::::::::::::
./mysql-rc.yaml
::::::::::::::
apiVersion: v1
kind: ReplicationController
metadata:
  name: mysql
spec:
  replicas: 1
  selector:
    app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
        - name: mysql
          image: 10.0.0.11:5000/mysql:5.7
          ports:
          - containerPort: 3306
          env:
          - name: MYSQL_ROOT_PASSWORD
            value: '123456'

::::::::::::::
./mysql-svc.yaml
::::::::::::::
apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  ports:
    - port: 3306
      targetPort: 3306
  selector:
    app: mysql

::::::::::::::
./tomcat-rc.yaml
::::::::::::::
apiVersion: v1
kind: ReplicationController
metadata:
  name: myweb
spec:
  replicas: 1
  selector:
    app: myweb
  template:
    metadata:
      labels:
        app: myweb
    spec:
      containers:
        - name: myweb
          image: 10.0.0.11:5000/tomcat-app:v2
          ports:
          - containerPort: 8080
          env:
          - name: MYSQL_SERVICE_HOST
            value: 'mysql'
          - name: MYSQL_SERVICE_PORT
            value: '3306'

::::::::::::::
./tomcat-svc.yaml
::::::::::::::
apiVersion: v1
kind: Service
metadata:
  name: myweb
spec:
  type: NodePort
  ports:
    - port: 8080
      nodePort: 30008
  selector:
    app: myweb

::::::::::::::
mysql_pvc.yaml
::::::::::::::
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: tomcat-mysql
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi

::::::::::::::
mysql_pv.yaml
::::::::::::::
apiVersion: v1
kind: PersistentVolume
metadata:
  name: tomcat-mysql
  labels:
    type: nfs001
spec:
  capacity:
    storage: 10Gi 
  accessModes:
    - ReadWriteMany 
  persistentVolumeReclaimPolicy: Recycle
  nfs:
    path: "/data/pv"
    server: 10.0.0.11
    readOnly: false

::::::::::::::
mysql-rc-pvc.yaml
::::::::::::::
apiVersion: v1
kind: ReplicationController
metadata:
  name: mysql
spec:
  replicas: 1
  selector:
    app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
        - name: mysql
          image: 10.0.0.11:5000/mysql:5.7
          ports:
          - containerPort: 3306
          env:
          - name: MYSQL_ROOT_PASSWORD
            value: '123456'
          volumeMounts:
          - name: nfs
            mountPath: /var/lib/mysql
      volumes:
      - name: nfs
        persistentVolumeClaim:
          claimName: tomcat-mysql

前期准备:

# 清空前面创建的所有资源
[root@k8s-master deploy]# kubectl get all
[root@k8s-master deploy]# kubectl delete svc --all
[root@k8s-master deploy]# kubectl delete rc --all
[root@k8s-master deploy]# kubectl delete deployment --all
[root@k8s-master deploy]# kubectl get all

# 上传事先写好的Yaml文件的压缩包到Master节点(与上方Yaml文件一致)
[root@k8s-master k8s_yaml]# wget http://192.168.2.254:8888/tomcat_demo.zip
# 解压并进入该目录
[root@k8s-master k8s_yaml]# unzip tomcat_demo.zip && cd /root/k8s_yaml/tomcat_demo/
# 将文件名包含"pv"的yaml文件存放到“bak_pv”目录下,因为我们还没涉及到持久化存储的知识
[root@k8s-master tomcat_demo]# mkdir bak_pv && mv *pv*.yaml bak_pv && ls
	bak_pv  mysql-rc.yaml  mysql-svc.yaml  tomcat-rc.yaml  tomcat-svc.yaml

# 上传本地镜像包并导入到镜像仓库中
[root@k8s-node-1 images]# wget http://192.168.2.254:8888/tomcat-app-v2.tar.gz
[root@k8s-node-1 images]# wget http://192.168.2.254:8888/docker-mysql-5.7.tar.gz
[root@k8s-node-1 images]# docker load -i tomcat-app-v2.tar.gz 
[root@k8s-node-1 images]# docker load -i docker-mysql-5.7.tar.gz 
[root@k8s-node-1 images]# docker tag docker.io/kubeguide/tomcat-app:v2 10.0.0.11:5000/tomcat-app:v2
[root@k8s-node-1 images]# docker tag docker.io/mysql:5.7 10.0.0.11:5000/mysql:5.7
[root@k8s-node-1 images]# docker push 10.0.0.11:5000/mysql:5.7
[root@k8s-node-1 images]# docker push 10.0.0.11:5000/tomcat-app:v2

操作步骤:

# 创建Mysql5.7的RC和SVC;SVC的目的在于,如果容器重启了,还能关联到SVC,Tomcat容器需要连接Mysql数据库,也就是SVC的VIP
# 创建RC -- Mysql5.7
[root@k8s-master tomcat_demo]# kubectl create -f mysql-rc.yaml 
	replicationcontroller "mysql" created
# 创建SVC -- mysql5.7
[root@k8s-master tomcat_demo]# kubectl create -f mysql-svc.yaml 
	service "mysql" created
# 查看创建的资源,记录下“svc/mysql”的VIP地址,用于Tomcat容器连接
[root@k8s-master tomcat_demo]# kubectl get all
    NAME       DESIRED   CURRENT   READY     AGE
    rc/mysql   1         1         1         46s

    NAME             CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
    svc/kubernetes   10.254.0.1      <none>        443/TCP    2d
svc/mysql        10.254.34.217   <none>        3306/TCP   39s

    NAME             READY     STATUS    RESTARTS   AGE
    po/mysql-nb2g8   1/1       Running   0          46s

# 修改“tomcat-rc.yaml”中Mysql的地址;因为没有创建DNS,所以需要将主机名替换为刚刚获取的“svc/mysql”的VIP地址
[root@k8s-master tomcat_demo]# vim tomcat-rc.yaml 
          - name: MYSQL_SERVICE_HOST
            value: '10.254.34.217'
          - name: MYSQL_SERVICE_PORT
            value: '3306'
# 创建RC -- tomcat(myweb)
[root@k8s-master tomcat_demo]# kubectl create -f tomcat-rc.yaml
	replicationcontroller "myweb" created
# 创建SVC -- tomcat(myweb)
[root@k8s-master tomcat_demo]# kubectl create -f tomcat-svc.yaml 
	service "myweb" created
# 查看所有资源
[root@k8s-master tomcat_demo]# kubectl get all
    NAME       DESIRED   CURRENT   READY     AGE
    rc/mysql   1         1         1         11m
    rc/myweb   1         1         1         58s

    NAME             CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
    svc/kubernetes   10.254.0.1      <none>        443/TCP          2d
    svc/mysql        10.254.34.217   <none>        3306/TCP         11m
svc/myweb        10.254.100.94   <nodes>       8080:30008/TCP   53s

    NAME             READY     STATUS    RESTARTS   AGE
    po/mysql-nb2g8   1/1       Running   0          11m
po/myweb-9ghl7   1/1       Running   0          58s

# 验证:浏览器访问
http://10.0.0.12:30008/demo/
http://10.0.0.13:30008/demo/
# 随后点击“ADD”插入一条数据,进行数据插入验证
# 登录mysql -- Pod;查看数据库中是否有相同数据
[root@k8s-master tomcat_demo]# kubectl exec -it mysql-nb2g8 /bin/bash
# 默认创建的库为“HPE_APP”
root@mysql-nb2g8:/# mysql -uroot -p123456
mysql> show databases;
    +--------------------+
    | Database           |
    +--------------------+
    | information_schema |
    | HPE_APP            |
    | mysql              |
    | performance_schema |
    | sys                |
    +--------------------+
    5 rows in set (0.00 sec)

mysql> use HPE_APP;
    Reading table information for completion of table and column names
    You can turn off this feature to get a quicker startup with -A

    Database changed
mysql> show tables;
    +-------------------+
    | Tables_in_HPE_APP |
    +-------------------+
    | T_USERS           |
    +-------------------+
    1 row in set (0.00 sec)

mysql> select * from T_USERS; 
    +----+-----------+-------+
    | ID | USER_NAME | LEVEL |
    +----+-----------+-------+
    |  1 | me        | 100   |
    |  2 | our team  | 100   |
    |  3 | HPE       | 100   |
    |  4 | teacher   | 100   |
    |  5 | docker    | 100   |
    |  6 | google    | 100   |
    |  7 | Chiron    | 10000 |
    +----+-----------+-------+
    7 rows in set (0.00 sec)

mysql> exit
Bye
root@mysql-nb2g8:/# exit
exit

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-WyPvFkEO-1576134541636)(F:\Docker与K8S训练营\Mysql+Tomcat.jpg)]

4:k8s的附加组件

k8s集群中dns服务的作用,就是将SVC的名字解析成VIP地址

4.1 dns服务

安装dns服务(指定Node2节点进行部署)

1:上传本地dns_docker镜像包并导入dns_docker镜像包(node2节点)

[root@k8s-node-2 ~]# mkdir /opt/images && cd /opt/images
[root@k8s-node-2 images]# wget http://192.168.2.254:8888/docker_k8s_dns.tar.gz
[root@k8s-node-2 images]# docker load -i docker_k8s_dns.tar.gz
[root@k8s-node-2 images]# docker images|grep gcr
    gcr.io/google_containers/kubedns-amd64           1.9                 26cf1ed9b144        3 years ago         47 MB
    gcr.io/google_containers/dnsmasq-metrics-amd64   1.0                 5271aabced07        3 years ago         14 MB
    gcr.io/google_containers/kube-dnsmasq-amd64      1.4                 3ec65756a89b        3 years ago         5.13 MB
    gcr.io/google_containers/exechealthz-amd64       1.2                 93a43bfb39bf        3 years ago         8.37 MB

2:将DNS服务的相关Yaml文件上传到Master节点指定位置

# Master节点创建存放DNS服务相关的Yaml文件存放位置
[root@k8s-master tomcat_demo]# mkdir /root/k8s_yaml/dns && cd /root/k8s_yaml/dns
rz skydns-rc.yaml  skydns-svc.yaml

3:修改skydns-rc.yaml, 指定在node2 创建dns服务

    spec:
      nodeName: k8s-node-2    #固定在Node2节点创建资源
      containers:
      - name: kubedns
        image: gcr.io/google_containers/kubedns-amd64:1.9

4:修改skydns-svc.yaml,指定创建SVC资源的clusterIP – VIP

spec:
  selector:
    k8s-app: kube-dns
  clusterIP: 10.254.230.254    #固定SVC的VIP地址

5:创建dns服务资源RC/Deployment

[root@k8s-master dns]# kubectl create -f skydns-rc.yaml
	deployment "kube-dns" created
[root@k8s-master dns]# kubectl create -f skydns-svc.yaml 
	service "kube-dns" created

6:检查

[root@k8s-master dns]# kubectl get all --namespace=kube-system
或者
[root@k8s-master dns]# kubectl get all -n kube-system
    NAME              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
    deploy/kube-dns   1         1         1            1           40s

    NAME           CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
    svc/kube-dns   10.254.230.254   <none>        53/UDP,53/TCP   39s

	NAME                     DESIRED   CURRENT   READY     AGE
    rs/kube-dns-1656925611   1         1         1         40s

    NAME                           READY     STATUS    RESTARTS   AGE
    po/kube-dns-1656925611-96k7m   4/4       Running   0          40s

7:修改所有node节点kubelet的配置文件:指定docker启动容器使用K8S-DNS服务器

vim  /etc/kubernetes/kubelet
	KUBELET_ARGS="--cluster_dns=10.254.230.254 --cluster_domain=cluster.local"
或者
grep -Ev '^$|#' /etc/kubernetes/kubelet
sed -i 's#KUBELET_ARGS=""#KUBELET_ARGS="--cluster_dns=10.254.230.254 --cluster_domain=cluster.local"#g' /etc/kubernetes/kubelet
grep -Ev '^$|#' /etc/kubernetes/kubelet

# 重启服务
systemctl   restart kubelet

8:示例Mysql+Tomcat,将之前我们修改的“tomcat-rc.yaml”中Mysql的地址;由VIP替换为原来的主机名“mysql”

# 清除之前创建的Mysql+Tomcat资源
[root@k8s-master dns]# cd /root/k8s_yaml/tomcat_demo/
[root@k8s-master tomcat_demo]# kubectl delete -f .
    replicationcontroller "mysql" deleted
    service "mysql" deleted
    replicationcontroller "myweb" deleted
    service "myweb" deleted

# 修改“tomcat-rc.yaml”中Mysql的地址
[root@k8s-master tomcat_demo]# vim tomcat-rc.yaml 
          env:
          - name: MYSQL_SERVICE_HOST
            value: 'mysql'
          - name: MYSQL_SERVICE_PORT
            value: '3306'

# 创建Mysql+Tomcat资源
[root@k8s-master tomcat_demo]# kubectl create -f .
    replicationcontroller "mysql" created
    service "mysql" created
    replicationcontroller "myweb" created
    service "myweb" created

# 访问验证服务是否正常;浏览器亦可
[root@k8s-master tomcat_demo]# curl -I http://10.0.0.13:30008/demo/
    HTTP/1.1 200 OK
    Server: Apache-Coyote/1.1

# 服务器进入Tomcat容器中验证;看到resolv.conf中先走k8S的DNS解析
[root@k8s-master tomcat_demo]# kubectl exec -it myweb-qjc5x bash
root@myweb-qjc5x:/usr/local/tomcat# cat /etc/resolv.conf 
    search default.svc.cluster.local svc.cluster.local cluster.local
nameserver 10.254.230.254
    nameserver 180.76.76.76
    nameserver 223.5.5.5
    options ndots:5
# 当ping -- Mysql时,将返回mysql的VIP地址
# 注意:在容器中,ping -- IP 是永远不通的
root@myweb-qjc5x:/usr/local/tomcat# ping mysql
PING mysql.default.svc.cluster.local (10.254.100.255): 56 data bytes

4.2 namespace命令空间

namespace做资源隔离
默认K8S的Namespace为:default
生产中一般建议,一个业务对应一个单独的Namepace

# 查看K8S集群中所有namespace 
[root@k8s-master tomcat_demo]# kubectl get namespace 
    NAME          STATUS    AGE
    default       Active    2d
    kube-system   Active    2d

# 查看Pod,默认选择的Namespace为:default
[root@k8s-master tomcat_demo]# kubectl get pod
    NAME          READY     STATUS    RESTARTS   AGE
    mysql-dr2fz   1/1       Running   0          57m
    myweb-qjc5x   1/1       Running   0          57m
[root@k8s-master tomcat_demo]# kubectl get pod -n default
    NAME          READY     STATUS    RESTARTS   AGE
    mysql-dr2fz   1/1       Running   0          57m
    myweb-qjc5x   1/1       Running   0          57m

# 查看指定Namespace:kube-system的资源
[root@k8s-master tomcat_demo]# kubectl get pod -n kube-system 
    NAME                        READY     STATUS    RESTARTS   AGE
    kube-dns-1656925611-c2270   4/4       Running   0          58m
    
# 将Namespace:default中Mysql+Tomcat资源删除
[root@k8s-master tomcat_demo]# kubectl delete -f .
    replicationcontroller "mysql" deleted
    service "mysql" deleted
    replicationcontroller "myweb" deleted
    service "myweb" deleted
# 创建Namespace:tomcat
[root@k8s-master tomcat_demo]# kubectl create namespace tomcat
    namespace "tomcat" created
# 查看Namespace列表
[root@k8s-master tomcat_demo]# kubectl get namespace 
    NAME          STATUS    AGE
    default       Active    2d
    kube-system   Active    2d
tomcat        Active    48s

# 更改“yaml”文件;指定Namespace;不指定就默认选择Namespace为:default
# 测试在第三行之后追加一行指定Namespace;并查看前五行效果
[root@k8s-master tomcat_demo]# sed '3a \ \ namespace: tomcat' mysql-svc.yaml |head -5
    apiVersion: v1
    kind: Service
    metadata:
      namespace: tomcat   #新增的行
      name: mysql
# 将当前目录下所有“Yaml”文件指定Namespace
[root@k8s-master tomcat_demo]# sed -i '3a \ \ namespace: tomcat' *.yaml

# 创建Mysql+Tomcat资源
[root@k8s-master tomcat_demo]# kubectl create -f .
    replicationcontroller "mysql" created
    service "mysql" created
    replicationcontroller "myweb" created
    service "myweb" created
# 查看验证Mysql+Tomcat资源;分配到了Namespace:tomcat中
[root@k8s-master tomcat_demo]# kubectl get all -n default 
    NAME             CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
    svc/kubernetes   10.254.0.1   <none>        443/TCP   2d
[root@k8s-master tomcat_demo]# kubectl get all -n tomcat 
    NAME       DESIRED   CURRENT   READY     AGE
    rc/mysql   1         1         1         40s
    rc/myweb   1         1         1         40s

    NAME        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
    svc/mysql   10.254.170.16    <none>        3306/TCP         40s
    svc/myweb   10.254.130.189   <nodes>       8080:30008/TCP   39s

    NAME             READY     STATUS    RESTARTS   AGE
    po/mysql-pwtsp   1/1       Running   0          40s
    po/myweb-jbvd6   1/1       Running   0          40s
# 访问Tomcat依旧是正常的
[root@k8s-master tomcat_demo]# curl -I http://10.0.0.13:30008/demo/
    HTTP/1.1 200 OK
    Server: Apache-Coyote/1.1

4.3 健康检查

4.3.1 探针的种类

livenessProbe:健康状态检查,周期性检查服务是否存活,检查结果失败,将重启容器

readinessProbe:可用性检查,周期性检查服务是否可用,不可用将从service的endpoints中移除

4.3.2 探针的检测方法

  • exec:执行一段命令 返回值为0, 非0
  • httpGet:检测某个 http 请求的返回状态码 2xx,3xx正常, 4xx,5xx错误
  • tcpSocket:测试某个端口是否能够连接
# Master节点创建存放Health健康检查相关的Yaml文件存放位置
[root@k8s-master tomcat_demo]# mkdir /root/k8s_yaml/health && cd /root/k8s_yaml/health

4.3.3 liveness探针的exec使用:

​ nginx_pod_exec.yaml – 通过检测文件是否存在(即执行一段命令返回的状态码是否正常)

apiVersion: v1
kind: Pod
metadata:
  name: exec
spec:
  containers:
    - name: nginx
      image: 10.0.0.11:5000/nginx:1.13
      ports:
        - containerPort: 80
      args:
        - /bin/sh
        - -c
        - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
      livenessProbe:
        exec:
          command:
            - cat
            - /tmp/healthy
        initialDelaySeconds: 5  
        periodSeconds: 5
        timeoutSeconds: 5
        successThreshold: 1
        failureThreshold: 1

​ 扩展:
​ K8S中容器的初始命令:
​ args:
​ command:
​ Dockerfile中容器的初始命令:
​ CMD:
​ ENTRYPOINT:

# 创建nginx_pod_exec资源
[root@k8s-master health]# kubectl create -f nginx_pod_exec.yaml 
	pod "exec" created
# 查看nginx_pod_exec资源的Pod状态;发现每过一段时间,Pod将执行一次重启
[root@k8s-master health]# kubectl get pod -w
    NAME      READY     STATUS    RESTARTS   AGE
    exec      1/1       Running   0          2s
    NAME      READY     STATUS    RESTARTS   AGE
    exec      1/1       Running   1          1m
    exec      1/1       Running   2         2m

# 查看nginx_pod_exec资源的Pod详情,也是可以看到Pod的状态变化
[root@k8s-master ~]# kubectl describe pod exec
	container "nginx" is unhealthy, it will be killed and re-created.

4.3.4 liveness探针的httpGet使用:

​ nginx_pod_httpGet.yaml – 通过检测服务访问的URL返回的状态码是否正常(即观察访问的值是否为“302”等)

apiVersion: v1
kind: Pod
metadata:
  name: httpget
spec:
  containers:
    - name: nginx
      image: 10.0.0.11:5000/nginx:1.13
      ports:
        - containerPort: 80
      livenessProbe:
        httpGet:
          path: /index.html
          port: 80
        initialDelaySeconds: 3
        periodSeconds: 3
# 创建nginx_pod_httpGet资源
[root@k8s-master health]# kubectl create -f nginx_pod_httpGet.yaml 
    pod "httpget" created
# 进入容器删除首页
[root@k8s-master health]# kubectl exec -it httpget bash
root@httpget:/# rm -rf /usr/share/nginx/html/index.html 
root@httpget:/# exit
exit
# 查看容器状态,发现立马进行了一次重启;这时容器有了首页就正常了
[root@k8s-master ~]# kubectl get pod httpget -w
    NAME      READY     STATUS    RESTARTS   AGE
    httpget   1/1       Running   0          1m
    httpget   1/1       Running   1         2m
# 查看容器详情,发现检测到Nginx状态404,就有了下面的重启操作
[root@k8s-master ~]# kubectl describe pod httpget
	Liveness probe failed: HTTP probe failed with statuscode: 404

4.3.5 liveness探针的tcpSocket使用:

​ nginx_pod_tcpSocket.yaml – 通过检测“80”端口是否启用,来判断容器的正常与否(想要正常,需进入容器将“80”端口的Nginx服务启动)

apiVersion: v1
kind: Pod
metadata:
  name: tcpsocket
spec:
  containers:
    - name: nginx
      image: 10.0.0.11:5000/nginx:1.13
      ports:
        - containerPort: 80
      args:
        - /bin/sh
        - -c
        - tail -f /etc/hosts
      livenessProbe:
        tcpSocket:
          port: 80
        initialDelaySeconds: 10
        periodSeconds: 3
# 创建nginx_pod_tcpSocket资源
[root@k8s-master health]# kubectl create -f nginx_pod_tcpSocket.yaml 
    pod "tcpsocket" created
# 因为我们未进入容器将Nginx服务启动;80端口检测不到,经过一个周期,容器被重启了
[root@k8s-master ~]# kubectl get pod tcpsocket -w
    NAME        READY     STATUS    RESTARTS   AGE
    tcpsocket   1/1       Running   0          15s
    tcpsocket   1/1       Running   1         48s
# 此时我们进入容器将Nginx服务启动
[root@k8s-master health]# kubectl exec -it tcpsocket bash
root@tcpsocket:/# /usr/sbin/nginx
root@tcpsocket:/# exit
exit
# 这时再看到Pod状态和详情,发现没有任何变化
[root@k8s-master ~]# kubectl get pod tcpsocket -w
[root@k8s-master ~]# kubectl describe pod tcpsocket

4.3.6 readiness探针的httpGet使用:

​ nginx-rc-httpGet.yaml – 通过检测服务访问的URL返回的状态码是否正常(即容器启动并不会显示正常关联到SVC;只有一切正常时,才会关联到SVC;这就诠释了“可用性”)

apiVersion: v1
kind: ReplicationController
metadata:
  name: readiness
spec:
  replicas: 2
  selector:
    app: readiness
  template:
    metadata:
      labels:
        app: readiness
    spec:
      containers:
      - name: readiness
        image: 10.0.0.11:5000/nginx:1.13
        ports:
        - containerPort: 80
        readinessProbe:
          httpGet:
            path: /Chiron.html
            port: 80
          initialDelaySeconds: 3
          periodSeconds: 3
# 创建nginx-rc-httpGet资源
[root@k8s-master health]# kubectl create -f nginx-rc-httpGet.yaml 
	replicationcontroller "readiness" created
# 查看Pod状态,发现是运行的,但是容器并没有准备好
[root@k8s-master health]# kubectl get pod |grep readiness
    readiness-db8cw   0/1       Running            0          1m
    readiness-rmpjc   0/1       Running            0          1m
# 创建SVC资源,管理POD:readiness
[root@k8s-master health]# kubectl expose rc readiness --port=80 --target-port=80 --type=NodePort
    service "readiness" exposed
# 查看SVC:readiness资源详情;并没有关联到Pod
[root@k8s-master health]# kubectl describe svc readiness 
    Name:                   readiness
    Namespace:              default
    Labels:                 app=readiness
    Selector:               app=readiness
    Type:                   NodePort
    IP:                     10.254.227.196
    Port:                   <unset> 80/TCP
    NodePort:               <unset> 32089/TCP
    Endpoints:
    Session Affinity:       None
    No events.
# 进入其中一个Pod,添加Chiron.html页面
[root@k8s-master health]# kubectl exec -it readiness-db8cw bash
root@readiness-db8cw:/# echo 'web01' > /usr/share/nginx/html/Chiron.html
root@readiness-db8cw:/# exit
	exit
# 再次查看SVC:readiness资源详情;关联到了一个;验证了“可用性”
[root@k8s-master health]# kubectl describe svc readiness 
    Name:                   readiness
    Namespace:              default
    Labels:                 app=readiness
    Selector:               app=readiness
    Type:                   NodePort
    IP:                     10.254.227.196
    Port:                   <unset> 80/TCP
    NodePort:               <unset> 32089/TCP
Endpoints:              172.18.95.4:80
    Session Affinity:       None
    No events.
# 查看Pod:readiness状态;一个正常(准备好的),一个异常(未准备好)
[root@k8s-master health]# kubectl get pod -o wide |grep readiness
    readiness-db8cw   1/1       Running            0          10m       172.18.95.4   k8s-node-2
    readiness-rmpjc   0/1       Running            0          10m       172.18.37.6   k8s-node-1

4.4 dashboard服务

1:Node2节点上传并导入镜像,打标签

rz kubernetes-dashboard-amd64_v1.4.1.tar.gz
[root@k8s-node-2 images]# docker load -i kubernetes-dashboard-amd64_v1.4.1.tar.gz
[root@k8s-node-2 images]# docker tag index.tenxcloud.com/google_containers/kubernetes-dashboard-amd64:v1.4.1 10.0.0.10:5000/kubernetes-dashboard-amd64:v1.4.1

2:Master节点创建dashborad的deployment和service

# 创建dashborad相关的Yaml文件存放位置
[root@k8s-master health]# mkdir /root/k8s_yaml/dashboard && cd /root/k8s_yaml/dashboard

# 上传事先准备好的Yaml文件
rz dashboard.yaml  &&  rz dashboard-svc.yaml
# 修改dashboard.yaml文件,指定Pod创建时运行在Node2节点上
[root@k8s-master dashboard]# vim dashboard.yaml 
    spec:
      nodeName: k8s-node-2
      containers:
      - name: kubernetes-dashboard

# 创建dashborad相关资源
[root@k8s-master dashboard]# kubectl create -f dashboard.yaml 
	deployment "kubernetes-dashboard-latest" created
[root@k8s-master dashboard]# kubectl create -f dashboard-svc.yaml 
	service "kubernetes-dashboard" created

# 查看Namespace:kube-system 中的资源
[root@k8s-master dashboard]# kubectl get all -n kube-system 
    NAME                                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
    deploy/kube-dns                      1         1         1            1           4h
deploy/kubernetes-dashboard-latest   1         1         1            1           2s

    NAME           CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
    svc/kube-dns   10.254.230.254   <none>        53/UDP,53/TCP   4h
svc/kubernetes-dashboard   10.254.1.236     <none>        80/TCP          1s

    NAME                                        DESIRED   CURRENT   READY     AGE
    rs/kube-dns-1656925611                      1         1         1         4h
rs/kubernetes-dashboard-latest-1975875540   1         1         1         2s

    NAME                                              READY     STATUS    RESTARTS   AGE
    po/kube-dns-1656925611-c2270                      4/4       Running   0          4h
po/kubernetes-dashboard-latest-1975875540-lnd2d   1/1       Running   0          2s

3:访问Dashboard:http://10.0.0.11:8080/ui/

4.5 通过apiservicer反向代理访问service

第一种:NodePort类型 
  type: NodePort
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30008

第二种:ClusterIP类型
  type: ClusterIP
  ports:
    - port: 80
      targetPort: 80
      
http://10.0.0.11:8080/api/v1/proxy/namespaces/命令空间/services/service的名字/

http://10.0.0.13:30008/
http://10.0.0.11:8080/api/v1/proxy/namespaces/tomcat/services/myweb/	#同样可以访问Tomcat

5: k8s弹性伸缩

k8s弹性伸缩,需要附加插件heapster监控

扩展:
默认在Node节点中Kubelet集成了"cAdvisor"
http://10.0.0.12:4194/containers/
http://10.0.0.13:4194/containers/

5.1 安装heapster监控

1:在Node2中上传并导入镜像,打标签

# 创建放置Heapster监控用到的镜像包目录
[root@k8s-node-2 images]# mkdir /opt/images/heapster && cd /opt/images/heapster
# 上传本地的镜像包
[root@k8s-node-2 images]# wget http://192.168.2.254:8888/docker_heapster_grafana.tar.gz
[root@k8s-node-2 images]# wget http://192.168.2.254:8888/docker_heapster.tar.gz
[root@k8s-node-2 images]# wget http://192.168.2.254:8888/docker_heapster_influxdb.tar.gz
# 批量导入images
[root@k8s-node-2 heapster]# for n in `ls *.tar.gz`;do docker load -i $n ;done
# 为导入的images打标签
[root@k8s-node-2 heapster]# docker tag docker.io/kubernetes/heapster_grafana:v2.6.0 10.0.0.11:5000/heapster_grafana:v2.6.0
[root@k8s-node-2 heapster]# docker tag  docker.io/kubernetes/heapster_influxdb:v0.5 10.0.0.11:5000/heapster_influxdb:v0.5
[root@k8s-node-2 heapster]# docker tag docker.io/kubernetes/heapster:canary 10.0.0.11:5000/heapster:canary

2:Maste节点创建存放“heapster”相关“Yaml”文件的命令;上传事先准备好的Yaml配置文件

[root@k8s-master dashboard]# mkdir /root/k8s_yaml/heapster && cd /root/k8s_yaml/heapster
[root@k8s-master heapster]# wget http://192.168.2.254:8888/grafana-service.yaml
[root@k8s-master heapster]# wget http://192.168.2.254:8888/heapster-service.yaml
[root@k8s-master heapster]# wget http://192.168.2.254:8888/influxdb-service.yaml
[root@k8s-master heapster]# wget http://192.168.2.254:8888/heapster-controller.yaml
[root@k8s-master heapster]# wget http://192.168.2.254:8888/influxdb-grafana-controller.yaml

3:修改配置文件指定在“Node2节点”创建资源

# 修改配置文件:heapster-controller.yaml
    spec:
      nodeName: k8s-node-2   #指定节点
      containers:
      - name: heapster
        image: 10.0.0.11:5000/heapster:canary
        imagePullPolicy: IfNotPresent   #更改镜像下载策略;只要本地有镜像,就不进行更新
# 修改配置文件:influxdb-grafana-controller.yaml
    spec:
      nodeName: k8s-node-2
      containers:
      - name: influxdb
        image: 10.0.0.11:5000/heapster_influxdb:v0.5

# 查看所有与Namespace中的Pod资源
[root@k8s-master heapster]# kubectl get pod --all-namespaces -o wide

# 创建资源
[root@k8s-master heapster]# kubectl create -f /root/k8s_yaml/heapster/.
    service "monitoring-grafana" created
    replicationcontroller "heapster" created
    service "heapster" created
    replicationcontroller "influxdb-grafana" created
    service "monitoring-influxdb" created

4:打开dashboard验证

在这里插入图片描述

5.2 弹性伸缩

1:在deploy目录中创建弹性伸缩RC的配置文件

[root@k8s-master heapster]# cd /root/k8s_yaml/deploy
[root@k8s-master deploy]# vim k8s_deploy_nginx.yaml 
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: 10.0.0.11:5000/nginx:1.13
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: 100m
          requests:
            cpu: 100m
# 创建deploy_nginx资源
[root@k8s-master deploy]# kubectl create -f k8s_deploy_nginx.yaml 
	deployment "nginx" created
# 获取Pod的IP地址
[root@k8s-master deploy]# kubectl get pod -o wide |grep nginx
	nginx-2807576163-pkgp9   1/1       Running   0          1m        172.18.37.9   k8s-node-1

# 查看资源(这里只截取了deploy_nginx资源)
[root@k8s-master deploy]# kubectl get all -o wide
    NAME           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
    deploy/nginx   1         1         1            1           8m

    NAME        REFERENCE          TARGET    CURRENT   MINPODS   MAXPODS   AGE
    hpa/nginx   Deployment/nginx   10%       0%        1         8         4m

    NAME           DESIRED   CURRENT   READY     AGE       CONTAINER(S)   IMAGE(S)                    SELECTOR
    rc/readiness   3         3         0         4h        readiness      10.0.0.11:5000/nginx:1.15   app=readiness

    NAME             CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE       SELECTOR
    svc/kubernetes   10.254.0.1       <none>        443/TCP        3d        <none>
    svc/readiness    10.254.227.196   <nodes>       80:32089/TCP   4h        app=readiness

    NAME                  DESIRED   CURRENT   READY     AGE       CONTAINER(S)   IMAGE(S)                    SELECTOR
    rs/nginx-2807576163   1         1         1         8m        nginx          10.0.0.11:5000/nginx:1.13   app=nginx,pod-template-hash=2807576163

    NAME                        READY     STATUS    RESTARTS   AGE       IP            NODE
    po/nginx-2807576163-pkgp9   1/1       Running   0          8m        172.18.37.9   k8s-node-1

[root@k8s-master deploy]# kubectl get hpa nginx
    NAME      REFERENCE          TARGET    CURRENT   MINPODS   MAXPODS   AGE
    nginx     Deployment/nginx   10%       0%        1         8         1m

2:创建弹性伸缩规则

# 创建SVC,与当前deploy_nginx资源进行关联
[root@k8s-master deploy]# kubectl expose deployment nginx --type=NodePort --port=80
	service "nginx" exposed
# 查看SVC资源;记录SVC:nginx 的随机端口;作为后面压测时使用
# 启用SVC是为了,当压力过大,Pod水平扩展时,SVC会做负载,压测时流量会相对平均分配
[root@k8s-master deploy]# kubectl get svc 
    NAME         CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
    kubernetes   10.254.0.1       <none>        443/TCP        3d
nginx        10.254.249.86    <nodes>       80:30138/TCP   11s
	readiness    10.254.227.196   <nodes>       80:32089/TCP   4h

[root@k8s-master deploy]# kubectl  autoscale  deploy  nginx  --max=8  --min=1 --cpu-percent=10
	deployment "nginx" autoscaled

3:压力测试

 # 安装httpd-tools工具,进行压力测试
 [root@k8s-master deploy]# yum install -y httpd-tools
 # 查看“ab”命令是否存在
 [root@k8s-master deploy]# which ab
	/usr/bin/ab
# 压力测试
[root@k8s-master deploy]# ab -n 1000000 -c 40 http://10.0.0.12:30138/index.html

# 注意:
	当压力减小时,水平扩展的Pod默认是五分钟后才会根据实际情况进行水平缩容;运行时间越少,将会优先删除

扩容截图

在这里插入图片描述

缩容:

在这里插入图片描述

6:持久化存储

​ 前面我们部署的Mysql+Tomcat,是没有做持久化存储的,当Mysql_Pod重启后,之前添加的数据将丢失不存在。

数据持久化类型:

6.1 emptyDir:

​ emptryDir,顾名思义是一个空目录,它的生命周期和所属的 Pod 是完全一致的。emptyDir 类型的 Volume 在 Pod 分配到 Node 上时会被创建,Kubernetes 会在 Node 上自动分配一个目录,因此无需指定 Node 宿主机上对应的目录文件。这个目录的初始内容为空,当 Pod 从 Node 上移除(Pod 被删除或者 Pod 发生迁移)时,emptyDir 中的数据会被永久删除。
​ emptyDir Volume 主要用于某些应用程序无需永久保存的临时目录,在多个容器之间共享数据等。缺省情况下,emptryDir 是使用主机磁盘进行存储的。你也可以使用其它介质作为存储,比如:网络存储、内存等。设置 emptyDir.medium 字段的值为 Memory 就可以使用内存进行存储,使用内存做为存储可以提高整体速度,但是要注意一旦机器重启,内容就会被清空,并且也会受到容器内存的限制。
​ emptyDir类型的volume在pod分配到node上时被创建,kubernetes会在node上自动分配 一个目录,因此无需指定宿主机node上对应的目录文件。这个目录的初始内容为空,当Pod从node上移除时,emptyDir中的数据会被永久删除。

emptyDir Volume主要用于某些应用程序无需永久保存的临时目录,多个容器的共享目录等。

[root@k8s-master deploy]# cd /root/k8s_yaml/tomcat_demo/
# 创建mysql-rc-V2.yaml文件;添加“emptyDir”持久化机制;后面创建的Mysql-Pod资源使用这个文件
[root@k8s-master tomcat_demo]# vim mysql-rc.yaml
apiVersion: v1
kind: ReplicationController
metadata:
  namespace: tomcat
  name: mysql
spec:
  replicas: 1
  selector:
    app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      volumes:
      - name: mysql
        emptyDir: {}
      containers:
        - name: mysql
          image: 10.0.0.11:5000/mysql:5.7
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 3306
          volumeMounts:
          - mountPath: /var/lib/mysql
            name: mysql
          env:
          - name: MYSQL_ROOT_PASSWORD
            value: '123456'
# 删除旧的Namespace:Tomcat中所有资源
[root@k8s-master tomcat_demo]# kubectl delete -f /root/k8s_yaml/tomcat_demo/.
    replicationcontroller "mysql" deleted
    service "mysql" deleted
    replicationcontroller "myweb" deleted
    service "myweb" deleted
# 将旧版本mysql-rc.yaml文件移动到下一级目录中;不影响后面的批量创建资源
[root@k8s-master tomcat_demo]# mv mysql-rc.yaml bak_pv/

# 批量创建Mysql+Tomcat资源
[root@k8s-master tomcat_demo]# kubectl create -f /root/k8s_yaml/tomcat_demo/.
    replicationcontroller "mysql" created
    service "mysql" created
    replicationcontroller "myweb" created
    service "myweb" created

# 查看创建的资源Tomcat:Pod运行的节点
[root@k8s-master tomcat_demo]# kubectl get pod -o wide -n tomcat 
    NAME          READY     STATUS    RESTARTS   AGE       IP            NODE
    mysql-c2v1j   1/1       Running   0          26s       172.18.37.7   k8s-node-1
    myweb-0lzls   1/1       Running   0          26s       172.18.37.2   k8s-node-1

# 访问Web_URL,插入一条数据:Name=Chiron Level=10000
http://10.0.0.13:30008/demo/index.jsp

# 在“Node1节点”删除刚刚创建的Pod资源
[root@k8s-node-1 images]# docker rm -f `docker ps |grep my`
    9213f64fe0f2
    ffd769cd457b
    7b05a7711987
    861e6f27af6e
    
# 等待RC新起Pod,然后访问Web_URL,插入的数据依然存在
http://10.0.0.13:30008/demo/index.jsp

# 缺点:不能删除Mysql-Pod,如果删除Pod,数据将丢失
[root@k8s-master tomcat_demo]# kubectl delete pod -n tomcat mysql-c2v1j
	pod "mysql-c2v1j" deleted
[root@k8s-master tomcat_demo]# kubectl get pod -o wide -n tomcat 
    NAME          READY     STATUS    RESTARTS   AGE       IP             NODE
mysql-vn36w   1/1       Running   0          27s       172.18.37.10   k8s-node-1
    myweb-0lzls   1/1       Running   0          14m       172.18.37.2    k8s-node-1
# 等待Mysql新起Pod,然后访问Web_URL,插入的数据已经丢失
http://10.0.0.13:30008/demo/index.jsp

# 在“Node1节点”查看Mysql-Pod的volumes-mysql数据目录
[root@k8s-node-1 images]# find / -name "mysql" |grep volumes
    /var/lib/kubelet/pods/57970a59-1250-11ea-b36a-000c29c6d194/volumes/kubernetes.io~empty-dir/mysql
    /var/lib/kubelet/pods/57970a59-1250-11ea-b36a-000c29c6d194/volumes/kubernetes.io~empty-dir/mysql/mysql

# 进入mysql数据目录,并查看目录下有哪些东西
[root@k8s-node-1 images]# cd /var/lib/kubelet/pods/57970a59-1250-11ea-b36a-000c29c6d194/volumes/kubernetes.io~empty-dir/mysql && ls
    auto.cnf  HPE_APP  ib_buffer_pool  ibdata1  ib_logfile0  ib_logfile1  ibtmp1  mysql  performance_schema  sys
# 我们访问tomcat时,数据库是“HPE_APP”,查看它的下面有哪些表
[root@k8s-node-1 mysql]# ls HPE_APP/
	db.opt  T_USERS.frm  T_USERS.ibd

6.2 HostPath:

​ hostPath Volume为pod挂载宿主机上的目录或文件,使得容器可以使用宿主机的高速文件系统进行存储。
缺点:
  在k8s中,pod都是动态在各node节点上调度。当一个pod在当前node节点上启动并通过hostPath存储了文件到本地以后,下次调度到另一个节点上启动时,就无法使用在之前节点上存储的文件。
​ 所以使用“HostPath”方式做持久化时,不必须固定Pod资源运行在哪个节点。

# 删除旧的“Mysql+Tomcat”资源
[root@k8s-master tomcat_demo]# kubectl delete -f /root/k8s_yaml/tomcat_demo/.
    replicationcontroller "mysql" deleted
    service "mysql" deleted
    replicationcontroller "myweb" deleted
    service "myweb" deleted
# 移动“mysql-rc-V2.yaml”文件到“bak_pv”目录下
[root@k8s-master tomcat_demo]# mv mysql-rc-V2.yaml /root/k8s_yaml/tomcat_demo/bak_pv/
# 创建mysql-rc-V3.yaml文件
[root@k8s-master tomcat_demo]# vim mysql-rc-V3.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  namespace: tomcat
  name: mysql
spec:
  replicas: 1
  selector:
    app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      volumes:
      - name: mysql
        hostPath:
          path: /data/tomcat_mysql
      containers:
        - name: mysql
          image: 10.0.0.11:5000/mysql:5.7
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 3306
          volumeMounts:
          - mountPath: /var/lib/mysql
            name: mysql
          env:
          - name: MYSQL_ROOT_PASSWORD
            value: '123456'
# 批量创建Mysql+Tomcat资源
[root@k8s-master tomcat_demo]# kubectl create -f /root/k8s_yaml/tomcat_demo/.
    replicationcontroller "mysql" created
    service "mysql" created
    replicationcontroller "myweb" created
    service "myweb" created
# 查看Pod被分配到了哪个节点
[root@k8s-master tomcat_demo]# kubectl get pod -o wide -n tomcat 
    NAME          READY     STATUS    RESTARTS   AGE       IP            NODE
    mysql-hc0h2   1/1       Running   0          3m        172.18.37.7   k8s-node-1
    myweb-q7d2m   1/1       Running   0          3m        172.18.37.2   k8s-node-1

# 访问Web_URL,触发Tomcat页面;插入一条数据:Name=Chiron Level=10000
http://10.0.0.13:30008/demo/index.jsp

# 在Node1节点查看我们挂载的目录是否存有数据
[root@k8s-node-1 mysql]# cd /data/tomcat_mysql/ && ls
    auto.cnf  ib_buffer_pool  ib_logfile0  ibtmp1  performance_schema
    HPE_APP   ibdata1         ib_logfile1  mysql   sys
    
# 删除“Mysql-Pod”;待重新运行一个新的后,查看之前的数据是否依然存在
[root@k8s-master tomcat_demo]# kubectl delete pod -n tomcat mysql-hc0h2
[root@k8s-master tomcat_demo]# kubectl get pod -o wide -n tomcat 
    NAME          READY     STATUS    RESTARTS   AGE       IP             NODE
mysql-sp6sg   1/1       Running   0          4s        172.18.37.10   k8s-node-1
    myweb-q7d2m   1/1       Running   0          10m       172.18.37.2    k8s-node-1

# 等待Mysql新起Pod,然后访问Web_URL,插入的数据依然存在
http://10.0.0.13:30008/demo/index.jsp
# 注意:
	只要再次创建的Pod没有被分配到其他节点,数据将始终存在;
	再次创建的Pod如果被分配到了其他节点,数据将丢失。

6.3 nfs:

​ NFS是网络文件系统Network File System的缩写,NFS服务器可以让PC将网络中的NFS服务器共享的目录挂载到本地的文件系统中,而在本地的系统中来看,那个远程主机的目录就好像是自己的一个磁盘分区一样。

Kubernetes使用NFS共享存储有两种方式:

​ 1.手动方式静态创建所需要的PV和PVC;
​ 2.通过创建PVC动态地创建对应PV,无需手动创建PV。

# 所有节点安装NFS工具
yum install nfs-utils -y
# Master节点建立存放持久化数据的目录;后面会在Mysql_RC的Yaml文件中用到
[root@k8s-master tomcat_demo]# mkdir /data

# Master节点作为服务端;更改NFS配置文件,共享哪个目录,允许谁访问,目录权限,同步方式;不做任何用户映射
[root@k8s-master tomcat_demo]# cat <<EOF>>/etc/exports
> /data 10.0.0.0/24(rw,async,no_all_squash,no_root_squash)
> EOF
[root@k8s-master tomcat_demo]# more /etc/exports
	/data 10.0.0.0/24(rw,async,no_all_squash,no_root_squash)

# 启动并查看 NFS相关服务
[root@k8s-master tomcat_demo]# systemctl restart rpcbind nfs
[root@k8s-master tomcat_demo]# systemctl enable rpcbind nfs
[root@k8s-master tomcat_demo]# systemctl status rpcbind nfs

# 在Node节点验证是否能查看到共享的目录
[root@k8s-node-1 tomcat_mysql]# showmount -e 10.0.0.11
    Export list for 10.0.0.11:
    /data 10.0.0.0/24
[root@k8s-node-2 images]# showmount -e 10.0.0.11
    Export list for 10.0.0.11:
    /data 10.0.0.0/24
    
# 删除旧的“Mysql+Tomcat”资源
[root@k8s-master tomcat_demo]# kubectl delete -f /root/k8s_yaml/tomcat_demo/.
    replicationcontroller "mysql" deleted
    service "mysql" deleted
    replicationcontroller "myweb" deleted
    service "myweb" deleted
# 移动“mysql-rc-V3.yaml”文件到“bak_pv”目录下
[root@k8s-master tomcat_demo]# mv mysql-rc-V3.yaml /root/k8s_yaml/tomcat_demo/bak_pv/
# 创建mysql-rc-V4.yaml文件
[root@k8s-master tomcat_demo]# vim mysql-rc-V4.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  namespace: tomcat
  name: mysql
spec:
  replicas: 1
  selector:
    app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      volumes:
      - name: mysql
        nfs:
          path: /data/tomcat_mysql
          server: 10.0.0.11
      containers:
        - name: mysql
          image: 10.0.0.11:5000/mysql:5.7
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 3306
          volumeMounts:
          - mountPath: /var/lib/mysql
            name: mysql
          env:
          - name: MYSQL_ROOT_PASSWORD
            value: '123456'
# 批量创建Mysql+Tomcat资源
[root@k8s-master tomcat_demo]# kubectl create -f /root/k8s_yaml/tomcat_demo/.
    replicationcontroller "mysql" created
    service "mysql" created
    replicationcontroller "myweb" created
    service "myweb" created
# 查看Pod被分配到了哪个节点
[root@k8s-master tomcat_demo]# kubectl get pod -o wide -n tomcat
    NAME          READY     STATUS    RESTARTS   AGE       IP            NODE
    mysql-3rxjv   1/1       Running   0          6s        172.18.37.7   k8s-node-1
    myweb-vcd21   1/1       Running   0          6s        172.18.37.2   k8s-node-1

# 访问Web_URL,触发Tomcat页面;插入一条数据:Name=Chiron Level=10000
http://10.0.0.13:30008/demo/index.jsp

# 在“Node1节点”查看是否有Mount挂载记录
[root@k8s-node-1 tomcat_mysql]# df -Th |grep nfs
    10.0.0.11:/data/tomcat_mysql  nfs4  48G  3.5G   45G   8% /var/lib/kubelet/pods/169c35cb-128e-11ea-b36a-000c29c6d194/volumes/kubernetes.io~nfs/mysql
# 在Master节点查看本地共享目录下,是否存在数据
[root@k8s-master tomcat_demo]# ls /data/tomcat_mysql/
	auto.cnf  HPE_APP  ib_buffer_pool  ibdata1  ib_logfile0  ib_logfile1  ibtmp1  mysql  performance_schema  sys

# 修改“mysql-rc-V4.yaml”文件,指定在“Node2节点”运行Mysql-Pod资源,方便后面验证数据依然存在
[root@k8s-master tomcat_demo]# vim mysql-rc-V4.yaml
    spec:
      nodeName: k8s-node-2   #指定运行节点
      volumes:
      - name: mysql
        nfs:
          path: /data/tomcat_mysql
          server: 10.0.0.11

# 批量删除“Mysql+Tomcat”资源
[root@k8s-master tomcat_demo]# kubectl delete -f /root/k8s_yaml/tomcat_demo/.
    replicationcontroller "mysql" deleted
    service "mysql" deleted
    replicationcontroller "myweb" deleted
    service "myweb" deleted
# 批量创建Mysql+Tomcat资源
[root@k8s-master tomcat_demo]# kubectl create -f /root/k8s_yaml/tomcat_demo/.
    replicationcontroller "mysql" created
    service "mysql" created
    replicationcontroller "myweb" created
    service "myweb" created

# 查看Tomcat创建的资源中Pod:Mysql是否运行在“Node2”节点中
[root@k8s-master tomcat_demo]# kubectl get pod -o wide -n tomcat 
    NAME          READY     STATUS    RESTARTS   AGE       IP             NODE
mysql-f4npf   1/1       Running   0          5s        172.18.95.10   k8s-node-2
    myweb-qlr3p   1/1       Running   0          5s        172.18.37.2    k8s-node-1

# 访问Web_URL,验证插入的数据依然存在
http://10.0.0.13:30008/demo/index.jsp

# 在“Node2节点”查看是否有Mount挂载记录;挂载的为同一份数据
[root@k8s-node-2 images]# df -Th |grep nfs
	10.0.0.11:/data/tomcat_mysql nfs4       48G  3.5G   45G   8% /var/lib/kubelet/pods/b3083a2b-128f-11ea-b36a-000c29c6d194/volumes/kubernetes.io~nfs/mysql

# 注意:
	存在单点故障;
	当磁盘空间不足,无法扩容

6.4 pvc:

​ pv: persistent volume 全局资源,k8s集群

​ pvc: persistent volume claim, 局部资源,属于某一个namespace

​ Pod --> PVC --> PV --> NFS

6.4.1:创建pv和pvc

上传yaml配置文件,创建pv和pvc

# 创建存放PV(C)相关Yaml文件的目录
[root@k8s-master tomcat_demo]# mkdir /root/k8s_yaml/volume && cd /root/k8s_yaml/volume
# 将之前随Mysql+Tomcat一起上传的PV(C)相关Yaml文件复制到本地
[root@k8s-master volume]# cp /root/k8s_yaml/tomcat_demo/bak_pv/mysql_pv* /root/k8s_yaml/volume/ && ls
	mysql_pvc.yaml  mysql_pv.yaml
# 创建单独存放PV数据的相关目录
[root@k8s-master volume]# mkdir /data/pv

# 创建"mysql_PV"资源
[root@k8s-master volume]# kubectl create -f mysql_pv.yaml 
	persistentvolume "tomcat-mysql" created
# 验证"PV"属于全局资源,无论查看PV资源是否指定Namespace都可以查看到
[root@k8s-master volume]# kubectl get pv
    NAME           CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      CLAIM     REASON    AGE
    tomcat-mysql   10Gi       RWX           Recycle         Available                       1m
[root@k8s-master volume]# kubectl get pv -n tomcat
    NAME           CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      CLAIM     REASON    AGE
    tomcat-mysql   10Gi       RWX           Recycle         Available                       1m
# 并且在Dashboard中也能查看到创建的PV资源
	http://10.0.0.11:8080/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard/#/persistentvolume?namespace=_all
	
# 再创建一个“PV2”;后面会根据这个设置它的挑选机制
[root@k8s-master volume]# cp mysql_pv.yaml mysql_pv2.yaml 
[root@k8s-master volume]# vim mysql_pv2.yaml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: tomcat-mysql2   #名称指定为“PV2”
  labels:
    type: nfs001
spec:
  capacity:
    storage: 50Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Recycle
  nfs:
    path: "/data/pv2"   #数据存放在“PV2”中
    server: 10.0.0.11
    readOnly: false
# 创建“PV2”的数据存储目录
[root@k8s-master volume]# mkdir /data/pv2
# 创建"mysql_PV2"资源
[root@k8s-master volume]# kubectl create -f mysql_pv2.yaml 
	persistentvolume "tomcat-mysql2" created
# 查看所有“PV”资源
[root@k8s-master volume]# kubectl get pv
    NAME            CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      CLAIM     REASON    AGE
    tomcat-mysql    10Gi       RWX           Recycle         Available                       25m
tomcat-mysql2   50Gi       RWX           Recycle         Available                       26s

# 修改PVC文件;指定挑选机制
[root@k8s-master volume]# vim mysql_pvc.yaml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: tomcat-mysql
  namespace: tomcat
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 30Gi
# 创建“PVC”资源
[root@k8s-master volume]# kubectl create -f mysql_pvc.yaml 
	persistentvolumeclaim "tomcat-mysql" created
# 再次查看“PV”资源,发现符合机制的PV:“tomcat-mysql2”被分配给了“tomcat/tomcat-mysql”
[root@k8s-master volume]# kubectl get pv
NAME            CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      CLAIM                 REASON    AGE
tomcat-mysql    10Gi       RWX           Recycle         Available                                   33m
tomcat-mysql2   50Gi       RWX           Recycle         Bound       tomcat/tomcat-mysql             8m

# 验证“PVC”为局部资源,必须指定查看的“Namespace”
[root@k8s-master volume]# kubectl get pvc -n tomcat
    NAME           STATUS    VOLUME          CAPACITY   ACCESSMODES   AGE
    tomcat-mysql   Bound     tomcat-mysql2   50Gi       RWX           2m
[root@k8s-master volume]# kubectl get pvc
	No resources found.

6.4.2:创建mysql-rc,pod模板里使用volume:PVC

[root@k8s-master volume]# cd /root/k8s_yaml/tomcat_demo/
# 移动“mysql-rc-V4.yaml”文件到“bak_pv”目录下
[root@k8s-master tomcat_demo]# mv mysql-rc-V4.yaml /root/k8s_yaml/tomcat_demo/bak_pv/
# 创建mysql-rc-V5.yaml文件
[root@k8s-master tomcat_demo]# vim mysql-rc-V5.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  namespace: tomcat
  name: mysql
spec:
  replicas: 1
  selector:
    app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      nodeName: k8s-node-2
      volumes:
      - name: mysql
        persistentVolumeClaim:
          claimName: tomcat-mysql
      containers:
        - name: mysql
          image: 10.0.0.11:5000/mysql:5.7
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 3306
          volumeMounts:
          - mountPath: /var/lib/mysql
            name: mysql
          env:
          - name: MYSQL_ROOT_PASSWORD
            value: '123456'
# 批量删除“Mysql+Tomcat”资源
[root@k8s-master tomcat_demo]# kubectl delete -f /root/k8s_yaml/tomcat_demo/.
    replicationcontroller "mysql" deleted
    service "mysql" deleted
    replicationcontroller "myweb" deleted
    service "myweb" deleted
# 批量创建Mysql+Tomcat资源
[root@k8s-master tomcat_demo]# kubectl create -f /root/k8s_yaml/tomcat_demo/.
    replicationcontroller "mysql" created
    service "mysql" created
    replicationcontroller "myweb" created
    service "myweb" created

# 查看Pod被分配到了哪个节点
[root@k8s-master tomcat_demo]# kubectl get pod -o wide -n tomcat
    NAME          READY     STATUS    RESTARTS   AGE       IP            NODE
    mysql-lfrnc   1/1       Running   0          6s        172.18.95.9   k8s-node-2
    myweb-b6dvc   1/1       Running   0          6s        172.18.37.3   k8s-node-1

# 访问Web_URL,触发Tomcat页面;插入一条数据:Name=Chiron Level=10000
http://10.0.0.13:30008/demo/index.jsp

6.4.3: 验证持久化

验证方法1:删除mysql的pod,数据库不丢

[root@k8s-master tomcat_demo]# kubectl delete -n tomcat pod mysql-lfrnc
	pod "mysql-lfrnc" deleted

[root@k8s-master tomcat_demo]# kubectl get pod -n tomcat -o wide
    NAME          READY     STATUS    RESTARTS   AGE       IP             NODE
mysql-0kcf4   1/1       Running   0          14s       172.18.95.10   k8s-node-2
    myweb-b6dvc   1/1       Running   0          7m        172.18.37.3    k8s-node-1
# 待Pod:Mysql重新运行后;查看到插入的数据依然存在
http://10.0.0.13:30008/demo/index.jsp

验证方法2:查看nfs服务端,是否有mysql的数据文件

# 查看验证指定的“PVC”下的“/data/pv2/”目录中是否有数据
[root@k8s-master tomcat_demo]# ls /data/pv2/
	auto.cnf  HPE_APP  ib_buffer_pool  ibdata1  ib_logfile0  ib_logfile1  ibtmp1  mysql  performance_schema  sys

1563694431121

6.5: 分布式存储glusterfs

a: 什么是glusterfs

Glusterfs是一个开源分布式文件系统,具有强大的横向扩展能力,可支持数PB级存储容量和数千客户端,通过网络互联成一个并行的网络文件系统。具有可扩展性、高性能、高可用性等特点。

b: 安装glusterfs

# 所有节点安装glusterfs:
yum install  centos-release-gluster -y
yum install  install glusterfs-server -y
systemctl start glusterd.service
systemctl enable glusterd.service
# 为gluster集群每个节点增加两个存储单元brick
mkdir -p /gfs/test1 /gfs/test2

c: 添加存储资源池

# master节点:
# 查看资源池节点列表
[root@k8s-master tomcat_demo]# gluster pool list
    UUID                                    Hostname        State
    cd5fe643-be60-45ed-8e10-64f11592b829    localhost       Connected 
# 加入类外两个节点到资源池
[root@k8s-master tomcat_demo]# gluster peer probe k8s-node-1
	peer probe: success. 
[root@k8s-master tomcat_demo]# gluster peer probe k8s-node-2
	peer probe: success. 
# 再次查看资源池节点列表
[root@k8s-master tomcat_demo]# gluster pool list 
UUID                                    Hostname        State
da21e75e-aacd-42bd-8f8d-9e0307280606    k8s-node-1      Connected 
b90e3f52-956e-4769-80d6-cbd7ebe3f70e    k8s-node-2      Connected 
cd5fe643-be60-45ed-8e10-64f11592b829    localhost       Connected 

d: glusterfs卷管理

#创建分布式复制卷:副本数2;不指定默认创建分布式卷
[root@k8s-master tomcat_demo]# gluster volume create Chiron replica 2 k8s-master:/gfs/test1 k8s-node-1:/gfs/test1 k8s-master:/gfs/test2  k8s-node-1:/gfs/test2 force
	volume create: Chiron: success: please start the volume to access data
# 参数解释:
	Chiron:卷名称
	replica 2:指定副本数
	force:因为官方建议副本数为"3",加上这个参数,忽略强制创建
# 启动卷
[root@k8s-master tomcat_demo]# gluster volume start Chiron
# 查看卷
[root@k8s-master tomcat_demo]# gluster volume info Chiron 
# 查看卷列表
[root@k8s-master tomcat_demo]# gluster volume list

# 所有Node节点挂载卷;其实可以挂载任何一个节点的卷
[root@k8s-node]# mount -t glusterfs 10.0.0.11:/Chiron /mnt
# 所有Node节点查看挂载
[root@k8s-node]# df -Th |grep Chiron
	10.0.0.11:/Chiron   fuse.glusterfs   48G  5.6G   43G  12% /mnt
	
# 在"Node3节点"向“/mnt”目录导入一些数据并查看里面文件与目录数量
[root@k8s-node-2 images]# cp -a /etc/rc* /mnt/
[root@k8s-node-2 images]# tree /mnt/ |tail -n 1
	16 directories, 20 files
# 在"Master节点"分别查看"/gfs/test1/"与"/gfs/test2/"目录下文件与目录数量
# 想查看完整数据,需要将卷挂载到一个挂载点钟进行查看
[root@k8s-master tomcat_demo]# tree /gfs/test1 |tail -n 1
	12 directories, 8 files
[root@k8s-master tomcat_demo]# tree /gfs/test2 |tail -n 1
	13 directories, 12 files

e: 分布式复制卷讲解

在这里插入图片描述

f: 分布式复制卷扩容

# 扩容前查看容量:
[root@k8s-node-2 images]# df -h |grep mnt
	10.0.0.11:/Chiron     48G  5.6G   43G  12% /mnt

# 扩容命令:
[root@k8s-node-2 images]# gluster volume add-brick Chiron k8s-node-2:/gfs/test1 k8s-node-2:/gfs/test2 force
	volume add-brick: success

# 扩容后查看容量:
[root@k8s-node-2 images]# df -h |grep mnt
	10.0.0.11:/Chiron     72G  9.3G   63G  13% /mnt

# 查看“/gfs/test”目录下文件与目录数量;只是放进来了目录,并没有文件
[root@k8s-node-2 images]# tree /gfs/test1/ |tail -n 1
	9 directories, 0 files
[root@k8s-node-2 images]# tree /gfs/test2/ |tail -n 1
	9 directories, 0 files
	
# 新增存储单元后,将数据均衡到新的存储单元
# 均衡数据会造成磁盘I/O过高,建议在访问量比较小时操作
[root@k8s-node-2 images]# gluster volume rebalance Chiron start
	volume rebalance: Chiron: success: Rebalance on Chiron has been started successfully. Use rebalance status command to check status of the rebalance process.
	ID: 42052f66-c0ed-41fd-ab13-805823d3ebd0
# 再次“tree”查看时,数据已经均衡到了新增存储单元
[root@k8s-node-2 images]# tree /gfs/test2 |tail -n 1
11 directories, 6 files
[root@k8s-node-2 images]# tree /gfs/test1 |tail -n 1
11 directories, 6 files

6.6 k8s 对接glusterfs存储

扩展:
endpoint资源可以理解为与SVC资源同名

[root@k8s-master tomcat_demo]# kubectl get endpoints -n tomcat 
    NAME      ENDPOINTS           AGE
    mysql     172.18.95.10:3306   5h
    myweb     172.18.37.3:8080    5h
[root@k8s-master tomcat_demo]# kubectl get  svc -n tomcat 
    NAME      CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
    mysql     10.254.84.147    <none>        3306/TCP         5h
    myweb     10.254.229.158   <nodes>       8080:30008/TCP   5h
[root@k8s-master tomcat_demo]# kubectl get pod -n tomcat -o wide
    NAME          READY     STATUS    RESTARTS   AGE       IP             NODE
    mysql-0kcf4   1/1       Running   0          5h        172.18.95.10   k8s-node-2
    myweb-b6dvc   1/1       Running   0          5h        172.18.37.3    k8s-node-1
# 创建属于glusterfs相关的yaml文件的存放目录
[root@k8s-master tomcat_demo]# mkdir /root/k8s_yaml/glusterfs && cd /root/k8s_yaml/glusterfs
[root@k8s-master glusterfs]# vim glusterfs-ep.yaml

a:创建endpoint

apiVersion: v1
kind: Endpoints
metadata:
  name: glusterfs
  namespace: tomcat
subsets:
- addresses:
  - ip: 10.0.0.11
  - ip: 10.0.0.12
  - ip: 10.0.0.13
  ports:
  - port: 49152
    protocol: TCP
[root@k8s-master glusterfs]# kubectl create -f glusterfs-ep.yaml 
	endpoints "glusterfs" created
[root@k8s-master glusterfs]# kubectl get endpoints -n tomcat 
    NAME        ENDPOINTS                                         AGE
    glusterfs   10.0.0.11:49152,10.0.0.12:49152,10.0.0.13:49152   12s
    mysql       172.18.95.10:3306                                 5h
    myweb       172.18.37.3:8080                                  5h
# 还缺少“glusterfs”的SVC做负载均衡
[root@k8s-master glusterfs]# kubectl get svc -n tomcat 
    NAME      CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
    mysql     10.254.84.147    <none>        3306/TCP         5h
    myweb     10.254.229.158   <nodes>       8080:30008/TCP   5h

b: 创建service:glusterfs-svc.yaml

[root@k8s-master glusterfs]# kubectl describe svc mysql -n tomcat
    Name:                   mysql
Namespace:              tomcat   #必须和“glusterfs-svc.yaml”中的“namespace”一致,否则无法进行关联
    Labels:                 <none>
    Selector:               app=mysql
    Type:                   ClusterIP
    IP:                     10.254.84.147
    Port:                   <unset> 3306/TCP
    Endpoints:              172.18.95.10:3306
    Session Affinity:       None
    No events.

[root@k8s-master glusterfs]# vim glusterfs-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: glusterfs
  namespace: tomcat      #必须与SVC:mysql中的“namespace”一致,否则无法进行关联
spec:
  ports:
  - port: 49152
    protocol: TCP
    targetPort: 49152
  sessionAffinity: None
  type: ClusterIP
  
[root@k8s-master glusterfs]# kubectl create -f glusterfs-svc.yaml 
	service "glusterfs" created
[root@k8s-master glusterfs]# kubectl get svc -n tomcat
	NAME        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
glusterfs   10.254.93.25     <none>        49152/TCP        6s
	mysql       10.254.84.147    <none>        3306/TCP         6h
	myweb       10.254.229.158   <nodes>       8080:30008/TCP   6h
[root@k8s-master glusterfs]# kubectl describe svc glusterfs -n tomcat
    Name:                   glusterfs
Namespace:              tomcat
    Labels:                 <none>
    Selector:               <none>
    Type:                   ClusterIP
IP:                     10.254.93.25
Port:                   <unset> 49152/TCP
Endpoints:              10.0.0.11:49152,10.0.0.12:49152,10.0.0.13:49152
    Session Affinity:       None
    No events.
[root@k8s-master glusterfs]# cd /root/k8s_yaml/tomcat_demo/
# 删除旧的“Mysql+Tomcat”资源
[root@k8s-master tomcat_demo]# kubectl delete -f /root/k8s_yaml/tomcat_demo/.
    replicationcontroller "mysql" deleted
    service "mysql" deleted
    replicationcontroller "myweb" deleted
    service "myweb" deleted
# 移动“mysql-rc-V5.yaml”文件到“bak_pv”目录下
[root@k8s-master tomcat_demo]# mv mysql-rc-V5.yaml /root/k8s_yaml/tomcat_demo/bak_pv/
# 创建mysql-rc-V6.yaml文件
[root@k8s-master tomcat_demo]# vim mysql-rc-V6.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  namespace: tomcat
  name: mysql
spec:
  replicas: 1
  selector:
    app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      nodeName: k8s-node-2
      volumes:
      - name: mysql
        glusterfs:
          endpoints: glusterfs
          path: /Chiron
      containers:
        - name: mysql
          image: 10.0.0.11:5000/mysql:5.7
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 3306
          volumeMounts:
          - mountPath: /var/lib/mysql
            name: mysql
          env:
          - name: MYSQL_ROOT_PASSWORD
            value: '123456'
# 清空前边测试用的数据;否则后面启动Pod:mysql会失败:初始化时,目录中含有数据
[root@k8s-node-2 images]# ls /mnt/
	rc0.d  rc1.d  rc2.d  rc3.d  rc4.d  rc5.d  rc6.d  rc.d  rc.local
[root@k8s-node-2 images]# rm -rf /mnt/*

# 批量创建Mysql+Tomcat资源
[root@k8s-master tomcat_demo]# kubectl create -f /root/k8s_yaml/tomcat_demo/.
    replicationcontroller "mysql" created
    service "mysql" created
    replicationcontroller "myweb" created
    service "myweb" created
# 查看创建的Namespace:tomcat中所有资源
[root@k8s-master tomcat_demo]# kubectl get all -n tomcat
    NAME       DESIRED   CURRENT   READY     AGE
    rc/mysql   1         1         1         12s
    rc/myweb   1         1         1         12s

    NAME            CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
    svc/glusterfs   10.254.93.25     <none>        49152/TCP        18m
    svc/mysql       10.254.204.230   <none>        3306/TCP         12s
    svc/myweb       10.254.106.148   <nodes>       8080:30008/TCP   12s

    NAME             READY     STATUS    RESTARTS   AGE
    po/mysql-xxg78   1/1       Running   0          12s
    po/myweb-582tl   1/1       Running   0          12s
    
# 访问Web_URL,触发Tomcat页面;插入一条数据:Name=Chiron Level=10000
http://10.0.0.13:30008/demo/index.jsp

# 在"glusterfs:Chiron"挂载出的"mnt目录"查看我们挂载的目录已经生成数据
[root@k8s-node-2 images]# ls /mnt/
	auto.cnf  HPE_APP  ib_buffer_pool  ibdata1  ib_logfile0  ib_logfile1  ibtmp1  mysql  performance_schema  sys
扩展(不做演示):

c: 创建gluster类型pv

apiVersion: v1
kind: PersistentVolume
metadata:
  name: gluster
  labels:
    type: glusterfs
spec:
  capacity:
    storage: 50Gi
  accessModes:
    - ReadWriteMany
  glusterfs:
    endpoints: "glusterfs"
    path: "Chiron"
    readOnly: false

d: 创建pvc

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: gluster
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 200Gi

e:在pod中使用gluster

vi  nginx_pod.yaml
…… 
        volumeMounts:
        - name: nfs-vol2
          mountPath: /usr/share/nginx/html
     volumes:
     - name: nfs-vol2
       persistentVolumeClaim:
         claimName: gluster

7:与jenkins集成实现cd

ip地址 服务 内存
10.0.0.11 kube-apiserver 8080 2G
10.0.0.12 jenkins(tomcat + jdk) 8080 2G
借助码云Gitee gitlab 8080,80

在这里插入图片描述

7. 1 :将本地代码上传到码云Gitee的仓库中

​ 创建“码云Gitee”存储仓库"K8S_yiliao";创建时,将初始化仓库勾选取消,其他地方保持默认即可

# 本地仓库在"Node2节点"中
# Git 全局设置:
[root@k8s-node-2 images]# git config --global user.name "ChironW"
[root@k8s-node-2 images]# git config --global user.email "18438074712@163.com"

# 创建 git 仓库:
[root@k8s-node-2 images]# mkdir /opt/gitee && cd /opt/gitee
[root@k8s-node-2 gitee]# wget http://192.168.2.254:8888/yiliaoqixie.zip
[root@k8s-node-2 gitee]# unzip yiliaoqixie.zip
[root@k8s-node-2 gitee]# cd yiliaoqixie/

# 初始化仓库
[root@k8s-node-2 yiliaoqixie]# git init
	Initialized empty Git repository in /opt/gitee/yiliaoqixie/.git/

# 提交到本地仓库
[root@k8s-node-2 yiliaoqixie]# git add /opt/gitee/yiliaoqixie/.
[root@k8s-node-2 yiliaoqixie]# git commit -m "第一版"

# 由本地仓库推送到远端仓库
[root@k8s-node-2 yiliaoqixie]# git remote add origin git@gitee.com:chironW/K8S_yiliao.git
[root@k8s-node-2 yiliaoqixie]# git push -u origin master
    The authenticity of host 'gitee.com (212.64.62.174)' can't be established.
    ECDSA key fingerprint is SHA256:FQGC9Kn/eye1W8icdBgrQp+KkGYoFgbVr17bmjey0Wc.
    ECDSA key fingerprint is MD5:27:e5:d3:f7:2a:9e:eb:6c:93:cd:1f:c1:47:a3:54:b1.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added 'gitee.com,212.64.62.174' (ECDSA) to the list of known hosts.
    Permission denied (publickey).
    fatal: Could not read from remote repository.

    Please make sure you have the correct access rights
    and the repository exists.
# 报错原因:权限被拒绝;远端没有配置“SSH-publickey”

# 本地主机生成公钥,将获取的公钥添加到“Gitee”的设置“SSH公钥”中
[root@k8s-node-2 yiliaoqixie]# ssh-keygen -t rsa
[root@k8s-node-2 yiliaoqixie]# cat /root/.ssh/id_rsa.pub
再次推送本地代码
[root@k8s-node-2 yiliaoqixie]# git push -u origin master

7.2 :安装jenkins,并自动构建docker镜像

1:安装jenkins

# “k8s-node-1节点”安装部署Jenkins:
[root@k8s-node-1 images]# cd /usr/local/src/
[root@k8s-node-1 src]# wget http://192.168.2.254:8888/apache-tomcat-8.0.27.tar.gz
[root@k8s-node-1 src]# wget http://192.168.2.254:8888/jenkin-data.tar.gz
[root@k8s-node-1 src]# wget http://192.168.2.254:8888/jdk-8u102-linux-x64.rpm
[root@k8s-node-1 src]# wget http://192.168.2.254:8888/jenkins.war

# 安装JDK
[root@k8s-node-1 src]# rpm -ivh jdk-8u102-linux-x64.rpm

# 解压部署Tomcat
[root@k8s-node-1 src]# mkdir /usr/local/src/tomcat
[root@k8s-node-1 src]# tar xf apache-tomcat-8.0.27.tar.gz -C tomcat
[root@k8s-node-1 src]# mv tomcat/apache-tomcat-8.0.27/* /usr/local/src/tomcat/
[root@k8s-node-1 src]# rm -rf tomcat/apache-tomcat-8.0.27/ /usr/local/src/tomcat/webapps/*

# 部署Jenkins
[root@k8s-node-1 src]# cd /usr/local/src/tomcat/webapps/
[root@k8s-node-1 webapps]# mv /usr/local/src/jenkins.war /usr/local/src/tomcat/webapps/ROOT.war
# 这里将之前部署好的Jenkins归档包,进行解压即可,插件等已经配置完成
[root@k8s-node-1 webapps]# tar xf /usr/local/src/jenkin-data.tar.gz -C /root/
[root@k8s-node-1 webapps]# ls /root/.jenkins/

# 启动Tomcat
[root@k8s-node-1 webapps]# /usr/local/src/tomcat/bin/startup.sh 
[root@k8s-node-1 webapps]# netstat -lntup |grep java

2:访问jenkins

​ 访问:http://10.0.0.12:8080/
​ 默认账号密码:admin/123456

3:编写Dockerfile,进行本地测试,然后推送到Git仓库中

# 编写Dockerfile文件
[root@k8s-node-2 yiliaoqixie]# vim dockerfile
FROM 10.0.0.11:5000/nginx:1.17
ADD . /usr/share/nginx/html

# Build测试镜像并运行查看
[root@k8s-node-2 yiliaoqixie]# docker build -t yiliao:v1 .
[root@k8s-node-2 yiliaoqixie]# docker run -d -P yiliao:v1 
[root@k8s-node-2 yiliaoqixie]# docker ps -a -l
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                   NAMES
afb48da6470b        yiliao:v1           "nginx -g 'daemon ..."   20 seconds ago      Up 19 seconds       0.0.0.0:32768->80/tcp   festive_hypatia

# Web浏览器访问验证:
	http://10.0.0.13:32768/
	
# 推送“dockerfile”文件到gitee仓库
[root@k8s-node-2 yiliaoqixie]# git add .
[root@k8s-node-2 yiliaoqixie]# git commit -m 'add dockerfile'
[root@k8s-node-2 yiliaoqixie]# git push -u origin master

4:配置jenkins拉取Git仓库代码凭据

a:在jenkins上生成秘钥对;将获取的公钥添加到“K8S_yiliao”仓库的管理-公钥中

[root@k8s-node-1 webapps]# ssh-keygen -t rsa
[root@k8s-node-1 webapps]# cat /root/.ssh/id_rsa.pub

b:将一起生成的私钥添加到Jenkins的全局凭据中;类型:SSH Username with private key

[root@k8s-node-1 webapps]# cat /root/.ssh/id_rsa

在这里插入图片描述

5:Jenkins创建自由风格的工程“yiliao”;拉取代码测试

在这里插入图片描述

​ Jenkins构建后;可以查看到拉取的代码

[root@k8s-node-1 webapps]# ls /root/.jenkins/workspace/yiliao/

6:更改Jenkins配置中执行的shell命令;使其创建Docker镜像,并推送到私有仓库

docker build -t 10.0.0.11:5000/yiliao:v1 .
docker push 10.0.0.11:5000/yiliao:v1

​ 再次构建后,在私有仓库中查看是否有“yiliao:v1”的镜像

[root@k8s-master ~]# ls /opt/myregistry/docker/registry/v2/repositories/yiliao/_manifests/tags/
	v1

7:Jenkins使用参数化构建,自动构建docker镜像并上传到私有仓库

Gitee仓库中直接修改index.html文件中的标题为“ChironW医疗器械有限公司”;添加提交信息为“修改了首页标题v2”
使用参数化构建(文本参数:version),指定构建时Build镜像的版本号:执行shell语句变为:

if [ -z $version ];then
	echo "请输入版本!"
	exit 4
fi

docker build -t 10.0.0.11:5000/yiliao:$version .
docker push 10.0.0.11:5000/yiliao:$version

在这里插入图片描述

​ 再次构建;指定构建版本号为“v2”;在私有仓库中查看是否有“yiliao:v2”的镜像

[root@k8s-master ~]# ls /opt/myregistry/docker/registry/v2/repositories/yiliao/_manifests/tags/
	v1  v2

7.3 手动部署实现“yiliao”的发布,升级,回滚;方面理解后面通过 Jenkins自动化部署应用

# Node节点执行“kubectl”命令方法:
[root@k8s-node-1 webapps]# kubectl -s 10.0.0.11:8080 get pod -n tomcat

# 发布
# Master节点部署“yiliao”相关资源
# 创建Namespace资源
[root@k8s-master ~]# kubectl create namespace yiliao
	namespace "yiliao" created
# 创建Deployment资源
[root@k8s-master ~]# kubectl run -n yiliao yiliao --image=10.0.0.11:5000/yiliao:v1 --replicas=2 --record 
	deployment "yiliao" created
# 查看历史版本操作
[root@k8s-master ~]# kubectl rollout history -n yiliao deployment yiliao
	deployments "yiliao"
	REVISION        CHANGE-CAUSE
	1               kubectl run -n yiliao yiliao --image=10.0.0.11:5000/yiliao:v1 --replicas=2 --record
# 创建Service资源
[root@k8s-master ~]# kubectl expose -n yiliao deployment yiliao --port=80 --target-port=80 --type=NodePort
	service "yiliao" exposed
#  查看"Namespace:yiliao"中的所有资源
[root@k8s-master ~]# kubectl get all -n yiliao 
    NAME            DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
    deploy/yiliao   2         2         2            2           1m

    NAME         CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
    svc/yiliao   10.254.32.77   <nodes>       80:32175/TCP   35s

    NAME                   DESIRED   CURRENT   READY     AGE
    rs/yiliao-4098334581   2         2         2         1m

    NAME                         READY     STATUS    RESTARTS   AGE
    po/yiliao-4098334581-9148v   1/1       Running   0          1m
    po/yiliao-4098334581-vd5j0   1/1       Running   0          1m
# Web浏览器访问验证:标题为“某某医疗”
http://10.0.0.12:32175/
http://10.0.0.13:32175/

# 升级
# 升级版本:
[root@k8s-master ~]# kubectl set image -n yiliao deployment yiliao yiliao=10.0.0.11:5000/yiliao:v2
	deployment "yiliao" image updated
# 查看历史版本操作
[root@k8s-master ~]# kubectl rollout history -n yiliao deployment yiliao
	deployments "yiliao"
	REVISION        CHANGE-CAUSE
	1               kubectl run -n yiliao yiliao --image=10.0.0.11:5000/yiliao:v1 --replicas=2 --record
	2               kubectl set image -n yiliao deployment yiliao yiliao=10.0.0.11:5000/yiliao:v2
# Web浏览器访问验证:标题为“ChironW医疗”
http://10.0.0.12:32175/
http://10.0.0.13:32175/

# 回滚
# 回滚到上一个版本:
[root@k8s-master ~]# kubectl rollout undo -n yiliao deployment yiliao
	deployment "yiliao" rolled back
# 查看历史版本操作
[root@k8s-master ~]# kubectl rollout history -n yiliao deployment yiliao
	deployments "yiliao"
	REVISION        CHANGE-CAUSE
	2               kubectl set image -n yiliao deployment yiliao yiliao=10.0.0.11:5000/yiliao:v2
	3               kubectl run -n yiliao yiliao --image=10.0.0.11:5000/yiliao:v1 --replicas=2 --record
# Web浏览器访问验证:标题为“某某医疗”
http://10.0.0.12:32175/
http://10.0.0.13:32175/

# 回滚到指定版本“REVISION:2”
[root@k8s-master ~]# kubectl rollout undo -n yiliao deployment yiliao --to-revision=2
	deployment "yiliao" rolled back
# 查看历史版本操作
[root@k8s-master ~]# kubectl rollout history -n yiliao deployment yiliao
    deployments "yiliao"
    REVISION        CHANGE-CAUSE
    3               kubectl run -n yiliao yiliao --image=10.0.0.11:5000/yiliao:v1 --replicas=2 --record
    4               kubectl set image -n yiliao deployment yiliao yiliao=10.0.0.11:5000/yiliao:v2
# Web浏览器访问验证:标题为“ChironW医疗”
http://10.0.0.12:32175/
http://10.0.0.13:32175/
# 删除相关"Namespace:yiliao"资源;方便后面演示Jenkins自动部署应用到K8S
[root@k8s-master ~]# kubectl delete namespace yiliao 
	namespace "yiliao" deleted
[root@k8s-master ~]# kubectl get all -n yiliao 
	No resources found.

7.4 jenkins自动部署应用到k8s

使用参数化构建(文本参数:version)发布与更新:
更改Jenkins工程中执行Shell为:

if [ -z $version ];then
	echo "请输入版本!"
	exit 4
fi

if [ -f /tmp/yiliao.lock ];then
	docker  build  -t  10.0.0.11:5000/yiliao:$version  .
	docker  push 10.0.0.11:5000/yiliao:$version
	kubectl -s 10.0.0.11:8080 set image  -n yiliao deploy yiliao yiliao=10.0.0.11:5000/yiliao:$version
	port=`kubectl -s 10.0.0.11:8080  get svc -n yiliao|grep -oP '(?<=80:)\d+'`
	echo "你的项目地址访问是http://10.0.0.13:$port"
	echo "你的项目地址访问是http://10.0.0.12:$port"
	echo "更新成功"
else
	docker  build  -t  10.0.0.11:5000/yiliao:$version  .
	docker  push 10.0.0.11:5000/yiliao:$version
	kubectl  -s 10.0.0.11:8080  create  namespace  yiliao
	kubectl  -s 10.0.0.11:8080  run   yiliao  -n yiliao  --image=10.0.0.11:5000/yiliao:$version --replicas=3 --record
	kubectl  -s 10.0.0.11:8080   expose -n yiliao deployment yiliao --port=80 --type=NodePort
	port=`kubectl -s 10.0.0.11:8080  get svc -n yiliao|grep -oP '(?<=80:)\d+'`
	echo "你的项目地址访问是http://10.0.0.13:$port"
	echo "你的项目地址访问是http://10.0.0.12:$port"
	echo "发布成功"
	touch /tmp/yiliao.lock
	chattr +i /tmp/yiliao.lock
fi
# 发布
# 再次构建;指定构建版本号为“v1”;
# Web浏览器访问验证:标题为“ChironW医疗”
http://10.0.0.13:31900
http://10.0.0.12:31900

# 升级
# Gitee中直接修改index.html文件中的标题为“Chiron-2医疗器械有限公司”;添加提交信息为“修改了首页标题v3”
# 再次构建;指定构建版本号为“v2”;
# Web浏览器访问验证:标题为“ChironW-2医疗”
http://10.0.0.13:31900
http://10.0.0.12:31900

​ 回滚版本:

# Jenkins自动化回滚

# 回滚到上一版本
# Jenkins创建自由风格的工程“yiliao_undo”;执行Shell语句为:(未做参数化构建默认回滚上一版本)
kubectl -s 10.0.0.11:8080 rollout undo -n yiliao deployment yiliao
port=`kubectl -s 10.0.0.11:8080  get svc -n yiliao|grep -oP '(?<=80:)\d+'`
echo "你的项目地址访问是http://10.0.0.13:$port"
echo "你的项目地址访问是http://10.0.0.12:$port"
echo "回滚成功"
# 构建工程“yiliao_undo”
# Web浏览器访问验证:标题为“ChironW医疗”
http://10.0.0.13:31900
http://10.0.0.12:31900

# 回滚到指定版本
# Jenkins使用参数化构建回滚到指定版本;执行Shell语句为:(参数化构建回滚到指定版本)
if [ -z $version ];then
	echo "未输入回滚版本号,默认回滚到上一版本!"
	kubectl -s 10.0.0.11:8080 rollout undo -n yiliao deployment yiliao
	port=`kubectl -s 10.0.0.11:8080  get svc -n yiliao|grep -oP '(?<=80:)\d+'`
	echo "你的项目地址访问是http://10.0.0.13:$port"
	echo "你的项目地址访问是http://10.0.0.12:$port"
	echo "回滚上一版本成功"
else
	echo "回滚到指定版本:$REVISION!"
	kubectl -s 10.0.0.11:8080 rollout undo -n yiliao deployment yiliao --to-revision=$REVISION
	port=`kubectl -s 10.0.0.11:8080  get svc -n yiliao|grep -oP '(?<=80:)\d+'`
	echo "你的项目地址访问是http://10.0.0.13:$port"
	echo "你的项目地址访问是http://10.0.0.12:$port"
	echo "回滚到指定版本:$REVISION成功"
fi

	echo "你的所有历史版本如下"
	kubectl -s 10.0.0.11:8080 rollout history -n yiliao deployment yiliao
# 构建工程“yiliao_undo”;不填写构建参数使其回滚到上一版本
# Web浏览器访问验证:标题为“ChironW-2医疗”
	http://10.0.0.13:31900
	http://10.0.0.12:31900
# 再次构建工程“yiliao_undo”;填写构建参数指定回滚到“REVISION=3”版本
# Web浏览器访问验证:标题为“ChironW医疗”
	http://10.0.0.13:31900
	http://10.0.0.12:31900

8:部署Prometheus+Grafana

Node节点安装cadvisor和node-exporter

​ cadvisor在安装“kubelet”时,已经包含进去了,此处可省略

# 所有“Node节点”上传cadvisor和node-exporter镜像文件并导入
[root@k8s-node images]# wget http://192.168.2.254:8888/docker_monitor_node.tar.gz
[root@k8s-node images]# docker load -i docker_monitor_node.tar.gz
# 运行“node-exporter”容器
[root@k8s-node images]# docker run -d -p 9100:9100 -v "/:/host:ro,rslave" --name=node_exporter   quay.io/prometheus/node-exporter --path.rootfs /host

# cadvisor在安装“kubelet”时,已经包含进去了,此处可省略
[root@k8s-node images]# docker run --volume=/:/rootfs:ro  --volume=/var/run:/var/run:rw --volume=/sys:/sys:ro --volume=/var/lib/docker/:/var/lib/docker:ro  --publish=8080:8080 --detach=true --name=cadvisor google/cadvisor:latest

Master节点部署Prometheus+Grafana

# Master节点部署Prometheus+Grafana
# 部署Prometheus
[root@k8s-master /]# cd /opt/
# 上传本地prometheus归档包并解压
[root@k8s-master opt]# wget http://192.168.2.254:8888/prometheus-2.12.0.linux-amd64.tar.gz
[root@k8s-master opt]# tar xf prometheus-2.12.0.linux-amd64.tar.gz 
[root@k8s-master opt]# cd prometheus-2.12.0.linux-amd64/
[root@k8s-master prometheus-2.12.0.linux-amd64]# cp prometheus.yml{,.bak}
# 修改文件“prometheus.yml”为以下内容
[root@k8s-master prometheus-2.12.0.linux-amd64]# grep -Ev '^$|#' prometheus.yml
global:
alerting:
  alertmanagers:
  - static_configs:
    - targets:
rule_files:
scrape_configs:
  - job_name: 'prometheus'
    static_configs:
    - targets: ['localhost:9090']
  - job_name: 'cadvisor'
    static_configs:
    - targets: ['10.0.0.12:4194','10.0.0.13:4194']
  - job_name: 'node'
    static_configs:
    - targets: ['10.0.0.12:9100','10.0.0.13:9100']
# 启动prometheus服务,默认运行在前台
[root@k8s-master prometheus-2.12.0.linux-amd64]# ./prometheus --config.file="prometheus.yml"
# Web浏览器访问
http://10.0.0.11:9090/

# 部署Grafana
# 上传本地RPM包:Grafana进行安装
[root@k8s-master ~]# wget http://192.168.2.254:8888/grafana-6.3.3-1.x86_64.rpm
[root@k8s-master ~]# yum localinstall grafana-6.3.3-1.x86_64.rpm -y
# 启动Grafana服务并加入开机自启
[root@k8s-master ~]# systemctl start grafana-server.service
[root@k8s-master ~]# systemctl enable grafana-server.service
# 查看Grafana服务占用端口号
[root@k8s-master ~]# netstat -tnpl |grep grafana
	tcp6       0      0 :::3000                 :::*                    LISTEN      14962/grafana-serve 
# Web浏览器访问
http://10.0.0.11:3000/login
默认用户名密码:admin/admin
# 配置prometheus数据源不做详解

#导入事先准备好的dashboard出图模板“grafana_docker_dashboard.json”文件

# 模板文件可以从“grafana”官方进行搜索下载:
https://grafana.com/grafana/dashboards

标题为“ChironW医疗”
http://10.0.0.13:31900
http://10.0.0.12:31900

回滚到指定版本

Jenkins使用参数化构建回滚到指定版本;执行Shell语句为:(参数化构建回滚到指定版本)

if [ -z KaTeX parse error: Undefined control sequence: \d at position 182: …p -oP '(?<=80:)\̲d̲+'` echo "你的项目…port"
echo “你的项目地址访问是http://10.0.0.12:port"echo""elseecho":port" echo "回滚上一版本成功" else echo "回滚到指定版本:REVISION!”
kubectl -s 10.0.0.11:8080 rollout undo -n yiliao deployment yiliao --to-revision=KaTeX parse error: Undefined control sequence: \d at position 80: …p -oP '(?<=80:)\̲d̲+'` echo "你的项目…port"
echo “你的项目地址访问是http://10.0.0.12:port"echo":port" echo "回滚到指定版本:REVISION成功”
fi

echo "你的所有历史版本如下"
kubectl -s 10.0.0.11:8080 rollout history -n yiliao deployment yiliao

构建工程“yiliao_undo”;不填写构建参数使其回滚到上一版本

Web浏览器访问验证:标题为“ChironW-2医疗”

http://10.0.0.13:31900
http://10.0.0.12:31900

再次构建工程“yiliao_undo”;填写构建参数指定回滚到“REVISION=3”版本

Web浏览器访问验证:标题为“ChironW医疗”

http://10.0.0.13:31900
http://10.0.0.12:31900

## 8:部署Prometheus+Grafana

### Node节点安装cadvisor和node-exporter

​	cadvisor在安装“kubelet”时,已经包含进去了,此处可省略

```shell
# 所有“Node节点”上传cadvisor和node-exporter镜像文件并导入
[root@k8s-node images]# wget http://192.168.2.254:8888/docker_monitor_node.tar.gz
[root@k8s-node images]# docker load -i docker_monitor_node.tar.gz
# 运行“node-exporter”容器
[root@k8s-node images]# docker run -d -p 9100:9100 -v "/:/host:ro,rslave" --name=node_exporter   quay.io/prometheus/node-exporter --path.rootfs /host

# cadvisor在安装“kubelet”时,已经包含进去了,此处可省略
[root@k8s-node images]# docker run --volume=/:/rootfs:ro  --volume=/var/run:/var/run:rw --volume=/sys:/sys:ro --volume=/var/lib/docker/:/var/lib/docker:ro  --publish=8080:8080 --detach=true --name=cadvisor google/cadvisor:latest

Master节点部署Prometheus+Grafana

# Master节点部署Prometheus+Grafana
# 部署Prometheus
[root@k8s-master /]# cd /opt/
# 上传本地prometheus归档包并解压
[root@k8s-master opt]# wget http://192.168.2.254:8888/prometheus-2.12.0.linux-amd64.tar.gz
[root@k8s-master opt]# tar xf prometheus-2.12.0.linux-amd64.tar.gz 
[root@k8s-master opt]# cd prometheus-2.12.0.linux-amd64/
[root@k8s-master prometheus-2.12.0.linux-amd64]# cp prometheus.yml{,.bak}
# 修改文件“prometheus.yml”为以下内容
[root@k8s-master prometheus-2.12.0.linux-amd64]# grep -Ev '^$|#' prometheus.yml
global:
alerting:
  alertmanagers:
  - static_configs:
    - targets:
rule_files:
scrape_configs:
  - job_name: 'prometheus'
    static_configs:
    - targets: ['localhost:9090']
  - job_name: 'cadvisor'
    static_configs:
    - targets: ['10.0.0.12:4194','10.0.0.13:4194']
  - job_name: 'node'
    static_configs:
    - targets: ['10.0.0.12:9100','10.0.0.13:9100']
# 启动prometheus服务,默认运行在前台
[root@k8s-master prometheus-2.12.0.linux-amd64]# ./prometheus --config.file="prometheus.yml"
# Web浏览器访问
http://10.0.0.11:9090/

# 部署Grafana
# 上传本地RPM包:Grafana进行安装
[root@k8s-master ~]# wget http://192.168.2.254:8888/grafana-6.3.3-1.x86_64.rpm
[root@k8s-master ~]# yum localinstall grafana-6.3.3-1.x86_64.rpm -y
# 启动Grafana服务并加入开机自启
[root@k8s-master ~]# systemctl start grafana-server.service
[root@k8s-master ~]# systemctl enable grafana-server.service
# 查看Grafana服务占用端口号
[root@k8s-master ~]# netstat -tnpl |grep grafana
	tcp6       0      0 :::3000                 :::*                    LISTEN      14962/grafana-serve 
# Web浏览器访问
http://10.0.0.11:3000/login
默认用户名密码:admin/admin
# 配置prometheus数据源不做详解

#导入事先准备好的dashboard出图模板“grafana_docker_dashboard.json”文件

# 模板文件可以从“grafana”官方进行搜索下载:
https://grafana.com/grafana/dashboards

本文链接: http://www.dtmao.cc/news_show_150351.shtml

附件下载

相关教程

    暂无相关的数据...

共有条评论 网友评论

验证码: 看不清楚?