Kubernetes集群组件:

• etcd 一个高可用的K/V键值对存储和服务发现系统
• flannel 实现夸主机的容器网络的通信
• kube-apiserver 提供kubernetes集群的API调用
• kube-controller-manager 确保集群服务
• kube-scheduler 调度容器，分配到Node
• kubelet 在Node节点上按照配置文件中定义的容器规格启动容器
• kube-proxy 提供网络代理服务

部署kubernetes集群

集群环境： 系统:centos7u4

本次实验使用三台机器用于部署k8s的运行环境，1台master,2台node：细节如下

设置三台机器的主机名：

Master上执行：
# hostnamectl --static set-hostname  k8s-master
Node1上执行：
# hostnamectl --static set-hostname  k8s-node-1
Node2上执行：
# hostnamectl --static set-hostname  k8s-node-2

修改每台机器的hosts文件互相解析：

192.168.245.250 k8s-master
192.168.245.250 etcd
192.168.245.250 registry
192.168.245.251 k8s-node-1
192.168.245.252 k8s-node-2

所有机器关闭防火墙和selinux:

# systemctl stop firewalld && systemctl disable firewalld && setenforce 0 

所有机器安装epel-release源

# yum -y install epel-release

一、部署master

1.安装etcd数据库

使用yum安装etcd

etcd服务作为Kubernetes集群的主数据库,在安装Kubernetes各服务之前需要首先安装和启动。

yum -y install etcd

编辑/etc/etcd/etcd.conf文件(修改开头带两个星号部分3行)

#[Member]
#ETCD_CORS=""
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_WAL_DIR=""
#ETCD_LISTEN_PEER_URLS="http://localhost:2380"
#指定监听的网卡和端口
**ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379,http://0.0.0.0:4001"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#指定etcd的服务名称
**ETCD_NAME="master"
#ETCD_SNAPSHOT_COUNT="100000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
#ETCD_QUOTA_BACKEND_BYTES="0"
#ETCD_MAX_REQUEST_BYTES="1572864"
#ETCD_GRPC_KEEPALIVE_MIN_TIME="5s"
#ETCD_GRPC_KEEPALIVE_INTERVAL="2h0m0s"
#ETCD_GRPC_KEEPALIVE_TIMEOUT="20s"
# #[Clustering]
#ETCD_INITIAL_ADVERTISE_PEER_URLS="http://localhost:2380"
#别人访问etcd数据库可以使用的路径和端口
**ETCD_ADVERTISE_CLIENT_URLS="http://etcd:2379,http://etcd:4001"

启动并验证状态:

# systemctl start etcd

#测试etcd数据的可用性
# etcdctl set testdir/testkey0 0
0
# etcdctl get testdir/testkey0 
0
#测试访问地址是否是可用的
# etcdctl -C http://etcd:4001 cluster-health
member 8e9e05c52164694d is healthy: got healthy result from http://0.0.0.0:2379
cluster is healthy
# etcdctl -C http://etcd:2379 cluster-health
member 8e9e05c52164694d is healthy: got healthy result from http://0.0.0.0:2379
cluster is healthy

2.安装Docker

使用yum安装docker

[root@k8s-master ~]# yum install docker -y

配置Docker配置文件，使其允许从registry中拉取镜像

[root@k8s-master ~]# vim /etc/sysconfig/docker

# /etc/sysconfig/docker
# Modify these options if you want to change the way the docker daemon runs
OPTIONS='--selinux-enabled --log-driver=journald --signature-verification=false'
if [ -z "${DOCKER_CERT_PATH}" ]; then
    DOCKER_CERT_PATH=/etc/docker
fi
#设置docker的5000端口可用
**OPTIONS='--insecure-registry registry:5000'

设置开机自启动并开启服务

[root@k8s-master ~]# systemctl enable docker

[root@k8s-master ~]# systemctl start docker 

3.安装kubernetes

使用yum安装kubernetes

[root@k8s-master ~]# yum install kubernetes -y

配置并启动kubernetes

在kubernetes master上需要运行以下组件：

kubernetes API Server
kubernetes Controller Manager
kubernetes Scheduler

相应的要更改以下几个配置中带**部分信息：

[root@k8s-master ~]# vim /etc/kubernetes/apiserver
###
# kubernetes system config
#
# The following values are used to configure the kube-apiserver
#
# The address on the local server to listen to.
#需要监听的地址
**KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"

# The port on the local server to listen on.
#端口
**KUBE_API_PORT="--port=8080"

# Port minions listen on
# KUBELET_PORT="--kubelet-port=10250"

# Comma separated list of nodes in the etcd cluster
#告诉etcd的访问地址
**KUBE_ETCD_SERVERS="--etcd-servers=http://etcd:2379"

# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"

# default admission control policies
#KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"
#api sever可以管理的控制器有哪些
**KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"

# Add your own!
KUBE_API_ARGS=""

[root@k8s-master ~]# vim /etc/kubernetes/config

###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
# kube-apiserver.service
# kube-controller-manager.service
# kube-scheduler.service
# kubelet.service
# kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"

# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"

# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=false"

# How the controller-manager, scheduler, and proxy find the apiserver
#设置master的访问路径和端口
**KUBE_MASTER="--master=http://k8s-master:8080"

启动服务并设置开机自启动

[root@k8s-master ~]# systemctl enable kube-apiserver.service
[root@k8s-master ~]# systemctl start kube-apiserver.service

#控制器服务
[root@k8s-master ~]# systemctl enable kube-controller-manager.service
[root@k8s-master ~]# systemctl start kube-controller-manager.service

#调度器服务
[root@k8s-master ~]# systemctl enable kube-scheduler.service
[root@k8s-master ~]# systemctl start kube-scheduler.service

二、部署node

1.安装docker

安装配置启动docker(两台node一样)

# yum install docker -y

配置Docker配置文件，使其允许从registry中拉取镜像

# vim /etc/sysconfig/docker
# /etc/sysconfig/docker

# Modify these options if you want to change the way the docker daemon runs
OPTIONS='--selinux-enabled --log-driver=journald --signature-verification=false'
if [ -z "${DOCKER_CERT_PATH}" ]; then
    DOCKER_CERT_PATH=/etc/docker
fi
#5000端口打开
**OPTIONS='--insecure-registry registry:5000'

设置开机自启动并开启服务

[root@k8s-master ~]# systemctl enable docker
[root@k8s-master ~]# systemctl start docker 

2.安装kubernetes

安装配置启动kubernetes(两台node一样)

# yum install kubernetes -y

配置并启动kubernetes

在kubernetes node上需要运行以下组件：

Kubelet
kubernetes Proxy

相应的要更改以下几个配置文中带**部分信息：

[root@K8s-node-1 ~]# vim /etc/kubernetes/config
###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
# kube-apiserver.service
# kube-controller-manager.service
# kube-scheduler.service
# kubelet.service
# kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"

# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"

# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=false"

# How the controller-manager, scheduler, and proxy find the apiserver
#指定master的地址和端口
**KUBE_MASTER="--master=http://k8s-master:8080"

[root@K8s-node-1 ~]# vim /etc/kubernetes/kubelet
###
# kubernetes kubelet (minion) config
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
#监听地址
**KUBELET_ADDRESS="--address=0.0.0.0"

# The port for the info server to serve on
# KUBELET_PORT="--port=10250"

# You may leave this blank to use the actual hostname
#自己的主机名称 k8s-node-2里面写k8s-node-1
**KUBELET_HOSTNAME="--hostname-override=k8s-node-1"

# location of the api-server
#指定master
**KUBELET_API_SERVER="--api-servers=http://k8s-master:8080"

# pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"

# Add your own!
KUBELET_ARGS=""

启动服务并设置开机自启动

[root@k8s-master ~]# systemctl enable kubelet.service
[root@k8s-master ~]# systemctl start kubelet.service
[root@k8s-master ~]# systemctl enable kube-proxy.service
[root@k8s-master ~]# systemctl start kube-proxy.service

查看状态：

在master上查看集群中节点及节点状态

#测试k8s路径和端口对不对
[root@k8s-master ~]# kubectl -s http://k8s-master:8080 get node
NAME         STATUS    AGE
127.0.0.1    Ready     27d    #这行不用管
k8s-node-1   Ready     3m
k8s-node-2   Ready     16s
[root@k8s-master ~]# kubectl get nodes
NAME         STATUS    AGE
k8s-node-1   Ready     3m
k8s-node-2   Ready     43s

三、创建覆盖网络——Flannel

安装Flannel

在master、node上均执行如下命令，进行安装
[root@k8s-master ~]# yum install flannel -y

配置Flannel

master、node上均编辑/etc/sysconfig/flanneld，修改**部分

[root@k8s-master ~]# vi /etc/sysconfig/flanneld
# Flanneld configuration options
# etcd url location. Point this to the server where etcd runs
#连接etcd
**FLANNEL_ETCD_ENDPOINTS="http://etcd:2379"

# etcd config key. This is the configuration key that flannel queries
# For address range assignment
FLANNEL_ETCD_PREFIX="/atomic.io/network"

# Any additional options that you want to pass
#FLANNEL_OPTIONS=""

配置etcd中关于flannel的key（不用做）

Flannel使用Etcd进行配置，来保证多个Flannel实例之间的配置一致性，所以需要在etcd上进行如下配置：
（‘/atomic.io/network/config’这个key与上文/etc/sysconfig/flanneld中的配置项FLANNEL_ETCD_PREFIX是相对应的，错误的话启动就会出错）

master   10.0.0.0
node1  10.18.0.0
node2  10.17.0.0

管理员配置flannel使用的network，并将配置保存在 etcd 中:

#所有的节点都使用这个网段 确保命令中的路径和FLANNEL_ETCD_PREFIX是一致的
[root@k8s-master ~]# etcdctl mk /atomic.io/network/config '{ "Network": "10.0.0.0/16" }'
{ "Network": "10.0.0.0/16" }

在每个 minion 节点上，flannel 启动。

它从 etcd 中获取 network 配置，并为本节点产生一个 subnet，也保存在 etcd 中。并且产生 /run/flannel/subnet.env 文件：

FLANNEL_NETWORK=10.0.0.0/16  #这是全局的 falnnel network
FLANNEL_SUBNET=10.0.78.1/24   #这是本节点上 falnnel subnet
FLANNEL_MTU=1400             #本节点上 flannel mtu
FLANNEL_IPMASQ=false

四、启动

启动Flannel之后，需要依次重启docker、kubernete。

在master执行：

systemctl enable flanneld.service 
systemctl start flanneld.service 
#看节点运行的状态
systemctl status flanneld.service 

systemctl restart  docker
systemctl restart kube-apiserver.service
systemctl restart kube-controller-manager.service
systemctl restart kube-scheduler.service  

重启所有服务(排错或者节点关闭的时候，需要重新启动时用)

systemctl restart flanneld.service &&  service docker restart &&  systemctl restart kube-apiserver.service &&  systemctl restart kube-controller-manager.service && systemctl restart kube-scheduler.service

在node上执行：

systemctl enable flanneld.service 
systemctl start flanneld.service 
systemctl restart docker 
systemctl restart kubelet.service
systemctl restart kube-proxy.service

重启所有服务(排错或者节点关闭的时候，需要重新启动时用)

systemctl restart flanneld.service &&  service docker restart &&  systemctl restart kubelet.service &&  systemctl restart kube-proxy.service