centos7 使用kubeadm部署k8s
作者：互联网
配置环境
本次配置共准备master、node1、node2三台服务器，环境配置三台机器保持同步
1.安装docker
参考文章centos安装docker
2.关闭防火墙
关闭防火墙并禁用开机启动
systemctl stop firewalld & systemctl disable firewalld
3.关闭Swap
目的在于防止内存交换影响性能以及稳定性
swapoff -a可临时关闭，系统重启后恢复- 编辑
/etc/fstab注释包含swap一行重启之后可永久关闭
快速命令：
swapoff -a
sed -i '/swap/s/^/#/' /etc/fstab
4.关闭SeLinux
目的在于允许容器访问宿主机的文件系统
setenforce 0
sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
5.配置k8s国内yum源
由于墙的存在，需要配置国内的下载源，使用如下命令添加
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
6.修改docker引擎
修改docker cgroup驱动，与k8s一致，防止因为引擎不一致而导致的启动错误
vi /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
],
"registry-mirrors": ["https://registry.docker-cn.com","http://hub-mirror.c.163.com"]
}
执行如下命令重启docker
systemctl daemon-reload
systemctl restart docker
7.安装k8s组件
执行命令(所有节点机都需要安装)
# 下载相关组件
yum install -y kubelet-1.15.1 kubeadm-1.15.1 kubectl-1.15.1
# 设置开机启动 kubelet
systemctl enable --now kubelet
8.设置路由
centos7需要设置路由
yum install -y bridge-utils.x86_64
# 加载br_netfilter模块，使用lsmod查看开启的模块
modprobe br_netfilter
# 修改配置文件
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
# 重新加载配置文件
sysctl --system
创建集群
集群创建master机器与node机器有所区别，文中会做出区分
1.拉取集群所需镜像
把所需的镜像下载好，init的时候就不会再拉镜像，由于无法连接google镜像库导致出错
首先拉取国内对应镜像资源
- master服务器：
docker pull mirrorgooglecontainers/kube-apiserver:v1.15.1
docker pull mirrorgooglecontainers/kube-controller-manager:v1.15.1
docker pull mirrorgooglecontainers/kube-scheduler:v1.15.1
docker pull mirrorgooglecontainers/kube-proxy:v1.15.1
docker pull mirrorgooglecontainers/pause:3.1
docker pull mirrorgooglecontainers/etcd:3.3.10
docker pull coredns/coredns:1.3.1
- node服务器：
docker pull mirrorgooglecontainers/kube-proxy:v1.15.1
docker pull mirrorgooglecontainers/pause:3.1
将镜像tag名称修改为k8s官方一致
- master服务器：
docker tag mirrorgooglecontainers/kube-apiserver:v1.15.1 k8s.gcr.io/kube-apiserver:v1.15.1
docker tag mirrorgooglecontainers/kube-controller-manager:v1.15.1 k8s.gcr.io/kube-controller-manager:v1.15.1
docker tag mirrorgooglecontainers/kube-scheduler:v1.15.1 k8s.gcr.io/kube-scheduler:v1.15.1
docker tag mirrorgooglecontainers/kube-proxy:v1.15.1 k8s.gcr.io/kube-proxy:v1.15.1
docker tag mirrorgooglecontainers/pause:3.1 k8s.gcr.io/pause:3.1
docker tag mirrorgooglecontainers/etcd:3.3.10 k8s.gcr.io/etcd:3.3.10
docker tag coredns/coredns:1.3.1 k8s.gcr.io/coredns:1.3.1
- node服务器：
docker tag mirrorgooglecontainers/kube-proxy:v1.15.1 k8s.gcr.io/kube-proxy:v1.15.1
docker tag mirrorgooglecontainers/pause:3.1 k8s.gcr.io/pause:3.1
删除原来的镜像
- master服务器：
docker rmi mirrorgooglecontainers/kube-apiserver:v1.15.1
docker rmi mirrorgooglecontainers/kube-controller-manager:v1.15.1
docker rmi mirrorgooglecontainers/kube-scheduler:v1.15.1
docker rmi mirrorgooglecontainers/kube-proxy:v1.15.1
docker rmi mirrorgooglecontainers/pause:3.1
docker rmi mirrorgooglecontainers/etcd:3.3.10
docker rmi coredns/coredns:1.3.1
- node服务器：
docker rmi mirrorgooglecontainers/kube-proxy:v1.15.1
docker rmi mirrorgooglecontainers/pause:3.1
2.master机器创建集群
修改master机器hostname为master
hostnamectl set-hostname master
创建集群
# --apiserver-advertise-address 本机ip
# --kubernetes-version 指定镜像版本
# --service-cidr 服务使用IP地址的替代范围
# --pod-network-cidr 指定Pod网络的IP地址范围。如果设置，控制平面将自动为每个节点分配CIDR
kubeadm init --apiserver-advertise-address 192.168.0.108 --kubernetes-version 1.15.1 --service-cidr 10.1.0.0/16 --pod-network-cidr 10.244.0.0/16
看到显示下列信息代表集群启动成功
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.0.108:6443 --token y9e8yj.a2olglq2lurpmow7 \
--discovery-token-ca-cert-hash sha256:43e3035c08345a303afd696ed8a87d3b8a2ebd2c875d2fb30d2eeea47594447c
上述信息表示我们可以通过运行此行命令将node服务器加入集群，需要记录以免遗忘
kubeadm join 192.168.0.108:6443 --token y9e8yj.a2olglq2lurpmow7 \
--discovery-token-ca-cert-hash sha256:43e3035c08345a303afd696ed8a87d3b8a2ebd2c875d2fb30d2eeea47594447c
根据kubeadm信息提示设置普通用户权限
.kube/config文件可以放在任意安装有kubectl的机器上，通过该文件可以使用kubectl远程控制集群
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
应用flannel网络
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml
若报错The connection to the server raw.githubusercontent.com was refused - did you specify the right host or port?则是网络导致的，改用手动添加解决
kubectl apply -f - <<EOF
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: flannel
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- apiGroups:
- ""
resources:
- nodes
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes/status
verbs:
- patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: flannel
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: flannel
subjects:
- kind: ServiceAccount
name: flannel
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: flannel
namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
name: kube-flannel-cfg
namespace: kube-system
labels:
tier: node
app: flannel
data:
cni-conf.json: |
{
"name": "cbr0",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
net-conf.json: |
{
"Network": "10.244.0.0/16",
"Backend": {
"Type": "vxlan"
}
}
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: kube-flannel-ds-amd64
namespace: kube-system
labels:
tier: node
app: flannel
spec:
template:
metadata:
labels:
tier: node
app: flannel
spec:
hostNetwork: true
nodeSelector:
beta.kubernetes.io/arch: amd64
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay.io/coreos/flannel:v0.11.0-amd64
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay.io/coreos/flannel:v0.11.0-amd64
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: true
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: kube-flannel-ds-arm64
namespace: kube-system
labels:
tier: node
app: flannel
spec:
template:
metadata:
labels:
tier: node
app: flannel
spec:
hostNetwork: true
nodeSelector:
beta.kubernetes.io/arch: arm64
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay.io/coreos/flannel:v0.11.0-arm64
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay.io/coreos/flannel:v0.11.0-arm64
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: true
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: kube-flannel-ds-arm
namespace: kube-system
labels:
tier: node
app: flannel
spec:
template:
metadata:
labels:
tier: node
app: flannel
spec:
hostNetwork: true
nodeSelector:
beta.kubernetes.io/arch: arm
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay.io/coreos/flannel:v0.11.0-arm
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay.io/coreos/flannel:v0.11.0-arm
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: true
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: kube-flannel-ds-ppc64le
namespace: kube-system
labels:
tier: node
app: flannel
spec:
template:
metadata:
labels:
tier: node
app: flannel
spec:
hostNetwork: true
nodeSelector:
beta.kubernetes.io/arch: ppc64le
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay.io/coreos/flannel:v0.11.0-ppc64le
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay.io/coreos/flannel:v0.11.0-ppc64le
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: true
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: kube-flannel-ds-s390x
namespace: kube-system
labels:
tier: node
app: flannel
spec:
template:
metadata:
labels:
tier: node
app: flannel
spec:
hostNetwork: true
nodeSelector:
beta.kubernetes.io/arch: s390x
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay.io/coreos/flannel:v0.11.0-s390x
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay.io/coreos/flannel:v0.11.0-s390x
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: true
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg
EOF
查看节点信息
kubectl get nodes
出现如下信息
NAME STATUS ROLES AGE VERSION
master Ready master 35ms v1.15.1
若出现节点依旧是NotReady则参阅文章:工作节点NotReady处理
node节点加入集群
node服务器分别更改名称为node1、node2
# node1服务器
hostnamectl set-hostname node1
# node2服务器
hostnamectl set-hostname node2
运行kubeadm join命令加入集群
# node1服务器
kubeadm join 192.168.0.108:6443 --token y9e8yj.a2olglq2lurpmow7 \
--discovery-token-ca-cert-hash sha256:43e3035c08345a303afd696ed8a87d3b8a2ebd2c875d2fb30d2eeea47594447c
# node2服务器
kubeadm join 192.168.0.108:6443 --token y9e8yj.a2olglq2lurpmow7 \
--discovery-token-ca-cert-hash sha256:43e3035c08345a303afd696ed8a87d3b8a2ebd2c875d2fb30d2eeea47594447c
若出现如下报错
error execution phase preflight: unable to fetch the kubeadm-config ConfigMap: failed to get config map: Unauthorized
说明距离master生成的join命令已经超过24小时，token已经过期需要重新生成。
运行以下命令生成新的token
kubeadm token create
将生成的token值替换join命令中的token值再次运行join命令
然后回到master服务器，查看节点信息
kubectl get nodes
出现如下信息
NAME STATUS ROLES AGE VERSION
master Ready master 5m v1.15.1
node1 Ready <none> 42ms v1.15.1
node2 Ready <none> 22ms v1.15.1
至此，k8s部署完毕
附加
由于构建集群的服务器时间可能存在一些误差，在一些验证token的服务里可能会出现因误差所导致的异常，建议构建集群的同时将集群内部所有服务器的时间同步。
服务器时间同步可以查看另一篇文章《centos服务器同步时间》
若需要针对gpu节点机进行配置，参考文章docker19.03以上版本k8s配置GPU支持
标签：kube,--,v1.15,mirrorgooglecontainers,centos7,kubeadm,k8s,docker 来源： https://blog.csdn.net/a914541185/article/details/111474785