k8s的服务暴露插件-fraefik
作者：互联网
CoreDNS实现了服务的自动发现，那么如何对外暴露我们的服务呢？
第一种方式是nodePort型的service：但是这中方式无法使用ipvs模型，只能使用iptables模型
第二种方式是ingress：注意ingress资源只能调度7层网络资源，特指http/https
ingress是k8s API的标准资源类型之一，也是一种核心资源，它其实就是一组基于域名的URL路径，把用户的请求转发给制定的serivce资源的规则，将外部流量，转发指内部，从而实现服务的暴露
通常用来实现ingress的软件有：
Haproxy
ingress-nginx
fraefik
我们这里使用fraefik来当作我们的ingress控制器：
准备fraefik镜像：
[root@hdss7-200 ~]# docker pull traefik:v1.7.2-alpine v1.7.2-alpine: Pulling from library/traefik 4fe2ade4980c: Pull complete 8d9593d002f4: Pull complete 5d09ab10efbd: Pull complete 37b796c58adc: Pull complete Digest: sha256:cf30141936f73599e1a46355592d08c88d74bd291f05104fe11a8bcce447c044 Status: Downloaded newer image for traefik:v1.7.2-alpine docker.io/library/traefik:v1.7.2-alpine [root@hdss7-200 ~]# [root@hdss7-200 ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE goharbor/chartmuseum-photon v0.9.0-v1.8.3 ec654bcf3624 6 months ago 131MB goharbor/harbor-migrator v1.8.3 6f945bb96ea3 6 months ago 362MB goharbor/redis-photon v1.8.3 cda8fa1932ec 6 months ago 109MB goharbor/clair-photon v2.0.8-v1.8.3 5630fa937f6d 6 months ago 165MB goharbor/notary-server-photon v0.6.1-v1.8.3 e0a54affd0c8 6 months ago 136MB goharbor/notary-signer-photon v0.6.1-v1.8.3 72708cdfb905 6 months ago 133MB goharbor/harbor-registryctl v1.8.3 9dc783842a19 6 months ago 97.2MB goharbor/registry-photon v2.7.1-patch-2819-v1.8.3 a05e085842f5 6 months ago 82.3MB goharbor/nginx-photon v1.8.3 3a016e0dc7de 6 months ago 37MB goharbor/harbor-log v1.8.3 b92621c47043 6 months ago 82.6MB goharbor/harbor-jobservice v1.8.3 53bc2359083f 6 months ago 120MB goharbor/harbor-core v1.8.3 a3ccc3897bc0 6 months ago 136MB goharbor/harbor-portal v1.8.3 514f2fb70e90 6 months ago 43.9MB goharbor/harbor-db v1.8.3 d1b8adbed58f 6 months ago 147MB goharbor/prepare v1.8.3 a37e777b7fe7 6 months ago 147MB coredns/coredns 1.6.1 c0f6e815079e 7 months ago 42.2MB harbor.od.com/public/coredns v1.6.1 c0f6e815079e 7 months ago 42.2MB traefik v1.7.2-alpine add5fac61ae5 18 months ago 72.4MB nginx 1.7.9 84581e99d807 5 years ago 91.7MB harbor.od.com/public/nginx v1.7.9 84581e99d807 5 years ago 91.7MB kubernetes/pause latest f9d5de079539 5 years ago 240kB harbor.od.com/public/pause latest f9d5de079539 5 years ago 240kB [root@hdss7-200 ~]# docker tag add5fac61ae5 harbor.od.com/public/traefik:v1.7.2 [root@hdss7-200 ~]# docker push harbor.od.com/public/traefik:v1.7.2 The push refers to repository [harbor.od.com/public/traefik] a02beb48577f: Pushed ca22117205f4: Pushed 3563c211d861: Pushed df64d3292fd6: Pushed v1.7.2: digest: sha256:6115155b261707b642341b065cd3fac2b546559ba035d0262650b3b3bbdd10ea size: 1157
准备资源配置清单：
# cat rbac.yaml apiVersion: v1 kind: ServiceAccount metadata: name: traefik-ingress-controller namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: traefik-ingress-controller rules: - apiGroups: - "" resources: - services - endpoints - secrets verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses verbs: - get - list - watch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: traefik-ingress-controller subjects: - kind: ServiceAccount name: traefik-ingress-controller namespace: kube-system # cat ds.yaml apiVersion: extensions/v1beta1 kind: DaemonSet metadata: name: traefik-ingress namespace: kube-system labels: k8s-app: traefik-ingress spec: template: metadata: labels: k8s-app: traefik-ingress name: traefik-ingress spec: serviceAccountName: traefik-ingress-controller terminationGracePeriodSeconds: 60 containers: - image: harbor.od.com/public/traefik:v1.7.2 name: traefik-ingress ports: - name: controller containerPort: 80 hostPort: 81 - name: admin-web containerPort: 8080 securityContext: capabilities: drop: - ALL add: - NET_BIND_SERVICE args: - --api - --kubernetes - --logLevel=INFO - --insecureskipverify=true - --kubernetes.endpoint=https://10.4.7.10:7443 - --accesslog - --accesslog.filepath=/var/log/traefik_access.log - --traefiklog - --traefiklog.filepath=/var/log/traefik.log - --metrics.prometheus # cat ingress.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: traefik-web-ui namespace: kube-system annotations: kubernetes.io/ingress.class: traefik spec: rules: - host: traefik.od.com http: paths: - path: / backend: serviceName: traefik-ingress-service servicePort: 8080 # cat svc.yaml kind: Service apiVersion: v1 metadata: name: traefik-ingress-service namespace: kube-system spec: selector: k8s-app: traefik-ingress ports: - protocol: TCP port: 80 name: controller - protocol: TCP port: 8080 name: admin-web
使用陈述式资源管理方法来应用我们的声明式资源配置清单：
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/traefik/rbac.yaml serviceaccount/traefik-ingress-controller created clusterrole.rbac.authorization.k8s.io/traefik-ingress-controller created clusterrolebinding.rbac.authorization.k8s.io/traefik-ingress-controller created [root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/traefik/ds.yaml daemonset.extensions/traefik-ingress created [root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/traefik/svc.yaml service/traefik-ingress-service created [root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/traefik/ingress.yaml ingress.extensions/traefik-web-ui created
检查pod状态是否已经起来了：
~]# kubectl get pod -n kube-system NAME READY STATUS RESTARTS AGE coredns-6b6c4f9648-j7cv9 1/1 Running 0 82m traefik-ingress-4pdm5 0/1 ContainerCreating 0 4s traefik-ingress-rgcqp 0/1 ContainerCreating 0 29s # kubectl describe pod -n kube-system traefik-ingress-4pdm5 Warning FailedCreatePodSandBox 7s kubelet, hdss7-22.host.com Failed create pod sandbox: rpc error: code = Unknown desc = failed to start sandbox container for pod "traefik-ingress-4pdm5": Error response from daemon: driver failed programming external connectivity on endpoint k8s_POD_traefik-ingress-4pdm5_kube-system_8d6fb147-074c-46b3-b5a0-7cff176671ec_8 (a840cdb6e9da00aefc7ce6d233a373acf4ecef3ee06890fb647208069ed59f25): (iptables failed: iptables --wait -t filter -A DOCKER ! -i docker0 -o docker0 -p tcp -d 172.7.22.3 --dport 80 -j ACCEPT: iptables: No chain/target/match by that name.
重启docker进程后发现可以了
[root@hdss7-21 ~]# systemctl restart docker [root@hdss7-22 ~]# systemctl restart docker [root@hdss7-21 ~]# kubectl get pod -n kube-system -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES coredns-6b6c4f9648-j7cv9 1/1 Running 0 85m 172.7.21.4 hdss7-21.host.com <none> <none> traefik-ingress-4pdm5 1/1 Running 0 2m59s 172.7.22.3 hdss7-22.host.com <none> <none> traefik-ingress-rgcqp 1/1 Running 0 3m24s 172.7.21.5 hdss7-21.host.com <none> <none>
配置fraefik域名解析：
[root@hdss7-11 named]# cat od.com.zone $ORIGIN od.com. $TTL 600; 10 minutes @ IN SOAdns.od.com. dnsadmin.od.com. ( 2019111004 ; serial 10800 ; refresh (3 hours) 900 ; retry (15 minutes) 604800 ; expire (1 week) 86400 ; minimum (1 day) ) NS dns.od.com. $TTL 60; 1 minute dns A 10.4.7.11 harbor A 10.4.7.200 k8s-yaml A 10.4.7.200 fraefik A 10.4.7.11 [root@hdss7-11 named]# systemctl restart named [root@hdss7-11 named]# dig @10.4.7.11 fraefik.od.com +short 10.4.7.11
然后我们在ingress的入口主机上，添加如下nginx的配置，说明：我们将业务域进行一个泛匹配，然后将所有规则抛给ingress的节点上的81端口，这样，nginx的配置如果ingress没有机器上下线的操作，等于说我们根本不用在操作nginx，只需要在资源配置清单中添加我们的规则即可，将业务的路由规则完全交给资源配置清单
[root@hdss7-200 conf.d]# cat od.com.conf
upstream default_backend_traefik {
server 10.4.7.21:81 max_fails=3 fail_timeout=10s;
server 10.4.7.22:81 max_fails=3 fail_timeout=10s;
}
server {
server_name *.od.com;
location / {
proxy_pass http://default_backend_traefik;
proxy_set_header Host $http_host;
proxy_set_header x-forwarded-for $proxy_add_x_forwarded_for;
}
}
[root@hdss7-200 conf.d]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@hdss7-200 conf.d]# nginx -s reload标签：hdss7,ago,插件,traefik,od,ingress,fraefik,k8s,com 来源： https://blog.51cto.com/13520772/2482614