StorageClass 简单学习
作者：互联网
StorageClass 简单学习
学习资料来源
https://www.jianshu.com/p/5e565a8049fc
https://zhuanlan.zhihu.com/p/289501984
https://blog.csdn.net/ag1942/article/details/115371793
https://stackoverflow.com/questions/65376314/kubernetes-nfs-provider-selflink-was-empty
等等..
第一部分: StorageClass 资源获取
- 搭建nfs, 以及提供服务, 我这边采用的是 * 指定ip地址, 避免k8s内部无法连接.
- k8s集群内每个节点都需要打开 nfs
systemctl enable nfs && systemctl restart nfs
下载资料
git clone https://github.com/kubernetes-incubator/external-storage.git
- 公司网络太差, 也可以直接使用浏览器下载然后解压缩的方式来实现.
第二部分: 部署使用
2.1 部署deployment
注意下载好的文件内有多种外部存储, 为了简单起见使用nfs-client的方式进行
- 进入到nfs-client 文件夹中进行相关工作
- 注意我自己手动下载了一些文件.最开始并没有这么多文件夹的.
/k8ssc/external-storage-master/nfs-client/deploy
[root@k8s-master01 deploy]# ll
总用量 36
-rw-r--r-- 1 root root 226 10月 21 2020 class.yaml
-rw-r--r-- 1 root root 1030 10月 21 2020 deployment-arm.yaml
-rw-r--r-- 1 root root 1008 5月 4 09:58 deployment.yaml
-rw-r--r-- 1 root root 278 5月 4 10:09 nfs-clusterrolebinding.yaml
-rw-r--r-- 1 root root 836 5月 4 10:08 nfs-clusterrole.yaml
drwxr-xr-x 2 root root 214 10月 21 2020 objects
-rw-r--r-- 1 root root 1656 5月 4 09:44 rbac2.yaml
-rw-r--r-- 1 root root 1819 10月 21 2020 rbac.yaml
-rw-r--r-- 1 root root 241 5月 4 09:24 test-claim.yaml
-rw-r--r-- 1 root root 424 10月 21 2020 test-pod.yaml
- 需要注意的是 deployment 需要修改之后再进行部署
- 修改内容为将服务修改成自己本地的nfs-server以及目录
[root@k8s-master01 deploy]# exportfs
/nfs <world>
# 注意 这个目录是我本地的环境信息.
[root@k8s-master01 deploy]# cat deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nfs-client-provisioner
labels:
app: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: nfs-client-provisioner
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: quay.io/external_storage/nfs-client-provisioner:latest
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: fuseim.pri/ifs
- name: NFS_SERVER
value: 10.110.83.201 # 需要修改为自己的nfs-server
- name: NFS_PATH
value: /nfs #需要修改成自己的服务目录
volumes:
- name: nfs-client-root
nfs:
server: 10.110.83.201 # 同样处理.
path: /nfs #同样处理.
- 修改完deployment.yaml 之后就可以执行部署了.
kubectl apply -f deployment.yaml - 部署完成之后可以进行查看
[root@k8s-master01 deploy]# kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-7b9cf5df76-78n5d 1/1 Running 0 12h
nfs-client-provisioner-7f547686c4-59294 1/1 Running 0 80m
redis-master-0 1/1 Running 0 69m
redis-slave-0 0/1 CrashLoopBackOff 18 69m
[root@k8s-master01 deploy]# kubectl get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
mysql 1/1 1 1 12h
nfs-client-provisioner 1/1 1 1 80m
- 注意: 为了简单可以先在一个节点将这个images 拉取到 然后分发到各个node 节点中.
- 注意: 有问题需要
kubectl describe pods podid或者是kubectl logs -f podname查看相关错误信息.
2.2 部署 StorageClass
- 不需要修改相关文件, 直接部署即可.
cat class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: managed-nfs-storage
provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
archiveOnDelete: "false"
- 直接部署即可
kubectl apply -f class.yaml - 部署完之后可以进行查看:
[root@k8s-master01 deploy]# kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE
managed-nfs-storage fuseim.pri/ifs Delete Immediate false 96m
- 著有需要记住这个name 后面会有用.
2.3 授权
- 注意: 这里面我遇到了一个很大的坑.
按照原始文档创建之后一直报错
leaderelection.go:234] error retrieving resource lock default/fuseim.pri-ifs: Unauthorized
后来经过多发查找发现问题原因 如果创建到default 的表空间, 可能对backends的获取有问题,除了需要执行的kubectl apply -f rbac.yaml之后还需要执行另外的权限设置.
注意 default 的命令空间 设置.
nfs-clusterrole.yaml
新增内容:
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nfs-provisioner-runner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["services"]
verbs: ["get"]
- apiGroups: ["extensions"]
resources: ["podsecuritypolicies"]
resourceNames: ["nfs-provisioner"]
verbs: ["use"]
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: run-nfs-provisioner
subjects:
- kind: ServiceAccount
name: default
namespace: default
roleRef:
kind: ClusterRole
name: nfs-provisioner-runner
apiGroup: rbac.authorization.k8s.io
2.4 简要测试验证.
-
创建pvc
kubectl apply -f test-claim.yaml -
注意文件里面的 class 名字.
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: test-claim
annotations:
volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Mi
- 可以查看相关信息:
[root@k8s-master01 deploy]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
mysql Bound pv-volume-mysql 8Gi RWO 12h
pvc-test Bound pv-test 1Gi RWO 12h
redis-data-redis-master-0 Bound pvc-34e2b8de-13af-4f60-9b45-3259c45e2338 8Gi RWO managed-nfs-storage 83m
redis-data-redis-slave-0 Bound pvc-b651443d-ecff-46a4-9507-977180cde3be 8Gi RWO managed-nfs-storage 83m
test-claim Bound pvc-cd79f7a1-2eab-443a-b183-e7730e38840a 1Mi RWX managed-nfs-storage 107m
[root@k8s-master01 deploy]# cd /nfs
[root@k8s-master01 nfs]# ll
总用量 0
-rw-r--r-- 1 root root 0 5月 4 09:49 1
drwxrwxrwx 2 root root 28 5月 4 10:34 default-redis-data-redis-master-0-pvc-34e2b8de-13af-4f60-9b45-3259c45e2338
drwxrwxrwx 2 root root 28 5月 4 10:34 default-redis-data-redis-slave-0-pvc-b651443d-ecff-46a4-9507-977180cde3be
drwxrwxrwx 2 root root 6 5月 4 10:11 default-test-claim-pvc-cd79f7a1-2eab-443a-b183-e7730e38840a
第三部分: 其他问题.
- 除了default 命令空间之后 还遇到一个 一直提示 avaible 但是不能 bound的错误 经过一番查看, 找到一个合理的解释为:
In v1.16, we will deprecate the SelfLink field in both ObjectMeta and ListMeta objects by: documenting in field definition that it is deprecated and is going to be removed adding a release-note about field deprecation We will also introduce a feature gate to allow disabling setting SelfLink fields and opaque the logic setting it behind this feature gate.
In v1.20 (12 months and 4 release from v1.16) we will switch off the feature gate which will automatically disable setting SelfLinks. However it will still be possible to revert the behavior by changing value of a feature gate.
In v1.21, we will get rid of the whole code propagating those fields and fields themselves. In the meantime, we will go over places referencing that field (see below) and get rid of those too.
- 解决方法为在kube-apiserver.yaml 里面添加内容为:
在文件: /etc/kubernetes/manifests/kube-apiserver.yaml
添加内容:
- --feature-gates=RemoveSelfLink=false
# 注意多主节点, 每个都需要修改.
# 注意kubeadm搭建的 修改了 次文件 apiserver 会自动重启.
修改之后为:
apiVersion: v1
kind: Pod
metadata:
annotations:
kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 10.110.83.201:6443
creationTimestamp: null
labels:
component: kube-apiserver
tier: control-plane
name: kube-apiserver
namespace: kube-system
spec:
containers:
- command:
- kube-apiserver
- --feature-gates=RemoveSelfLink=false
- 但是怀疑跟这个没关系, 因为 我用的1.21 说已经完全废除了.
第四部分: 简单使用
- 设置storageclass 为默认 default 的存储类
kubectl patch storageclass managed-nfs-storage -p '{"metadata": {"annotations":{"storageclass.beta.kubernetes.io/is-default-class":"true"}}}'
执行 mysql的安装
helm install mysqltestsc stable/mysql
然后可以执行
kubectl exec -it mysqltestscxxxx /bin/bash 的方式来连接查看
查看密码的命令为:
kubectl get secret --namespace default mysqltestsc -o jsonpath="{.data.mysql-root-password}" | base64 --decode; echo
标签：name,--,学习,yaml,nfs,StorageClass,简单,k8s,root 来源： https://www.cnblogs.com/jinanxiaolaohu/p/14729232.html