首页 > 其他分享> > nfs storageclass 创建pvc pending

nfs storageclass 创建pvc pending

2022-08-21 21:31:58 作者：互联网

nfs storageclass 创建pvc pending

使用nfs 作为外部存储，一直启动不起来，查看 pvc 和 pods 信息如下：

1、PVC 一直处于 pending 状态【www-nfs-web-0 Pending k8s-nfs-storage 11s】；
2、 pods 显示【FailedScheduling】 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims.

PVC 没有创建成功，动态 PVC 中，是 provisioner 中来负责创建。分析主要原因是，官方在 k8s 1.20 中基于对性能和统一apiserver调用方式的初衷，移除了对 SelfLink 的支持，而 nfs-provisioner 需要 SelfLink 该项功能

报错信息

#kubectl get pvc
NAME            STATUS    VOLUME     CAPACITY   ACCESS MODES   STORAGECLASS     AGE
app-pvc         Bound     pv01       50Gi       RWX     10m
local-pvc       Bound     local-pv   50Gi       RWO            local-storage     14m
www-nfs-web-0   Pending                                        k8s-nfs-storage   7m40s

#kubectl get pod -o wide
NAME                                      READY   STATUS    RESTARTS   AGE   IP             NODE     NOMINATED NODE   READINESS GATES
nfs-client-provisioner-76d45db677-bxc77   1/1     Running   0          3m26s   10.244.1.102   node1    <none>           <none>
nfs-web-0                                 0/1     Pending   0          3m26s   <none>         <none>   <none>           <none>


#kubectl describe pod nfs-web-0
Name:           nfs-web-0
Namespace:      default
Priority:       0
Node:           <none>
Labels:         app=nfs-web
                controller-revision-hash=nfs-web-64b64845df
                statefulset.kubernetes.io/pod-name=nfs-web-0
Annotations:    <none>
Status:         Pending
IP:
IPs:            <none>
Controlled By:  StatefulSet/nfs-web
Containers:
  nginx:
    Image:        ikubernetes/myapp:v1
    Port:         80/TCP
    Host Port:    0/TCP
    Environment:  <none>
    Mounts:
      /usr/share/nginx/html from www (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-dnwgx (ro)
Conditions:
  Type           Status
  PodScheduled   False
Volumes:
  www:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  www-nfs-web-0
    ReadOnly:   false
  default-token-dnwgx:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-dnwgx
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age    From               Message
  ----     ------            ----   ----               -------
  Warning  FailedScheduling  3m40s  default-scheduler  0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims.
  Warning  FailedScheduling  3m40s  default-scheduler  0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims.

nfs storageclass yaml

#cat nfs-storageclass.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nfs-client-provisioner
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: quay.io/external_storage/nfs-client-provisioner:latest
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: nfs-provisioner
            - name: NFS_SERVER
              value: 10.0.0.127
            - name: NFS_PATH
              value: /data/volumes
      volumes:
        - name: nfs-client-root
          nfs:
            server: 10.0.0.127
            path: /data/volumes
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update", "create"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch", "create"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["list", "watch", "create", "update", "patch"]
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: k8s-nfs-storage
provisioner: nfs-provisioner
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nfs-web
spec:
  serviceName: "nginx"
  replicas: 1
  selector:
    matchLabels:
      app: nfs-web
  template:
    metadata:
      labels:
        app: nfs-web
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: nginx
        image: ikubernetes/myapp:v1
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: www
      annotations:
        volume.beta.kubernetes.io/storage-class: k8s-nfs-storage
    spec:
      accessModes: [ "ReadWriteMany" ]
      resources:
        requests:
          storage: 1Gi


# kubectl apply -f nfs-storageclass.yaml
deployment.apps/nfs-client-provisioner created
serviceaccount/nfs-client-provisioner created
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner created
storageclass.storage.k8s.io/k8s-nfs-storage created
statefulset.apps/nfs-web created
#kubectl get pv
#kubectl get pvc

方法一：修改 apiserver 的配置文件

修改 apiserver 的配置文件，重新启用 SelfLink 功能。针对 K8S，可添加如下配置：

#vim /etc/kubernetes/manifests/kube-apiserver.yaml
spec:
  containers:
  - command:
    - kube-apiserver
...

    - --feature-gates=RemoveSelfLink=false   #新增此行


#apiserver 是基于静态pod部署，需要重新加载
mv /etc/kubernetes/manifests/kube-apiserver.yaml
mv kube-apiserver.yaml /etc/kubernetes/manifests/

#kubectl get pvc
NAME            STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
www-nfs-web-0   Bound    pvc-b7793ca6-43ae-42b1-a397-91acc9e11172   1Gi   RWX            k8s-nfs-storage   26m

#kubectl get pod
NAME                                      READY   STATUS    RESTARTS   AGE
nfs-client-provisioner-76d45db677-bxc77   1/1     Running   3          26m
nfs-web-0                                 1/1     Running   0          26m

方法二：修改k3s的启动文件

K3S 中没有 apiserver 的配置文件，可通过 systemd 的启动文件添加该参数

# /etc/systemd/system/k3s.service

ExecStart=/usr/local/bin/k3s \
    server \
        ...
        '--kube-apiserver-arg' \   # 新增
        'feature-gates=RemoveSelfLink=false' \  # 新增

方法三：新安装操作

$ curl -sfL https://get.k3s.io | sh -s - --kube-apiserver-arg "feature-gates=RemoveSelfLink=false"

方法四：使用不基于 SelfLink 功能的 provisioner 镜像

使用新的不基于 SelfLink 功能的 provisioner 镜像，重新创建 provisioner 容器。

gcr.io/k8s-staging-sig-storage/nfs-subdir-external-provisioner:v4.0.0

registry.cn-beijing.aliyuncs.com/pylixm/nfs-subdir-external-provisioner:v4.0.0

标签：web,name,pvc,client,nfs,storageclass,provisioner,k8s
来源： https://www.cnblogs.com/weiweirui/p/16610908.html

nfs storageclass 创建pvc pending

nfs storageclass 创建pvc pending

报错信息

nfs storageclass yaml

方法一： 修改 apiserver 的配置文件

方法二：修改k3s的启动文件

方法三： 新安装操作

方法四： 使用不基于 SelfLink 功能的 provisioner 镜像

方法一：修改 apiserver 的配置文件

方法三：新安装操作

方法四：使用不基于 SelfLink 功能的 provisioner 镜像