<div dir="ltr"><div>Hi,</div><div><br></div><div>I am trying to attach a cinder volume to my pod, but it does not work.</div><div><br></div><div>The long story, the default version of kubernetes used in Yoga is 1.23.3 fcore35. When creating a default kubernetes cluster we got :</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>    Image:         <a href="http://quay.io/k8scsi/csi-attacher:v2.0.0">quay.io/k8scsi/csi-attacher:v2.0.0</a><br>    Image:         <a href="http://quay.io/k8scsi/csi-provisioner:v1.4.0">quay.io/k8scsi/csi-provisioner:v1.4.0</a><br>    Image:         <a href="http://quay.io/k8scsi/csi-snapshotter:v1.2.2">quay.io/k8scsi/csi-snapshotter:v1.2.2</a><br>    Image:         <a href="http://quay.io/k8scsi/csi-resizer:v0.3.0">quay.io/k8scsi/csi-resizer:v0.3.0</a><br>    Image:         <a href="http://docker.io/k8scloudprovider/cinder-csi-plugin:v1.18.0">docker.io/k8scloudprovider/cinder-csi-plugin:v1.18.0</a><br>    Image:         <a href="http://quay.io/k8scsi/csi-node-driver-registrar:v1.1.0">quay.io/k8scsi/csi-node-driver-registrar:v1.1.0</a></div><div>    Image:         <a href="http://docker.io/k8scloudprovider/openstack-cloud-controller-manager:v1.18.1">docker.io/k8scloudprovider/openstack-cloud-controller-manager:v1.18.1</a></div></blockquote><div><br></div><div>Which <br></div><div>1 - Does not correspond to the documentation of Magnum, the documentation states these defaults for yoga :</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>    Image:         <a href="http://10.0.0.165:4000/csi-attacher:v3.3.0">10.0.0.165:4000/csi-attacher:v3.3.0</a><br>    Image:         <a href="http://10.0.0.165:4000/csi-provisioner:v3.0.0">10.0.0.165:4000/csi-provisioner:v3.0.0</a><br>    Image:         <a href="http://10.0.0.165:4000/csi-snapshotter:v4.2.1">10.0.0.165:4000/csi-snapshotter:v4.2.1</a><br>    Image:         <a href="http://10.0.0.165:4000/csi-resizer:v1.3.0">10.0.0.165:4000/csi-resizer:v1.3.0</a><br>    Image:         <a href="http://10.0.0.165:4000/cinder-csi-plugin:v1.26.2">10.0.0.165:4000/cinder-csi-plugin:v1.26.2</a><br>    Image:         <a href="http://10.0.0.165:4000/csi-node-driver-registrar:v2.4.0">10.0.0.165:4000/csi-node-driver-registrar:v2.4.0</a><br>    Image:         <a href="http://10.0.0.165:4000/cinder-csi-plugin:v1.26.2">10.0.0.165:4000/cinder-csi-plugin:v1.26.2</a>  (cinder-csi-plugin:v1.23.0 which does not exists anymore)<br></div></blockquote><div><br></div><div>2 - And does not work, csi-cinder-controllerplugin keeps crashing.</div><div><br></div><div>I tried to use the updates images (using a local registry), but I couldn't attach the cinder-volume, I got :</div><div><br></div><div>Volumes:<br>  html-volume:<br>    Type:       Cinder (a Persistent Disk resource in OpenStack)<br>    VolumeID:   f780cb46-ed2a-405d-b901-7201b49c3df1<br>    FSType:     ext4<br>    ReadOnly:   false<br>    SecretRef:  nil<br>  kube-api-access-slqf4:<br>    Type:                    Projected (a volume that contains injected data from multiple sources)<br>    TokenExpirationSeconds:  3607<br>    ConfigMapName:           kube-root-ca.crt<br>    ConfigMapOptional:       <nil><br>    DownwardAPI:             true<br>QoS Class:                   Burstable<br>Node-Selectors:              <none><br>Tolerations:                 <a href="http://node.kubernetes.io/not-ready:NoExecute">node.kubernetes.io/not-ready:NoExecute</a> op=Exists for 300s<br>                             <a href="http://node.kubernetes.io/unreachable:NoExecute">node.kubernetes.io/unreachable:NoExecute</a> op=Exists for 300s<br>Events:<br>  Type     Reason              Age                    From                     Message<br>  ----     ------              ----                   ----                     -------<br><b>  Warning  FailedMount         26m (x10 over 135m)    kubelet                  Unable to attach or mount volumes: unmounted volumes=[html-volume], unattached volumes=[kube-api-access-slqf4 html-volume]: timed out waiting for the condition<br>  Warning  FailedAttachVolume  3m39s (x40 over 146m)  attachdetach-controller  AttachVolume.Attach failed for volume "cinder.csi.openstack.org-f780cb46-ed2a-405d-b901-7201b49c3df1" : Attach timeout for volume f780cb46-ed2a-405d-b901-7201b49c3df1<br>  Warning  FailedMount         104s (x54 over 146m)   kubelet                  Unable to attach or mount volumes: unmounted volumes=[html-volume], unattached volumes=[html-volume kube-api-access-slqf4]: timed out waiting for the condition</b></div><div><br></div><div><br></div><div>"volume":{"capacity_bytes":5368709120,"volume_id":"7e377933-4ae6-47b7-a685-f484d35153af"}},{"status":{"published_node_ids":["c2531ccf-842e-44d1-85bd-72c811cea199"]},"volume":{"capacity_bytes":1073741824,"volume_id":"f9d5273b-e73d-4b37-8b50-1fcecb910b2a"}}]}<br>I0319 12:36:50.910443       1 connection.go:201] GRPC error: <nil><br>I0319 12:36:56.925658       1 controller.go:210] Started VA processing "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br>I0319 12:36:56.925682       1 csi_handler.go:224] CSIHandler: processing VA "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br>I0319 12:36:56.925687       1 csi_handler.go:251] Attaching "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br>I0319 12:36:56.925691       1 csi_handler.go:421] Starting attach operation for "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br>I0319 12:36:56.925705       1 csi_handler.go:740] Found NodeID 472bf42d-5ce0-4751-8fec-57bede0024d6 in CSINode k8intcalnewer-56bgom6jntbm-node-0<br>I0319 12:36:56.925828       1 csi_handler.go:312] VA finalizer added to "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br>I0319 12:36:56.925836       1 csi_handler.go:326] NodeID annotation added to "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br>I0319 12:36:56.947632       1 connection.go:193] GRPC call: /csi.v1.Controller/ControllerPublishVolume<br>I0319 12:36:56.947646       1 connection.go:194] GRPC request: {"node_id":"472bf42d-5ce0-4751-8fec-57bede0024d6","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_id":"f780cb46-ed2a-405d-b901-7201b49c3df1"}<br>I0319 12:36:58.343821       1 connection.go:200] GRPC response: {"publish_context":{"DevicePath":"/dev/vdc"}}<br>I0319 12:36:58.343834       1 connection.go:201] GRPC error: <nil><br>I0319 12:36:58.343841       1 csi_handler.go:264] Attached "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br>I0319 12:36:58.343848       1 util.go:38] Marking as attached "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br><b>I0319 12:36:58.348467       1 csi_handler.go:234] Error processing "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7": failed to mark as attached: <a href="http://volumeattachments.storage.k8s.io">volumeattachments.storage.k8s.io</a> "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7" is forbidden: User "system:serviceaccount:kube-system:csi-cinder-controller-sa" cannot patch resource "volumeattachments/status" in API group "<a href="http://storage.k8s.io">storage.k8s.io</a>" at the cluster scope</b><br>I0319 12:36:58.348503       1 controller.go:210] Started VA processing "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br>I0319 12:36:58.348509       1 csi_handler.go:224] CSIHandler: processing VA "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br>I0319 12:36:58.348513       1 csi_handler.go:251] Attaching "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br>I0319 12:36:58.348517       1 csi_handler.go:421] Starting attach operation for "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br>I0319 12:36:58.348525       1 csi_handler.go:740] Found NodeID 472bf42d-5ce0-4751-8fec-57bede0024d6 in CSINode k8intcalnewer-56bgom6jntbm-node-0<br>I0319 12:36:58.348540       1 csi_handler.go:304] VA finalizer is already set on "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br>I0319 12:36:58.348552       1 csi_handler.go:318] NodeID annotation is already set on "csi-9f81405424dc2cf210b6465f8b649ef20f85024f169b660fab235c03f64753b7"<br>I0319 12:36:58.348564       1 connection.go:193] GRPC call: /csi.v1.Controller/ControllerPublishVolume<br>I0319 12:36:58.348567       1 connection.go:194] GRPC request: {"node_id":"472bf42d-5ce0-4751-8fec-57bede0024d6","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext:</div><div><br></div><div><br></div><div>The I tried even the most updated images :</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>  <a href="http://10.0.0.165:4000/cinder-csi-plugin:v1.26.2">10.0.0.165:4000/cinder-csi-plugin:v1.26.2</a><br>  <a href="http://10.0.0.165:4000/csi-provisioner:v3.4.0">10.0.0.165:4000/csi-provisioner:v3.4.0</a><br>  <a href="http://10.0.0.165:4000/csi-resizer:v1.7.0">10.0.0.165:4000/csi-resizer:v1.7.0</a><br>  <a href="http://10.0.0.165:4000/csi-snapshotter:v6.2.1">10.0.0.165:4000/csi-snapshotter:v6.2.1</a><br>  <a href="http://10.0.0.165:4000/csi-attacher:v4.2.0">10.0.0.165:4000/csi-attacher:v4.2.0</a><br>  <a href="http://10.0.0.165:4000/csi-node-driver-registrar:v2.7.0">10.0.0.165:4000/csi-node-driver-registrar:v2.7.0</a></div></blockquote><div><br></div><div>I had the same problem. <br></div><div><br></div><div>Then I tried to use an  older version of kubernetes : 1.21.11 with the older images shown above (following this link <a href="https://www.roksblog.de/deploy-kubernetes-clusters-in-openstack-within-minutes-with-magnum/">https://www.roksblog.de/deploy-kubernetes-clusters-in-openstack-within-minutes-with-magnum/</a>), and it worked, the cinder volume was successfully mounted inside my nginx pod.</div><div><br></div><div><br></div><div><br></div><div>- What is the meaning of the error I am having?</div><div>- Is it magnum related or kubernetes related or both?<br></div><div><br></div><div><br></div><div>Regards.<br></div></div>