Thanks for the quick response. 

I would like to have this in the MCAPI driver so we have more control on variables by labels. Because each deployment is different. 

I have created issue here: https://github.com/vexxhost/magnum-cluster-api/issues/366 

On Mon, Apr 29, 2024 at 5:26 PM Mohammed Naser <mnaser@vexxhost.com> wrote:

Hi Satish,


Replies inline, as I’m trying to “decompose” this. 😊

 

From: Satish Patel <satish.txt@gmail.com>
Date: Monday, April 29, 2024 at 5:13 PM
To: OpenStack Discuss <openstack-discuss@lists.openstack.org>
Subject: [magnum][magnum-capi][cinder] Override csi plugin to override availability zone for cinder

Folks,

 

I have deployed a magnum with magnum-capi driver. Now trying to work with a cinder for PV stuff and found some strange issue related to the Availability Zone. 

 

My k8s cluster running in AZ: foo 

My Ceph/Cinder running in AZ: nova (default) 

 

I have created k8s cluster using magnum-capi selecting AZ: foo but when I am trying to create/attach PV then getting an error saying invalid AZ. Ofc because its in different AZ so I have add following snippet in Storage class to override AZ of CSI

 

allowedTopologies: - matchLabelExpressions: - key: topology.cinder.csi.openstack.org/zone values: - nova

 

So in this case, you do allow cross az attach inside OpenStack in order to allow this, is this correct?  Since the AZ of your storage != the AZ of your compute nodes.



PV creation completed successfully after applying the above change. Now when I try to deploy ngnix and pass that PV then again I get a volume affinity issue because worker nodes and  volume are in different AZ. 

 

I did the following to change CSI plugin az to nova for worker node to make it fix. 

 

kubectl label nodes kube-acrsf-default-worker-rxmg6-t7wzd topology.cinder.csi.openstack.org/zone=nova --overwrite

 

What is the proper workaround here for the above scenario and what would be the proper way to fix it? 

 

In this case, what you’re doing is telling the node that it is running in the `nova` AZ (which it is not!)

 

Does magnum have Cinder CSI AZ label to override availability zone in magnum template? 

 

https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/cinder-csi-plugin/using-cinder-csi-plugin.md#block-storage

 

I believe in this case, you’ll need to set the ignore-volume-az if I understand correctly, it seems Magnum does not have a default label for this.  If you want to file an issue to the CAPI driver we can add an option to enable this and make it ignore it.