Add netapp storage in edge site | Wallaby | DCN
Hi, I have a DCN setup and there is a requirement to use a netapp storage device in one of the edge sites. Can someone please confirm if it is possible? And if so then should i add the parameters in the edge deployment script or the central deployment script. Any suggestions? With regards, Swogat Pradhan
On Wed, May 24, 2023 at 3:15 AM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi, I have a DCN setup and there is a requirement to use a netapp storage device in one of the edge sites. Can someone please confirm if it is possible?
I see from prior email to this list that you're using tripleo, so I'll respond with that in mind. There are many factors that come into play, but I suspect the short answer to your question is no. Tripleo's DCN architecture requires the cinder-volume service running at edge sites to run in active-active mode, where there are separate instances running on three nodes in to for the service to be highly available (HA).The problem is that only a small number of cinder drivers support running A/A, and NetApp's drivers do not support A/A. It's conceivable you could create a custom tripleo role that deploys just a single node running cinder-volume with a NetApp backend, but it wouldn't be HA. It's also conceivable you could locate the NetApp system in the central site's controlplane, but there are extremely difficult constraints you'd need to overcome: - Network latency between the central and edge sites would mean the disk performance would be bad. - You'd be limited to using iSCSI (FC wouldn't work) - Tripleo disables cross-AZ attachments, so the only way for an edge site to access a NetApp volume would be to configure the cinder-volume service running in the controlplane with a backend availability zone set to the edge site's AZ. You mentioned the NetApp is needed "in one of the edge sites," but in reality the NetApp would be available in one, AND ONLY ONE edge site, and it would also not be available to any instances running in the central site. Alan
And if so then should i add the parameters in the edge deployment script or the central deployment script. Any suggestions?
With regards, Swogat Pradhan
Hi Alan, So, can I include the cinder-netapp-storage.yaml file in the central site and then use the new backend to add storage to edge VM's? I believe it is not possible right?? as the cinder volume in the edge won't have the config for the netapp. With regards, Swogat Pradhan On Thu, May 25, 2023 at 2:17 AM Alan Bishop <abishop@redhat.com> wrote:
On Wed, May 24, 2023 at 3:15 AM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi, I have a DCN setup and there is a requirement to use a netapp storage device in one of the edge sites. Can someone please confirm if it is possible?
I see from prior email to this list that you're using tripleo, so I'll respond with that in mind.
There are many factors that come into play, but I suspect the short answer to your question is no.
Tripleo's DCN architecture requires the cinder-volume service running at edge sites to run in active-active mode, where there are separate instances running on three nodes in to for the service to be highly available (HA).The problem is that only a small number of cinder drivers support running A/A, and NetApp's drivers do not support A/A.
It's conceivable you could create a custom tripleo role that deploys just a single node running cinder-volume with a NetApp backend, but it wouldn't be HA.
It's also conceivable you could locate the NetApp system in the central site's controlplane, but there are extremely difficult constraints you'd need to overcome: - Network latency between the central and edge sites would mean the disk performance would be bad. - You'd be limited to using iSCSI (FC wouldn't work) - Tripleo disables cross-AZ attachments, so the only way for an edge site to access a NetApp volume would be to configure the cinder-volume service running in the controlplane with a backend availability zone set to the edge site's AZ. You mentioned the NetApp is needed "in one of the edge sites," but in reality the NetApp would be available in one, AND ONLY ONE edge site, and it would also not be available to any instances running in the central site.
Alan
And if so then should i add the parameters in the edge deployment script or the central deployment script. Any suggestions?
With regards, Swogat Pradhan
On Thu, May 25, 2023 at 12:09 AM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi Alan, So, can I include the cinder-netapp-storage.yaml file in the central site and then use the new backend to add storage to edge VM's?
Where is the NetApp physically located? Tripleo's DCN architecture assumes the storage is physically located at the same site where the cinder-volume service will be deployed. If you include the cinder-netapp-storage.yaml environment file in the central site's controlplane, then VMs at the edge site will encounter the problems I outlined earlier (network latency, no ability to do cross-AZ attachments).
I believe it is not possible right?? as the cinder volume in the edge won't have the config for the netapp.
The cinder-volume services at an edge site are meant to manage storage devices at that site. If the NetApp is at the edge site, ideally you'd include some variation of the cinder-netapp-storage.yaml environment file in the edge site's deployment. However, then you're faced with the fact that the NetApp driver doesn't support A/A, which is required for c-vol services running at edge sites (In case you're not familiar with these details, tripleo runs all cinder-volume services in active/passive mode under pacemaker on controllers in the controlplane. Thus, only a single instance runs at any time, and pacemaker provides HA by moving the service to another controller if the first one goes down. However, pacemaker is not available at edge sites, and so to get HA, multiple instances of the cinder-volume service run simultaneously on 3 nodes (A/A), using etcd as a Distributed Lock Manager (DLM) to coordinate things. But drivers must specifically support running A/A, and the NetApp driver does NOT.) Alan
With regards, Swogat Pradhan
On Thu, May 25, 2023 at 2:17 AM Alan Bishop <abishop@redhat.com> wrote:
On Wed, May 24, 2023 at 3:15 AM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi, I have a DCN setup and there is a requirement to use a netapp storage device in one of the edge sites. Can someone please confirm if it is possible?
I see from prior email to this list that you're using tripleo, so I'll respond with that in mind.
There are many factors that come into play, but I suspect the short answer to your question is no.
Tripleo's DCN architecture requires the cinder-volume service running at edge sites to run in active-active mode, where there are separate instances running on three nodes in to for the service to be highly available (HA).The problem is that only a small number of cinder drivers support running A/A, and NetApp's drivers do not support A/A.
It's conceivable you could create a custom tripleo role that deploys just a single node running cinder-volume with a NetApp backend, but it wouldn't be HA.
It's also conceivable you could locate the NetApp system in the central site's controlplane, but there are extremely difficult constraints you'd need to overcome: - Network latency between the central and edge sites would mean the disk performance would be bad. - You'd be limited to using iSCSI (FC wouldn't work) - Tripleo disables cross-AZ attachments, so the only way for an edge site to access a NetApp volume would be to configure the cinder-volume service running in the controlplane with a backend availability zone set to the edge site's AZ. You mentioned the NetApp is needed "in one of the edge sites," but in reality the NetApp would be available in one, AND ONLY ONE edge site, and it would also not be available to any instances running in the central site.
Alan
And if so then should i add the parameters in the edge deployment script or the central deployment script. Any suggestions?
With regards, Swogat Pradhan
Hi Alan, My netapp storage is located in edge site itself. As the networks are routable my central site is able to reach the netapp storage ip address (ping response is 30ms-40ms). Let's say i included the netapp storage yaml in central site deployment script (which is not recommended) and i am able to create the volumes as it is reachable from controller nodes. Will i be able to mount those volumes in edge site VM's?? And if i am able to do so, then how will the data flow?? When storing something in the netapp volume will the data flow through the central site controller and get stored in the storage space? With regards, Swogat Pradhan On Fri, 26 May 2023, 10:03 am Alan Bishop, <abishop@redhat.com> wrote:
On Thu, May 25, 2023 at 12:09 AM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi Alan, So, can I include the cinder-netapp-storage.yaml file in the central site and then use the new backend to add storage to edge VM's?
Where is the NetApp physically located? Tripleo's DCN architecture assumes the storage is physically located at the same site where the cinder-volume service will be deployed. If you include the cinder-netapp-storage.yaml environment file in the central site's controlplane, then VMs at the edge site will encounter the problems I outlined earlier (network latency, no ability to do cross-AZ attachments).
I believe it is not possible right?? as the cinder volume in the edge won't have the config for the netapp.
The cinder-volume services at an edge site are meant to manage storage devices at that site. If the NetApp is at the edge site, ideally you'd include some variation of the cinder-netapp-storage.yaml environment file in the edge site's deployment. However, then you're faced with the fact that the NetApp driver doesn't support A/A, which is required for c-vol services running at edge sites (In case you're not familiar with these details, tripleo runs all cinder-volume services in active/passive mode under pacemaker on controllers in the controlplane. Thus, only a single instance runs at any time, and pacemaker provides HA by moving the service to another controller if the first one goes down. However, pacemaker is not available at edge sites, and so to get HA, multiple instances of the cinder-volume service run simultaneously on 3 nodes (A/A), using etcd as a Distributed Lock Manager (DLM) to coordinate things. But drivers must specifically support running A/A, and the NetApp driver does NOT.)
Alan
With regards, Swogat Pradhan
On Thu, May 25, 2023 at 2:17 AM Alan Bishop <abishop@redhat.com> wrote:
On Wed, May 24, 2023 at 3:15 AM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi, I have a DCN setup and there is a requirement to use a netapp storage device in one of the edge sites. Can someone please confirm if it is possible?
I see from prior email to this list that you're using tripleo, so I'll respond with that in mind.
There are many factors that come into play, but I suspect the short answer to your question is no.
Tripleo's DCN architecture requires the cinder-volume service running at edge sites to run in active-active mode, where there are separate instances running on three nodes in to for the service to be highly available (HA).The problem is that only a small number of cinder drivers support running A/A, and NetApp's drivers do not support A/A.
It's conceivable you could create a custom tripleo role that deploys just a single node running cinder-volume with a NetApp backend, but it wouldn't be HA.
It's also conceivable you could locate the NetApp system in the central site's controlplane, but there are extremely difficult constraints you'd need to overcome: - Network latency between the central and edge sites would mean the disk performance would be bad. - You'd be limited to using iSCSI (FC wouldn't work) - Tripleo disables cross-AZ attachments, so the only way for an edge site to access a NetApp volume would be to configure the cinder-volume service running in the controlplane with a backend availability zone set to the edge site's AZ. You mentioned the NetApp is needed "in one of the edge sites," but in reality the NetApp would be available in one, AND ONLY ONE edge site, and it would also not be available to any instances running in the central site.
Alan
And if so then should i add the parameters in the edge deployment script or the central deployment script. Any suggestions?
With regards, Swogat Pradhan
Hi Alan, If you please could help me clarify this. With regards, Swogat Pradhan On Fri, 26 May 2023, 10:09 am Swogat Pradhan, <swogatpradhan22@gmail.com> wrote:
Hi Alan, My netapp storage is located in edge site itself. As the networks are routable my central site is able to reach the netapp storage ip address (ping response is 30ms-40ms). Let's say i included the netapp storage yaml in central site deployment script (which is not recommended) and i am able to create the volumes as it is reachable from controller nodes. Will i be able to mount those volumes in edge site VM's?? And if i am able to do so, then how will the data flow?? When storing something in the netapp volume will the data flow through the central site controller and get stored in the storage space?
With regards, Swogat Pradhan
On Fri, 26 May 2023, 10:03 am Alan Bishop, <abishop@redhat.com> wrote:
On Thu, May 25, 2023 at 12:09 AM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi Alan, So, can I include the cinder-netapp-storage.yaml file in the central site and then use the new backend to add storage to edge VM's?
Where is the NetApp physically located? Tripleo's DCN architecture assumes the storage is physically located at the same site where the cinder-volume service will be deployed. If you include the cinder-netapp-storage.yaml environment file in the central site's controlplane, then VMs at the edge site will encounter the problems I outlined earlier (network latency, no ability to do cross-AZ attachments).
I believe it is not possible right?? as the cinder volume in the edge won't have the config for the netapp.
The cinder-volume services at an edge site are meant to manage storage devices at that site. If the NetApp is at the edge site, ideally you'd include some variation of the cinder-netapp-storage.yaml environment file in the edge site's deployment. However, then you're faced with the fact that the NetApp driver doesn't support A/A, which is required for c-vol services running at edge sites (In case you're not familiar with these details, tripleo runs all cinder-volume services in active/passive mode under pacemaker on controllers in the controlplane. Thus, only a single instance runs at any time, and pacemaker provides HA by moving the service to another controller if the first one goes down. However, pacemaker is not available at edge sites, and so to get HA, multiple instances of the cinder-volume service run simultaneously on 3 nodes (A/A), using etcd as a Distributed Lock Manager (DLM) to coordinate things. But drivers must specifically support running A/A, and the NetApp driver does NOT.)
Alan
With regards, Swogat Pradhan
On Thu, May 25, 2023 at 2:17 AM Alan Bishop <abishop@redhat.com> wrote:
On Wed, May 24, 2023 at 3:15 AM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi, I have a DCN setup and there is a requirement to use a netapp storage device in one of the edge sites. Can someone please confirm if it is possible?
I see from prior email to this list that you're using tripleo, so I'll respond with that in mind.
There are many factors that come into play, but I suspect the short answer to your question is no.
Tripleo's DCN architecture requires the cinder-volume service running at edge sites to run in active-active mode, where there are separate instances running on three nodes in to for the service to be highly available (HA).The problem is that only a small number of cinder drivers support running A/A, and NetApp's drivers do not support A/A.
It's conceivable you could create a custom tripleo role that deploys just a single node running cinder-volume with a NetApp backend, but it wouldn't be HA.
It's also conceivable you could locate the NetApp system in the central site's controlplane, but there are extremely difficult constraints you'd need to overcome: - Network latency between the central and edge sites would mean the disk performance would be bad. - You'd be limited to using iSCSI (FC wouldn't work) - Tripleo disables cross-AZ attachments, so the only way for an edge site to access a NetApp volume would be to configure the cinder-volume service running in the controlplane with a backend availability zone set to the edge site's AZ. You mentioned the NetApp is needed "in one of the edge sites," but in reality the NetApp would be available in one, AND ONLY ONE edge site, and it would also not be available to any instances running in the central site.
Alan
And if so then should i add the parameters in the edge deployment script or the central deployment script. Any suggestions?
With regards, Swogat Pradhan
On Thu, May 25, 2023 at 9:39 PM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi Alan, My netapp storage is located in edge site itself. As the networks are routable my central site is able to reach the netapp storage ip address (ping response is 30ms-40ms). Let's say i included the netapp storage yaml in central site deployment script (which is not recommended) and i am able to create the volumes as it is reachable from controller nodes. Will i be able to mount those volumes in edge site VM's?? And if i am able to do so, then how will the data flow?? When storing something in the netapp volume will the data flow through the central site controller and get stored in the storage space?
A cinder-volume service running in the central site's controplane will be able to work with a netapp backend that's physically located at an edge site. The good news is the c-vol service will be HA because it will be controlled by pacemaker running on the controllers. In order for VMs at the edge site to access volumes on the netapp, you'll need to set the CinderNetappAvailabilityZone [1] to the edge site's AZ. [1] https://opendev.org/openstack/tripleo-heat-templates/src/branch/stable/walla... To attach a netapp volume, nova-compute at the edge will interact with cinder-volume in the controlplane, and cinder-volume will in turn interact with the netapp. This will happen over central <=> edge network connections. Eventually, nova will directly connect to the netapp, so all traffic from the VM to the netapp will occur within the edge site. Data will not flow through the cinder-volume service, but there are restrictions and limitations: - Only that one edge site can access the netapp backend - If the central <=> edge network connection then you won't be able to attach or detach a netapp volume (but active connections will continue to work) Of course, there are operations where cinder services are in the data path (e.g. creating a volume from an image), but not when a VM is accessing a volume. Alan
With regards, Swogat Pradhan
On Fri, 26 May 2023, 10:03 am Alan Bishop, <abishop@redhat.com> wrote:
On Thu, May 25, 2023 at 12:09 AM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi Alan, So, can I include the cinder-netapp-storage.yaml file in the central site and then use the new backend to add storage to edge VM's?
Where is the NetApp physically located? Tripleo's DCN architecture assumes the storage is physically located at the same site where the cinder-volume service will be deployed. If you include the cinder-netapp-storage.yaml environment file in the central site's controlplane, then VMs at the edge site will encounter the problems I outlined earlier (network latency, no ability to do cross-AZ attachments).
I believe it is not possible right?? as the cinder volume in the edge won't have the config for the netapp.
The cinder-volume services at an edge site are meant to manage storage devices at that site. If the NetApp is at the edge site, ideally you'd include some variation of the cinder-netapp-storage.yaml environment file in the edge site's deployment. However, then you're faced with the fact that the NetApp driver doesn't support A/A, which is required for c-vol services running at edge sites (In case you're not familiar with these details, tripleo runs all cinder-volume services in active/passive mode under pacemaker on controllers in the controlplane. Thus, only a single instance runs at any time, and pacemaker provides HA by moving the service to another controller if the first one goes down. However, pacemaker is not available at edge sites, and so to get HA, multiple instances of the cinder-volume service run simultaneously on 3 nodes (A/A), using etcd as a Distributed Lock Manager (DLM) to coordinate things. But drivers must specifically support running A/A, and the NetApp driver does NOT.)
Alan
With regards, Swogat Pradhan
On Thu, May 25, 2023 at 2:17 AM Alan Bishop <abishop@redhat.com> wrote:
On Wed, May 24, 2023 at 3:15 AM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi, I have a DCN setup and there is a requirement to use a netapp storage device in one of the edge sites. Can someone please confirm if it is possible?
I see from prior email to this list that you're using tripleo, so I'll respond with that in mind.
There are many factors that come into play, but I suspect the short answer to your question is no.
Tripleo's DCN architecture requires the cinder-volume service running at edge sites to run in active-active mode, where there are separate instances running on three nodes in to for the service to be highly available (HA).The problem is that only a small number of cinder drivers support running A/A, and NetApp's drivers do not support A/A.
It's conceivable you could create a custom tripleo role that deploys just a single node running cinder-volume with a NetApp backend, but it wouldn't be HA.
It's also conceivable you could locate the NetApp system in the central site's controlplane, but there are extremely difficult constraints you'd need to overcome: - Network latency between the central and edge sites would mean the disk performance would be bad. - You'd be limited to using iSCSI (FC wouldn't work) - Tripleo disables cross-AZ attachments, so the only way for an edge site to access a NetApp volume would be to configure the cinder-volume service running in the controlplane with a backend availability zone set to the edge site's AZ. You mentioned the NetApp is needed "in one of the edge sites," but in reality the NetApp would be available in one, AND ONLY ONE edge site, and it would also not be available to any instances running in the central site.
Alan
And if so then should i add the parameters in the edge deployment script or the central deployment script. Any suggestions?
With regards, Swogat Pradhan
Hi Alan, Thanks for your clarification, the way you suggested will solve my issue. But i already have a netapp backend in my central site and to add another backend should i follow this documentation: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/17.... And should i remove the old netapp environment file and use the new custom environment file created using the above mentioned guide?? I already have a prod workload in the currently deployed netapp and I do not want to cause any issues in that netapp storage. With regards, Swogat Pradhan On Tue, May 30, 2023 at 8:43 PM Alan Bishop <abishop@redhat.com> wrote:
On Thu, May 25, 2023 at 9:39 PM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi Alan, My netapp storage is located in edge site itself. As the networks are routable my central site is able to reach the netapp storage ip address (ping response is 30ms-40ms). Let's say i included the netapp storage yaml in central site deployment script (which is not recommended) and i am able to create the volumes as it is reachable from controller nodes. Will i be able to mount those volumes in edge site VM's?? And if i am able to do so, then how will the data flow?? When storing something in the netapp volume will the data flow through the central site controller and get stored in the storage space?
A cinder-volume service running in the central site's controplane will be able to work with a netapp backend that's physically located at an edge site. The good news is the c-vol service will be HA because it will be controlled by pacemaker running on the controllers.
In order for VMs at the edge site to access volumes on the netapp, you'll need to set the CinderNetappAvailabilityZone [1] to the edge site's AZ.
[1] https://opendev.org/openstack/tripleo-heat-templates/src/branch/stable/walla...
To attach a netapp volume, nova-compute at the edge will interact with cinder-volume in the controlplane, and cinder-volume will in turn interact with the netapp. This will happen over central <=> edge network connections. Eventually, nova will directly connect to the netapp, so all traffic from the VM to the netapp will occur within the edge site. Data will not flow through the cinder-volume service, but there are restrictions and limitations: - Only that one edge site can access the netapp backend - If the central <=> edge network connection then you won't be able to attach or detach a netapp volume (but active connections will continue to work)
Of course, there are operations where cinder services are in the data path (e.g. creating a volume from an image), but not when a VM is accessing a volume.
Alan
With regards, Swogat Pradhan
On Fri, 26 May 2023, 10:03 am Alan Bishop, <abishop@redhat.com> wrote:
On Thu, May 25, 2023 at 12:09 AM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi Alan, So, can I include the cinder-netapp-storage.yaml file in the central site and then use the new backend to add storage to edge VM's?
Where is the NetApp physically located? Tripleo's DCN architecture assumes the storage is physically located at the same site where the cinder-volume service will be deployed. If you include the cinder-netapp-storage.yaml environment file in the central site's controlplane, then VMs at the edge site will encounter the problems I outlined earlier (network latency, no ability to do cross-AZ attachments).
I believe it is not possible right?? as the cinder volume in the edge won't have the config for the netapp.
The cinder-volume services at an edge site are meant to manage storage devices at that site. If the NetApp is at the edge site, ideally you'd include some variation of the cinder-netapp-storage.yaml environment file in the edge site's deployment. However, then you're faced with the fact that the NetApp driver doesn't support A/A, which is required for c-vol services running at edge sites (In case you're not familiar with these details, tripleo runs all cinder-volume services in active/passive mode under pacemaker on controllers in the controlplane. Thus, only a single instance runs at any time, and pacemaker provides HA by moving the service to another controller if the first one goes down. However, pacemaker is not available at edge sites, and so to get HA, multiple instances of the cinder-volume service run simultaneously on 3 nodes (A/A), using etcd as a Distributed Lock Manager (DLM) to coordinate things. But drivers must specifically support running A/A, and the NetApp driver does NOT.)
Alan
With regards, Swogat Pradhan
On Thu, May 25, 2023 at 2:17 AM Alan Bishop <abishop@redhat.com> wrote:
On Wed, May 24, 2023 at 3:15 AM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi, I have a DCN setup and there is a requirement to use a netapp storage device in one of the edge sites. Can someone please confirm if it is possible?
I see from prior email to this list that you're using tripleo, so I'll respond with that in mind.
There are many factors that come into play, but I suspect the short answer to your question is no.
Tripleo's DCN architecture requires the cinder-volume service running at edge sites to run in active-active mode, where there are separate instances running on three nodes in to for the service to be highly available (HA).The problem is that only a small number of cinder drivers support running A/A, and NetApp's drivers do not support A/A.
It's conceivable you could create a custom tripleo role that deploys just a single node running cinder-volume with a NetApp backend, but it wouldn't be HA.
It's also conceivable you could locate the NetApp system in the central site's controlplane, but there are extremely difficult constraints you'd need to overcome: - Network latency between the central and edge sites would mean the disk performance would be bad. - You'd be limited to using iSCSI (FC wouldn't work) - Tripleo disables cross-AZ attachments, so the only way for an edge site to access a NetApp volume would be to configure the cinder-volume service running in the controlplane with a backend availability zone set to the edge site's AZ. You mentioned the NetApp is needed "in one of the edge sites," but in reality the NetApp would be available in one, AND ONLY ONE edge site, and it would also not be available to any instances running in the central site.
Alan
And if so then should i add the parameters in the edge deployment script or the central deployment script. Any suggestions?
With regards, Swogat Pradhan
Hi Alan, Can you please check if the environment file attached is in correct format and how it should be? Also should i remove the old netapp environment file and use the new custom environment file created (attached). With regards, Swogat Pradhan On Wed, May 31, 2023 at 10:46 PM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi Alan, Thanks for your clarification, the way you suggested will solve my issue. But i already have a netapp backend in my central site and to add another backend should i follow this documentation: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/17....
And should i remove the old netapp environment file and use the new custom environment file created using the above mentioned guide??
I already have a prod workload in the currently deployed netapp and I do not want to cause any issues in that netapp storage.
With regards, Swogat Pradhan
On Tue, May 30, 2023 at 8:43 PM Alan Bishop <abishop@redhat.com> wrote:
On Thu, May 25, 2023 at 9:39 PM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi Alan, My netapp storage is located in edge site itself. As the networks are routable my central site is able to reach the netapp storage ip address (ping response is 30ms-40ms). Let's say i included the netapp storage yaml in central site deployment script (which is not recommended) and i am able to create the volumes as it is reachable from controller nodes. Will i be able to mount those volumes in edge site VM's?? And if i am able to do so, then how will the data flow?? When storing something in the netapp volume will the data flow through the central site controller and get stored in the storage space?
A cinder-volume service running in the central site's controplane will be able to work with a netapp backend that's physically located at an edge site. The good news is the c-vol service will be HA because it will be controlled by pacemaker running on the controllers.
In order for VMs at the edge site to access volumes on the netapp, you'll need to set the CinderNetappAvailabilityZone [1] to the edge site's AZ.
[1] https://opendev.org/openstack/tripleo-heat-templates/src/branch/stable/walla...
To attach a netapp volume, nova-compute at the edge will interact with cinder-volume in the controlplane, and cinder-volume will in turn interact with the netapp. This will happen over central <=> edge network connections. Eventually, nova will directly connect to the netapp, so all traffic from the VM to the netapp will occur within the edge site. Data will not flow through the cinder-volume service, but there are restrictions and limitations: - Only that one edge site can access the netapp backend - If the central <=> edge network connection then you won't be able to attach or detach a netapp volume (but active connections will continue to work)
Of course, there are operations where cinder services are in the data path (e.g. creating a volume from an image), but not when a VM is accessing a volume.
Alan
With regards, Swogat Pradhan
On Fri, 26 May 2023, 10:03 am Alan Bishop, <abishop@redhat.com> wrote:
On Thu, May 25, 2023 at 12:09 AM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi Alan, So, can I include the cinder-netapp-storage.yaml file in the central site and then use the new backend to add storage to edge VM's?
Where is the NetApp physically located? Tripleo's DCN architecture assumes the storage is physically located at the same site where the cinder-volume service will be deployed. If you include the cinder-netapp-storage.yaml environment file in the central site's controlplane, then VMs at the edge site will encounter the problems I outlined earlier (network latency, no ability to do cross-AZ attachments).
I believe it is not possible right?? as the cinder volume in the edge won't have the config for the netapp.
The cinder-volume services at an edge site are meant to manage storage devices at that site. If the NetApp is at the edge site, ideally you'd include some variation of the cinder-netapp-storage.yaml environment file in the edge site's deployment. However, then you're faced with the fact that the NetApp driver doesn't support A/A, which is required for c-vol services running at edge sites (In case you're not familiar with these details, tripleo runs all cinder-volume services in active/passive mode under pacemaker on controllers in the controlplane. Thus, only a single instance runs at any time, and pacemaker provides HA by moving the service to another controller if the first one goes down. However, pacemaker is not available at edge sites, and so to get HA, multiple instances of the cinder-volume service run simultaneously on 3 nodes (A/A), using etcd as a Distributed Lock Manager (DLM) to coordinate things. But drivers must specifically support running A/A, and the NetApp driver does NOT.)
Alan
With regards, Swogat Pradhan
On Thu, May 25, 2023 at 2:17 AM Alan Bishop <abishop@redhat.com> wrote:
On Wed, May 24, 2023 at 3:15 AM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
> Hi, > I have a DCN setup and there is a requirement to use a netapp > storage device in one of the edge sites. > Can someone please confirm if it is possible? >
I see from prior email to this list that you're using tripleo, so I'll respond with that in mind.
There are many factors that come into play, but I suspect the short answer to your question is no.
Tripleo's DCN architecture requires the cinder-volume service running at edge sites to run in active-active mode, where there are separate instances running on three nodes in to for the service to be highly available (HA).The problem is that only a small number of cinder drivers support running A/A, and NetApp's drivers do not support A/A.
It's conceivable you could create a custom tripleo role that deploys just a single node running cinder-volume with a NetApp backend, but it wouldn't be HA.
It's also conceivable you could locate the NetApp system in the central site's controlplane, but there are extremely difficult constraints you'd need to overcome: - Network latency between the central and edge sites would mean the disk performance would be bad. - You'd be limited to using iSCSI (FC wouldn't work) - Tripleo disables cross-AZ attachments, so the only way for an edge site to access a NetApp volume would be to configure the cinder-volume service running in the controlplane with a backend availability zone set to the edge site's AZ. You mentioned the NetApp is needed "in one of the edge sites," but in reality the NetApp would be available in one, AND ONLY ONE edge site, and it would also not be available to any instances running in the central site.
Alan
> And if so then should i add the parameters in the edge deployment > script or the central deployment script. > Any suggestions? > > With regards, > Swogat Pradhan >
On Wed, May 31, 2023 at 10:44 AM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi Alan, Can you please check if the environment file attached is in correct format and how it should be? Also should i remove the old netapp environment file and use the new custom environment file created (attached).
You could retain the original environment file, and merge in just a portion of the sample file that you attached. The original file contains all the settings, including the resource_registry entry, to deploy the first netapp in the controlplane. You want to enhance this with just a couple of items in order to deploy the second netapp at the edge site. The important thing to know is the CinderNetappMultiConfig entries are necessary only for the second netapp. The 'tripleo_netapp' entry could be removed, because it's basically taken care of by the original env file. You only need to include the 'tripleo_netapp_dcn02' settings. Furthermore, you don't need to list every setting; you only need to list the ones that differ from the settings in the original env file. Basically, the settings in the original env file (including the resource_registry) define the default values for all netapp backends, and the entries in the CinderNetappMultiConfig define the overrides for specific backends. For example, if the original env file sets "CinderNetappStorageProtocol: iscsi" then that will be the default value for all netapp backends, unless you override it in the CinderNetappMultiConfig section. Lastly, your CinderNetappBackendName setting looks correct (it specifies both the 'tripleo_netapp' and 'tripleo_netapp_dcn02' backends). Alan
With regards, Swogat Pradhan
On Wed, May 31, 2023 at 10:46 PM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi Alan, Thanks for your clarification, the way you suggested will solve my issue. But i already have a netapp backend in my central site and to add another backend should i follow this documentation: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/17....
And should i remove the old netapp environment file and use the new custom environment file created using the above mentioned guide??
I already have a prod workload in the currently deployed netapp and I do not want to cause any issues in that netapp storage.
With regards, Swogat Pradhan
On Tue, May 30, 2023 at 8:43 PM Alan Bishop <abishop@redhat.com> wrote:
On Thu, May 25, 2023 at 9:39 PM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi Alan, My netapp storage is located in edge site itself. As the networks are routable my central site is able to reach the netapp storage ip address (ping response is 30ms-40ms). Let's say i included the netapp storage yaml in central site deployment script (which is not recommended) and i am able to create the volumes as it is reachable from controller nodes. Will i be able to mount those volumes in edge site VM's?? And if i am able to do so, then how will the data flow?? When storing something in the netapp volume will the data flow through the central site controller and get stored in the storage space?
A cinder-volume service running in the central site's controplane will be able to work with a netapp backend that's physically located at an edge site. The good news is the c-vol service will be HA because it will be controlled by pacemaker running on the controllers.
In order for VMs at the edge site to access volumes on the netapp, you'll need to set the CinderNetappAvailabilityZone [1] to the edge site's AZ.
[1] https://opendev.org/openstack/tripleo-heat-templates/src/branch/stable/walla...
To attach a netapp volume, nova-compute at the edge will interact with cinder-volume in the controlplane, and cinder-volume will in turn interact with the netapp. This will happen over central <=> edge network connections. Eventually, nova will directly connect to the netapp, so all traffic from the VM to the netapp will occur within the edge site. Data will not flow through the cinder-volume service, but there are restrictions and limitations: - Only that one edge site can access the netapp backend - If the central <=> edge network connection then you won't be able to attach or detach a netapp volume (but active connections will continue to work)
Of course, there are operations where cinder services are in the data path (e.g. creating a volume from an image), but not when a VM is accessing a volume.
Alan
With regards, Swogat Pradhan
On Fri, 26 May 2023, 10:03 am Alan Bishop, <abishop@redhat.com> wrote:
On Thu, May 25, 2023 at 12:09 AM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi Alan, So, can I include the cinder-netapp-storage.yaml file in the central site and then use the new backend to add storage to edge VM's?
Where is the NetApp physically located? Tripleo's DCN architecture assumes the storage is physically located at the same site where the cinder-volume service will be deployed. If you include the cinder-netapp-storage.yaml environment file in the central site's controlplane, then VMs at the edge site will encounter the problems I outlined earlier (network latency, no ability to do cross-AZ attachments).
I believe it is not possible right?? as the cinder volume in the edge won't have the config for the netapp.
The cinder-volume services at an edge site are meant to manage storage devices at that site. If the NetApp is at the edge site, ideally you'd include some variation of the cinder-netapp-storage.yaml environment file in the edge site's deployment. However, then you're faced with the fact that the NetApp driver doesn't support A/A, which is required for c-vol services running at edge sites (In case you're not familiar with these details, tripleo runs all cinder-volume services in active/passive mode under pacemaker on controllers in the controlplane. Thus, only a single instance runs at any time, and pacemaker provides HA by moving the service to another controller if the first one goes down. However, pacemaker is not available at edge sites, and so to get HA, multiple instances of the cinder-volume service run simultaneously on 3 nodes (A/A), using etcd as a Distributed Lock Manager (DLM) to coordinate things. But drivers must specifically support running A/A, and the NetApp driver does NOT.)
Alan
With regards, Swogat Pradhan
On Thu, May 25, 2023 at 2:17 AM Alan Bishop <abishop@redhat.com> wrote:
> > > On Wed, May 24, 2023 at 3:15 AM Swogat Pradhan < > swogatpradhan22@gmail.com> wrote: > >> Hi, >> I have a DCN setup and there is a requirement to use a netapp >> storage device in one of the edge sites. >> Can someone please confirm if it is possible? >> > > I see from prior email to this list that you're using tripleo, so > I'll respond with that in mind. > > There are many factors that come into play, but I suspect the short > answer to your question is no. > > Tripleo's DCN architecture requires the cinder-volume service > running at edge sites to run in active-active > mode, where there are separate instances running on three nodes in > to for the service to be highly > available (HA).The problem is that only a small number of cinder > drivers support running A/A, and NetApp's > drivers do not support A/A. > > It's conceivable you could create a custom tripleo role that deploys > just a single node running cinder-volume > with a NetApp backend, but it wouldn't be HA. > > It's also conceivable you could locate the NetApp system in the > central site's controlplane, but there are > extremely difficult constraints you'd need to overcome: > - Network latency between the central and edge sites would mean the > disk performance would be bad. > - You'd be limited to using iSCSI (FC wouldn't work) > - Tripleo disables cross-AZ attachments, so the only way for an edge > site to access a NetApp volume > would be to configure the cinder-volume service running in the > controlplane with a backend availability > zone set to the edge site's AZ. You mentioned the NetApp is needed > "in one of the edge sites," but in > reality the NetApp would be available in one, AND ONLY ONE edge > site, and it would also not be available > to any instances running in the central site. > > Alan > > >> And if so then should i add the parameters in the edge deployment >> script or the central deployment script. >> Any suggestions? >> >> With regards, >> Swogat Pradhan >> >
Thank you so much Alan for your help. I am using the following template to deploy the netapp. I have my original template and this is the custom template I am adding to it. I will check and confirm if this works. With regards, Swogat Pradhan On Thu, Jun 1, 2023 at 9:51 AM Alan Bishop <abishop@redhat.com> wrote:
On Wed, May 31, 2023 at 10:44 AM Swogat Pradhan <swogatpradhan22@gmail.com> wrote:
Hi Alan, Can you please check if the environment file attached is in correct format and how it should be? Also should i remove the old netapp environment file and use the new custom environment file created (attached).
You could retain the original environment file, and merge in just a portion of the sample file that you attached.
The original file contains all the settings, including the resource_registry entry, to deploy the first netapp in the controlplane. You want to enhance this with just a couple of items in order to deploy the second netapp at the edge site.
The important thing to know is the CinderNetappMultiConfig entries are necessary only for the second netapp. The 'tripleo_netapp' entry could be removed, because it's basically taken care of by the original env file. You only need to include the 'tripleo_netapp_dcn02' settings. Furthermore, you don't need to list every setting; you only need to list the ones that differ from the settings in the original env file. Basically, the settings in the original env file (including the resource_registry) define the default values for all netapp backends, and the entries in the CinderNetappMultiConfig define the overrides for specific backends. For example, if the original env file sets "CinderNetappStorageProtocol: iscsi" then that will be the default value for all netapp backends, unless you override it in the CinderNetappMultiConfig section.
Lastly, your CinderNetappBackendName setting looks correct (it specifies both the 'tripleo_netapp' and 'tripleo_netapp_dcn02' backends).
Alan
With regards, Swogat Pradhan
On Wed, May 31, 2023 at 10:46 PM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi Alan, Thanks for your clarification, the way you suggested will solve my issue. But i already have a netapp backend in my central site and to add another backend should i follow this documentation: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/17....
And should i remove the old netapp environment file and use the new custom environment file created using the above mentioned guide??
I already have a prod workload in the currently deployed netapp and I do not want to cause any issues in that netapp storage.
With regards, Swogat Pradhan
On Tue, May 30, 2023 at 8:43 PM Alan Bishop <abishop@redhat.com> wrote:
On Thu, May 25, 2023 at 9:39 PM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
Hi Alan, My netapp storage is located in edge site itself. As the networks are routable my central site is able to reach the netapp storage ip address (ping response is 30ms-40ms). Let's say i included the netapp storage yaml in central site deployment script (which is not recommended) and i am able to create the volumes as it is reachable from controller nodes. Will i be able to mount those volumes in edge site VM's?? And if i am able to do so, then how will the data flow?? When storing something in the netapp volume will the data flow through the central site controller and get stored in the storage space?
A cinder-volume service running in the central site's controplane will be able to work with a netapp backend that's physically located at an edge site. The good news is the c-vol service will be HA because it will be controlled by pacemaker running on the controllers.
In order for VMs at the edge site to access volumes on the netapp, you'll need to set the CinderNetappAvailabilityZone [1] to the edge site's AZ.
[1] https://opendev.org/openstack/tripleo-heat-templates/src/branch/stable/walla...
To attach a netapp volume, nova-compute at the edge will interact with cinder-volume in the controlplane, and cinder-volume will in turn interact with the netapp. This will happen over central <=> edge network connections. Eventually, nova will directly connect to the netapp, so all traffic from the VM to the netapp will occur within the edge site. Data will not flow through the cinder-volume service, but there are restrictions and limitations: - Only that one edge site can access the netapp backend - If the central <=> edge network connection then you won't be able to attach or detach a netapp volume (but active connections will continue to work)
Of course, there are operations where cinder services are in the data path (e.g. creating a volume from an image), but not when a VM is accessing a volume.
Alan
With regards, Swogat Pradhan
On Fri, 26 May 2023, 10:03 am Alan Bishop, <abishop@redhat.com> wrote:
On Thu, May 25, 2023 at 12:09 AM Swogat Pradhan < swogatpradhan22@gmail.com> wrote:
> Hi Alan, > So, can I include the cinder-netapp-storage.yaml file in the central > site and then use the new backend to add storage to edge VM's? >
Where is the NetApp physically located? Tripleo's DCN architecture assumes the storage is physically located at the same site where the cinder-volume service will be deployed. If you include the cinder-netapp-storage.yaml environment file in the central site's controlplane, then VMs at the edge site will encounter the problems I outlined earlier (network latency, no ability to do cross-AZ attachments).
> I believe it is not possible right?? as the cinder volume in the > edge won't have the config for the netapp. >
The cinder-volume services at an edge site are meant to manage storage devices at that site. If the NetApp is at the edge site, ideally you'd include some variation of the cinder-netapp-storage.yaml environment file in the edge site's deployment. However, then you're faced with the fact that the NetApp driver doesn't support A/A, which is required for c-vol services running at edge sites (In case you're not familiar with these details, tripleo runs all cinder-volume services in active/passive mode under pacemaker on controllers in the controlplane. Thus, only a single instance runs at any time, and pacemaker provides HA by moving the service to another controller if the first one goes down. However, pacemaker is not available at edge sites, and so to get HA, multiple instances of the cinder-volume service run simultaneously on 3 nodes (A/A), using etcd as a Distributed Lock Manager (DLM) to coordinate things. But drivers must specifically support running A/A, and the NetApp driver does NOT.)
Alan
> With regards, > Swogat Pradhan > > On Thu, May 25, 2023 at 2:17 AM Alan Bishop <abishop@redhat.com> > wrote: > >> >> >> On Wed, May 24, 2023 at 3:15 AM Swogat Pradhan < >> swogatpradhan22@gmail.com> wrote: >> >>> Hi, >>> I have a DCN setup and there is a requirement to use a netapp >>> storage device in one of the edge sites. >>> Can someone please confirm if it is possible? >>> >> >> I see from prior email to this list that you're using tripleo, so >> I'll respond with that in mind. >> >> There are many factors that come into play, but I suspect the short >> answer to your question is no. >> >> Tripleo's DCN architecture requires the cinder-volume service >> running at edge sites to run in active-active >> mode, where there are separate instances running on three nodes in >> to for the service to be highly >> available (HA).The problem is that only a small number of cinder >> drivers support running A/A, and NetApp's >> drivers do not support A/A. >> >> It's conceivable you could create a custom tripleo role that >> deploys just a single node running cinder-volume >> with a NetApp backend, but it wouldn't be HA. >> >> It's also conceivable you could locate the NetApp system in the >> central site's controlplane, but there are >> extremely difficult constraints you'd need to overcome: >> - Network latency between the central and edge sites would mean the >> disk performance would be bad. >> - You'd be limited to using iSCSI (FC wouldn't work) >> - Tripleo disables cross-AZ attachments, so the only way for an >> edge site to access a NetApp volume >> would be to configure the cinder-volume service running in the >> controlplane with a backend availability >> zone set to the edge site's AZ. You mentioned the NetApp is needed >> "in one of the edge sites," but in >> reality the NetApp would be available in one, AND ONLY ONE edge >> site, and it would also not be available >> to any instances running in the central site. >> >> Alan >> >> >>> And if so then should i add the parameters in the edge deployment >>> script or the central deployment script. >>> Any suggestions? >>> >>> With regards, >>> Swogat Pradhan >>> >>
participants (2)
-
Alan Bishop
-
Swogat Pradhan