[Kolla][nova] /var/lib/nova/instances on nfs
Hello Stackers, is there any configuration parameter in kolla for sharing nova on nfs between compute nodes ? Or I must insert an entry in fstab ? Thanks Ignazio
Hello Ignazio, If you are not using Cinder NFS backend already, you need to set: enable_shared_var_lib_nova_mnt: yes And yes, you need to manage fstab yourself, mounting the shared nfs at /var/lib/nova/mnt It must happen before the containers are started (so before deploy or redeploy). -yoctozepto On Tue, Jun 22, 2021 at 12:11 PM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hello Stackers, is there any configuration parameter in kolla for sharing nova on nfs between compute nodes ? Or I must insert an entry in fstab ? Thanks Ignazio
On Tue, 2021-06-22 at 13:02 +0200, Radosław Piliszek wrote:
Hello Ignazio,
If you are not using Cinder NFS backend already, you need to set:
enable_shared_var_lib_nova_mnt: yes
And yes, you need to manage fstab yourself, mounting the shared nfs at /var/lib/nova/mnt
It must happen before the containers are started (so before deploy or redeploy). i dont think they were refering to cinder nfs. we have support for deploying novas state directory and libvirts stroage on nfs in nvoa when usign the raw/qcow image backend.
in general i advise against that but it is supported. you should ensure that you use nfs v4 preferable nfs v4.2 or newer with my downstream hat on we droped supprot for nfs v3 many years ago and the last lts release we hadd that supported it was based on newton. technially we dont have a min nfs version requirement ustream but at some point i think we shoudl enforce at least nfs v4 upstream too. there are several known locking issues with nfs v3 that make it generally problematic to use at scale with nova that manifest intermietnly during move operations. the same may or may not be true with nfs via cinder but that is one of the less well tested and hardened cinder backends to use with nova.
-yoctozepto
On Tue, Jun 22, 2021 at 12:11 PM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hello Stackers, is there any configuration parameter in kolla for sharing nova on nfs between compute nodes ? Or I must insert an entry in fstab ? Thanks Ignazio
Hello , I am using cinder with netapp driver but if I do not mount a share under /var/lib/docker/volumes/nova_compute/_data live migration does not work because an error is disblayed: shared storage in needes or something like that. I do not understand why is does not notice that volumes are shared Ignazio Il giorno mar 22 giu 2021 alle ore 13:43 Sean Mooney <smooney@redhat.com> ha scritto:
On Tue, 2021-06-22 at 13:02 +0200, Radosław Piliszek wrote:
Hello Ignazio,
If you are not using Cinder NFS backend already, you need to set:
enable_shared_var_lib_nova_mnt: yes
And yes, you need to manage fstab yourself, mounting the shared nfs at /var/lib/nova/mnt
It must happen before the containers are started (so before deploy or redeploy). i dont think they were refering to cinder nfs. we have support for deploying novas state directory and libvirts stroage on nfs in nvoa when usign the raw/qcow image backend.
in general i advise against that but it is supported. you should ensure that you use nfs v4 preferable nfs v4.2 or newer
with my downstream hat on we droped supprot for nfs v3 many years ago and the last lts release we hadd that supported it was based on newton. technially we dont have a min nfs version requirement ustream but at some point i think we shoudl enforce at least nfs v4 upstream too. there are several known locking issues with nfs v3 that make it generally problematic to use at scale with nova that manifest intermietnly during move operations.
the same may or may not be true with nfs via cinder but that is one of the less well tested and hardened cinder backends to use with nova.
-yoctozepto
On Tue, Jun 22, 2021 at 12:11 PM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hello Stackers, is there any configuration parameter in kolla for sharing nova on nfs between compute nodes ? Or I must insert an entry in fstab ? Thanks Ignazio
On Tue, 2021-06-22 at 13:57 +0200, Ignazio Cassano wrote:
Hello , I am using cinder with netapp driver but if I do not mount a share under /var/lib/docker/volumes/nova_compute/_data live migration does not work because an error is disblayed: shared storage in needes or something like that. I do not understand why is does not notice that volumes are shared
so these are cinder boot form volume guests? if you are usign the correct microverion it should detect that its shared storage automaticaly when you do a migration can you confirm the command you are usign to do the migration and that its a boot form volumen guest not an image backed guest with a data volumn.
Ignazio
Il giorno mar 22 giu 2021 alle ore 13:43 Sean Mooney < smooney@redhat.com> ha scritto:
On Tue, 2021-06-22 at 13:02 +0200, Radosław Piliszek wrote:
Hello Ignazio,
If you are not using Cinder NFS backend already, you need to set:
enable_shared_var_lib_nova_mnt: yes
And yes, you need to manage fstab yourself, mounting the shared nfs at /var/lib/nova/mnt
It must happen before the containers are started (so before deploy or redeploy). i dont think they were refering to cinder nfs. we have support for deploying novas state directory and libvirts stroage on nfs in nvoa when usign the raw/qcow image backend.
in general i advise against that but it is supported. you should ensure that you use nfs v4 preferable nfs v4.2 or newer
with my downstream hat on we droped supprot for nfs v3 many years ago and the last lts release we hadd that supported it was based on newton. technially we dont have a min nfs version requirement ustream but at some point i think we shoudl enforce at least nfs v4 upstream too. there are several known locking issues with nfs v3 that make it generally problematic to use at scale with nova that manifest intermietnly during move operations.
the same may or may not be true with nfs via cinder but that is one of the less well tested and hardened cinder backends to use with nova.
-yoctozepto
On Tue, Jun 22, 2021 at 12:11 PM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hello Stackers, is there any configuration parameter in kolla for sharing nova on nfs between compute nodes ? Or I must insert an entry in fstab ? Thanks Ignazio
Hello Sean, you are right. Setting cinder nfs it works fine and we do not need do share /var/lib/nova. Probably there was some errors in my previous installation. Ignazio Il giorno mar 22 giu 2021 alle ore 14:32 Sean Mooney <smooney@redhat.com> ha scritto:
On Tue, 2021-06-22 at 13:57 +0200, Ignazio Cassano wrote:
Hello , I am using cinder with netapp driver but if I do not mount a share under /var/lib/docker/volumes/nova_compute/_data live migration does not work because an error is disblayed: shared storage in needes or something like that. I do not understand why is does not notice that volumes are shared
so these are cinder boot form volume guests? if you are usign the correct microverion it should detect that its shared storage automaticaly when you do a migration can you confirm the command you are usign to do the migration and that its a boot form volumen guest not an image backed guest with a data volumn.
Ignazio
Il giorno mar 22 giu 2021 alle ore 13:43 Sean Mooney < smooney@redhat.com> ha scritto:
On Tue, 2021-06-22 at 13:02 +0200, Radosław Piliszek wrote:
Hello Ignazio,
If you are not using Cinder NFS backend already, you need to set:
enable_shared_var_lib_nova_mnt: yes
And yes, you need to manage fstab yourself, mounting the shared nfs at /var/lib/nova/mnt
It must happen before the containers are started (so before deploy or redeploy). i dont think they were refering to cinder nfs. we have support for deploying novas state directory and libvirts stroage on nfs in nvoa when usign the raw/qcow image backend.
in general i advise against that but it is supported. you should ensure that you use nfs v4 preferable nfs v4.2 or newer
with my downstream hat on we droped supprot for nfs v3 many years ago and the last lts release we hadd that supported it was based on newton. technially we dont have a min nfs version requirement ustream but at some point i think we shoudl enforce at least nfs v4 upstream too. there are several known locking issues with nfs v3 that make it generally problematic to use at scale with nova that manifest intermietnly during move operations.
the same may or may not be true with nfs via cinder but that is one of the less well tested and hardened cinder backends to use with nova.
-yoctozepto
On Tue, Jun 22, 2021 at 12:11 PM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hello Stackers, is there any configuration parameter in kolla for sharing nova on nfs between compute nodes ? Or I must insert an entry in fstab ? Thanks Ignazio
participants (3)
-
Ignazio Cassano
-
Radosław Piliszek
-
Sean Mooney