[openstack-dev][magnum] Using Fedora Atomic 29 for k8s cluster
Hi all, At this moment, Magnum is still using Fedora Atomic 27 as the default image in devstack. But you can definitely use Fedora Atomic 29 and it works fine. But you may run into a performance issue when booting Fedora Atomic 29 if your compute host doesn't have enough entropy. There are two steps you need for that case: 1. Adding property hw_rng_model='virtio' to Fedora Atomic 29 image 2. Adding property hw_rng:allowed='True' to flavor, and we also need hw_rng:rate_bytes=4096 and hw_rng:rate_period=1 to get a reasonable rate limit to avoid the VM drain the hypervisor. We are working on a patch for Magnum devstack to support FA29 out of box. Meanwhile, we're starting to test Fedora CoreOS 30. Please popup in #openstack-containers channel if you have any question. Cheers. -- Cheers & Best regards, Feilong Wang (王飞龙) -------------------------------------------------------------------------- Head of R&D Catalyst Cloud - Cloud Native New Zealand Tel: +64-48032246 Email: flwang@catalyst.net.nz Level 6, Catalyst House, 150 Willis Street, Wellington --------------------------------------------------------------------------
On Thu, Aug 22, 2019 at 11:46 PM Feilong Wang <feilong@catalyst.net.nz> wrote:
Hi all,
At this moment, Magnum is still using Fedora Atomic 27 as the default image in devstack. But you can definitely use Fedora Atomic 29 and it works fine. But you may run into a performance issue when booting Fedora Atomic 29 if your compute host doesn't have enough entropy. There are two steps you need for that case:
1. Adding property hw_rng_model='virtio' to Fedora Atomic 29 image
2. Adding property hw_rng:allowed='True' to flavor, and we also need hw_rng:rate_bytes=4096 and hw_rng:rate_period=1 to get a reasonable rate limit to avoid the VM drain the hypervisor.
We are working on a patch for Magnum devstack to support FA29 out of box. Meanwhile, we're starting to test Fedora CoreOS 30. Please popup in #openstack-containers channel if you have any question. Cheers.
Neat! I think it's important for us to get off Fedora Atomic given that RedHat seems to be ending it soon. Is the plan to move towards Fedora CoreOS 30 or has there been consideration of using something like an Ubuntu-base (and leveraging something like kubeadm+ansible to drive the deployment?)-
-- Cheers & Best regards, Feilong Wang (王飞龙) -------------------------------------------------------------------------- Head of R&D Catalyst Cloud - Cloud Native New Zealand Tel: +64-48032246 Email: flwang@catalyst.net.nz Level 6, Catalyst House, 150 Willis Street, Wellington --------------------------------------------------------------------------
-- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser@vexxhost.com W. http://vexxhost.com
On 2019-08-23 10:58:11 -0400 (-0400), Mohammed Naser wrote: [...]
Neat! I think it's important for us to get off Fedora Atomic given that RedHat seems to be ending it soon. Is the plan to move towards Fedora CoreOS 30 or has there been consideration of using something like an Ubuntu-base (and leveraging something like kubeadm+ansible to drive the deployment?)-
This seems like it's particularly time-sensitive as Fedora 29 will be EOL and no longer receiving security fixes within just a few short months from now. If it's not fixed before Train is finalized, you're basically releasing an unusable service since it will pretty much immediately be depending on an insecure distribution nobody wants to put into production, right? -- Jeremy Stanley
On Fri, Aug 23, 2019, at 8:21 AM, Jeremy Stanley wrote:
On 2019-08-23 10:58:11 -0400 (-0400), Mohammed Naser wrote: [...]
Neat! I think it's important for us to get off Fedora Atomic given that RedHat seems to be ending it soon. Is the plan to move towards Fedora CoreOS 30 or has there been consideration of using something like an Ubuntu-base (and leveraging something like kubeadm+ansible to drive the deployment?)-
This seems like it's particularly time-sensitive as Fedora 29 will be EOL and no longer receiving security fixes within just a few short months from now. If it's not fixed before Train is finalized, you're basically releasing an unusable service since it will pretty much immediately be depending on an insecure distribution nobody wants to put into production, right?
Note that 27 is still the default and has been EOL for a long time. Getting to 29 is a good improvement on top of that even if it EOLs sooner than we would like.
-- Jeremy Stanley
Attachments: * signature.asc
On Fri, Aug 23, 2019 at 11:44 AM Clark Boylan <cboylan@sapwetik.org> wrote:
On Fri, Aug 23, 2019, at 8:21 AM, Jeremy Stanley wrote:
On 2019-08-23 10:58:11 -0400 (-0400), Mohammed Naser wrote: [...]
Neat! I think it's important for us to get off Fedora Atomic given that RedHat seems to be ending it soon. Is the plan to move towards Fedora CoreOS 30 or has there been consideration of using something like an Ubuntu-base (and leveraging something like kubeadm+ansible to drive the deployment?)-
This seems like it's particularly time-sensitive as Fedora 29 will be EOL and no longer receiving security fixes within just a few short months from now. If it's not fixed before Train is finalized, you're basically releasing an unusable service since it will pretty much immediately be depending on an insecure distribution nobody wants to put into production, right?
Note that 27 is still the default and has been EOL for a long time. Getting to 29 is a good improvement on top of that even if it EOLs sooner than we would like.
+1 to this
-- Jeremy Stanley
Attachments: * signature.asc
-- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser@vexxhost.com W. http://vexxhost.com
On 2019-08-24 03:20, Jeremy Stanley wrote:
On 2019-08-23 10:58:11 -0400 (-0400), Mohammed Naser wrote: [...]
Neat! I think it's important for us to get off Fedora Atomic given that RedHat seems to be ending it soon. Is the plan to move towards Fedora CoreOS 30 or has there been consideration of using something like an Ubuntu-base (and leveraging something like kubeadm+ansible to drive the deployment?)-
This seems like it's particularly time-sensitive as Fedora 29 will be EOL and no longer receiving security fixes within just a few short months from now.
If it's not fixed before Train is finalized,
you're basically releasing an unusable service since it will pretty much immediately be depending on an insecure distribution nobody wants to put into production, right?
I don't think so. Devstack is just a dev environment. If you're going to use FA29 on production, then you can set the properties on image/flavor for your prod.
On 2019-08-24 06:53:00 +1200 (+1200), feilong@catalyst.net.nz wrote:
On 2019-08-24 03:20, Jeremy Stanley wrote: [...]
If it's not fixed before Train is finalized, you're basically releasing an unusable service since it will pretty much immediately be depending on an insecure distribution nobody wants to put into production, right?
I don't think so. Devstack is just a dev environment. If you're going to use FA29 on production, then you can set the properties on image/flavor for your prod.
I'm probably missing something, but when Fedora 29 reaches EOL there will be no security supported Atomic, right? Are you saying Magnum will work fine in production without Atomic, that it's only used when testing via DevStack? -- Jeremy Stanley
On 2019-08-24 07:40, Jeremy Stanley wrote:
On 2019-08-24 06:53:00 +1200 (+1200), feilong@catalyst.net.nz wrote:
On 2019-08-24 03:20, Jeremy Stanley wrote: [...]
If it's not fixed before Train is finalized, you're basically releasing an unusable service since it will pretty much immediately be depending on an insecure distribution nobody wants to put into production, right?
I don't think so. Devstack is just a dev environment. If you're going to use FA29 on production, then you can set the properties on image/flavor for your prod.
I'm probably missing something, but when Fedora 29 reaches EOL there will be no security supported Atomic, right? Are you saying Magnum will work fine in production without Atomic, that it's only used when testing via DevStack?
Firstly, I don't know when Fedora Atomic 29 will EOL at this moment, given the Fedora CoreOS 30 is still in testing status. I'm saying at this moment, in production, user can use Fedora Atomic 29.
On 2019-08-24 08:00:41 +1200 (+1200), feilong@catalyst.net.nz wrote: [...]
Firstly, I don't know when Fedora Atomic 29 will EOL at this moment, given the Fedora CoreOS 30 is still in testing status.
I'm saying at this moment, in production, user can use Fedora Atomic 29.
I see. In the past, Fedora versions have reached EOL around 13 months from their initial release: https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle#Maintenance_Schedul... Fedora 29 was released (slightly behind schedule) at the end of October 2018: https://fedoraproject.org/wiki/Releases/29/Schedule This means it's likely to be EOL at the end of November 2019, roughly 6 weeks after we plan to release Train: https://releases.openstack.org/train/schedule.html If users of the Train release of Magnum are expected to rely on a feature which will only be available in Fedora 29, then basically Magnum will only have a viability of 6 weeks after Train releases before those users are stuck running a release of Fedora which has no security support from Red Hat. The development cycle for Train is rapidly approaching the station, if you'll forgive the simile, with coordinated feature freeze only 2 weeks away now. It would be unfortunate for Magnum to release something nobody can safely use, and it looks to me like time is running out if we want to avoid that. I sincerely hope I'm missing some important detail in all of this, and am eager to find out what it is. -- Jeremy Stanley
On 2019-08-24 08:15, Jeremy Stanley wrote:
On 2019-08-24 08:00:41 +1200 (+1200), feilong@catalyst.net.nz wrote: [...]
Firstly, I don't know when Fedora Atomic 29 will EOL at this moment, given the Fedora CoreOS 30 is still in testing status.
I'm saying at this moment, in production, user can use Fedora Atomic 29.
I see. In the past, Fedora versions have reached EOL around 13 months from their initial release:
https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle#Maintenance_Schedul...
Fedora 29 was released (slightly behind schedule) at the end of October 2018:
https://fedoraproject.org/wiki/Releases/29/Schedule
This means it's likely to be EOL at the end of November 2019, roughly 6 weeks after we plan to release Train:
https://releases.openstack.org/train/schedule.html
If users of the Train release of Magnum are expected to rely on a feature which will only be available in Fedora 29, then basically Magnum will only have a viability of 6 weeks after Train releases before those users are stuck running a release of Fedora which has no security support from Red Hat. The development cycle for Train is rapidly approaching the station, if you'll forgive the simile, with coordinated feature freeze only 2 weeks away now. It would be unfortunate for Magnum to release something nobody can safely use, and it looks to me like time is running out if we want to avoid that.
I sincerely hope I'm missing some important detail in all of this, and am eager to find out what it is.
I understand that and that's why we worked on the upgrade feature so that user can upgrade to a newer version operating system. Given the Fedora Atomic to Fedora CoreOS is a big jump, I don't know if the 13 months life is still the case. TBH, I assume it could be longer. Again, we're working hard on this but we don't have a perfect solution due to the limited resources.
On 2019-08-24 02:58, Mohammed Naser wrote:
On Thu, Aug 22, 2019 at 11:46 PM Feilong Wang <feilong@catalyst.net.nz> wrote:
Hi all,
At this moment, Magnum is still using Fedora Atomic 27 as the default image in devstack. But you can definitely use Fedora Atomic 29 and it works fine. But you may run into a performance issue when booting Fedora Atomic 29 if your compute host doesn't have enough entropy. There are two steps you need for that case:
1. Adding property hw_rng_model='virtio' to Fedora Atomic 29 image
2. Adding property hw_rng:allowed='True' to flavor, and we also need hw_rng:rate_bytes=4096 and hw_rng:rate_period=1 to get a reasonable rate limit to avoid the VM drain the hypervisor.
We are working on a patch for Magnum devstack to support FA29 out of box. Meanwhile, we're starting to test Fedora CoreOS 30. Please popup in #openstack-containers channel if you have any question. Cheers.
Neat! I think it's important for us to get off Fedora Atomic given that RedHat seems to be ending it soon. Is the plan to move towards Fedora CoreOS 30 or has there been consideration of using something like an Ubuntu-base (and leveraging something like kubeadm+ansible to drive the deployment?)-
Personally, I would like we can stay at Fedora Atomic/CoreOS since Magnum has already been benefited from the container-based readonly operating system. But we did have the discussion about using kubeadm+ansible, however, as you can see, it's a quite big refactoring, I'm not sure if we can get it done with current limited resources.
-- Cheers & Best regards, Feilong Wang (王飞龙) -------------------------------------------------------------------------- Head of R&D Catalyst Cloud - Cloud Native New Zealand Tel: +64-48032246 Email: flwang@catalyst.net.nz Level 6, Catalyst House, 150 Willis Street, Wellington --------------------------------------------------------------------------
pá 23. 8. 2019 v 20:59 odesílatel <feilong@catalyst.net.nz> napsal:
On 2019-08-24 02:58, Mohammed Naser wrote:
On Thu, Aug 22, 2019 at 11:46 PM Feilong Wang <feilong@catalyst.net.nz> wrote:
Hi all,
At this moment, Magnum is still using Fedora Atomic 27 as the default image in devstack. But you can definitely use Fedora Atomic 29 and it works fine. But you may run into a performance issue when booting Fedora Atomic 29 if your compute host doesn't have enough entropy. There are two steps you need for that case:
1. Adding property hw_rng_model='virtio' to Fedora Atomic 29 image
2. Adding property hw_rng:allowed='True' to flavor, and we also need hw_rng:rate_bytes=4096 and hw_rng:rate_period=1 to get a reasonable rate limit to avoid the VM drain the hypervisor.
We are working on a patch for Magnum devstack to support FA29 out of box. Meanwhile, we're starting to test Fedora CoreOS 30. Please popup in #openstack-containers channel if you have any question. Cheers.
Neat! I think it's important for us to get off Fedora Atomic given that RedHat seems to be ending it soon. Is the plan to move towards Fedora CoreOS 30 or has there been consideration of using something like an Ubuntu-base (and leveraging something like kubeadm+ansible to drive the deployment?)-
Personally, I would like we can stay at Fedora Atomic/CoreOS since Magnum has already been benefited from the container-based readonly operating system. But we did have the discussion about using kubeadm+ansible, however, as you can see, it's a quite big refactoring, I'm not sure if we can get it done with current limited resources.
According to some notes from Magnum Train PTG meeting there is plan to move all drivers out of tree. Despite there is no time schedule for that we started to work on Ubuntu K8s driver which is completely independent to openstack/magnum project and which is based on kubeadm, Heat SW Deployments and external Openstack cloud provider. So we do not think there is much refactoring needed. At the moment it is under heavy development, however, at some point we plan to open the source and we will certainly appreciate any help.
-- Cheers & Best regards, Feilong Wang (王飞龙)
Head of R&D Catalyst Cloud - Cloud Native New Zealand Tel: +64-48032246 Email: flwang@catalyst.net.nz Level 6, Catalyst House, 150 Willis Street, Wellington
Jakub Slíva Ultimum Technologies s.r.o. Na Poříčí 1047/26, 11000 Praha 1 Czech Republic jakub.sliva@ultimum.io *https://ultimum.io <https://ultimum.io>* LinkedIn <https://www.linkedin.com/company/ultimum-technologies> | Twitter <https://twitter.com/ultimumtech> | Facebook <https://www.facebook.com/ultimumtechnologies/timeline>
On Fri, Aug 30, 2019 at 5:02 AM Jakub Slíva <jakub.sliva@ultimum.io> wrote:
pá 23. 8. 2019 v 20:59 odesílatel <feilong@catalyst.net.nz> napsal:
On 2019-08-24 02:58, Mohammed Naser wrote:
On Thu, Aug 22, 2019 at 11:46 PM Feilong Wang <feilong@catalyst.net.nz> wrote:
Hi all,
At this moment, Magnum is still using Fedora Atomic 27 as the default image in devstack. But you can definitely use Fedora Atomic 29 and it works fine. But you may run into a performance issue when booting Fedora Atomic 29 if your compute host doesn't have enough entropy. There are two steps you need for that case:
1. Adding property hw_rng_model='virtio' to Fedora Atomic 29 image
2. Adding property hw_rng:allowed='True' to flavor, and we also need hw_rng:rate_bytes=4096 and hw_rng:rate_period=1 to get a reasonable rate limit to avoid the VM drain the hypervisor.
We are working on a patch for Magnum devstack to support FA29 out of box. Meanwhile, we're starting to test Fedora CoreOS 30. Please popup in #openstack-containers channel if you have any question. Cheers.
Neat! I think it's important for us to get off Fedora Atomic given that RedHat seems to be ending it soon. Is the plan to move towards Fedora CoreOS 30 or has there been consideration of using something like an Ubuntu-base (and leveraging something like kubeadm+ansible to drive the deployment?)-
Personally, I would like we can stay at Fedora Atomic/CoreOS since Magnum has already been benefited from the container-based readonly operating system. But we did have the discussion about using kubeadm+ansible, however, as you can see, it's a quite big refactoring, I'm not sure if we can get it done with current limited resources.
According to some notes from Magnum Train PTG meeting there is plan to move all drivers out of tree. Despite there is no time schedule for that we started to work on Ubuntu K8s driver which is completely independent to openstack/magnum project and which is based on kubeadm, Heat SW Deployments and external Openstack cloud provider. So we do not think there is much refactoring needed. At the moment it is under heavy development, however, at some point we plan to open the source and we will certainly appreciate any help.
Let's share efforts! https://review.opendev.org/#/c/678309/ This driver can get you a full deployment but the only thing missing (which shouldn't be much more) is using Magnum CA support. There is a document in that patch that you can try to use DevStack with! Thanks Mohammed
-- Cheers & Best regards, Feilong Wang (王飞龙) -------------------------------------------------------------------------- Head of R&D Catalyst Cloud - Cloud Native New Zealand Tel: +64-48032246 Email: flwang@catalyst.net.nz Level 6, Catalyst House, 150 Willis Street, Wellington --------------------------------------------------------------------------
Jakub Slíva
Ultimum Technologies s.r.o. Na Poříčí 1047/26, 11000 Praha 1 Czech Republic
jakub.sliva@ultimum.io https://ultimum.io
LinkedIn | Twitter | Facebook
-- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser@vexxhost.com W. http://vexxhost.com
participants (6)
-
Clark Boylan
-
Feilong Wang
-
feilong＠catalyst.net.nz
-
Jakub Slíva
-
Jeremy Stanley
-
Mohammed Naser