Open Stack

On Wed, Mar 3, 2021 at 12:57 AM Goutham Pacha Ravi
<gouthampravi at gmail.com> wrote:
>
> On Mon, Mar 1, 2021 at 10:29 AM Sean Mooney <smooney at redhat.com> wrote:
> >
> > On Mon, 2021-03-01 at 16:07 +0000, Lucas Alvares Gomes wrote:
> > > Hi all,
> > >
> > > As part of the Victoria PTG [0] the Neutron community agreed upon
> > > switching the default backend in Devstack to OVN. A lot of work has
> > > been done since, from porting the OVN devstack module to the DevStack
> > > tree, refactoring the DevStack module to install OVN from distro
> > > packages, implementing features to close the parity gap with ML2/OVS,
> > > fixing issues with tests and distros, etc...
> > >
> > > We are now very close to being able to make the switch and we've
> > > thought about sending this email to the broader community to raise
> > > awareness about this change as well as bring more attention to the
> > > patches that are current on review.
> > >
> > > Note that moving DevStack to ML2/OVN does not mean that ML2/OVS is
> > > discontinued and/or not supported anymore. The ML2/OVS driver is still
> > > going to be developed and maintained by the upstream Neutron
> > > community.
> > can we ensure that this does not happen until the xena release.
> > in generall i think its ok to change the default but not this late in the cycle.
> > i would also like to ensure we keep at least one non ovn based multi node job in
> > nova until https://review.opendev.org/c/openstack/nova/+/602432 is merged and possible after.
> > right now the event/neutorn interaction is not the same during move operations.
> > >
> > > Below is a e per project explanation with relevant links and issues of
> > > where we stand with this work right now:
> > >
> > > * Keystone:
> > >
> > > Everything should be good for Keystone, the gate is happy with the
> > > changes. Here is the test patch:
> > > https://review.opendev.org/c/openstack/keystone/+/777963
> > >
> > > * Glance:
> > >
> > > Everything should be good for Glace, the gate is happy with the
> > > changes. Here is the test patch:
> > > https://review.opendev.org/c/openstack/glance/+/748390
> > >
> > > * Swift:
> > >
> > > Everything should be good for Swift, the gate is happy with the
> > > changes. Here is the test patch:
> > > https://review.opendev.org/c/openstack/swift/+/748403
> > >
> > > * Ironic:
> > >
> > > Since chainloading iPXE by the OVN built-in DHCP server is work in
> > > progress, we've changed most of the Ironic jobs to explicitly enable
> > > ML2/OVS and everything is merged, so we should be good for Ironic too.
> > > Here is the test patch:
> > > https://review.opendev.org/c/openstack/ironic/+/748405
> > >
> > > * Cinder:
> > >
> > > Cinder is almost complete. There's one test failure in the
> > > "tempest-slow-py3" job run on the
> > > "test_port_security_macspoofing_port" test.
> > >
> > > This failure is due to a bug in core OVN [1]. This bug has already
> > > been fixed upstream [2] and the fix has been backported down to the
> > > branch-20.03 [3] of the OVN project. However, since we install OVN
> > > from packages we are currently waiting for this fix to be included in
> > > the packages for Ubuntu Focal (it's based on OVN 20.03). I already
> > > contacted the package maintainer which has been very supportive of
> > > this work and will work on the package update, but he maintain a
> > > handful of backports in that package which is not yet included in OVN
> > > 20.03 upstream and he's now working with the core OVN community [4] to
> > > include it first in the branch and then create a new package for it.
> > > Hopefully this will happen soon.
> > >
> > > But for now we have a few options moving on with this issue:
> > >
> > > 1- Wait for the new package version
> > > 2- Mark the test as unstable until we get the new package version
> > > 3- Compile OVN from source instead of installing it from packages
> > > (OVN_BUILD_FROM_SOURCE=True in local.conf)
> > i dont think we should default to ovn untill a souce build is not required.
> > compiling form souce while not supper expensice still adds time to the job
> > execution and im not sure we should be paying that cost on every devstack job run.
> >
> > we could maybe compile it once and bake the package into the image or host it on a mirror
> > but i think we should avoid this option if we have alternitives.
> > >
> > > What do you think about it ?
> > >
> > > Here is the test patch for Cinder:
> > > https://review.opendev.org/c/openstack/cinder/+/748227
> > >
> > > * Nova:
> > >
> > > There are a few patches waiting for review for Nova, which are:
> > >
> > > 1- Adapting the live migration scripts to work with ML2/OVN: Basically
> > > the scripts were trying to stop the Neutron agent (q-agt) process
> > > which is not part of an ML2/OVN deployment. The patch changes the code
> > > to check if that system unit exists before trying to stop it.
> > >
> > > Patch: https://review.opendev.org/c/openstack/nova/+/776419
> > >
> > > 2- Explicitly set grenade job to ML2/OVS: This is a temporary change
> > > which can be removed one release cycle after we switch DevStack to
> > > ML2/OVN. Grenade will test updating from the release version to the
> > > master branch but, since the default of the released version is not
> > > ML2/OVN, upgrading from ML2/OVS to ML2/OVN as part of the grenade job
> > > is not supported.
> > >
> > > Patch: https://review.opendev.org/c/openstack/nova/+/776934
> > >
> > > 3- Explicitly set nova-next job to ML2/OVS: This job uses the QoS
> > > minimum bandwidth feature which is not yet supported by ML2/OVN [5][6]
> > > therefore we are temporarily enabling ML2/OVS for this job until that
> > > feature lands in core OVN.
> > >
> > > Patch: https://review.opendev.org/c/openstack/nova/+/776944
> > >
> > > I also spoke briefly with Sean Mooney (irc: sean-k-mooney) about these
> > > changes and he suggested keeping all the Nova jobs on ML2/OVS for now
> > > because he feels like a change in the default network driver a few
> > > weeks prior to the upstream code freeze can be concerning. We do not
> > > know yet precisely when we are changing the default due to the current
> > > patches we need to get merged but, if this is a shared feeling among
> > > the Nova community I can work on enabling ML2/OVS on all jobs in Nova
> > > until we get a new release in OpenStack.
> > yep this is still my view.
> > i would suggest we do the work required in the repos but not merge it until the xena release
> > is open. thats technically at RC1 so march 25th
> > i think we can safely do the swich after that but i would not change the defualt in any project
> > before then.
> > >
> > > Here's the test patch for Nova:
> > > https://review.opendev.org/c/openstack/nova/+/776945
> > >
> > > * DevStack:
> > >
> > > And this is the final patch that will make this all happen:
> > > https://review.opendev.org/c/openstack/devstack/+/735097
> > >
> > > It changes the default in DevStack from ML2/OVS to ML2/OVN. It's been
> > > a long and bumpy road to get to this point and I would like to say
> > > thanks to everyone involved so far and everyone that read the whole
> > > email, please let me know your thoughts.
> > thanks for working on this.
> > >
> > > [0] https://etherpad.opendev.org/p/neutron-victoria-ptg
> > > [1] https://bugs.launchpad.net/tempest/+bug/1728886
> > > [2] https://patchwork.ozlabs.org/project/openvswitch/patch/20200319122641.473776-1-numans@ovn.org/
> > > [3] https://github.com/ovn-org/ovn/commit/0c26bc03064f2c21d208f0f860b48d8ab39380cb
> > > [4] https://mail.openvswitch.org/pipermail/ovs-discuss/2021-February/050961.html
> > > [5] https://github.com/openstack/nova/blob/ded25f33c734ebff963f06984707a99fe76a9ee1/gate/post_test_hook.sh#L129-L143
> > > [6] https://docs.openstack.org/neutron/latest/ovn/gaps.html
> > >
> > > Cheers,
> > > Lucas
>
> ++ Thank you indeed for working diligently on this important change.
>
> Please do note that devstack, and the base job that you're modifying
> is used by many other projects besides the ones that you have
> enumerated in the subject line.
> I suggest using [all] as a better subject line indicator to get the
> attention of folks like me who have filters based on the subject line.
> Also, the network substrate is important for the project I help
> maintain: Manila, which provides shared file systems over a network -
> so I followed your lead and submitted a dependent patch. I hope to
> reach out to you in case we see some breakages:
> https://review.opendev.org/c/openstack/manila-tempest-plugin/+/778346
>

Thanks for the suggestion, you are right the [ALL] would make sense. I
will try to raise more awareness as possible on this change in the
default.

Also thanks for proposing the test patch for Manilla, I see the gate
is happy there! But yeah, feel free to reach out to me if anything
breaks and I will glad looking into it.

> > >
> >
> >
> >
>