[openstack-dev] [neutron][upgrades] Potential issues when performing Neutron upgrades

Kevin Benton blak111 at gmail.com
Mon Jul 13 08:09:12 UTC 2015


>because you won't have to run Neutron agents on compute nodes anymore.

How will upgrades work for OVN?

On Thu, Jul 9, 2015 at 2:30 PM, Russell Bryant <rbryant at redhat.com> wrote:

> On 07/09/2015 10:11 AM, Ihar Hrachyshka wrote:
> > On 07/09/2015 09:01 AM, Artur Korzen wrote:
> >> Hi all,
> >
> >> I've been researching the Neutron project as a part of work on
> >> Openstack rolling upgrades, my primary assignments included
> >> testing if there is no VM access downtime when performing
> >> upgrade.
> >
> >
> >
> >> Are you aware of any issues that are present in Liberty release
> >> of Neutron, that may cause the VM access downtime and may stop
> >> ops to pick up newest versions of code?
> >
> >
> >
> >> When talking about no VM access downtime during upgrade, the
> >> sensitive operations are:
> >
> >> 1. ensure that even after neutron services is
> >> shutoff/uninstalled, the traffic can go into the VM 2. take care
> >> on neutron services startup/installation that existing
> >> configuration is preserved (routing tables, forwarding entries
> >> and security rules) 3. the underlying network technologies (OVS,
> >> linux bridges etc.) is working without distraction when upgrading
> >> neutron code: no dropping table flows or applying the traffic
> >> rules to drop the packages
> >
> >
> >
> >> To be compatible between different versions of service talking
> >> to each other, the important areas are:
> >
> >> 1. RPC API versioning 2. Objects exchanged between services via
> >> RPC 3. Database schema (bp [2]) and data migration
> >
> >
> >
> >> I have researched the ML2 plugin with ovs agent, including the
> >> neutron-ovs-agent, dhcp-server, l3-agent, neutron-server.
> >
> >> One of the VM access downtime during upgrade is addressed in [1],
> >>  removing the flows in ovs after neutron agent restarts.
> >
> >
> >
> >> Is Neutron ready to be upgradable with minimal downtime of
> >> services and no VM access downtime?
> >
> > As the ovs bug you refer to above, no, at least not in reference
> > implementation. That's for data plane.
> >
> > As for other services, neutron-server online schema migration
> > should help, and I hope to get it implemented in L (though there
> > are some obstacles that may block us; we'll see).
> >
> > As for live data migration, neutron code base is not ready yet to
> > reasonably require live data migration being implemented in all
> > patches that need data moves in database. The very first obstacle
> > for that is that there is no middleware layer between sqlalchemy
> > and the rest of neutron that would allow us to hide migration
> > details. Oslo.versionedobjects is such a middleware. See below.
> >
> >
> >> Are there any guidelines on using the oslo versionobjects and its
> >>  priority in Liberty cycle?
> >
> > It's not a common priority for L, but we've started on this road
> > inside feature/qos branch that will hopefully get into master some
> > time after L-2. If interested, see:
> >
> > http://git.openstack.org/cgit/openstack/neutron/tree/neutron/objects?h=f
> >
> >
> eature/qos
> >
> > Once the feature branch is merged into master, I plan to start
> > converting existing resources to objects. It may take time and
> > will definitely span to M. Depending on progress in this regard,
> > we'll see whether we will be able to consider live data migration.
> > At the moment, I don't see it happening in M, at least not at the
> > start of it.
> >
> >
> >> Are there use-cases written down when talking about Neutron
> >> upgrades?
> >
> > One thing that is currently in review are partial upgrades. They
> > are tested for nova (including nova-network) but not for neutron,
> > so it's considered a nova-network/neutron parity issue.
> >
> > You should find most of relevant patches in:
> >
> > https://review.openstack.org/#/q/owner:%22Russell+Bryant%22+status:open,
> >
> >
> n,z
>
> Yes, there's a ton of work to make rolling upgrades as robust as what
> Nova has done.  There's significant limitations to what Neutron can do
> without breaking it, but hopefully the grenade job would help us catch
> things that would break it sooner.  At the moment those jobs have been
> blocked pending some re-work of how the partial upgrade jobs work.
>
> As of the last release (Kilo), rolling upgrades from Juno with Neutron
> worked when verified manually.  I'm hoping we'll have the same for
> Liberty, but I'm not sure if anyone has tried it out recently.
>
> All of the things discussed here sound like good things to work on.  I
> actually thought a group was going to work on versioned objects for
> Kilo, but that never materialized.
>
> I'll also plug a project I'm working on (OVN, part of the Open vSwitch
> project), which I also think simplifies this a good bit for Neutron,
> because you won't have to run Neutron agents on compute nodes anymore.
>
> --
> Russell Bryant
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Kevin Benton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150713/0e1fa2ed/attachment.html>


More information about the OpenStack-dev mailing list