[openstack-dev] [nova] Order of n-api (placement) and n-sch upgrades for Ocata

Eoghan Glynn eglynn at redhat.com
Fri Jan 20 10:53:31 UTC 2017



> >> What are these issues? My original message was to highlight one
> >> particular deployment type which is completely independent of
> >> how things get packaged in the traditional sense of the word
> >> (rpms/deb/tar.gz).  Perhaps it's getting lost in terminology,
> >> but packaging the software in one way and how it's run can be two
> >> separate issues.  So what I'd like to know is how is that
> >> impacted by whatever ordering is necessary, and if there's anyway
> >> way not to explicitly have special cases that need to be handled
> >> by the end user when applying updates.  It seems like we all want
> >> similar things. I would like not to have to do anything different
> >> from the install for upgrade. Why can't apply configs, restart
> >> all services?  Or can I?  I seem to be getting mixed messages...
> >> 
> >> 
> > 
> > Sorry for being unclear on the issue. As Jay pointed out, if
> > nova-scheduler is upgraded before the placement service, the
> > nova-scheduler service will continue to start and take requests.
> > The problem is if the filter scheduler code is requesting a
> > microversion in the placement API which isn't available yet, in
> > particular this 1.4 microversion, then scheduling requests will
> > fail which to the end user means NoValidHost (the same as if we
> > don't have any compute nodes yet, or available).
> > 
> > So as Jay also pointed out, if placement and n-sch are upgraded
> > and restarted at the same time, the window for hitting this is
> > minimal. If deployment tooling is written to make sure to restart
> > the placement service *before* nova-scheduler, then there should be
> > no window for issues.
> > 
> 
> 
> Thanks all for providing insights. I'm trying to see a consensus here,
> and while I understand the concerns from Alex about the upgrade, I
> think it's okay for a deployer having a "controller" node (disclaimer:
> Nova doesn't have this concept, rather a list of components that are
> not compute nodes) to have a very quick downtime (I mean getting
> NoValidHosts if an user asks for an instance while the "controller" is
> upgraded).

Do we also need to be concerned about the placement API "warm-up" time?

i.e. if a placement-less newton deployment is upgraded to placement-ful
ocata, then would there surely be a short period during which placement
is able to respond to the incoming queries from the scheduler, but only
with incomplete information since all the computes haven't yet triggered
their first reporting cycle?

In that case, it wouldn't necessarily lead to a NoValidHost failure on a
instance boot request, but rather a potentially faulty placement decision,
being based on incomplete information. I mean "faulty" there in the sense
of not strictly following the configured scheduling strategy.

Is that a concern, or an acceptable short degradation of service?

Cheers,
Eoghan

> To be honest, Nova has never supported (yet) having rolling upgrades
> for services that are not computes. If you look at the upgrade devref,
> we ask for a maintenance window [1]. During that maintenance window,
> we say it's safer to upgrade "nova-conductor first and nova-api last"
> for coherence reasons but since that's during the maintenance window,
> we're not supposed to have user requests coming in.
> 
> So, to circle back with the original problem, I think having the
> nova-scheduler upgraded *before* placement is not a problem. If
> deployers don't want to implement an upgrade scenario where placement
> is upgraded before scheduler, that's fine. No need of extra work for
> deployers. That's just that *if* you implement that path, the
> scheduler could still get requests.
> 
> -Sylvain
> 
> [1]
> http://docs.openstack.org/developer/nova/upgrade.html#rolling-upgrade-process
> 
> 
> > --
> > 
> > Thanks,
> > 
> > Matt
> > 
> > __________________________________________________________________________
> >
> > 
> OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 



More information about the OpenStack-dev mailing list