Open Stack

On 02/02/15 19:52, Steve Baker wrote:
> A spec has been raised to add a config option to allow operators to
> choose whether to use the new convergence engine for stack operations.
> For some context you should read the spec first [1]
>
> Rather than doing this, I would like to propose the following:

I am strongly, strongly opposed to making this part of the API.

> * Users can (optionally) choose which engine to use by specifying an
> engine parameter on stack-create (choice of classic or convergence)
> * Operators can set a config option which determines which engine to use
> if the user makes no explicit choice
> * Heat developers will set the default config option from classic to
> convergence when convergence is deemed sufficiently mature

We'd also need a way for operators to prevent users from enabling 
convergence if they're not ready to support it.

> I realize it is not ideal to expose this kind of internal implementation
> detail to the user, but choosing convergence _will_ result in different
> stack behaviour (such as multiple concurrent update operations) so there
> is an argument for giving the user the choice. Given enough supporting
> documentation they can choose whether convergence might be worth trying
> for a given stack (for example, a large stack which receives frequent
> updates)

It's supposed to be a strict improvement; we don't need to ask 
permission. We have made major changes of this type in practically every 
Heat release. When we switched from creating resources serially to 
creating them in parallel in Havana we didn't ask permission. We just 
did it. We when started allowing users to recover from a failed 
operation in Juno we didn't ask permission. We just did it. We don't 
need to ask permission to allow concurrent updates. We can just do it.

The only difference here is that we are being a bit smarter and 
uncoupling our development schedule from the release cycle. There are 15 
other blueprints, essentially all of which have to be complete before 
convergence is usable at all. It won't do *anything at all* until we are 
at least 12 blueprints in. The config option buys us time to land them 
without the risk of something half-finished appearing in the release 
(trunk-chasers will also thank us). It has no other legitimate purpose IMO.

The goal is IN NO WAY to maintain separate code paths in the long term. 
The config option is simply a development strategy to allow us to land 
code without screwing up a release and while maintaining as much test 
coverage as possible.

> Operators likely won't feel they have enough knowledge to make the call
> that a heat install should be switched to using all convergence, and
> users will never be able to try it until the operators do (or the
> default switches).

Hardly anyone should have to make a call. We should flip the default as 
soon as all of the blueprints have landed (i.e. as soon as it works at 
all), provided that a release is not imminent. (Realistically, at this 
point I think we have to say the target is to do it as early as in 
Lizard as we can.) That means for those chasing trunk they get it as 
soon as it works at all, and for those using stable releases they get it 
at the next release, just like every other feature we have ever added.

As a bonus, trunk-chasing operators who need to can temporarily delay 
enabling of convergence until a point of their choosing in the release 
cycle by overriding the default. Anybody in that position likely has 
enough knowledge to make the right call for them.

So I believe that all of our stakeholders are catered to by the config 
option: operators & users who want a stable, tested release; 
operator/users who want to experiment on the bleeding edge; and 
operators who chase trunk but whose users require stability.

The only group that benefits from enshrining the choice in the API - 
users who want to experiment with the bleeding edge, but who don't 
control their own OpenStack deployment - doesn't actually exist, and if 
it did then this would be among the least of their problems.

> Finally, there are also some benefits to heat developers. Creating a
> whole new gate job to test convergence-enabled heat will consume its
> share of CI resource. I'm hoping to make it possible for some of our
> functional tests to run against a number of scenarios/environments.
> Being able to run tests under classic and convergence scenarios in one
> test run will be a great help (for performance profiling too).

I think this is the strongest argument in favour. However, I'd like to 
think it would be possible to run the functional tests twice in the 
gate, changing the config and restarting the engine in between.

But if the worst comes to the worst, then although I think it's 
preferable to use one VM for twice as long vs. two VMs for the same 
length of time, I don't think the impact on resource utilisation in the 
gate of choosing one over the other is likely to be huge. And I don't 
see this situation persisting for a long time. The purpose of running 
both sets of tests is to buy us time to write a migration tool without 
having to delay flipping the config switch until it is ready. So we'd 
likely only have to continue running the legacy tests for one release cycle.

> If there is enough agreement then I'm fine with taking over and updating
> the convergence-config-option spec.

Be my guest, but I'd certainly vote -1.

Thanks for starting this discussion though. After this thread it is 
clear to me that I wasn't nearly specific enough in the spec about when 
we should flip the default, so I updated the wording to match better 
what I said above.

cheers,
Zane.

> [1]
> https://review.openstack.org/#/c/152301/2/specs/kilo/convergence-config-option.rst
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev