Open Stack

On Fri, 2016-08-05 at 13:56 +0200, Dmitry Tantsur wrote:
> On 08/05/2016 01:21 PM, Steven Hardy wrote:
> > 
> > On Fri, Aug 05, 2016 at 12:27:40PM +0200, Dmitry Tantsur wrote:
> > > 
> > > On 08/04/2016 11:48 PM, Dan Prince wrote:
> > > > 
> > > > Last week I started some prototype work on what could be a new
> > > > way to
> > > > install the Undercloud. The driving force behind this was some
> > > > of the
> > > > recent "composable services" work we've done in TripleO so
> > > > initially I
> > > > called in composable undercloud. There is an etherpad here with
> > > > links
> > > > to some of the patches already posted upstream (many of which
> > > > stand as
> > > > general imporovements on their own outside the scope of what
> > > > I'm
> > > > talking about here).
> > > > 
> > > > https://etherpad.openstack.org/p/tripleo-composable-undercloud
> > > > 
> > > > The idea in short is that we could spin up a small single
> > > > process all-
> > > > in-one heat-all (engine and API) and thereby avoid things like
> > > > Rabbit,
> > > > and MySQL. Then we can use Heat templates to drive the
> > > > Undercloud
> > > > deployment just like we do in the Overcloud.
> > > I don't want to sound rude, but please no. The fact that you have
> > > a hammer
> > > does not mean everything around is nails :( What problem are you
> > > trying to
> > > solve by doing it?
> > I think Dan explains it pretty well in his video, and your comment
> > indicates a fundamental misunderstanding around the entire TripleO
> > vision,
> > which is about symmetry and reuse between deployment tooling and
> > the
> > deployed cloud.
> Well, except for you need some non-openstack starting point, because 
> unlike with e.g. ansible installing any openstack service(s) does
> not 
> end at "dnf install <package-name>".
> 
> > 
> > 
> > The problems this would solve are several:
> > 
> > 1. Remove divergence between undercloud and overcloud puppet
> > implementation
> > (instead of having an undercloud specific manifest, we reuse the
> > *exact*
> > same stuff we use for overcloud deployments)
> I'm not against reusing puppet bits, I'm against building the same
> heavy 
> abstraction layer with heat around it.

What do you mean by heavy exactly. The entire point here was to
demonstrate that this can work and *is* actually quite lightweight I
think.

We are already building an abstraction layer. So why not just use it in
2 places instead of one.

> 
> > 
> > 
> > 2. Better modularity, far easier to enable/disable services
> Why? Do you expect enabling/disabling Nova, for example? In this
> regard 
> undercloud is fundamentally different from overcloud: for the former
> we 
> have a list of required services and a pretty light list of optional 
> services.

I think this is a very narrow view of the Undercloud and ignores the
fact that continually adding booleans to enable or disable features is
not scalable. Using the same composability and deployment framework we
have developed for the Overcloud might make better sense to me.

There is also real potential here to re-use this as a means to install
other package based types of setups. An "anything is an undercloud"
sort of approach could be the next logic step... all of this for free
because we are building abstractions to install these things in the
Overcloud as well.

> 
> > 
> > 
> > 3. Get container integration "for free" when we land it in the
> > overcloud
> > 
> > 4. Any introspection and debugging workflow becomes identical
> > between the
> > undercloud and overcloud
> I would love a defined debugging workflow for the overcloud first..

The nice thing about demo I showed for debugging is all the output
comes back to the console. Heat, os-collect-config, puppet, etc. all
there at your fingertips. Set 'debug=True' and you have everything you
need I think.

After building it I've quite enjoyed how fast it is to test and debug
creating a prototype undercloud.yaml.

> 
> > 
> > 
> > 5. We remove dependencies on a bunch of legacy scripts which run
> > outside of
> > puppet
> If you mean instack-undercloud element, we're getting rid of them 
> anyway, no?

We mean all of the elements. Besides a few bootstrapping things we have
gradually moved towards using Heat hooks to run things as opposed to
the traditional os-apply-config/os-refresh-config hooks. This provides
better signalling back to heat and arguably makes debugging much easier
when something fails too.

> 
> > 
> > 
> > 6. Whenever someone lands support for a new service in the
> > overcloud, we
> > automatically get undercloud support for it, completely for free.
> Again, why? A service won't integrate itself into the deployment. And
> to 
> be honest, the amount of options TripleO has already cases real
> world 
> problems. I would rather see a well defined set of functionality for
> it..
> 
> > 
> > 
> > 7. Potential for much easier implementation of a multi-node
> > undercloud
> Ideally, I would love to see:
> 
>   for node in nodes:
>     ssh $node puppet apply blah-blah
> 
> Maybe we're not there, but it only means we have to improve our
> puppet 
> modules.
> 
> > 
> > 
> > > 
> > > Undercloud installation is already sometimes fragile, but it's
> > > probably the
> > > least fragile part right now (at least from my experience) And at
> > > the very
> > > least it's pretty obviously debuggable in most cases. THT is hard
> > > to
> > > understand and often impossible to debug. I'd prefer we move away
> > > from THT
> > > completely rather than trying to fix it in one more place where
> > > heat does
> > > not fit..
> > These are some strong but unqualified assertions, so it's hard to
> > really
> > respond.
> We'll talk about "unqualified" assertions the next time I'll try to
> get 
> answers on #tripleo after seeing error messages like
> "controller_step42 
> failed with code 1" ;)
> 
> > 
> > Yes, there is complexity, but it's a set of abstractions which
> > actually work pretty well for us, so there is value in having just
> > one set
> > of abstractions used everywhere vs special-casing the undercloud.
> There should be a point where we stop. What entity is going to
> install 
> heat to install undercloud (did I just say "seed")? What will provide
> HA 
> for it? Authentication, templates storage and versioning? How do you 
> reuse the same abstractions (that's the whole point after all)?
> 
> > 
> > 
> > Re moving away from THT completely, this is not a useful statement
> > -
> > yes, there are alternative tools, but if you were to remove THT and
> > just
> > use some other tool with Ironic, the result would simply not be
> > TripleO.
> > There would be zero migration/upgrade path for existing users and
> > all
> > third-party integrations (and our API/UI) would break.
> I don't agree it would not be TripleO. OpenStack does not end on
> heat 
> templates, some deployments don't even use heat.
> 
> > 
> > 
> > FWIW I think this prototyping work is very interesting, and I'm
> > certainly
> > keen to get wider (more constructive) feedback and see where it
> > leads.
> > 
> > Thanks,
> > 
> > Steve
> > 
> > ___________________________________________________________________
> > _______
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsu
> > bscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> 
> _____________________________________________________________________
> _____
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubs
> cribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev