Open Stack

On Mon, 2016-01-25 at 06:55 -0800, Clint Byrum wrote:
> Excerpts from Jiri Tomasek's message of 2016-01-25 06:09:33 -0800:
> > On 01/25/2016 12:42 AM, Clint Byrum wrote:
> > > Excerpts from Dan Prince's message of 2016-01-22 16:19:07 -0800:
> > > > On Fri, 2016-01-22 at 11:24 -0600, Ben Nemec wrote:
> > > > > So I haven't weighed in on this yet, in part because I was on
> > > > > vacation
> > > > > when it was first proposed and missed a lot of the initial
> > > > > discussion,
> > > > > and also because I wanted to take some time to order my
> > > > > thoughts on
> > > > > it.
> > > > >   Also because my initial reaction...was not conducive to
> > > > > calm and
> > > > > rational discussion. ;-)
> > > > > 
> > > > > The tldr is that I don't like it.  To explain why, I'm going
> > > > > to make
> > > > > a
> > > > > list (everyone loves lists, right? Top $NUMBER reasons we
> > > > > should stop
> > > > > expecting other people to write our API for us):
> > > > > 
> > > > > 1) We've been down this road before.  Except last time it was
> > > > > with
> > > > > Heat.
> > > > >   I'm being somewhat tongue-in-cheek here, but expecting a
> > > > > general
> > > > > service to provide us a user-friendly API for our specific
> > > > > use case
> > > > > just
> > > > > doesn't make sense to me.
> > > > We've been down this road with Heat yes. But we are currently
> > > > using
> > > > Heat for some things that we arguable should be (a workflows
> > > > tool might
> > > > help offload some stuff out of Heat). Also we haven't
> > > > implemented
> > > > custom Heat resources for TripleO either. There are mixed
> > > > opinions on
> > > > this but plugging in your code to a generic API is quite nice
> > > > sometimes.
> > > > 
> > > > That is the beauty of Mistral I think. Unlike Heat it actually
> > > > encourages you to customize it with custom Python actions.
> > > > Anything we
> > > > want in tripleo-common can become our own Mistral action (these
> > > > get
> > > > registered with stevedore entry points so we'd own the code)
> > > > and the
> > > > YAML workflows just tie them together via tasks.
> > > > 
> > > Since TripleO's undercloud is a special, single-purpose cloud
> > > just for
> > > deploying OpenStack, I think that is a perfectly fine idea. But
> > > it also
> > > somewhat invalidates the reasons to use Mistral. If you can
> > > control all
> > > the pieces, IMO you may want to use the best tool for that job,
> > > not the
> > > one that is the most OpenStack. The virtuous circle only works if
> > > you're
> > > actually improving the OpenStack components for their general use
> > > cases.
> > 
> > Maybe it is just me but I'd like to point out that we'd not be 
> > changing/updating Mistral API to serve our purpose. We'd just write
> > our 
> > own custom Mistral Actions and use Mistral API to organize them in 
> > workflows and run them.
> > 
> 
> If you're doing it with a stevedore entry point, you're making a
> custom
> piece of python which any other Mistral service of the same version
> would
> not have.
> 
> > Mistral seems to be best _existent_ and _usable_ tool for the job
> > which 
> > is IMO very important point.
> > 
> 
> Agreed, that's a very big +1. However, as in poker, before pushing in
> all the chips, be sure the player next to you is actually beat, and
> not
> just playing it slow.
> 
> > > 
> > > > We don't have to go off and build our own proxy deployment
> > > > workflow
> > > > API. The structure to do just about anything we need already
> > > > exists so
> > > > why not go and use it?
> > > > 
> > > > > 2) The TripleO API is not a workflow API.  I also largely
> > > > > missed this
> > > > > discussion, but the TripleO API is a _Deployment_ API.  In
> > > > > some cases
> > > > > there also happens to be a workflow going on behind the
> > > > > scenes, but
> > > > > honestly that's not something I want our users to have to
> > > > > care about.
> > > > Agree that users don't have to care about this.
> > > > 
> > > > Users can get as involved as they want here. Most users I think
> > > > will
> > > > use python-tripleoclient to drive the deployment or the new UI.
> > > > They
> > > > don't have to interact with Mistral directly unless they really
> > > > want
> > > > to. So whether we choose to build our own API or use a generic
> > > > one I
> > > > think this point is mute.
> > > > 
> > > > > 3) It ties us 100% to a given implementation.  If Mistral
> > > > > proves to
> > > > > be a
> > > > > poor choice for some reason, or insufficient for a particular
> > > > > use
> > > > > case,
> > > > > we have no alternative.  If we have an API and decide to
> > > > > change our
> > > > > implementation, nobody has to know or care.  This is kind of
> > > > > the
> > > > > whole
> > > > > point of having an API - it shields users from all the nasty
> > > > > implementation details under the surface.
> > > > 
> > > > Mistal's API is a generic workflow API. It is very much the
> > > > same layer
> > > > that I think we would get if we were to integrate with
> > > > something like
> > > > Ansible Tower... except that Mistral is part of OpenStack. It
> > > > integrates very nicely with OpenStack services and is very
> > > > customizable
> > > > with custom actions. The fact that Mistral sits much closer to
> > > > OpenStack and is essentially a light shim on top of it is to
> > > > our
> > > > advantage (being TripleO). To think that we can build up a
> > > > proxy API in
> > > > such a manner that we might be able to swap in an entirely new
> > > > backend
> > > > (without even having a fully implement backend yet to begin
> > > > with) is
> > > > for me a bit of a stretch. We've got a lot of "TripleO API"
> > > > maturing
> > > > before we'll get to this point. Which is why I lean towards
> > > > using a
> > > > generic workflow API to accomplis the same task.
> > > > 
> > > Ansible 2.0 integrates extremely well with OpenStack too. Even
> > > better,
> > > you don't need an inside-the-cloud service to use it to interact
> > > with OpenStack. Now, sometimes you want an "aaS" helping you, and
> > > as
> > > you've alluded to, Tower is sort of like that. As is Mistral for
> > > its
> > > language. I've suggested before that Mistral would be well served
> > > by
> > > adopting Ansible as a language for workflow, since the concepts
> > > they're
> > > encoding are very similar. In fact, learning Mistral's DSL feels
> > > like
> > > learning Spanish after already having studied Latin. The
> > > structure is
> > > the same, the meanings are the same. It's just different words.
> > 
> > So is it possible to use Ansible (Ansible Tower) instead of
> > Mistral 
> > _right now_?
> > 
> 
> AFAIK, Tower hasn't been open sourced, so no. However, what I'm
> suggesting
> is that it is more important to align with what deployers want, than
> what OpenStack needs.
> 
> > > 
> > > > I actually think rather than shielding users we should be more
> > > > transparent about the actual workflows that are driving
> > > > deployment.
> > > > Smaller more focused workflows that we string together to drive
> > > > the
> > > > deployment.
> > > > 
> > > I tend to agree, especially when your workflows may need to be
> > > customized.
> > 
> > Customizing underlining deployment workflows is IMO potentially
> > very 
> > dangerous regarding updates etc. and should be avoided as much as
> > possible.
> > 
> 
> Speaking from experience and interviewing potential users of TripleO,
> People have requirements to install on homogeneous racks of gear,
> incorporate their own security frameworks, and generally DON'T want a
> black-box product at anything but the smallest scale. Of course, we
> all
> want them to bolt things on where we made a space for that, but
> assuming
> that only the crazy users will do something like add actions to a
> workflow is only going to confuse you _and_ the users.
> 
> > > 
> > > > > 4) It raises the bar even further for both new deployers and
> > > > > developers.
> > > > >   You already need to have a pretty firm grasp of Puppet and
> > > > > Heat
> > > > > templates to understand how our stuff works, not to mention a
> > > > > decent
> > > > > understanding of quite a number of OpenStack services.
> > > > > 
> > > > > This presents a big chicken and egg problem for people new to
> > > > > OpenStack.
> > > > >   It's great that we're based on OpenStack and that allows
> > > > > people to
> > > > > peek
> > > > > under the hood and do some tinkering, but it can't be
> > > > > required for
> > > > > everyone.  A lot of our deployers are going to have little to
> > > > > no
> > > > > OpenStack experience, and TripleO is already a daunting task
> > > > > for
> > > > > those
> > > > > people (hell, it's daunting for people who _are_
> > > > > experienced).
> > > > > 
> > > > And on the flipside you will get more of a community around
> > > > using an
> > > > OpenStack project than you ever would going off and building
> > > > your own
> > > > "Deployment/Workflow API".
> > > > 
> > > I actually think you'll get an even bigger community if you use a
> > > generic deployment workflow tool than if you use an openstack
> > > specific
> > > workflow tool. So IMO, the end-goal should be an API which drives
> > > Ansible workflows. Mistral would likely be able to do this with
> > > some
> > > refactoring around multiple languages.
> > > 
> > > > I would actually argue this is less of a deployers thing and
> > > > more of a
> > > > development tool choice. IMO most deployers will use python-
> > > > tripleoclient or some UI and not mistralclient directly. The
> > > > code I've
> > > > posted this week shows a prototype of just this, Mistral is
> > > > swapped in
> > > > such that you would never know it was involved because python-
> > > > tripleoclient works like it always did. Deployers use our CLI
> > > > and UI
> > > > tools like they always have, and developers gain a community of
> > > > Mistral
> > > > developers (and documentation) which they can interact with on
> > > > common
> > > > problems. Sounds like a win/win to me.
> > > > 
> > > > > 5) What does reimplementing all of our tested, well-
> > > > > understood Python
> > > > > into a new YAML format gain us?  This is maybe the biggest
> > > > > thing I'm
> > > > > missing from this whole discussion.  We lose a bunch of
> > > > > things (ease
> > > > > of
> > > > > transition from other Python projects, excellent existing
> > > > > testing
> > > > > framework, etc.), but what are we actually gaining other than
> > > > > the
> > > > > ability to say that we use N + 1 OpenStack services?  Because
> > > > > we're
> > > > > way
> > > > > past the point where "It's OpenStack deploying OpenStack" is
> > > > > sufficient
> > > > > reason for people to pay attention to us.  We need less "Ooh,
> > > > > neat"
> > > > > and
> > > > > more "Ooh, that's easy to use and works well."  It's still
> > > > > not clear
> > > > > to
> > > > > me that Mistral helps in any way with the latter.
> > > > Nobody suggested we reimplement everything. Much of the plan to
> > > > move
> > > > code into tripleo-common would stay. Instead of building our
> > > > own API
> > > > we'd just skip all that and focus on the code that is actually
> > > > about
> > > > our deployments in the form of custom Mistral actions and YAML
> > > > workflows.
> > > > 
> > > > The YAML workflows just ties together actions which are
> > > > actually all
> > > > written in Python. YAML works quite well for this and is a
> > > > whole lot
> > > > less verbose than writting everything we have in Python. There
> > > > is a
> > > > reason Heat, Ansible, and Mistral use YAML for these things...
> > > > and I
> > > > think it works well. Understood you have an opinion on this,
> > > > but I
> > > > don't share the view that everything works better when written
> > > > in
> > > > Python. Take Puppet for example, we interface with that via
> > > > Hiera.
> > > > 
> > > > People will pay attention because we'll be able to add features
> > > > faster.
> > > > By not having to build our own API and plumbing we can focus on
> > > > actual
> > > > problems rather than boilerplate Python API code.
> > > > 
> > > I totally agree with you Dan, and I'd only add that if you can
> > > make
> > > it work with Ansible you gain a huge amount of potential
> > > development
> > > capability in the greater Ansible community.
> > 
> > So is it possible to use Ansible instead of Mistral right now? If I
> > am 
> > not mistaken, there has been a discussion whether to use Mistral
> > some 
> > time back and it was pushed away because it's API was not finished.
> > Now 
> > it is. Same seems to happen with Ansible now. As you describe, the 
> > Mistral and Ansible are very similar just the DSL is different, why
> > not 
> > implement the actions and use Mistral which is available now and
> > then 
> > consider migrating to Ansible when it is ready, which by your 
> > description should not be difficult task.
> > 
> > This also relates to the question of shielding these 'technology 
> > changes' behind a TripleO API about which I am not very much
> > convinced 
> > is worth the effort because TripleO API itself would be unstable
> > anyway 
> > and would bring another 'unnecessary' layer to the project.
> > 
> 
> Perhaps I lack the proper context to make this judgement, but isn't
> this already a large effort to improve development velocity and adopt
> a tool that users can be comfortable modifying? I'm suggesting that
> TripleO should consider the ramifications of adopting another
> OpenStack
> specific language in the deployment tooling before doing it. If there
> is
> an urgency attached to this, then that of course plays into the
> decision.
> And also, to be clear, I _do_ think an in-cloud workflow tool should
> be
> the answer. I also think that the current in-cloud workflow tool with
> a
> strong community presence in OpenStack should be enhanced to just be
> a
> frontend for Ansible. That would be a _huge_ boon to adoption of
> Mistral,
> since now you have a language operators are adopting in droves,
> without
> the requirement that it be hosted.

Interesting angle. I've also consider the what-if Mistral was an front
end to run Ansible playbooks and I quite like the idea of it perhaps
being a more OpenStack integrated solution for this tool. I'm not sure
if that fits into Mistral's vision but it could perhaps be worth
exploring... :)

The important part here to me is that this mechanism might be generally
useful for everyone, not just TripleO. We could build a community
around it as possibly a generally useful way to drive Ansible workflows
in a more OpenStack centric way. So again, like TripleO's use of
OpenStack services in other places you got potential for a feedback
look of community goodness to take off and flourish.

The Ansible integration doesn't exist yet.. but Mistral is already
quite capable where it is. And the more I've used it the more I've
liked the idea of it becoming a central part of the TripleO tooling. So
as is, today I'm advocating that we make use of it as an alternative to
our own Deployment/workflow REST API. I can't tell you what the future
holds here other than it doesn't feel like we need to go and build our
own API for these things yet.

With TripleO API I think many people are trying to envision a grand
Deployment API that would do a similar sort of things. Perhaps you
could have a Mistral/Heat backend. Or an Ansible backend, etc. The
problem with this API is it is totally focused on the prescibed way
TripleO does things... and I think outside of the TripleO community
we'd have a hard time getting anyone to consider using it for anything
else. So in this regard I'm taking the stance that TripleO should have
less code that is specific to our take on deployment, not more. If
TripleO is to be a success < code helps us I think, not more.

Perhaps we will eventually need to build an API for some things. Ironic
inspector comes to mind here. That got built and could have easily been
done under the "TripleO" umbrella I think, although it definitely feels
better under Ironic. By putting this API in the "right" (Ironic)
project we gain a larger community, all the while still getting to use
the features just the same within TripleO. So if we do need an API for
something lets just make sure it is for the right reason (and under the
right project).

Dan


> 
> _____________________________________________________________________
> _____
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubs
> cribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev