[openstack-dev] [TripleO] Should we have a TripleO API, or simply use Mistral?

Jay Dobies jason.dobies at redhat.com
Fri Jan 22 21:16:29 UTC 2016

I fall very much in the same mentality as Ben. I'm +1 to all of his 
points, with a few comments inline.

On 01/22/2016 12:24 PM, Ben Nemec wrote:
> So I haven't weighed in on this yet, in part because I was on vacation
> when it was first proposed and missed a lot of the initial discussion,
> and also because I wanted to take some time to order my thoughts on it.
>   Also because my initial reaction...was not conducive to calm and
> rational discussion. ;-)
> The tldr is that I don't like it.  To explain why, I'm going to make a
> list (everyone loves lists, right? Top $NUMBER reasons we should stop
> expecting other people to write our API for us):
> 1) We've been down this road before.  Except last time it was with Heat.
>   I'm being somewhat tongue-in-cheek here, but expecting a general
> service to provide us a user-friendly API for our specific use case just
> doesn't make sense to me.

I think it's important to think about outside integrations here. The 
current model is to tell other clients to manipulate Heat environments 
and understand how to parse/inspect templates*. Now it will be to 
understand/parse/manipulate Mistral workflows. Neither of those are 
conducive to the types of UI wireframes we've proposed in the past, much 
less friendly to completely outside integrators.

* I realize some of that inspection is moving into Heat, but it's still 
at the mechanical template level rather than providing insight into how 
to actually use them.

> 2) The TripleO API is not a workflow API.  I also largely missed this
> discussion, but the TripleO API is a _Deployment_ API.  In some cases
> there also happens to be a workflow going on behind the scenes, but
> honestly that's not something I want our users to have to care about.

I'm glad Ben mentioned this, because I always viewed the workflow 
aspects as a subset of what actually needs to be done.

> 3) It ties us 100% to a given implementation.  If Mistral proves to be a
> poor choice for some reason, or insufficient for a particular use case,
> we have no alternative.  If we have an API and decide to change our
> implementation, nobody has to know or care.  This is kind of the whole
> point of having an API - it shields users from all the nasty
> implementation details under the surface.

I strongly agree with this one. It's not even generic speculation; we've 
told people in the past to deal with Heat templates and now we're 
telling them to deal with workflows. We already have a history of the 
backend changing and an API would give us much more flexibility (and 
less annoyed users).

> 4) It raises the bar even further for both new deployers and developers.
>   You already need to have a pretty firm grasp of Puppet and Heat
> templates to understand how our stuff works, not to mention a decent
> understanding of quite a number of OpenStack services.
> This presents a big chicken and egg problem for people new to OpenStack.
>   It's great that we're based on OpenStack and that allows people to peek
> under the hood and do some tinkering, but it can't be required for
> everyone.  A lot of our deployers are going to have little to no
> OpenStack experience, and TripleO is already a daunting task for those
> people (hell, it's daunting for people who _are_ experienced).
> 5) What does reimplementing all of our tested, well-understood Python
> into a new YAML format gain us?  This is maybe the biggest thing I'm
> missing from this whole discussion.  We lose a bunch of things (ease of
> transition from other Python projects, excellent existing testing
> framework, etc.), but what are we actually gaining other than the
> ability to say that we use N + 1 OpenStack services?  Because we're way
> past the point where "It's OpenStack deploying OpenStack" is sufficient
> reason for people to pay attention to us.  We need less "Ooh, neat" and
> more "Ooh, that's easy to use and works well."  It's still not clear to
> me that Mistral helps in any way with the latter.
> 6) On the testing note, how do we test these workflows?  Do we know what
> happens when step X fails?  How do we test that they handle it properly
> in an automated and repeatable way?  In Python these are largely easy
> questions to answer: unit tests.  How do you unit test YAML?  This is a
> big reason I'm not even crazy about having Mistral on the back end of a
> TripleO API.  We'd be going from code that we can test and prove works
> in a variety of scenarios, to YAML that is tested and proven to work in
> exactly the three scenarios we run in CI.  This is basically the same
> situation we had with tripleo-incubator, and it was bad there too.
> I dunno.  Maybe I'm too late to this party to have any impact on the
> discussion, but I very much do not like the direction we're going and I
> would be remiss if I didn't at least point out my concerns with it.
> -Ben
> On 01/13/2016 03:41 AM, Tzu-Mainn Chen wrote:
>> Hey all,
>> I realize now from the title of the other TripleO/Mistral thread [1] that
>> the discussion there may have gotten confused.  I think using Mistral for
>> TripleO processes that are obviously workflows - stack deployment, node
>> registration - makes perfect sense.  That thread is exploring practicalities
>> for doing that, and I think that's great work.
>> What I inappropriately started to address in that thread was a somewhat
>> orthogonal point that Dan asked in his original email, namely:
>> "what it might look like if we were to use Mistral as a replacement for the
>> TripleO API entirely"
>> I'd like to create this thread to talk about that; more of a 'should we'
>> than 'can we'.  And to do that, I want to indulge in a thought exercise
>> stemming from an IRC discussion with Dan and others.  All, please correct me
>> if I've misstated anything.
>> The IRC discussion revolved around one use case: deploying a Heat stack
>> directly from a Swift container.  With an updated patch, the Heat CLI can
>> support this functionality natively.  Then we don't need a TripleO API; we
>> can use Mistral to access that functionality, and we're done, with no need
>> for additional code within TripleO.  And, as I understand it, that's the
>> true motivation for using Mistral instead of a TripleO API: avoiding custom
>> code within TripleO.
>> That's definitely a worthy goal... except from my perspective, the story
>> doesn't quite end there.  A GUI needs additional functionality, which boils
>> down to: understanding the Heat deployment templates in order to provide
>> options for a user; and persisting those options within a Heat environment
>> file.
>> Right away I think we hit a problem.  Where does the code for 'understanding
>> options' go?  Much of that understanding comes from the capabilities map
>> in tripleo-heat-templates [2]; it would make sense to me that responsibility
>> for that would fall to a TripleO library.
>> Still, perhaps we can limit the amount of TripleO code.  So to give API
>> access to 'getDeploymentOptions', we can create a Mistral workflow.
>>    Retrieve Heat templates from Swift -> Parse capabilities map
>> Which is fine-ish, except from an architectural perspective
>> 'getDeploymentOptions' violates the abstraction layer between storage and
>> business logic, a problem that is compounded because 'getDeploymentOptions'
>> is not the only functionality that accesses the Heat templates and needs
>> exposure through an API.  And, as has been discussed on a separate TripleO
>> thread, we're not even sure Swift is sufficient for our needs; one possible
>> consideration right now is allowing deployment from templates stored in
>> multiple places, such as the file system or git.
>> Are we going to have duplicate 'getDeploymentOptions' workflows for each
>> storage mechanism?  If we consolidate the storage code within a TripleO
>> library, do we really need a *workflow* to call a single function?  Is a
>> thin TripleO API that contains no additional business logic really so bad
>> at that point?
>> My gut reaction is to say that proposing Mistral in place of a TripleO API
>> is to look at the engineering concerns from the wrong direction.  The
>> Mistral alternative comes from a desire to limit custom TripleO code at all
>> costs.  I think that is an extremely dangerous attitude that leads to
>> compromises and workarounds that will quickly lead to a shaky code base
>> full of design flaws that make it difficult to implement or extend any
>> functionality cleanly.
>> I think the correct attitude is to simply look at the problem we're
>> trying to solve and find the correct architecture.  For these get/set
>> methods that the API needs, it's pretty simple: storage -> some logic ->
>> a REST API.  Adding a workflow engine on top of that is unneeded, and I
>> believe that means it's an incorrect solution.
>> Thanks,
>> Tzu-Mainn Chen
>> [1] http://lists.openstack.org/pipermail/openstack-dev/2016-January/083757.html
>> [2] https://github.com/openstack/tripleo-heat-templates/blob/master/capabilities_map.yaml
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

More information about the OpenStack-dev mailing list