[openstack-dev] [TripleO] Should we have a TripleO API, or simply use Mistral?

Tzu-Mainn Chen tzumainn at redhat.com
Wed Jan 13 19:01:58 UTC 2016

----- Original Message -----
> On Wed, 2016-01-13 at 04:41 -0500, Tzu-Mainn Chen wrote:
> > Hey all,
> > 
> > I realize now from the title of the other TripleO/Mistral thread [1]
> > that
> > the discussion there may have gotten confused.  I think using Mistral
> > for
> > TripleO processes that are obviously workflows - stack deployment,
> > node
> > registration - makes perfect sense.  That thread is exploring
> > practicalities
> > for doing that, and I think that's great work.
> > 
> > What I inappropriately started to address in that thread was a
> > somewhat
> > orthogonal point that Dan asked in his original email, namely:
> > 
> > "what it might look like if we were to use Mistral as a replacement
> > for the
> > TripleO API entirely"
> > 
> > I'd like to create this thread to talk about that; more of a 'should
> > we'
> > than 'can we'.  And to do that, I want to indulge in a thought
> > exercise
> > stemming from an IRC discussion with Dan and others.  All, please
> > correct me
> > if I've misstated anything.
> > 
> > The IRC discussion revolved around one use case: deploying a Heat
> > stack
> > directly from a Swift container.  With an updated patch, the Heat CLI
> > can
> > support this functionality natively.  Then we don't need a TripleO
> > API; we
> > can use Mistral to access that functionality, and we're done, with no
> > need
> > for additional code within TripleO.  And, as I understand it, that's
> > the
> > true motivation for using Mistral instead of a TripleO API: avoiding
> > custom
> > code within TripleO.
> The True motivation for investigating Mistral was to counter the
> assertion that we needed to build our own REST API for workflows in the
> TripleO API spec. This:
>  "We need a REST API that supports the overcloud deployment workflow."
> https://review.openstack.org/#/c/230432/13/specs/mitaka/tripleo-overclo
> ud-deployment-api.rst
> In doing that I was trying to wrap some of the existing code in
> tripleo-common in Mistral actions and it occurred to me that some
> things (like the ability to deploy heat templates from Swift) might
> benefit others more if we put them in the respective client libraries
> (like heatclient) instead of carrying them in tripleo-common where they
> only benefit us. Especially since in the heatclient case it already had
> a --template-object option to begin with (https://bugs.launchpad.net/py
> thon-heatclient/+bug/1532326)
> This goes towards a similar pattern we follow with other components
> like Puppet which is that instead of trying to create our own
> functionality in puppet-tripleo we are trying to as much as possible
> add the features we need to the individual modules for each project
> (puppet-nova, puppet-swift, etc).
> In other words avoiding custom code within TripleO is just good
> practice in general, not something that is specific to the Mistral vs
> TripleO API discussion. When someone looks at TripleO as a project I
> would very much like them to admire our architecture... not what we've
> had to build to support it. As an example I'm very glad to see TripleO
> be a bit more friendly to config management tooling... rather than
> trying to build our own version (os-apply-config, etc.). And adding
> code to the right place usually works out better for everyone and can
> help build up the larger OpenStack community too.
> > 
> > That's definitely a worthy goal... except from my perspective, the
> > story
> > doesn't quite end there.  A GUI needs additional functionality, which
> > boils
> > down to: understanding the Heat deployment templates in order to
> > provide
> > options for a user; and persisting those options within a Heat
> > environment
> > file.
> TripleO API was previously described as what would be our "workflows
> API" for TripleO. A places for common workflows that are used for the
> CLI, and UI together (one code path for all!).
> What you describe here sounds like a different sort of idea which is a
> GUI helper sort of API. FWIW I think it is totally fine for a GUI to
> maintain its own API for caching reasons etc if the members of that
> team find it to be a requirement. Please don't feel that any of the
> workflow API comparisons block requirements for things to build a GUI
> though.
> > 
> > Right away I think we hit a problem.  Where does the code for
> > 'understanding
> > options' go?  Much of that understanding comes from the capabilities
> > map
> > in tripleo-heat-templates [2]; it would make sense to me that
> > responsibility
> > for that would fall to a TripleO library.
> > 
> > Still, perhaps we can limit the amount of TripleO code.  So to give
> > API
> > access to 'getDeploymentOptions', we can create a Mistral workflow.
> > 
> >   Retrieve Heat templates from Swift -> Parse capabilities map
> I have no issues with this workflow if it helps the UI team to create
> say a cached copy of the capabilities map.
> Similar to the above heatclient discussion I would ask why put this
> into a TripleO library if we don't have to. Could this code perhaps be
> made into a more generic Heat feature? If not immediately, then perhaps
> eventually? What I mean is... it would be totally fine to write this
> code in say TripleO common with an eye towards moving it eventually to
> someplace better like Heat API (proper) or heatclient perhaps.
> Regardless of where the code for parsing the capabilities map lives
> though I think it would be a fine use of a Mistral workflow.
> > 
> > Which is fine-ish, except from an architectural perspective
> > 'getDeploymentOptions' violates the abstraction layer between storage
> > and
> > business logic, a problem that is compounded because
> > 'getDeploymentOptions'
> > is not the only functionality that accesses the Heat templates and
> > needs
> > exposure through an API.  And, as has been discussed on a separate
> > TripleO
> > thread, we're not even sure Swift is sufficient for our needs; one
> > possible
> > consideration right now is allowing deployment from templates stored
> > in
> > multiple places, such as the file system or git.
> > 
> > Are we going to have duplicate 'getDeploymentOptions' workflows for
> > each
> > storage mechanism?  If we consolidate the storage code within a
> > TripleO
> > library, do we really need a *workflow* to call a single
> > function?  Is a
> > thin TripleO API that contains no additional business logic really so
> > bad
> > at that point?
> Do we really need our TripleO libaries to support multiple storage
> backends at all right now. As a cloud deployer I can store my heat
> templates wherever I like... git, svn, cvs... whatever. When I'm ready
> to deploy I would then use python-tripleoclient or the UI to "upload"
> my local copy of the templates into Swift.  At this point I could
> either use the UI or python-tripleoclient to drive the rest of the
> deployment to completion. The fact that the TripleO workflows use Swift
> for storage is actually abstracted from me such that I don't really
> care much.
> > 
> > My gut reaction is to say that proposing Mistral in place of a
> > TripleO API
> > is to look at the engineering concerns from the wrong direction.  The
> > Mistral alternative comes from a desire to limit custom TripleO code
> > at all
> > costs.  I think that is an extremely dangerous attitude that leads to
> > compromises and workarounds that will quickly lead to a shaky code
> > base
> > full of design flaws that make it difficult to implement or extend
> > any
> > functionality cleanly.
> Similar to what I said above limiting code in TripleO isn't necessarily
> the primary desire here I think. It is more of a question of can we use
> a generic workflow API like Mistral, or should we go and build our own.
> Whether more or less of the code lives in a TripleO common library is
> something we can debate and take a step at a time I think.
> > 
> > I think the correct attitude is to simply look at the problem we're
> > trying to solve and find the correct architecture.  For these get/set
> > methods that the API needs, it's pretty simple: storage -> some logic
> > ->
> > a REST API.  Adding a workflow engine on top of that is unneeded, and
> > I
> > believe that means it's an incorrect solution.
> Totally agree on finding the correct architecture. What you suggest as
> a "dangerous attitude" is actually just trying to ask the question that
> to me have not yet been asked. Trying out tools and solutions we may
> not have tried yet. (FWIW I found the dangerous attitude bit to be a
> bit un-constructive to this conversation and would prefer to leave that
> verbiage elsewhere)
> If you feel strongly that this particular case of parsing the
> capabilities map and managing config settings is not suitably
> implemented with a generic workflow tool then perhaps we implement both
> and see how it works out. Continue work on creating TripleO API...
> I would however like to perhaps see us scale back selling TripleO API
> as our generic workflow API. Perhaps there still is a niche where it
> helps with some things and I've got no issue with pursuing this idea. I
> will likely continue to ask the questions about putting code into the
> right place (like could some of this be implemented as a Heat API
> feature instead). Perhaps this just comes in time after we've refined
> our deployment workflows a bit to accomidate but UI and CLI.

That seems more than fair (and I apologize for the 'dangerous attitude'
comment, which I did not intend to be construed as an attack).  I think
I'll give the API spec one more polish and then re-submit, with the hope
that the discussion here brings us much closer to consensus as to the
path forward.  Thanks!


> Dan
> > 
> > 
> > Thanks,
> > Tzu-Mainn Chen
> > 
> > 
> > 
> > [1] http://lists.openstack.org/pipermail/openstack-dev/2016-January/0
> > 83757.html
> > [2] https://github.com/openstack/tripleo-heat-templates/blob/master/c
> > apabilities_map.yaml
> > 
> > _____________________________________________________________________
> > _____
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubs
> > cribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

More information about the OpenStack-dev mailing list