[openstack-dev] [TripleO] Should we have a TripleO API, or simply use Mistral?
openstack at nemebean.com
Mon Jan 25 21:49:19 UTC 2016
On 01/25/2016 05:56 AM, Dougal Matthews wrote:
> Great post Ben, I largely agree with what you are saying, a lot of your
> are valid concerns that I share. Just a couple of comments inline as I
> try to
> avoid what others have said.
> On 22 January 2016 at 17:24, Ben Nemec <openstack at nemebean.com
> <mailto:openstack at nemebean.com>> wrote:
> So I haven't weighed in on this yet, in part because I was on vacation
> when it was first proposed and missed a lot of the initial discussion,
> and also because I wanted to take some time to order my thoughts on it.
> Also because my initial reaction...was not conducive to calm and
> rational discussion. ;-)
> The tldr is that I don't like it. To explain why, I'm going to make a
> list (everyone loves lists, right? Top $NUMBER reasons we should stop
> expecting other people to write our API for us):
> 1) We've been down this road before. Except last time it was with Heat.
> I'm being somewhat tongue-in-cheek here, but expecting a general
> service to provide us a user-friendly API for our specific use case just
> doesn't make sense to me.
> I don't really expect the API to be used directly by many users - how many
> users directly used the Tuskar API? I suspect they all used it via the CLI
> tools or the GUI. I could be completely wrong with the assumption but I
> the primary entry points will be the CLI and GUI we expose. Only advanced
> users would need to consider Mistral.
I touched on this in my reply to Dan as well, but I don't believe this
is true. The CLI needs to be a very thin wrapper around whatever REST
API we're calling. Basically it should be taking params from the
command-line and passing them through to $API. If it does _any_
processing of them, then that's something the GUI has to duplicate
(assuming it can) and we have an architectural problem.
That being said, it does appear as though I misunderstood the initial
proposal. I thought we were talking about replacing openstack overcloud
* (and friends) commands with mistral calls, but it appears the proposal
was to simply pull certain pieces of the commands out and replace them
with Mistral (which as I noted in my other reply doesn't actually solve
the problem the TripleO API was trying to solve), so it's possible some
of the things I was arguing against weren't actually being proposed. :-)
> Anyone directly interfacing with the API's will need to learn a ton of
> I don't think Mistral adds much more complexity here.
> 2) The TripleO API is not a workflow API. I also largely missed this
> discussion, but the TripleO API is a _Deployment_ API. In some cases
> there also happens to be a workflow going on behind the scenes, but
> honestly that's not something I want our users to have to care about.
> Yeah, I agree that one of the downsides of the Mistral approach is that we
> blur the lines between these two.
> 3) It ties us 100% to a given implementation. If Mistral proves to be a
> poor choice for some reason, or insufficient for a particular use case,
> we have no alternative. If we have an API and decide to change our
> implementation, nobody has to know or care. This is kind of the whole
> point of having an API - it shields users from all the nasty
> implementation details under the surface.
> 4) It raises the bar even further for both new deployers and developers.
> You already need to have a pretty firm grasp of Puppet and Heat
> templates to understand how our stuff works, not to mention a decent
> understanding of quite a number of OpenStack services.
> This presents a big chicken and egg problem for people new to OpenStack.
> It's great that we're based on OpenStack and that allows people to peek
> under the hood and do some tinkering, but it can't be required for
> everyone. A lot of our deployers are going to have little to no
> OpenStack experience, and TripleO is already a daunting task for those
> people (hell, it's daunting for people who _are_ experienced).
> 5) What does reimplementing all of our tested, well-understood Python
> into a new YAML format gain us? This is maybe the biggest thing I'm
> missing from this whole discussion. We lose a bunch of things (ease of
> transition from other Python projects, excellent existing testing
> framework, etc.), but what are we actually gaining other than the
> ability to say that we use N + 1 OpenStack services? Because we're way
> past the point where "It's OpenStack deploying OpenStack" is sufficient
> reason for people to pay attention to us. We need less "Ooh, neat" and
> more "Ooh, that's easy to use and works well." It's still not clear to
> me that Mistral helps in any way with the latter.
> 6) On the testing note, how do we test these workflows? Do we know what
> happens when step X fails? How do we test that they handle it properly
> in an automated and repeatable way? In Python these are largely easy
> questions to answer: unit tests. How do you unit test YAML? This is a
> big reason I'm not even crazy about having Mistral on the back end of a
> TripleO API. We'd be going from code that we can test and prove works
> in a variety of scenarios, to YAML that is tested and proven to work in
> exactly the three scenarios we run in CI. This is basically the same
> situation we had with tripleo-incubator, and it was bad there too.
> I dunno. Maybe I'm too late to this party to have any impact on the
> discussion, but I very much do not like the direction we're going and I
> would be remiss if I didn't at least point out my concerns with it.
> On 01/13/2016 03:41 AM, Tzu-Mainn Chen wrote:
> > Hey all,
> > I realize now from the title of the other TripleO/Mistral thread
>  that
> > the discussion there may have gotten confused. I think using
> Mistral for
> > TripleO processes that are obviously workflows - stack deployment,
> > registration - makes perfect sense. That thread is exploring
> > for doing that, and I think that's great work.
> > What I inappropriately started to address in that thread was a
> > orthogonal point that Dan asked in his original email, namely:
> > "what it might look like if we were to use Mistral as a
> replacement for the
> > TripleO API entirely"
> > I'd like to create this thread to talk about that; more of a
> 'should we'
> > than 'can we'. And to do that, I want to indulge in a thought
> > stemming from an IRC discussion with Dan and others. All, please
> correct me
> > if I've misstated anything.
> > The IRC discussion revolved around one use case: deploying a Heat
> > directly from a Swift container. With an updated patch, the Heat
> CLI can
> > support this functionality natively. Then we don't need a TripleO
> API; we
> > can use Mistral to access that functionality, and we're done, with
> no need
> > for additional code within TripleO. And, as I understand it,
> that's the
> > true motivation for using Mistral instead of a TripleO API:
> avoiding custom
> > code within TripleO.
> > That's definitely a worthy goal... except from my perspective, the
> > doesn't quite end there. A GUI needs additional functionality,
> which boils
> > down to: understanding the Heat deployment templates in order to
> > options for a user; and persisting those options within a Heat
> > file.
> > Right away I think we hit a problem. Where does the code for
> > options' go? Much of that understanding comes from the
> capabilities map
> > in tripleo-heat-templates ; it would make sense to me that
> > for that would fall to a TripleO library.
> > Still, perhaps we can limit the amount of TripleO code. So to
> give API
> > access to 'getDeploymentOptions', we can create a Mistral workflow.
> > Retrieve Heat templates from Swift -> Parse capabilities map
> > Which is fine-ish, except from an architectural perspective
> > 'getDeploymentOptions' violates the abstraction layer between
> storage and
> > business logic, a problem that is compounded because
> > is not the only functionality that accesses the Heat templates and
> > exposure through an API. And, as has been discussed on a separate
> > thread, we're not even sure Swift is sufficient for our needs; one
> > consideration right now is allowing deployment from templates
> stored in
> > multiple places, such as the file system or git.
> > Are we going to have duplicate 'getDeploymentOptions' workflows
> for each
> > storage mechanism? If we consolidate the storage code within a
> > library, do we really need a *workflow* to call a single
> function? Is a
> > thin TripleO API that contains no additional business logic really
> so bad
> > at that point?
> > My gut reaction is to say that proposing Mistral in place of a
> TripleO API
> > is to look at the engineering concerns from the wrong direction. The
> > Mistral alternative comes from a desire to limit custom TripleO
> code at all
> > costs. I think that is an extremely dangerous attitude that leads to
> > compromises and workarounds that will quickly lead to a shaky code
> > full of design flaws that make it difficult to implement or extend any
> > functionality cleanly.
> > I think the correct attitude is to simply look at the problem we're
> > trying to solve and find the correct architecture. For these get/set
> > methods that the API needs, it's pretty simple: storage -> some
> logic ->
> > a REST API. Adding a workflow engine on top of that is unneeded,
> and I
> > believe that means it's an incorrect solution.
> > Thanks,
> > Tzu-Mainn Chen
> > 
> > 
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> OpenStack Development Mailing List (not for usage questions)
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
More information about the OpenStack-dev