[openstack-dev] [TripleO] Driving workflows with Mistral
tzumainn at redhat.com
Tue Jan 12 21:07:32 UTC 2016
> On 01/11/2016 11:09 PM, Tzu-Mainn Chen wrote:
> > ----- Original Message -----
> >> Background info:
> >> We've got a problem in TripleO at the moment where many of our
> >> workflows can be driven by the command line only. This causes some
> >> problems for those trying to build a UI around the workflows in that
> >> they have to duplicate deployment logic in potentially multiple places.
> >> There are specs up for review which outline how we might solve this
> >> problem by building what is called TripleO API .
> >> Late last year I began experimenting with an OpenStack service called
> >> Mistral which contains a generic workflow API. Mistral supports
> >> defining workflows in YAML and then creating, managing, and executing
> >> them via an OpenStack API. Initially the effort was focused around the
> >> idea of creating a workflow in Mistral which could supplant our
> >> "baremetal introspection" workflow which currently lives in python-
> >> tripleoclient. I create a video presentation which outlines this effort
> >> . This particular workflow seemed to fit nicely within the Mistral
> >> tooling.
> >> ----
> >> More recently I've turned my attention to what it might look like if we
> >> were to use Mistral as a replacement for the TripleO API entirely. This
> >> brings forth the question of would TripleO be better off building out
> >> its own API... or would relying on existing OpenStack APIs be a better
> >> solution?
> >> Some things I like about the Mistral solution:
> >> - The API already exists and is generic.
> >> - Mistral already supports interacting with many of the OpenStack API's
> >> we require . Integration with keystone is baked in. Adding support
> >> for new clients seems straightforward (I've had no issues in adding
> >> support for ironic, inspector, and swift actions).
> >> - Mistral actions are pluggable. We could fairly easily wrap some of
> >> our more complex workflows (perhaps those that aren't easy to replicate
> >> with pure YAML workflows) by creating our own TripleO Mistral actions.
> >> This approach would be similar to creating a custom Heat resource...
> >> something we have avoided with Heat in TripleO but I think it is
> >> perhaps more reasonable with Mistral and would allow us to again build
> >> out our YAML workflows to drive things. This might allow us to build
> >> off some of the tripleo-common consolidation that is already underway
> >> ...
> >> - We could achieve a "stable API" by simply maintaining input
> >> parameters for workflows in a stable manner. Or perhaps workflows get
> >> versioned like a normal API would be as well.
> >> - The purist part of me likes Mistral quite a bit. It fits nicely with
> >> the deploy OpenStack with OpenStack. I sort of feel like if we have to
> >> build our own API in TripleO part of this vision has failed and could
> >> even be seen as a massive technical debt which would likely be hard to
> >> build a community around outside of TripleO.
> >> - Some of the proposed validations could perhaps be implemented as new
> >> Mistral actions as well. I'm not convinced we require TripleO API just
> >> to support a validations mechanism yet. Perhaps validations seem hard
> >> because we are simply trying to do them in the wrong places anyway?
> >> (like for example perhaps we should validate network connectivity at
> >> inspection time rather than during provisioning).
> >> - Power users might find a workflow built around a Mistral API more
> >> easy to interact with and expand upon. Perhaps this ends up being
> >> something that gets submitted as a patchset back to the TripleO that we
> >> accept into our upstream "stock" workflow sets.
> > Hiya! Thanks for putting down your thoughts.
> > I think I fundamentally disagree with the idea of using Mistral, simply
> > because many of the actions we'd like to expose through a REST API
> > (described in the tripleo-common deployment library spec ) aren't
> > workflows; they're straightforward get/set methods.
> Right, because this spec describes nearly nothing from what is present
> in tripleoclient now. And what we realistically have now is workflows,
> which we'll have to reimplement in API somehow. So maybe we need both:
> the get/set TripleO API for deployment plans and Mistral for workflows.
This would make sense to me. I have no objections to using Mistral for the
bits where we have actual workflows, only for the get/set-type methods
proposed in the spec. Using it for a latter seems like a major stretch,
as I argued in my previous email.
> > Putting a workflow
> > engine in front of that feels like overkill and an added complication
> > that simply isn't needed. And added complications can lead to unneeded
> > complications: for instance, one open Mistral bug details how it may
> > not scale well .
> Let's not talk about scaling in the context of what we have in
> tripleoclient now ;)
> > The Mistral solution feels like we're trying to force a circular peg
> > into a round-ish hole. In a vacuum, if we were to consider the
> > engineering problem of exposing a code base to outside consumers in a
> > non-language specific fashion - I'm pretty sure we'd just suggest the
> > creation of a REST API and be done with it; the thought of using a
> > workflow engine as the frontend would not cross our minds.
> > I don't really agree with the 'purist' argument. We already have custom
> > business logic written in the TripleO CLI; accepting that within TripleO,
> > but not a very thin API layer, feels like an arbitrary line to me. And
> > if that line exists, I'd argue that if it forces overcomplicated
> > solutions to straightforward engineering problems, then that line needs
> > to be moved.
> > Mainn
> > 
> > https://github.com/openstack/tripleo-specs/blob/master/specs/mitaka/tripleo-overcloud-deployment-library.rst
> >  https://bugs.launchpad.net/mistral/+bug/1423054
> >> ----
> >> Last week we landed the last patches  to our undercloud to enable
> >> installing Mistral by simply setting: enable_mistral = true in
> >> undercloud.conf. NOTE: you'll need to be using a recent trunk repo from
> >> Delorean so that you have the recently added Mistral packages for this
> >> to work. Although the feature is disable by default this should enable
> >> those wishing to tinker with Mistral as a new TripleO undercloud
> >> service an easy path forwards.
> >>  https://review.openstack.org/#/c/230432
> >>  https://www.youtube.com/watch?v=bnAT37O-sdw
> >>  http://git.openstack.org/cgit/openstack/mistral/tree/mistral/action
> >> s/openstack/mapping.json
> >>  https://etherpad.openstack.org/p/tripleo-undercloud-workflow
> >> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
More information about the OpenStack-dev