Open Stack

On 01/12/2016 10:50 AM, Jiri Tomasek wrote:
> On 01/12/2016 04:22 PM, Ryan Brown wrote:
>> On 01/12/2016 06:52 AM, Jiri Tomasek wrote:
>>> On 01/11/2016 04:51 PM, Dan Prince wrote:
>>>> Background info:
>>>>
>>>> We've got a problem in TripleO at the moment where many of our
>>>> workflows can be driven by the command line only. This causes some
>>>> problems for those trying to build a UI around the workflows in that
>>>> they have to duplicate deployment logic in potentially multiple places.
>>>> There are specs up for review which outline how we might solve this
>>>> problem by building what is called TripleO API [1].
>>>>
>>>> Late last year I began experimenting with an OpenStack service called
>>>> Mistral which contains a generic workflow API. Mistral supports
>>>> defining workflows in YAML and then creating, managing, and executing
>>>> them via an OpenStack API. Initially the effort was focused around the
>>>> idea of creating a workflow in Mistral which could supplant our
>>>> "baremetal introspection" workflow which currently lives in python-
>>>> tripleoclient. I create a video presentation which outlines this effort
>>>> [2]. This particular workflow seemed to fit nicely within the Mistral
>>>> tooling.
>>>>
>>>> ----
>>>>
>>>> More recently I've turned my attention to what it might look like if we
>>>> were to use Mistral as a replacement for the TripleO API entirely. This
>>>> brings forth the question of would TripleO be better off building out
>>>> its own API... or would relying on existing OpenStack APIs be a better
>>>> solution?
>>>>
>>>> Some things I like about the Mistral solution:
>>>>
>>>> - The API already exists and is generic.
>>>>
>>>> - Mistral already supports interacting with many of the OpenStack API's
>>>> we require [3]. Integration with keystone is baked in. Adding support
>>>> for new clients seems straightforward (I've had no issues in adding
>>>> support for ironic, inspector, and swift actions).
>>>>
>>>> - Mistral actions are pluggable. We could fairly easily wrap some of
>>>> our more complex workflows (perhaps those that aren't easy to replicate
>>>> with pure YAML workflows) by creating our own TripleO Mistral actions.
>>>> This approach would be similar to creating a custom Heat resource...
>>>> something we have avoided with Heat in TripleO but I think it is
>>>> perhaps more reasonable with Mistral and would allow us to again build
>>>> out our YAML workflows to drive things. This might allow us to build
>>>> off some of the tripleo-common consolidation that is already underway
>>>> ...
>>>>
>>>> - We could achieve a "stable API" by simply maintaining input
>>>> parameters for workflows in a stable manner. Or perhaps workflows get
>>>> versioned like a normal API would be as well.
>>>>
>>>> - The purist part of me likes Mistral quite a bit. It fits nicely with
>>>> the deploy OpenStack with OpenStack. I sort of feel like if we have to
>>>> build our own API in TripleO part of this vision has failed and could
>>>> even be seen as a massive technical debt which would likely be hard to
>>>> build a community around outside of TripleO.
>>>>
>>>> - Some of the proposed validations could perhaps be implemented as new
>>>> Mistral actions as well. I'm not convinced we require TripleO API just
>>>> to support a validations mechanism yet. Perhaps validations seem hard
>>>> because we are simply trying to do them in the wrong places anyway?
>>>> (like for example perhaps we should validate network connectivity at
>>>> inspection time rather than during provisioning).
>>>>
>>>> - Power users might find a workflow built around a Mistral API more
>>>> easy to interact with and expand upon. Perhaps this ends up being
>>>> something that gets submitted as a patchset back to the TripleO that we
>>>> accept into our upstream "stock" workflow sets.
>>>>
>>>> ----
>>>>
>>>> Last week we landed the last patches [4] to our undercloud to enable
>>>> installing Mistral by simply setting: enable_mistral = true in
>>>> undercloud.conf. NOTE: you'll need to be using a recent trunk repo from
>>>> Delorean so that you have the recently added Mistral packages for this
>>>> to work. Although the feature is disable by default this should enable
>>>> those wishing to tinker with Mistral as a new TripleO undercloud
>>>> service an easy path forwards.
>>>>
>>>> [1] https://review.openstack.org/#/c/230432
>>>> [2] https://www.youtube.com/watch?v=bnAT37O-sdw
>>>> [3] http://git.openstack.org/cgit/openstack/mistral/tree/mistral/action
>>>> s/openstack/mapping.json
>>>> [4] https://etherpad.openstack.org/p/tripleo-undercloud-workflow
>>>>
>>>>
>>>> __________________________________________________________________________
>>>>
>>>>
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>> Hi, I have a few questions:
>>>
>>> Is Mistral action able to access/manipulate local files? E.g. access the
>>> templates installed at undercloud's
>>> /usr/share/openstack-tripleo-heat-templates?
>>
>> I believe with mistral there would be an intermediate step of
>> uploading the templates to Swift first. Heat can read templates from
>> swift, and any mistral workflows would be able to read the templates
>> out, modify them, and save back to swift.
>
> Correct, but from the Mistral usage standpoint, having the flexibility
> is a good thing IMO regardless of the example I have chosen.
>
>>
>> Object stores FTW
>>
>>> I Mistral action able to call either OpenStack service python client or
>>> OpenStack service API directly?
>>>
>>> What is the response from the Mistral action in the workflow? Lets say
>>> we'd use Mistral to get a list of available environments (we do this in
>>> tripleo-common now) So we call Mistral API to trigger a workflow that
>>> has single action which gets the list of environments. Is mistral able
>>> to provide this list as a response, or it is able just to trigger a
>>> workflow?
>>>
>>> In similar manner, is Mistral able to provide a way to get workflow in
>>> progress data? E.g. We have a Mistral workflow for nodes introspection.
>>> This workflow contains several actions calling to ironic and
>>> ironic-inspector apis. GUI calls Mistral API to trigger the workflow and
>>> now it needs to have a way to track the progress of that workflow. How
>>> would this be achieved? I think forcing GUI to poll the APIs that are
>>> called in the actions and try to implement logic that estimates what
>>> state the workflow is to report about it is not a valid solution. We
>>> need the workflow API (Mistral) to provide a lets say web sockets
>>> connection and push the status of actions along with relevant data, so
>>> GUI can listen to those.
>>
>> I can see a few options for progress polling/push.
>>
>> Mistral:
>> 1. Send job to REST API (we didn't have to build)
>> 1. It spins off as many jobs as it needs (we build)
>> 1. Poll mistral to see how many are done
>>
>> TripleO API
>> 1. Build the REST API (we build)
>> 1. Send requests to start introspection (we build)
>> 1. Poll TripleO API until things are done
>>
>> Mistral + Zaqar:
>> 1. Send job to REST API (we didn't have to build)
>> 1. It spins off as many jobs as it needs (we build)
>> 1. Send updates to Zaqar (we didn't have to build)
>> 1. Get websocket updates to the GUI from Zaqar (we didn't have to build)
>>
>> Zaqar is (of course) designed to deliver updates like this, so every
>> project on the face of the planet doesn't have to rebuild websocket
>> notifications, which is a good thing.
>
> Thanks, this sounds very good. So the Mistral + Zaqar workflow means
> that as part of workflow action, Mistral notifies Zaqar that certain
> thing happened. Zaqar then notifies listening GUI providing information
> to identify what API call the GUI needs to make to retrieve relevant
> data or even provide the data itself (e.g. Mistral fails, some error
> occurs etc.)

I don't know what the mistral-zaqar integration is at the moment, but if 
it's not there you could post to zaqar in the custom TripleO actions.

>>
>>> I am about to implement your nodes workflow in the GUI to test how it
>>> works.
>>>
>>> Jirka
>>>
>>> __________________________________________________________________________
>>>
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> Jirka
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
Ryan Brown / Senior Software Engineer, Openstack / Red Hat, Inc.