Open Stack

On 01/12/2016 04:22 PM, Ryan Brown wrote:
> On 01/12/2016 06:52 AM, Jiri Tomasek wrote:
>> On 01/11/2016 04:51 PM, Dan Prince wrote:
>>> Background info:
>>>
>>> We've got a problem in TripleO at the moment where many of our
>>> workflows can be driven by the command line only. This causes some
>>> problems for those trying to build a UI around the workflows in that
>>> they have to duplicate deployment logic in potentially multiple places.
>>> There are specs up for review which outline how we might solve this
>>> problem by building what is called TripleO API [1].
>>>
>>> Late last year I began experimenting with an OpenStack service called
>>> Mistral which contains a generic workflow API. Mistral supports
>>> defining workflows in YAML and then creating, managing, and executing
>>> them via an OpenStack API. Initially the effort was focused around the
>>> idea of creating a workflow in Mistral which could supplant our
>>> "baremetal introspection" workflow which currently lives in python-
>>> tripleoclient. I create a video presentation which outlines this effort
>>> [2]. This particular workflow seemed to fit nicely within the Mistral
>>> tooling.
>>>
>>> ----
>>>
>>> More recently I've turned my attention to what it might look like if we
>>> were to use Mistral as a replacement for the TripleO API entirely. This
>>> brings forth the question of would TripleO be better off building out
>>> its own API... or would relying on existing OpenStack APIs be a better
>>> solution?
>>>
>>> Some things I like about the Mistral solution:
>>>
>>> - The API already exists and is generic.
>>>
>>> - Mistral already supports interacting with many of the OpenStack API's
>>> we require [3]. Integration with keystone is baked in. Adding support
>>> for new clients seems straightforward (I've had no issues in adding
>>> support for ironic, inspector, and swift actions).
>>>
>>> - Mistral actions are pluggable. We could fairly easily wrap some of
>>> our more complex workflows (perhaps those that aren't easy to replicate
>>> with pure YAML workflows) by creating our own TripleO Mistral actions.
>>> This approach would be similar to creating a custom Heat resource...
>>> something we have avoided with Heat in TripleO but I think it is
>>> perhaps more reasonable with Mistral and would allow us to again build
>>> out our YAML workflows to drive things. This might allow us to build
>>> off some of the tripleo-common consolidation that is already underway
>>> ...
>>>
>>> - We could achieve a "stable API" by simply maintaining input
>>> parameters for workflows in a stable manner. Or perhaps workflows get
>>> versioned like a normal API would be as well.
>>>
>>> - The purist part of me likes Mistral quite a bit. It fits nicely with
>>> the deploy OpenStack with OpenStack. I sort of feel like if we have to
>>> build our own API in TripleO part of this vision has failed and could
>>> even be seen as a massive technical debt which would likely be hard to
>>> build a community around outside of TripleO.
>>>
>>> - Some of the proposed validations could perhaps be implemented as new
>>> Mistral actions as well. I'm not convinced we require TripleO API just
>>> to support a validations mechanism yet. Perhaps validations seem hard
>>> because we are simply trying to do them in the wrong places anyway?
>>> (like for example perhaps we should validate network connectivity at
>>> inspection time rather than during provisioning).
>>>
>>> - Power users might find a workflow built around a Mistral API more
>>> easy to interact with and expand upon. Perhaps this ends up being
>>> something that gets submitted as a patchset back to the TripleO that we
>>> accept into our upstream "stock" workflow sets.
>>>
>>> ----
>>>
>>> Last week we landed the last patches [4] to our undercloud to enable
>>> installing Mistral by simply setting: enable_mistral = true in
>>> undercloud.conf. NOTE: you'll need to be using a recent trunk repo from
>>> Delorean so that you have the recently added Mistral packages for this
>>> to work. Although the feature is disable by default this should enable
>>> those wishing to tinker with Mistral as a new TripleO undercloud
>>> service an easy path forwards.
>>>
>>> [1] https://review.openstack.org/#/c/230432
>>> [2] https://www.youtube.com/watch?v=bnAT37O-sdw
>>> [3] http://git.openstack.org/cgit/openstack/mistral/tree/mistral/action
>>> s/openstack/mapping.json
>>> [4] https://etherpad.openstack.org/p/tripleo-undercloud-workflow
>>>
>>>
>>> __________________________________________________________________________ 
>>>
>>>
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> Hi, I have a few questions:
>>
>> Is Mistral action able to access/manipulate local files? E.g. access the
>> templates installed at undercloud's
>> /usr/share/openstack-tripleo-heat-templates?
>
> I believe with mistral there would be an intermediate step of 
> uploading the templates to Swift first. Heat can read templates from 
> swift, and any mistral workflows would be able to read the templates 
> out, modify them, and save back to swift.

Correct, but from the Mistral usage standpoint, having the flexibility 
is a good thing IMO regardless of the example I have chosen.

>
> Object stores FTW
>
>> I Mistral action able to call either OpenStack service python client or
>> OpenStack service API directly?
>>
>> What is the response from the Mistral action in the workflow? Lets say
>> we'd use Mistral to get a list of available environments (we do this in
>> tripleo-common now) So we call Mistral API to trigger a workflow that
>> has single action which gets the list of environments. Is mistral able
>> to provide this list as a response, or it is able just to trigger a
>> workflow?
>>
>> In similar manner, is Mistral able to provide a way to get workflow in
>> progress data? E.g. We have a Mistral workflow for nodes introspection.
>> This workflow contains several actions calling to ironic and
>> ironic-inspector apis. GUI calls Mistral API to trigger the workflow and
>> now it needs to have a way to track the progress of that workflow. How
>> would this be achieved? I think forcing GUI to poll the APIs that are
>> called in the actions and try to implement logic that estimates what
>> state the workflow is to report about it is not a valid solution. We
>> need the workflow API (Mistral) to provide a lets say web sockets
>> connection and push the status of actions along with relevant data, so
>> GUI can listen to those.
>
> I can see a few options for progress polling/push.
>
> Mistral:
> 1. Send job to REST API (we didn't have to build)
> 1. It spins off as many jobs as it needs (we build)
> 1. Poll mistral to see how many are done
>
> TripleO API
> 1. Build the REST API (we build)
> 1. Send requests to start introspection (we build)
> 1. Poll TripleO API until things are done
>
> Mistral + Zaqar:
> 1. Send job to REST API (we didn't have to build)
> 1. It spins off as many jobs as it needs (we build)
> 1. Send updates to Zaqar (we didn't have to build)
> 1. Get websocket updates to the GUI from Zaqar (we didn't have to build)
>
> Zaqar is (of course) designed to deliver updates like this, so every 
> project on the face of the planet doesn't have to rebuild websocket 
> notifications, which is a good thing.

Thanks, this sounds very good. So the Mistral + Zaqar workflow means 
that as part of workflow action, Mistral notifies Zaqar that certain 
thing happened. Zaqar then notifies listening GUI providing information 
to identify what API call the GUI needs to make to retrieve relevant 
data or even provide the data itself (e.g. Mistral fails, some error 
occurs etc.)

>
>> I am about to implement your nodes workflow in the GUI to test how it
>> works.
>>
>> Jirka
>>
>> __________________________________________________________________________ 
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: 
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Jirka