Open Stack

On Thu, Sep 21, 2017 at 8:53 PM, Jiří Stránský <jistr at redhat.com> wrote:

> On 21.9.2017 18:04, Marios Andreou wrote:
>
>> On Thu, Sep 21, 2017 at 3:53 PM, Jiří Stránský <jistr at redhat.com> wrote:
>>
>> On 21.9.2017 12:31, Giulio Fidente wrote:
>>>
>>> On 09/20/2017 07:36 PM, James Slagle wrote:
>>>>
>>>> On Tue, Sep 19, 2017 at 8:37 AM, Giulio Fidente <gfidente at redhat.com>
>>>>> wrote:
>>>>>
>>>>> On 09/18/2017 05:37 PM, James Slagle wrote:
>>>>>>
>>>>>> - The entire sequence and flow is driven via Mistral on the Undercloud
>>>>>>> by default. This preserves the API layer and provides a clean
>>>>>>> reusable
>>>>>>> interface for the CLI and GUI.
>>>>>>>
>>>>>>>
>>>>>> I think it's worth saying that we want to move the deployment steps
>>>>>> out
>>>>>> of heat and in ansible, not in mistral so that mistral will run the
>>>>>> workflow only once and let ansible go through the steps
>>>>>>
>>>>>> I think having the steps in mistral would be a nice option to be able
>>>>>> to
>>>>>> rerun easily a particular deployment step from the GUI, versus having
>>>>>> them in ansible which is instead a better option for CLI users ... but
>>>>>> it looks like having them in ansible is the only option which permits
>>>>>> us
>>>>>> to reuse the same code to deploy an undercloud because having the
>>>>>> steps
>>>>>> in mistral would require the undercloud installation itself to depend
>>>>>> on
>>>>>> mistral which we don't want to
>>>>>>
>>>>>> James, Dan, please comment on the above if I am wrong
>>>>>>
>>>>>>
>>>>> That's correct. We don't want to require Mistral to install the
>>>>> Undercloud. However, I don't think that necessarily means it has to be
>>>>> a single call to ansible-playbook. We could have multiple invocations
>>>>> of ansible-playbook. Both Mistral and CLI code for installing the
>>>>> undercloud could handle that easily.
>>>>>
>>>>> You wouldn't be able to interleave an external playbook among the
>>>>> deploy steps however. That would have to be done under a single call
>>>>> to ansible-playbook (at least how that is written now). We could
>>>>> however have hooks that could serve as integration points to call
>>>>> external playbooks after each step.
>>>>>
>>>>>
>>>> the benefits of driving the steps from mistral are that then we could
>>>> also interleave the deployment steps and we won't need the
>>>> ansible-playbook hook for the "external" services:
>>>>
>>>> 1) collect the ansible tasks *and* the workflow_tasks (per step) from
>>>> heat
>>>>
>>>> 2) launch the stepN deployment workflow (ansible-playbook)
>>>>
>>>> 3) execute any workflow_task defined for stepN (like ceph-ansible
>>>> playbook)
>>>>
>>>> 4) repeat 2 and 3 for stepN+1
>>>>
>>>> I think this would also provide a nice interface for the UI ... but then
>>>> we'd need mistral to be able to deploy the undercloud
>>>>
>>>>
>>>>
>> Why not launch the _single_  deploy_steps playbook (so we have
>> when/conditionals with step numbers), passing in the step you want to have
>> executed (we can pass this in as a parameter to the mistral workflow and
>> pass through to the ansible-playbook invocation?), otherwise execute all
>> the steps.
>>
>
> +1 that's the sort of thing i meant by "it's not baked in by default but
> we could make it so". We could even give it a list of steps like
> `tripleo_run_steps: [4, 5, 6]`.
>
> In either case whether it should be ansible handling the loop
>> based on a passed in parameter.
>> 'Ansible-native' looping is currently the
>> case for the current deploy_steps_playbook here
>> https://github.com/openstack/tripleo-heat-templates/blob/259
>> cf512b3b7e3539105cdb52421e3239701ef73/common/deploy-steps.j2#L335
>> - can we not work parameterise that playbook so that we either do loop
>> with
>> the variable, or just step X?
>>
>> Then for the upgrade workflow it is as above but  1.5 first launch the
>> upgrade_tasks_playbook  then the deploy_steps playbook for all the steps
>> (consider this
>> https://review.openstack.org/#/c/505624/3/scripts/upgrade-no
>> n-controller.sh at 162
>> - download and run the playbooks for non-controllers in O->P upgrade
>> pointing this out to illustrate the workflow I have in mind). So I don't
>> see why we can't have mistral invoking ansible-playbook and taking
>> parameters like which playbook, which step etc.
>>
>>
>>
>>> Alternatively we could do the main step loop in Ansible directly, and
>>> have
>>> the tasks do whatever they need to get the particular service deployed,
>>> from  to launching a nested ansible-playbook run if that's what it takes.
>>>
>>>
>>
>> I think you can do both, I mean mistral invoking ansible-playbook and
>> still
>> let ansible handle the steps with a loop.
>>
>
> +1 exactly. FWIW i'm totally on board with wrapping everything in Mistral
> on the outermost level, as that's required for UI. I'm just not keen on
> having Mistral enter the process in between each step and drive the step
> loop.
>
> In fact that is what the current
>> upgrade_steps_playbook does here
>> https://github.com/openstack/tripleo-heat-templates/blob/259
>> cf512b3b7e3539105cdb52421e3239701ef73/common/deploy-steps.j2#L363-L365
>> with a loop variable 'step'. The upgrade_tasks have their 'tags: stepX'
>> converted to 'when: step == X' in the client here
>> https://github.com/openstack/python-tripleoclient/blob/4d342
>> 826d6c3db38ee88dccc92363b655b1161a5/tripleoclient/v1/overclo
>> ud_config.py#L63
>> - we must come up with a better solution than that; ultimately we can just
>> update the existing upgrade_tasks to have 'when' and the main reason for
>> not doing so already was not to break the heat-driven upgrade workflow but
>> that is going away for Q.
>>
>>
> +1 for `when` instead of `tags`, ideally we'd switch the main deployment
> tasks to that too.
>
>
>
>>
>>
>>> That way we could run the whole thing end-to-end via ansible-playbook, or
>>> if needed one could execute smaller bits by themselves (steps or nested
>>> playbook runs) -- that capability is not baked in by default, but i think
>>> we could make it so.
>>>
>>> Also the interface for services would be clean and simple -- it's always
>>> the ansible tasks.
>>>
>>> And Mistral-less use cases become easier to handle too (= undercloud
>>> installation when Mistral isn't present yet, or development envs when you
>>> want to tune the playbook directly without being forced to go through
>>> Mistral).
>>>
>>>
>> You don't *have* to go through mistral either way I mean you can always
>> just run ansible-playbook directly using the generated playbooks if that
>> is
>> what you need for dev/debug etc.
>>
>>
>>
>>> Logging becomes a bit more unwieldy in this scenario though, as for the
>>> nested ansible-playbook execution, all output would go into a task in the
>>> outer playbook, which would be harder to follow and the log of the outer
>>> playbook could be huge.
>>>
>>> So this solution is no silver bullet, but from my current point of view
>>> it
>>> seems a bit less conceptually foreign than using Mistral to provide step
>>> loop functionality to Ansible, which should be able to handle that on its
>>> own.
>>>
>>>
>>> just saying using mistral to invoke ansible-playbook doesn't imply having
>> mistral do the looping/step control. I think it was already mentioned that
>> we can/will have multiple invocations of ansible-playbook. Having the loop
>> in the playbook then means organising our templates a certain way so that
>> there is a _single_ parent playbook which we can parameterise to then run
>> all or some of the steps (which as pointed above is currently the case for
>> the upgrade and deployment steps playbooks).
>>
>
> Yup, +1 again :) However, the 1)2)3)4) approach discussed earlier in the
> thread suggested to hand over the step loop control to Mistral and keep
> using the Mistral workflow_tasks, which would make it impossible to have a
> single parent playbook, if i understood correctly. So Mistral would be a
> requirement for running all steps via a single command (impacting UC
> install and developer workflow).
>
> Or if you really wanted to create a single playbook from the 1)2)3)4)
> approach, it could create a Mistral->Ansible[->Mistral->Ansible] chain
> for things like Ceph install, which sounds suboptimal as well.
>
> Having loop control in Ansible allows going just
> Mistral->Ansible[->Ansible], with the option to run just Ansible[->Ansible]
> for UC install and dev workflow.
>
> (^ The bracketed parts only come into play with the external playbooks
> like for Ceph and Kubernetes.)
>
> Sorry for the overcommunication here in case it's clear. Just want to make
> sure that we're on the same page about what are the options that we're
> discussing.
>


+1 thanks Jirka in fact I got the impression you were advocating more for
mistral to own the workflow and steps rather than having it just invoke the
playbooks & passing through params about what would be executed.

marios



>
> Jirka
>
>
>
>>
>> For me the main advantage of using mistral is the integration with UI -
>> we've never had 'proper' client/common support for upgrades or updates and
>> getting that is a goal for Q (
>> https://etherpad.openstack.org/p/tripleo-ptg-queens-upgrades - if nothing
>> else, we need a more decent way of handling the ansible-playbook
>> invocations than a bash script like upgrade-non-controller). There is
>> already some relevant work under way here FYI
>> https://review.openstack.org/#/c/487496/
>> https://review.openstack.org/#/c/487488/ that will be used for P minor
>> update.
>>
>> thanks, marios
>>
>>
>>
>>
>>> - It would still be possible to run ansible-playbook directly for
>>>
>>>> various use cases (dev/test/POC/demos). This preserves the quick
>>>>>>> iteration via Ansible that is often desired.
>>>>>>>
>>>>>>> - The remaining SoftwareDeployment resources in
>>>>>>> tripleo-heat-templates
>>>>>>> need to be supported by config download so that the entire
>>>>>>> configuration can be driven with Ansible, not just the deployment
>>>>>>> steps. The success criteria for this point would be to illustrate
>>>>>>> using an image that does not contain a running os-collect-config.
>>>>>>>
>>>>>>> - The ceph-ansible implementation done in Pike could be reworked to
>>>>>>> use this model. "config download" could generate playbooks that have
>>>>>>> hooks for calling external playbooks, or those hooks could be
>>>>>>> represented in the templates directly. The result would be the same
>>>>>>> either way though in that Heat would no longer be triggering a
>>>>>>> separate Mistral workflow just for ceph-ansible.
>>>>>>>
>>>>>>>
>>>>>> I'd say for ceph-ansible, kubernetes and in general anything else
>>>>>> which
>>>>>> needs to run with a standard playbook installed on the undercloud and
>>>>>> not one generated via the heat templates... these "external" services
>>>>>> usually require the inventory file to be in different format, to
>>>>>> describe the hosts to use on a per-service basis, not per-role (and I
>>>>>> mean tripleo roles here, not ansible roles obviously)
>>>>>>
>>>>>> About that, we discussed a more long term vision where the playbooks
>>>>>> (static data) needd to describe how to deploy/upgrade a given service
>>>>>> is
>>>>>> in a separate repo (like tripleo-apb) and we "compose" from heat the
>>>>>> list of playbooks to be executed based on the roles/enabled services;
>>>>>> in
>>>>>> this scenario we'd be much closer to what we had to do for
>>>>>> ceph-ansible
>>>>>> and I feel like that might finally allow us merge back the ceph
>>>>>> deployment (or kubernetes deployment) process into the more general
>>>>>> approach driven by tripleo
>>>>>>
>>>>>> James, Dan, comments?
>>>>>>
>>>>>>
>>>>> Agreed, I think this is the longer term plan in regards to using
>>>>> APB's, where everything consumed is an external playbook/role.
>>>>>
>>>>> We definitely want to consider this plan in parallel with the POC work
>>>>> that Flavio is pulling together and make sure that they are aligned so
>>>>> that we're not constantly reworking the framework.
>>>>>
>>>>> I've not yet had a chance to review the material he sent out this
>>>>> morning, but perhaps we could work together to update the sequence
>>>>> diagram to also have a "future" state to indicate where we are going
>>>>> and what it would look like with APB's and external paybooks.
>>>>>
>>>>>
>>>> Indeed that would be great :) IIUC, APBs are deployed by running a
>>> short-lived container with Ansible inside, which then connects to
>>> Kubernetes endpoint to create resources. So this should be a less
>>> complicated case than running non-containerized external playbooks.
>>>
>>>
>>> this would be awesome, note that it isn't only ceph and kubernetes
>>>> anymore in this scenario ... I just spotted a submission for the Skydive
>>>> composable service and it uses the same mistral/ansible-playbook
>>>> approach ... so it's already 3 looking forward for this!
>>>>
>>>> https://review.openstack.org/#/c/502353/
>>>>
>>>>
>>>> [1] https://github.com/ansibleplaybookbundle/ansible-playbook-
>>> bundle/blob/master/docs/design.md#deploy
>>>
>>>
>>> ____________________________________________________________
>>> ______________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.op
>>> enstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170925/81104292/attachment.html>