Open Stack

On Fri, Mar 20, 2020 at 6:07 PM John Fulton <johfulto at redhat.com> wrote:
>
> On Thu, Mar 19, 2020 at 5:37 AM Saravanan KR <skramaja at redhat.com> wrote:
> >
> > On Thu, Mar 19, 2020 at 1:02 AM John Fulton <johfulto at redhat.com> wrote:
> > >
> > > On Sat, Mar 14, 2020 at 8:06 AM Rabi Mishra <ramishra at redhat.com> wrote:
> > > >
> > > > On Sat, Mar 14, 2020 at 2:10 AM John Fulton <johfulto at redhat.com> wrote:
> > > >>
> > > >> On Fri, Mar 13, 2020 at 3:27 PM Kevin Carter <kecarter at redhat.com> wrote:
> > > >> >
> > > >> > Hello stackers,
> > > >> >
> > > >> > In the pursuit to remove Mistral from the TripleO undercloud, we've discovered an old capability that we need to figure out how best to handle. Currently, we provide the ability for an end-user (operator / deployer) to pass in "N" Mistral workflows as part of a given deployment plan which is processed by python-tripleoclient at runtime [0][1]. From what we have documented, and what we can find within the code-base, we're not using this feature by default. That said, we do not remove something if it is valuable in the field without an adequate replacement. The ability to run arbitrary Mistral workflows at deployment time was first created in 2017 [2] and while present all this time, its documented [3] and intra-code-base uses are still limited to samples [4].
> > > >>
> > > >> As it stands now, we're on track to making Mistral inert this cycle and if our progress holds over the next couple of weeks the capability to run arbitrary Mistral workflows will be the only thing left within our codebase that relies on Mistral running on the Undercloud.
> > > >>
> > > >> >
> > > >> > So the question is what do we do with functionality. Do we remove this ability out right, do we convert the example workflow [5] into a stand-alone Ansible playbook and change the workflow runner to an arbitrary playbook runner, or do we simply leave everything as-is and deprecate it to be removed within the next two releases?
> > > >
> > > >
> > > > Yeah, as John mentioned, tripleo.derive_params.v1.derive_parameters workflow is surely being used for NFV (DPDK/SR-IOV) and HCI use cases and  can't be deprecated or dropped. Though we've a generic interface in tripleoclient to run any workflow in plan-environment, I have not seen it being used for anything other than the mentioned workflow.
> > > >
> > > > In the scope of 'mistral-ansible' work, we seem to  have two options.
> > > >
> > > > 1. Convert the workflow to ansible playbook 'as-is' i.e calculate and merge the derived parameters in plan-environment and as you've mentioned, change tripleoclient code to call any playbook in plan-environment.yaml and the parameters/vars.
> > >
> > > Nice idea. I hadn't thought of that.
> > >
> > > If there's a "hello world" example of this (which results in a THT
> > > param in the deployment plan being set to "hello world"), then I could
> > > try writing an ansible module to derive the HCI parameters and set
> > > them in place of the "hello world".
> > >
> > I am fine with the approach, but the only concern is, we have plans to
> > remove Heat in the coming cycles. One of inputs for the Mistral derive
> > parameters is fetched from the heat stack. If we are going to retain
> > it, then it has to be re-worked during the Heat removal. Mistral to
> > ansible could be the first step towards it.
>
> Hey Saravanan,
>
> That works for me. I'm glad we were able to come up with a way to do this.
>
> Kevin put some patches together today that will help a lot on this.
>
> 1. tht: https://review.opendev.org/#/c/714217/
> 2. tripleo-ansible: https://review.opendev.org/#/c/714232/
> 3. trilpeoclient: https://review.opendev.org/#/c/714198/
>
> If I put these on my undercloud, then I think I can run:
>
> 'openstack overcloud deploy ... -p plan-environment-derived-params.yaml'
>
> as usual and then the updated tripleoclient and tht patch should
> trigger the new tripleo-ansible playbook in place of the Mistral
> workbook.
>
> I think I can then just update that tripleo-ansible patch to have it
> include a new derive_params_hci role and add a new derive_params_hci
> module where I'll stick code from the original Python prototype I did
> for it. I'll probably just shell to `openstack baremetal introspection
> data save ID` from ansible to get the Ironic data. I'll give it a try
> next week and update this thread. Even if Heat is not in the flow, at
> least the Ansible role and module can be reused.
>
> Note that it uses the new tripleo_plan_parameters_update module that
> Rabi wrote so that should make it easier to deal with the deployment
> plan itself (https://review.opendev.org/712604).

Kevin and Rabi have made a lot of progress and with their unmerged
patches [1] 'openstack overcloud deploy -p
plan-environment-derived-params.yaml' has Ansible is running a
playbook [2] with place holders for us to use derive params which
looks like it will push the changes back to the deployment plan as
discussed above.

As far as I can tell 'openstack overcloud deploy -p
plan-environment-derived-params.yaml' from master won't work the old
way as Mistral isn't in the picture anymore when 'openstack overcloud
deploy -p' is run (someone please correct me if I'm wrong). Thus,
derive params are not going to work with the U release unless we
finish the above. I don't think we should undo the progress. We're now
in the RC so time is short. As long as we still ship the Mistral
container on the undercloud in U, a deployer could have it derive the
params in theory if the workbook is run manually and the resultant
params are applied as overrides. Do we GA with that as a known issue
with workaround and then circle back to fix the above and backport it?
I don't think we should delay the release for it. I think we should
instead push harder on getting the playbook and roles that Kevin
started [2] (they could still land in U).

  John

[1] https://review.opendev.org/#/q/topic:mistral_to_ansible+(status:open+OR+status:merged)
[2] https://review.opendev.org/#/c/719466/13/tripleo_ansible/roles/tripleo_derived_parameters/tasks/main.yml@170

>
>   John
>
> > Regards,
> > Saravanan KR
> >
> > >   John
> > >
> > > > 2. Move the functionality further down the component chain in TripleO to have the required ansible host/group_vars set for them to be used by config-download playbooks/ansible/puppet.
> > > >
> > > > I guess option 1 would be easier within the timelines. I've done some preliminary work to move some of the functionality in relevant mistral actions to utils modules[1], so that they can be called from ansible modules without depending on mistral/mistral-lib and use those in a playbook that kinda replicate the tasks in the mistral workflow.
> > > >
> > > > Having said that, it would be good to know what the DFG:NFV folks think, as they are the original authors/maintainers of that workflow.
> > > >
> > > > <snip>
> > > >
> > > >> The Mistral based workflow took advantage of the deployment plan which
> > > >> was stored in Swift on the undercloud. My understanding is that too is
> > > >> going away.
> > > >
> > > >
> > > > I'm not sure that would be in the scope of 'mstral-to-ansible' work. Dropping swift would probably be a bit more complex, as we use it to store templates, plan-environment, plan backups (possibly missing few more) etc  and would require significant design rework (may be possible when we get rid of heat in undercloud). In spite of heat using the templates from swift and merging environments on the client side, we've had already bumped heat's REST API json body size limit (max_json_body_size) on the undercloud to 4MB[2] from the default 1MB and sending all required templates as part of API request would not be a good idea from undercloud scalability pov.
> > > >
> > > > [1] https://review.opendev.org/#/c/709546/
> > > > [2] https://github.com/openstack/tripleo-heat-templates/blob/master/environments/undercloud.yaml#L109
> > > >
> > > > --
> > > > Regards,
> > > > Rabi Mishra
> > > >
> > >
> >