Open Stack

On Wed, Nov 8, 2017 at 11:09 AM, Saravanan KR <skramaja at redhat.com> wrote:

> Thanks Steven for the update.
>
> Current CLI flow:
> ----------------------
> * User need to add -p parameter for the overcloud deploy command with
> workflows to be invoked [1]
> * Plan will update updated to the swift container
> * Derived parameters workflow is initiated
>     - For each role
>         * Get the introspection data of first node assigned to the role
>         * Find the list features based on the services or parameters
>         * If dpdk present, run dpdk formulas workflow
>         * if sriov is present, run sriov formulas workfow (under
> development)
>         * if sriov or dpdk is present, run host formulas workflow
>         * if hci present, run hci formulas workflow
>
> Here the order of the formulas workflow invocation is important. For
> example,  in Compute-DPDK-HCI role, HCI formulas should exclude the
> CPUs allocated for DPDK PMD threads, while calculating cpu allocation
> ratio.
>
> I am trying to understand the proposed changes. Is it for assisting UI
> only or changing the existing CLI flow too? If the idea is to invoke
> the individual formulas workflow, it will not be possible with
> existing implementation, need to be re-worked. We need to introduce
> order for formulas workflow and direct fetching and merging of derived
> parameters in plan.
>

So there are several problems we're trying to solve with this proposal. In
general is to provide feature based workflows which will configure these
features, as well as provide means to get current configuration of these
features and provide sensible information about the input for these
workflows.

I think one of the main problem of current implementation is that user is
not able to get any information about input required to provide to run
derivation workflows. That information is purely documentation based and
also involves tweaking deployment-plan which I am convinced is not a good
way to provide the input.

So what we're proposing is to bring up a mechanism of mapping the
derivation workflows to services (roles/environments) so as Steven
described we're able to identify which workflows are possible to run and
provide extensive input definition so user can see what he is configuring
and why (input type, description, label).

This also means that there is several parameter derivation workflows rather
than just one and the input for the workflow is the actual input passed to
mistral (no plan-environment.yaml changes involved). Using this whole
approach means that for each such identified feature, we can provide -
Input details, general feature description (mistral workflow description)
and current configuration (by reaching to mistral workflow execution if
that was run before)

If as you're saying certain workflows depend on each other those should
probably be in one workflow. On the other hand, I think it is not goo
approach to try to put all the parameter derivation workflows into single
workflow.


-- Jirka


>
> As per earlier discussion jtomasek, to invoke derived parameters
> workflow (existing) for a plan, UI requires following information:
> * Whether derived parameters should be invoked for this deployment
> (based on roles and enabled services)
> * If yes, list of parameters, its types, and its default values (and
> choices if present), are required
>
> Did I miss anything?
>
> Regards,
> Saravanan KR
>
> [1] https://github.com/openstack/tripleo-heat-templates/blob/
> master/plan-samples/plan-environment-derived-params.yaml
>
> On Wed, Nov 8, 2017 at 2:39 PM, Bogdan Dobrelya <bdobreli at redhat.com>
> wrote:
> > On 11/8/17 6:09 AM, Steven Hardy wrote:
> >>
> >> Hi all,
> >>
> >> Today I had a productive hallway discussion with jtomasek and
> >> stevebaker re $subject, so I wanted to elaborate here for the benefit
> >> of those folks not present.  Hopefully we can get feedback on the
> >> ideas and see if it makes sense to continue and work on some patches:
> >>
> >> The problem under discussion is how do we run pre-deployment workflows
> >> (such as those integrated recently to calculate derived parameters,
> >> and in future perhaps also those which download container images etc),
> >> and in particular how do we make these discoverable via the UI
> >> (including any input parameters).
> >>
> >> The idea we came up with has two parts:
> >>
> >> 1. Add a new optional section to roles_data for services that require
> >> pre-deploy workflows
> >>
> >> E.g something like this:
> >>
> >>       pre_deploy_workflows:
> >>          - derive_params:
> >>                workflow_name:
> >> tripleo.derive_params_formulas.v1.dpdk_derive_params
> >>                inputs:
> >>                    ...
> >>
> >> This would allow us to associate a specific mistral workflow with a
> >> given service template, and also work around the fact that currently
> >> mistral inputs don't have any schema (only key/value input) as we
> >> could encode the required type and any constraints in the inputs block
> >> (clearly this could be removed in future should typed parameters
> >> become available in mistral).
> >>
> >> 2. Add a new workflow that calculates the enabled services and returns
> >> all pre_deploy_workflows
> >>
> >> This would take all enabled environments, then use heat to validate
> >> the configuration and return the merged resource registry (which will
> >> require https://review.openstack.org/#/c/509760/), then we would
> >> iterate over all enabled services in the registry and extract a given
> >> roles_data key (e.g pre_deploy_workflows)
> >>
> >> The result of the workflow would be a list of all pre_deploy_workflows
> >> for all enabled services, which the UI could then use to run the
> >> workflows as part of the pre-deploy process.
> >>
> >> If this makes sense I can go ahead and push some patches so we can
> >> iterate on the implementation?
> >
> >
> > I apologise for a generic/non-techy comment: it would be nice to keep
> > required workflows near the services' definition templates, to keep it as
> > much self-contained as possible. IIUC, that's covered by #1.
> > For future steps, I'd like to see all of the "bulk processing" to sit in
> > those templates as well.
> >
> >>
> >> Thanks,
> >>
> >> Steve
> >>
> >> ____________________________________________________________
> ______________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> > --
> > Best regards,
> > Bogdan Dobrelya,
> > Irc #bogdando
> >
> >
> > ____________________________________________________________
> ______________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20171108/d2fc2a5c/attachment.html>