Open Stack

On Thu, Feb 15, 2018 at 2:00 AM, Jiri Tomasek <jtomasek at redhat.com> wrote:

>
> On Wed, Feb 14, 2018 at 11:16 PM, Ben Nemec <openstack at nemebean.com>
> wrote:
>
>>
>>
>> On 02/09/2018 08:49 AM, Jiri Tomasek wrote:
>>
>>> *Step 2. network-environment -> NIC configs*
>>>
>>> Second step of network configuration is NIC config. For this
>>> network-environment.yaml is used which references NIC config templates
>>> which define network_config in their resources section. User is currently
>>> required to configure these templates manually. We would like to provide
>>> interactive view which would allow user to setup these templates using
>>> TripleO UI. A good example is a standalone tool created by Ben Nemec [3].
>>>
>>> There is currently work aimed for Pike to introduce jinja templating for
>>> network environments and templates [4] (single-nic-with-vlans,
>>> bond-with-vlans) to support composable networks and roles (integrate data
>>> from roles_data.yaml and network_data.yaml) It would be great if we could
>>> move this one step further by using these samples as a starting point and
>>> let user specify full NIC configuration.
>>>
>>> Available information at this point:
>>> - list of roles and networks as well as which networks need to be
>>> configured at which role's NIC Config template
>>> - os-net-config schema which defines NIC configuration elements and
>>> relationships [5]
>>> - jinja templated sample NIC templates
>>>
>>> Requirements:
>>> - provide feedback to the user about networks assigned to role and have
>>> not been configured in NIC config yet
>>>
>>
>> I don't have much to add on this point, but I will note that because my
>> UI is standalone and pre-dates composable networks it takes the opposite
>> approach.  As a user adds a network to a role, it exposes the configuration
>> for that network.  Since you have the networks ahead of time, you can
>> obviously expose all of those settings up front and ensure the correct
>> networks are configured for each nic-config.
>>
>> I say this mostly for everyone's awareness so design elements of my tool
>> don't get copied where they don't make sense.
>>
>> - let user construct network_config section of NIC config templates for
>>> each role (brigdes/bonds/vlans/interfaces...)
>>> - provide means to assign network to vlans/interfaces and automatically
>>> construct network_config section parameter references
>>>
>>
>> So obviously your UI code is going to differ, but I will point out that
>> the code in my tool for generating the actual os-net-config data is
>> semi-standalone: https://github.com/cybertron/t
>> ripleo-scripts/blob/master/net_processing.py
>>
>> It's also about 600 lines of code and doesn't even handle custom roles or
>> networks yet.  I'm not clear whether it ever will at this point given the
>> change in my focus.
>>
>> Unfortunately the input JSON schema isn't formally documented, although
>> the unit tests do include a number of examples.
>> https://github.com/cybertron/tripleo-scripts/blob/master/tes
>> t-data/all-the-things/nic-input.json covers quite a few different cases.
>>
>> - populate parameter definitions in NIC config templates based on
>>> role/networks assignment
>>> - populate parameter definitions in NIC config templates based on
>>> specific elements which use them e.g. BondInterfaceOvsOptions in case when
>>> ovs_bond is used
>>>
>>
>> I guess there's two ways to handle this - you could use the new jinja
>> templating to generate parameters, or you could handle it in the generation
>> code.
>>
>> I'm not sure if there's a chicken-and-egg problem with the UI generating
>> jinja templates, but that's probably the simplest option if it works. The
>> approach I took with my tool was to just throw all the parameters into all
>> the files and if they're unused then oh well.  With jinja templating you
>> could do the same thing - just copy a single boilerplate parameter header
>> that includes the jinja from the example nic-configs and let the templating
>> handle all the logic for you.
>>
>> It would be cleaner to generate static templates that don't need to be
>> templated, but it would require re-implementing all of the custom network
>> logic for the UI.  I'm not sure being cleaner is sufficient justification
>> for doing that.
>>
>> - store NIC config templates in deployment plan and reference them from
>>> network-environment.yaml
>>>
>>> Problems to solve:
>>> As a biggest problem to solve I see defining logic which would
>>> automatically handle assigning parameters to elements in network_config
>>> based on Network which user assigns to the element. For example: Using GUI,
>>> user is creating network_config for compute role based on
>>> network/config/multiple-nics/compute.yaml, user adds an interface and
>>> assigns the interface to Tenant network. Resulting template should then
>>> automatically populate addresses/ip_netmask: get_param: TenantIpSubnet.
>>> Question is whether all this logic should live in GUI or should GUI pass
>>> simplified format to Mistral workflow which will convert it to proper
>>> network_config format and populates the template with it.
>>>
>>
>> I guess the fact that I separated the UI and config generation code in my
>> tool is my answer to this question.  I don't remember all of my reasons for
>> that design, but I think the main thing was to keep the input and
>> generation cleanly separated.  Otherwise there was a danger of making a UI
>> change and having it break the generation process because they were tightly
>> coupled.  Having a JSON interface between the two avoids a lot of those
>> problems.  It also made it fairly easy to unit test the generation code,
>> whereas trying to mock out all of the UI elements would have been a fragile
>> nightmare.
>>
>> It does require a bunch of translation code[1], but a lot of it is fairly
>> boilerplate (just map UI inputs to JSON keys).
>>
>> 1: https://github.com/cybertron/tripleo-scripts/blob/171aedabfe
>> ad1f27f4dc0fce41a8b82da28923ed/net-iso-gen.py#L515
>>
>> Hope this helps.
>
>
> Ben, thanks a lot for your input. I think this makes the direction with
> NIC configs clearer:
>
> 1. The generated template will include all possible parameters definitions
> unless we find a suitable way of populating parameters section part of
> template generation process. Note that current jinja templates for NIC
> config (e.g. network/config/multiple-nics/role.role.j2.yaml:127) create
> these definitions conditionally by specific role name which is not very
> elegant in terms of custom roles.
>

This patch recently landed, which generates all the needed parameters in
the sample NIC configs based on the composable networks defined in
network_data.yaml:
https://review.openstack.org/#/c/523638

Furthermore, this patch removes all the role-specific hard-coded templates,
and generates templates based on the role-to-network association in
roles_data.yaml.

I think we could use this method to generate the needed parameters for the
templates generated in the UI. I would personally like to see a workflow
where the user chose one of the built-in NIC config designs to generate
samples, which could then be further edited. Presenting a blank slate to
the user, and requiring them to build up the hierarchy is very confusing
unless the installer is very familiar with the desired architecture (first
add a bridge, then add a bond to the bridge, then add interfaces to the
bond, then add VLANs to the bridge). It's better to start with a basic
example (VLANs on a single NIC, one NIC per network, DPDK, etc.), and allow
the user to customize from there.


>
> 2. GUI is going to define forms to add/configure network elements
> (interface, bridge, bond, vlan, ...) and provide user friendly way to
> combine these together. The whole data construct (per Role) is going to be
> sent to tripleo-common workflow as json. Workflow consumes json input and
> produces final template yaml. I think we should be able to reuse bunch of
> the logic which Ben already created.
>
> Example:
> json input from GUI:
> ..., {
>   type: 'interface',
>   name: 'nic1',
>   network_name_lower: 'external'
> },...
> transformed by tripleo-common:
> ...
> - type: interface
>   name: nic{{loop.index + 1}}
>   use_dhcp: false
>   addresses:
>   - ip_netmask:
>   get_param: {{network.name}}IpSubnet
> ...
>
> With this approach, we'll create common API provided by Mistral to
> generate NIC config templates which can be reused by CLI and other clients,
> not TripleO UI specifically. Note that we will also need a 'reverse'
> Mistral workflow which is going to convert template yaml network_config
> into the input json format, so GUI can display current configuration to the
> user and let him change that.
>
> Liz has updated network configuration wireframes which can be found here
> https://lizsurette.github.io/OpenStack-Design/
> tripleo-ui/3-tripleo-ui-edge-cases/7.advancednetworkconfigurationan
> dtopology . The goal is to provide a graphical network configuration
> overview and let user perform actions from it. This ensures that with every
> action performed, user immediately gets clear feedback on how does the
> network configuration look.
>
> -- Jirka
>

I like the wireframes overall. However, I'm trying to avoid a flexible and
open-ended configuration if it isn't clear what the final configuration
should look like. We want to present the user with some basic forms, and
let them modify those forms to their needs.


>
>
>
>>
>>
>> -Ben
>>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Dan Sneddon         |  Senior Principal OpenStack Engineer
dsneddon at redhat.com |  redhat.com/openstack
dsneddon:irc        |  @dxs:twitter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180222/26dac6ad/attachment.html>