Open Stack

On 12/08/2016 08:10 PM, Jason Rist wrote:
> On 12/08/2016 05:28 PM, Dan Sneddon wrote:
>> On 12/08/2016 06:05 AM, Jiri Tomasek wrote:
>>> Hi all,
>>>
>>> I've been investigating how to implement TripleO network configuration
>>> in TripleO UI. Based on my findings I'd like to propose a solution.
>>>
>>> tl;dr proposal: Slightly refactor Network environment files to match
>>> GUI usage, Use Jinja Templating to generate dynamic parts of the
>>> templates/environments
>>>
>>>
>>> # Overview
>>>
>>> I've used Ben Nemec's amazing Network template generator as a reference
>>> to help me understand how the network configuration works [1]. In
>>> general the process of configuring the network in TripleO is:
>>>
>>> Define which Networks we intend to use -> Assign Roles to the Networks
>>> (+ Assign Role Services to the Network) -> Generate NIC config
>>> templates based on previous information
>>>
>>>
>>> # Deeper dive into templates
>>>
>>> We currently have 2 environment files in THT [2] which define network
>>> configuration:
>>>
>>> network-environment.yaml [3] - holds the information on NIC
>>> configuration for each Role using
>>> OS::TripleO::<RoleName>::Net::SoftwareConfig resource + related
>>> parameter configuration
>>>
>>> network-isolation.yaml [4]
>>> - defines the list of networks using
>>> OS::TripleO::Network::<NetworkName> resource
>>> - defines ports configuration for each network using
>>> OS::TripleO::Network::Ports::<NetworkName>VipPort (note that both
>>> resources point to the static templates - those templates don't require
>>> any manual modification)
>>> - holds  Roles - Networks assignment using
>>> OS::TripleO::<RoleName>::Ports::<NetworkName>Port for each role and
>>> storage (again, templates referenced by those resources don't require
>>> any modification)
>>>
>>> User is intended to go ahead and modify those environments and provide
>>> NIC config templates to achieve a network configuration that matches
>>> his needs.
>>>
>>>
>>> # How GUI works
>>>
>>> Before proceeding to proposed changes I need to describe briefly how
>>> TripleO UI works. TripleO UI is using THT as a source of truth, which
>>> means that it is trying not to add any additional business logic or
>>> manipulate templates. Rather it uses environment files as a 'features'
>>> which user can enable or disable depending on the needs of the
>>> deployment. The information about inter-environment relationships is
>>> tracked in capabilities-map.yaml which is also part of the THT. Based
>>> on these choices, UI allows user to configure parameters for those
>>> features. The parameter values and information about which environments
>>> are selected is stored in mistral environment. This approach leaves the
>>> plan templates intact. Huge benefit of this approach is that UI (or
>>> tripleo-common) does not need to hold explicit business logic related
>>> to certain deployment features as it is purely driven by THT. Also
>>> Adding a new feature involves only providing the templates/environments
>>> and it automatically appears as an option in UI.
>>>
>>> To achieve best user experience while using this approach, the
>>> environment files need to be defined in a granular manner, so they
>>> don't require user to modify them and each describe an isolated 'feature'.
>>>
>>> Roles and Network Configuration are exceptions to this concept as they
>>> require modification/generation of the templates/environments and
>>> therefore they use Jinja templating to achieve that.
>>>
>>>
>>> # The proposal
>>>
>>> So having described previous, here is the approach I think we should
>>> use to achieve network configuration using TripleO UI:
>>>
>>> 1. Put networks definitions into separate environment for each network:
>>> - this way GUI can provide a list of networks available to use and let
>>> user select which of them he wants to use. These environments are not
>>> dynamic and if user wants to add a new network, he does so by creating
>>> new templates and environment for it. UI also provides means to
>>> configure parameters for each network at this point (if needed).
>>>
>>> For example the environment for a Storage Network looks like this:
>>>
>>> resource_registry:
>>>   OS::TripleO::Network::Storage: ../network/storage.yaml
>>>   OS::TripleO::Network::Ports::StorageVipPort:
>>> ../network/ports/storage.yaml
>>>
>>> 2. Assign Roles to Networks
>>> Having the Networks selected as well as Roles defined, TripleO UI
>>> provides user with means to assign Roles to Networks. This step
>>> involves generating the network-environment.yaml file. So TripleO UI
>>> sends the mapping of roles to network in json format to tripleo-common
>>> which in turn uses network-isolation.j2.yaml Jinja template to generate
>>> the environment file. I expect that pre-defined network-isolation.yaml
>>> will be included in default plan so the user does not need to start
>>> from scratch. Tripleo-common also provides an action to fetch
>>> network-roles assignment data by parsing the network-isolation.yaml
>>>
>>> In addition, user is able to assign individual Role Services to a
>>> Network. ServiceNetMap parameter is currently used for this. GUI needs
>>> to make sure that it represents Services-Networks assignment grouped by
>>> Role so it is ensured that user assigns Services to only networks where
>>> their Role is assigned.
>>>
>>> 3. Generate NIC Config templates
>>> TripleO UI provides means to configure NICS, Bonds etc. for each Role,
>>> using the information from previous steps. It sends the data in json
>>> format to tripleo-common which then generates nic config templates for
>>> each Role based on network/config/nic-configs/role.j2.yaml Jinja
>>> template and generates network-environment.yaml based on
>>> network-environment.j2.yaml which references those templates.
>>>
>>> Note that network-environment.j2.yaml probably can't be combined with
>>> network-isolation.j2.yaml as every time that environment would need to
>>> get updated, all data the template needs would need to be provided.
>>>
>>> There are wireframes made by Liz Blanchard currently available [5],
>>> althought they are not exactly up to date to this proposal. Ideally
>>> whole network configuration would happen on a screen based on the
>>> graphical representation of network [6].
>>>
>>>
>>> Any comments to this proposal are very welcome, please note that I am
>>> not a networking expert so I might be missing something.
>>>
>>> There is a spec [7] in progress aimed for Ocata, but the feature will
>>> highly probably not land in Ocata, so we'll need to update the spec and
>>> move it to next cycle.
>>>
>>>
>>> [1]
>>> http://blog.nemebean.com/content/tripleo-network-isolation-template-generator
>>>
>>> [2] https://github.com/openstack/tripleo-heat-templates
>>> [3]
>>> https://github.com/openstack/tripleo-heat-templates/blob/master/environments/network-environment.yaml
>>>
>>> [4]
>>> https://github.com/openstack/tripleo-heat-templates/blob/master/environments/network-isolation.yaml
>>>
>>> [5] https://openstack.invisionapp.com/share/UM87J4NBQ#/screens/179046668
>>> [6] https://openstack.invisionapp.com/share/UM87J4NBQ#/screens/179046679
>>> [7] https://review.openstack.org/#/c/396383/
>>>
>>> Thanks
>>> Jirka
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> Jirka,
>>
>> Thanks for putting this together, I think this is a great summary of
>> the issue and a good thing to discuss.
>>
>> I have done put some thought into how we might convert the existing
>> YAML NIC config templates to Jinja. Most of the templates are pretty
>> modular if you look closely. For instance, working from the
>> bond-with-vlans templates, the bridge definition is the same for all nodes:
>>
>>       - type: ovs_bridge
>>         name: bridge_name
>>         dns_servers:
>>           get_param: DnsServers
>>         members:
>>         - type: ovs_bond
>>           name: bond1
>>           ovs_options:
>>             get_param: BondInterfaceOvsOptions
>>           members:
>>           - type: interface
>>             name: nic2
>>             primary: true
>>           - type: interface
>>             name: nic3
>>
>> Then, we have each of the VLANs, which can be enabled or disabled based
>> on whether a network is enabled or not on a node. For instance, the
>> external network might be defined like this:
>>
>> {% if role.has_net_external %}
>>         - type: vlan
>>           device: bond1
>>           vlan_id:
>>             get_param: ExternalNetworkVlanID
>>           addresses:
>>           - ip_netmask:
>>               get_param: ExternalIpSubnet
>>   {% if role.default_route_network == 'external' }
>>           routes:
>>           - default: true
>>             next_hop:
>>               get_param: ExternalInterfaceDefaultRoute
>>   {% endif %}
>> {% endif %}
>>
>> (note that there might be a better way to test for the external
>> network, consider the above psuedo-code)
>>
>> The drawback to this approach is that we need a different template for
>> each physical configuration.
> 
> How many are we talking?

We would probably want at least as many as we have currently:
single-nic-vlans, single-nic-linux-bridge-vlans, bond-with-vlans, and
multiple NICs.

We could add other topologies, such as two bonds, but at a certain
point I think we need to recognize that we can't cover every possible
physical topology. I think we just need to demonstrate how to create
the physical NIC config template, and the deployer can work from the
samples to develop a custom config.

> 
>> If the controllers have more NICs than the
>> computes, then that requires a different base configuration. One
>> potential advantage to this approach is that you could even use one
>> unified template with different physical ports depending on the role.
>> For instance:
>>
>>         - type: ovs_bond
>>           name: bond1
>>           ovs_options:
>>             get_param: BondInterfaceOvsOptions
>>           members:
>>           - type: interface
>>             name: nic2
>>             primary: true
>>           - type: interface
>>             name: nic3
>> {% if role.name == Controller %}
>>           - type: interface
>>             name: nic4
>>             primary: true
>>           - type: interface
>>             name: nic5
>> {% endif %}
>>
>> However, the above might be a little difficult to model in a graphical
>> interface, especially if the configuration is significantly different.
>>
>> So this would change the workflow slightly. You would first develop a
>> template that included all the possible networks that could appear in
>> that physical configuration, then enable them conditionally based on
>> which networks were assigned to each role. It would also increase the
>> complexity of developing templates by hand, but it would probably
>> still be easier to manage one or two complex template than 5 or more
>> simple configurations.
>>
>> Since the physical template might be something that we could develop
>> automatically based on LLDP data collected during introspection, we
>> could potentially automate all parts of the configuration other than
>> customizing which networks to use where. Note that while it's easy
>> to conceptualize automatic template generation based on LLDP data
>> received from the switch, I also expect this to be pretty
>> error-prone. For instance, it may be difficult to detect which
>> interfaces should be part of a bond or bridge. Also, in cases where
>> a VLAN appears on more than one interface, it isn't easy to figure
>> out which interface Director should use for which. This could happen,
>> for instance, when the External VLAN is configured on one interface
>> for the public API, but it's also being shared on a bridge with many
>> VLANs trunked for Neutron routers.
>>
>> [1] - https://openstack.invisionapp.com/share/UM87J4NBQ#/screens/179046679
>>
> Dan - see above comment.
> 
> I think the combination of Jirka's and Dan's approach seems very reasonable.
> 
> -J
> 


-- 
Dan Sneddon         |  Senior Principal OpenStack Engineer
dsneddon at redhat.com |  redhat.com/openstack
dsneddon:irc        |  @dxs:twitter