Open Stack

On 12/08/2016 06:05 AM, Jiri Tomasek wrote:
> Hi all,
> 
> I've been investigating how to implement TripleO network configuration
> in TripleO UI. Based on my findings I'd like to propose a solution.
> 
> tl;dr proposal: Slightly refactor Network environment files to match
> GUI usage, Use Jinja Templating to generate dynamic parts of the
> templates/environments
> 
> 
> # Overview
> 
> I've used Ben Nemec's amazing Network template generator as a reference
> to help me understand how the network configuration works [1]. In
> general the process of configuring the network in TripleO is:
> 
> Define which Networks we intend to use -> Assign Roles to the Networks
> (+ Assign Role Services to the Network) -> Generate NIC config
> templates based on previous information
> 
> 
> # Deeper dive into templates
> 
> We currently have 2 environment files in THT [2] which define network
> configuration:
> 
> network-environment.yaml [3] - holds the information on NIC
> configuration for each Role using
> OS::TripleO::<RoleName>::Net::SoftwareConfig resource + related
> parameter configuration
> 
> network-isolation.yaml [4]
> - defines the list of networks using
> OS::TripleO::Network::<NetworkName> resource
> - defines ports configuration for each network using
> OS::TripleO::Network::Ports::<NetworkName>VipPort (note that both
> resources point to the static templates - those templates don't require
> any manual modification)
> - holds  Roles - Networks assignment using
> OS::TripleO::<RoleName>::Ports::<NetworkName>Port for each role and
> storage (again, templates referenced by those resources don't require
> any modification)
> 
> User is intended to go ahead and modify those environments and provide
> NIC config templates to achieve a network configuration that matches
> his needs.
> 
> 
> # How GUI works
> 
> Before proceeding to proposed changes I need to describe briefly how
> TripleO UI works. TripleO UI is using THT as a source of truth, which
> means that it is trying not to add any additional business logic or
> manipulate templates. Rather it uses environment files as a 'features'
> which user can enable or disable depending on the needs of the
> deployment. The information about inter-environment relationships is
> tracked in capabilities-map.yaml which is also part of the THT. Based
> on these choices, UI allows user to configure parameters for those
> features. The parameter values and information about which environments
> are selected is stored in mistral environment. This approach leaves the
> plan templates intact. Huge benefit of this approach is that UI (or
> tripleo-common) does not need to hold explicit business logic related
> to certain deployment features as it is purely driven by THT. Also
> Adding a new feature involves only providing the templates/environments
> and it automatically appears as an option in UI.
> 
> To achieve best user experience while using this approach, the
> environment files need to be defined in a granular manner, so they
> don't require user to modify them and each describe an isolated 'feature'.
> 
> Roles and Network Configuration are exceptions to this concept as they
> require modification/generation of the templates/environments and
> therefore they use Jinja templating to achieve that.
> 
> 
> # The proposal
> 
> So having described previous, here is the approach I think we should
> use to achieve network configuration using TripleO UI:
> 
> 1. Put networks definitions into separate environment for each network:
> - this way GUI can provide a list of networks available to use and let
> user select which of them he wants to use. These environments are not
> dynamic and if user wants to add a new network, he does so by creating
> new templates and environment for it. UI also provides means to
> configure parameters for each network at this point (if needed).
> 
> For example the environment for a Storage Network looks like this:
> 
> resource_registry:
>   OS::TripleO::Network::Storage: ../network/storage.yaml
>   OS::TripleO::Network::Ports::StorageVipPort:
> ../network/ports/storage.yaml
> 
> 2. Assign Roles to Networks
> Having the Networks selected as well as Roles defined, TripleO UI
> provides user with means to assign Roles to Networks. This step
> involves generating the network-environment.yaml file. So TripleO UI
> sends the mapping of roles to network in json format to tripleo-common
> which in turn uses network-isolation.j2.yaml Jinja template to generate
> the environment file. I expect that pre-defined network-isolation.yaml
> will be included in default plan so the user does not need to start
> from scratch. Tripleo-common also provides an action to fetch
> network-roles assignment data by parsing the network-isolation.yaml
> 
> In addition, user is able to assign individual Role Services to a
> Network. ServiceNetMap parameter is currently used for this. GUI needs
> to make sure that it represents Services-Networks assignment grouped by
> Role so it is ensured that user assigns Services to only networks where
> their Role is assigned.
> 
> 3. Generate NIC Config templates
> TripleO UI provides means to configure NICS, Bonds etc. for each Role,
> using the information from previous steps. It sends the data in json
> format to tripleo-common which then generates nic config templates for
> each Role based on network/config/nic-configs/role.j2.yaml Jinja
> template and generates network-environment.yaml based on
> network-environment.j2.yaml which references those templates.
> 
> Note that network-environment.j2.yaml probably can't be combined with
> network-isolation.j2.yaml as every time that environment would need to
> get updated, all data the template needs would need to be provided.
> 
> There are wireframes made by Liz Blanchard currently available [5],
> althought they are not exactly up to date to this proposal. Ideally 
> whole network configuration would happen on a screen based on the
> graphical representation of network [6].
> 
> 
> Any comments to this proposal are very welcome, please note that I am
> not a networking expert so I might be missing something.
> 
> There is a spec [7] in progress aimed for Ocata, but the feature will
> highly probably not land in Ocata, so we'll need to update the spec and
> move it to next cycle.
> 
> 
> [1]
> http://blog.nemebean.com/content/tripleo-network-isolation-template-generator
> 
> [2] https://github.com/openstack/tripleo-heat-templates
> [3]
> https://github.com/openstack/tripleo-heat-templates/blob/master/environments/network-environment.yaml
> 
> [4]
> https://github.com/openstack/tripleo-heat-templates/blob/master/environments/network-isolation.yaml
> 
> [5] https://openstack.invisionapp.com/share/UM87J4NBQ#/screens/179046668
> [6] https://openstack.invisionapp.com/share/UM87J4NBQ#/screens/179046679
> [7] https://review.openstack.org/#/c/396383/
> 
> Thanks
> Jirka
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Jirka,

Thanks for putting this together, I think this is a great summary of
the issue and a good thing to discuss.

I have done put some thought into how we might convert the existing
YAML NIC config templates to Jinja. Most of the templates are pretty
modular if you look closely. For instance, working from the
bond-with-vlans templates, the bridge definition is the same for all nodes:

      - type: ovs_bridge
        name: bridge_name
        dns_servers:
          get_param: DnsServers
        members:
        - type: ovs_bond
          name: bond1
          ovs_options:
            get_param: BondInterfaceOvsOptions
          members:
          - type: interface
            name: nic2
            primary: true
          - type: interface
            name: nic3

Then, we have each of the VLANs, which can be enabled or disabled based
on whether a network is enabled or not on a node. For instance, the
external network might be defined like this:

{% if role.has_net_external %}
        - type: vlan
          device: bond1
          vlan_id:
            get_param: ExternalNetworkVlanID
          addresses:
          - ip_netmask:
              get_param: ExternalIpSubnet
  {% if role.default_route_network == 'external' }
          routes:
          - default: true
            next_hop:
              get_param: ExternalInterfaceDefaultRoute
  {% endif %}
{% endif %}

(note that there might be a better way to test for the external
network, consider the above psuedo-code)

The drawback to this approach is that we need a different template for
each physical configuration. If the controllers have more NICs than the
computes, then that requires a different base configuration. One
potential advantage to this approach is that you could even use one
unified template with different physical ports depending on the role.
For instance:

        - type: ovs_bond
          name: bond1
          ovs_options:
            get_param: BondInterfaceOvsOptions
          members:
          - type: interface
            name: nic2
            primary: true
          - type: interface
            name: nic3
{% if role.name == Controller %}
          - type: interface
            name: nic4
            primary: true
          - type: interface
            name: nic5
{% endif %}

However, the above might be a little difficult to model in a graphical
interface, especially if the configuration is significantly different.

So this would change the workflow slightly. You would first develop a
template that included all the possible networks that could appear in
that physical configuration, then enable them conditionally based on
which networks were assigned to each role. It would also increase the
complexity of developing templates by hand, but it would probably
still be easier to manage one or two complex template than 5 or more
simple configurations.

Since the physical template might be something that we could develop
automatically based on LLDP data collected during introspection, we
could potentially automate all parts of the configuration other than
customizing which networks to use where. Note that while it's easy
to conceptualize automatic template generation based on LLDP data
received from the switch, I also expect this to be pretty
error-prone. For instance, it may be difficult to detect which
interfaces should be part of a bond or bridge. Also, in cases where
a VLAN appears on more than one interface, it isn't easy to figure
out which interface Director should use for which. This could happen,
for instance, when the External VLAN is configured on one interface
for the public API, but it's also being shared on a bridge with many
VLANs trunked for Neutron routers.

[1] - https://openstack.invisionapp.com/share/UM87J4NBQ#/screens/179046679

-- 
Dan Sneddon         |  Senior Principal OpenStack Engineer
dsneddon at redhat.com |  redhat.com/openstack
dsneddon:irc        |  @dxs:twitter