Open Stack

On 05/10/2016 08:54 AM, Vladimir Eremin wrote:
> Hi,
> 
> I've investigated status of nodepool multi node testing and fuel-qa
> approaches, and I wanna share my opinion on moving Fuel testing on
> OpenStack and nodepool.

Awesome! This is a great writeup - and hopefully will be useful as we
validate our theory that zuul v3 should provide a richer environment for
doing complex things like fuel testing than the current multi-node work.

> Our CI pipeline consists of next stages:
> 
> 1. Artifact building and publishing
> 2. QA jobs:
> 2.1. Master node installation from ISO
> 2.2. Slave nodes provisioning
> 2.3. Software deployment
> 2.4. Workload verification
> 
> Current upstream nodepool limitations are pre-spwaned nodes, small
> flavors and, only l3 connectivity. Also, we have no PXE booting and VLAN
> trunking in OpenStack itself. So, the main problem with moving this
> pipeline to nodepool is to emulate IT tasks: installation from ISO and
> nodes provisioning.
> 
> Actually the point is: to test Fuel and test rest of OpenStack
> components against Fuel we mostly need to test stage artifact building,
> deployment and verification. So we need to make Fuel installable from
> packages and create overlay L2 networking. I've found no unsolvable
> problems right now to check most of scenarios with this approach.



> Besides artifact building step, there are next actions items to do to
> run Fuel QA test:
> 
> 1. Automate overlay networking setup. I've
> used https://www.tinc-vpn.org/ as a L2 switching overlay, but OpenVPN
> could be tool of choice. Action items:
>  - overlay networking setup should be integrated in fuel-devops

There is overlay work in the multi-node stuff for devstack. I believe
clarkb has a todo-list item to make that networking setup more general
and more generally available. (it's currently done in devstack-gate
script) I'm not sure if you saw that or if it's suitable for what you
need? If not, it would be good to understand deficiencies.

> 2. Automate Fuel master node codebase installation from packages,
> including repo adding and deployment. Action items:
> - installation should be integrated in fuel-devops or nodepool infra
> - make bootstrap scripts working with more than one network on master
> node ("Bringing down ALL network interfaces except...")
> - fix iptables and ssh for underlay networking

We've talked a few times about handling packages and repos of packages
for patches that have not yet landed, but have done exactly zero work on
it. Since you're a concrete use case, perhaps we can design things with
you in mind.

> 3. Automate Fuel slave node codebase installation and node enrollment.
> Action items:
> - nailgun-agent installation should be integrated in fuel-devops or
> nodepool infra
> - mcollective and ssh keys setup should be automated
> - nailgun ang/or astute should be extended to allow pre-provisioned
> nodes enrollment (I'm doing this part now)
> - nailgun-agent and l23network should support overlay network interfaces

Exciting. I look forward to working on this with you - there are fun
problems in here. :)