Open Stack

On Thu, Aug 20, 2015, at 09:33 AM, Clayton O'Neill wrote:
> We're using nodepool for our internal CI process and find it to be a
> great
> tool.  We've recently made a change to our development environment build
> tools to allow us to test VRRP load balancing and overlay networking
> inside
> of instances.  We use Vagrant for building those dev environments and
> we've
> extended it to selectively turn off Neutron port security, when needed,
> to
> enable these features to work.
> 
> Now we find ourselves wanting to do the same thing as part of our
> multi-node integration test that we use as a gate for deployments.  Jim
> and
> I discussed this briefly at the Operator's Mid-cycle and he suggested I
> get
> some feedback before trying to implement new functionality to fix this.
> Right now I've been able to come up with three general approaches:
> 
>    1. Implement a new mechanism similar in theory to the existing "ready"
>    script functionality.  The differences would be that it would allow
>    running
>    a program on the nodepool server instead of the instance that was
>    newly
>    created.  This would set environment variables to allow the external
>    script
>    to access the configured provider and make changes via API or CLI
>    clients
>    as needed before the node was added to Jenkins.
>    2. Add the ability to disable port security support directly into
>    Nodepool.  This has the advantage that it would automatically use the
>    existing API rate limiting in nodepool.  However, it's not clear if
>    the
>    functionality we're looking for would be useful for any other existing
>    users of nodepool, and this may be more work.
>    3. Add an extension point using stevedore (or something similar?) that
>    would allow writing plugins for this sort of behavior.  Initially I'd
>    probably only add a "node ready" extension point, but it seems like
>    this
>    would be easy to extend.
> 
> Not having delved into any of these approaches deeply, I'm not sure which
> is the least amount of work but I suspect it is the first option.  If
> people feel strongly that this sort of thing might not be accepted into
> nodepool, or have a strong opinion on a specific approach, I'd be glad to
> hear it before writing any code.
>
I'd like to back up a bunch and think about what is necessary to do
things like VRRP on multinode testing before we worry about nodepool.
You need shared L2 connectivity between the hosts, this is not something
that neutron provides to us in either of our clouds so disabling port
security won't help at all. (Also note that we completely open up the
security groups for all of our test nodes and manage iptables firewalls
locally on the nodes instead, is this not the same as disabling port
security?).

Because we don't have shared L2 between nodes on the multinode tests and
things like nova net and neutron expect it we set up at least one
overlay L2 network using OVS and GRE between all of the hosts running
devstack. You should be able to run VRRP over an overlay like this in
order to get L2 connectivity and avoid any neutron port security.

This is clearly a semi common need and not something that is easily
fixed by nodepool so my preference would be to continue using overlays
as we do today rather than make nodepool more complicated. Will this
meet your needs?

The overlay setup code can be found at
https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/functions.sh#n907
if you want to look through what is currently done.

Hope this helps and happy to answer questions about the existing setup
and how it works around nested cloud problems.

Clark