Open Stack

On 01/13/2017 11:32 AM, Clark Boylan wrote:
> On Fri, Jan 13, 2017, at 08:18 AM, Monty Taylor wrote:
>> Hey all,
>>
>> Part of the rollout plan for zuul v3 involves rolling out the new
>> zookpeer-based nodepool launchers before we roll out zuul v3 itself.
>> We've mostly spoken about this in a hand-wavy mannr so far, but I think
>> we may have a fairly simple answer of how to approach it - so I'd like
>> to propose the following:
>>
>> * Make a branch of current nodepool master that we don't intend to merge
>> back ever.
>>
>> * Replace the OpenStack interactions in the provider_manager with zk api
>> calls.
>>
>> * Make a copy of our nodepool.yaml file that has min-ready set to zero
>> for everything.
>>
>> * Run a copy of nodepool from the branch pointed at the new v3
>> zookeeper-based nodepool.
>>
>> This should allow the shim nodepool to make real-time requests for nodes
>> of the new nodepool and attach them to the 2.5 ansiblelaunchers. It
>> makes the v3 nodepool the system of record. Once that's all in place, we
>> should be able to also roll out a zuul v3 installation that is also
>> pointed at the new nodepool, and have both shim-nodepool and zuul v3 be
>> clients of nodepool v3.
>>
>> Once we've finished migrating to zuul v3, we'll just delete the shim
>> server.
>>
>> Sound good to everyone?
> 
> I'll admit I had to read this email several times before I understood
> what you are planning to do. To summarize in case it helps anyone else
> the idea is to have a throwaway version of nodepool that sits between
> zuul and the new zookeeper based nodepool launchers. This throwaway
> nodepool will read in demand from gearman and instead of running
> OpenStack api calls to clouds to fulfill that demand will make launch
> requests of the new zookeeper based nodepool.

Yes - that's a great summary.

> Just to throw the idea out there would an alternative daemon that
> imported nodepool's demand and allocation calculator and zuul's
> zookeeper request abstraction (however that turns out) make sense? This
> way you don't have to map nodepool requests through an abstraction for
> the various OpenStack APIs. And you'd be running the actual code that
> zuul will be using. (Not sure which would be simpler/quicker to get
> working though).

That was actually my original plan - but the demand and allocation stuff
is all implemented, and the "get a node" part of the provider manager
API is actually a really small surface area, so my hypothesis is that
just replacing the node launching api parts with the zk is less work.
However, if starting to implement it winds up getting hairy, I think
pivoting to alternative daemon is certainly the other sane choice.