Open Stack

On 01/16/2017 11:48 AM, James E. Blair wrote:
> Clint Byrum <clint at fewbar.com> writes:
> 
>> Are the jobs so complicated that you can't just write a dedicated gearman
>> worker to do the translation to/from zk? I mean, gearman is really really
>> really simple for a reason.
>>
>> Just saying.. might be simpler to write a throw-away 500 line python gear
>> worker, than modifying nodepool. And you know how against rewrites I am,
>> but in this case.. it's not a rewrite, but a shim.
> 
> That would be easy, but there are no jobs to reimplement in that way.
> The current version of nodepool is not a gearman worker, rather it is a
> gearman administrative client which inspects the status of the gearman
> queue to infer whether there are enough nodes on-line to satisfy the
> demand based on guesses it makes as to which jobs it expects to run on
> each of the available image types.
> 
> Notably, there is no zuul -> nodepool request process.  That is what
> we're adding in v3.  And with good reason.
> 
> In order for the current version of nodepool to operate, it requires a
> significant amount of state information which it uses in the (rather
> complex) calculations it performs to decide when, where, and which nodes
> to launch.
> 
> I believe that by modifying nodepool to replace the relatively simple
> (thanks to shade) "request a node from the cloud" bit with "request a
> node from v3 nodepool", Monty will be making the minimal change needed
> to create the shim.  Anything else likely involves a change to the
> complex side-channel interactions that nodepool performs now.

FWIW, I believe the specific effort can be thought of in two bits:

1) Replace impl of these methods of provider_manager.ProviderManager

createServer
waitForServer
cleanupServer
waitForServerDeletion

2) Collapse subnode logic (rather than making one request for the
primary node and then a request for each secondary node, nodepool-shim
will need to request len(subnodes+1) nodes in the initial request and
then assign one as the primary and the rest as subnodes in the nodepool
db. It's still not that much code - it is essentially just the code in
Nodepool._run - but won't be _quite_ as trivial as (1)

Based on today's infra meeting, I'll make a branch for this today.