Open Stack

On 09/05/2014 06:32 AM, Sean Dague wrote:
> While reviewing this zookeeper service group fix in Nova -
> https://review.openstack.org/#/c/102639/ it was exposed that the
> zookeeper tests aren't running in infra.
>
> The crux of the issue is that zookeeper python modules are C extensions.
> So you have to either install from packages (which we don't do in unit
> tests) or install from pip, which means forcing zookeeper dev packages
> locally. Realistically this is the same issue we end up with for mysql
> and pg, but given their wider usage we just forced that pain on developers.
>
> But it seems like a bad stand off between testing upstream and testing
> normal path locally.
>
> Big picture it would be nice to not require a ton of dev libraries
> locally for optional components, but still test them upstream. So that
> in the base case I'm not running zookeeper locally, but if it fails
> upstream because I broke something in zookeeper, it's easy enough to
> spin up that dev env that has it.
>
> Which feels like we need some decoupling on our requirements vs. tox
> targets to get there. CC to Monty and Clark as our super awesome tox
> hackers to help figure out if there is a path forward here that makes sense.

Funny story - I've come to dislike what we're doing here, so I've been 
slowly working on an idea in this area:

https://github.com/emonty/dox

The tl;dr is "it's like tox, except it uses docker instead of 
virtualenv" - which means we can express all of our requirements, not 
just pip ones.

It's not quite ready yet - although I'd be happy to move it in to 
stackforge or even openstack-dev and get other people hacking on it with 
me until it is. The main problem that needs solving, I think, is how to 
sanely express multiple target environments (like py26,py27) without 
making a stupidly baroque config file. OTOH, tox's insistence of making 
a new virtualenv for each "environment" is heavyweight and has led to 
some pretty crazy hacks across the project. Luckily, docker itself does 
an EXCELLENT job at handling caching and reuse - so I think we can have 
a set of containers that something in infra (waves hands) publishes to 
dockerhub, like:

   infra/py27
   infra/py26

And then have things like nova build on those, like:

   infra/nova/py27

Which would have zookeeper as well

The _really_ fun part, again, if we can figure out how to express it in 
config without reimplementing make accidentally, is that we could start 
to have things like:

   infra/mysql
   infra/postgres
   infra/mongodb

And have dox files say things like:

   Nova unittests want a python27 environment, this means we want an 
infra/mysql container, an infra/postgres container and for those to be 
linked to the infra/nova/py27 container where the tests will run.

Since those are all reusable, the speed should be _Excellent_ and we 
should be able to more easily get more things runnable locally without 
full devstack.

Thoughts? Anybody wanna hack on it with me? I think it could wind up 
being a pretty useful tool for folks outside of OpenStack too if we get 
it right.

Monty