Open Stack

On 09/05/2014 07:21 AM, Monty Taylor wrote:
> On 09/05/2014 06:32 AM, Sean Dague wrote:
>> While reviewing this zookeeper service group fix in Nova -
>> https://review.openstack.org/#/c/102639/ it was exposed that the
>> zookeeper tests aren't running in infra.
>>
>> The crux of the issue is that zookeeper python modules are C extensions.
>> So you have to either install from packages (which we don't do in unit
>> tests) or install from pip, which means forcing zookeeper dev packages
>> locally. Realistically this is the same issue we end up with for mysql
>> and pg, but given their wider usage we just forced that pain on
>> developers.
>>
>> But it seems like a bad stand off between testing upstream and testing
>> normal path locally.
>>
>> Big picture it would be nice to not require a ton of dev libraries
>> locally for optional components, but still test them upstream. So that
>> in the base case I'm not running zookeeper locally, but if it fails
>> upstream because I broke something in zookeeper, it's easy enough to
>> spin up that dev env that has it.
>>
>> Which feels like we need some decoupling on our requirements vs. tox
>> targets to get there. CC to Monty and Clark as our super awesome tox
>> hackers to help figure out if there is a path forward here that makes
>> sense.
>
> Funny story - I've come to dislike what we're doing here, so I've been
> slowly working on an idea in this area:
>
> https://github.com/emonty/dox
>
> The tl;dr is "it's like tox, except it uses docker instead of
> virtualenv" - which means we can express all of our requirements, not
> just pip ones.
>
> It's not quite ready yet - although I'd be happy to move it in to
> stackforge or even openstack-dev and get other people hacking on it with
> me until it is. The main problem that needs solving, I think, is how to
> sanely express multiple target environments (like py26,py27) without
> making a stupidly baroque config file. OTOH, tox's insistence of making
> a new virtualenv for each "environment" is heavyweight and has led to
> some pretty crazy hacks across the project. Luckily, docker itself does
> an EXCELLENT job at handling caching and reuse - so I think we can have
> a set of containers that something in infra (waves hands) publishes to
> dockerhub, like:
>
>    infra/py27
>    infra/py26
>
> And then have things like nova build on those, like:
>
>    infra/nova/py27
>
> Which would have zookeeper as well
>
> The _really_ fun part, again, if we can figure out how to express it in
> config without reimplementing make accidentally, is that we could start
> to have things like:
>
>    infra/mysql
>    infra/postgres
>    infra/mongodb
>
> And have dox files say things like:
>
>    Nova unittests want a python27 environment, this means we want an
> infra/mysql container, an infra/postgres container and for those to be
> linked to the infra/nova/py27 container where the tests will run.
>
> Since those are all reusable, the speed should be _Excellent_ and we
> should be able to more easily get more things runnable locally without
> full devstack.
>
> Thoughts? Anybody wanna hack on it with me? I think it could wind up
> being a pretty useful tool for folks outside of OpenStack too if we get
> it right.

I'd like to follow up on this real quick - just to set some expectations 
appropriately...

Firstly, I'm super thrilled that folks are interested in hacking on this 
- and excited that people have piled on. It's extremely exciting.

I want to make sure people know that goal number 1 of this is to allow 
for a tox-like experience for developers on their laptops that isn't 
specific to python. We have growing numbers of folks in our numbers who 
work on things that are not python, and there are a ton of 
non-OpenStack/non-Python folks out there working on projects that could 
potentially benefit from something that does for them what tox has so 
far done for us.

Additionally, I personally am bothered by the fact that as a project we 
declare quite strongly what version of libraries we use that are 
_python_ but have no current way to do the same for non-python 
libraries. I may draw the ire ultimately from the distros from where I 
want my stance on this to get - but I would like very much for us to be 
able to stop caring about what version of libvirt is _Currently_ in 
Fedora, RHEL or Ubuntu and get to a point where we _tell_ people that to 
use OpenStack Kilo they'll need libvirt v7 or something. I think that's 
a ways off, but I personally want to get there nonetheless.

That said - although I'm hopeful it will be, I'd like to be clear that 
I'm not convinced this will be a useful tool for any of our automation. 
There are no problems we have in the gate infrastructure that dox is 
aiming at helping us solve. We already have single-use slaves that 
already can install anything we need at any time.

There are at least four outcomes I can see for the current dox effort:

a) dox winds up being a simple and powerful tool for developers both 
OpenStack and not to use in their day to day life
b) a is _so_ true that we feel it's an essential part of openstack 
developer tooling, like git-review, and we pull it in from stackforge to 
openstack-infra and make it part of the infra program
c) a is true AND we learn ways in which it makes things better in the gate
d) we hack on it for a while and decide that docker isn't actually very 
useful and scrap the whole thing

It's early enough that I'd like to focus purely on (a) for a while. I 
think it's early enough that I don't want too many eggs to be placed in 
its basket too soon - things get complex around these parts really 
quickly, and I think we'll be well served by letting this one find its feet.

Thanks!
Monty