Open Stack

I agree with Flavio. This looks really cool, and I had a very similar idea
recently. I'll try to find some time to give this a whirl.

-Dave


On Fri, Sep 5, 2014 at 10:37 AM, Flavio Percoco <flavio at redhat.com> wrote:

> On 09/05/2014 04:21 PM, Monty Taylor wrote:
> > On 09/05/2014 06:32 AM, Sean Dague wrote:
> >> While reviewing this zookeeper service group fix in Nova -
> >> https://review.openstack.org/#/c/102639/ it was exposed that the
> >> zookeeper tests aren't running in infra.
> >>
> >> The crux of the issue is that zookeeper python modules are C extensions.
> >> So you have to either install from packages (which we don't do in unit
> >> tests) or install from pip, which means forcing zookeeper dev packages
> >> locally. Realistically this is the same issue we end up with for mysql
> >> and pg, but given their wider usage we just forced that pain on
> >> developers.
> >>
> >> But it seems like a bad stand off between testing upstream and testing
> >> normal path locally.
> >>
> >> Big picture it would be nice to not require a ton of dev libraries
> >> locally for optional components, but still test them upstream. So that
> >> in the base case I'm not running zookeeper locally, but if it fails
> >> upstream because I broke something in zookeeper, it's easy enough to
> >> spin up that dev env that has it.
> >>
> >> Which feels like we need some decoupling on our requirements vs. tox
> >> targets to get there. CC to Monty and Clark as our super awesome tox
> >> hackers to help figure out if there is a path forward here that makes
> >> sense.
> >
> > Funny story - I've come to dislike what we're doing here, so I've been
> > slowly working on an idea in this area:
> >
> > https://github.com/emonty/dox
> >
> > The tl;dr is "it's like tox, except it uses docker instead of
> > virtualenv" - which means we can express all of our requirements, not
> > just pip ones.
> >
> > It's not quite ready yet - although I'd be happy to move it in to
> > stackforge or even openstack-dev and get other people hacking on it with
> > me until it is. The main problem that needs solving, I think, is how to
> > sanely express multiple target environments (like py26,py27) without
> > making a stupidly baroque config file. OTOH, tox's insistence of making
> > a new virtualenv for each "environment" is heavyweight and has led to
> > some pretty crazy hacks across the project. Luckily, docker itself does
> > an EXCELLENT job at handling caching and reuse - so I think we can have
> > a set of containers that something in infra (waves hands) publishes to
> > dockerhub, like:
> >
> >   infra/py27
> >   infra/py26
> >
> > And then have things like nova build on those, like:
> >
> >   infra/nova/py27
> >
> > Which would have zookeeper as well
> >
> > The _really_ fun part, again, if we can figure out how to express it in
> > config without reimplementing make accidentally, is that we could start
> > to have things like:
> >
> >   infra/mysql
> >   infra/postgres
> >   infra/mongodb
> >
> > And have dox files say things like:
> >
> >   Nova unittests want a python27 environment, this means we want an
> > infra/mysql container, an infra/postgres container and for those to be
> > linked to the infra/nova/py27 container where the tests will run.
> >
> > Since those are all reusable, the speed should be _Excellent_ and we
> > should be able to more easily get more things runnable locally without
> > full devstack.
> >
> > Thoughts? Anybody wanna hack on it with me? I think it could wind up
> > being a pretty useful tool for folks outside of OpenStack too if we get
> > it right.
> >
>
> I think it is sexy - I don't describe ideas/software as sexy that often
> but this one deserves it. I'm interested in helping out.
>
> I'll clone it and give it a try - or at least take a look at it.
>
> Flavio
>
> > Monty
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> --
> @flaper87
> Flavio Percoco
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
David Shrewsbury (Shrews)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140905/9c7f268f/attachment.html>