Open Stack

On Mon, Jun 10, 2013 at 4:32 PM, Mark McLoughlin <markmc at redhat.com> wrote:

> Hi
>
> On Mon, 2013-06-10 at 20:50 +1200, Robert Collins wrote:
> > We have a hole in our otherwise pretty robust dependency management :
> > binary packages. At the moment, the most reliable way to get a dev
> > environment working is to either copy from a wiki page like
> > https://wiki.openstack.org/wiki/DependsOnUbuntu, or to run devstack
> > once :).
> >
> > This is something I've said before we should automate... I was
> > starting to look at tempest in the weekend, and yet again running up
> > the curve of 'what packages are needed for this specific project',
> > when I hit my tolerance threshold and thus, rage-fueled-code.
> >
> > https://github.com/rbtcollins/bindep is a draft project which would
> > allow capturing in machine readable form the dependencies needed for
> > our CI environments, for developers, and for binary deployment. It
> > *isn't* a replacement or wrapper for tools like dpkg and rpm : there
> > are good tools for that already. What it would let us do is record
> > within each project the dependencies the project actually has for each
> > use case : no more patching devstack to educate it about new binary
> > dependencies.
>
> Having metadata in each project describing its non-python requirements
> is definitely a good thing. As is having some way of turning that
> metadata into a "check these deps are satisfied" or "install these deps"
> disto-specific command.
>
> I think it's great you've kicked this off - it's a great starting point
> for finding a workable solution. I'm actually kinda surprised we never
> had e.g. tools/install-deps.sh type scripts long ago.
>
> The first thing I notice is that the list of requirements is a list of
> debian packages and versions. Obviously, anyone who doesn't use a debian
> distro wouldn't like to have to cater first for debian and then for
> their preferred distro when adding a dep. I think the project has gone
> beyond having a single first class distro, so I think this needs
> improving.
>
> (I've an obvious bias here - but the concern honestly does go beyond
> just me)
>
> What might work is if we built in a dependence on the
> openstack/requirements repo. Like for python deps, we'd use that repo to
> describe the unified set of dependencies for all of openstack and
> individual projects would specify a subset of those dependencies.
>
> We could also use openstack/requirements to maintain a mapping of distro
> agnostic requirement name/version to distro specific package
> name/version. Maybe bindep would look use the distro mapping table to
> apply other-requirements.txt or maybe we'd have a utility to copy the
> appropriate parts of the mapping table into other-requirements.txt.
>

Joshua Harlow and I did some work on that as part of Anvil (nee devstackpy)
prior to the Folsom summit. I don't see the Ubuntu configuration files any
more, but the RHEL stuff is still there (
https://github.com/stackforge/anvil/blob/master/conf/distros/rhel.yaml).

It works by defining standard names for expressing dependencies, and then
mapping those names to one or more system packages, just as you describe.
It also included classes for installing and configuring packages on
different platforms. At one point it was a full devstack replacement and
worked on Fedora as well as Ubuntu. It probably does more than would be
required for something like bindep, but there may be bits and pieces that
could be reused.

Doug


>
> Bikeshed - other-requirements could be system-requirements.txt or
> distro-requirements.txt ?
>
> Anyway, just some ideas ...
>
> > I think this could live very happily as an oslo component, like pbr
> > does. It provides a small CLI, and a Python library which can be used
> > from things like pbr or CI scripts, if they want to layer on it.
>
> It could come under oslo, but I'd like to see it gain a good deal of
> traction first. Obviously adding other-requirements.txt to projects
> would be a good first step. Using it in devstack would be next?
>
> Basically, I think this could move to stackforge and we could see how it
> evolves before moving it to oslo.
>
> Why wait?
>
> Well, if we include it in oslo, it will imply some interface stability
> guarantees that will make it harder to evolve the idea.
>
> Also, oslo-core is already finding the review workload pretty hard to
> keep on top of ... so I'm reluctant to pull in another project which is
> very new and isn't in use anywhere yet. We have lots of work with our
> core focus on the project's copy-and-paste technical debt.
>
> So, again - I'm up for including it oslo at some point, but let's get it
> in use in a bunch of places first.
>
> Cheers,
> Mark.
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130610/d3e814de/attachment.html>