Open Stack

On Mon, Jun 10, 2013 at 5:15 PM, Robert Collins
<robertc at robertcollins.net>wrote:

> On 11 June 2013 08:32, Mark McLoughlin <markmc at redhat.com> wrote:
> > Hi
> >
> > Having metadata in each project describing its non-python requirements
> > is definitely a good thing. As is having some way of turning that
> > metadata into a "check these deps are satisfied" or "install these deps"
> > disto-specific command.
> >
> > I think it's great you've kicked this off - it's a great starting point
> > for finding a workable solution. I'm actually kinda surprised we never
> > had e.g. tools/install-deps.sh type scripts long ago.
>
> :) We do in devstack, but eesh thats harsh on a workstation.
>
> > The first thing I notice is that the list of requirements is a list of
> > debian packages and versions. Obviously, anyone who doesn't use a debian
> > distro wouldn't like to have to cater first for debian and then for
> > their preferred distro when adding a dep. I think the project has gone
> > beyond having a single first class distro, so I think this needs
> > improving.
> >
> > (I've an obvious bias here - but the concern honestly does go beyond
> > just me)
>
> I agree; here are my thoughts on some approaches for this.
>
> A - have a golden list, and describe everything in those terms. When
> platforms (debian/ubuntu/fedora/rhel) etc disagree, whoever doesn't
> match the golden list has to translate. We can build and ship a global
> translation list, as well as per-project translation lists.
>
> B - have platform specific rules in other-requirements.txt. e.g.
> linux-image-generic [platform:ubuntu]
> linux-image [platform:debian]
> kernel [platform:rpm]
>
> In diskimage-builder we took approach A and it's working well - we
> support Fedora easily using it.
>

Are we sure there is always a 1:1 mapping? It seems like we had at least
one case where a dependency was in a single package on one platform but in
split across multiple packages on another. I can't think of that case,
though, so maybe I'm imagining things.


>
> For bindep I've design in *both* A and B from the get-go. B is working
> right now, but A is missing a way to describe additional mappings
> without changing the bindep code (e.g. embedding extra metadata in
> other-requirements.txt, or a new file, or $whatever).
>
> > What might work is if we built in a dependence on the
> > openstack/requirements repo. Like for python deps, we'd use that repo to
> > describe the unified set of dependencies for all of openstack and
> > individual projects would specify a subset of those dependencies.
>
> I think thats a good idea in it's own right, but orthogonal to the
> mapping problem? How is it relevant to the mapping problem - what have
> I missed?
>

Keeping a single golden list with the right version numbers consistent
across all of the projects.

Could we release a package that just contains the list as a data file that
bindep can use? That would keep the dependency list for OpenStack separate
from bindep, making the tool more likely to be reusable and avoiding the
copying problem.


>
> > We could also use openstack/requirements to maintain a mapping of distro
> > agnostic requirement name/version to distro specific package
> > name/version. Maybe bindep would look use the distro mapping table to
> > apply other-requirements.txt or maybe we'd have a utility to copy the
> > appropriate parts of the mapping table into other-requirements.txt.
> >
> > Bikeshed - other-requirements could be system-requirements.txt or
> > distro-requirements.txt ?
>
> Certainly, I have no strong attachments here. Blue is fine :). What I
> will say is that if 'distro agnostic' means 'pick a name *no* distro
> is using if *any distro deviates from another distro*', then that way
> lies a lot of pain and burden. I'd *much* rather say 'distro agnostic
> means Debian' or 'distro agnostic means Ubuntu', not to "bless" such a
> distro as far as OpenStack is concerned, but to just give us a stable
> and comprehensive list of packages as a starting point.
>

I assume we'll want to pin versions of these packages, just as we do with
the Python requirements. If that's the case, we need a mapping list for
each distro anyway. The golden list would map some standard name to the
distro package with a version. The dependency lists in a given project
would include the "generic" name without the version.

Doug


>
> > Anyway, just some ideas ...
> >
> >> I think this could live very happily as an oslo component, like pbr
> >> does. It provides a small CLI, and a Python library which can be used
> >> from things like pbr or CI scripts, if they want to layer on it.
> >
> > It could come under oslo, but I'd like to see it gain a good deal of
> > traction first. Obviously adding other-requirements.txt to projects
> > would be a good first step. Using it in devstack would be next?
>
> Sure. If folk are happy using non-integrated things in (integrated
> things|the gate), then I'll be delighted to move this onto stackforge
> and start submitting patches here and there.
>
> > Basically, I think this could move to stackforge and we could see how it
> > evolves before moving it to oslo.
> >
> > Why wait?
> >
> > Well, if we include it in oslo, it will imply some interface stability
> > guarantees that will make it harder to evolve the idea.
> >
> > Also, oslo-core is already finding the review workload pretty hard to
> > keep on top of ... so I'm reluctant to pull in another project which is
> > very new and isn't in use anywhere yet. We have lots of work with our
> > core focus on the project's copy-and-paste technical debt.
> >
> > So, again - I'm up for including it oslo at some point, but let's get it
> > in use in a bunch of places first.
>
> Cool, sure thing.
>
> -Rob
> --
> Robert Collins <rbtcollins at hp.com>
> Distinguished Technologist
> HP Cloud Services
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130610/2f00ffc9/attachment.html>