Open Stack

On 11 June 2013 08:32, Mark McLoughlin <markmc at redhat.com> wrote:
> Hi
>
> Having metadata in each project describing its non-python requirements
> is definitely a good thing. As is having some way of turning that
> metadata into a "check these deps are satisfied" or "install these deps"
> disto-specific command.
>
> I think it's great you've kicked this off - it's a great starting point
> for finding a workable solution. I'm actually kinda surprised we never
> had e.g. tools/install-deps.sh type scripts long ago.

:) We do in devstack, but eesh thats harsh on a workstation.

> The first thing I notice is that the list of requirements is a list of
> debian packages and versions. Obviously, anyone who doesn't use a debian
> distro wouldn't like to have to cater first for debian and then for
> their preferred distro when adding a dep. I think the project has gone
> beyond having a single first class distro, so I think this needs
> improving.
>
> (I've an obvious bias here - but the concern honestly does go beyond
> just me)

I agree; here are my thoughts on some approaches for this.

A - have a golden list, and describe everything in those terms. When
platforms (debian/ubuntu/fedora/rhel) etc disagree, whoever doesn't
match the golden list has to translate. We can build and ship a global
translation list, as well as per-project translation lists.

B - have platform specific rules in other-requirements.txt. e.g.
linux-image-generic [platform:ubuntu]
linux-image [platform:debian]
kernel [platform:rpm]

In diskimage-builder we took approach A and it's working well - we
support Fedora easily using it.

For bindep I've design in *both* A and B from the get-go. B is working
right now, but A is missing a way to describe additional mappings
without changing the bindep code (e.g. embedding extra metadata in
other-requirements.txt, or a new file, or $whatever).

> What might work is if we built in a dependence on the
> openstack/requirements repo. Like for python deps, we'd use that repo to
> describe the unified set of dependencies for all of openstack and
> individual projects would specify a subset of those dependencies.

I think thats a good idea in it's own right, but orthogonal to the
mapping problem? How is it relevant to the mapping problem - what have
I missed?

> We could also use openstack/requirements to maintain a mapping of distro
> agnostic requirement name/version to distro specific package
> name/version. Maybe bindep would look use the distro mapping table to
> apply other-requirements.txt or maybe we'd have a utility to copy the
> appropriate parts of the mapping table into other-requirements.txt.
>
> Bikeshed - other-requirements could be system-requirements.txt or
> distro-requirements.txt ?

Certainly, I have no strong attachments here. Blue is fine :). What I
will say is that if 'distro agnostic' means 'pick a name *no* distro
is using if *any distro deviates from another distro*', then that way
lies a lot of pain and burden. I'd *much* rather say 'distro agnostic
means Debian' or 'distro agnostic means Ubuntu', not to "bless" such a
distro as far as OpenStack is concerned, but to just give us a stable
and comprehensive list of packages as a starting point.

> Anyway, just some ideas ...
>
>> I think this could live very happily as an oslo component, like pbr
>> does. It provides a small CLI, and a Python library which can be used
>> from things like pbr or CI scripts, if they want to layer on it.
>
> It could come under oslo, but I'd like to see it gain a good deal of
> traction first. Obviously adding other-requirements.txt to projects
> would be a good first step. Using it in devstack would be next?

Sure. If folk are happy using non-integrated things in (integrated
things|the gate), then I'll be delighted to move this onto stackforge
and start submitting patches here and there.

> Basically, I think this could move to stackforge and we could see how it
> evolves before moving it to oslo.
>
> Why wait?
>
> Well, if we include it in oslo, it will imply some interface stability
> guarantees that will make it harder to evolve the idea.
>
> Also, oslo-core is already finding the review workload pretty hard to
> keep on top of ... so I'm reluctant to pull in another project which is
> very new and isn't in use anywhere yet. We have lots of work with our
> core focus on the project's copy-and-paste technical debt.
>
> So, again - I'm up for including it oslo at some point, but let's get it
> in use in a bunch of places first.

Cool, sure thing.

-Rob
-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services