[openstack-dev] supported dependency versioning and testing

Mark McLoughlin markmc at redhat.com
Fri Feb 21 13:33:09 UTC 2014


On Thu, 2014-02-20 at 10:31 -0800, Joe Gordon wrote:
> Hi All,
> 
> I discussion recently came up inside of nova about what it means
> supported version for a dependency means.  in libvirt we gate on the
> minimal version that we support but for all python dependencies we
> gate on the highest version that passes our requirements. While we all
> agree that having two different ways of choosing which version to test
> (min and max) is bad, there are good arguments for doing both.
> 
> testing most recent version:
>     * We want to make sure we support the latest and greatest
>     * Bug fixes
>     * Quickly discover backwards incompatible changes so we can deal
> with them as they arise instead of in batch
> 
> Testing lowest version supported:
>     * Make sure we don't land any code that breaks compatibility with
> the lowest version we say we support

Good summary.

> A few questions and ideas on how to move forward.
>     * How do other projects deal with this? This problem isn't unique
> in OpenStack.
>     * What are the issues with making one gate job use the latest
> versions and one use the lowest supported versions?

I think this would be very useful.

Obviously it would take some effort on someone's part to set this up
initially, but I don't immediately see it being much of an ongoing
burden on developers.

>     * Only test some things on every commit or every day (periodic
> jobs)? But no one ever fixes those things when they break? who wants
> to own them? distros? deployers?

The gate job above would most likely lead to us trying hard to maintain
support for older.

A periodic job would either go stale or we'd keep it going simply by
dropping support for older libraries. (Maybe that's ok)

>     * Other solutions?
>     * Does it make sense to gate on the lowest version of libvirt but
> the highest version of python libs?

We might be unconsciously drawing a platform vs app line here - that
libvirt is part of the platform and the python library stack is part of
our app - while still giving a nod to supporting the notion that the
python library stack is part of the platform.

Put it another way - we'd be wary of dropping support for an older
libvirt (because it rules out support for a platform) but not so much
with dropping support for an older python library (because meh, it's not
*really* part of the platform).

Or another way, we give explicit guidance about what exact versions of
libvirt we support (i.e. test with specific distros and whatever
versions of libvirt they have) and leave those versions newer than the
oldest version we explicitly support as a grey area. Similarly, we give
explicit guidance about the exact python library stack we support (i.e.
what we test now in the gate) and leave the older versions as a grey
area.

Perhaps rather than focusing on making this absolutely black and white,
we should focus on better communicating what we actually focus our
testing on? (i.e. rather than making the grey areas black, improve the
white areas)

Concretely, for every commit merged, we could publish:

  - the set of commits tested
  - details of the jobs passed:
      - the distro
      - installed packages and versions
      - output of pip freeze
      - configuration used
      - tests passed

Meh, feeling like I'm going off-topic a bit.

Mark.




More information about the OpenStack-dev mailing list