Open Stack

On Fri, Feb 21, 2014 at 10:17 AM, Sean Dague <sean at dague.net> wrote:
> On 02/21/2014 11:02 AM, Daniel P. Berrange wrote:
>> On Fri, Feb 21, 2014 at 10:46:22AM -0500, Sean Dague wrote:
>>> On 02/21/2014 09:45 AM, Daniel P. Berrange wrote:
>>>> On Thu, Feb 20, 2014 at 02:45:03PM -0500, Sean Dague wrote:
>>>>>
>>>>> So I'm one of the first people to utter "if it isn't tested, it's
>>>>> probably broken", however I also think we need to be realistic about the
>>>>> fact that if you did out the permutations of dependencies and config
>>>>> options, we'd have as many test matrix scenarios as grains of sand on
>>>>> the planet.
>>>>>
>>>>> I do think in some ways this is unique to OpenStack, in that our
>>>>> automated testing is head and shoulders above any other Open Source
>>>>> project out there, and most proprietary software systems I've seen.
>>>>>
>>>>> So this is about being pragmatic. In our dependency testing we are
>>>>> actually testing with most recent versions of everything. So I would
>>>>> think that even with libvirt, we should err in that direction.
>>>>
>>>> I'm very much against that, because IME, time & time again across
>>>> all open source projects I've worked on, people silently introduce
>>>> use of features/apis that only exist in newer versions without anyone
>>>> ever noticing until it is too late.
>>>>
>>>>> That being said, we also need to be a little bit careful about taking
>>>>> such a hard line about "supported vs. not" based on only what's in the
>>>>> gate. Because if we did the following things would be listed as
>>>>> unsupported (in increasing level of ridiculousness):
>>>>>
>>>>>  * Live migration
>>>>>  * Using qpid or zmq
>>>>>  * Running on anything other than Ubuntu 12.04
>>>>>  * Running on multiple nodes
>>>>>
>>>>> Supported to me means we think it should work, and if it doesn't, it's a
>>>>> high priority bug that will get fixed quickly. Testing is our sanity
>>>>> check. But it can't be considered that it will catch everything, at
>>>>> least not before the heat death of the universe.
>>>>
>>>> I agree we should be pragmatic here to some extent. We shouldn't aim to
>>>> test every single intermediate version, or every possible permutation of
>>>> versions - just a representative sample. Testing both lowest and highest
>>>> versions is a reasonable sample set IMHO.
>>>
>>> Testing lower bounds is interesting, because of the way pip works. That
>>> being said, if someone wants to take ownership of building that job to
>>> start as a periodic job, I'm happy to point in the right direction. Just
>>> right now, it's a lower priority item than things like Tempest self
>>> testing, Heat actually gating, Neutron running in parallel, Nova API
>>> coverage.
>>
>> If it would be hard work to do it for python modules, we can at least
>> not remove the existing testing of an old libvirt version - simply add
>> an additional test with newer libvirt.
>
> Simply adding a test with newer libvirt isn't all that simple at the end
> of the day, as it requires building a new nodepool image. Because
> getting new libvirt in the existing test environment means cloud
> archive, and cloud archive means a ton of other new code as well. Plus
> in Juno we're presumably going to jump to 14.04 as our test base, which
> is going to be it's own big transition.
>
> So, I'm not opposed, but I also think bifurcating libvirt testing is a
> big enough change in the pipeline that it needs some pretty dedicated
> folks looking at it, and the implications there in. This isn't just a
> yaml change, set and forget it. And given where we are in the
> development cycle, I'm not sure trying to keep the gate stable with a
> new libvirt which we've known to be problematic, is the right time to do
> this.
>
> But, if someone is stepping up to work through it, can definitely mentor
> them on the right places to be poking.



So it sounds like the consensus here is:

* We should have a uniform policy (unless we take the platform vs app
distinction)
* Long term we want to have a lower bound gate job as well, but that
no one has stepped up to work on it yet
* Setting up libvirt min and libvirt max tests is non-trivial and
needs someone to work on it

So in the short term we shouldn't be forced to to hold libvirt back to
the minimal supported version in
gate(https://blueprints.launchpad.net/nova/+spec/support-libvirt-1x),
hopefully while someone steps up to get a minimal libvirt (and python
deps?) job in the gate.

>
>         -Sean
>
> --
> Sean Dague
> Samsung Research America
> sean at dague.net / sean.dague at samsung.com
> http://dague.net
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>