Open Stack

Thanks Salvatore and Kyle for your feedback.

Kyle, you're right, my question has been kicked off by the ML2 ODL bug.
I didn't want to point fingers but rather understand the mid/long-term
plan for 3rd party testing. I'm happy to see that this is taken into
account and hopefully the Juno cycle will provide time to implement the
correct level of testing.

Regards,

Simon

On 03/04/2014 15:26, Kyle Mestery wrote:
> I agree 100% on this in fact. One of the other concerns I have with
> the existing 3rd party
> CI systems is that, other than the "audit" review Salvatore mentions,
> who is ensuring
> they continue to run ok? Once they've been given voting rights, is
> anyone auditing these
> to ensure they continue to function ok?
> 
> I suspect also that Simon is referring to the ODL ML2 MechanismDriver,
> which was broken
> with this commit [1] pushed in at the very end of Icehouse, and in
> fact is still broken unless
> you use the wonky workaround of telling Nova that VIF plugging isn't
> fatal and give it a timeout
> to wait. Better CI for ODL would have caught this, but I'm still
> somewhat saddened this was
> merged so late because now ODL is broken by default and the work to
> fix this is turning out
> to be more challenging than initially thought. :(
> 
> Thanks,
> Kyle
> 
> [1] https://review.openstack.org/#/c/75253/
> 
> On Thu, Apr 3, 2014 at 7:56 AM, Salvatore Orlando <sorlando at nicira.com> wrote:
>> Hi Simon,
>>
>> I agree with your concern.
>> Let me point out however that VMware mine sweeper runs almost all the smoke
>> suite.
>> It's been down a few days for an internal software upgrade, so perhaps you
>> have not seen any recent report from it.
>>
>> I've seen some CI systems testing as little as tempest.api.network.
>> Since a criterion on the minimum set of tests to run was not defined prior
>> to the release cycle, it was also not ok to enforce it once the system went
>> live.
>> The only thing active at the moment is a sort of purpose built lie detector
>> [1].
>>
>> I hope stricter criteria will be enforced for Juno; I personally think every
>> CI should run at least the smoketest suite for L2/L3 services (eg: load
>> balancer scenario will stay optional).
>>
>> Salvatore
>>
>> [1] https://review.openstack.org/#/c/75304/
>>
>>
>>
>> On 3 April 2014 12:28, Simon Pasquier <simon.pasquier at bull.net> wrote:
>>>
>>> Hi,
>>>
>>> I'm looking at [1] but I see no requirement of which Tempest tests
>>> should be executed.
>>>
>>> In particular, I'm a bit puzzled that it is not mandatory to boot an
>>> instance and check that it gets connected to the network. To me, this is
>>> the very minimum for asserting that your plugin or driver is working
>>> with Neutron *and* Nova (I'm not even talking about security groups). I
>>> had a quick look at the existing 3rd party CI systems and I found none
>>> running this kind of check (correct me if I'm wrong).
>>>
>>> Thoughts?
>>>
>>> [1] https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers
>>> --
>>> Simon Pasquier
>>> Software Engineer (OpenStack Expertise Center)
>>> Bull, Architect of an Open World
>>> Phone: + 33 4 76 29 71 49
>>> http://www.bull.com
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>