Open Stack

Hi Simon,

You are absolutely right in your train of thoughts: unless the
third-party CI monitors and vets all the potential changes it cares
about there's always a chance something might break. This is why I
think it's important that each Neutron third party CI should not only
test Neutron changes, but also Nova's, DevStack's and Tempest's.
Filters may be added to test only the relevant subtrees.

For instance, the VMware CI runs the full suite of tempest smoke
tests, as they come from upstream and it vets all the changes that go
in Tempest made to API and scenario tests as well as configuration
changes. As for Nova, we test changes to the vif parts, and for
DevStack, we validate changes made to lib/neutron*.

Vetting all the changes coming in VS only the ones that can
potentially break third-party support is a balancing act when you
don't have infinite resources at your disposal, or you're just ramping
up the CI infrastructure.

Cheers,
Armando

On 4 April 2014 02:00, Simon Pasquier <simon.pasquier at bull.net> wrote:
> Hi Salvatore,
>
> On 03/04/2014 14:56, Salvatore Orlando wrote:
>> Hi Simon,
>>
> <snip>
>>
>> I hope stricter criteria will be enforced for Juno; I personally think
>> every CI should run at least the smoketest suite for L2/L3 services (eg:
>> load balancer scenario will stay optional).
>
> I had a little thinking about this and I feel like it might not have
> caught _immediately_ the issue Kyle talked about [1].
>
> Let's rewind the time line:
> 1/ Change to *Nova* adding external events API is merged
> https://review.openstack.org/#/c/76388/
> 2/ Change to *Neutron* notifying Nova when ports are ready is merged
> https://review.openstack.org/#/c/75253/
> 3/ Change to *Nova* making libvirt wait for Neutron notifications is merged
> https://review.openstack.org/#/c/74832/
>
> At this point and assuming that the external ODL CI system were running
> the L2/L3 smoke tests, change #3 could have passed since external
> Neutron CI aren't voting for Nova. Instead it would have voted against
> any subsequent change to Neutron.
>
> Simon
>
> [1] https://bugs.launchpad.net/neutron/+bug/1301449
>
>>
>> Salvatore
>>
>> [1] https://review.openstack.org/#/c/75304/
>>
>>
>>
>> On 3 April 2014 12:28, Simon Pasquier <simon.pasquier at bull.net
>> <mailto:simon.pasquier at bull.net>> wrote:
>>
>>     Hi,
>>
>>     I'm looking at [1] but I see no requirement of which Tempest tests
>>     should be executed.
>>
>>     In particular, I'm a bit puzzled that it is not mandatory to boot an
>>     instance and check that it gets connected to the network. To me, this is
>>     the very minimum for asserting that your plugin or driver is working
>>     with Neutron *and* Nova (I'm not even talking about security groups). I
>>     had a quick look at the existing 3rd party CI systems and I found none
>>     running this kind of check (correct me if I'm wrong).
>>
>>     Thoughts?
>>
>>     [1] https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers
>>     --
>>     Simon Pasquier
>>     Software Engineer (OpenStack Expertise Center)
>>     Bull, Architect of an Open World
>>     Phone: + 33 4 76 29 71 49 <tel:%2B%2033%204%2076%2029%2071%2049>
>>     http://www.bull.com
>>
>>     _______________________________________________
>>     OpenStack-dev mailing list
>>     OpenStack-dev at lists.openstack.org
>>     <mailto:OpenStack-dev at lists.openstack.org>
>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev