[openstack-dev] [puppet] What's up in our CI?

Emilien Macchi emilien at redhat.com
Thu Feb 25 03:39:24 UTC 2016

On 02/24/2016 09:37 PM, Emilien Macchi wrote:
> On 02/24/2016 09:28 PM, Emilien Macchi wrote:
>> 1/ CI failures
>> For those who contributed to Puppet OpenStack modules over the last
>> days, you might have noticed our CI was pretty unstable.
>> The main reason is that our 2 scenarios are overloaded by the number of
>> services & tests, so they randomly timeout.
>> Here are some actions that should bring back our CI in a good shape:
>> * Add scenario003 and move some services from 001/002 to 003
>>   https://review.openstack.org/284388
>> * Optimize Zuul layout to consume less resources in OpenStack Infra
>>   https://review.openstack.org/284431
>> * Continue to reduce to 2 the number of workers when possible
>>   https://review.openstack.org/284289
>> * Investigate why `gem install r10k` takes 12 minutes on Internap cloud
>>   (instead of a few seconds on other clouds)
>>   https://review.openstack.org/#/c/283696/
>> 2/ scenario001 is the RBD scenario
>> scenario001 is now deploying Glance, Nova, Cinder and Gnocchi with RBD
>> backend (Ceph).
>> We can call it a Compute + Telemetry + Ceph scenario.
>> 3/ What's next
>> Currently in the pipe:
>> * running tempest with plugins
>> * add Zaqar
>> * add Neutron FWaaS, LBaaS
>> * on scenario002: use Swift for Glance backand and Neutron ML2
>> linuxbridge (instead of ML2 OVS).
>> * more (please bring ideas & feedback)
>> Any questions / suggestions are welcome,
> I realize we have a new thing:
> 4/ Canonical updated UCA packaging for Mitaka (last update was in
> December). Our integration jobs running on Trusty are all broken now.
> We're investigating that but if we don't find a fix by tomorrow morning
> I'll propose to disable voting on Ubuntu.

So I've noticed multiple issues, some of them are going to be fixed
soon, but others need further investigation.
We also have chicken & egg problem where we need a patch in
puppet-neutron required by our integration CI to pass, and vice-versa.

I propose we disable voting on Ubuntu integration jobs:

Until we stabilize a little bit.

Once we sort this out:
* we'll re-enable voting for trusty jobs
* I'll cut a Mitaka beta release
Emilien Macchi

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160224/715c97b5/attachment.pgp>

More information about the OpenStack-dev mailing list