[openstack-dev] [puppet] today was a bad day for CI

Emilien Macchi emilien at redhat.com
Fri Sep 9 00:15:44 UTC 2016


I'm writing this email because it's end of day for me and I want our
community know the status of our CI.

I spent my day trying to bring our CI back and we found new blockers
almost at every hour:
https://review.openstack.org/367551

1) Add 2GB swap by default
Add some swap when running CI jobs, so we can reduce the number of
failures in CI related to missing memory.
By default 2GB, can be disabled or increased/decreased with parameters.

2) Promote RDO repository to have Oslo DB from master
python-oslo-db 4.13.x releases are known to break Gnocchi.
https://review.openstack.org/367221 will fix it, but it is not yet
in any release, so RDO unpinned oslo.db until that happens.

3) Disable Ironic testing on Ubuntu. Packages are broken in recent
Newton upgrade. They are working on it.

4) Enable br_netfilter kernel module on Ubuntu.
See https://bugs.launchpad.net/cloud-archive/+bug/1621651
Even if it's not critical, it's a nice-to-have because it removed the
ERROR that we had in neutron logs. We'll see how the bug report evolve
and maybe remove this workaround.

5) Export lsmod in logs, easier for us to debug later.

6) Disable linuxbridge on scenario003 for Ubuntu
Also see https://bugs.launchpad.net/cloud-archive/+bug/1621651


And now, the best one (we don't have any fix yet):
Puppetlabs released a new version of Puppet that required an uncertain
version of redhat-release.
http://logs.openstack.org/51/367551/8/check/gate-puppet-openstack-integration-3-scenario001-tempest-centos-7/fc2c291/console.html#_2016-09-09_00_05_47_317332

They created a new dependency on puppetlabs-release that breaks
everything. I reported the bug here:
https://tickets.puppetlabs.com/browse/CPR-378

So CI is now broken and there is nothing more we can do I guess. Any
help from Puppetlabs is welcome.
-- 
Emilien Macchi



More information about the OpenStack-dev mailing list