[openstack-dev] [puppet] today was a bad day for CI

Emilien Macchi emilien at redhat.com
Fri Sep 9 01:07:44 UTC 2016


On Thu, Sep 8, 2016 at 8:15 PM, Emilien Macchi <emilien at redhat.com> wrote:
> I'm writing this email because it's end of day for me and I want our
> community know the status of our CI.
>
> I spent my day trying to bring our CI back and we found new blockers
> almost at every hour:
> https://review.openstack.org/367551
>
> 1) Add 2GB swap by default
> Add some swap when running CI jobs, so we can reduce the number of
> failures in CI related to missing memory.
> By default 2GB, can be disabled or increased/decreased with parameters.
>
> 2) Promote RDO repository to have Oslo DB from master
> python-oslo-db 4.13.x releases are known to break Gnocchi.
> https://review.openstack.org/367221 will fix it, but it is not yet
> in any release, so RDO unpinned oslo.db until that happens.
>
> 3) Disable Ironic testing on Ubuntu. Packages are broken in recent
> Newton upgrade. They are working on it.
>
> 4) Enable br_netfilter kernel module on Ubuntu.
> See https://bugs.launchpad.net/cloud-archive/+bug/1621651
> Even if it's not critical, it's a nice-to-have because it removed the
> ERROR that we had in neutron logs. We'll see how the bug report evolve
> and maybe remove this workaround.
>
> 5) Export lsmod in logs, easier for us to debug later.
>
> 6) Disable linuxbridge on scenario003 for Ubuntu
> Also see https://bugs.launchpad.net/cloud-archive/+bug/1621651
>
>
> And now, the best one (we don't have any fix yet):
> Puppetlabs released a new version of Puppet that required an uncertain
> version of redhat-release.
> http://logs.openstack.org/51/367551/8/check/gate-puppet-openstack-integration-3-scenario001-tempest-centos-7/fc2c291/console.html#_2016-09-09_00_05_47_317332
>
> They created a new dependency on puppetlabs-release that breaks
> everything. I reported the bug here:
> https://tickets.puppetlabs.com/browse/CPR-378
>
> So CI is now broken and there is nothing more we can do I guess. Any
> help from Puppetlabs is welcome.

Puppetlabs was super responsive and they are fixing the problem here:
https://github.com/puppetlabs/puppetlabs-release/pull/71

In the meantime, we will pin the package in our CI and it should work again.
-- 
Emilien Macchi



More information about the OpenStack-dev mailing list