[openstack-dev] [Fuel] Current Fuel CI status and issues related to CentOS7 migration

Sergii Golovatiuk sgolovatiuk at mirantis.com
Mon Dec 7 16:08:10 UTC 2015


Hi,


On Mon, Dec 7, 2015 at 4:29 PM, Aleksandra Fedorova <afedorova at mirantis.com>
wrote:

> Hi everyone,
>
> let me describe CI status and changes in CI caused by CentOS7
> migration. Please read carefully as your local and custom test
> environments might be affected as well.
>
>
> * CentOS7 in docker with aufs on Ubuntu Trusty host is not stable
>
> Containers of such type are required for Fuel ISO building process,
> but this configuration is extremely sensitive to the versions of
> kernel and docker packages.
>
> ISO builds fail on 3.16 kernel or docker 1.6.1, see [1].
>
> The current known-to-work configuration is:
>
>       kernel 3.19
>       docker 1.6.2
>
> We have updated main build slaves to this configuration, so now
> community build with CentOS7 is available at
>
>    https://ci.fuel-infra.org/view/ISO/


There are some plans to remove docker from master node completely. I think
we should force that action.

>
>
> * fuel-web is not compatible with python2.6
>
> verify-fuel-web job, which was used to test fuel-web against python2.6
> is now disabled. Work on cleaning up other jobs is currently in
> progress in [2]
>
> * fuel-library deployment tests have been refactored and now use
> perestroika build scripts, see [3], to build fuel-library packages.
>
> This is a huge step forward for Fuel CI, as we simplified job
> configurations and configured deployment tests to use common template
> [4]. This template can also be used for deployment tests for fuel-ostf
> and other Fuel repositories.
>
> * regression in upstream Ubuntu kernel 3.13.0-72 [5]
>
> This issue is not related to CentOS7 merge but appeared in the same
> timeframe and affected the testing process.
>
> Deployment tests use external Ubuntu repository to install base OS on
> OpenStack nodes. In nightly tests we use daily snapshots of
> archive.ubuntu.com. You can find them at
> http://mirror.fuel-infra.org/pkgs/
>
> It appears that on 3rd of December new kernel was uploaded to
> trusty-proposed repo which caused certain issues with LVM and leads to
> Smoke failures.
>
> For now to unblock CI we've pinned Ubuntu mirror used in tests to
> known "good" snapshot with 3.13.0-71. See [6].
>
> Please use this snapshot when running local or custom tests
>
>   http://mirror.fuel-infra.org/pkgs/ubuntu-2015-12-02-170158/
>
> or disable trusty-proposed repository till issue is resolved.
>

It's very dangerous to disable proposed, as it will break client's
deployments when package is moved from proposed to updates. There is a
standard process to block the package flow from proposed to updates, so
let's use it


>
>
> [1] https://bugs.launchpad.net/mos/+bug/1522788
> [2] https://bugs.launchpad.net/fuel/+bug/1523514
> [3]
> https://github.com/fuel-infra/jenkins-jobs/blob/a15c63461bddf857e0548d8b861bd4154e5f8f3a/servers/fuel-ci/builders/build-pkgs.sh#L59
> [4]
> https://github.com/fuel-infra/jenkins-jobs/blob/a15c63461bddf857e0548d8b861bd4154e5f8f3a/servers/fuel-ci/build-and-test-pkgs.yaml
> [5] https://bugs.launchpad.net/fuel/+bug/1523092
> [6]
> https://github.com/fuel-infra/jenkins-jobs/commit/6745eef4964a8c365178953f5f5f45524bfeeb19
>
> --
> Aleksandra Fedorova
> Fuel CI Team Lead
> bookwar
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151207/54176e77/attachment.html>


More information about the OpenStack-dev mailing list