Open Stack

Greetings,

The TripleO squads for CI and Tempest have just completed Sprint 12.  The
following is a summary of activities during this sprint.   Details on our
team structure can be found in the spec [1].

---

# Sprint 12 Epic (CI): Libvirt Reproducer

* Epic Card: https://trello.com/c/JEGLSVh6/51-reproduce-ci-jobs-with-libvirt
* Tasks: http://ow.ly/O1vZ30jTSc3

"Allow developers to reproduce a multinode CI job on a bare metal host
using libvirt"
"Enable the same workflows used in upstream CI / reproducer using libvirt
instead of OVB as the provisioning mechanism"

The CI Squad prototyped, designed, and implemented new functionality for
our CI reproducer.   “Reproducers” are scripts generated by each CI job
that allow the job/test to be recreated.  These are useful to both CI team
members when investigating failures, as well as developers creating
failures with the intent to iteratively debug and/or fix issues.  Prior to
this sprint, the reproducer scripts supported reproduction of upstream CI
jobs using OVB, typically on RDO Cloud.  This sprint we extended this
capability to support reproduction of jobs in libvirt.

This work was done for a few reasons:

* (short term) enable the team to work on upgrades and other CI team tasks
more efficiently by mitigating recurring RDO Cloud infrastructure issues.
This was the primary motivator for doing this work at this time.
* (mid-longer term) enhance / enable iterative workflows such as THT
development, debugging deployment scenarios, etc.  Snapshots in particular
have proven quite useful.  As we look towards a future with a viable
single-node deployment capability, libvirt has clear benefits for common
developer scenarios.

It is expected that further iteration and refinement of this initial
implementation will be required before the tripleo-ci team is able to
support this broadly.  What we’ve done works as designed.  While we welcome
folks to explore, please note that we are not announcing a supported
libvirt reproducer meant for use outside the tripleo-ci team at this time.
We expect some degree of change, and have a number of RFE’s resulting from
our testing as well as documentation patches that we’re iterating on.

That said, we think it’s really cool, works well in its current form, and
are optimistic about its future.

## We did the following (CI):

* Add support to the reproducer script [2,3] generated by CI to enable
libvirt.
* Basic snapshot create/restore [4] capability.
* Tested Scenarios: featureset 3 (UC idem), 10 (multinode containers), 37
(min OC + minor update).  See sprint cards for details.
* 14-18 RFE’s identified as part of testing for future work
http://ow.ly/J2u830jTSLG

---

# Sprint 12 Epic (Tempest):

* Epic Card: https://trello.com/c/ifIYQsxs/75-sprint-12-undercloud-tempest
* Tasks: http://ow.ly/GGvc30jTSfV

“Run tempest on undercloud by using containerized and packaged tempest”
“Complete work items carried from sprint 11 or another side work going on.”

## We did the following (Tempest):

* Create tripleo-ci jobs that run containerized tempest on all stable
branches.
* Create documentation for configuring and running tempest using
containerized tempest on UC @tripleo.org, and blog posts. [5,6,7]
* Run certification tests via new Jenkins job using ansible role [8]
* Refactor validate-tempest CI role for UC and containers

---

# Ruck and Rover

Each sprint two of the team members assume the roles of Ruck and Rover
(each for half of the sprint).

* Ruck is responsible to monitoring the CI, checking for failures, opening
bugs, participate on meetings, and this is your focal point to any CI
issues.
* Rover is responsible to work on these bugs, fix problems and the rest of
the team are focused on the sprint. For more information about our
structure, check [1]

## Ruck & Rover (Sprint 12), Etherpad [9,10]:

* Quique Llorente(quiquell)
* Gabriele Cerami (panda)

A few notable issues where substantial time was spent were:

1767099 periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset030-master
vxlan tunnel fails randomly
1758899 reproducer-quickstart.sh building wrong gating package.
1767343 gate tripleo-ci-centos-7-containers-multinode fails to update
packages in cron container
1762351
periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-queens-upload is
timeout Depends on https://bugzilla.redhat.com/show_bug.cgi?id=1565179
1766873 quickstart on ovb doesn't yield a deployment
1767049 Error during test discovery : 'must specify exactly one of host or
intercept' Depends on https://bugzilla.redhat.com/show_bug.cgi?id=1434385
1767076 Creating pingtest_sack fails: Failed to schedule instances:
NoValidHost_Remote: No valid host was found
1763634 devmode.sh --ovb fails to deploy overcloud
1765680 Incorrect branch used for not gated tripleo-upgrade repo

If you have any questions and/or suggestions, please contact us in #oooq or
#tripleo

Thanks,

Matt


tq: https://github.com/openstack/tripleo-quickstart
tqe: https://github.com/openstack/tripleo-quickstart-extras

[1]
https://specs.openstack.org/openstack/tripleo-specs/specs/policy/ci-team-structure.html
[2] {{tq}}/roles/libvirt/setup/overcloud/tasks/libvirt_nodepool.yml
[3]
{{tqe}}/roles/create-reproducer-script/templates/reproducer-quickstart.sh.j2#L50
[4] {{tqe}}/roles/snapshot-libvirt
[5]
https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/12/html-single/openstack_integration_test_suite_guide
[6]
https://blogs.rdoproject.org/2018/05/running-tempest-tests-against-a-tripleo-undercloud
[7]
https://blogs.rdoproject.org/2018/05/consuming-kolla-tempest-container-image-for-running-tempest-tests

[8] https://github.com/redhat-cip/ansible-role-openstack-certification
[9] https://review.rdoproject.org/etherpad/p/ruckrover-sprint12
[10] https://etherpad.openstack.org/p/rover-030518
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180508/e977d1fa/attachment.html>