[openstack-dev] [tripleo] Contributing to TripleO is challenging

Emilien Macchi emilien at redhat.com
Mon Mar 21 14:19:47 UTC 2016


On Mon, Mar 21, 2016 at 9:59 AM, Steven Hardy <shardy at redhat.com> wrote:
> On Mon, Mar 21, 2016 at 09:41:42AM -0400, Emilien Macchi wrote:
>> On Mon, Mar 21, 2016 at 6:57 AM, Steven Hardy <shardy at redhat.com> wrote:
>> > On Fri, Mar 18, 2016 at 01:27:33PM +0000, Arkady_Kanevsky at DELL.com wrote:
>> >>    Emilien,
>> >>
>> >>    Agree on the rant. But not clear on concrete proposal to fix it.
>> >>
>> >>    Spend more time “fixing” CI and use Tempest as a gate is a bit wage.
>> >>
>> >>    Unless we test known working version of each project in TripleO CI you are
>> >>    dependent on health of other components.
>> >
>> > I've so far resisted replying to this thread, because while valid, many of
>> > the concerns expressed by Emilien are quite general complaints, and it's
>> > hard to reply with specific solutions.
>> >
>> > However work *is* going on to improve many of these problems, let's see if
>> > I can provide a summary, to clarify the various "concrete proposals" which
>> > do exist.
>> >
>> > 1. Core team & review velocity
>> >
>> > We've had a small and very overloaded core team for a while now, and this
>> > will be helped by expanding our community to include those who've been
>> > regularly contributing excellent work and reviews as core reviewers:
>> >
>> > http://lists.openstack.org/pipermail/openstack-dev/2016-February/087774.html
>> > http://lists.openstack.org/pipermail/openstack-dev/2016-March/089235.html
>> > http://lists.openstack.org/pipermail/openstack-dev/2016-March/089912.html
>> > http://lists.openstack.org/pipermail/openstack-dev/2016-March/089913.html
>> >
>> > Note that I personally think it's absolutely fine for folks to be more
>> > expert in some subsystem and to focus review extra attention on e.g API,
>> > UI, Puppet or whatever.  This subsystem-core model has been well proven in
>> > other projects, and folks will naturally broaden their areas of deeper
>> > knowledge over time.
>> >
>> > Related to this is movement of code, such as the puppet-tripleo refactoring
>> > mentioned by Michael - this has already started, and will help with
>> > providing a cleaner interface between the puppet and heat pieces (which
>> > will also help focus reviewer attention appropriately).
>>
>> Indeed, Michael, Dan & I are working on moving out the Puppet code
>> from THT to puppet-tripleo.
>> That's a nice move, and I appreciate TripleO team support on it.
>>
>> > 2. Day 1 developer experience
>> >
>> > This is closely related to the CI failure rate - there are efforts to
>> > integrate with the RDO tripleo-quickstart tooling, which simplifies the
>> > initial undercloud setup, and potentially makes consuming pre-built,
>> > validated undercloud images (probably output artefacts from our new
>> > periodic CI job) much easier.
>> >
>> > So, this will mean that both developers and CI can potentially be less
>> > regularly impacted by trunk regressions which often cause CI to fail, and
>> > break developer environments.
>> >
>> > https://review.openstack.org/#/c/276810/5
>> >
>> > 3. CI coverage and trunk failure rate
>> >
>> > We've been working really hard to improve things here, which are really
>> > several inter-related issues:
>> >
>> > - Lack of Hardware capacity in the tripleo CI cloud
>> > - Frequent trunk regressions breaking our CI
>> > - Lack of coverage of some key features (network isolation, SSL, IPv6, upgrades)
>> > - Lack of coverage for vendor plugin templates/puppet code
>> >
>> > There's work ongoing to improve this from multiple perspectives:
>> >
>> > New periodic CI job (to be used for automated promotion of the
>> > current-tripleo repo, and for pre-built undercloud images):
>> > https://review.openstack.org/#/c/271370/
>> >
>> > Add network isolation support to CI:
>> > https://review.openstack.org/#/c/288163/
>> >
>> > Test SSL enabled in overcloud:
>> > https://review.openstack.org/#/c/281988/
>> >
>> > CI coverage of IPv6:
>> > https://review.openstack.org/#/c/289445/
>> >
>> > Discussion around better documented integration for third-party CI:
>> > http://lists.openstack.org/pipermail/openstack-dev/2016-March/088972.html
>>
>> Do we have plans to execute Tempest?
>
> This is something which has been discussed several times, but right now we
> don't have the time available to run it per-commit because we'll hit the
> job timeout.
>
> This situation will improve as we gain time e.g through use of cached
> pre-built images, but right now I think we could look at enabling it only
> on the periodic job when that is fully proven.
>
> Having said that, I should point out that tempest doesn't get us great
> coverage of some newer projects - e.g all Heat scenario coverage was moved
> out of the tempest tree, and other projects have done similar AFAIK, so we
> may end up with very sparse API surface tests (or nothing at all) in these cases.

In Puppet OpenStack CI, we execute smoke tests (a few tests of each
service, and some scenarios), and some tests not in smoke, for Ironic,
Aodh, Horizon.
It takes ~15 minutes to execute them.

This is how we proceed:
https://github.com/openstack/puppet-openstack-integration/blob/master/run_tests.sh#L116-L134

I'm asking because to me Tempest is the official project for OpenStack
testing and if we enable third party CI later, with external CI, we
need to find a common way to test TripleO clouds.

> There is probably more we can do within the existing pingtest though, it
> creates the instance inside a heat stack, so we could just add a bunch more
> resources for all the overcloud services, and pretty quickly prove that the
> deployed services are at least running (without extending the runtime much).
>
> Steve
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi



More information about the OpenStack-dev mailing list