Open Stack

On 09/01/2016 05:48 PM, Emilien Macchi wrote:
> On Thu, Aug 25, 2016 at 9:16 AM, Steven Hardy <shardy at redhat.com> wrote:
>> On Wed, Aug 17, 2016 at 07:20:59AM -0400, James Slagle wrote:
>>> On Wed, Aug 17, 2016 at 4:04 AM, Dmitry Tantsur <dtantsur at redhat.com> wrote:
>>>> However, the current gate system allows to run jobs based on files affected.
>>>> So we can also run a scenario covering ironic on THT check/gate if
>>>> puppet/services/*ironic* is affected, but not in the other cases. This won't
>>>> cover all potential failures, but it would be of great help anyway. It
>>>> should also run in experimental pipeline, so that it can be triggered on any
>>>> patch.
>>>>
>>>> This is in addition to periodic jobs you're proposing, not replacing them.
>>>> WDYT?
>>>
>>> Using the files affected to trigger a scenario test that uses the
>>> affected composable service sounds like a really good idea to me.
>>
>> +1 I think this sounds like a really good idea.
>>
>> Now that we're doing almost all per-service configuration in the respective
>> templates and puppet profiles, this should be much easier to implement I
>> think so definitely +1 on giving it a go.
>>
>
> So I would like to give an update about this.
> The work has been done to have the structure in place.
> I first used experimental pipeline to test the new jobs but they are
> now in check pipeline, as non-voting.
>
> I created a multinode jobs CI matrix here:
> https://github.com/openstack-infra/tripleo-ci#service-testing-matrix
> I encourage everyone to have a quick look at it.
>
> What is happening now?
>
> - If you submit a patch in THT (in puppet/services/ceilometer*) it
> will trigger scenario001 job (example with Telemetry). Same thing for
> Puppet profiles in puppet-tripleo. Note: with Zuul v3 we'll be able to
> make things more granular and specific to projects and files. We're
> looking for having it!
> - Since multinode-nonha initially created by slagle is now a bit
> redundant with scenarios, I'll make it lighter to test basic compute
> services with the less services as possible.
> - I'll continue to extend the scenarios complexity and start testing
> different backends (ie, ceph/rbd on scenario001 with Telemetry, etc),
> like we're doing in Puppet CI:
> https://github.com/openstack/puppet-openstack-integration#description
> - For now, we're using pingtest to test that services actually work. I
> guess it's good for now, but we also might want to consider Tempest
> sometimes. I know Tempest already runs on periodic jobs, but should we
> also consider it in those multinode jobs? The feedback would be
> valuable though but we would have to make tempest configuration
> composable, depending on the services we actually run (maybe using
> discovery, etc).
>
> Any help and feedback on this topic is highly welcome,
>

It's pretty cool, unfortunately putting nova-compute on controllers is 
incompatible with the approach we currently take for ironic... so we 
either need a separate scenario or another approach here.