Open Stack

On 04/05/2017 10:49 PM, Emilien Macchi wrote:
> Greetings dear owls,
>
> I would like to bring back an old topic: running tempest in the gate.
>
> == Context
>
> Right now, TripleO gate is running something called pingtest to
> validate that the OpenStack cloud is working. It's an Heat stack, that
> deploys a Nova server, some volumes, a glance image, a neutron network
> and sometimes a little bit more.
> To deploy the pingtest, you obviously need Heat deployed in your overcloud.
>
> == Problems:
>
> Although pingtest has been very helpful over the last years:
> - easy to understand, it's an Heat template, like an OpenStack user
> would do to deploy their apps.
> - fast: the stack takes a few minutes to be created and validated
>
> It has some limitations:
> - Limitation to what Heat resources support (example: some OpenStack
> resources can't be managed from Heat)
> - Impossible to run a dynamic workflow (test a live migration for example)
>
> == Solutions
>
> 1) Switch pingtest to Tempest run on some specific tests, with feature
> parity of what we had with pingtest.
> For example, we could imagine to run the scenarios that deploys VM and
> boot from volume. It would test the same thing as pingtest (details
> can be discussed here).
> Each scenario would run more tests depending on the service that they
> run (scenario001 is telemetry, so it would run some tempest tests for
> Ceilometer, Aodh, Gnocchi, etc).
> We should work at making the tempest run as short as possible, and the
> close as possible from what we have with a pingtest.

A lot of work is going into Tempest itself and its various plugins, so that it 
becomes a convenient and universal tool to test OpenStack clouds. While we're 
not quite there in terms of convenience, it's hard to match the coverage of 
tempest + plugins. I'd prefer TripleO use (some subset of) Tempest test suite(s).

>
> 2) Run custom scripts in TripleO CI tooling, called from the pingtest
> (heat template), that would run some validations commands (API calls,
> etc).
> It has been investigated in the past but never implemented AFIK.
>
> 3) ?

Unless you want to duplicate all the work that goes into Tempest ecosystem now, 
this is probably not a good idea.

>
> I tried to make this text short and go straight to the point, please
> bring feedback now. I hope we can make progress on $topic during Pike,
> so we can increase our testing coverage and detect deployment issues
> sooner.
>
> Thanks,
>