Open Stack

Sagi,

I think Rally or Browbeat and other performance oriented solutions won't
> serve our needs, because we run TripleO CI on virtualized environment with
> very limited resources. Actually we are pretty close to full utilizing
> these resources when deploying openstack, so very little is available for
> test.


You can run Rally with any load. Including just starting 1 smallest VM.


It may be useful to run a "limited edition" of API tests that maximize
> coverage and don't duplicate, for example just to check service working
> basically, without covering all its functionality. It will take very little
> time (i.e. 5 tests for each service) and will give a general picture of
> deployment success. It will cover fields that are not covered by pingtest
> as well.


You can actually pick few of scenarios that we have in Rally and cover most
of the functionality.
If you specify what exactly you want to test I can help with writing Rally
Task for that. (it will use as minimum as possible resources)


Best regards,
Boris Pavlovic



On Thu, Apr 6, 2017 at 2:38 AM, Dmitry Tantsur <dtantsur at redhat.com> wrote:

> On 04/05/2017 10:49 PM, Emilien Macchi wrote:
>
>> Greetings dear owls,
>>
>> I would like to bring back an old topic: running tempest in the gate.
>>
>> == Context
>>
>> Right now, TripleO gate is running something called pingtest to
>> validate that the OpenStack cloud is working. It's an Heat stack, that
>> deploys a Nova server, some volumes, a glance image, a neutron network
>> and sometimes a little bit more.
>> To deploy the pingtest, you obviously need Heat deployed in your
>> overcloud.
>>
>> == Problems:
>>
>> Although pingtest has been very helpful over the last years:
>> - easy to understand, it's an Heat template, like an OpenStack user
>> would do to deploy their apps.
>> - fast: the stack takes a few minutes to be created and validated
>>
>> It has some limitations:
>> - Limitation to what Heat resources support (example: some OpenStack
>> resources can't be managed from Heat)
>> - Impossible to run a dynamic workflow (test a live migration for example)
>>
>> == Solutions
>>
>> 1) Switch pingtest to Tempest run on some specific tests, with feature
>> parity of what we had with pingtest.
>> For example, we could imagine to run the scenarios that deploys VM and
>> boot from volume. It would test the same thing as pingtest (details
>> can be discussed here).
>> Each scenario would run more tests depending on the service that they
>> run (scenario001 is telemetry, so it would run some tempest tests for
>> Ceilometer, Aodh, Gnocchi, etc).
>> We should work at making the tempest run as short as possible, and the
>> close as possible from what we have with a pingtest.
>>
>
> A lot of work is going into Tempest itself and its various plugins, so
> that it becomes a convenient and universal tool to test OpenStack clouds.
> While we're not quite there in terms of convenience, it's hard to match the
> coverage of tempest + plugins. I'd prefer TripleO use (some subset of)
> Tempest test suite(s).
>
>
>> 2) Run custom scripts in TripleO CI tooling, called from the pingtest
>> (heat template), that would run some validations commands (API calls,
>> etc).
>> It has been investigated in the past but never implemented AFIK.
>>
>> 3) ?
>>
>
> Unless you want to duplicate all the work that goes into Tempest ecosystem
> now, this is probably not a good idea.
>
>
>> I tried to make this text short and go straight to the point, please
>> bring feedback now. I hope we can make progress on $topic during Pike,
>> so we can increase our testing coverage and detect deployment issues
>> sooner.
>>
>> Thanks,
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170406/462f68ae/attachment.html>