Open Stack

On 06/04/17 12:32 +0300, Sagi Shnaidman wrote:
>HI,
>
>I think Rally or Browbeat and other performance oriented solutions won't
>serve our needs, because we run TripleO CI on virtualized environment with
>very limited resources. Actually we are pretty close to full utilizing
>these resources when deploying openstack, so very little is available for
>test.
>It's not a problem to run tempest API tests because they are cheap - take
>little time, little resources, but also gives little coverage. Scenario
>test are more interesting and gives us more coverage, but also takes a lot
>of resources (which we don't have sometimes).
>
>It may be useful to run a "limited edition" of API tests that maximize
>coverage and don't duplicate, for example just to check service working
>basically, without covering all its functionality. It will take very little
>time (i.e. 5 tests for each service) and will give a general picture of
>deployment success. It will cover fields that are not covered by pingtest
>as well.

Without much of the history you guys have, I think this makes sense. This will
also increase consistency throughout CI environments in OpenStack. I like the
idea of starting with a small set of tests from tempest and eventually increase
the number of tests that are executed if needed.

Flavio

>I think could be an option to develop a special scenario tempest tests for
>TripleO which would fit our needs.
>
>Thanks
>
>
>On Wed, Apr 5, 2017 at 11:49 PM, Emilien Macchi <emilien at redhat.com> wrote:
>
>> Greetings dear owls,
>>
>> I would like to bring back an old topic: running tempest in the gate.
>>
>> == Context
>>
>> Right now, TripleO gate is running something called pingtest to
>> validate that the OpenStack cloud is working. It's an Heat stack, that
>> deploys a Nova server, some volumes, a glance image, a neutron network
>> and sometimes a little bit more.
>> To deploy the pingtest, you obviously need Heat deployed in your overcloud.
>>
>> == Problems:
>>
>> Although pingtest has been very helpful over the last years:
>> - easy to understand, it's an Heat template, like an OpenStack user
>> would do to deploy their apps.
>> - fast: the stack takes a few minutes to be created and validated
>>
>> It has some limitations:
>> - Limitation to what Heat resources support (example: some OpenStack
>> resources can't be managed from Heat)
>> - Impossible to run a dynamic workflow (test a live migration for example)
>>
>> == Solutions
>>
>> 1) Switch pingtest to Tempest run on some specific tests, with feature
>> parity of what we had with pingtest.
>> For example, we could imagine to run the scenarios that deploys VM and
>> boot from volume. It would test the same thing as pingtest (details
>> can be discussed here).
>> Each scenario would run more tests depending on the service that they
>> run (scenario001 is telemetry, so it would run some tempest tests for
>> Ceilometer, Aodh, Gnocchi, etc).
>> We should work at making the tempest run as short as possible, and the
>> close as possible from what we have with a pingtest.
>>
>> 2) Run custom scripts in TripleO CI tooling, called from the pingtest
>> (heat template), that would run some validations commands (API calls,
>> etc).
>> It has been investigated in the past but never implemented AFIK.
>>
>> 3) ?
>>
>> I tried to make this text short and go straight to the point, please
>> bring feedback now. I hope we can make progress on $topic during Pike,
>> so we can increase our testing coverage and detect deployment issues
>> sooner.
>>
>> Thanks,
>> --
>> Emilien Macchi
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
>-- 
>Best regards
>Sagi Shnaidman

>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-- 
@flaper87
Flavio Percoco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 862 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170406/3d1f3be2/attachment.sig>