Open Stack

Hi Diarmuid,

Even in our setup Sravanthi by mistake has deleted all the ports
we are trying to bring up again, if it is done today wil ping you and schedule again

Otherwise i will schedule for Monday

Thanks,
Ravi
----- Original Message -----
From: jtaleric at redhat.com
To: openstack-dev at lists.openstack.org, jkilpatr at redhat.com
Sent: Thursday, April 6, 2017 4:15:58 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi
Subject: Re: [openstack-dev] [tripleo] pingtest vs tempest

On Thu, Apr 6, 2017 at 5:32 AM, Sagi Shnaidman <sshnaidm at redhat.com> wrote:
> HI,
>
> I think Rally or Browbeat and other performance oriented solutions won't
> serve our needs, because we run TripleO CI on virtualized environment with
> very limited resources. Actually we are pretty close to full utilizing these
> resources when deploying openstack, so very little is available for test.
> It's not a problem to run tempest API tests because they are cheap - take
> little time, little resources, but also gives little coverage. Scenario test
> are more interesting and gives us more coverage, but also takes a lot of
> resources (which we don't have sometimes).

Sagi,
In my original message I mentioned a "targeted" test, I should
explained that more. We could configure the specific scenario so that
the load on the virt overcloud would be minimal. Justin Kilpatrick
already have Browbeat integrated with TripleO Quickstart[1], so there
shouldn't be much work to try this proposed solution.

>
> It may be useful to run a "limited edition" of API tests that maximize
> coverage and don't duplicate, for example just to check service working
> basically, without covering all its functionality. It will take very little
> time (i.e. 5 tests for each service) and will give a general picture of
> deployment success. It will cover fields that are not covered by pingtest as
> well.
>
> I think could be an option to develop a special scenario tempest tests for
> TripleO which would fit our needs.

I haven't looked at Tempest in a long time, so maybe its functionality
has improved. I just saw the opportunity to integrate Browbeat/Rally
into CI to test the functionality of OpenStack services, while also
capturing performance metrics.

Joe

[1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openstack_browbeat_tree_master_ci-2Dscripts&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=rFCQ76TW5HZUgA7b20ApVcXgXru6mvz4fvCm1_H6w1k&m=c7EeLf1PQSsV2XbWBhv6CWOzUFDRnDiIheN4lDKjyq8&s=Z0jGFw40ezDmSb3F6ns5SXRvacH6AgU0TK5dKBSRgEs&e= 

>
> Thanks
>
>
> On Wed, Apr 5, 2017 at 11:49 PM, Emilien Macchi <emilien at redhat.com> wrote:
>>
>> Greetings dear owls,
>>
>> I would like to bring back an old topic: running tempest in the gate.
>>
>> == Context
>>
>> Right now, TripleO gate is running something called pingtest to
>> validate that the OpenStack cloud is working. It's an Heat stack, that
>> deploys a Nova server, some volumes, a glance image, a neutron network
>> and sometimes a little bit more.
>> To deploy the pingtest, you obviously need Heat deployed in your
>> overcloud.
>>
>> == Problems:
>>
>> Although pingtest has been very helpful over the last years:
>> - easy to understand, it's an Heat template, like an OpenStack user
>> would do to deploy their apps.
>> - fast: the stack takes a few minutes to be created and validated
>>
>> It has some limitations:
>> - Limitation to what Heat resources support (example: some OpenStack
>> resources can't be managed from Heat)
>> - Impossible to run a dynamic workflow (test a live migration for example)
>>
>> == Solutions
>>
>> 1) Switch pingtest to Tempest run on some specific tests, with feature
>> parity of what we had with pingtest.
>> For example, we could imagine to run the scenarios that deploys VM and
>> boot from volume. It would test the same thing as pingtest (details
>> can be discussed here).
>> Each scenario would run more tests depending on the service that they
>> run (scenario001 is telemetry, so it would run some tempest tests for
>> Ceilometer, Aodh, Gnocchi, etc).
>> We should work at making the tempest run as short as possible, and the
>> close as possible from what we have with a pingtest.
>>
>> 2) Run custom scripts in TripleO CI tooling, called from the pingtest
>> (heat template), that would run some validations commands (API calls,
>> etc).
>> It has been investigated in the past but never implemented AFIK.
>>
>> 3) ?
>>
>> I tried to make this text short and go straight to the point, please
>> bring feedback now. I hope we can make progress on $topic during Pike,
>> so we can increase our testing coverage and detect deployment issues
>> sooner.
>>
>> Thanks,
>> --
>> Emilien Macchi
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=rFCQ76TW5HZUgA7b20ApVcXgXru6mvz4fvCm1_H6w1k&m=c7EeLf1PQSsV2XbWBhv6CWOzUFDRnDiIheN4lDKjyq8&s=W6U8XqdHMEh8UmMqQtZNoLw8eWnRvoFgYl-sKUgkiBY&e= 
>
>
>
>
> --
> Best regards
> Sagi Shnaidman
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=rFCQ76TW5HZUgA7b20ApVcXgXru6mvz4fvCm1_H6w1k&m=c7EeLf1PQSsV2XbWBhv6CWOzUFDRnDiIheN4lDKjyq8&s=W6U8XqdHMEh8UmMqQtZNoLw8eWnRvoFgYl-sKUgkiBY&e= 
>

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=rFCQ76TW5HZUgA7b20ApVcXgXru6mvz4fvCm1_H6w1k&m=c7EeLf1PQSsV2XbWBhv6CWOzUFDRnDiIheN4lDKjyq8&s=W6U8XqdHMEh8UmMqQtZNoLw8eWnRvoFgYl-sKUgkiBY&e=