Open Stack

We haven't got much feedback from TripleO core reviewers, who are
usually more involved on this topic. I'll give a chance to let them
talk because we take some actions based on the feedback brought in
this discussion.

Thanks,

On Fri, Apr 7, 2017 at 4:26 AM, Ravi Sekhar Reddy Konda
<ravisekhar.konda at oracle.com> wrote:
> Hi Diarmuid,
>
> Even in our setup Sravanthi by mistake has deleted all the ports
> we are trying to bring up again, if it is done today wil ping you and schedule again
>
> Otherwise i will schedule for Monday
>
> Thanks,
> Ravi
> ----- Original Message -----
> From: jtaleric at redhat.com
> To: openstack-dev at lists.openstack.org, jkilpatr at redhat.com
> Sent: Thursday, April 6, 2017 4:15:58 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi
> Subject: Re: [openstack-dev] [tripleo] pingtest vs tempest
>
> On Thu, Apr 6, 2017 at 5:32 AM, Sagi Shnaidman <sshnaidm at redhat.com> wrote:
>> HI,
>>
>> I think Rally or Browbeat and other performance oriented solutions won't
>> serve our needs, because we run TripleO CI on virtualized environment with
>> very limited resources. Actually we are pretty close to full utilizing these
>> resources when deploying openstack, so very little is available for test.
>> It's not a problem to run tempest API tests because they are cheap - take
>> little time, little resources, but also gives little coverage. Scenario test
>> are more interesting and gives us more coverage, but also takes a lot of
>> resources (which we don't have sometimes).
>
> Sagi,
> In my original message I mentioned a "targeted" test, I should
> explained that more. We could configure the specific scenario so that
> the load on the virt overcloud would be minimal. Justin Kilpatrick
> already have Browbeat integrated with TripleO Quickstart[1], so there
> shouldn't be much work to try this proposed solution.
>
>>
>> It may be useful to run a "limited edition" of API tests that maximize
>> coverage and don't duplicate, for example just to check service working
>> basically, without covering all its functionality. It will take very little
>> time (i.e. 5 tests for each service) and will give a general picture of
>> deployment success. It will cover fields that are not covered by pingtest as
>> well.
>>
>> I think could be an option to develop a special scenario tempest tests for
>> TripleO which would fit our needs.
>
> I haven't looked at Tempest in a long time, so maybe its functionality
> has improved. I just saw the opportunity to integrate Browbeat/Rally
> into CI to test the functionality of OpenStack services, while also
> capturing performance metrics.
>
> Joe
>
> [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openstack_browbeat_tree_master_ci-2Dscripts&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=rFCQ76TW5HZUgA7b20ApVcXgXru6mvz4fvCm1_H6w1k&m=c7EeLf1PQSsV2XbWBhv6CWOzUFDRnDiIheN4lDKjyq8&s=Z0jGFw40ezDmSb3F6ns5SXRvacH6AgU0TK5dKBSRgEs&e=
>
>>
>> Thanks
>>
>>
>> On Wed, Apr 5, 2017 at 11:49 PM, Emilien Macchi <emilien at redhat.com> wrote:
>>>
>>> Greetings dear owls,
>>>
>>> I would like to bring back an old topic: running tempest in the gate.
>>>
>>> == Context
>>>
>>> Right now, TripleO gate is running something called pingtest to
>>> validate that the OpenStack cloud is working. It's an Heat stack, that
>>> deploys a Nova server, some volumes, a glance image, a neutron network
>>> and sometimes a little bit more.
>>> To deploy the pingtest, you obviously need Heat deployed in your
>>> overcloud.
>>>
>>> == Problems:
>>>
>>> Although pingtest has been very helpful over the last years:
>>> - easy to understand, it's an Heat template, like an OpenStack user
>>> would do to deploy their apps.
>>> - fast: the stack takes a few minutes to be created and validated
>>>
>>> It has some limitations:
>>> - Limitation to what Heat resources support (example: some OpenStack
>>> resources can't be managed from Heat)
>>> - Impossible to run a dynamic workflow (test a live migration for example)
>>>
>>> == Solutions
>>>
>>> 1) Switch pingtest to Tempest run on some specific tests, with feature
>>> parity of what we had with pingtest.
>>> For example, we could imagine to run the scenarios that deploys VM and
>>> boot from volume. It would test the same thing as pingtest (details
>>> can be discussed here).
>>> Each scenario would run more tests depending on the service that they
>>> run (scenario001 is telemetry, so it would run some tempest tests for
>>> Ceilometer, Aodh, Gnocchi, etc).
>>> We should work at making the tempest run as short as possible, and the
>>> close as possible from what we have with a pingtest.
>>>
>>> 2) Run custom scripts in TripleO CI tooling, called from the pingtest
>>> (heat template), that would run some validations commands (API calls,
>>> etc).
>>> It has been investigated in the past but never implemented AFIK.
>>>
>>> 3) ?
>>>
>>> I tried to make this text short and go straight to the point, please
>>> bring feedback now. I hope we can make progress on $topic during Pike,
>>> so we can increase our testing coverage and detect deployment issues
>>> sooner.
>>>
>>> Thanks,
>>> --
>>> Emilien Macchi
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=rFCQ76TW5HZUgA7b20ApVcXgXru6mvz4fvCm1_H6w1k&m=c7EeLf1PQSsV2XbWBhv6CWOzUFDRnDiIheN4lDKjyq8&s=W6U8XqdHMEh8UmMqQtZNoLw8eWnRvoFgYl-sKUgkiBY&e=
>>
>>
>>
>>
>> --
>> Best regards
>> Sagi Shnaidman
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=rFCQ76TW5HZUgA7b20ApVcXgXru6mvz4fvCm1_H6w1k&m=c7EeLf1PQSsV2XbWBhv6CWOzUFDRnDiIheN4lDKjyq8&s=W6U8XqdHMEh8UmMqQtZNoLw8eWnRvoFgYl-sKUgkiBY&e=
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=rFCQ76TW5HZUgA7b20ApVcXgXru6mvz4fvCm1_H6w1k&m=c7EeLf1PQSsV2XbWBhv6CWOzUFDRnDiIheN4lDKjyq8&s=W6U8XqdHMEh8UmMqQtZNoLw8eWnRvoFgYl-sKUgkiBY&e=
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi