[openstack-dev] [QA] The end-user test suite for OpenStack clusters

Ken'ichi Ohmichi ken1ohmichi at gmail.com
Fri Oct 7 19:43:50 UTC 2016


Hi Timur,

2016-10-06 4:08 GMT-07:00 Timur Nurlygayanov <tnurlygayanov at mirantis.com>:
> Ken, it is a good idea!
>
> The plan is to develop os-faults as a library which will be able
> to manage cluster nodes and OpenStack services on these nodes.
> It is a good idea to add some Tempest tests which will use
> os-faults library as well for some API tests.

Sorry for my misleading, that is not I wanted to say.
I am thinking Tempest doesn't use os-faults as library.
I just wanted to say some destructive tests which will use REST APIs
can be implemented in Tempest instead of os-faults.

For example, the compute service client contains disable_service which
disables nova service but Tempest doesn't contain the corresponding
test cases because Tempest tests need to run in parallel.
https://github.com/openstack/tempest/blob/master/tempest/lib/services/compute/services_client.py#L54

If some tests use this method, the other tests which run in parallel
can be failed as unexpected on the gate.
Such tests also are destructive and I am hoping these tests also will
be able to run the gate by finding better way.

> The Stepler framework [1] will use os-faults to perform all destructive
> actions in the clouds (the reboot of nodes, restart of OpenStack services,
> enable/disable network interfaces or some firewall rules and etc).
>
> We need to get +1 from you in [1] to create the repository with
> advanced end-user scenario tests.

OK, I'd like to summarize my thinking here for adding os-faults under
QA project:

Under QA project, the first target of most programs(tempest, devstack,
etc) is for the gate testing.
Of course, some of them are available on production clouds also as the
design, but the first is for the gate.
That means devstack clouds as the first target/purpose.
At this time, this os-faults doesn't seem the gate as the first
purpose based on current implementation.
So I feel os-faults seems different from the existing programs.

os-faults is very young and we will be able to extend it for devstack
clouds also later, I hope.
In addition, I heard this kind of tests(destructive, HA, etc) are
requested from the other users.
So this os-faults seems very valuable for users.

Just in my opinion, I am find to add this os-faults under QA project.
But before that, I'd like to get opinions from the other people of QA team.

Thanks
Ken Ohmichi












> On Tue, Oct 4, 2016 at 8:53 PM, Yaroslav Lobankov <ylobankov at mirantis.com>
> wrote:
>>
>> Hi Ken,
>>
>> OS-Faults doesn't have any scenarios in the tree yet (the project is two
>> months old), but you can find some examples of the use in the
>> os-faults/examples directory.
>>
>> Regards,
>> Yaroslav Lobankov.
>>
>> On Tue, Oct 4, 2016 at 8:02 PM, Ken'ichi Ohmichi <ken1ohmichi at gmail.com>
>> wrote:
>>>
>>> Hi Timur,
>>>
>>> Thanks for your explanation.
>>>
>>> 2016-09-29 6:22 GMT-07:00 Timur Nurlygayanov
>>> <tnurlygayanov at mirantis.com>:
>>> >
>>> >> I am guessing the above "restart nodes" is for verifying each
>>> >> OpenStack service restarts successfully, right?
>>> >
>>> > Yes, this is right. And we also will check that HA logic for these
>>> > services works correctly (for example, rescheduling of L3 Neutron
>>> > agents for networks).
>>> >
>>> >> But these service scripts are provided by distributors, and Devstack
>>> >> itself doesn't contain service scripts IIUC.
>>> >> So I'd like to know how to verify it on Devstack clouds.
>>> >
>>> > Yes, DevStack doesn't support many scenarios which are actual
>>> > and should be supported on the production clouds.
>>> > It will be not possible to run all advanced test scenarios for DevStack
>>> > clouds,
>>> > just because DevStack can't deploy OpenStack cloud with 3 controllers
>>> > now (so, probably it will be possible in the future).
>>> >
>>> > Of course, some advanced scenarios will support DevStack clouds,
>>> > for example, some test scenarios which are based on customer-found
>>> > issues from the real production clouds, like upload of the large images
>>> > (100+ Gb)
>>> > to Glance with Swift backend. Such cases are important for verification
>>> > of
>>> > pre-production environments, but not very important for CI gate jobs.
>>> >
>>> > It is also important to note that in these advanced cases we are
>>> > targeting
>>> > to check not only the logic of Python code, but also the correct
>>> > configuration
>>> > of all OpenStack components on some pre-production OpenStack clusters.
>>>
>>> I guessed some part of os-faults can be moved to Tempest if os-faults
>>> contains API tests for enabling/disabling OpenStack services.
>>> Then, os-faults would be able to concentrate on more destructive tests
>>> like rebooting physical nodes, etc.
>>> However, I could not find any actual scenarios on current os-faults
>>> (https://github.com/openstack/os-faults).
>>> That seems to just contain some abstraction layers and unit tests. Can
>>> we see actual test scenarios of os-faults ?
>>> Maybe I missed something.
>>>
>>> Thanks
>>> Ken Ohmichi
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
>
> Timur,
> Senior QA Manager
> OpenStack Projects
> Mirantis Inc
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



More information about the OpenStack-dev mailing list