Open Stack

Hi,

My first impression is "It's interesting" :)

I actually don't read whole of this threads, but, I re-read the QA
mission statement[1].
I feel os-faults is a little bit far from the mission statement.
Because os-faults seems like focusing on operator testing.

But I think this can be under the QA project. Because tempest scope
has also operator testing.

[1] https://wiki.openstack.org/wiki/QA#Project_Team_Definition


On Sat, Oct 8, 2016 at 4:43 AM, Ken'ichi Ohmichi <ken1ohmichi at gmail.com> wrote:
> Hi Timur,
>
> 2016-10-06 4:08 GMT-07:00 Timur Nurlygayanov <tnurlygayanov at mirantis.com>:
>> Ken, it is a good idea!
>>
>> The plan is to develop os-faults as a library which will be able
>> to manage cluster nodes and OpenStack services on these nodes.
>> It is a good idea to add some Tempest tests which will use
>> os-faults library as well for some API tests.
>
> Sorry for my misleading, that is not I wanted to say.
> I am thinking Tempest doesn't use os-faults as library.
> I just wanted to say some destructive tests which will use REST APIs
> can be implemented in Tempest instead of os-faults.
>
> For example, the compute service client contains disable_service which
> disables nova service but Tempest doesn't contain the corresponding
> test cases because Tempest tests need to run in parallel.
> https://github.com/openstack/tempest/blob/master/tempest/lib/services/compute/services_client.py#L54
>
> If some tests use this method, the other tests which run in parallel
> can be failed as unexpected on the gate.
> Such tests also are destructive and I am hoping these tests also will
> be able to run the gate by finding better way.
>
>> The Stepler framework [1] will use os-faults to perform all destructive
>> actions in the clouds (the reboot of nodes, restart of OpenStack services,
>> enable/disable network interfaces or some firewall rules and etc).
>>
>> We need to get +1 from you in [1] to create the repository with
>> advanced end-user scenario tests.
>
> OK, I'd like to summarize my thinking here for adding os-faults under
> QA project:
>
> Under QA project, the first target of most programs(tempest, devstack,
> etc) is for the gate testing.
> Of course, some of them are available on production clouds also as the
> design, but the first is for the gate.
> That means devstack clouds as the first target/purpose.
> At this time, this os-faults doesn't seem the gate as the first
> purpose based on current implementation.
> So I feel os-faults seems different from the existing programs.
>
> os-faults is very young and we will be able to extend it for devstack
> clouds also later, I hope.
> In addition, I heard this kind of tests(destructive, HA, etc) are
> requested from the other users.
> So this os-faults seems very valuable for users.
>
> Just in my opinion, I am find to add this os-faults under QA project.
> But before that, I'd like to get opinions from the other people of QA team.
>
> Thanks
> Ken Ohmichi
>
>
>
>
>
>
>
>
>
>
>
>
>> On Tue, Oct 4, 2016 at 8:53 PM, Yaroslav Lobankov <ylobankov at mirantis.com>
>> wrote:
>>>
>>> Hi Ken,
>>>
>>> OS-Faults doesn't have any scenarios in the tree yet (the project is two
>>> months old), but you can find some examples of the use in the
>>> os-faults/examples directory.
>>>
>>> Regards,
>>> Yaroslav Lobankov.
>>>
>>> On Tue, Oct 4, 2016 at 8:02 PM, Ken'ichi Ohmichi <ken1ohmichi at gmail.com>
>>> wrote:
>>>>
>>>> Hi Timur,
>>>>
>>>> Thanks for your explanation.
>>>>
>>>> 2016-09-29 6:22 GMT-07:00 Timur Nurlygayanov
>>>> <tnurlygayanov at mirantis.com>:
>>>> >
>>>> >> I am guessing the above "restart nodes" is for verifying each
>>>> >> OpenStack service restarts successfully, right?
>>>> >
>>>> > Yes, this is right. And we also will check that HA logic for these
>>>> > services works correctly (for example, rescheduling of L3 Neutron
>>>> > agents for networks).
>>>> >
>>>> >> But these service scripts are provided by distributors, and Devstack
>>>> >> itself doesn't contain service scripts IIUC.
>>>> >> So I'd like to know how to verify it on Devstack clouds.
>>>> >
>>>> > Yes, DevStack doesn't support many scenarios which are actual
>>>> > and should be supported on the production clouds.
>>>> > It will be not possible to run all advanced test scenarios for DevStack
>>>> > clouds,
>>>> > just because DevStack can't deploy OpenStack cloud with 3 controllers
>>>> > now (so, probably it will be possible in the future).
>>>> >
>>>> > Of course, some advanced scenarios will support DevStack clouds,
>>>> > for example, some test scenarios which are based on customer-found
>>>> > issues from the real production clouds, like upload of the large images
>>>> > (100+ Gb)
>>>> > to Glance with Swift backend. Such cases are important for verification
>>>> > of
>>>> > pre-production environments, but not very important for CI gate jobs.
>>>> >
>>>> > It is also important to note that in these advanced cases we are
>>>> > targeting
>>>> > to check not only the logic of Python code, but also the correct
>>>> > configuration
>>>> > of all OpenStack components on some pre-production OpenStack clusters.
>>>>
>>>> I guessed some part of os-faults can be moved to Tempest if os-faults
>>>> contains API tests for enabling/disabling OpenStack services.
>>>> Then, os-faults would be able to concentrate on more destructive tests
>>>> like rebooting physical nodes, etc.
>>>> However, I could not find any actual scenarios on current os-faults
>>>> (https://github.com/openstack/os-faults).
>>>> That seems to just contain some abstraction layers and unit tests. Can
>>>> we see actual test scenarios of os-faults ?
>>>> Maybe I missed something.
>>>>
>>>> Thanks
>>>> Ken Ohmichi
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>> --
>>
>> Timur,
>> Senior QA Manager
>> OpenStack Projects
>> Mirantis Inc
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev