Open Stack

Hi team,

here is a short update:

1) The QA user story for destructive testing of OpenStack cloud is on
review [1].
2) The spec for a new framework which will focus on HA/failover and
destructive testing is no review [2].
3) The commit for the new repository is on review [3] as well.

[1] https://review.openstack.org/#/c/396142
[2] https://review.openstack.org/#/c/399618
[3] https://review.openstack.org/#/c/374667


On Fri, Oct 14, 2016 at 3:47 AM, Ghanshyam Mann <
ghanshyam.mann at nectechnologies.in> wrote:

> I like os-faults library which can provide the abstraction over different
> destructive actions like reboot/poweroff node etc.
>
>
>
> But not much clear about Stepler that what all tests it will contain.
> Tempest do have scenario tests which can hit the production env with
> significant way of testing scenario.
>
> It can cover the end user scenario also which involves the interaction of
> public OpenStack APIs and always in enhancement state by adding more and
> more scenario tests.
>
>
>
> Few query over Stepler as separate project:
>
> 1.       Is Stepler will cover only tests which hits the node level
> action(reboot, HA etc)?
>
> 2.       This should not mix the scenario tests which are in Tempest
> scope because that can make confusion for developers (where to add scenario
> tests) as well as for tester(from where to run what scenario tests).
>
> 3.       How we make sure those tests run fine or at least run while
> adding.
>
> a.       I think devstack enhancement for multi-node etc can do this as
> mentioned by you also.
>
> b.      If so then why not adding those tests in Tempest only using
> os-faults lib ?
>
>
>
> Overall I feel os-faults  as lib is really nice idea but tests scope can
> go in Tempest under HW_scenario  (or something else) till it does not break
> basic principle of Tempest.
>
>
>
> Thanks
>
> gmann
>
>
>
> *From:* Timur Nurlygayanov [mailto:tnurlygayanov at mirantis.com]
> *Sent:* 06 October 2016 20:09
> *To:* OpenStack Development Mailing List (not for usage questions) <
> openstack-dev at lists.openstack.org>
> *Subject:* Re: [openstack-dev] [QA] The end-user test suite for OpenStack
> clusters
>
>
>
> Ken, it is a good idea!
>
>
>
> The plan is to develop os-faults as a library which will be able
>
> to manage cluster nodes and OpenStack services on these nodes.
>
> It is a good idea to add some Tempest tests which will use
>
> os-faults library as well for some API tests.
>
>
>
> The Stepler framework [1] will use os-faults to perform all destructive
>
> actions in the clouds (the reboot of nodes, restart of OpenStack services,
>
> enable/disable network interfaces or some firewall rules and etc).
>
>
>
> We need to get +1 from you in [1] to create the repository with
>
> advanced end-user scenario tests.
>
>
>
> Thank you!
>
>
>
> [1] https://review.openstack.org/#/c/374667/
>
>
>
> On Tue, Oct 4, 2016 at 8:53 PM, Yaroslav Lobankov <ylobankov at mirantis.com>
> wrote:
>
> Hi Ken,
>
>
>
> OS-Faults doesn't have any scenarios in the tree yet (the project is two
> months old), but you can find some examples of the use in the
> os-faults/examples directory.
>
>
>
> Regards,
>
> Yaroslav Lobankov.
>
>
>
> On Tue, Oct 4, 2016 at 8:02 PM, Ken'ichi Ohmichi <ken1ohmichi at gmail.com>
> wrote:
>
> Hi Timur,
>
> Thanks for your explanation.
>
>
> 2016-09-29 6:22 GMT-07:00 Timur Nurlygayanov <tnurlygayanov at mirantis.com>:
> >
> >> I am guessing the above "restart nodes" is for verifying each
> >> OpenStack service restarts successfully, right?
> >
> > Yes, this is right. And we also will check that HA logic for these
> > services works correctly (for example, rescheduling of L3 Neutron
> > agents for networks).
> >
> >> But these service scripts are provided by distributors, and Devstack
> >> itself doesn't contain service scripts IIUC.
> >> So I'd like to know how to verify it on Devstack clouds.
> >
> > Yes, DevStack doesn't support many scenarios which are actual
> > and should be supported on the production clouds.
> > It will be not possible to run all advanced test scenarios for DevStack
> > clouds,
> > just because DevStack can't deploy OpenStack cloud with 3 controllers
> > now (so, probably it will be possible in the future).
> >
> > Of course, some advanced scenarios will support DevStack clouds,
> > for example, some test scenarios which are based on customer-found
> > issues from the real production clouds, like upload of the large images
> > (100+ Gb)
> > to Glance with Swift backend. Such cases are important for verification
> of
> > pre-production environments, but not very important for CI gate jobs.
> >
> > It is also important to note that in these advanced cases we are
> targeting
> > to check not only the logic of Python code, but also the correct
> > configuration
> > of all OpenStack components on some pre-production OpenStack clusters.
>
> I guessed some part of os-faults can be moved to Tempest if os-faults
> contains API tests for enabling/disabling OpenStack services.
> Then, os-faults would be able to concentrate on more destructive tests
> like rebooting physical nodes, etc.
> However, I could not find any actual scenarios on current os-faults
> (https://github.com/openstack/os-faults).
> That seems to just contain some abstraction layers and unit tests. Can
> we see actual test scenarios of os-faults ?
> Maybe I missed something.
>
> Thanks
> Ken Ohmichi
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
>
> --
>
>
>
> Timur,
>
> Senior QA Manager
>
> OpenStack Projects
>
> Mirantis Inc
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 

Timur,
QA Manager
OpenStack Projects
Mirantis Inc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20161128/f9c3c169/attachment.html>