[openstack-dev] [QA] The end-user test suite for OpenStack clusters

Ghanshyam Mann ghanshyammann at gmail.com
Fri Feb 24 11:57:47 UTC 2017


Yea, agree with dims.

Sampath ,

Thanks for taking over this, it is really great help. Please update the
current spec with approaches you have. Timur help will be great if he show
up sometime.

Also we will add destructive testing as one of weekly meeting agenda and
make sure you will get all help & required attention from QA team.

-gmann

On Fri, Feb 24, 2017 at 7:26 AM, Davanum Srinivas <davanum at gmail.com> wrote:

> Sampath,
>
> I am not sure if you will hear back from Timur soon as he may not be
> working on this stuff anymore (in Mirantis). So i'd recommend picking
> up the work if you don't hear from him soon.
>
> Thanks,
> Dims
>
> On Thu, Feb 23, 2017 at 3:41 PM, Sam P <sam47priya at gmail.com> wrote:
> > Hi Timur,
> >
> >  The current status of this work is,
> > 1) The QA user story for destructive testing of OpenStack cloud [1] is
> merged .
> > 2) The spec for a new framework which will focus on HA/failover and
> > destructive testing  [2] has no update since Nov 30 2016.
> > 3) The commit for the new repository [3]  has abandoned due to no
> > update after Nov 29 2016.
> >
> > Currently, I am working on 3rd party destructive/HA testing CI for
> > Masakari[4] and very much interested in this work.
> > I will keep working on [1] with PWG.
> > Please let me know your plans for [2], and [3].
> > If it is difficult for you to continue, I would love to continue your
> > work on [2], and [3].
> >
> > [1] https://review.openstack.org/#/c/396142
> > [2] https://review.openstack.org/#/c/399618
> > [3] https://review.openstack.org/#/c/374667
> > [4] https://wiki.openstack.org/wiki/Masakari
> > --- Regards,
> > Sampath
> >
> >
> >
> > On Mon, Nov 28, 2016 at 6:37 AM, Timur Nurlygayanov
> > <tnurlygayanov at mirantis.com> wrote:
> >> Hi team,
> >>
> >> here is a short update:
> >>
> >> 1) The QA user story for destructive testing of OpenStack cloud is on
> review
> >> [1].
> >> 2) The spec for a new framework which will focus on HA/failover and
> >> destructive testing is no review [2].
> >> 3) The commit for the new repository is on review [3] as well.
> >>
> >> [1] https://review.openstack.org/#/c/396142
> >> [2] https://review.openstack.org/#/c/399618
> >> [3] https://review.openstack.org/#/c/374667
> >>
> >>
> >> On Fri, Oct 14, 2016 at 3:47 AM, Ghanshyam Mann
> >> <ghanshyam.mann at nectechnologies.in> wrote:
> >>>
> >>> I like os-faults library which can provide the abstraction over
> different
> >>> destructive actions like reboot/poweroff node etc.
> >>>
> >>>
> >>>
> >>> But not much clear about Stepler that what all tests it will contain.
> >>> Tempest do have scenario tests which can hit the production env with
> >>> significant way of testing scenario.
> >>>
> >>> It can cover the end user scenario also which involves the interaction
> of
> >>> public OpenStack APIs and always in enhancement state by adding more
> and
> >>> more scenario tests.
> >>>
> >>>
> >>>
> >>> Few query over Stepler as separate project:
> >>>
> >>> 1.       Is Stepler will cover only tests which hits the node level
> >>> action(reboot, HA etc)?
> >>>
> >>> 2.       This should not mix the scenario tests which are in Tempest
> scope
> >>> because that can make confusion for developers (where to add scenario
> tests)
> >>> as well as for tester(from where to run what scenario tests).
> >>>
> >>> 3.       How we make sure those tests run fine or at least run while
> >>> adding.
> >>>
> >>> a.       I think devstack enhancement for multi-node etc can do this as
> >>> mentioned by you also.
> >>>
> >>> b.      If so then why not adding those tests in Tempest only using
> >>> os-faults lib ?
> >>>
> >>>
> >>>
> >>> Overall I feel os-faults  as lib is really nice idea but tests scope
> can
> >>> go in Tempest under HW_scenario  (or something else) till it does not
> break
> >>> basic principle of Tempest.
> >>>
> >>>
> >>>
> >>> Thanks
> >>>
> >>> gmann
> >>>
> >>>
> >>>
> >>> From: Timur Nurlygayanov [mailto:tnurlygayanov at mirantis.com]
> >>> Sent: 06 October 2016 20:09
> >>> To: OpenStack Development Mailing List (not for usage questions)
> >>> <openstack-dev at lists.openstack.org>
> >>> Subject: Re: [openstack-dev] [QA] The end-user test suite for OpenStack
> >>> clusters
> >>>
> >>>
> >>>
> >>> Ken, it is a good idea!
> >>>
> >>>
> >>>
> >>> The plan is to develop os-faults as a library which will be able
> >>>
> >>> to manage cluster nodes and OpenStack services on these nodes.
> >>>
> >>> It is a good idea to add some Tempest tests which will use
> >>>
> >>> os-faults library as well for some API tests.
> >>>
> >>>
> >>>
> >>> The Stepler framework [1] will use os-faults to perform all destructive
> >>>
> >>> actions in the clouds (the reboot of nodes, restart of OpenStack
> services,
> >>>
> >>> enable/disable network interfaces or some firewall rules and etc).
> >>>
> >>>
> >>>
> >>> We need to get +1 from you in [1] to create the repository with
> >>>
> >>> advanced end-user scenario tests.
> >>>
> >>>
> >>>
> >>> Thank you!
> >>>
> >>>
> >>>
> >>> [1] https://review.openstack.org/#/c/374667/
> >>>
> >>>
> >>>
> >>> On Tue, Oct 4, 2016 at 8:53 PM, Yaroslav Lobankov <
> ylobankov at mirantis.com>
> >>> wrote:
> >>>
> >>> Hi Ken,
> >>>
> >>>
> >>>
> >>> OS-Faults doesn't have any scenarios in the tree yet (the project is
> two
> >>> months old), but you can find some examples of the use in the
> >>> os-faults/examples directory.
> >>>
> >>>
> >>>
> >>> Regards,
> >>>
> >>> Yaroslav Lobankov.
> >>>
> >>>
> >>>
> >>> On Tue, Oct 4, 2016 at 8:02 PM, Ken'ichi Ohmichi <
> ken1ohmichi at gmail.com>
> >>> wrote:
> >>>
> >>> Hi Timur,
> >>>
> >>> Thanks for your explanation.
> >>>
> >>>
> >>> 2016-09-29 6:22 GMT-07:00 Timur Nurlygayanov <
> tnurlygayanov at mirantis.com>:
> >>> >
> >>> >> I am guessing the above "restart nodes" is for verifying each
> >>> >> OpenStack service restarts successfully, right?
> >>> >
> >>> > Yes, this is right. And we also will check that HA logic for these
> >>> > services works correctly (for example, rescheduling of L3 Neutron
> >>> > agents for networks).
> >>> >
> >>> >> But these service scripts are provided by distributors, and Devstack
> >>> >> itself doesn't contain service scripts IIUC.
> >>> >> So I'd like to know how to verify it on Devstack clouds.
> >>> >
> >>> > Yes, DevStack doesn't support many scenarios which are actual
> >>> > and should be supported on the production clouds.
> >>> > It will be not possible to run all advanced test scenarios for
> DevStack
> >>> > clouds,
> >>> > just because DevStack can't deploy OpenStack cloud with 3 controllers
> >>> > now (so, probably it will be possible in the future).
> >>> >
> >>> > Of course, some advanced scenarios will support DevStack clouds,
> >>> > for example, some test scenarios which are based on customer-found
> >>> > issues from the real production clouds, like upload of the large
> images
> >>> > (100+ Gb)
> >>> > to Glance with Swift backend. Such cases are important for
> verification
> >>> > of
> >>> > pre-production environments, but not very important for CI gate jobs.
> >>> >
> >>> > It is also important to note that in these advanced cases we are
> >>> > targeting
> >>> > to check not only the logic of Python code, but also the correct
> >>> > configuration
> >>> > of all OpenStack components on some pre-production OpenStack
> clusters.
> >>>
> >>> I guessed some part of os-faults can be moved to Tempest if os-faults
> >>> contains API tests for enabling/disabling OpenStack services.
> >>> Then, os-faults would be able to concentrate on more destructive tests
> >>> like rebooting physical nodes, etc.
> >>> However, I could not find any actual scenarios on current os-faults
> >>> (https://github.com/openstack/os-faults).
> >>> That seems to just contain some abstraction layers and unit tests. Can
> >>> we see actual test scenarios of os-faults ?
> >>> Maybe I missed something.
> >>>
> >>> Thanks
> >>> Ken Ohmichi
> >>>
> >>>
> >>> ____________________________________________________________
> ______________
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> unsubscribe
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>>
> >>>
> >>>
> >>> ____________________________________________________________
> ______________
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> unsubscribe
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>>
> >>>
> >>>
> >>> Timur,
> >>>
> >>> Senior QA Manager
> >>>
> >>> OpenStack Projects
> >>>
> >>> Mirantis Inc
> >>>
> >>>
> >>> ____________________________________________________________
> ______________
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> unsubscribe
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>
> >>
> >>
> >> --
> >>
> >> Timur,
> >> QA Manager
> >> OpenStack Projects
> >> Mirantis Inc
> >>
> >> ____________________________________________________________
> ______________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> > ____________________________________________________________
> ______________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Davanum Srinivas :: https://twitter.com/dims
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170224/2d535a9c/attachment-0001.html>


More information about the OpenStack-dev mailing list