[openstack-dev] [qa] Clarification of policy for qa-specs around adding new tests
oomichi at mxs.nes.nec.co.jp
Thu Jun 19 10:40:55 UTC 2014
> -----Original Message-----
> From: Matthew Treinish [mailto:mtreinish at kortar.org]
> Sent: Wednesday, June 18, 2014 10:33 PM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [qa] Clarification of policy for qa-specs around adding new tests
> On Tue, Jun 17, 2014 at 01:45:55AM +0000, Kenichi Oomichi wrote:
> > > -----Original Message-----
> > > From: Matthew Treinish [mailto:mtreinish at kortar.org]
> > > Sent: Monday, June 16, 2014 11:58 PM
> > > To: OpenStack Development Mailing List (not for usage questions)
> > > Subject: Re: [openstack-dev] [qa] Clarification of policy for qa-specs around adding new tests
> > >
> > > On Mon, Jun 16, 2014 at 10:46:51AM -0400, David Kranz wrote:
> > > > I have been reviewing some of these specs and sense a lack of clarity around
> > > > what is expected. In the pre-qa-specs world we did not want tempest
> > > > blueprints to be used by projects to track their tempest test submissions
> > > > because the core review team did not want to have to spend a lot of time
> > > > dealing with that. We said that each project could have one tempest
> > > > blueprint that would point to some other place (project blueprints,
> > > > spreadsheet, etherpad, etc.) that would track specific tests to be added.
> > > > I'm not sure what aspect of the new qa-spec process would make us feel
> > > > differently about this. Has this policy changed? We should spell out the
> > > > expectation in any event. I will update the README when we have a
> > > > conclusion.
> > > >
> > >
> > > The policy has not changed. There should be 1 BP (or maybe 2 or 3 if they want
> > > to split the effort a bit more granularly for tracking different classes of
> > > tests, but still 1 BP series) for improving project tests. For individual tests
> > > part of a bigger effort should be tracked outside of the Tempest LP. IMO after
> > > it's approved the spec/BP for tracking test additions is only really useful to
> > > have a unified topic to use for review classification.
> > +1 to use a single blueprint for adding new tests of each project.
> > The unified topic of each project would be useful to get each project
> > reviewers' effort on the Tempest tests reviews.
> > To add new tests, do we need to have qa-specs, or is it OK to have
> > blueprints only?
> So I've been asking all the new BPs for project testing being opened this cycle
> to have a spec too. My feeling is that we should only have one process for doing
> BPs/specs that way we get all the artifacts in the same place. It should also
> hopefully get everyone more involved with the qa-specs workflow.
I see, thank you for clarifying it.
> The specs for adding project test should be pretty simple, they just basically
> need to outline what project is going to be tested, what types of tests are
> going to be worked on, (API, CLI, etc..) and how the test development is going
> to be tracked. (etherpad, google doc, etc.)
That is nice advice, it seems easy to review also :-)
More information about the OpenStack-dev