Open Stack

On Wed, 2014-02-12 at 14:17 -0800, Maru Newby wrote:
> On Feb 12, 2014, at 1:59 PM, Sean Dague <sean at dague.net> wrote:
> 
> > On 02/12/2014 04:25 PM, Maru Newby wrote:
> >> 
> >> On Feb 12, 2014, at 12:36 PM, Sean Dague <sean at dague.net> wrote:
> >> 
> >>> On 02/12/2014 01:48 PM, Maru Newby wrote:
> >>>> At the last 2 summits, I've suggested that API tests could be maintained in the Neutron tree and reused by Tempest.  I've finally submitted some patches that demonstrate this concept:
> >>>> 
> >>>> https://review.openstack.org/#/c/72585/  (implements a unit test for the lifecycle of the network resource)
> >>>> https://review.openstack.org/#/c/72588/  (runs the test with tempest rest clients)
> >>>> 
> >>>> My hope is to make API test maintenance a responsibility of the Neutron team.  The API compatibility of each Neutron plugin has to be validated by Neutron tests anyway, and if the tests are structured as I am proposing, Tempest can reuse those efforts rather than duplicating them.
> >>>> 
> >>>> I've added this topic to this week's agenda, and I would really appreciate it interested parties would take a look at the patches in question to prepare themselves to participate in the discussion.
> >>>> 
> >>>> 
> >>>> m.
> >>>> _______________________________________________
> >>>> OpenStack-dev mailing list
> >>>> OpenStack-dev at lists.openstack.org
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>> 
> >>> 
> >>> Realistically, having API tests duplicated in the Tempest tree is a
> >>> feature, not a bug.
> >>> 
> >>> tempest/api is there for double book keep accounting, and it has been
> >>> really effective at preventing accidental breakage of our APIs (which
> >>> used to happen all the time), so I don't think putting API testing in
> >>> neutron obviates that.
> >> 
> >> Given how limited our testing resources are, might it be worth considering whether 'double-entry accounting' is actually the best way to preventing accidental breakage going forward?  Might reasonable alternatives exist, such as clearly separating api tests from other tests in the neutron tree and giving review oversight only to qualified individuals?
> > 
> > Our direct experience is that if we don't do this, within 2 weeks some
> > project will have landed API breaking changes. This approach actually
> > takes a lot of review load off the core reviewers, so reverting to a
> > model which puts more work back on the review team (given the current
> > review load), isn't something I think we want.
> 
> Just so I'm clear, is there anything I could say that would change your mind?

I'd like to discuss this at tomorrow's IRC meeting, if possible. I think
that the model that Maru came up with (PoC code in the two patches
referenced above) are actually a pretty slick way of dealing with this
issue, and along with the ongoing efforts to "libify" Tempest, I think
that we should head in this direction if possible.

Best,
-jay