Open Stack

On Tuesday, 26 June 2018 11:52:53 CEST Ghanshyam Mann wrote:
>  ---- On Tue, 26 Jun 2018 18:28:03 +0900 Luigi Toscano <ltoscano at redhat.com>
> wrote ----
>  > On Tuesday, 26 June 2018 11:18:52 CEST Ghanshyam Mann wrote:
>  > > Hello Everyone,
>  > > 
>  > > In Queens cycle,  community goal to split the Tempest Plugin has been
>  > > completed [1] and i think almost all the projects have separate repo
>  > > for
>  > > tempest plugin [2]. Which means each tempest plugins are being
>  > > separated
>  > > from their project release model.  Few projects have started the
>  > > independent release model for their plugins like kuryr-tempest-plugin,
>  > > ironic-tempest-plugin etc [3].  I think neutron-tempest-plugin also
>  > > planning as chatted with amotoki.
>  > > 
>  > > There might be some changes in Tempest which might not work with older
>  > > version of Tempest Plugins.  For example, If I am testing any
>  > > production
>  > > cloud which has Nova, Neutron, Cinder, Keystone , Aodh, Congress etc  i
>  > > will be using Tempest and Aodh's , Congress's Tempest plugins. With
>  > > Independent release model of each Tempest Plugins, there might be
>  > > chance
>  > > that the Aodh's or Congress's Tempest plugin versions are not
>  > > compatible
>  > > with latest/known Tempest versions. It will become hard to find the
>  > > compatible tag/release of Tempest and Tempest Plugins or in some cases
>  > > i
>  > > might need to patch up the things.
>  > > 
>  > > During QA feedback sessions at Vancouver Summit, there was feedback to
>  > > coordinating the release of all Tempest plugins and Tempest [4] (also
>  > > amotoki talked to me on this as neutron-tempest-plugin is planning
>  > > their
>  > > first release). Idea is to release/tag all the Tempest plugins and
>  > > Tempest
>  > > together so that specific release/tag can be identified as compatible
>  > > version of all the Plugins and Tempest for testing the complete stack.
>  > > That
>  > > way user can get to know what version of Tempest Plugins is compatible
>  > > with
>  > > what version of Tempest.
>  > > 
>  > > For above use case, we need some coordinated release model among
>  > > Tempest and all the Tempest Plugins. There should be single release of
>  > > all Tempest Plugins with well defined tag whenever any Tempest release
>  > > is happening. For Example, Tempest version 19.0.0 is to mark the
>  > > "support of the Rocky release". When releasing the Tempest 19.0, we
>  > > will release all the Tempest plugins also to tag the compatibility of
>  > > plugins with Tempest for "support of the Rocky release".
>  > > 
>  > > One way to make this coordinated release (just a initial thought):
>  > > 1. Release Each Tempest Plugins whenever there is any major version
>  > > release
>  > > of Tempest (like marking the support of OpenStack release in Tempest,
>  > > EOL
>  > > of OpenStack release in Tempest) 1.1 Each plugin or Tempest can do
>  > > their
>  > > intermediate release of minor version change which are in backward
>  > > compatible way. 1.2 This coordinated Release can be started from latest
>  > > Tempest Version for simple reading.  Like if we start this coordinated
>  > > release from Tempest version 19.0.0 then, each plugins will be released
>  > > as
>  > > 19.0.0 and so on.
>  > > 
>  > > Giving the above background and use case of this coordinated release,
>  > > A) I would like to ask each plugins owner if you are agree on this
>  > > coordinated release?  If no, please give more feedback or issue we can
>  > > face
>  > > due to this coordinated release.
>  > 
>  > The Sahara PTL may disagree with me, but I disagree with forcing each
>  > team to release in a coordinate model.
>  > 
>  > I already take care of releasing sahara-tests, which contains both the
>  > tempest plugin and the scenario tests, when a new major version of
>  > OpenStack is released, keeping the compatibility with the relevant
>  > versions of Tempest.
>  > 
>  > tl;dr I agree with having Tempest plugins follow the same lifecycle of
>  > Tempest, but please allow me to do so manually.
> 
> But with coordinated release,  we can make sure we have particular tags
> which can be used in OpenStack Complete testing. With independent release
> model, there is no guarantee that all tempest plugins will be compatible
> with Tempest versions.

With the independent release model (in general, not just for Tempest plugins) 
it's up to the program maintainer to ensure that things are compatible. 
Let me make sure that:
- I agree that Tempest plugins should follow the same lifecycle than Tempest. 
It's the policy that I applied for the Tempest plugin part of sahara-tests 
since Kilo.
- most of plugins maintainer don't care, can't care or shouldn't care about 
the release process: then coordinated release is fine (as it happen also for 
the general usage of coordinated release vs independent relase in OpenStack)
- I still prefer to manage this myself, because the repository does not 
contain only the Tempest plugin. I may end up with the need of another release 
at a different time, because other things are broken. And no, I'm not going to 
split the repository, because the part which is not a Tempest plugin still 
uses tempest.lib.


-- 
Luigi