Open Stack

On Mon, Mar 13, 2017 at 7:37 PM, Andrea Frittoli <andrea.frittoli at gmail.com>
wrote:

> On Sat, Mar 11, 2017 at 10:45 PM Matt Riedemann <mriedemos at gmail.com>
> wrote:
>
> On 3/10/2017 3:02 PM, Andrea Frittoli wrote:
> >
> > We had a couple of sessions related to this topic at the PTG [0][1].
> >
> > We agreed that we want to still maintain integration tests only in
> > Tempest, which means that API micro versions that have no integration
> > impact can be tested via functional tests.
>
> To be clear, "integration" here means tests that span operations across
> multiple services, correct? Like a compute test that first creates a
> port in the networking service and a volume in the block storage service
> and then uses those to create a server and maybe take a snapshot of it
> which is then verified was uploaded to the image service.
>
> Yes, indeed.
>
>
>
> The non-integration things are self-contained in a single service, like
> if all you need to do is create an aggregate, show it's details and
> validate the response, at a particular microversion, we can just do that
> in nova functional tests, and it's not necessary in Tempest.
>
> It might be worth having a definition of this policy in the Tempest docs
> so when people ask this question again you can just point at the docs.
>
>
> I agree, we need to go a better job at documenting this.
> I just want to avoid having a black and white rule that would not work.
>
> To me tests that should be in Tempest are tests that make sense to run
> against any change in any repo, and the reasons for that can be many:
> - the test involves multiple services (more that "it uses a token though")
> - the test covers a feature which is key for interoperability
> - the test must be reliable and not take too long
>
> ​+1. I added those in https://review.openstack.org/#/c/444727/​



> Based on this I don't it's possible to completely avoid the discussion
> about
> which test should in Tempest and which not, it's something to be considered
> on a test by test basis, based on a set of guidelines.
>
> We have an initial definition of scope in [0], but it's probably worth to
> elaborate
> more on it. I'll put up a patch so that the discussion on it can continue
> in
> gerrit.
>
> [0] https://docs.openstack.org/developer/tempest/test-
> removal.html#tempest-scope
>
>
> >
> > In terms of which versions we test in the gate, for nova we always run
> > with min_microversion = None and max_microversion = latest, which means
> > that all tests will be executed.
> > Since micro versions are incremental, and each micro version usually
> > involves no or one test on Tempest side, I think it will be a while
> > before this becomes an issue for the common gate.
>
> We test max_microversion=latest only on master. On the devstack stable
> branch we cap the max_microversion, e.g.:
>
> Thanks, good point.
>
>
>
> https://github.com/openstack-dev/devstack/blob/stable/
> newton/lib/tempest#L339
>
> --
>
> Thanks,
>
> Matt
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170314/37fd2551/attachment.html>