Open Stack

Hi, folks.

This is the email I sent to the interop at openstack.orb and the co-chairs of DefCore.  From today's DefCore meeting, I am posting this to the mailing list for discussion and hopefully resolution.  Mark Voelker said that other vendors had also had the same issue.  TL;DR  a set of Nova tests changed between the time 2015.05 build SHA was captured and the 2015.07 SHA was captured.  The changed tests were intended for the LIberty release, as they were not proposed/merged until after the Kilo release  Kilo release was April; changes to tests were merged June 19 (definitely in the Liberty cycle).

Thanks,
--Rocky

Below, is more info on Huawei's situation:


Huawei has a problem we hope you can help us with.  Our FusionSphere 6.0, based on Juno, qualified for  the "interop" OpenStack Powered (tm) program on the 2015.05 DefCore tests.  We are now trying to qualify our Public cloud offering, running the exact same FusionSphere/OpenStack software but cannot pass a subset of the Nova tests.

Huawei uses Nova api V2 (not V2.1).  As such, our implementation has "Additional Properties" enabled and used.  These are supposedly valid operations for the Juno release.

The tests in question were changed via https://review.openstack.org/#/c/156130.  AdditionalProperties were changed to "False" in all of the tests included in the review.

The problem is that Juno was released in October of 2014, with AdditionalProperties (extenstions) allowed in the v2 apis.  The tests were changed for the Kilo release cycle, but affect the Juno releases under the Interop/DefCore/Refstack test selection process because of the nonbranching nature of Tempest.

We think a waiver is appropriate in this situation, as the Huawei code has not changed, yet it now fails API tests that were valid three months earlier.  We think that including the v2 api in the changes in theseDefCore  tests was a mistake, as the discussion below indicates, and that more care needs to be exercised when test changes are made to enforce new behaviors for later release cycles.  We would be happy to demonstrate that our release still passes the failed tests under the older version of those tests that exist in 2015.05.

We have been working diligently to get the certification on our Public Cloud product, and had hoped to qualify/certify before the release of the 2016.01 standards, as our development resources are currently focused on moving our product to a more current  OpenStack release.

Please advise us as to how we proceed from here.


Respectfully,
Rocky Grober and our FusionSphere team

an excerpt from one of the changed files:
[cid:image002.gif at 01D13983.9475C460]

The problem of a backward incompatibility to an API being introduced by test changes was mentioned in one an exchange in comments:

afazekas
Mar 17, 2015
↩

Patch Set 31: Code-Review-1

Does not makes sense to prevent additional properties in tempest on something which allows extensions.
Ghanshyam Mann
Mar 18, 2015
↩

Patch Set 31:

@afazekas - Thanks for review.

Actually all schema has been modified to work for all extension (as current tests runs for all extension enable).

As v2 (v2.1) API are frozen, tempest should have additionalProperties False to capture any unwanted changes in those API.
afazekas
Mar 18, 2015
↩

Patch Set 31:

https://wiki.openstack.org/wiki/APIChangeGuidelines

"Adding a property to a resource representation" is generally acceptable.

"Changing or removing a property in a resource representation" OR "Changing the semantics of a property in a resource representation which may be supplied by clients" is generally not acceptable.

We do not need to prevent additional properties.
Ghanshyam Mann
Mar 19, 2015
↩

Patch Set 31:

Yea, those are acceptable and it used to done by adding new extension etc. But as v2 nd v2.1 are frozen and no new extension for thosse. All new attributes etc can be added with microversion only. As it is done for first microversion - https://review.openstack.org/#/c/140313/

Please do let me know if m missing anything on this.
Ken'ichi Ohmichi
Mar 19, 2015
↩

Patch Set 31: Code-Review+2

Yeah, Ghanshyam is right.

As http://lists.openstack.org/pipermail/openstack-dev/2015-February/057613.html , new attributes will be added with microversions and v2(v2.1) should be frozen. So this change is necessary for blocking additional attributes on v2 and v2.1 API.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/defcore-committee/attachments/20160107/39cc5ee7/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 29270 bytes
Desc: image001.gif
URL: <http://lists.openstack.org/pipermail/defcore-committee/attachments/20160107/39cc5ee7/attachment-0001.gif>