Open Stack

> On Jun 14, 2016, at 11:21 AM, Matthew Treinish <mtreinish at kortar.org> wrote:
> 
> On Tue, Jun 14, 2016 at 10:57:05AM -0700, Chris Hoge wrote:
>> Last year, in response to Nova micro-versioning and extension updates[1],
>> the QA team added strict API schema checking to Tempest to ensure that
>> no additional properties were added to Nova API responses[2][3]. In the
>> last year, at least three vendors participating the the OpenStack Powered
>> Trademark program have been impacted by this change, two of which
>> reported this to the DefCore Working Group mailing list earlier this year[4].
>> 
>> The DefCore Working Group determines guidelines for the OpenStack Powered
>> program, which includes capabilities with associated functional tests
>> from Tempest that must be passed, and designated sections with associated
>> upstream code [5][6]. In determining these guidelines, the working group
>> attempts to balance the future direction of development with lagging
>> indicators of deployments and user adoption.
>> 
>> After a tremendous amount of consideration, I believe that the DefCore
>> Working Group needs to implement a temporary waiver for the strict API
>> checking requirements that were introduced last year, to give downstream
>> deployers more time to catch up with the strict micro-versioning
>> requirements determined by the Nova/Compute team and enforced by the
>> Tempest/QA team.
> 
> I'm very much opposed to this being done. If we're actually concerned with
> interoperability and verify that things behave in the same manner between multiple
> clouds then doing this would be a big step backwards. The fundamental disconnect
> here is that the vendors who have implemented out of band extensions or were
> taking advantage of previously available places to inject extra attributes
> believe that doing so means they're interoperable, which is quite far from
> reality. **The API is not a place for vendor differentiation.**

Yes, it’s bad practice, but it’s also a reality, and I honestly believe that
vendors have received the message and are working on changing.

> As a user of several clouds myself I can say that having random gorp in a
> response makes it much more difficult to use my code against multiple clouds. I
> have to determine which properties being returned are specific to that vendor's
> cloud and if I actually need to depend on them for anything it makes whatever
> code I'm writing incompatible for using against any other cloud. (unless I
> special case that block for each cloud) Sean Dague wrote a good post where a lot
> of this was covered a year ago when microversions was starting to pick up steam:
> 
> https://dague.net/2015/06/05/the-nova-api-in-kilo-and-beyond-2 <https://dague.net/2015/06/05/the-nova-api-in-kilo-and-beyond-2>
> 
> I'd recommend giving it a read, he explains the user first perspective more
> clearly there.
> 
> I believe Tempest in this case is doing the right thing from an interoperability
> perspective and ensuring that the API is actually the API. Not an API with extra
> bits a vendor decided to add.

A few points on this, though. Right now, Nova is the only API that is
enforcing this, and the clients. While this may change in the
future, I don’t think it accurately represents the reality of what’s
happening in the ecosystem.

As mentioned before, we also need to balance the lagging nature of
DefCore as an interoperability guideline with the needs of testing
upstream changes. I’m not asking for a permanent change that
undermines the goals of Tempest for QA, rather a temporary
upstream modification that recognizes the challenges faced by
vendors in the market right now, and gives them room to continue
to align themselves with upstream. Without this, the two other 
alternatives are to:

* Have some vendors leave the Powered program unnecessarily,
  weakening it.
* Force DefCore to adopt non-upstream testing, either as a fork
  or an independent test suite.

Neither seem ideal to me.

One of my goals is to transparently strengthen the ties between
upstream and downstream development. There is a deadline
built into this proposal, and my intention is to enforce it.

> I don't think a cloud or product that does this
> to the api should be considered an interoperable OpenStack cloud and failing the
> tests is the correct behavior.

I think it’s more nuanced than this, especially right now.
Only additions to responses will be considered, not changes.
These additions will be clearly labelled as variations,
signaling the differences to users. Existing clients in use
will not break. Correct behavior will eventually be enforced,
and this would be clearly signaled by both the test tool and
through the administrative program.

<snip>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160614/4daaec4c/attachment.html>