[openstack-dev] [tempest][nova][defcore] Add option to disable some strict response checking for interop testing

Jay Faulkner jay at jvf.cc
Wed Jun 22 19:13:40 UTC 2016

On 6/22/16 12:01 PM, Sean Dague wrote:
> On 06/22/2016 02:37 PM, Chris Hoge wrote:
>>> On Jun 22, 2016, at 11:24 AM, Sean Dague <sean at dague.net> wrote:
>>> On 06/22/2016 01:59 PM, Chris Hoge wrote:
>>>>> On Jun 20, 2016, at 5:10 AM, Sean Dague <sean at dague.net
>>>>> <mailto:sean at dague.net>> wrote:
>>>>> On 06/14/2016 07:19 PM, Chris Hoge wrote:
>>>>>>> On Jun 14, 2016, at 3:59 PM, Edward Leafe <ed at leafe.com
>>>>>>> <mailto:ed at leafe.com>> wrote:
>>>>>>> On Jun 14, 2016, at 5:50 PM, Matthew Treinish <mtreinish at kortar.org
>>>>>>> <mailto:mtreinish at kortar.org>> wrote:
>>>>>>>> But, if we add another possible state on the defcore side like
>>>>>>>> conditional pass,
>>>>>>>> warning, yellow, etc. (the name doesn't matter) which is used to
>>>>>>>> indicate that
>>>>>>>> things on product X could only pass when strict validation was
>>>>>>>> disabled (and
>>>>>>>> be clear about where and why) then my concerns would be alleviated.
>>>>>>>> I just do
>>>>>>>> not want this to end up not being visible to end users trying to
>>>>>>>> evaluate
>>>>>>>> interoperability of different clouds using the test results.
>>>>>>> +1
>>>>>>> Don't fail them, but don't cover up their incompatibility, either.
>>>>>>> -- Ed Leafe
>>>>>> That’s not my proposal. My requirement is that vendors who want to do
>>>>>> this
>>>>>> state exactly which APIs are sending back additional data, and that this
>>>>>> information be published.
>>>>>> There are different levels of incompatibility. A response with
>>>>>> additional data
>>>>>> that can be safely ignored is different from a changed response that
>>>>>> would
>>>>>> cause a client to fail.
>>>>> It's actually not different. It's really not.
>>>>> This idea that it's safe to add response data is based on an assumption
>>>>> that software versions only move forward. If you have a single deploy of
>>>>> software, that's fine.
>>>>> However as noted, we've got production clouds on Juno <-> Mitaka in the
>>>>> wild. Which means if we want to support horizontal transfer between
>>>>> clouds, the user experienced timeline might be start on a Mitaka cloud,
>>>>> then try to move to Juno. So anything added from Juno -> Mitaka without
>>>>> signaling has exactly the same client breaking behavior as removing
>>>>> attributes.
>>>>> Which is why microversions are needed for attribute adds.
>>>> I’d like to note that Nova v2.0 is still a supported API, which
>>>> as far as I understand allows for additional attributes and
>>>> extensions. That Tempest doesn’t allow for disabling strict
>>>> checking when using a v2.0 endpoint is a problem.
>>>> The reporting of v2.0 in the Marketplace (which is what we do
>>>> right now) is also a signal to a user that there may be vendor
>>>> additions to the API.
>>>> DefCore doesn’t disallow the use of a 2.0 endpoint as part
>>>> of the interoperability standard.
>>> This is a point of confusion.
>>> The API definition did not allow that. The implementation of the API
>>> stack did.
>> And downstream vendors took advantage of that. We may
>> not like it, but it’s a reality in the current ecosystem.
> And we started saying "stop it" 2 years ago. And we've consistently been
> saying stop it all along. And now it's gone.
> And yes, for people that did not get ahead of this issue and engage the
> community, it now hurts. But this has been a quite long process.
I don't wanna wade fully into this discussion, but a question about this
here as there seems to be somewhat of a double standard. I know
upstream, we generally "pay the price" for bad API decisions almost
indefinitely, because we don't want to break users. Is it reasonable to
expecting a public/vendor cloud, who will typically has even more
change-averse users, to change that API out from under users without a
version bump?

>>> In Liberty the v2.0 API is optionally provided by a different backend
>>> stack that doesn't support extensions.
>>> In Mitaka it is default v2.0 API on a non extensions backend
>>> In Newton the old backend is deleted.
>>> From Newton forward there is still a v2.0 API, but all the code hooks
>>> that provided facilities for extensions are gone.
>> It’s really important that the current documentation reflect the
>> code and intent of the dev team. As of writing this e-mail, 
>> "• v2 (SUPPORTED) and v2 extensions (SUPPORTED) (Will
>> be deprecated in the near future.)”[1]
>> Even with this being removed in Newton, DefCore still has
>> to allow for it in every supported version.
> The v2 extensions link there, you will notice, is upstream extensions.
> All of which default on for the new code stack.
> Everything documented there still works on the new code stack. The v2 +
> v2 extensions linked there remains supported in Newton.
> The wording on this page should be updated, it is in the Nova developer
> docs, intended for people working on Nova upstream. They lag a bit from
> where reality is, as does documentation everywhere.
> 	-Sean

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160622/2095eabc/attachment.pgp>

More information about the OpenStack-dev mailing list