Open Stack

> On Jun 19, 2015, at 12:37 PM, John Garbutt <john at johngarbutt.com> wrote:
> 
> On 19 June 2015 at 15:01, Mark Voelker <mvoelker at vmware.com> wrote:
>> Thanks for bringing this up Ken’ichi!  Folks who aren’t familiar with Nova’s move to microversioning may want to peruse the spec that was implemented in Kilo [1] and note that some other projects like Neutron are headed in this direction as well [2].
> 
> There quite a bit of miss-information out there about microversions.
> 
> Please do read this for more context:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__dague.net_2015_06_05_the-2Dnova-2Dapi-2Din-2Dkilo-2Dand-2Dbeyond-2D2_&d=BQIFaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=Q8IhPU-EIzbG5YDx5LYO7zEJpGZykn7RwFg-UTPWvDc&m=u-7GQNCU48f6JTrRvtRMgsRIbr_CgD8m6uW3vB4Skys&s=JBEprEZ2TrL5L4u4cidb1AEs60l1N05K4mg0OClCJUA&e= 
> 
>> The basic idea of this patch is to not allow any additional properties in API responses (even if all the nova-supported properties are included) because Nova has moved to a microversioned API as of Kilo [1], correct?
> 
> Its a way bigger change than that, please read above blog.
> 
> We do allow for an API to admit it has been altered. To quote the spec[1]:
> "
> X-OpenStack-Nova-API-Version: version_number, experimental
> 
> Experimental is only returned if the operator has made a modification
> to the API behaviour that is non standard. This is only intended to be
> a transitional mechanism while some functionality used by cloud
> operators is upstreamed and it will be removed within a small number
> of releases..
> “


Sure, but I’m referring to Ken’ichi’s Tempest patch here, not to microversioning (of which I’m a fan, FWIW) in general.  To wit: we are in the transitional period you mention right now since microversioning is brand new and that 'small number of releases' hasn’t yet occurred.  The Tempest change doesn’t take into account the ‘experimental’ flag at all, does it?  I think it just says “if there are more attributes than these in the response then the response is invalid and this test fails.”  As a development gate test, that makes sense: it provides very strong incentive to get stuff moved into the API proper or split out into a separate API.  For DefCore purposes though, it’s a bit tough as Tempest is going to fail both on older clouds where attributes have been added (e.g. before microversioning existed and people had the experimental/bump options) and newer (potentially even Liberty?) clouds where the experimental flag has been set as folks do the good-faith work of transitioning things.  

This sort of feels like one of those tricky areas where Tempest is being used for more than one purpose and they move at a different cadence.  I may well have missed something about the change, so please shout if I’m misinterpreting. =) 


> 
>> And if a cloud wants to provide additional attributes in API responses, it should now bump the monotonic version counter to make clear to clients that it does so?
> 
> Not as such. All changes must be upstream, or create a separate API
> for vendor specific things.
> 
>> From a DefCore perspective, I imagine the concern here is how this change affects clouds built on older versions of OpenStack since Tempest is branchless. A DefCore Guideline typically covers three OpenStack releases: the next Guideline will likely cover Juno, Kilo, and Liberty.  Clouds built on Juno predate the move to Nova microversioning.  In theory there could be Juno-based clouds that provide all of the “OpenStack” properties in API responses, but add some additional properties as well.  Such clouds might run into trouble when running RefStack because Tempest will now enforce that no additional properties may be added to API responses even on clouds that predate the move to microversioning.
> 
> It feels like that should only run against v2.1 API and should not be
> run against the v2.0 API, really. 

+1

I’d welcome more opinions but that’s sort of the conclusion I was mentally meandering toward as well.

> Although, as a test for Nova I would
> always want that run, but as a test for DefCore is a different
> question.


+1

I think that API microversioning and Tempest changes like this clearly communicate the community’s intent that API responses aren’t something vendors should add attributes to, and it incents them to do things in-tree (or at least create a different API which is more explicit for users), which is great.  I think the concern is just that the DefCore use case for Tempest is a bit different than the development gate-test use case.  Because DefCore is using Tempest as it’s means of validation even against older versions of OpenStack and because Tempest is branchless, we may not be giving existing OpenStack Powered Platforms reasonable time to make the transition.  If this were applied to 2.1 only, that would alleviate at least part of that concern (e.g. Juno clouds).  

Another option is that we just be ok with this and tell vendors that if they’ve added attributes to the API, those need to go away before the next time they apply for OpenStack Powered status.  For those applying under the 2015.05 Guideline today through late January (which is the next time a Guideline should be going up to the Board for approval [1]) we could advise using a version of Tempest that predates this change.  For those who will be applying under the next Guideline, they’ve got ~7 months from today to split out their additional attributes into a separate API (if they’re going to apply with a Juno or Kilo based product) or get it accepted in-tree (if they’re planning to apply with a Liberty or master-based product).  However, it’s worth pointing out that pragmatically speaking, the window is probably more like 3 months, which feels like not a lot of time to make product API changes.  That’s because the Tokyo Summit is when we’ll have a “solid” draft Guideline out [2], when we’ll generally be talking with the community a lot, and it’s pragmatically when we’d expect a lot of vendors to start testing and discovering they have a problem.

[1] http://git.openstack.org/cgit/openstack/defcore/tree/process/2015A.rst#n10
[2] http://git.openstack.org/cgit/openstack/defcore/tree/process/2015A.rst#n22

At Your Service,

Mark T. Voelker

> 
> Thanks,
> John
> 
>> Have I understood the concern correctly?
>> 
>> [1] http://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/api-microversions.html
>> [2] Such as Neutron: http://specs.openstack.org/openstack/neutron-specs/specs/liberty/microversioning.html
>> 
>> At Your Service,
>> 
>> Mark T. Voelker
>> 
>>> On Jun 19, 2015, at 5:16 AM, Ken'ichi Ohmichi <ken1ohmichi at gmail.com> wrote:
>>> 
>>> Hi,
>>> 
>>> There is a related patch of Tempest[1].
>>> With this patch, Tempest will block additional(including
>>> vendor-specific) parameters on a response of Nova REST API.
>>> So after that, I think Refstack also will deny clouds which provide
>>> such REST API services, because it uses Tempest tests.
>>> I believe this way will make OpenStack interoperability better and
>>> upstream-first trend will be more strong.
>>> But this change affects DefCore thing, so I'd like to notice this and
>>> get feedback.
>>> # During writing this mail, the patch is approved anyway.
>>> 
>>> Thanks
>>> Ken Ohmichi
>>> 
>>> ---
>>> [1]: https://review.openstack.org/#/c/156130/
>>> 
>>> 2015-06-18 12:07 GMT+09:00 Mark Voelker <mvoelker at vmware.com>:
>>>>> On Jun 17, 2015, at 8:51 PM, Rochelle Grober <rochelle.grober at huawei.com> wrote:
>>>>> 
>>>>> I'm replying to this version of John's mail to expand on his "use case" point.
>>>>> 
>>>>> What John calls a use case is, in this instance, the same as a high level "Test Case".  Essentially, this is the documentation/comments that define what the test(s) for this case are doing.  DefCore (or others) could create these high level test cases and use them as the framework into which new tests, specific for the capability, could be written.  This, I think, would be an ideal way to start documenting missing capabilities tests.  For some capabilities, there could be multiple use cases (or test cases) that define the capability.
>>>>> 
>>>>> This could also be used for existing tests, but instead of including code for each case or step in a case, the git repo link to the existing test could be included.  This process could also be used with new tests being linked into the test case document as they are merged into the trunk.
>>>>> 
>>>>> This process provides the higher level test "specs" that developers implement into tests.  It provides both a way to track what tests exist and what don't, and a description of the capability that is understandable by end users and other interested parties not concerned with the "guts," but just the behavior of the capability.
>>>>> 
>>>>> For the case of multiple ways of performing an action, the tests would need to implement all methods that meet DefCore criteria.  If any of the methods result in the appropriate end result, then for DefCore purposes, the test has passed.
>>>> 
>>>> Maybe I’m misunderstanding, but this seems hugely problematic for interoperability.  If we say there’s more than one way to do a thing but you pass if you implement any one of those ways, then app developers still don’t know what methods they can depend on being available.  Instead of interoperable clouds we have clouds that can do the same things but for which I still have to write a bunch of this**:
>>>> 
>>>> def list_images():
>>>>  “”” A function to list images. Because all OpenStack Powered Platforms can do that…somehow.””"
>>>>  if $cloud == ‘vendorA’:
>>>>     # TODO: this also works for vendorX
>>>>     list_images_via_nova_image_api()
>>>>  elif $cloud == ‘vendorB’:
>>>>     # TODO: this also worked for vendorY last week but now, um?
>>>>     list_images_via_glance_v1()
>>>>  elif $cloud == ‘vendorC’:
>>>>     list_images_via_glance_v2()
>>>>  else:
>>>>     # I dunno what cloud this is, but it’s OpenStack Powered! So something must work.
>>>>     try:
>>>>        list_images_via_nova_image_api()
>>>>     except NopeError:
>>>>         # D’oh, guess that wasn’t it…
>>>>         try:
>>>>            list_images_via_glance_v1()
>>>>         except StillNopeError:
>>>>            # Aww…well third time’s the charm?
>>>>            try:
>>>>               list_images_via_glance_v2()
>>>>            except NopeNopeNopeError:
>>>>               rage_quit()
>>>> 
>>>> into any app that I want to be able to run on OpenStack Powered Platforms.  Which is a Bad Thing and basically where we’re at now.  How you expose a capability matters tremendously.  IMHO, interoperability by if/else loop isn’t interoperability at all.
>>>> 
>>>> ** I’m strawmanning slightly here in that I’m pretending these three methods all somehow met DefCore criteria, but you get the picture.
>>>> 
>>>> At Your Service,
>>>> 
>>>> Mark T. Voelker
>>>> 
>>>>> 
>>>>> Does this make sense for DefCore?  As OpenStack matures and acts more like a collection of products rather than a collection of code bases, the processes and collateral of software process that exist to turn code into product come more into play. Hence, the QA as not just verification gate, but a measurement and metrics repository.
>>>>> 
>>>>> Sorry!  Had to get this thought out.  You don't need the QA that most SW companies have until you have product. We are getting close to one and so are now in need of expanding OpenStack QA to encompass the other.
>>>>> 
>>>>> --Rocky
>>>>> 
>>>>> 
>>>>> 
>>>>> -----Original Message-----
>>>>> From: John Garbutt [mailto:john at johngarbutt.com]
>>>>> Sent: Wednesday, June 17, 2015 09:57
>>>>> To: defcore-commit.
>>>>> Cc: Nikhil Komawar
>>>>> Subject: [OpenStack-DefCore] Image APIs in Glance and Nova
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> So this was raised in the cross project meeting, and thank you for that.
>>>>> 
>>>>> I keep mentioning use cases, so here we go...
>>>>> * create from cloud base image for ubuntu 12.04
>>>>> * above but with boot from volume
>>>>> * create a snapshot
>>>>> * download snapshot
>>>>> * upload image into another openstack cloud
>>>>> * boot server from uploaded image
>>>>> 
>>>>> Now, at a higher level:
>>>>> * user does above using custom script for Cloud A and Cloud B
>>>>> * user keeps to just the APIs that are defcore tested
>>>>> * user gets access to Cloud C and Cloud D
>>>>> * user wants to point script at new clouds, and everything should just work
>>>>> 
>>>>> So I think thats where we want to be. Now lets dig in...
>>>>> 
>>>>> To list images, the user could:
>>>>> * use nova to list images (stable, but project wants to delete it)
>>>>> * use glance v1 (should never be exposed to end users, was designed as
>>>>> internal only API)
>>>>> * use glance v2 (only just released, not really deployed anywhere)
>>>>> 
>>>>> For the
>>>>> 
>>>>> _______________________________________________
>>>>> Defcore-committee mailing list
>>>>> Defcore-committee at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
>>>>> 
>>>>> _______________________________________________
>>>>> Defcore-committee mailing list
>>>>> Defcore-committee at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
>>>> 
>>>> _______________________________________________
>>>> Defcore-committee mailing list
>>>> Defcore-committee at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
>> 
>> _______________________________________________
>> Defcore-committee mailing list
>> Defcore-committee at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee