[OpenStack-DefCore] Identifying and defining missing/new tests for capabilities :Was Image APIs in Glance and Nova

John Garbutt john at johngarbutt.com
Fri Jun 19 17:15:18 UTC 2015


Sorry for going back in time here...

On 18 June 2015 at 15:43, Monty Taylor <mordred at inaugust.com> wrote:
> On 06/18/2015 10:38 AM, Rob Hirschfeld wrote:
>> top posting because I'm not sure where to inject this....
>>
>> At some point, reading this leads me to wonder if DefCore may have to
>> pick a "right way" (tm) from these choices.  If we believe that to be
>> true then the sooner we start on that journey the better.  It will need
>> to be a major topic at the mid-cycle.
>
> Yah. I mean, I'm hoping that the TC can start picking some "right ways",
> communicate that clearly to DefCore and have DefCore validate it as a
> good idea (maybe even using something similar to the current "future
> direction" concept we have now)

Its generally not about "picking" a way, its more about "creating" way
that works for all cases.

Right now, its usually unrelated technology choices that force a
deployer to only enable one for three methods, because when using
upstream code only one of the three methods actually work in your
situation. That the thing that really sucks here :(

> However, whether it's always like that, or whether some of those
> thoughts originate on the defcore set - it's still certainly a heavy
> topic since it's a bit of a step from being reflective towards
> influencing what folks _Should_ be doing.

I find you influence developers by giving them fun problems to solve.

Lets define a scenario:
* User wants to use script against any DefCore compliant cloud
* User restricts themselves to only DefCore tested APIs
* They should write script against cloud A, and it should work against cloud B

Now there are two sides to this:
* what use cases are possible today
* what use cases are considered the most important soon

That really helps concentrate the mind on whats required here.

Thanks,
John

>> On 06/18/2015 05:20 AM, Monty Taylor wrote:
>>> On 06/17/2015 11:07 PM, Mark Voelker wrote:
>>>>> On Jun 17, 2015, at 8:51 PM, Rochelle Grober
>>>>> <rochelle.grober at huawei.com> wrote:
>>>>>
>>>>> I'm replying to this version of John's mail to expand on his "use
>>>>> case" point.
>>>>>
>>>>> What John calls a use case is, in this instance, the same as a high
>>>>> level "Test Case".  Essentially, this is the documentation/comments
>>>>> that define what the test(s) for this case are doing.  DefCore (or
>>>>> others) could create these high level test cases and use them as the
>>>>> framework into which new tests, specific for the capability, could
>>>>> be written.  This, I think, would be an ideal way to start
>>>>> documenting missing capabilities tests.  For some capabilities,
>>>>> there could be multiple use cases (or test cases) that define the
>>>>> capability.
>>>>>
>>>>> This could also be used for existing tests, but instead of including
>>>>> code for each case or step in a case, the git repo link to the
>>>>> existing test could be included.  This process could also be used
>>>>> with new tests being linked into the test case document as they are
>>>>> merged into the trunk.
>>>>>
>>>>> This process provides the higher level test "specs" that developers
>>>>> implement into tests.  It provides both a way to track what tests
>>>>> exist and what don't, and a description of the capability that is
>>>>> understandable by end users and other interested parties not
>>>>> concerned with the "guts," but just the behavior of the capability.
>>>>>
>>>>> For the case of multiple ways of performing an action, the tests
>>>>> would need to implement all methods that meet DefCore criteria.  If
>>>>> any of the methods result in the appropriate end result, then for
>>>>> DefCore purposes, the test has passed.
>>>> Maybe I’m misunderstanding, but this seems hugely problematic for
>>>> interoperability.  If we say there’s more than one way to do a thing
>>>> but you pass if you implement any one of those ways, then app
>>>> developers still don’t know what methods they can depend on being
>>>> available.  Instead of interoperable clouds we have clouds that can
>>>> do the same things but for which I still have to write a bunch of
>>>> this**:
>>> +10000
>>>
>>>> def list_images():
>>>>     “”” A function to list images. Because all OpenStack Powered
>>>> Platforms can do that…somehow.””"
>>>>     if $cloud == ‘vendorA’:
>>>>        # TODO: this also works for vendorX
>>>>        list_images_via_nova_image_api()
>>>>     elif $cloud == ‘vendorB’:
>>>>        # TODO: this also worked for vendorY last week but now, um?
>>>>        list_images_via_glance_v1()
>>>>     elif $cloud == ‘vendorC’:
>>>>        list_images_via_glance_v2()
>>>>     else:
>>>>        # I dunno what cloud this is, but it’s OpenStack Powered! So
>>>> something must work.
>>>>        try:
>>>>           list_images_via_nova_image_api()
>>>>        except NopeError:
>>>>            # D’oh, guess that wasn’t it…
>>>>            try:
>>>>               list_images_via_glance_v1()
>>>>            except StillNopeError:
>>>>               # Aww…well third time’s the charm?
>>>>               try:
>>>>                  list_images_via_glance_v2()
>>>>               except NopeNopeNopeError:
>>>>                  rage_quit()
>>> This is exactly why we had to write shade - that WAS in fact what we
>>> were having to put into all of our apps - complete with the rage_quit.
>>>
>>>> into any app that I want to be able to run on OpenStack Powered
>>>> Platforms.  Which is a Bad Thing and basically where we’re at now.
>>>> How you expose a capability matters tremendously.  IMHO,
>>>> interoperability by if/else loop isn’t interoperability at all.
>>>>
>>>> ** I’m strawmanning slightly here in that I’m pretending these three
>>>> methods all somehow met DefCore criteria, but you get the picture.
>>> /me hands Mark a fruit basket of agreement
>>>
>>>> At Your Service,
>>>>
>>>> Mark T. Voelker
>>>>
>>>>> Does this make sense for DefCore?  As OpenStack matures and acts
>>>>> more like a collection of products rather than a collection of code
>>>>> bases, the processes and collateral of software process that exist
>>>>> to turn code into product come more into play. Hence, the QA as not
>>>>> just verification gate, but a measurement and metrics repository.
>>>>>
>>>>> Sorry!  Had to get this thought out.  You don't need the QA that
>>>>> most SW companies have until you have product.  We are getting close
>>>>> to one and so are now in need of expanding OpenStack QA to encompass
>>>>> the other.
>>>>>
>>>>> --Rocky
>>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: John Garbutt [mailto:john at johngarbutt.com]
>>>>> Sent: Wednesday, June 17, 2015 09:57
>>>>> To: defcore-commit.
>>>>> Cc: Nikhil Komawar
>>>>> Subject: [OpenStack-DefCore] Image APIs in Glance and Nova
>>>>>
>>>>> Hi,
>>>>>
>>>>> So this was raised in the cross project meeting, and thank you for
>>>>> that.
>>>>>
>>>>> I keep mentioning use cases, so here we go...
>>>>> * create from cloud base image for ubuntu 12.04
>>>>> * above but with boot from volume
>>>>> * create a snapshot
>>>>> * download snapshot
>>>>> * upload image into another openstack cloud
>>>>> * boot server from uploaded image
>>>>>
>>>>> Now, at a higher level:
>>>>> * user does above using custom script for Cloud A and Cloud B
>>>>> * user keeps to just the APIs that are defcore tested
>>>>> * user gets access to Cloud C and Cloud D
>>>>> * user wants to point script at new clouds, and everything should
>>>>> just work
>>>>>
>>>>> So I think thats where we want to be. Now lets dig in...
>>>>>
>>>>> To list images, the user could:
>>>>> * use nova to list images (stable, but project wants to delete it)
>>>>> * use glance v1 (should never be exposed to end users, was designed as
>>>>> internal only API)
>>>>> * use glance v2 (only just released, not really deployed anywhere)
>>>>>
>>>>> For the
>>>>>
>>>>> _______________________________________________
>>>>> Defcore-committee mailing list
>>>>> Defcore-committee at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
>>>>>
>>>>> _______________________________________________
>>>>> Defcore-committee mailing list
>>>>> Defcore-committee at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
>>>> _______________________________________________
>>>> Defcore-committee mailing list
>>>> Defcore-committee at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
>>>>
>>>
>>> _______________________________________________
>>> Defcore-committee mailing list
>>> Defcore-committee at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
>>
>
>
> _______________________________________________
> Defcore-committee mailing list
> Defcore-committee at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee



More information about the Defcore-committee mailing list