Open Stack

I think this also brings up a good point from yesterday's discussion about "what is an interim test?". We all vigorously agree that compliance to the API contract falls into interop testing. We also agreed that non-happy path (negative responses) are also in the scope of interop. I think this test brings up another aspect important to interop, which for lack of a better term I will call application behavior.


It would be easy for the API responses to the create image/delete image to meet the correct response codes and schemas, but the behavior of the application to not be correct (the delete request is ignored, the image goes into the error state). Given that we consider non-happy path API tests to be in consideration for interop, I think non-happy path application behavior would also fall into scope. I do think we can be more clear about what aspects of the OpenStack projects fall into scope (the artifacts created by an application like VMs, volumes, etc are others that we include in our tests now) so that it is more apparent what tests should fall into scope.


Daryl

________________________________
From: Gema Gomez-Solano <gema.gomez-solano at canonical.com>
Sent: Thursday, March 10, 2016 9:59 AM
To: Catherine Cuong Diep
Cc: Daryl Walleck; defcore-committee at lists.openstack.org
Subject: Re: [OpenStack-DefCore] Fw: Discussing tests

In my view the call to delete image should work regardless of the status of the image (from interoperability viewpoint). This is an async call, right? so it should be queued and dealt with in due time (i.e. whenever the save operation is finished). This is the behavior I would expect as an end user, havent looked at the test yet, will do when i get home!

Cheers,
Gema

Sent from my iPhone

On 10 Mar 2016, at 09:42, Catherine Cuong Diep <cdiep at us.ibm.com<mailto:cdiep at us.ibm.com>> wrote:


I think the question for DefCore to review is:

Is "being able to delete an image that is not done saving " an important capability for interoperability ?

Catherine Diep
IBM Silicon Valley Laboratory, San Jose, California 95141
cdiep at us.ibm.com<mailto:cdiep at us.ibm.com>, Tel: (408) 463-4352 T/L: 543-4352
----- Forwarded by Catherine Cuong Diep/San Jose/IBM on 03/10/2016 07:37 AM -----

From: Daryl Walleck <daryl.walleck at RACKSPACE.COM<mailto:daryl.walleck at rackspace.com>>
To: Jean-Daniel Bonnetot <jean-daniel.bonnetot at corp.ovh.com<mailto:jean-daniel.bonnetot at corp.ovh.com>>, "defcore-committee at lists.openstack.org<mailto:defcore-committee at lists.openstack.org>" <defcore-committee at lists.openstack.org<mailto:defcore-committee at lists.openstack.org>>
Date: 03/10/2016 07:25 AM
Subject: Re: [OpenStack-DefCore] Discussing tests

________________________________



My understanding of that test is that it is validating the system behavior being able to delete an image that is not done saving.

As to your question of atomicity, one of the outcomes of the DefCore midcycle is an audit of the existing tests to provide a precise list of what API calls each test makes and what assertions are really being made. The outcome of that audit should help with these types of questions about individual tests.

Daryl

From: Jean-Daniel Bonnetot<mailto:jean-daniel.bonnetot at corp.ovh.com>
Sent: Thursday, March 10, 2016 9:05 AM
To: defcore-committee at lists.openstack.org<mailto:defcore-committee at lists.openstack.org>
Subject: [OpenStack-DefCore] Discussing tests

Hi defcore,

Not sure it’s the right place to discuss about it but I try ;)

I’m using refstack to test our Public Cloud at OVH and I have question about a test which make trouble on our solution.

I run the 2015.07 guidelines.
The test is tempest.api.compute.images.test_images.ImagesTestJSON.test_delete_saving_image.

I saw this test corresponds to the compute-images-delete capability.
I understand that image deletion is required, but why the test is deletion on a saving image?

In this particular case, at OVH we had to patch Glance to forbid deletion if the image is in saving state because of some Ceph issue.
I saw that during the last meetup you talked about "Atomicity of tests », I think that’s what I’m talking about too and I agree that it’s a problem we need to solve.

compute-images-delete is marked as atomic but it’s not from my point of view.


--
Jean-Daniel Bonnetot
http://www.ovh.com<http://www.ovh.com/>
@pilgrimstack




_______________________________________________
Defcore-committee mailing list
Defcore-committee at lists.openstack.org<mailto:Defcore-committee at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee_______________________________________________
Defcore-committee mailing list
Defcore-committee at lists.openstack.org<mailto:Defcore-committee at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


_______________________________________________
Defcore-committee mailing list
Defcore-committee at lists.openstack.org<mailto:Defcore-committee at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/defcore-committee/attachments/20160310/4f720ffe/attachment.html>