[OpenStack-DefCore] Discussing tests

Chris Hoge chris at openstack.org
Fri Mar 11 15:25:09 UTC 2016


> On Mar 11, 2016, at 6:36 AM, Jean-Daniel Bonnetot <jean-daniel.bonnetot at corp.ovh.com> wrote:
> 
> Agree, we are doing a non-standard stuffs when it’s required and we have to do it.
> I’m not saying we can’t manage it in a different way, I’m saying that with different constraints we often need to deal and choose between answering customers needs, make our infrastructure stable and scalable and being fully compliant.
> 
> I need to be more precise on the things we corrected on our infrastructure. We raise a 409 error on instance deletion when task_state is image_pending_upload.
> So the modification is on Nova side (my bad, I told about Glance).
> 
> Anyway, the steps in tempest.api.compute.images.test_images.ImagesTestJSON.test_delete_saving_image are:
> 1. Creating an instance
> 2. Snapshotting the instance
> 3. Deleting the image while the image is still uploading
> 4. Deleting the instance while the instance is still snapshotting (here we have our hack on the API)
> 
> So the capability compute-images-delete can’t passe because of a non-standard API answer on … DELETE /v2/{tenantid}/server/{serverid} in a particular case which is "task_state is image_pending_upload".
> 
> What I’m trying to say is that I’m not sure this kind of strange situation serve the interop goal.
> Interop capabilities defined compute-images-delete as a requirement and it’s good, but the workflow used to test it is a little bit overkill to answer this question I think.

I’m not a fan of throwing out a test because of a breaking downstream change.
I understand your concern, but my preference is to have the vendor attempt to
fix either the upstream problem, the downstream problem, or the test problem. This
would indicate a commitment to interoperability rather than a desire to just pass
a guideline to get a trademark.

This indicates a larger issue that keeps coming up, where operators are deploying
Ceph as supporting storage infrastructure, and breaking interoperability in the
process. To me this is an indication that we should consider adding a ceph-backed
deployment to the integrated gate so we can catch these issues earlier rather than
later.

Despite not being an OpenStack project, Ceph is an important part of the
operating ecosystem.

-Chris

> --
> Jean-Daniel Bonnetot
> http://www.ovh.com
> @pilgrimstack
> 
> 
> 
> 
>> Le 10 mars 2016 à 20:05, Daryl Walleck <daryl.walleck at RACKSPACE.COM> a écrit :
>> 
>> ​Jean-Daniel,
>> 
>> Actually, I needed to take a step back to catch one important detail that I missed.
>> 
>> "In this particular case, at OVH we had to patch Glance to forbid deletion if the image is in saving state because of some Ceph issue."
>> 
>> All the original thoughts I had still hold true, but the purpose of the DefCore tests is too identify non-standard behavior from the cloud-under-test, which it did. The question of whether this particular snag is relevant for interop is still valid.
>> 
>> Daryl
>> 
>> From: Catherine Cuong Diep <cdiep at us.ibm.com>
>> Sent: Thursday, March 10, 2016 9:42 AM
>> To: Daryl Walleck
>> Cc: Jean-Daniel Bonnetot; defcore-committee at lists.openstack.org
>> Subject: Fw: [OpenStack-DefCore] Discussing tests
>> 
>> I think the question for DefCore to review is:
>> 
>> Is "being able to delete an image that is not done saving " an important capability for interoperability ?
>> 
>> Catherine Diep
>> IBM Silicon Valley Laboratory, San Jose, California 95141
>> cdiep at us.ibm.com, Tel: (408) 463-4352 T/L: 543-4352
>> ----- Forwarded by Catherine Cuong Diep/San Jose/IBM on 03/10/2016 07:37 AM -----
>> 
>> From: Daryl Walleck <daryl.walleck at RACKSPACE.COM>
>> To: Jean-Daniel Bonnetot <jean-daniel.bonnetot at corp.ovh.com>, "defcore-committee at lists.openstack.org" <defcore-committee at lists.openstack.org>
>> Date: 03/10/2016 07:25 AM
>> Subject: Re: [OpenStack-DefCore] Discussing tests
>> 
>> 
>> 
>> My understanding of that test is that it is validating the system behavior being able to delete an image that is not done saving.
>> 
>> As to your question of atomicity, one of the outcomes of the DefCore midcycle is an audit of the existing tests to provide a precise list of what API calls each test makes and what assertions are really being made. The outcome of that audit should help with these types of questions about individual tests.
>> 
>> Daryl
>> 
>> From: Jean-Daniel Bonnetot
>> Sent: Thursday, March 10, 2016 9:05 AM
>> To: defcore-committee at lists.openstack.org
>> Subject: [OpenStack-DefCore] Discussing tests
>> Hi defcore,
>> 
>> Not sure it’s the right place to discuss about it but I try ;)
>> 
>> I’m using refstack to test our Public Cloud at OVH and I have question about a test which make trouble on our solution.
>> 
>> I run the 2015.07 guidelines.
>> The test is tempest.api.compute.images.test_images.ImagesTestJSON.test_delete_saving_image.
>> 
>> I saw this test corresponds to the compute-images-delete capability. 
>> I understand that image deletion is required, but why the test is deletion on a saving image?
>> 
>> In this particular case, at OVH we had to patch Glance to forbid deletion if the image is in saving state because of some Ceph issue.
>> I saw that during the last meetup you talked about "Atomicity of tests », I think that’s what I’m talking about too and I agree that it’s a problem we need to solve.
>> 
>> compute-images-delete is marked as atomic but it’s not from my point of view.
>> 
>> 
>> --
>> Jean-Daniel Bonnetot
>> http://www.ovh.com
>> @pilgrimstack
>> 
>> 
>> 
>> 
>> _______________________________________________
>> Defcore-committee mailing list
>> Defcore-committee at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee_______________________________________________
>> Defcore-committee mailing list
>> Defcore-committee at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
> 
> 
> _______________________________________________
> Defcore-committee mailing list
> Defcore-committee at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee




More information about the Defcore-committee mailing list