Open Stack

Hi,

We've got a couple of problems running original Tempest EC2 API test against new standalone stackforge/ec2-api project and
I wanted to ask for some advice about how to deal with it.

Tempest now is running against our ec2-api after this review was closed -
https://review.openstack.org/#/c/170258/

And now we face two problems (that can also be found in tempest logs of this review -
https://review.openstack.org/#/c/170668/)
For now I switched tempest gating job to non-voting until these problems are resolved in the following review - 
https://review.openstack.org/#/c/170646/

Problems are:
1) tempest.thirdparty.boto.test_ec2_network.EC2NetworkTest.test_disassociate_not_associated_floating_ip
this test tries to allocate address and disassociate it without association.
Amazon allows to do it and does not throw error. But EC2 implementation in Nova throws error.
We have the same test in our own test suite against stackforge/ec2-api (but it's not merged yet) and I checked it against Amazon.
I suggest to remove this test from tempest as incompatible with Amazon.
Also it can be skipped but for me it has no sense.

2) tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_compute_with_volumes
This test registers three images by their manifests, run instance with image/kernel/ramdisk parameters,
and ssh into this instance to check something.
This is not the only test that runs instance with such parameters but this is the only one
that ssh-s into such an instance.
This instance runs but test can't ssh into it and it fails. Because this instance doesn't have ramdisk and kernel.
It runs supplied with image property only. The VM comes up semi-functional and instance can't boot up as a result.
Problem is in the ec2-api/nova communication. Nova public API doesn't support kernel and ramdisk parameters during instance creation.

Next I'll file a bug to ec2-api with this description.

In the long run we should discuss adding this feature to public API but for now we'd like to put Tempest
in our project back to voting state.
We've got several options about what to do for this and we need some help to pick one (or several):
1) skip this test in tempest and switch tempest back to voting state in our project.
The problem is that this test is still also employed against nova's EC2 so it'll get skipped there as well.
2) Leave it as non-voting until extension is added to nova.
Great solution but it'll take way too long I understand.
3) add special condition to skipping the test so that it's skipped only when employed against stackforge/ec2-api, 
while still working against nova if it's possible at all and not too much hassle.

Kind regards,
Andrey.