Open Stack

Ala Rezmerita just created an etherpad with the issues description (Thanks
Ala!)

https://etherpad.openstack.org/gate-tempest-devstack-vm-quantum-full


On Wed, Jun 19, 2013 at 9:50 AM, Miguel Lavalle <miguel at mlavalle.com> wrote:

> Hi,
>
> During the last Quantum IRC meeting, I mentioned that in working with
> Tempest tests we have identified a good number of situations where the nova
> and quantum api's behaviors are not consistent. Below you will find a
> document that describes the identified issues and who is assigned to fix
> them. If anyone knows of other issues not listed below or wants to help,
> please let us know.
>
> Fix Jenkins gate-tempest-devstack-vm-quantum-full
>
> This document lists all the Tempest tests that are failing on a Devstack +
> Quantum Trunk setup. The goal is to make these tests pass for the Havana
> Milestone 3. Alla, Miguel and Jordan are working on this.
>
> For each test :
>
>    -
>
>    Provide the full path of the test. If the test has a double interface
>    (JSON and XML), it’s enough to provide only the JSON path.
>    -
>
>    Provide a small stacktrace or a link to a Paste service such as
>    http://paste.openstack.org/
>    -
>
>    Would be great to provide the CURL URL that triggers the bug
>    -
>
>    Provide  a small analysis of what the bug may be and the components
>    involved (Quantum, Quantum API on Nova’s side, Nova-network etc.)
>    -
>
>    As far as possible, provide the file and the line number where the
>    Python exception is raised or translated (“casted”)
>    -
>
>    If already filed, an URL to the bug report in Launchpad
>
> Tests related to the Fixed IPs Compute API extension (Miguel)
>
> 1)
> tempest.api.compute.admin.test_fixed_ips:FixedIPsTestJson.test_list_fixed_ip_details
>
>    -
>
>    Trace : http://paste.openstack.org/show/38363/
>    -
>
>    curl -H "X-Auth-Token:$TOKEN" -X GET http://
>    $IP:8774/v2/$TENANT_ID/os-fixed-ips/10.0.0.3
>    -
>
>    API : http://api.openstack.org/api-ref.html#ext-os-fixed-ips
>    -
>
>    Explanation : Call to
>    nova.api.openstack.compute.contrib.fixed_ips.FixedIPController::show() This
>    file doesn’t use the Quantum API nor the Nova-Network API. It interacts
>    directly with the DB, which is bad.
>    -
>
>    A possible fix would be to :
>    1.
>
>       Change
>       nova.api.openstack.compute.contrib.fixed_ips.FixedIPController::show() to
>       use either nova.network.api.API::get_fixed_ip() (for Nova Network) or
>       nova.network.quantumv2.api.API:get_fixed_ip() (for Nova Network)
>       2.
>
>       Implement nova.network.quantumv2.api.API:get_fixed_ip() which
>       currently raises a NotImplementedError exception
>
> 2)
> tempest.api.compute.admin.test_fixed_ips:FixedIPsTestJson.test_set_reserve
>
>    -
>
>    Trace : http://paste.openstack.org/show/38372/
>    -
>
>    curl  -H "X-Auth-Token:$TOKEN" -X POST http://
>    $IP:8774/v2/$TENANT_ID/os-fixed-ips/10.0.0.3/action
>    -
>
>    API : http://api.openstack.org/api-ref.html#ext-os-fixed-ips
>    -
>
>    Possible Fix : Should call
>    nova.api.openstack.compute.contrib.fixed_ips.FixedIPController::_set_reserved()
>    (once the result of db.fixed_ip_get_by_address() is made through the API).
>    Here the problem is that neither Nova-Network nor Quatum API implement an
>    equivalent of db.fixed_ip_update()
>
> 3)
> tempest.api.compute.admin.test_fixed_ips:FixedIPsTestJson.test_set_unreserve
>
>    -
>
>    Same as 2)
>
>
>
> Tests related to Quotas Admin(Miguel)
>
>
> 4)tempest.api.compute.admin.test_quotas:QuotasAdminTestJSON.test_security_groups_exceed_limit
>
>
> 5)tempest.api.compute.admin.test_quotas:QuotasAdminTestJSON.test_security_groups_rules_exceed_limit
>
> Tests related to Floating IPs
>
> 6)
> tempest.api.compute.floating_ips.test_floating_ips_actions:FloatingIPsTestJSON.test_associate_ip_to_server_without_passing_floating_ip
>
>    -
>
>    Trace Tempest http://paste.openstack.org/show/38431/
>    -
>
>    Bug : https://bugs.launchpad.net/quantum/+bug/1190242
>
>
>    -
>
>    curl -H "Content-Type: application/json" -H "X-Auth-Token:$TOKEN" -X
>     POST http://$IP:8774/v2/$TENANT_ID/servers/$SERVER_ID/action<http://10.1.59.157:8774/v2/de6c5fbc55a34cbcaa3d79eb6b21a784/servers/0b2ad3b6-c14a-4d89-b2a0-c015f0a88a1f/action>-d ‘{"addFloatingIp": {"address": ""}}’
>    -
>
>    Notice that the address of the FloatingIP is empty.
>    -
>
>    API : http://api.openstack.org/api-ref.html#ext-floating-ips
>    -
>
>    Explanation : Mismatch in the 2 API. If there is only one IP in the
>    pool, Quantum allows the floating IP to be empty and returns the one and
>    only IP in the pool. Nova-network doesn’t allow this and returns a 404
>    -
>
>    Review https://review.openstack.org/#/c/32740/
>
>
> 7)
> tempest.api.compute.floating_ips.test_floating_ips_actions:FloatingIPsTestJSON.test_delete_floating_ip
>
>    -
>
>    Trace Tempest : http://paste.openstack.org/show/38505/
>    -
>
>    Bug : https://bugs.launchpad.net/tempest/+bug/1160309 (see comment #2)
>    -
>
>    This is related to bug 9)
>
>
> 8)tempest.api.compute.floating_ips.test_floating_ips_actions:FloatingIPsTestJSON.test_delete_nonexistant_floating_ip
>
>
>    -
>
>    Related to 9)
>
>
> 9)
> tempest.api.compute.floating_ips.test_list_floating_ips:FloatingIPDetailsTestJSON.test_get_nonexistant_floating_ip_details
>
>    -
>
>    Bug :https://bugs.launchpad.net/tempest/+bug/1160309
>
>
>    -
>
>    Trace tempest:http://paste.openstack.org/show/38430/
>    -
>
>    Trace nova: http://paste.openstack.org/show/38433/
>    -
>
>    Curl:curl -H "X-Auth-Token:$TOKEN" -X GET http://
>    $IP:8774/v2/$PROJECT_ID/os-floating-ips/99987878787878
>
> Proposed Fix: https://review.openstack.org/33024
>
> Tests Related to Security Groups (Jordan)
>
> 10)t
> empest.api.compute.security_groups.test_security_group_rules:SecurityGroupRulesTestJSON.test_security_group_rules_create_with_invalid_id
>
>    -
>
>    TRACE :  http://paste.openstack.org/show/38373/
>    -
>
>    curl -H "Content-Type: application/json" -H "X-Auth-Token:$TOKEN" -X
>     POST http://$IP:8774/v2/$TENANT_ID/os-security-group-rules -d
>    ‘{"security_group_rule": {"from_port": 22, "ip_protocol": "tcp", "to_port":
>    22, "parent_group_id": "9991393497170", "cidr": null, "group_id":
>    null}}’
>
>
>    -
>
>    Explanation : Notice that the parent_goup_id is  a numerical ID and
>    not a UUID. Quantum has an additional check to validate that the ID is an
>    UUID (see nova/network/security_group/quantum_driver.py::validate_id())
>
>
>    -
>
>    API : http://api.openstack.org/api-ref.html#ext-os-security-groups
>    -
>
>    Bug : https://bugs.launchpad.net/tempest/+bug/1182384
>    -
>
>    Review : https://review.openstack.org/#/c/29899/
>
>
> 11)
> tempest.api.compute.security_groups.test_security_group_rules:SecurityGroupRulesTestJSON.test_security_group_rules_delete_with_invalid_id
>
>    -
>
>    http://paste.openstack.org/show/38424/
>    -
>
>    curl  -H "X-Auth-Token:$TOKEN" -X  DELETE http://
>    $IP:8774/v2$TENANT_ID/os-security-group-rules/9991407551273
>    -
>
>    Explanation : Same bug as 10
>    -
>
>    API : http://api.openstack.org/api-ref.html#ext-os-security-groups
>    -
>
>    Review : https://review.openstack.org/#/c/29899/
>
> 12)
> tempest.api.compute.security_groups.test_security_groups:SecurityGroupsTestJSON.test_delete_nonexistant_security_group
>
>    -
>
>    Same as bug 10
>    -
>
>    Review : https://review.openstack.org/#/c/29899/
>
>
>
> 13)tempest.api.compute.security_groups.test_security_groups:SecurityGroupsTestJSON.test_security_group_get_nonexistant_group
>
>    -
>
>    Same as bug 10
>    -
>
>    Review : https://review.openstack.org/#/c/29899/
>
>
> 14)
> tempest.api.compute.security_groups.test_security_groups:SecurityGroupsTestJSON.test_security_group_create_with_duplicate_name
>
> Security Group with duplicate name should not be created, but two groups
> with the same name can be created in quantum. We have here the same problem
> as in 15 and in 16. With Quantum, there is no validation that a  group with
> given name exists already or if the given SG name is empty or is composed
> of white spaces or is more than 255 chars.
>
> In the description of bug https://bugs.launchpad.net/nova/+bug/1161411this issue is generally discussed. SecurityGroup API  are
> based on the ID and not the names, except for adding an instance to a
> security group.  In order to solve the last problem the bug
> https://bugs.launchpad.net/nova/+bug/1161473 was added.
>
> The major question is if these 3 tests (14, 15, 16) : does the name of a
> security group is really that important? If so, we must add some validation
> methods. If not the test suit concerning this part must be disable in
> tempest. What do you think Miguel?
>
>
>  15)
> tempest.api.compute.security_groups.test_security_groups:SecurityGroupsTestJSON.test_security_group_create_with_invalid_group_description
>
>
> 16)tempest.api.compute.security_groups.test_security_groups:SecurityGroupsTestJSON.test_security_group_create_with_invalid_group_name
>
> BUG:https://bugs.launchpad.net/nova/+bug/1161411 +
> https://bugs.launchpad.net/nova/+bug/1161473
>
> Traceback(tempest) :http://paste.openstack.org/show/38423/
>
> The Security Group should not be created with group name an empty string
> or with white spaces/chars more than 255
>
> CURL:curl -H "Content-Type: application/json" -H "X-Auth-Token:$TOKEN" -X
> POST http://$IP:8774/v2/$PROJECT_ID/os-security-groups -d
>  '{"security_group": {"name": " ", "description":
> "description-1554950088"}}'
>
> curl -H "Content-Type: application/json" -H "X-Auth-Token:$TOKEN" -X POST
> http://$IP:8774/v2/$PROJECT_ID/os-security-groups -d  '{"security_group":
> {"name": "  ", "description": "description-1554950088"}}'
>
>
> 17)tempest.api.compute.security_groups.test_security_groups:SecurityGroupsTestJSON.test_server_security_groups
>
>    -
>
>    Tempest Trace : http://paste.openstack.org/show/38427/
>    -
>
>    Fixed by https://review.openstack.org/#/c/32288/ (merged on June,12th)
>
>
>
> Tests related to servers(Ala)
>
> 18)
> tempest.api.compute.servers.test_list_server_filters:ListServerFiltersTestXML.test_list_servers_filtered_by_ip_regex
>
>    -
>
>    BUG: https://bugs.launchpad.net/quantum/+bug/1182883
>    -
>
>    BP: https://blueprints.launchpad.net/quantum/+spec/like-op-list
>    -
>
>    CURL : GET http://$IP:8774/v2/$PROJECT_ID/servers?ip=10.
>    -
>
>    Explanation : The  regex search is not supported by Quantum. Thus
>    Quantum returns a 404 Not Found (0 server match) where Tempest expects one
>    server to be found.
>    -
>
>    Possible Fix : For "search port by IP with regex" feature, I think the
>    best place to hack it would be in file
>    quantum/db/db_base_plugin_v2.py::_get_ports_query()
>
>
>
> 19)tempest.api.compute.servers.test_servers_negative:ServersNegativeTest.test_create_with_nonexistent_security_group
>
> FIXED : https://review.openstack.org/#/c/30271/
>
>
> 20)tempest.api.compute.servers.test_virtual_interfaces:VirtualInterfacesTestXML.test_list_virtual_interfaces
>
>    -
>
>    TRACE (NOVA): http://paste.openstack.org/show/38371/
>    -
>
>    BUG: https://bugs.launchpad.net/tempest/+bug/1183436
>    -
>
>    CURL: GET http://
>    $IP:8774/v2/$PROJECT_ID/servers/$SERVER/os-virtual-interfaces
>    -
>
>    Explanation: This HTTP request calls the Quantum API
>    (nova/nova/network/quantumv2/api.py) and specifically the get_vifs_by_*
>    methods which are not implemented (raise NotImplementedError())
>    -
>
>    Possible Fix:
>    -
>
>       skip this test if Quantum is enabled as set in Tempest
>       configuration. Or
>       -
>
>       Implement the get_vifs_by_* methods
>
> PATCH: (still not approved)  https://review.openstack.org/#/c/31755/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130619/1c797364/attachment.html>