We are satisfied to announce the release of: tempest 17.1.0: OpenStack Integration Testing This release is part of the queens release series. Download the package from: https://pypi.python.org/pypi/tempest For more details, please see below. 17.1.0 ^^^^^^ Prelude ******* This is an intermediate release during the Queens development cycle to make new functionality available to plugins and other consumers. New Features ************ * Adds a new cli option to tempest run, --load-list <list-file> to specify target tests to run from a list-file. The list-file supports the output format of the tempest run --list-tests command. * Add reset group snapshot status API to v3 group_snapshots_client library, min_microversion of this API is 3.19. This feature enables the possibility to reset group snapshot status. * Add reset group status API to v3 groups_client library, min_microversion of this API is 3.20. This feature enables the possibility to reset group status. * Add the *validation_resources* module to tempest.lib. The module provides a set of helpers that can be used to provision and cleanup all the resources required to perform ping / ssh tests against a virtual machine: a keypair, a security group with targeted rules and a floating IP. * Add the proxy_url optional parameter to the get_credentials method in tempest/lib/auth.py so that that helper can be used when going through and HTTP proxy. * Add a new function called "compare_version_header_to_response" to "tempest.lib.common.api_version_utils", which compares the API micoversion in the response header to another microversion using the comparators defined in "tempest.lib.common.api_version_request.APIVersionRequest". It is now possible to determine how to retrieve an attribute from a response body of an API call, depending on the returned microversion. Add a new exception type called "InvalidParam" to "tempest.lib.exceptions", allowing the possibility of raising an exception if an invalid parameter is passed to a library function. * Adds a new config options, "proxy_url". This options is used to configure running tempest through a proxy server. * The RestClient class in tempest.lib.rest_client has a new kwarg parameters, "proxy_url", that is used to set a proxy server. * A new class was added to tempest.lib.http, ClosingProxyHttp. This behaves identically to ClosingHttp except that it requires a proxy url and will establish a connection through a proxy * A new boolean config option "domain_specific_drivers" is added to the section "identity-feature-enabled". This option must be enabled when testing an environment that is configured to use domain- specific identity drivers. * Define below object storage service clients as libraries. Add new service clients to the library interface so the other projects can use these modules as stable libraries without any maintenance changes. * account_client * container_client * object_client * Two extra modules are now marked as stable for plugins, test.py and clients.py. The former includes the test base class with its automatic credentials provisioning and test resource managing fixtures. The latter is built on top of ServiceClients and it adds aliases and a few custom configurations to it. Upgrade Notes ************* * As of the Queens release, tempest no longer tests the identity v2.0 API because the majority of the v2.0 API have been removed from the identity project. Once the Queens release reaches end-of-life, we can remove the v2.0 tempest tests and clean up v2.0 testing cruft. * Remove deprecated APIs from volume v2 volumes_client, and the deprecated APIs are re-realized in volume v2 transfers_client. * create_volume_transfer * show_volume_transfer * list_volume_transfers * delete_volume_transfer * accept_volume_transfer * Remove two deprecated skip decorators in "config" module: "skip_unless_config" and "skip_if_config". * Remove deprecated get_ipv6_addr_by_EUI64 method from data_utils. Use the same method from oslo_utils.netutils. Bug Fixes ********* * Allow to specify new domain parameters: * *user_domain_name* * *user_domain_id* * *project_domain_name* * *project_domain_id* for CLIClient class, whose values will be substituted to "--os-user- domain-name", "--os-user-domain-id", "--os-project-domain-name" and "--os-project-domain-id" respectively during command execution. This allows to prevent possible test failures with authentication in Keystone v3. Bug: #1719687 * Fix list_group_snapshots API in v3 group_snapshots_client： Bug#1715786. The url path for list group snapshots with details API is changed from "?detail=True" to "/detail". Changes in tempest 17.0.0..17.1.0 --------------------------------- a22794d Add release notes for an queens intermediate release. b74a207 Fix test case for updating volume type extra specs bf142fc Make test.py and clients.py as stable for plugins cb94b5e Add proxy_url to get_credentials in auth 065eb4a Cap compute floating IPs tests f082079 Don't read config in Manager class definition 1fbad23 Disable testing of the v2.0 identity API 1e446b5 Rename scenario.manager._create_port to create_port 529b7ec Fix "import xx as xx" grammer 986407d Move the object client to tempest.lib 7451440 Add config options to set proxy_url 98fe4ce Remove scenario.manager.rebuild_server aec0809 Updated from global requirements b61942d Do not set alias user_client for specific client e9877c6 Remove _check_remote_connectivity in scenario.manager b441134 Allow to specify user and project domains in CLIclient 0d4551b Only attempt to detach an in-use volume during cleanup ba0e553 Rename base.rebuild_server to base.recreate_server 420a019 Refactor of _check_tenant_network_connectivity 75891d8 Setup networking for live migration tests 712dafa Remove method get_ipv6_addr_by_EUI64 ebe4698 Update README to use stestr instead of testr for unit tests f36476e Fix object_client methods to accept headers and query param 88eadcf Remove wrapper methods from object_client 50688db Fix senario test: test_swift_acl_anonymous_download cddfdce Move object storage container_client to lib interface ae0e27b Add release note for --load-list eed4031 Fix create container method 0dc4f8e Add compare header version to test images oneserver 5d65cd7 Remove _project_network_cidr in security group tests 1056c13 Remove unnecessary global variables dc6e2a8 Fill microversion doc for implemented volume tests bd7ef1b Remove unnecessary client alias in AvailabilityZoneTestJson 23df2c0 Add test case for reset group snapshot status 1bac618 Fix volume group test bbabe54 Remove unnecessary assertIn fc6280f Remove redundant volume check in nova_volume_detach e65ec45 Add compare header version to create_image_from_server helper b0b272e Remove internal helper _default_security_group 6c9ca4e Add missing addCleanup of _delete_group_snapshot 5db1f11 Delete a volume by Non-admin privileges 7351cbd Fix create, update or delete container metadata method 871b1a8 Make list methods consistent for container client afe96b8 Move internal helpers to the class that uses them 0befe81 Remove deprecated APIs from volume v2 volumes_client f92e6d4 Remove deprecated skip decorators ce3bb9b Fix the 'service' decorator path in doc b1de7a1 Remove unneeded resource_cleanups in compute tests 1fc499e Compute volumes via addClassResourceCleanup 238818c Compute SG via addClassResourceCleanup 0d0a3f3 Compute servers via addClassResourceCleanup decdcf6 Remove one-line helper _delete_volume cca9974 Add test case for reset group status 3b2a215 Fix list_group_snapshots API in v3 group_snapshots_client 991d62d Updated from global requirements fe879ea Fix a typo of a missing letter d0b8ebd Switch to use stestr for unit tests directly e1ed695 Move identity_utils to common.identity ba712ac Add unit tests for not overriding setUpClass 08e42d4 Add unit tests for test class fixtures 9958367 Prepare setup_clients and resource_setup a6c885a Prepare setup_credentials as stable b72cf49 Move object storage account_client to lib interface f78565d Use glance client to delete an image 1b9e59b Updated from global requirements b17f7a3 Compute images via addClassResourceCleanup c527df5 Remove unused _add_router_interface_with_subnet_id helper a5440c8 Prepare skip_checks as stable interface 421dc3c Make skip_checks and setup_creds safe bcbf1af1 Prepare network_resources as a stable interface 0477acc Make validation_resources a stable interface 9f416dd Stop implicit validation_resources provisioning d48fc09 Remove unnecessary executable permissions 3be5748 Make resource_cleanup stable 9806f2d Add a validation resources fixture 8f018cb Capture logs when running unit tests b0dbe4b Fix the redundant use of netaddr.IPNetwork a54f170 Remove unnecessary class variable 7446709 Remove unused RFCViolation aeef897 Remove "test_create_with_nonexistent_volume_type" 63ad7b3 Remove unused helper in RoutersTest e84f818 Remove useless check in AttachInterfacesTestJSON 0a1f6c4 Remove unnecessary back slash b4ec494 Make validation resources leak safe ec61bac Get cidr/mask_bits according to ip_version in resource_setup a1c3af3 [DOC] Fix "Title level inconsistent" warning 1dc2e23 Only choose available compute node as migration dest 6d5a48c Move data directory under doc/source 477fd02 doc migration: update the doc link address 510c0f9 Fix addCleanup in AutoAllocateNetworkTest 0dba54c Add --load-list argument to tempest to accept a non-regex whitelist 4315b63 Fix addCleanup order in test_volume_list_param_tenant b830fae1d Use create_volume wrapper for volume creation 019ad73 Add cost time printing in wait_for_volume_resource_status 9ff5c28 Add compare header version function to tempest.lib 126f5a1 Fix for resetting snapshot status af55ae3 Fix the test case for showing host 4f0887c Avoid using v3 volume clients when v3 is disabled d2eb092 Add release notes page for v17 c9f271d Fix volume microversion link in doc 9cafd3d Fix identity tests when domain specific drivers are enabled 0cc76bf Test size extend for an attached volume 5febd73 [Doc] Remove html_use_smartypants config in doc 5a60991 Unskip test_reassign_port_between_servers Diffstat (except docs and test files) ------------------------------------- .gitignore | 1 + .stestr.conf | 4 + HACKING.rst | 12 +- README.rst | 10 +- data/tempest-plugins-registry.header | 23 - ...domain-param-in-cliclient-a270fcf35c8f09e6.yaml | 17 + .../notes/add-load-list-cmd-35a4a2e6ea0a36fd.yaml | 7 + ...v3-group-snapshots-client-248d41827daf2a0c.yaml | 6 + ...s-api-to-v3-groups-client-9aa048617c66756a.yaml | 6 + ...lidation-resources-to-lib-dc2600c4324ca4d7.yaml | 7 + ...proxy_url_get_credentials-aef66b085450513f.yaml | 6 + ...mpare-header-version-func-de5139b2161b3627.yaml | 15 + ...sable-identity-v2-testing-4ef1565d1a5aedcf.yaml | 7 + ...-list-group-snapshots-api-969d9321002c566c.yaml | 6 + .../notes/http_proxy_config-cb39b55520e84db5.yaml | 9 + ...termediate-queens-release-2f9f305775fca454.yaml | 4 + ...lient-as-stable-interface-d1b07c7e8f17bef6.yaml | 11 + ...is-from-v2-volumes-client-3ca4a5db5fea518f.yaml | 11 + ...eprecated-skip-decorators-f8b42d812d20b537.yaml | 5 + ...ve-get-ipv6-addr-by-EUI64-c79972d799c7a430.yaml | 5 + ...clients-stable-for-plugin-90b1e7dc83f28ccd.yaml | 8 + releasenotes/source/conf.py | 4 - releasenotes/source/index.rst | 1 + releasenotes/source/v17.0.0.rst | 6 + requirements.txt | 18 +- .../api/compute/admin/test_aggregates_negative.py | 13 +- .../compute/admin/test_auto_allocate_network.py | 6 +- tempest/api/compute/admin/test_create_server.py | 34 +- .../api/compute/admin/test_flavors_extra_specs.py | 11 +- .../admin/test_flavors_extra_specs_negative.py | 11 +- tempest/api/compute/admin/test_live_migration.py | 12 + tempest/api/compute/admin/test_quotas.py | 25 +- tempest/api/compute/admin/test_servers_negative.py | 4 +- tempest/api/compute/base.py | 183 +++--- tempest/api/compute/flavors/test_flavors.py | 10 +- tempest/api/compute/floating_ips/base.py | 18 + .../floating_ips/test_floating_ips_actions.py | 56 +- .../test_floating_ips_actions_negative.py | 28 +- .../compute/floating_ips/test_list_floating_ips.py | 29 +- .../test_list_floating_ips_negative.py | 18 +- tempest/api/compute/images/test_image_metadata.py | 5 +- .../api/compute/images/test_images_oneserver.py | 8 +- .../images/test_images_oneserver_negative.py | 17 +- .../api/compute/images/test_list_image_filters.py | 6 +- .../security_groups/test_security_groups.py | 3 +- .../api/compute/servers/test_attach_interfaces.py | 4 - tempest/api/compute/servers/test_create_server.py | 16 +- tempest/api/compute/servers/test_device_tagging.py | 12 +- tempest/api/compute/servers/test_server_actions.py | 51 +- .../api/compute/servers/test_server_personality.py | 58 +- tempest/api/compute/servers/test_servers.py | 40 +- .../api/compute/servers/test_servers_negative.py | 4 +- tempest/api/compute/volumes/test_attach_volume.py | 51 +- tempest/api/identity/admin/v3/test_groups.py | 12 +- tempest/api/identity/admin/v3/test_list_users.py | 21 +- .../api/identity/admin/v3/test_oauth_consumers.py | 2 +- tempest/api/identity/admin/v3/test_tokens.py | 6 +- tempest/api/identity/v3/test_catalog.py | 0 tempest/api/identity/v3/test_projects.py | 3 +- tempest/api/network/admin/test_quotas.py | 8 +- tempest/api/network/admin/test_routers.py | 21 +- tempest/api/network/base.py | 6 + tempest/api/network/test_allowed_address_pair.py | 7 +- tempest/api/network/test_networks.py | 26 +- tempest/api/network/test_ports.py | 16 +- tempest/api/network/test_routers.py | 38 +- tempest/api/network/test_routers_negative.py | 7 +- tempest/api/network/test_security_groups.py | 4 +- .../api/network/test_security_groups_negative.py | 4 +- tempest/api/object_storage/base.py | 4 +- tempest/api/object_storage/test_account_bulk.py | 2 +- .../api/object_storage/test_account_services.py | 2 +- tempest/api/object_storage/test_container_acl.py | 14 +- .../object_storage/test_container_acl_negative.py | 30 +- .../api/object_storage/test_container_quotas.py | 4 +- .../api/object_storage/test_container_services.py | 119 ++-- .../test_container_services_negative.py | 53 +- .../api/object_storage/test_container_staticweb.py | 33 +- tempest/api/object_storage/test_container_sync.py | 2 +- tempest/api/object_storage/test_object_expiry.py | 8 +- tempest/api/object_storage/test_object_services.py | 117 ++-- tempest/api/object_storage/test_object_slo.py | 2 +- tempest/api/object_storage/test_object_version.py | 8 +- tempest/api/volume/admin/test_groups.py | 157 ++++-- tempest/api/volume/admin/test_multi_backend.py | 18 +- tempest/api/volume/admin/test_snapshots_actions.py | 27 +- tempest/api/volume/admin/test_user_messages.py | 0 tempest/api/volume/admin/test_volume_hosts.py | 37 +- .../api/volume/admin/test_volume_quota_classes.py | 6 +- tempest/api/volume/admin/test_volume_quotas.py | 10 +- .../volume/admin/test_volume_types_extra_specs.py | 18 + .../api/volume/admin/test_volume_types_negative.py | 9 - tempest/api/volume/admin/test_volumes_list.py | 2 +- tempest/api/volume/base.py | 19 +- tempest/api/volume/test_availability_zone.py | 10 +- tempest/api/volume/test_volumes_extend.py | 129 +++++ tempest/api/volume/test_volumes_negative.py | 3 +- tempest/clients.py | 27 +- tempest/cmd/run.py | 14 + tempest/cmd/verify_tempest_config.py | 8 +- tempest/common/compute.py | 2 + tempest/common/identity.py | 38 ++ tempest/common/validation_resources.py | 292 ---------- tempest/common/waiters.py | 2 + tempest/config.py | 109 +--- tempest/exceptions.py | 4 - tempest/lib/auth.py | 13 +- tempest/lib/cli/base.py | 24 +- tempest/lib/common/api_version_utils.py | 56 ++ tempest/lib/common/dynamic_creds.py | 74 +-- tempest/lib/common/http.py | 41 ++ tempest/lib/common/preprov_creds.py | 58 +- tempest/lib/common/rest_client.py | 17 +- tempest/lib/common/utils/data_utils.py | 33 -- tempest/lib/common/validation_resources.py | 457 +++++++++++++++ tempest/lib/exceptions.py | 4 + tempest/lib/services/clients.py | 44 +- tempest/lib/services/object_storage/__init__.py | 25 + .../lib/services/object_storage/account_client.py | 78 +++ .../services/object_storage/container_client.py | 124 ++++ .../lib/services/object_storage/object_client.py | 165 ++++++ .../services/volume/v1/encryption_types_client.py | 0 .../services/volume/v2/encryption_types_client.py | 0 tempest/lib/services/volume/v2/volumes_client.py | 32 -- .../services/volume/v3/group_snapshots_client.py | 16 +- tempest/lib/services/volume/v3/groups_client.py | 11 + tempest/scenario/manager.py | 139 +---- .../scenario/test_network_advanced_server_ops.py | 2 +- tempest/scenario/test_network_basic_ops.py | 39 +- tempest/scenario/test_object_storage_basic_ops.py | 14 +- tempest/services/object_storage/__init__.py | 24 - tempest/services/object_storage/account_client.py | 99 ---- .../services/object_storage/container_client.py | 150 ----- tempest/services/object_storage/object_client.py | 231 -------- tempest/test.py | 538 +++++++++++++----- .../services/object_storage/test_object_client.py | 108 ++++ .../volume/v3/test_group_snapshots_client.py | 37 +- .../lib/services/volume/v3/test_groups_client.py | 9 + .../services/object_storage/test_object_client.py | 6 +- test-requirements.txt | 2 +- tools/generate-tempest-plugins-list.sh | 12 +- tools/tempest-plugin-sanity.sh | 2 +- tox.ini | 6 +- 169 files changed, 4282 insertions(+), 2318 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 36b9efa..8a2fa99 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5,2 +5,2 @@ pbr!=2.1.0,>=2.0.0 # Apache-2.0 -cliff>=2.8.0 # Apache-2.0 -jsonschema!=2.5.0,<3.0.0,>=2.0.0 # MIT +cliff!=2.9.0,>=2.8.0 # Apache-2.0 +jsonschema<3.0.0,>=2.6.0 # MIT @@ -9 +9 @@ paramiko>=2.0.0 # LGPLv2.1+ -netaddr!=0.7.16,>=0.7.13 # BSD +netaddr>=0.7.18 # BSD @@ -11,2 +11,2 @@ testrepository>=0.0.18 # Apache-2.0/BSD -oslo.concurrency>=3.8.0 # Apache-2.0 -oslo.config!=4.3.0,!=4.4.0,>=4.0.0 # Apache-2.0 +oslo.concurrency>=3.20.0 # Apache-2.0 +oslo.config>=4.6.0 # Apache-2.0 @@ -14,2 +14,2 @@ oslo.log>=3.30.0 # Apache-2.0 -oslo.serialization!=2.19.1,>=1.10.0 # Apache-2.0 -oslo.utils>=3.20.0 # Apache-2.0 +oslo.serialization!=2.19.1,>=2.18.0 # Apache-2.0 +oslo.utils>=3.28.0 # Apache-2.0 @@ -22 +22 @@ PrettyTable<0.8,>=0.7.1 # BSD -os-testr>=0.8.0 # Apache-2.0 +os-testr>=1.0.0 # Apache-2.0 @@ -25 +25 @@ debtcollector>=1.2.0 # Apache-2.0 -unittest2 # BSD +unittest2>=1.1.0 # BSD diff --git a/test-requirements.txt b/test-requirements.txt index 29f0865..37644d0 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -7 +7 @@ sphinx>=1.6.2 # BSD -openstackdocstheme>=1.16.0 # Apache-2.0 +openstackdocstheme>=1.17.0 # Apache-2.0