We enthusiastically announce the release of: tempest 22.0.0: OpenStack Integration Testing This release is part of the train release series. The source is available from: https://opendev.org/openstack/tempest Download the package from: https://pypi.org/project/tempest Please report issues through: https://bugs.launchpad.net/tempest/+bugs For more details, please see below. 22.0.0 ^^^^^^ Prelude ******* This release is to tag the Tempest for OpenStack Train release. This release marks the start of Train release support in Tempest. After this release, Tempest will support below OpenStack Releases: * Train * Stein * Rocky * Queens Current development of Tempest is for OpenStack Ussuri development cycle. Every Tempest commit is also tested against master during the Ussuri cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Ussuri (or future release) cloud. To be on safe side, use this tag to test the OpenStack Train release. New Features ************ * Add show api version details function to v3 versions_client library for cinder. * show_version * A new kwarg, "log_req_body", was added to the "tempest.lib.common.rest_client.RestClient" method "raw_request()". This kwarg takes in a string which will be used in place of the request body, which is logged by default. The intent of this option is to be used for security reasons to avoid logging sensitive information that are part of request bodies. * Add "segments" client to Tempest to make possible the testing of the Routed Provider Networks feature. The following API calls are available for tempest from now: * POST /segments * PUT /segments/{segment_id} * GET /segment/{segment_id} * DELETE /segments/{segment_id} * GET /segments Deprecation Notes ***************** * The config option "CONF.compute.vnc_server_header" is deprecated because it has become obsolete with the usage of different response header fields to accomplish the same goal in accordance with RFC7231 Section 6.2.2. Bug Fixes ********* * Adequately validates WebSocket upgrade in test_novnc and removes unneeded configuration complexity. Closes bug #1838777 and #1840788. Changes in tempest 21.0.0..22.0.0 --------------------------------- 13c190d62 Add release note for log_req_body kwarg 47a419932 Fix the logging of request body credentials 7155fdfdf Call tearDownClass when handling skip exception on setUpClass 8094512a1 Add releasenote to tag the Tempest for Train release 266024c8c Modify the comment for backups_client in v1 cb3cf038e Move logging from _request() to raw_request() 6fca957b0 Remove barbican-tempest-plugin from blacklist 152f1054b Address bug 1839794 on the Volume show version client f045a5297 Add assertNotEmpty to avoid IndexError e65682971 Added instructions about general IDE's in gitignore 730d5df0c add unit test for list_user_projects method 9e79f2cc2 Modify the list project date 3044a29a1 Add irrelevant files to .zuul.yaml e740d39d0 Bump the openstackdocstheme extension to 1.20 68ddf4189 Add workaround to handle the testtool skip exception issue 7e4ef4c82 Add PDF building 48a76dbcc Add response schema validation for volume transfer fd227dab7 Add a CONTRIBUTING file 391d2df83 Fix volume host schema: host name may not contain '@' f33ae3a5b Remove required projects for sanity job fa30d2fb5 Fix use of get_server_ip with fixed networks 5952f8534 Allow overridding OS_TEST_TIMEOUT via the role variables 8641a7073 "test_project_get_equals_list" allows extra fields 8913b1db6 Add group-based-policy to blacklist 941a9470b Add the api reference for roles_client in v3 692dcc19c Validation of openstack keystone version status 6cd8b61c6 Add response schema validation for volume quota_set 2b1cb507d Modify the list group date 6be496b5a Use create_image_from_server to create image for server 6a7472aa8 mke2fs: do not use full path fd01d15d1 Fix test_novnc to adequately validate websocket upgrade 8607b35da Remove unused project_ids in test_list_projects f03b3c125 Add api reference for extensions_client 24675aa30 Add rand_name to create port incase of leaks fbe8f6e5e Add response schema validation for snapshot manage aa605d503 Add api-ref link for some interfaces b9f75e90a Fix the skipException message to follow a pattern similar to other messages 4730b9dd4 Remove references to locking test accounts config doc abc48b4de Change ListProjectsStaticTestJSON to not fail with many domains 61f03b7fe Fix Tempest Plugin Registry page for plugins url 369e80623 Add assert statements for Volume Pagination test to get passed cdd2b24bf compute: fix missing img_signature_uuid property d7ac61432 Add API reference for metering_label_rules_client 9ae6ef82d Add api reference in quotas_client 41178cb8a Modify the group types date 411f4bbc5 Retry http requests to generate plugins list 219e322e7 tempest cleanup fix return code 377279095 Remove python shebangs from python modules db10fb070 add show detail for volume Api V3 4776b13ba Fix Upstream test cases to be case insensitive 11f5fbedc Skip some tests in "network.admin.test_ports" if nova is not enabled c3cd87a6e Do not delete server when task state is image_snapshot 18cf132af Modify the API reference 043533235 Modify the fake date to the correct value fdd964494 Make bashate E005,E042 error and ignore E006 be05dfab7 Remove x/ranger-tempest-plugin from BLACKLIST 924a03588 Revert "Use memcached based cache in nova in all devstack-tempest jobs" a45ae7404 Fix remaining api-ref references de1153b36 Introduce bashate a0b8ca43e Setup framework for IPv6-only deployment verification 4f6538e01 Fix indentation in releasenotes 9888e977d Add release notes page for version 21.0.0 0f348f79c Skip tests in UsersV3TestJSON with an immutable user source eb0a2cc5f Use memcached based cache in nova in all devstack-tempest jobs 36e543b9a Add segments client bf30ae787 Update api-ref location 802a2b101 Fix unittest of encryption_types_client 51c51fc85 Remove duplicated irrelevant-files for sanity check c00340623 Use skip_checks in ServersTestMultiNic 4f87f6bd2 Add ranger-tempest-plugin to blacklist 7409c72cb Add Python 3 Train unit tests 6fcbb349e Fix invalid original_name key in flavor check 209663193 Add 'single core approve' policy for plugins sanity jobs failure bae11b6dc Skip tests in TrustsV3TestJSON with immutable user source dce8059c6 Skip test_server_connectivity_cold_migration_revert until fixed bd5be763c tempest cleanup should use keystone v3 client 1114de6b0 Remove the Pike stable branch jobs from master gate 4006211b5 Make plugin-sanity-check job as voting c68287e17 Enable volume backup tests ba18426fd Unskip test_stamp_pattern d0b5d2893 Add response schema validation for volume messages 2f1806ad8 Add response schema validation for volume hosts 38a78be61 Add response schema validation for volume services 9acabc40d Add response schema validation for volume extensions d4f40a418 Add response schema validation for volume capabilities Diffstat (except docs and test files) ------------------------------------- .gitignore | 6 + .zuul.yaml | 173 ++++--------- CONTRIBUTING.rst | 17 ++ README.rst | 271 +------------------- REVIEWING.rst | 3 + playbooks/devstack-tempest-ipv6.yaml | 24 ++ ...dd_subunit_describe_calls-5498a37e6cd66c4b.yaml | 4 +- ...entity-clients-as-library-e663c6132fcac6c2.yaml | 10 +- ...-image-clients-as-library-86d17caa26ce3961.yaml | 14 +- ...-add-new-identity-clients-3c3afd674a395bde.yaml | 10 +- ...volume-clients-as-library-9a3444dd63c134b3.yaml | 20 +- ...entity-clients-as-library-5f7ndha733nwdsn9.yaml | 16 +- .../14.0.0-add-image-clients-af94564fb34ddca6.yaml | 2 +- ...domain-param-in-cliclient-a270fcf35c8f09e6.yaml | 8 +- ...api-to-v3-versions-client-4b408427379cabfe.yaml | 7 + ...precate-vnc-server-header-529f07d592aefb62.yaml | 12 + .../fix-credential-logging-98089c897d801355.yaml | 9 + ...ntermediate-train-release-0146c624cff9a57f.yaml | 8 +- ...lient-as-stable-interface-d1b07c7e8f17bef6.yaml | 6 +- ...orage-as-stable-interface-42014c7b43ecb254.yaml | 4 +- .../notes/segments-client-866f02948f40d4ff.yaml | 12 + .../tempest-rocky-release-0fc3312053923380.yaml | 10 +- .../tempest-stein-release-18bad34136a2e6ef.yaml | 10 +- .../tempest-train-release-a2ed0743a5eadeb6.yaml | 17 ++ releasenotes/source/conf.py | 15 -- releasenotes/source/index.rst | 1 + releasenotes/source/v21.0.0.rst | 6 + .../ipv6-only-deployments-verification/README.rst | 16 ++ .../defaults/main.yaml | 1 + .../tasks/main.yaml | 4 + roles/run-tempest/README.rst | 5 + roles/run-tempest/defaults/main.yaml | 1 + roles/run-tempest/tasks/main.yaml | 1 + setup.cfg | 2 + tempest/api/compute/admin/test_networks.py | 2 +- tempest/api/compute/images/test_images.py | 4 + .../api/compute/images/test_images_oneserver.py | 11 +- .../api/compute/servers/test_attach_interfaces.py | 15 +- .../servers/test_create_server_multi_nic.py | 11 +- tempest/api/compute/servers/test_device_tagging.py | 6 + tempest/api/compute/servers/test_novnc.py | 20 +- tempest/api/compute/servers/test_server_actions.py | 3 + tempest/api/compute/servers/test_servers.py | 3 +- .../api/compute/volumes/test_volume_snapshots.py | 2 +- tempest/api/compute/volumes/test_volumes_get.py | 2 +- tempest/api/compute/volumes/test_volumes_list.py | 2 +- .../api/compute/volumes/test_volumes_negative.py | 2 +- .../api/identity/admin/v3/test_list_projects.py | 50 ++-- tempest/api/identity/admin/v3/test_list_users.py | 8 + tempest/api/identity/admin/v3/test_projects.py | 10 +- tempest/api/identity/admin/v3/test_trusts.py | 4 + tempest/api/identity/v3/test_api_discovery.py | 12 + tempest/api/identity/v3/test_tokens.py | 31 ++- .../admin/test_external_networks_negative.py | 2 + tempest/api/network/admin/test_ports.py | 23 +- tempest/api/network/admin/test_routers_negative.py | 2 + tempest/api/network/base.py | 2 + tempest/api/network/test_allowed_address_pair.py | 10 +- tempest/api/network/test_extra_dhcp_options.py | 1 + tempest/api/network/test_floating_ips.py | 10 +- tempest/api/network/test_networks_negative.py | 3 +- tempest/api/network/test_ports.py | 53 ++-- tempest/api/network/test_routers.py | 4 +- .../api/volume/admin/test_backends_capabilities.py | 19 +- tempest/api/volume/admin/test_user_messages.py | 27 +- tempest/api/volume/admin/test_volume_hosts.py | 13 - tempest/api/volume/admin/test_volume_pools.py | 1 + tempest/api/volume/admin/test_volume_quotas.py | 23 +- tempest/api/volume/admin/test_volume_retype.py | 2 +- tempest/api/volume/test_versions.py | 12 + tempest/api/volume/test_volume_transfers.py | 4 - tempest/api/volume/test_volumes_backup.py | 1 - tempest/api/volume/test_volumes_list.py | 18 +- tempest/clients.py | 1 + tempest/cmd/cleanup.py | 20 +- tempest/cmd/cleanup_service.py | 6 +- tempest/cmd/list_plugins.py | 2 - tempest/common/utils/linux/remote_client.py | 2 +- tempest/config.py | 9 +- .../lib/api_schema/response/volume/capabilities.py | 55 ++++ .../lib/api_schema/response/volume/extensions.py | 43 ++++ tempest/lib/api_schema/response/volume/hosts.py | 81 ++++++ .../api_schema/response/volume/manage_snapshot.py | 49 ++++ tempest/lib/api_schema/response/volume/messages.py | 64 +++++ tempest/lib/api_schema/response/volume/quotas.py | 92 +++++++ tempest/lib/api_schema/response/volume/services.py | 92 +++++++ .../lib/api_schema/response/volume/transfers.py | 129 ++++++++++ tempest/lib/api_schema/response/volume/versions.py | 46 ++++ tempest/lib/common/rest_client.py | 30 ++- tempest/lib/services/compute/agents_client.py | 8 +- tempest/lib/services/compute/aggregates_client.py | 10 +- .../lib/services/compute/baremetal_nodes_client.py | 4 +- tempest/lib/services/compute/fixed_ips_client.py | 2 +- tempest/lib/services/compute/flavors_client.py | 28 +- .../lib/services/compute/floating_ips_client.py | 8 +- tempest/lib/services/compute/hosts_client.py | 4 +- tempest/lib/services/compute/images_client.py | 12 +- tempest/lib/services/compute/interfaces_client.py | 2 +- tempest/lib/services/compute/keypairs_client.py | 8 +- tempest/lib/services/compute/migrations_client.py | 2 +- .../lib/services/compute/quota_classes_client.py | 2 +- tempest/lib/services/compute/quotas_client.py | 10 +- .../compute/security_group_default_rules_client.py | 2 +- .../compute/security_group_rules_client.py | 2 +- .../lib/services/compute/security_groups_client.py | 10 +- .../lib/services/compute/server_groups_client.py | 2 +- tempest/lib/services/compute/servers_client.py | 125 ++++----- tempest/lib/services/compute/services_client.py | 12 +- tempest/lib/services/compute/snapshots_client.py | 4 +- .../lib/services/compute/tenant_usages_client.py | 4 +- tempest/lib/services/compute/volumes_client.py | 10 +- .../lib/services/identity/v2/endpoints_client.py | 2 +- .../lib/services/identity/v2/identity_client.py | 2 +- tempest/lib/services/identity/v2/roles_client.py | 14 +- .../lib/services/identity/v2/services_client.py | 4 +- tempest/lib/services/identity/v2/tenants_client.py | 12 +- tempest/lib/services/identity/v2/token_client.py | 5 +- tempest/lib/services/identity/v2/users_client.py | 12 +- .../identity/v3/application_credentials_client.py | 10 +- tempest/lib/services/identity/v3/catalog_client.py | 2 +- .../lib/services/identity/v3/credentials_client.py | 12 +- .../identity/v3/domain_configuration_client.py | 26 +- tempest/lib/services/identity/v3/domains_client.py | 10 +- .../services/identity/v3/endpoint_filter_client.py | 2 +- .../services/identity/v3/endpoint_groups_client.py | 4 +- .../lib/services/identity/v3/endpoints_client.py | 8 +- tempest/lib/services/identity/v3/groups_client.py | 10 +- .../services/identity/v3/oauth_consumers_client.py | 10 +- .../lib/services/identity/v3/oauth_token_client.py | 16 +- .../lib/services/identity/v3/policies_client.py | 24 +- .../services/identity/v3/project_tags_client.py | 2 +- .../lib/services/identity/v3/projects_client.py | 4 +- tempest/lib/services/identity/v3/regions_client.py | 6 +- .../identity/v3/role_assignments_client.py | 2 +- tempest/lib/services/identity/v3/roles_client.py | 12 +- .../lib/services/identity/v3/services_client.py | 8 +- tempest/lib/services/identity/v3/token_client.py | 6 +- tempest/lib/services/identity/v3/trusts_client.py | 4 +- tempest/lib/services/identity/v3/users_client.py | 12 +- .../lib/services/image/v1/image_members_client.py | 6 +- tempest/lib/services/image/v1/images_client.py | 6 +- .../lib/services/image/v2/image_members_client.py | 10 +- tempest/lib/services/image/v2/images_client.py | 22 +- .../services/image/v2/namespace_objects_client.py | 10 +- .../image/v2/namespace_properties_client.py | 10 +- .../lib/services/image/v2/namespace_tags_client.py | 14 +- tempest/lib/services/image/v2/namespaces_client.py | 10 +- .../lib/services/image/v2/resource_types_client.py | 8 +- tempest/lib/services/network/__init__.py | 5 +- tempest/lib/services/network/agents_client.py | 16 +- tempest/lib/services/network/extensions_client.py | 12 + .../lib/services/network/floating_ips_client.py | 8 +- .../network/metering_label_rules_client.py | 11 +- .../lib/services/network/metering_labels_client.py | 8 +- tempest/lib/services/network/networks_client.py | 10 +- tempest/lib/services/network/ports_client.py | 12 +- tempest/lib/services/network/qos_client.py | 10 +- .../network/qos_minimum_bandwidth_rules_client.py | 10 +- tempest/lib/services/network/quotas_client.py | 14 +- tempest/lib/services/network/routers_client.py | 12 +- .../network/security_group_rules_client.py | 6 +- .../lib/services/network/security_groups_client.py | 10 +- tempest/lib/services/network/segments_client.py | 63 +++++ .../services/network/service_providers_client.py | 2 +- tempest/lib/services/network/subnetpools_client.py | 8 +- tempest/lib/services/network/subnets_client.py | 10 +- tempest/lib/services/network/tags_client.py | 12 +- tempest/lib/services/network/versions_client.py | 4 +- .../lib/services/object_storage/account_client.py | 4 +- .../services/object_storage/container_client.py | 6 +- tempest/lib/services/placement/placement_client.py | 4 +- tempest/lib/services/volume/v1/backups_client.py | 7 +- .../services/volume/v1/encryption_types_client.py | 2 +- tempest/lib/services/volume/v1/hosts_client.py | 2 +- tempest/lib/services/volume/v1/qos_client.py | 6 +- tempest/lib/services/volume/v1/quotas_client.py | 2 +- tempest/lib/services/volume/v1/snapshots_client.py | 16 +- tempest/lib/services/volume/v1/types_client.py | 12 +- tempest/lib/services/volume/v1/volumes_client.py | 20 +- tempest/lib/services/volume/v3/backups_client.py | 12 +- .../lib/services/volume/v3/capabilities_client.py | 5 +- .../services/volume/v3/encryption_types_client.py | 4 +- .../lib/services/volume/v3/extensions_client.py | 3 +- .../services/volume/v3/group_snapshots_client.py | 12 +- .../lib/services/volume/v3/group_types_client.py | 14 +- tempest/lib/services/volume/v3/groups_client.py | 16 +- tempest/lib/services/volume/v3/hosts_client.py | 7 +- tempest/lib/services/volume/v3/messages_client.py | 7 +- tempest/lib/services/volume/v3/qos_client.py | 6 +- .../lib/services/volume/v3/quota_classes_client.py | 4 +- tempest/lib/services/volume/v3/quotas_client.py | 14 +- .../services/volume/v3/scheduler_stats_client.py | 2 +- tempest/lib/services/volume/v3/services_client.py | 29 ++- .../services/volume/v3/snapshot_manage_client.py | 10 +- tempest/lib/services/volume/v3/snapshots_client.py | 22 +- tempest/lib/services/volume/v3/transfers_client.py | 21 +- tempest/lib/services/volume/v3/types_client.py | 20 +- tempest/lib/services/volume/v3/versions_client.py | 23 +- .../lib/services/volume/v3/volume_manage_client.py | 2 +- tempest/lib/services/volume/v3/volumes_client.py | 50 ++-- .../scenario/test_network_advanced_server_ops.py | 4 +- tempest/scenario/test_stamp_pattern.py | 13 +- tempest/test.py | 52 +++- .../lib/services/identity/v2/test_token_client.py | 3 + .../services/identity/v3/test_projects_client.py | 12 +- .../lib/services/identity/v3/test_token_client.py | 3 + .../lib/services/identity/v3/test_users_client.py | 47 ++++ .../lib/services/network/test_segments_client.py | 140 ++++++++++ .../volume/v3/test_encryption_types_client.py | 35 ++- .../volume/v3/test_group_snapshots_client.py | 11 +- .../services/volume/v3/test_group_types_client.py | 30 ++- .../lib/services/volume/v3/test_groups_client.py | 21 +- .../lib/services/volume/v3/test_hosts_client.py | 4 +- .../lib/services/volume/v3/test_quotas_client.py | 19 +- .../lib/services/volume/v3/test_versions_client.py | 51 ++++ tools/format.sh | 3 +- tools/generate-tempest-plugins-list.py | 35 +-- tools/generate-tempest-plugins-list.sh | 4 +- tools/tempest-plugin-sanity.sh | 10 +- tools/verify-ipv6-only-deployments.sh | 92 +++++++ tox.ini | 37 ++- 230 files changed, 2748 insertions(+), 1316 deletions(-)