We contentedly announce the release of: tempest 24.0.0: OpenStack Integration Testing This release is part of the ussuri release series. The source is available from: https://opendev.org/openstack/tempest Download the package from: https://pypi.org/project/tempest Please report issues through: https://bugs.launchpad.net/tempest/+bugs For more details, please see below. 24.0.0 ^^^^^^ Prelude ******* Remove the support of python3.5. Tempest, its plugins dependencies in ussuri cycle are python-requires>=py3.6 which makes distro not having python 3.6 to do hack to install py3.6 etc. It time to drop the py3.5 from Tempest. Last supported version of Tempest for py3.5 is 23.0.0. This release is to tag the Tempest for OpenStack Ussuri release. This release marks the start of Ussuri release support in Tempest. After this release, Tempest will support below OpenStack Releases: * Ussuri * Train * Stein Current development of Tempest is for OpenStack Victoria development cycle. Every Tempest commit is also tested against master during the Victoria cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Ussuri (or future release) cloud. To be on safe side, use this tag to test the OpenStack Ussuri release. New Features ************ * A new config option 'subnet_id' is added to section 'network' to specify subnet which should be used for allocation of IPs for VMs created during testing. It should be used when the tested network contains more than one subnet otherwise test of external connectivity will fail. (Fixes bug * Add the option --worker-file in "tempest run" command. This is to give tempest more granularity to manually configure how the different sets of tests can be grouped to run with the different worker. You can configure tests regex to run under workers. You can also mix manual scheduling with standard one by mentioning concurrency. For example, the user can setup tempest to run with different concurrences, to be used with different regexps. * A new attachments client library has been introduced for the volume service. Initially only the show_attachment API is provided. This API requires a minimum volume API microversion of "3.27". Upgrade Notes ************* * Default value of config option "CONF.placement.region" is updated from "RegionOne" to empty string. As per tempest design, if tempest conf is not having any region for a service then identity region should be used. In case of placement the default value is "RegionOne" which is considered as placement region if region missing in tempest conf. In order to have identity region to be used as default we need to change to empty string for placement service. Empty string can be seen being used in other services like volume, image etc. * Python 2.7 support has been dropped. Last release of Tempest to support python 2.7 is Temepst 23.0.0. The minimum version of Python now supported by Tempest is Python 3.6. * Remove the deprecated argument "os-tenant-name" or "OS_TENANT_NAME" in favour of "os-project-name" argument. * Remove the deprecated CLI "verify-tempest-config" in favour of "tempest verify-config" command. You can use "tempest verify-config" CLI to verify the tempest conf file. Deprecation Notes ***************** * The "[compute-feature-enabled]/block_migrate_cinder_iscsi" is deprecated ahead of removal in a future release. Once removed the "[compute-feature-enabled]/block_migration_for_live_migration" configurable will then be used to determine when to run block migration based tests during live migration. * The config options "CONF.compute.spice_console" and "CONF.compute.rdp_console" are deprecated because test cases using them are removed. We can add them back when adding the test cases again. Bug Fixes ********* * Fixed bug #1858417. Adding consistency group exceptions "ConsistencyGroupException" and "ConsistencyGroupSnapshotException" that didn't exist before and caused failure in cinder-tempest- plugin. Changes in tempest 23.0.0..24.0.0 --------------------------------- 2c22e2833 Create default network for compute rescue tests b5b0cf490 Use stable constraint in tox to release new tag for Ussuri 0f8dd6bf1 Add releasenote to tag the Tempest for Ussuri release 95c410afc [ussuri][goal] Update contributor documentation 235c26445 Document the policy for Tempest & its plugins release c16883ff3 Adding description for testcases - identity part1 3fdd35ae6 Adding description for testcases - volume part1 aa7c14749 Add logs for plugins data f77b3e30e Catch json.loads exceptions in verify_device_metadata 1b3b03d46 Adding description for testcases - compute part2 5c191faae Explicitly handle baremetal ports when checking if they are active 5e5b8919a [Trivial]Do not use self in class method 747300b36 [Trivial]Add space between words in log message 2a6dcdbd0 Adding description for testcases - compute part1 14d66b6a2 Add glare and taas to blacklist c0ad55ccd compute: Add stable device rescue tests 803b4a7e8 compute: Add missing 2.73 response schema validation ca0fad075 Fix noVNC failed TCs 30638b53a Keystone - List user groups 'membership_expires_at' attribute 4b1085233 compute: Add negative volume update multiattach test 68f1e5e8c Pass clients to compute base's create_test_server b8435b27a Remove universal wheel building f27a334e2 Update hacking for Python3 8b29792f4 Check no specific group present in group list e743e5d39 Blacklist x/kingbird project from sanity-check c06c371b3 Add --worker-file option in tempest 7cd0186db Replace old requires-python with python-requires 61a3c8efa Fixes race condition in test_add_remove_fixed_ip 97b841130 Fix deletion of network quotas 33e0e069f add unittest for create and list resource types association 7d91b6908 Create default network for compute volume and negative metadata tests 61b9fd231 Fixed test related to tag-ext 2385e0417 Add subnet_id option 0fb306f2e Make tempest-multinode-full py2 job on py2 2c8312439 Cleanup images properly 65e37a2bf add unittest for delete resource types association 0d2b331cc Ensure bootable volume is gone in scenario test before tearDown ec3ae5ea0 services: Introduce a cinder v3 attachments client d2bdd8c0b Remove devstack-plugin-ceph-tempest py2 job 7eb1d2dc9 modify status code according to the latest API documentation 6e3f99e9d Do not force live migration outside of Availability zone faa163442 Added fix to 'test_delete_saving_image' testcase 2bdbc0a9e add unit test for delete_group_user method 8cb1b4748 Set 3h timeout for tempest-ipv6-only job d5bda7125 Deprecate spice_console and rdp_console config items e530f2416 Delete network quotas 9e3cc27e8 Add validation_resources in test_resize_volume_backed_server_confirm 4062a5ceb Re-enable the stable rocky job f644baa18 compute: Use wait_for_volume_attachment_remove when detaching multiattach volumes e30375e17 compute: Use valid extra specs to test 'os-extra_specs' API bcfa06d55 Prevent SSH client proxy loops 551e8d7a5 [ussuri][goal] Drop python 2.7 support and testing 4af65407b Update README for run-tempest role 4b95d4b14 compute: Add volume and placement microversion support to BaseV2ComputeTest 535cfb05f Disable stable/rocky temporarily on Tempest master 56149024a Use stable constraint in run-tempest for stable branch testing 03c428d5f Add missing waiter to test_tagged_attachment bc9fe8eca tox: Use python3 within tempestenv b0a8057fb Drop python 3.5 support from Tempest e5e2c7dd0 Modify the incorrect fake data a89eea855 Change default region for placement to empty string d19d7f8dc Improve response data of test_routers_client 48fe99faf compute: Deprecate [compute-feature-enabled]/block_migrate_cinder_iscsi c1b2a4a71 waiters: Add wait_for_volume_attachment_remove 195767a06 Removal of deprecated argument[os-tenant-name] 1d500e791 Accept custom registered endpoints abf299269 Remove indentation for bullet list 4b45f9e54 Fix test_stamp_pattern to remove dependency of user supplied volume name 1c680fdb7 Define python3 as basepython for Tempest tox env fbea823a3 Add ConsistencyGroup Exceptions in tempest e85f029ab Create default network for more compute tests 68600b18f Add doc for supported OpenStack release & py version f96146311 Add release notes page for version 23.0.0 64da1a140 Remove the Queens stable branch jobs from master gate d0ef93e01 Support force-deletion of servers. a8bfa172c Refactor methods to be used by other tests 4acf175de Remove the deprecated verify-tempest-config CLI 468f8fb18 add unittest for agent operation interface in network part 935e2a747 Remove the security_group_default_rules tests 1a2c8f362 add unittest for delete tag definition interface 4078afea7 Modify the response in test_snapshots_client 3ec41c20d Relax volume compare in test_minimum_basic_scenario 9a49aba5a Modify the application credentials fake data 51e474d5e Add response schema validation for volume types e75e3cb1a Add response schema validation for volume quota classes Diffstat (except docs and test files) ------------------------------------- .zuul.yaml | 90 ++++------- CONTRIBUTING.rst | 22 +-- HACKING.rst | 1 - ...mpest_and_plugins_compatible_version_policy.rst | 54 +++++++ ...sistency-group-exceptions-01cbb792cd710231.yaml | 6 + ...d-subnet-id-config-option-fac3d6f12abfc171.yaml | 8 + .../add-worker-file-option-d949121a61156968.yaml | 10 ++ ...placement-to-empty-string-394f1132c28345bb.yaml | 13 ++ ...lock-migrate-cinder-iscsi-fcda802d774dfeec.yaml | 8 + ...-spice-rdp-console-config-f2af173552axfb72.yaml | 6 + .../notes/drop-py-2-7-730baf411876d5d8.yaml | 6 + .../drop-py-3-5-support-76ca78f1a650fcad.yaml | 8 + ...termediate-ussuri-release-8aebeca312a6718c.yaml | 8 +- ...t-add-show-attachment-api-c3111f7e560a87b3.yaml | 8 + .../notes/os_tenant_name-3ee175763bff455b.yaml | 5 + .../tempest-ussuri-release-72b5770a3b97678f.yaml | 16 ++ .../verify-tempest-command-8e88452c7a08dd77.yaml | 7 + releasenotes/source/index.rst | 1 + releasenotes/source/v23.0.0.rst | 6 + roles/run-tempest/README.rst | 28 ++-- roles/run-tempest/defaults/main.yaml | 3 + roles/run-tempest/tasks/main.yaml | 17 +- setup.cfg | 9 +- setup.py | 8 - .../api/compute/admin/test_aggregates_negative.py | 2 + tempest/api/compute/admin/test_flavors.py | 40 +++-- .../api/compute/admin/test_flavors_extra_specs.py | 53 ++++--- .../admin/test_flavors_extra_specs_negative.py | 48 +++--- tempest/api/compute/admin/test_hosts.py | 5 + tempest/api/compute/admin/test_hosts_negative.py | 20 ++- tempest/api/compute/admin/test_hypervisor.py | 1 + .../api/compute/admin/test_hypervisor_negative.py | 12 +- tempest/api/compute/admin/test_live_migration.py | 5 + tempest/api/compute/admin/test_migrations.py | 14 +- tempest/api/compute/admin/test_networks.py | 2 + .../admin/test_security_group_default_rules.py | 132 ---------------- .../compute/admin/test_servers_on_multinodes.py | 6 +- tempest/api/compute/admin/test_services.py | 3 + .../api/compute/admin/test_services_negative.py | 14 +- tempest/api/compute/admin/test_volume_swap.py | 1 + tempest/api/compute/admin/test_volumes_negative.py | 65 ++++++++ tempest/api/compute/base.py | 66 +++++--- tempest/api/compute/images/test_images.py | 28 +++- .../images/test_images_oneserver_negative.py | 1 + .../api/compute/servers/test_attach_interfaces.py | 74 +++------ tempest/api/compute/servers/test_device_tagging.py | 17 +- .../api/compute/servers/test_multiple_create.py | 3 + .../servers/test_multiple_create_negative.py | 5 + tempest/api/compute/servers/test_server_actions.py | 15 +- .../servers/test_server_metadata_negative.py | 1 + tempest/api/compute/servers/test_server_rescue.py | 74 +++++++++ .../api/compute/servers/test_servers_negative.py | 1 + tempest/api/compute/volumes/test_attach_volume.py | 12 +- tempest/api/identity/admin/v2/test_endpoints.py | 3 + .../identity/admin/v3/test_default_project_id.py | 7 +- tempest/api/identity/admin/v3/test_groups.py | 7 + tempest/api/identity/admin/v3/test_list_users.py | 11 +- tempest/api/identity/admin/v3/test_policies.py | 5 +- tempest/api/identity/admin/v3/test_project_tags.py | 3 + tempest/api/identity/admin/v3/test_services.py | 6 +- tempest/api/identity/admin/v3/test_users.py | 9 +- tempest/api/identity/v3/test_catalog.py | 8 +- tempest/api/network/test_tags.py | 15 +- tempest/api/object_storage/test_crossdomain.py | 7 +- tempest/api/object_storage/test_healthcheck.py | 7 +- tempest/api/volume/admin/test_group_snapshots.py | 3 +- .../api/volume/admin/test_volume_quota_classes.py | 4 +- tempest/api/volume/admin/test_volume_types.py | 3 - tempest/api/volume/admin/test_volumes_actions.py | 1 + tempest/api/volume/base.py | 7 +- tempest/api/volume/test_versions.py | 3 + tempest/api/volume/test_volume_absolute_limits.py | 2 +- tempest/api/volume/test_volume_delete_cascade.py | 16 +- tempest/api/volume/test_volume_metadata.py | 3 + tempest/api/volume/test_volume_transfers.py | 3 + tempest/api/volume/test_volumes_actions.py | 1 + tempest/api/volume/test_volumes_clone.py | 3 + tempest/api/volume/test_volumes_clone_negative.py | 2 + tempest/api/volume/test_volumes_extend.py | 4 + tempest/api/volume/test_volumes_get.py | 6 + tempest/api/volume/test_volumes_snapshots.py | 1 + tempest/api/volume/test_volumes_snapshots_list.py | 2 +- tempest/clients.py | 2 + tempest/cmd/account_generator.py | 14 +- tempest/cmd/cleanup_service.py | 32 +++- tempest/cmd/run.py | 41 +++++ tempest/cmd/verify_tempest_config.py | 9 -- tempest/common/compute.py | 19 ++- tempest/common/utils/linux/remote_client.py | 4 +- tempest/common/waiters.py | 22 +++ tempest/config.py | 32 +++- tempest/hacking/checks.py | 59 +++---- .../api_schema/response/compute/v2_73/__init__.py | 0 .../api_schema/response/compute/v2_73/servers.py | 78 +++++++++ .../api_schema/response/volume/quota_classes.py | 68 ++++++++ .../lib/api_schema/response/volume/volume_types.py | 176 +++++++++++++++++++++ tempest/lib/auth.py | 4 +- tempest/lib/common/api_version_utils.py | 4 + tempest/lib/common/ssh.py | 21 ++- tempest/lib/exceptions.py | 14 ++ tempest/lib/services/compute/servers_client.py | 4 +- .../lib/services/image/v2/namespace_tags_client.py | 6 +- .../services/object_storage/capabilities_client.py | 5 +- tempest/lib/services/volume/v3/__init__.py | 18 +-- .../lib/services/volume/v3/attachments_client.py | 28 ++++ .../lib/services/volume/v3/quota_classes_client.py | 5 +- tempest/lib/services/volume/v3/types_client.py | 32 ++-- tempest/scenario/manager.py | 75 +++++++-- tempest/scenario/test_minimum_basic.py | 6 +- tempest/scenario/test_network_basic_ops.py | 15 +- tempest/scenario/test_stamp_pattern.py | 46 +++--- tempest/scenario/test_volume_boot_pattern.py | 67 +++----- tempest/test_discover/plugins.py | 6 + .../v3/test_application_credentials_client.py | 118 +++++++++----- .../lib/services/identity/v3/test_groups_client.py | 10 ++ .../image/v2/test_namespace_tags_client.py | 10 +- .../services/image/v2/test_namespaces_client.py | 66 ++++++-- .../image/v2/test_resource_types_client.py | 62 ++++++++ .../lib/services/network/test_agents_client.py | 133 ++++++++++++++++ .../lib/services/network/test_routers_client.py | 69 ++++++-- .../object_storage/test_capabilities_client.py | 2 +- .../services/object_storage/test_object_client.py | 4 +- .../services/volume/v3/test_attachments_client.py | 46 ++++++ .../services/volume/v3/test_snapshots_client.py | 118 ++++++++------ test-requirements.txt | 2 +- tools/generate-tempest-plugins-list.py | 3 + tox.ini | 63 ++++++-- 138 files changed, 2395 insertions(+), 787 deletions(-) Requirements updates -------------------- diff --git a/test-requirements.txt b/test-requirements.txt index 196387cb7..a50905fb7 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -4 +4 @@ -hacking>=1.1.0,<1.2.0 # Apache-2.0 +hacking>=3.0,<3.1.0;python_version>='3.5' # Apache-2.0