We are psyched to announce the release of: tempest 26.0.0: OpenStack Integration Testing This release is part of the wallaby release series. The source is available from: https://opendev.org/openstack/tempest Download the package from: https://pypi.org/project/tempest Please report issues through: https://bugs.launchpad.net/tempest/+bugs For more details, please see below. 26.0.0 ^^^^^^ Prelude ******* This is an intermediate release during the Wallaby development cycle to mark the end of support for EM Stein release in Tempest. After this release, Tempest will support below OpenStack Releases: * Victoria * Ussuri * Train Current development of Tempest is for OpenStack Wallaby development cycle. New Features ************ * Add a new config option can_migrate_between_any_hosts in the compute-feature-enabled section, which can be set to False for environment with non homogeneous compute nodes, so that it can select a destination host for migrating automatically, otherwise the testcase may fail unexpectedly. e.g., if source host is with CPU "E5-2699 v4" and the selected target host is with CPU "E5-2670 v3", the live-migration will fail because of the downgrade issue. * Add a new config option "[compute-feature-enabled] shelve_migrate" which enable test for environment that support cold migration of qcow2 unshelved instance. * A new config option in the validation section, image_alt_ssh_user, to specify the user name used to authenticate to an alternative instance (instance using image_ref_alt) in tests. By default this is set to root. * A new config option in the validation section, image_alt_ssh_password, to specify the password used to authenticate to an alternative instance (instance using image_ref_alt) in tests. By default this is set to password. * Added missing client methods for keystone's OS-ENDPOINT-POLICY and OS-EP-FILTER APIs. * Added missing clients and tests for keystone's v3 EC2 API which already existed for keystone v2. * Add show type API to v3 types_client library. * default_volume_type * Added associate_floating_ip() and dissociate_floating_ip() methods to the scenario manager. * Add glance image import APIs function to v2 images_client library. * stage_image_file * info_import * info_stores * image_import * [blueprint blueprint adopt-oslo-versioned-objects-for-db (https://blueprints.launchpad.net/neutron/+spec/adopt-oslo- versioned-objects-for-db)] Any reference to "tenant_id" in Network objects is replaced with "project_id". * New config option to "network-feature-enabled": "available_features". This is a list which can contain features that are not discoverable through Neutron API, or it can be the special entry "all". * Add "placement" API methods for testing Routed Provider Networks feature. The following API calls are available for tempest from now in the new resource_providers_client: * GET /resource_providers * GET /resource_providers/{uuid} * GET /resource_providers/{uuid}/inventories * GET /resource_providers/{uuid}/aggregates Upgrade Notes ************* * The following deprecated image scenario options are removed after a ~4 year deprecation period. * "ami_img_file" * "ari_img_file" * "aki_img_file" Starting Tempest 25.0.0 release, CONF.scenario.img_file need a full path for the image. CONF.scenario.img_dir was deprecated and will be removed in the next release. Till Tempest 25.0.0, old behavior is maintained and keep working but starting Tempest 26.0.0, you need to specify the full path in CONF.scenario.img_file config option. * The test_reboot_server_soft has been skipped for more than 6 years. Take into account that the minimum scenario test uses soft reboot and the nova functional test also covers reboot. * "Member" role has been deprecated and replaced by "member". Therefore the default value of config option "[object- storage].operator_role" is changed to "member". (Fixes bug #1330132) * Remove the deprecated CLI "tempest-account-generator" in favor of "tempest account-generator" command. You can use "tempest account- generator" CLI to generate the accounts yaml file. * A number of Compute APIs that only worked with the XenAPI virt driver have been removed in the Compute service. As a result, their corresponding tests are now disabled by default. They can be re- enabled using the new "[compute_feature_enabled] xenapi_apis" config option. Bug Fixes ********* * Fixed bug #1890060. tempest subunit_describe_calls --verbose not working with Cliff CLI. The subunit_describe_calls --verbose argument was a boolean and worked in the non Cliff CLI which is now deprecated, but does not work with cliff since --verbase is a standard cliff argument which is an int. Since the tool is in lib directory we cannot change the interface, so we add a new argument -a --all-stdout that will allow cliff CLI to support the feature in subunnit_describe_calls to print request and response headers and bodies to stdout. * is_resource_deleted method of v3 volumes_client might have returned a KeyError exception due to an incorrect accessing of a volume id in the case the volume was in error_deleting state. incorrect code - volume['id'] correct code - volume['volume']['id'] More details about the issue can be found at https://bugs.launchpad.net/tempest/+bug/1887980 * Concurrency parameter for account-generator command was accepting negative values and zero. The concurrency parameter now accepts only positive numbers. When a negative value or zero is passed to the program then the program ends and help is displayed. Other Notes *********** * New configuration options "CONF.glance.image_feature_enabled.image_import" has been introduced to enable the image import tests. If your glance deployement support image import functionality then you can enable the image import tests via this flag. Default value of this new config option is false. Changes in tempest 25.0.1..26.0.0 --------------------------------- f9d31f2c4 Use victoria stable constraint in tox to release 26.0.0 2c3da54a8 Add releasenote to tag the end of support for Stein b0b480c38 Remove unused LOG var in test_stamp_pattern.py 2e515fe59 Remove race due to 1907084 064a340dc New minimum guaranteed bandwidth tests edda62ab3 Fix indentation for the example code in documentation. a078ddd7c Need to add kwargs for the api 9ad9ca277 Allow kwargs in create_server_snapshot 2c230eb21 Allow kwargs in get_timestamp 27fd5cb38 Allow additional port_vnic_type 8581ea2cd compute: Skip AttachVolumeShelveTestJSON when cross_az_attach unavailable 7d9b50a81 Cleanup server before image at test level 0123f5212 Need to have stable implementation of apis ed66447aa Added apis to scenario manager 0216b3768 Allow kwargs in create_volume_from_image adcddceab compute: Cleanup server before image in AttachSCSIVolumeTestJSON b50eabe29 Allow kwargs in image_create 61f5733a6 Decentralize the zuul jobs into zuul.d 940a9e215 Fix TypeError when there exists image with size None cdcdf2ba5 Skip test_create_object_with_transfer_encoding 3d7010914 Change help string to use openstack instead of neutron 3ead99814 [Trivial]Remove unused variables 414ffbacb Fix "TypeError: format requires a mapping" in waiters 0a9b8235b Add related test to Bug #1732428 776185971 Add tag in compute interfaces schema for microversion 2.70 bf5ed0b67 [Trivial]Remove unused variables and methods d926b8d08 [Trivial]Do not use cls in instance method bc360aeb0 Skip test_server_rescue with volumes when there is no Cinder e3405ba80 Check VM's console log before trying to SSH to it. 25f7ba2e1 Allow kwargs in create_timestamp 76819faf4 Allow kwargs in nova_volume_attach f759bc1cf Allow kwargs in create_floating_ip 7d91c8734 Remove six.PY3/six.PY2 fec0dc182 Add glance multistore job (nv) in tempest gate 9cee53835 Remove 'gabbi_tempest_path' from devstack-tempest job 3fdbab4e3 Clarify tempest-full job description b0642f90e Allow kwargs in get_server_ip 9523d51cc Allow kwargs in _get_server_port_id_and_ip4 c54a91100 Set image_alt_ssh_user during stack d3ad5e94d Allow kwargs in create_volume bee673e74 Allow kwargs in boot_instance_from_resource 6aa0fa356 Replace assertItemsEqual with assertEqual 05c35eb79 Improve tempest-cleanup ansible role 5e761bfad Allow kwargs in create_floating_ip f9bb8b8b2 New list config option to list network features without API extension e1d873a5f Stabilize _create_loginable_secgroup_rule 721aeca8e Unskip Device detach compute tests on Focal 91ded0401 Allow kwargs in _log_console_output 4346a82ff Add glance image import copy-image tests fb1e4db2b Fix incorrect compute api microversion history links 35c2c1203 Fix url link for get_remote_console interface 0d6220824 Test attach scsi disk with config drive enabled cfd6159f9 Make neutron-tempest-dvr job non-voting 8e673a408 Allow kwargs in create_volume_type 20c87c795 Replace display_name by name in scenario manager 30b4d5335 Allow kwargs in create_keypair 4de3d536b Skip adding host to aggregate with az if host already in other zone 5f65bf580 Fix error that instance could not be found 104e0b731 Delete test_reboot_server_soft f0547d606 Add Tempest gate job for stable/victoria 9667217d5 zuul: Move tempest-integrated-compute to Focal efb5d6aba Retry on ServerFault in call_and_ignore_notfound_exc() ec60bfff2 add unittest for endpoints operation interface in identity part ce23482fd Skip additional test if 'xenapi_apis' flag is False e4f282051 Check port allocation for VM with resource request 23e0747e5 Add placement API methods for testing routed provider nets cdbfb5eb3 Fix victoria releasenote to align yaml format 491d63e12 Add glance multistore tests 5eaf6bdc9 Add keystone v3 EC2 credential tests and client 97756b5b2 Remove install unnecessary packages 4599b91fe Add missing keystone clients 1ca760cd2 Migrate tempest jobs to focal 7313bc461 Revert "Use stable constraint in tox to release new tag for Victoria" 13c743407 Updating the paramiko version to 2.7.0 or greater c00346cb2 Add missing replication_status in group schema c8b966ff6 Fix import line location 4e7b2f459 Fix flavor NotFound in test_resize_server_revert_deleted_flavor 9b442510e Adding description for testcases - swift part1 639c4700a Set max_microversion 2.56 for ServerPersonalityTestJSON cd9c2ea3a Add missing attachments schema in servers.py 0672955e1 Fix negative tests of update_volume for volume microversion 3.59 9542ad2fb Modify the fake data according to api reference 4bd9edbd3 volume: Introduce extend attached encrypted volume tests 5e8f04453 Add show types API to v3 types client library 46172435e Modify the fake data for test_networksks_client for v3 d2cbd3ad4 Add Access Rules tests e42bd9354 Added stestr for running test using pdb flag Diffstat (except docs and test files) ------------------------------------- .zuul.yaml | 748 --------------------- playbooks/devstack-tempest-ipv6.yaml | 5 - playbooks/devstack-tempest.yaml | 16 +- ...e-test_reboot_server_soft-48fa786f38cd94dc.yaml | 6 + ...te-feature-shelve-migrate-fdbd3633abe65c4e.yaml | 6 + ...lt-ssh-user-config-option-1b775af2f468aa5b.yaml | 10 + .../add-keystone-ep-clients-eeefd0904fbbe151.yaml | 5 + ...es-api-to-v3-types-client-44b2676f217d78dc.yaml | 6 + ...-disassociate-floating_ip-0b6cfebeef1304b0.yaml | 5 + .../end-of-support-for-stein-f795b968d83497a9.yaml | 12 + ...nabled_available_features-35f9ac5f253e2ca3.yaml | 6 + ...-placement-client-methods-e35c473e29494928.yaml | 11 + .../tempest-victoria-release-27000c02edc5a112.yaml | 1 + requirements.txt | 2 +- roles/tempest-cleanup/README.rst | 28 + roles/tempest-cleanup/defaults/main.yaml | 1 + roles/tempest-cleanup/tasks/dry_run.yaml | 7 + roles/tempest-cleanup/tasks/dry_run_checker.py | 71 ++ roles/tempest-cleanup/tasks/main.yaml | 37 +- tempest/api/compute/admin/test_migrations.py | 23 +- tempest/api/compute/admin/test_volume.py | 116 ++++ tempest/api/compute/base.py | 4 + .../security_groups/test_security_group_rules.py | 12 +- .../api/compute/servers/test_attach_interfaces.py | 30 + tempest/api/compute/servers/test_device_tagging.py | 4 +- tempest/api/compute/servers/test_novnc.py | 11 +- tempest/api/compute/servers/test_server_actions.py | 13 +- .../servers/test_server_metadata_negative.py | 8 +- .../api/compute/servers/test_server_personality.py | 1 + tempest/api/compute/servers/test_server_rescue.py | 9 + tempest/api/compute/volumes/test_attach_volume.py | 4 + .../admin/v3/test_application_credentials.py | 2 +- tempest/api/identity/base.py | 1 + tempest/api/identity/v3/test_access_rules.py | 84 +++ .../identity/v3/test_application_credentials.py | 23 +- tempest/api/identity/v3/test_ec2_credentials.py | 113 ++++ tempest/api/image/base.py | 10 + tempest/api/image/v2/admin/test_images.py | 68 ++ tempest/api/image/v2/test_images.py | 93 ++- tempest/api/network/test_floating_ips.py | 2 +- .../api/object_storage/test_account_services.py | 91 ++- tempest/api/object_storage/test_container_sync.py | 5 +- .../test_container_sync_middleware.py | 2 + tempest/api/object_storage/test_crossdomain.py | 5 +- tempest/api/object_storage/test_healthcheck.py | 3 - tempest/api/object_storage/test_object_expiry.py | 4 + tempest/api/object_storage/test_object_services.py | 121 ++-- tempest/api/object_storage/test_object_slo.py | 14 +- .../volume/admin/test_encrypted_volumes_extend.py | 35 + tempest/api/volume/base.py | 21 + tempest/api/volume/test_volumes_extend.py | 22 +- tempest/api/volume/test_volumes_negative.py | 4 +- tempest/clients.py | 9 +- tempest/cmd/account_generator.py | 2 +- tempest/cmd/run.py | 7 +- tempest/common/compute.py | 20 +- tempest/common/utils/__init__.py | 15 + tempest/common/waiters.py | 104 ++- tempest/config.py | 54 +- .../response/compute/v2_70/interfaces.py | 37 + .../api_schema/response/compute/v2_71/servers.py | 3 + .../api_schema/response/compute/v2_73/servers.py | 3 + tempest/lib/api_schema/response/volume/groups.py | 9 +- tempest/lib/auth.py | 4 +- tempest/lib/cli/base.py | 8 +- tempest/lib/cmd/check_uuid.py | 139 +++- tempest/lib/common/rest_client.py | 2 +- tempest/lib/common/thread.py | 13 +- tempest/lib/common/utils/data_utils.py | 2 +- tempest/lib/common/utils/test_utils.py | 17 +- tempest/lib/services/compute/interfaces_client.py | 9 + tempest/lib/services/compute/servers_client.py | 2 +- tempest/lib/services/identity/v3/__init__.py | 9 +- .../services/identity/v3/access_rules_client.py | 68 ++ .../services/identity/v3/endpoint_filter_client.py | 54 ++ tempest/lib/services/identity/v3/groups_client.py | 2 +- .../services/identity/v3/inherited_roles_client.py | 8 +- .../lib/services/identity/v3/oauth_token_client.py | 2 +- .../lib/services/identity/v3/policies_client.py | 24 + tempest/lib/services/identity/v3/roles_client.py | 24 +- tempest/lib/services/identity/v3/users_client.py | 27 + tempest/lib/services/placement/__init__.py | 4 +- .../placement/resource_providers_client.py | 82 +++ tempest/lib/services/volume/v1/volumes_client.py | 2 +- tempest/lib/services/volume/v3/types_client.py | 13 + tempest/scenario/manager.py | 242 ++++--- tempest/scenario/test_aggregates_basic_ops.py | 19 +- .../scenario/test_minbw_allocation_placement.py | 158 ++++- .../scenario/test_network_advanced_server_ops.py | 16 +- tempest/scenario/test_shelve_instance.py | 44 +- tempest/scenario/test_stamp_pattern.py | 2 - .../identity/v3/test_access_rules_client.py | 97 +++ .../identity/v3/test_endpoint_filter_client.py | 116 ++++ .../services/identity/v3/test_endpoints_client.py | 68 +- .../services/identity/v3/test_policies_client.py | 58 ++ .../lib/services/identity/v3/test_users_client.py | 83 +++ .../services/network/test_floating_ips_client.py | 22 +- .../lib/services/network/test_networks_client.py | 18 +- .../placement/test_resource_providers_client.py | 119 ++++ .../lib/services/volume/v3/test_types_client.py | 13 + tox.ini | 4 +- zuul.d/base.yaml | 86 +++ zuul.d/integrated-gate.yaml | 441 ++++++++++++ zuul.d/project.yaml | 146 ++++ zuul.d/stable-jobs.yaml | 23 + zuul.d/tempest-specific.yaml | 113 ++++ 114 files changed, 3765 insertions(+), 1274 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index d8738f08c..abc0bceda 100644 --- a/requirements.txt +++ b/requirements.txt @@ -8 +8 @@ testtools>=2.2.0 # MIT -paramiko>=2.0.0 # LGPLv2.1+ +paramiko>=2.7.0 # LGPLv2.1+