We are pumped to announce the release of: tempest 26.1.0: OpenStack Integration Testing This release is part of the wallaby release series. The source is available from: https://opendev.org/openstack/tempest Download the package from: https://pypi.org/project/tempest Please report issues through: https://bugs.launchpad.net/tempest/+bugs For more details, please see below. 26.1.0 ^^^^^^ Prelude ******* This is an intermediate release during the Wallaby development cycle to make new functionality available to plugins and other consumers. The integrated horizon dashboard test is now moved from tempest- horizon plugin into Tempest. You do not need to install tempest- horizon to run the horizon test which can be run using Tempest itself. Support for RBAC new system scope is added in Tempest. New Features ************ * The following tempest clients for keystone v3 OS_FEDERATION API were implemented in this release * identity_providers * protocols * mappings * service_providers * Add project alternate admin, member and reader role for dynamic credentials. * Added methods to the identity v3 roles client to support: * PUT /v3/system/users/{user}/roles/{role} * GET /v3/system/users/{user}/roles * GET /v3/system/users/{user}/roles/{role} * DELETE /v3/system/users/{user}/roles/{role} * PUT /v3/system/groups/{group}/roles/{role} * GET /v3/system/groups/{group}/roles * GET /v3/system/groups/{group}/roles/{role} * DELETE /v3/system/groups/{group}/roles/{role} * Added public interface create_loginable_secgroup_rule(). Since this interface is meant to be used by tempest plugins, It doesn't neccessarily require to be private api. * Added public interface create_security_group_rule(). Since this interface is meant to be used by tempest plugins, It doesn't neccessarily require to be private api. * Added public interface log_console_output(). It used to be a private method with name _log_console_output(). Since this interface is meant to be used by tempest plugins, It doesn't neccessarily require to be private api. * Keystone provides the new scoped token called "system" which can be used to query the system scoped API operation. Projects are moving towards the policy with new scope types, Keystone, Nova already provide the new policy for RBAC checks. Tempest has added the support to query the system scoped token from keystone to test the new policy. As next step, we will be moving all the Tempest tests on the project's new policy. * Dynamic credentials now support the scope type for specific roles too along with "admin", "member", "reader" role. Test can specify the scope in the prefix of "cls.credentials" name. If "system" is prefix in "cls.credentials" name then creds will be created with scope as "system". If "domain" is prefix in "cls.credentials" name then creds will be created with scope as "domain" otherwise default "project" scope will be used. For Example: credentials = [['my_role', 'role1'], # this will be old style and project scoped ['project_my_role', 'role1'], # this will be project scoped ['domain_my_role', 'role1'], # this will be domain scoped ['system_my_role', 'role1']] # this will be system scoped And below is how test can access the credential manager of respective credentials type: cls.os_my_role.any_client cls.os_project_my_role.any_client cls.os_domain_my_role.any_client cls.os_system_my_role.any_client For backward compatibility, we set the credentials manager class attribute in old style form too which is prefix with "os_roles_*", example "cls.os_roles_my_role" but we recommend to use the new style attribute as shown above. * Adds new personas that can be used to test service policies for all default scopes (project, domain, and system) and roles (reader, member, and admin). Both dynamic credentials and pre-provisioned credentials are supported. Known Issues ************ * Scope support for specific role is not yet added for pre- provisioned credentials. Upgrade Notes ************* * In this release tempest/manager.py is removed after more than 4 years of deprecation. * The "tempest.lib.common.utils.data_utils.random_bytes()" helper function will no longer allow a "size" of more than 1MiB. Tests generally do not need to generate and use large payloads for feature verification and it is easy to lose track of and duplicate large buffers. The sum total of such errors can become problematic in paralllelized and constrained CI environments. * Deprecated config options to select the Volume API version have been removed. Use "CONF.volume.catalog_type" to run volume tests under v3 or v2 APIs. * "CONF.volume-feature-enabled.api_v2" * "CONF.volume-feature-enabled.api_v3" Deprecation Notes ***************** * In this release the following tempest arguments are deprecated and replaced by new ones which are functionally equivalent: * --black-regex is replaced by --exclude-regex * --blacklist-file is replaced by --exclude-list * --whitelist-file is replaced by --include-list For now Tempest supports both (new and old ones) in order to make the transition for all consumers smoother. However, that's just a temporary case and the old options will be removed soon. Bug Fixes ********* * [bug 1613819 (https://bugs.launchpad.net/tempest/+bug/1613819)] admin_domain_name and default_credentials_domain_name parameters under [auth] now affect a domain used for creating test users just as they affect it for projects. Previously a domain with an id set to "default" had to be present in order for test user creation to succeed with Keystone v3. * Fixes the bug# 1917168 (https://bugs.launchpad.net/tempest/+bug/1917168) Changes in tempest stein-last..26.1.0 ------------------------------------- b8169f5a3 Add release notes for an Wallaby intermediate release. 2d0da049d Support scope in dynamic cred for specific roles ef13f406b Update dynamic creds to properly handle types during creation 24762611f Add bandwidth inventory to the subnodes in tempest-multinode-full-py3 f7d53297b Update subunit_describe_calls about new ports 795ef25b7 Implementation of create_subnet() varies in manila-tempest-plugin 420586c8b Add alt project persona in dynamic credential c37410fc1 Make _get_network_by_name() non-private 19b70a385 [part1] Remove six 4c3bf8f80 Remove usage of six 3bdafd897 Make _create_network() non-private 8c3dac1ab Remove usage of six 0fa206951 Remove usage of six f97c5cc1e Add log for all assigned roles for dynamic cred users 99d6e0098 Remove usage of six fd4dcf98c Make _cleanup_volume_type non-private 46d540d33 Add option to support volume_type in volume creation f4bd554b9 change ipv6 public network cidr on tempest-slow job 0ead75349 Enable swift in tempest-full-py3 a17cca4e7 Allow kwargs in create_volume_snapshot 8a8c85ddf Fix run-tempest black/exclude var setting 1a727de10 Modify the port fake data according to api reference 35eaed986 add unittest for trust client operation interface in identity part daf445a64 Optimize the check pipeline runtime 05bc7f17b Enable devstack parallel installation in tempest-full-parallel job 9e8729bc4 Implemetation of tempest client for keystone v3 OS_FEDERATION API 5b36c3689 Fix race condition for image test 7b98ea525 Fix --exlude typo in tox.ini 32e055780 Add release note and fix some TODO from system scope support 861c78f2f Disable file injection tests until bug/1882421 f5aef7bec Create default network for server advance scenario tests ef8e054b6 Make import wait check for store status f5e277c80 Add wait_for_resource_deletion for swift api clients cfd0a1526 Refactor skip_because decorator 945f5a1ce Make _create_security_group_rule non-private cb95bba84 Make _create_loginable_secgroup_rule non-private 1b0cddc90 Make _log_console_output non-private bd2acbfbc Allow use custom uppper constraints in stable/stein bfce1f219 Change Review merge policy from two +2 to single +2 requirement 6fb55c8fc Add config option for enforce_scope 249b30e0e test_live_migration: Rename test_iscsi_volume 7f3942522 Fix system & domain scoped admin dynamic credential 8b90b26cc Test for changing QoS policy with min bw of bound port 1ce92bf34 Make create_user domain-aware for V3CredsClient 26186cf96 test_encrypted_cinder_volumes: Do not create an image for this test eb3c7e37b Move horizon test from tempest-horizon to tempest a28849ff7 Fix project/domain scope in dynamic_creds 743d4953f Remove tempest/manager.py after 4 year deprecation dc84423b7 Inclusive jargon e7a513b46 Drop base grenade job for nova e044dada3 Test glance reserved properties 460084288 1912117: Fix the IP URL for metadata service in fixed network scenario 8663655af Make random_bytes() enforce sane size limits 8dfefcebe Fix memory explosion in multi-store image tests a811f44b5 Delete wrong argument from creating HTTP connection 8464cefbf Use CONF.identity.disable_ssl_certificate_validation in object_client 19213304d Add release note for new role client methods cbac9379a Add default RBAC personas to pre-provisioned creds 06374e2df Add default RBAC personas to dynamic credentials 5dbaaed88 Add client methods and tests for system grants cd0bbbdad Add system scope for admin auth 57d56fade Remove deprecated test.idempotent_id decorator 449287d45 Remove deprecated config options for volume API selection 983e1ceb4 Update microversion doc for compute microversion 2.3 9c8744172 Pass custom args to create_port through create_server f8625f0de Remove stable/stein testing jobs 5423c5309 scenario: Use wait_for_volume_attachment_remove_from_server in nova_volume_detach 77f2291c8 Add release notes page for version 26.0.0 be8ba2c27 Use proper kwargs in rebuild_server 5ffc28f19 add unitest for api of resource providers inventories cb4aff1a2 Revert "Use victoria stable constraint in tox to release 26.0.0" 4ad346e64 Also calculate external subnets when check cidr whether in use 4a14005dc Allow kwargs in create/restore_backup 23b57fdde Remove slow tag from test_minbw test 0090a1048 Allow kwargs in _get_router Diffstat (except docs and test files) ------------------------------------- REVIEWING.rst | 13 +- ... => tempest-non-active-plugins-registry.header} | 4 +- etc/{whitelist.yaml => allow-list.yaml} | 0 etc/rbac-persona-accounts.yaml.sample | 108 ++++++ ...N-APIs-as-tempest-clients-fe9e10a0fe5f09d4.yaml | 10 + .../notes/Inclusive-jargon-17621346744f0cf4.yaml | 13 + .../notes/Remove-manager-2e0b0af48f01294a.yaml | 5 + ...alt-project-dynamic-creds-1a3bc543e65d9433.yaml | 4 + ...tity-roles-system-methods-519dc144231993a3.yaml | 13 + ...e_loginable_secgroup_rule-73722fd4b4eb12d0.yaml | 6 + ...reate_security_group_rule-16d58a8f0f0ff262.yaml | 6 + ...ermediate-wallaby-release-55a0b31b1dee7b23.yaml | 4 + .../notes/log_console_output-dae6b8740b5a5821.yaml | 8 + ...aware-for-v3-creds-client-5054f58e715adc0c.yaml | 9 + ...ge-tempest-horizon-plugin-39d555339ab8c7ce.yaml | 6 + .../random-bytes-size-limit-ee94a8c6534fe916.yaml | 9 + ...ted-volume-config-options-4b7ea93b88e5b982.yaml | 9 + ...upport-for-rbac-new-scope-6ec8164ce1e7288c.yaml | 13 + ...n-get-roles-dynamic-creds-90bfab163c1c289a.yaml | 36 ++ .../notes/system-scope-44244cc955a7825f.yaml | 7 + releasenotes/source/index.rst | 1 + releasenotes/source/v26.0.0.rst | 5 + roles/run-tempest/README.rst | 15 +- roles/run-tempest/defaults/main.yaml | 1 - roles/run-tempest/tasks/main.yaml | 43 ++- .../compute/admin/test_instance_usage_audit_log.py | 3 +- .../test_instance_usage_audit_log_negative.py | 3 +- tempest/api/compute/admin/test_live_migration.py | 9 +- tempest/api/compute/admin/test_volume.py | 4 +- .../api/compute/flavors/test_flavors_negative.py | 5 +- tempest/api/compute/images/test_image_metadata.py | 4 +- .../images/test_images_oneserver_negative.py | 38 ++- .../api/compute/images/test_list_image_filters.py | 4 +- .../api/compute/servers/test_attach_interfaces.py | 5 +- tempest/api/compute/servers/test_novnc.py | 8 +- tempest/api/compute/servers/test_server_actions.py | 3 +- tempest/api/identity/admin/v3/test_roles.py | 37 +++ tempest/api/identity/admin/v3/test_tokens.py | 8 +- tempest/api/image/base.py | 4 +- tempest/api/image/v1/test_images.py | 8 +- tempest/api/image/v2/admin/test_images.py | 4 +- tempest/api/image/v2/test_images.py | 19 +- tempest/api/image/v2/test_images_negative.py | 42 +++ tempest/api/network/test_ports.py | 7 +- tempest/api/object_storage/base.py | 12 +- .../api/object_storage/test_account_services.py | 2 +- tempest/api/object_storage/test_container_sync.py | 2 +- tempest/api/object_storage/test_object_formpost.py | 3 +- .../test_object_formpost_negative.py | 3 +- tempest/api/object_storage/test_object_temp_url.py | 3 +- .../test_object_temp_url_negative.py | 3 +- tempest/api/volume/base.py | 24 +- tempest/api/volume/test_volumes_list.py | 2 +- tempest/api/volume/test_volumes_negative.py | 4 +- tempest/clients.py | 158 +++++---- tempest/cmd/cleanup_service.py | 3 +- tempest/cmd/init.py | 4 +- tempest/cmd/run.py | 116 +++++-- tempest/cmd/subunit_describe_calls.py | 24 +- tempest/cmd/verify_tempest_config.py | 14 - tempest/common/compute.py | 3 +- tempest/common/credentials_factory.py | 6 +- tempest/common/utils/__init__.py | 1 + tempest/common/waiters.py | 16 +- tempest/config.py | 106 ++++-- tempest/lib/auth.py | 26 +- tempest/lib/cmd/check_uuid.py | 2 +- tempest/lib/common/cred_client.py | 90 ++++- tempest/lib/common/cred_provider.py | 54 ++- tempest/lib/common/dynamic_creds.py | 205 +++++++++--- tempest/lib/common/jsonschema_validator.py | 3 +- tempest/lib/common/preprov_creds.py | 138 +++++++- tempest/lib/common/rest_client.py | 16 +- tempest/lib/common/ssh.py | 3 +- tempest/lib/common/utils/data_utils.py | 2 + tempest/lib/decorators.py | 16 +- tempest/lib/exceptions.py | 4 + tempest/lib/services/clients.py | 10 +- tempest/lib/services/compute/agents_client.py | 3 +- .../lib/services/compute/baremetal_nodes_client.py | 3 +- tempest/lib/services/compute/flavors_client.py | 3 +- .../services/compute/floating_ip_pools_client.py | 3 +- .../lib/services/compute/floating_ips_client.py | 3 +- tempest/lib/services/compute/hosts_client.py | 3 +- tempest/lib/services/compute/images_client.py | 3 +- tempest/lib/services/compute/keypairs_client.py | 3 +- tempest/lib/services/compute/migrations_client.py | 3 +- tempest/lib/services/compute/quotas_client.py | 3 +- .../lib/services/compute/security_groups_client.py | 3 +- tempest/lib/services/compute/servers_client.py | 2 +- tempest/lib/services/compute/services_client.py | 3 +- tempest/lib/services/compute/snapshots_client.py | 3 +- .../lib/services/compute/tenant_usages_client.py | 3 +- tempest/lib/services/compute/volumes_client.py | 3 +- .../lib/services/identity/v2/identity_client.py | 3 +- tempest/lib/services/identity/v2/roles_client.py | 3 +- .../lib/services/identity/v2/services_client.py | 3 +- tempest/lib/services/identity/v2/tenants_client.py | 3 +- tempest/lib/services/identity/v2/users_client.py | 3 +- .../services/identity/v3/access_rules_client.py | 3 +- .../identity/v3/application_credentials_client.py | 3 +- .../lib/services/identity/v3/credentials_client.py | 3 +- tempest/lib/services/identity/v3/domains_client.py | 3 +- .../lib/services/identity/v3/endpoints_client.py | 3 +- tempest/lib/services/identity/v3/groups_client.py | 3 +- .../identity/v3/identity_providers_client.py | 92 +++++ .../lib/services/identity/v3/mappings_client.py | 90 +++++ .../lib/services/identity/v3/oauth_token_client.py | 8 +- .../lib/services/identity/v3/projects_client.py | 3 +- .../lib/services/identity/v3/protocols_client.py | 96 ++++++ tempest/lib/services/identity/v3/regions_client.py | 3 +- .../identity/v3/role_assignments_client.py | 3 +- tempest/lib/services/identity/v3/roles_client.py | 58 +++- .../identity/v3/service_providers_client.py | 92 +++++ .../lib/services/identity/v3/services_client.py | 3 +- tempest/lib/services/identity/v3/token_client.py | 6 +- tempest/lib/services/identity/v3/trusts_client.py | 3 +- tempest/lib/services/identity/v3/users_client.py | 3 +- tempest/lib/services/image/v1/images_client.py | 2 +- tempest/lib/services/image/v2/images_client.py | 2 +- .../services/image/v2/namespace_objects_client.py | 3 +- .../lib/services/image/v2/namespace_tags_client.py | 3 +- tempest/lib/services/network/base.py | 3 +- .../lib/services/object_storage/account_client.py | 2 +- .../services/object_storage/container_client.py | 10 +- .../lib/services/object_storage/object_client.py | 43 ++- tempest/lib/services/placement/placement_client.py | 3 +- .../placement/resource_providers_client.py | 29 +- tempest/lib/services/volume/v1/hosts_client.py | 3 +- tempest/lib/services/volume/v1/quotas_client.py | 3 +- tempest/lib/services/volume/v1/services_client.py | 3 +- tempest/lib/services/volume/v1/snapshots_client.py | 3 +- tempest/lib/services/volume/v1/types_client.py | 3 +- tempest/lib/services/volume/v1/volumes_client.py | 3 +- tempest/lib/services/volume/v3/backups_client.py | 3 +- .../services/volume/v3/group_snapshots_client.py | 3 +- .../lib/services/volume/v3/group_types_client.py | 3 +- tempest/lib/services/volume/v3/groups_client.py | 3 +- tempest/lib/services/volume/v3/hosts_client.py | 3 +- tempest/lib/services/volume/v3/quotas_client.py | 3 +- tempest/lib/services/volume/v3/services_client.py | 3 +- tempest/lib/services/volume/v3/snapshots_client.py | 3 +- tempest/lib/services/volume/v3/transfers_client.py | 3 +- tempest/lib/services/volume/v3/types_client.py | 3 +- tempest/lib/services/volume/v3/versions_client.py | 2 +- tempest/lib/services/volume/v3/volumes_client.py | 3 +- tempest/manager.py | 62 ---- tempest/scenario/manager.py | 266 +++++++++------ tempest/scenario/test_dashboard_basic_ops.py | 141 ++++++++ tempest/scenario/test_encrypted_cinder_volumes.py | 6 +- .../scenario/test_minbw_allocation_placement.py | 245 ++++++++++++-- tempest/scenario/test_network_basic_ops.py | 2 +- tempest/scenario/test_network_v6.py | 6 +- tempest/scenario/test_security_groups_basic_ops.py | 16 +- tempest/scenario/test_server_advanced_ops.py | 2 +- tempest/scenario/test_server_basic_ops.py | 5 +- .../orchestration/json/orchestration_client.py | 2 +- tempest/test.py | 32 +- tempest/test_discover/plugins.py | 4 +- .../identity/v3/test_identity_providers_client.py | 142 ++++++++ .../services/identity/v3/test_mappings_client.py | 183 ++++++++++ .../services/identity/v3/test_protocols_client.py | 140 ++++++++ .../lib/services/identity/v3/test_roles_client.py | 73 ++++ .../identity/v3/test_service_providers_client.py | 157 +++++++++ .../lib/services/identity/v3/test_trusts_client.py | 66 ++++ .../lib/services/image/v2/test_images_client.py | 4 +- .../lib/services/network/test_ports_client.py | 87 ++++- .../services/object_storage/test_object_client.py | 9 +- .../placement/test_resource_providers_client.py | 63 ++++ tools/check_logs.py | 49 +-- tools/generate-tempest-plugins-list.py | 19 +- tools/generate-tempest-plugins-list.sh | 12 +- ...mpest-integrated-gate-compute-exclude-list.txt} | 6 - ...st-integrated-gate-networking-exclude-list.txt} | 0 ...ntegrated-gate-object-storage-exclude-list.txt} | 0 ...est-integrated-gate-placement-exclude-list.txt} | 0 .../tempest-integrated-gate-storage-blacklist.txt | 15 +- ...empest-integrated-gate-storage-exclude-list.txt | 14 + tools/tempest-plugin-sanity.sh | 6 +- tox.ini | 58 ++-- zuul.d/integrated-gate.yaml | 107 ++++-- zuul.d/project.yaml | 14 - zuul.d/stable-jobs.yaml | 6 - zuul.d/tempest-specific.yaml | 2 + 208 files changed, 4273 insertions(+), 1046 deletions(-)