[release-announce] tempest 26.1.0 (wallaby)
no-reply at openstack.org
no-reply at openstack.org
Wed Mar 10 09:47:01 UTC 2021
We are pumped to announce the release of:
tempest 26.1.0: OpenStack Integration Testing
This release is part of the wallaby release series.
The source is available from:
https://opendev.org/openstack/tempest
Download the package from:
https://pypi.org/project/tempest
Please report issues through:
https://bugs.launchpad.net/tempest/+bugs
For more details, please see below.
26.1.0
^^^^^^
Prelude
*******
This is an intermediate release during the Wallaby development cycle
to make new functionality available to plugins and other consumers.
The integrated horizon dashboard test is now moved from tempest-
horizon plugin into Tempest. You do not need to install tempest-
horizon to run the horizon test which can be run using Tempest itself.
Support for RBAC new system scope is added in Tempest.
New Features
************
* The following tempest clients for keystone v3 OS_FEDERATION API
were implemented in this release
* identity_providers
* protocols
* mappings
* service_providers
* Add project alternate admin, member and reader role for dynamic
credentials.
* Added methods to the identity v3 roles client to support:
* PUT /v3/system/users/{user}/roles/{role}
* GET /v3/system/users/{user}/roles
* GET /v3/system/users/{user}/roles/{role}
* DELETE /v3/system/users/{user}/roles/{role}
* PUT /v3/system/groups/{group}/roles/{role}
* GET /v3/system/groups/{group}/roles
* GET /v3/system/groups/{group}/roles/{role}
* DELETE /v3/system/groups/{group}/roles/{role}
* Added public interface create_loginable_secgroup_rule(). Since
this interface is meant to be used by tempest plugins, It doesn't
neccessarily require to be private api.
* Added public interface create_security_group_rule(). Since this
interface is meant to be used by tempest plugins, It doesn't
neccessarily require to be private api.
* Added public interface log_console_output(). It used to be a
private method with name _log_console_output(). Since this interface
is meant to be used by tempest plugins, It doesn't neccessarily
require to be private api.
* Keystone provides the new scoped token called "system" which can
be used to query the system scoped API operation. Projects are
moving towards the policy with new scope types, Keystone, Nova
already provide the new policy for RBAC checks. Tempest has added
the support to query the system scoped token from keystone to test
the new policy. As next step, we will be moving all the Tempest
tests on the project's new policy.
* Dynamic credentials now support the scope type for specific roles
too along with "admin", "member", "reader" role. Test can specify
the scope in the prefix of "cls.credentials" name. If "system" is
prefix in "cls.credentials" name then creds will be created with
scope as "system". If "domain" is prefix in "cls.credentials" name
then creds will be created with scope as "domain" otherwise default
"project" scope will be used. For Example:
credentials = [['my_role', 'role1'], # this will be old style and project scoped
['project_my_role', 'role1'], # this will be project scoped
['domain_my_role', 'role1'], # this will be domain scoped
['system_my_role', 'role1']] # this will be system scoped
And below is how test can access the credential manager of
respective credentials type:
cls.os_my_role.any_client
cls.os_project_my_role.any_client
cls.os_domain_my_role.any_client
cls.os_system_my_role.any_client
For backward compatibility, we set the credentials manager class
attribute in old style form too which is prefix with "os_roles_*",
example "cls.os_roles_my_role" but we recommend to use the new style
attribute as shown above.
* Adds new personas that can be used to test service policies for
all default scopes (project, domain, and system) and roles (reader,
member, and admin). Both dynamic credentials and pre-provisioned
credentials are supported.
Known Issues
************
* Scope support for specific role is not yet added for pre-
provisioned credentials.
Upgrade Notes
*************
* In this release tempest/manager.py is removed after more than 4
years of deprecation.
* The "tempest.lib.common.utils.data_utils.random_bytes()" helper
function will no longer allow a "size" of more than 1MiB. Tests
generally do not need to generate and use large payloads for feature
verification and it is easy to lose track of and duplicate large
buffers. The sum total of such errors can become problematic in
paralllelized and constrained CI environments.
* Deprecated config options to select the Volume API version have
been removed. Use "CONF.volume.catalog_type" to run volume tests
under v3 or v2 APIs.
* "CONF.volume-feature-enabled.api_v2"
* "CONF.volume-feature-enabled.api_v3"
Deprecation Notes
*****************
* In this release the following tempest arguments are deprecated and
replaced by new ones which are functionally equivalent:
* --black-regex is replaced by --exclude-regex
* --blacklist-file is replaced by --exclude-list
* --whitelist-file is replaced by --include-list
For now Tempest supports both (new and old ones) in order to make
the transition for all consumers smoother. However, that's just a
temporary case and the old options will be removed soon.
Bug Fixes
*********
* [bug 1613819 (https://bugs.launchpad.net/tempest/+bug/1613819)]
admin_domain_name and default_credentials_domain_name parameters
under [auth] now affect a domain used for creating test users just
as they affect it for projects. Previously a domain with an id set
to "default" had to be present in order for test user creation to
succeed with Keystone v3.
* Fixes the bug# 1917168
(https://bugs.launchpad.net/tempest/+bug/1917168)
Changes in tempest stein-last..26.1.0
-------------------------------------
b8169f5a3 Add release notes for an Wallaby intermediate release.
2d0da049d Support scope in dynamic cred for specific roles
ef13f406b Update dynamic creds to properly handle types during creation
24762611f Add bandwidth inventory to the subnodes in tempest-multinode-full-py3
f7d53297b Update subunit_describe_calls about new ports
795ef25b7 Implementation of create_subnet() varies in manila-tempest-plugin
420586c8b Add alt project persona in dynamic credential
c37410fc1 Make _get_network_by_name() non-private
19b70a385 [part1] Remove six
4c3bf8f80 Remove usage of six
3bdafd897 Make _create_network() non-private
8c3dac1ab Remove usage of six
0fa206951 Remove usage of six
f97c5cc1e Add log for all assigned roles for dynamic cred users
99d6e0098 Remove usage of six
fd4dcf98c Make _cleanup_volume_type non-private
46d540d33 Add option to support volume_type in volume creation
f4bd554b9 change ipv6 public network cidr on tempest-slow job
0ead75349 Enable swift in tempest-full-py3
a17cca4e7 Allow kwargs in create_volume_snapshot
8a8c85ddf Fix run-tempest black/exclude var setting
1a727de10 Modify the port fake data according to api reference
35eaed986 add unittest for trust client operation interface in identity part
daf445a64 Optimize the check pipeline runtime
05bc7f17b Enable devstack parallel installation in tempest-full-parallel job
9e8729bc4 Implemetation of tempest client for keystone v3 OS_FEDERATION API
5b36c3689 Fix race condition for image test
7b98ea525 Fix --exlude typo in tox.ini
32e055780 Add release note and fix some TODO from system scope support
861c78f2f Disable file injection tests until bug/1882421
f5aef7bec Create default network for server advance scenario tests
ef8e054b6 Make import wait check for store status
f5e277c80 Add wait_for_resource_deletion for swift api clients
cfd0a1526 Refactor skip_because decorator
945f5a1ce Make _create_security_group_rule non-private
cb95bba84 Make _create_loginable_secgroup_rule non-private
1b0cddc90 Make _log_console_output non-private
bd2acbfbc Allow use custom uppper constraints in stable/stein
bfce1f219 Change Review merge policy from two +2 to single +2 requirement
6fb55c8fc Add config option for enforce_scope
249b30e0e test_live_migration: Rename test_iscsi_volume
7f3942522 Fix system & domain scoped admin dynamic credential
8b90b26cc Test for changing QoS policy with min bw of bound port
1ce92bf34 Make create_user domain-aware for V3CredsClient
26186cf96 test_encrypted_cinder_volumes: Do not create an image for this test
eb3c7e37b Move horizon test from tempest-horizon to tempest
a28849ff7 Fix project/domain scope in dynamic_creds
743d4953f Remove tempest/manager.py after 4 year deprecation
dc84423b7 Inclusive jargon
e7a513b46 Drop base grenade job for nova
e044dada3 Test glance reserved properties
460084288 1912117: Fix the IP URL for metadata service in fixed network scenario
8663655af Make random_bytes() enforce sane size limits
8dfefcebe Fix memory explosion in multi-store image tests
a811f44b5 Delete wrong argument from creating HTTP connection
8464cefbf Use CONF.identity.disable_ssl_certificate_validation in object_client
19213304d Add release note for new role client methods
cbac9379a Add default RBAC personas to pre-provisioned creds
06374e2df Add default RBAC personas to dynamic credentials
5dbaaed88 Add client methods and tests for system grants
cd0bbbdad Add system scope for admin auth
57d56fade Remove deprecated test.idempotent_id decorator
449287d45 Remove deprecated config options for volume API selection
983e1ceb4 Update microversion doc for compute microversion 2.3
9c8744172 Pass custom args to create_port through create_server
f8625f0de Remove stable/stein testing jobs
5423c5309 scenario: Use wait_for_volume_attachment_remove_from_server in nova_volume_detach
77f2291c8 Add release notes page for version 26.0.0
be8ba2c27 Use proper kwargs in rebuild_server
5ffc28f19 add unitest for api of resource providers inventories
cb4aff1a2 Revert "Use victoria stable constraint in tox to release 26.0.0"
4ad346e64 Also calculate external subnets when check cidr whether in use
4a14005dc Allow kwargs in create/restore_backup
23b57fdde Remove slow tag from test_minbw test
0090a1048 Allow kwargs in _get_router
Diffstat (except docs and test files)
-------------------------------------
REVIEWING.rst | 13 +-
... => tempest-non-active-plugins-registry.header} | 4 +-
etc/{whitelist.yaml => allow-list.yaml} | 0
etc/rbac-persona-accounts.yaml.sample | 108 ++++++
...N-APIs-as-tempest-clients-fe9e10a0fe5f09d4.yaml | 10 +
.../notes/Inclusive-jargon-17621346744f0cf4.yaml | 13 +
.../notes/Remove-manager-2e0b0af48f01294a.yaml | 5 +
...alt-project-dynamic-creds-1a3bc543e65d9433.yaml | 4 +
...tity-roles-system-methods-519dc144231993a3.yaml | 13 +
...e_loginable_secgroup_rule-73722fd4b4eb12d0.yaml | 6 +
...reate_security_group_rule-16d58a8f0f0ff262.yaml | 6 +
...ermediate-wallaby-release-55a0b31b1dee7b23.yaml | 4 +
.../notes/log_console_output-dae6b8740b5a5821.yaml | 8 +
...aware-for-v3-creds-client-5054f58e715adc0c.yaml | 9 +
...ge-tempest-horizon-plugin-39d555339ab8c7ce.yaml | 6 +
.../random-bytes-size-limit-ee94a8c6534fe916.yaml | 9 +
...ted-volume-config-options-4b7ea93b88e5b982.yaml | 9 +
...upport-for-rbac-new-scope-6ec8164ce1e7288c.yaml | 13 +
...n-get-roles-dynamic-creds-90bfab163c1c289a.yaml | 36 ++
.../notes/system-scope-44244cc955a7825f.yaml | 7 +
releasenotes/source/index.rst | 1 +
releasenotes/source/v26.0.0.rst | 5 +
roles/run-tempest/README.rst | 15 +-
roles/run-tempest/defaults/main.yaml | 1 -
roles/run-tempest/tasks/main.yaml | 43 ++-
.../compute/admin/test_instance_usage_audit_log.py | 3 +-
.../test_instance_usage_audit_log_negative.py | 3 +-
tempest/api/compute/admin/test_live_migration.py | 9 +-
tempest/api/compute/admin/test_volume.py | 4 +-
.../api/compute/flavors/test_flavors_negative.py | 5 +-
tempest/api/compute/images/test_image_metadata.py | 4 +-
.../images/test_images_oneserver_negative.py | 38 ++-
.../api/compute/images/test_list_image_filters.py | 4 +-
.../api/compute/servers/test_attach_interfaces.py | 5 +-
tempest/api/compute/servers/test_novnc.py | 8 +-
tempest/api/compute/servers/test_server_actions.py | 3 +-
tempest/api/identity/admin/v3/test_roles.py | 37 +++
tempest/api/identity/admin/v3/test_tokens.py | 8 +-
tempest/api/image/base.py | 4 +-
tempest/api/image/v1/test_images.py | 8 +-
tempest/api/image/v2/admin/test_images.py | 4 +-
tempest/api/image/v2/test_images.py | 19 +-
tempest/api/image/v2/test_images_negative.py | 42 +++
tempest/api/network/test_ports.py | 7 +-
tempest/api/object_storage/base.py | 12 +-
.../api/object_storage/test_account_services.py | 2 +-
tempest/api/object_storage/test_container_sync.py | 2 +-
tempest/api/object_storage/test_object_formpost.py | 3 +-
.../test_object_formpost_negative.py | 3 +-
tempest/api/object_storage/test_object_temp_url.py | 3 +-
.../test_object_temp_url_negative.py | 3 +-
tempest/api/volume/base.py | 24 +-
tempest/api/volume/test_volumes_list.py | 2 +-
tempest/api/volume/test_volumes_negative.py | 4 +-
tempest/clients.py | 158 +++++----
tempest/cmd/cleanup_service.py | 3 +-
tempest/cmd/init.py | 4 +-
tempest/cmd/run.py | 116 +++++--
tempest/cmd/subunit_describe_calls.py | 24 +-
tempest/cmd/verify_tempest_config.py | 14 -
tempest/common/compute.py | 3 +-
tempest/common/credentials_factory.py | 6 +-
tempest/common/utils/__init__.py | 1 +
tempest/common/waiters.py | 16 +-
tempest/config.py | 106 ++++--
tempest/lib/auth.py | 26 +-
tempest/lib/cmd/check_uuid.py | 2 +-
tempest/lib/common/cred_client.py | 90 ++++-
tempest/lib/common/cred_provider.py | 54 ++-
tempest/lib/common/dynamic_creds.py | 205 +++++++++---
tempest/lib/common/jsonschema_validator.py | 3 +-
tempest/lib/common/preprov_creds.py | 138 +++++++-
tempest/lib/common/rest_client.py | 16 +-
tempest/lib/common/ssh.py | 3 +-
tempest/lib/common/utils/data_utils.py | 2 +
tempest/lib/decorators.py | 16 +-
tempest/lib/exceptions.py | 4 +
tempest/lib/services/clients.py | 10 +-
tempest/lib/services/compute/agents_client.py | 3 +-
.../lib/services/compute/baremetal_nodes_client.py | 3 +-
tempest/lib/services/compute/flavors_client.py | 3 +-
.../services/compute/floating_ip_pools_client.py | 3 +-
.../lib/services/compute/floating_ips_client.py | 3 +-
tempest/lib/services/compute/hosts_client.py | 3 +-
tempest/lib/services/compute/images_client.py | 3 +-
tempest/lib/services/compute/keypairs_client.py | 3 +-
tempest/lib/services/compute/migrations_client.py | 3 +-
tempest/lib/services/compute/quotas_client.py | 3 +-
.../lib/services/compute/security_groups_client.py | 3 +-
tempest/lib/services/compute/servers_client.py | 2 +-
tempest/lib/services/compute/services_client.py | 3 +-
tempest/lib/services/compute/snapshots_client.py | 3 +-
.../lib/services/compute/tenant_usages_client.py | 3 +-
tempest/lib/services/compute/volumes_client.py | 3 +-
.../lib/services/identity/v2/identity_client.py | 3 +-
tempest/lib/services/identity/v2/roles_client.py | 3 +-
.../lib/services/identity/v2/services_client.py | 3 +-
tempest/lib/services/identity/v2/tenants_client.py | 3 +-
tempest/lib/services/identity/v2/users_client.py | 3 +-
.../services/identity/v3/access_rules_client.py | 3 +-
.../identity/v3/application_credentials_client.py | 3 +-
.../lib/services/identity/v3/credentials_client.py | 3 +-
tempest/lib/services/identity/v3/domains_client.py | 3 +-
.../lib/services/identity/v3/endpoints_client.py | 3 +-
tempest/lib/services/identity/v3/groups_client.py | 3 +-
.../identity/v3/identity_providers_client.py | 92 +++++
.../lib/services/identity/v3/mappings_client.py | 90 +++++
.../lib/services/identity/v3/oauth_token_client.py | 8 +-
.../lib/services/identity/v3/projects_client.py | 3 +-
.../lib/services/identity/v3/protocols_client.py | 96 ++++++
tempest/lib/services/identity/v3/regions_client.py | 3 +-
.../identity/v3/role_assignments_client.py | 3 +-
tempest/lib/services/identity/v3/roles_client.py | 58 +++-
.../identity/v3/service_providers_client.py | 92 +++++
.../lib/services/identity/v3/services_client.py | 3 +-
tempest/lib/services/identity/v3/token_client.py | 6 +-
tempest/lib/services/identity/v3/trusts_client.py | 3 +-
tempest/lib/services/identity/v3/users_client.py | 3 +-
tempest/lib/services/image/v1/images_client.py | 2 +-
tempest/lib/services/image/v2/images_client.py | 2 +-
.../services/image/v2/namespace_objects_client.py | 3 +-
.../lib/services/image/v2/namespace_tags_client.py | 3 +-
tempest/lib/services/network/base.py | 3 +-
.../lib/services/object_storage/account_client.py | 2 +-
.../services/object_storage/container_client.py | 10 +-
.../lib/services/object_storage/object_client.py | 43 ++-
tempest/lib/services/placement/placement_client.py | 3 +-
.../placement/resource_providers_client.py | 29 +-
tempest/lib/services/volume/v1/hosts_client.py | 3 +-
tempest/lib/services/volume/v1/quotas_client.py | 3 +-
tempest/lib/services/volume/v1/services_client.py | 3 +-
tempest/lib/services/volume/v1/snapshots_client.py | 3 +-
tempest/lib/services/volume/v1/types_client.py | 3 +-
tempest/lib/services/volume/v1/volumes_client.py | 3 +-
tempest/lib/services/volume/v3/backups_client.py | 3 +-
.../services/volume/v3/group_snapshots_client.py | 3 +-
.../lib/services/volume/v3/group_types_client.py | 3 +-
tempest/lib/services/volume/v3/groups_client.py | 3 +-
tempest/lib/services/volume/v3/hosts_client.py | 3 +-
tempest/lib/services/volume/v3/quotas_client.py | 3 +-
tempest/lib/services/volume/v3/services_client.py | 3 +-
tempest/lib/services/volume/v3/snapshots_client.py | 3 +-
tempest/lib/services/volume/v3/transfers_client.py | 3 +-
tempest/lib/services/volume/v3/types_client.py | 3 +-
tempest/lib/services/volume/v3/versions_client.py | 2 +-
tempest/lib/services/volume/v3/volumes_client.py | 3 +-
tempest/manager.py | 62 ----
tempest/scenario/manager.py | 266 +++++++++------
tempest/scenario/test_dashboard_basic_ops.py | 141 ++++++++
tempest/scenario/test_encrypted_cinder_volumes.py | 6 +-
.../scenario/test_minbw_allocation_placement.py | 245 ++++++++++++--
tempest/scenario/test_network_basic_ops.py | 2 +-
tempest/scenario/test_network_v6.py | 6 +-
tempest/scenario/test_security_groups_basic_ops.py | 16 +-
tempest/scenario/test_server_advanced_ops.py | 2 +-
tempest/scenario/test_server_basic_ops.py | 5 +-
.../orchestration/json/orchestration_client.py | 2 +-
tempest/test.py | 32 +-
tempest/test_discover/plugins.py | 4 +-
.../identity/v3/test_identity_providers_client.py | 142 ++++++++
.../services/identity/v3/test_mappings_client.py | 183 ++++++++++
.../services/identity/v3/test_protocols_client.py | 140 ++++++++
.../lib/services/identity/v3/test_roles_client.py | 73 ++++
.../identity/v3/test_service_providers_client.py | 157 +++++++++
.../lib/services/identity/v3/test_trusts_client.py | 66 ++++
.../lib/services/image/v2/test_images_client.py | 4 +-
.../lib/services/network/test_ports_client.py | 87 ++++-
.../services/object_storage/test_object_client.py | 9 +-
.../placement/test_resource_providers_client.py | 63 ++++
tools/check_logs.py | 49 +--
tools/generate-tempest-plugins-list.py | 19 +-
tools/generate-tempest-plugins-list.sh | 12 +-
...mpest-integrated-gate-compute-exclude-list.txt} | 6 -
...st-integrated-gate-networking-exclude-list.txt} | 0
...ntegrated-gate-object-storage-exclude-list.txt} | 0
...est-integrated-gate-placement-exclude-list.txt} | 0
.../tempest-integrated-gate-storage-blacklist.txt | 15 +-
...empest-integrated-gate-storage-exclude-list.txt | 14 +
tools/tempest-plugin-sanity.sh | 6 +-
tox.ini | 58 ++--
zuul.d/integrated-gate.yaml | 107 ++++--
zuul.d/project.yaml | 14 -
zuul.d/stable-jobs.yaml | 6 -
zuul.d/tempest-specific.yaml | 2 +
208 files changed, 4273 insertions(+), 1046 deletions(-)
More information about the Release-announce
mailing list