Open Stack

We are psyched to announce the release of:

tempest 26.0.0: OpenStack Integration Testing

This release is part of the wallaby release series.

The source is available from:

    https://opendev.org/openstack/tempest

Download the package from:

    https://pypi.org/project/tempest

Please report issues through:

    https://bugs.launchpad.net/tempest/+bugs

For more details, please see below.

26.0.0
^^^^^^


Prelude
*******

This is an intermediate release during the Wallaby development cycle
to mark the end of support for EM Stein release in Tempest. After this
release, Tempest will support below OpenStack Releases:

* Victoria

* Ussuri

* Train

Current development of Tempest is for OpenStack Wallaby development
cycle.


New Features
************

* Add a new config option can_migrate_between_any_hosts in the
  compute-feature-enabled section, which can be set to False for
  environment with non homogeneous compute nodes, so that it can
  select a destination host for migrating automatically, otherwise the
  testcase may fail unexpectedly. e.g., if source host is with CPU
  "E5-2699 v4" and the selected target host is with CPU "E5-2670 v3",
  the live-migration will fail because of the downgrade issue.

* Add a new config option "[compute-feature-enabled] shelve_migrate"
  which enable test for environment that support cold migration of
  qcow2 unshelved instance.

* A new config option in the validation section, image_alt_ssh_user,
  to specify the user name used to authenticate to an alternative
  instance (instance using image_ref_alt) in tests. By default this is
  set to root.

* A new config option in the validation section,
  image_alt_ssh_password, to specify the password used to authenticate
  to an alternative instance (instance using image_ref_alt) in tests.
  By default this is set to password.

* Added missing client methods for keystone's OS-ENDPOINT-POLICY and
  OS-EP-FILTER APIs.

* Added missing clients and tests for keystone's v3 EC2 API which
  already existed for keystone v2.

* Add show type API to v3 types_client library.

  * default_volume_type

* Added associate_floating_ip() and dissociate_floating_ip() methods
  to the scenario manager.

* Add glance image import APIs function to v2 images_client library.

  * stage_image_file

  * info_import

  * info_stores

  * image_import

* [blueprint blueprint adopt-oslo-versioned-objects-for-db
  (https://blueprints.launchpad.net/neutron/+spec/adopt-oslo-
  versioned-objects-for-db)] Any reference to "tenant_id" in Network
  objects is replaced with "project_id".

* New config option to "network-feature-enabled":
  "available_features". This is a list which can contain features that
  are not discoverable through Neutron API, or it can be the special
  entry "all".

* Add "placement" API methods for testing Routed Provider Networks
  feature. The following API calls are available for tempest from now
  in the new resource_providers_client:

  * GET /resource_providers

  * GET /resource_providers/{uuid}

  * GET /resource_providers/{uuid}/inventories

  * GET /resource_providers/{uuid}/aggregates


Upgrade Notes
*************

* The following deprecated image scenario options are removed after
  a ~4 year deprecation period.

  * "ami_img_file"

  * "ari_img_file"

  * "aki_img_file"

  Starting Tempest 25.0.0 release, CONF.scenario.img_file need a full
  path for the image. CONF.scenario.img_dir was deprecated and will be
  removed in the next release. Till Tempest 25.0.0, old behavior is
  maintained and keep working but starting Tempest 26.0.0, you need to
  specify the full path in CONF.scenario.img_file config option.

* The test_reboot_server_soft has been skipped for more than 6
  years. Take into account that the minimum scenario test uses soft
  reboot and the nova functional test also covers reboot.

* "Member" role has been deprecated and replaced by "member".
  Therefore the default value of config option "[object-
  storage].operator_role" is changed to "member". (Fixes bug #1330132)

* Remove the deprecated CLI "tempest-account-generator" in favor of
  "tempest account-generator" command. You can use "tempest account-
  generator" CLI to generate the accounts yaml file.

* A number of Compute APIs that only worked with the XenAPI virt
  driver have been removed in the Compute service. As a result, their
  corresponding tests are now disabled by default. They can be re-
  enabled using the new "[compute_feature_enabled] xenapi_apis" config
  option.


Bug Fixes
*********

* Fixed bug #1890060. tempest subunit_describe_calls --verbose not
  working with Cliff CLI. The subunit_describe_calls --verbose
  argument was a boolean and worked in the non Cliff CLI which is now
  deprecated, but does not work with cliff since --verbase is a
  standard cliff argument which is an int.  Since the tool is in lib
  directory we cannot change the interface, so we add a new argument
  -a --all-stdout that will allow cliff CLI to support the feature in
  subunnit_describe_calls to print request and response headers and
  bodies to stdout.

* is_resource_deleted method of v3 volumes_client might have
  returned a KeyError exception due to an incorrect accessing of a
  volume id in the case the volume was in error_deleting state.
  incorrect code - volume['id'] correct code - volume['volume']['id']
  More details about the issue can be found at
  https://bugs.launchpad.net/tempest/+bug/1887980

* Concurrency parameter for account-generator command was accepting
  negative values and zero. The concurrency parameter now accepts only
  positive numbers. When a negative value or zero is passed to the
  program then the program ends and help is displayed.


Other Notes
***********

* New configuration options
  "CONF.glance.image_feature_enabled.image_import" has been introduced
  to enable the image import tests. If your glance deployement support
  image import functionality then you can enable the image import
  tests via this flag. Default value of this new config option is
  false.

Changes in tempest 25.0.1..26.0.0
---------------------------------

f9d31f2c4 Use victoria stable constraint in tox to release 26.0.0
2c3da54a8 Add releasenote to tag the end of support for Stein
b0b480c38 Remove unused LOG var in test_stamp_pattern.py
2e515fe59 Remove race due to 1907084
064a340dc New minimum guaranteed bandwidth tests
edda62ab3 Fix indentation for the example code in documentation.
a078ddd7c Need to add kwargs for the api
9ad9ca277 Allow kwargs in create_server_snapshot
2c230eb21 Allow kwargs in get_timestamp
27fd5cb38 Allow additional port_vnic_type
8581ea2cd compute: Skip AttachVolumeShelveTestJSON when cross_az_attach unavailable
7d9b50a81 Cleanup server before image at test level
0123f5212 Need to have stable implementation of apis
ed66447aa Added apis to scenario manager
0216b3768 Allow kwargs in create_volume_from_image
adcddceab compute: Cleanup server before image in AttachSCSIVolumeTestJSON
b50eabe29 Allow kwargs in image_create
61f5733a6 Decentralize the zuul jobs into zuul.d
940a9e215 Fix TypeError when there exists image with size None
cdcdf2ba5 Skip test_create_object_with_transfer_encoding
3d7010914 Change help string to use openstack instead of neutron
3ead99814 [Trivial]Remove unused variables
414ffbacb Fix "TypeError: format requires a mapping" in waiters
0a9b8235b Add related test to Bug #1732428
776185971 Add tag in compute interfaces schema for microversion 2.70
bf5ed0b67 [Trivial]Remove unused variables and methods
d926b8d08 [Trivial]Do not use cls in instance method
bc360aeb0 Skip test_server_rescue with volumes when there is no Cinder
e3405ba80 Check VM's console log before trying to SSH to it.
25f7ba2e1 Allow kwargs in create_timestamp
76819faf4 Allow kwargs in nova_volume_attach
f759bc1cf Allow kwargs in create_floating_ip
7d91c8734 Remove six.PY3/six.PY2
fec0dc182 Add glance multistore job (nv) in tempest gate
9cee53835 Remove 'gabbi_tempest_path' from devstack-tempest job
3fdbab4e3 Clarify tempest-full job description
b0642f90e Allow kwargs in get_server_ip
9523d51cc Allow kwargs in _get_server_port_id_and_ip4
c54a91100 Set image_alt_ssh_user during stack
d3ad5e94d Allow kwargs in create_volume
bee673e74 Allow kwargs in boot_instance_from_resource
6aa0fa356 Replace assertItemsEqual with assertEqual
05c35eb79 Improve tempest-cleanup ansible role
5e761bfad Allow kwargs in create_floating_ip
f9bb8b8b2 New list config option to list network features without API extension
e1d873a5f Stabilize _create_loginable_secgroup_rule
721aeca8e Unskip Device detach compute tests on Focal
91ded0401 Allow kwargs in _log_console_output
4346a82ff Add glance image import copy-image tests
fb1e4db2b Fix incorrect compute api microversion history links
35c2c1203 Fix url link for get_remote_console interface
0d6220824 Test attach scsi disk with config drive enabled
cfd6159f9 Make neutron-tempest-dvr job non-voting
8e673a408 Allow kwargs in create_volume_type
20c87c795 Replace display_name by name in scenario manager
30b4d5335 Allow kwargs in create_keypair
4de3d536b Skip adding host to aggregate with az if host already in other zone
5f65bf580 Fix error that instance could not be found
104e0b731 Delete test_reboot_server_soft
f0547d606 Add Tempest gate job for stable/victoria
9667217d5 zuul: Move tempest-integrated-compute to Focal
efb5d6aba Retry on ServerFault in call_and_ignore_notfound_exc()
ec60bfff2 add unittest for endpoints operation interface in identity part
ce23482fd Skip additional test if 'xenapi_apis' flag is False
e4f282051 Check port allocation for VM with resource request
23e0747e5 Add placement API methods for testing routed provider nets
cdbfb5eb3 Fix victoria releasenote to align yaml format
491d63e12 Add glance multistore tests
5eaf6bdc9 Add keystone v3 EC2 credential tests and client
97756b5b2 Remove install unnecessary packages
4599b91fe Add missing keystone clients
1ca760cd2 Migrate tempest jobs to focal
7313bc461 Revert "Use stable constraint in tox to release new tag for Victoria"
13c743407 Updating the paramiko version to 2.7.0 or greater
c00346cb2 Add missing replication_status in group schema
c8b966ff6 Fix import line location
4e7b2f459 Fix flavor NotFound in test_resize_server_revert_deleted_flavor
9b442510e Adding description for testcases - swift part1
639c4700a Set max_microversion 2.56 for ServerPersonalityTestJSON
cd9c2ea3a Add missing attachments schema in servers.py
0672955e1 Fix negative tests of update_volume for volume microversion 3.59
9542ad2fb Modify the fake data according to api reference
4bd9edbd3 volume: Introduce extend attached encrypted volume tests
5e8f04453 Add show types API to v3 types client library
46172435e Modify the fake data for test_networksks_client for v3
d2cbd3ad4 Add Access Rules tests
e42bd9354 Added stestr for running test using pdb flag


Diffstat (except docs and test files)
-------------------------------------

.zuul.yaml                                         | 748 ---------------------
playbooks/devstack-tempest-ipv6.yaml               |   5 -
playbooks/devstack-tempest.yaml                    |  16 +-
...e-test_reboot_server_soft-48fa786f38cd94dc.yaml |   6 +
...te-feature-shelve-migrate-fdbd3633abe65c4e.yaml |   6 +
...lt-ssh-user-config-option-1b775af2f468aa5b.yaml |  10 +
.../add-keystone-ep-clients-eeefd0904fbbe151.yaml  |   5 +
...es-api-to-v3-types-client-44b2676f217d78dc.yaml |   6 +
...-disassociate-floating_ip-0b6cfebeef1304b0.yaml |   5 +
.../end-of-support-for-stein-f795b968d83497a9.yaml |  12 +
...nabled_available_features-35f9ac5f253e2ca3.yaml |   6 +
...-placement-client-methods-e35c473e29494928.yaml |  11 +
.../tempest-victoria-release-27000c02edc5a112.yaml |   1 +
requirements.txt                                   |   2 +-
roles/tempest-cleanup/README.rst                   |  28 +
roles/tempest-cleanup/defaults/main.yaml           |   1 +
roles/tempest-cleanup/tasks/dry_run.yaml           |   7 +
roles/tempest-cleanup/tasks/dry_run_checker.py     |  71 ++
roles/tempest-cleanup/tasks/main.yaml              |  37 +-
tempest/api/compute/admin/test_migrations.py       |  23 +-
tempest/api/compute/admin/test_volume.py           | 116 ++++
tempest/api/compute/base.py                        |   4 +
.../security_groups/test_security_group_rules.py   |  12 +-
.../api/compute/servers/test_attach_interfaces.py  |  30 +
tempest/api/compute/servers/test_device_tagging.py |   4 +-
tempest/api/compute/servers/test_novnc.py          |  11 +-
tempest/api/compute/servers/test_server_actions.py |  13 +-
.../servers/test_server_metadata_negative.py       |   8 +-
.../api/compute/servers/test_server_personality.py |   1 +
tempest/api/compute/servers/test_server_rescue.py  |   9 +
tempest/api/compute/volumes/test_attach_volume.py  |   4 +
.../admin/v3/test_application_credentials.py       |   2 +-
tempest/api/identity/base.py                       |   1 +
tempest/api/identity/v3/test_access_rules.py       |  84 +++
.../identity/v3/test_application_credentials.py    |  23 +-
tempest/api/identity/v3/test_ec2_credentials.py    | 113 ++++
tempest/api/image/base.py                          |  10 +
tempest/api/image/v2/admin/test_images.py          |  68 ++
tempest/api/image/v2/test_images.py                |  93 ++-
tempest/api/network/test_floating_ips.py           |   2 +-
.../api/object_storage/test_account_services.py    |  91 ++-
tempest/api/object_storage/test_container_sync.py  |   5 +-
.../test_container_sync_middleware.py              |   2 +
tempest/api/object_storage/test_crossdomain.py     |   5 +-
tempest/api/object_storage/test_healthcheck.py     |   3 -
tempest/api/object_storage/test_object_expiry.py   |   4 +
tempest/api/object_storage/test_object_services.py | 121 ++--
tempest/api/object_storage/test_object_slo.py      |  14 +-
.../volume/admin/test_encrypted_volumes_extend.py  |  35 +
tempest/api/volume/base.py                         |  21 +
tempest/api/volume/test_volumes_extend.py          |  22 +-
tempest/api/volume/test_volumes_negative.py        |   4 +-
tempest/clients.py                                 |   9 +-
tempest/cmd/account_generator.py                   |   2 +-
tempest/cmd/run.py                                 |   7 +-
tempest/common/compute.py                          |  20 +-
tempest/common/utils/__init__.py                   |  15 +
tempest/common/waiters.py                          | 104 ++-
tempest/config.py                                  |  54 +-
.../response/compute/v2_70/interfaces.py           |  37 +
.../api_schema/response/compute/v2_71/servers.py   |   3 +
.../api_schema/response/compute/v2_73/servers.py   |   3 +
tempest/lib/api_schema/response/volume/groups.py   |   9 +-
tempest/lib/auth.py                                |   4 +-
tempest/lib/cli/base.py                            |   8 +-
tempest/lib/cmd/check_uuid.py                      | 139 +++-
tempest/lib/common/rest_client.py                  |   2 +-
tempest/lib/common/thread.py                       |  13 +-
tempest/lib/common/utils/data_utils.py             |   2 +-
tempest/lib/common/utils/test_utils.py             |  17 +-
tempest/lib/services/compute/interfaces_client.py  |   9 +
tempest/lib/services/compute/servers_client.py     |   2 +-
tempest/lib/services/identity/v3/__init__.py       |   9 +-
.../services/identity/v3/access_rules_client.py    |  68 ++
.../services/identity/v3/endpoint_filter_client.py |  54 ++
tempest/lib/services/identity/v3/groups_client.py  |   2 +-
.../services/identity/v3/inherited_roles_client.py |   8 +-
.../lib/services/identity/v3/oauth_token_client.py |   2 +-
.../lib/services/identity/v3/policies_client.py    |  24 +
tempest/lib/services/identity/v3/roles_client.py   |  24 +-
tempest/lib/services/identity/v3/users_client.py   |  27 +
tempest/lib/services/placement/__init__.py         |   4 +-
.../placement/resource_providers_client.py         |  82 +++
tempest/lib/services/volume/v1/volumes_client.py   |   2 +-
tempest/lib/services/volume/v3/types_client.py     |  13 +
tempest/scenario/manager.py                        | 242 ++++---
tempest/scenario/test_aggregates_basic_ops.py      |  19 +-
.../scenario/test_minbw_allocation_placement.py    | 158 ++++-
.../scenario/test_network_advanced_server_ops.py   |  16 +-
tempest/scenario/test_shelve_instance.py           |  44 +-
tempest/scenario/test_stamp_pattern.py             |   2 -
.../identity/v3/test_access_rules_client.py        |  97 +++
.../identity/v3/test_endpoint_filter_client.py     | 116 ++++
.../services/identity/v3/test_endpoints_client.py  |  68 +-
.../services/identity/v3/test_policies_client.py   |  58 ++
.../lib/services/identity/v3/test_users_client.py  |  83 +++
.../services/network/test_floating_ips_client.py   |  22 +-
.../lib/services/network/test_networks_client.py   |  18 +-
.../placement/test_resource_providers_client.py    | 119 ++++
.../lib/services/volume/v3/test_types_client.py    |  13 +
tox.ini                                            |   4 +-
zuul.d/base.yaml                                   |  86 +++
zuul.d/integrated-gate.yaml                        | 441 ++++++++++++
zuul.d/project.yaml                                | 146 ++++
zuul.d/stable-jobs.yaml                            |  23 +
zuul.d/tempest-specific.yaml                       | 113 ++++
114 files changed, 3765 insertions(+), 1274 deletions(-)


Requirements updates
--------------------

diff --git a/requirements.txt b/requirements.txt
index d8738f08c..abc0bceda 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -8 +8 @@ testtools>=2.2.0 # MIT
-paramiko>=2.0.0 # LGPLv2.1+
+paramiko>=2.7.0 # LGPLv2.1+