We are pumped to announce the release of: tempest 18.0.0: OpenStack Integration Testing This release is part of the queens release series. Download the package from: https://pypi.python.org/pypi/tempest For more details, please see below. 18.0.0 ^^^^^^ Prelude ******* When using OVS HW offload feature we need to create Neutron port with a certain capability. This is done by creating Neutron port with binding profile. To be able to test this we need profile capability support in Tempest as well. This release marks the start of Queens release support in Tempest. This release also marks the end of support for Newton in Tempest. New Features ************ * Add group type specs APIs to v3 group_types_client library. * create_or_update_group_type_specs * list_group_type_specs * show_group_type_specs_item * update_group_type_specs_item * delete_group_type_specs_item * A new config option 'port_profile' is added to the section 'network' to specify capabilities of the port. By default this is set to {}. * Add show default quotas API to network quotas_client library. This feature enables the possibility to show default network quotas for a specified project. * Add show encryption specs item API to v2 encryption_types_client library. This feature enables the possibility to show specific encryption specs for a volume type. * Add extension API show quota details to network quotas_client library. This feature enables the possibility to show a quota set for a specified project that includes the quota’s used, limit and reserved counts for per resource * Add update group types API to v3 "group_types_client" library; min_microversion of this API is 3.11. * A function has been added to the common library to allow mounting and unmounting of the config drive consistently. * The Tempest CLI commands have switched from calling testrepository internally to use stestr instead. This means that all of the features and bug fixes from moving to stestr are available to the tempest commands. * Adds a new CLI arg in tempest run, --black-regex, which is a regex to exclude the tests that match it. * A new boolean configuration option "[compute-feature- enabled]/volume_backed_live_migration" has been added. If enabled, tests which validate the behavior of Nova's *volume-backed live migration* feature will be executed. The option defaults to "False". Upgrade Notes ************* * Below config options or feature flags were deprecated for removal. It's time to remove them as all supported stable branches are good to handle them. * "[identity-feature-enabled].forbid_global_implied_dsr" * "[image-feature-enabled].deactivate_image" * "[default].resources_prefix" * config group "orchestration" * "[service_available].heat" * Remove the deprecated decorator "skip_unless_attr" in lib/decorators.py. * The tox ostestr job (normally invoked with "tox -eostestr") has been removed. This was lightly used, and in the near future ostestr will be removed from the tempest requirements file. If you were relying on this functionality you can replicate it by using the venv-tempest tox job. For example, simply running "tox -evenv- tempest -- ostestr" will do the same thing the old ostestr job did. * Tempest CLI commands will no long rely on anything from testr. This means any data in existing testr internals that were being exposed are no longer present. For example things like the .testr directories will be silently ignored. There is a potential incompatibility for existing users who are relying on test results being stored by testr. Anything relying on previous testr behavior will need to be updated to handle stestr. Bug Fixes ********* * Fixes tempest run CLI args mutually exclusive behavior which should not be the case anymore (Bug#1751201). Other Notes *********** * The CLIClient class, when it calls a command line client, uses --os- project-name instead of --os-tenant-name for the project, and passes --os-identity-api-version (default empty). All CLI clients still available in supported releases of OpenStack which are wrapped by the cmd_with_auth() method support those switches. * A new configuration option "[compute-feature- enabled]/volume_multiattach" has been added which defaults to False. Set this to True to enable volume multiattach testing. These tests require that compute API version 2.60 is available and block storage API version 3.44 is available. Note: In the Queens release, the only compute driver that supports volume multiattach is the libvirt driver, and only then when qemu<2.10 or libvirt>=3.10. The only volume backend in Queens that supports volume multiattach is lvm. * OpenStack Releases supported after this release are **Queens**, **Pike**, and **Ocata**. The release under current development of this tag is Rocky, meaning that every Tempest commit is also tested against master during the Rocky cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Rocky (or future release) cloud. Changes in tempest 17.2.0..18.0.0 --------------------------------- 891299e Add releasenotes to mark the start of Queens support, newton EOL 2a69e46 Reno for tempest run CLI new arg and fix ff07eac Add unit test for tempest run --config-file e2c2fb1 allow to config port binding:profile 0aae74a update the home-page url 2998117 Fixing the condition to run tests in tempest run 3c6b15d Add test for tempest run with a whitelist file 1386920 Add a black_regex flag to tempest run f9902ec Fix the -l/--list-tests argument for tempest run 3e97aae Cleanup selection list args in tempest run 5cc9289 Add missing env vars from unit test job 1d473f8 Remove duplicate unit test module 130f6d5 Move share code to _create_server_with_group e377cab Remove leftover call to _create_stestrepository 1c2a802 Enable tempest-full on stable/pike 396940a Fix misuse of assertTrue b6c6d2a Refactor config drive mounting 733bd26 Remove outdated comment in .zuul.yaml a23924f Make queens jobs voting 3bd6e27 Create image using 'owner' parameter c2aa112 Add default value for CONF.auth.admin_domain_name 8a4396e Switch Tempest CLI commands from testrepository to stestr 5cffc98 Fixes test_device_tagging IndexError c9d2c38 Add stable/queens version of the full neutron job b6f67f6 [TrivialFix] Remove unused statement in scenario/manager.py a9487f9 Add simple compute API test to create server with utf8 name 1ac13b2 Add show quota details to network quotas_client library e2556d6 Fix image namespace test logging d88a250 Add test for showing encryption specs item b69463e Add 'doc' tag for bug link on doc theme ef6a047 Fix a broken volume group api ref link 021e66b Deprecate list_servers_on_hypervisor and search_hypervisor in v253 55e4433 Remove redundant py3 integration job f979a04 Refresh server data after fip is assigned 4c24a1a Do not run stage-output again in Tempest d598009 Zuul: Remove project name 51cbf72 Delete identity domain must delete its contents 0ac946c Add test for IP substring filtering b295335 Fix the default value of stage_dir fb0b708 Fix compute api ref links 607cfbf Do not test create_interface_by_fixed_ips if no isolated network 09a66e6 Update api-ref for bug#1602400 d581844 Use '_live_migrate' instead of '_migrate_server_to' in test_live_migration.py 25bf8ce Remove unnecessary alias for service clients 69af2ce Delete base module not applicable definition. 2b4bf4b Delete meaningless and unused parameters 6ec582f Replace curly quotes with straight quotes da80cc9 Add the nova-multiattach job to the check/gate queues dfade25 Add multi-attach swap test 83a1412 Add resize test for multiattach volumes 81fa9b6 Add a few tests for cinder multiattach d5e9282 Fix creating release notes using tox -e venv -- reno new <slug> 85076bd Make tempest-full-py3 voting 270055c Updated from global requirements c4377ef Add a py3 integration test 0787a7c Neutron test cases create,list,update port failed 9433705 Fix broken compute api ref links 6283daa Remove deprecated 'orchestration' config group options b20f7e6 Remove deprecated config options 'resources_prefix' b4fdd68 Handle missing flavor.id in server response 60cb4b2 Restrict test_list_endpoints_for_token to tempest effafa3 Get func.__class__.__name__ if no attribute __name__ 20f4154 Fixed misspelling in test code. 98dc45c Add test case for group type specs APIs 6b79aff Remove unnecessary setup_clients methods 832f885 Updated from global requirements 24f82fe Updated from global requirements 6e695c9 Add show default quotas API to network quotas_client library 93d46ab Add status='ACTIVE' when using changes-since as filter 476f84f Add flavor check after resize server 173b11c Follow the new PTI for document build ba2f479 Rename tox_venvlist 7ac02f3 Remove 'deactivate_image' config options f110a4b Handle creating an image from a server with microversion > 2.35 039740a Decouple AttachVolumeShelveTestJSON from AttachVolumeTestJSON 8f09d27 Do not select inactive l3 agent in L3AgentSchedulerTestJSON 33289a2 Default to using neutron networks_client in get_tenant_network d3db306 CLI tests: Identity v3 (project name, api version) b57c8a4 Do not override stage-dir in tempest post ff3047f Relocate branch conditional for tempest-full job 1bdc3e7 Do not skip entire testcase if floating_ip is not enabled ff9779c Use tenant_network if any in testcases after microversion v2.37 c88e0a1 Add same extra specs as self.flavor_ref for created new flavor d513a7f Add NOTE related to 'on-demand' migration fac4aba Wait for server to be active in test_device_tagging e3b3884 Only run tempest-full on the master branch e72760f Add some missing implemented microversion in doc e4f9dfd Skip testcases when shared nets are found in AutoAllocateNetworkTest 15292a3 Support only volume api_v3 is enabled c4d1ad1 modify params transfer mode in test_snapshot_manage 88cd149 Change containers to list if string 65e4f9b Revert "Use a non admin privileges for retyping a volume" 18bcc9e Ignore a Cinder volume's "updated_at" field. 2aa431a Rename a method to avoid camel case. db5373d Remove BaseV*ImageAdminTest class d13acd1 Use wraps decorator e798bcc Fix the mistake of code indent bfa4b4e Do not use class variable for clients in ContainerSyncTest 515edd4 Document how to write multiline regex with comments cec6384 Remove redundant tempest-full 055b8f5 Fix an outdated comment referencing nova-volumes. 84b51f3 Add some api ref links for network service clients b15803a Fix broken swift ref links 35422ac Modify log message in debug_ssh ca15fe0 Update Octavia plugin ff76e49 Use addClassResourceCleanup in test_container_sync 2119ee1 Updated from global requirements be11739 Add release note about the ostestr tox job d5a0a72 Remove resource_cleanup from test_object_version b134589 Add skip checks for snapshot manage and group snapshots tests 334f313 Add CONF.compute_feature_enabled.volume_backed_live_migration 19355c9 Use addClassResourceCleanup in account service test bcc7ee8 Add negative test for attaching an attached volume to the same instance a2f69f1 Add update groups types API endpoint to volumes v3 library 0ba73df Use addClassResourceCleanup in test_inherits efb12be Remove deprecated 'forbid_global_implied_dsr' feature flag 4778205 Set OS_TEST_LOCK_PATH default value in fake_config 8d1c71e Fix a misspelled addClassResourceCleanp in tempest doc 544b3c8 Remove confusing comment in common/compute.py 7732faa Remove find_stack_traces.py e9241b5 Fix a missing preposition in plugin doc f1e7675 Use addClassResourceCleanup for cleanup in identity users test 90ef8f3 Use addClassResourceCleanup for cleanup in test_regions b36186b Revert "Update volume-status waiter for new cinder attach" aa27003 Add addOnException of unshelve_server when using sharing server 587efc0 Use addClassResourceCleanup for the cleanup of network resources 77ac45a Use addClassResourceCleanup for cleanup of service 880d943 Avoid tox_install.sh for constraints support 8e9e7f7 Initial tempest-full job dbfd7c2 use addClassResourceCleanup in test_roles 6bc422d Add IPv6 support to create_websocket() 069d828 Only use stestr and tempest run in tempest tox jobs 1c008c5 Use addClassResourceCleanup in test_list_projects 2abe00a Refactor volume groups and group snapshots tests 0648215 Remove deprecated skip_unless_attr decorator 26cd92b Updated from global requirements 1635334 Move wait_for_interface_status into _check_interface 7295698 Fix verbose grammar 7f60fa6 Stop using resource_cleanup in identity_v3 tests 917b555 Make shelved_offload_time's help message more specific cae2a75 Use addClassResourceCleanup for cleanup of volume resources b599242 Use service tagging 'decorators' b61314d Add the verify of network_id in _test_create_interface_by_port_id f312f15 Change Conflict's error message f5c1905 Remove a redundant self argument in decorators 9e25a57 Add docs for running a single test 437e31e Use addClassResourceCleanup to cleanup floatingips b3eab67 Fix a pointer to NotFound exception bbbaad6 Fix docs markup consistency 2e0ee04 Add some verify in test_create_ebs_image_and_check_boot cc40091 Don't check console output if not available a23512f Add compute test for ServerGroupAffinityFilter 989f911 Fix directory tree appearance in document aca29ed test_mtu_sized_frames: sanity check connectivity first 019088f Expose bug: Delete identity domain with contents d5dee88 Add Networks client unit tests Diffstat (except docs and test files) ------------------------------------- .stestr.conf | 1 - .testr.conf | 9 - .zuul.yaml | 114 +++++++++- HACKING.rst | 28 +-- README.rst | 19 ++ playbooks/post-tempest.yaml | 19 +- ...-to-v3-group-types-client-10390b52dedede54.yaml | 10 + ...ort-profile-config-option-2610b2fa67027960.yaml | 11 + ...-to-network-quotas-client-3a7c1159af9e56ff.yaml | 6 + ...2-encryption-types-client-290b421cd4bc0c0e.yaml | 6 + ...-to-network-quotas-client-3fffd302cc5d335f.yaml | 7 + ...api-to-group-types-client-09c06ccdf80d5003.yaml | 5 + ...config-volume-multiattach-ea8138dfa4fd308c.yaml | 12 + ...mount-config-drive-to-lib-1a6e912b8afbcc7e.yaml | 4 + ...-list-group-snapshots-api-969d9321002c566c.yaml | 2 +- ...deprecated-config-options-3db535b979fe3509.yaml | 12 + ...kip_unless_attr-decorator-02bde59a00328f5c.yaml | 4 + .../removed-tox-ostestr-8997a93d199c44f3.yaml | 9 + .../start-of-queens-support-fea9051ba1d85fc7.yaml | 12 + .../notes/switch-to-stestr-8c9f834b3f5a55d8.yaml | 13 ++ .../tempest-run-fix-updates-564b41706decbba1.yaml | 9 + .../volume-backed-live-mig-5a38b496ba1ec093.yaml | 7 + requirements.txt | 11 +- roles/process-stackviz/README.rst | 2 +- roles/process-stackviz/defaults/main.yaml | 2 +- roles/run-tempest/README.rst | 12 +- roles/run-tempest/defaults/main.yaml | 2 +- roles/run-tempest/tasks/main.yaml | 2 +- tempest/README.rst | 10 +- .../compute/admin/test_auto_allocate_network.py | 3 +- tempest/api/compute/admin/test_create_server.py | 12 + tempest/api/compute/admin/test_hypervisor.py | 34 +-- .../api/compute/admin/test_hypervisor_negative.py | 62 ++--- tempest/api/compute/admin/test_keypairs_v210.py | 6 +- tempest/api/compute/admin/test_live_migration.py | 18 +- tempest/api/compute/admin/test_migrations.py | 10 + tempest/api/compute/admin/test_networks.py | 2 +- .../compute/admin/test_servers_on_multinodes.py | 61 +++-- tempest/api/compute/admin/test_volume_swap.py | 101 +++++++-- tempest/api/compute/base.py | 62 +++-- .../api/compute/flavors/test_flavors_negative.py | 12 - tempest/api/compute/keypairs/base.py | 20 +- tempest/api/compute/keypairs/test_keypairs.py | 4 +- .../api/compute/keypairs/test_keypairs_negative.py | 7 +- tempest/api/compute/keypairs/test_keypairs_v22.py | 4 +- .../api/compute/servers/test_attach_interfaces.py | 63 ++++-- tempest/api/compute/servers/test_device_tagging.py | 22 +- .../compute/servers/test_list_servers_negative.py | 8 +- tempest/api/compute/servers/test_server_actions.py | 26 ++- .../api/compute/servers/test_server_password.py | 9 +- tempest/api/compute/servers/test_servers.py | 12 + .../api/compute/servers/test_servers_negative.py | 6 + .../servers/test_virtual_interfaces_negative.py | 7 +- tempest/api/compute/volumes/test_attach_volume.py | 198 +++++++++++++++- .../compute/volumes/test_attach_volume_negative.py | 15 ++ tempest/api/identity/admin/v2/test_tokens.py | 11 +- tempest/api/identity/admin/v3/test_credentials.py | 15 +- .../identity/admin/v3/test_domain_configuration.py | 12 - tempest/api/identity/admin/v3/test_domains.py | 21 ++ .../api/identity/admin/v3/test_endpoint_groups.py | 23 +- tempest/api/identity/admin/v3/test_endpoints.py | 49 ++-- .../identity/admin/v3/test_endpoints_negative.py | 11 +- tempest/api/identity/admin/v3/test_inherits.py | 13 +- .../api/identity/admin/v3/test_list_projects.py | 17 +- tempest/api/identity/admin/v3/test_list_users.py | 11 +- tempest/api/identity/admin/v3/test_regions.py | 8 +- tempest/api/identity/admin/v3/test_roles.py | 35 ++- tempest/api/image/base.py | 19 +- tempest/api/image/v2/admin/__init__.py | 0 tempest/api/image/v2/admin/test_images.py | 33 +++ tempest/api/image/v2/test_images.py | 4 - .../v2/test_images_metadefs_namespace_tags.py | 2 +- .../admin/test_external_network_extension.py | 2 - .../api/network/admin/test_l3_agent_scheduler.py | 3 +- tempest/api/network/admin/test_ports.py | 10 +- tempest/api/network/admin/test_quotas.py | 18 ++ tempest/api/network/base.py | 43 ++-- tempest/api/network/test_networks.py | 18 +- tempest/api/network/test_ports.py | 80 +++++++ tempest/api/network/test_routers_negative.py | 2 + tempest/api/object_storage/base.py | 6 +- .../api/object_storage/test_account_services.py | 11 +- tempest/api/object_storage/test_container_sync.py | 13 +- tempest/api/object_storage/test_object_version.py | 20 +- tempest/api/volume/admin/test_group_snapshots.py | 202 +++++++++++++++++ tempest/api/volume/admin/test_group_type_specs.py | 80 +++++++ tempest/api/volume/admin/test_group_types.py | 13 +- tempest/api/volume/admin/test_groups.py | 251 +++------------------ tempest/api/volume/admin/test_snapshot_manage.py | 5 +- tempest/api/volume/admin/test_volume_quotas.py | 9 +- .../api/volume/admin/test_volume_type_access.py | 12 +- tempest/api/volume/admin/test_volume_types.py | 6 + tempest/api/volume/base.py | 118 ++++------ tempest/api/volume/test_volumes_backup.py | 8 +- tempest/api/volume/test_volumes_extend.py | 9 +- tempest/api/volume/test_volumes_list.py | 46 ++-- tempest/clients.py | 4 +- tempest/cmd/account_generator.py | 96 ++++---- tempest/cmd/cleanup.py | 49 ++-- tempest/cmd/cleanup_service.py | 31 --- tempest/cmd/init.py | 35 ++- tempest/cmd/run.py | 212 ++++++----------- tempest/cmd/subunit_describe_calls.py | 19 +- tempest/cmd/verify_tempest_config.py | 1 - tempest/cmd/workspace.py | 16 +- tempest/common/compute.py | 22 +- tempest/common/credentials_factory.py | 2 +- tempest/common/utils/__init__.py | 9 +- tempest/common/waiters.py | 14 +- tempest/config.py | 122 ++-------- .../api_schema/response/compute/v2_45/__init__.py | 0 .../api_schema/response/compute/v2_45/images.py | 32 +++ tempest/lib/cli/base.py | 13 +- tempest/lib/common/fixed_network.py | 7 +- tempest/lib/common/preprov_creds.py | 4 +- tempest/lib/common/utils/linux/remote_client.py | 29 ++- tempest/lib/common/utils/test_utils.py | 6 +- tempest/lib/decorators.py | 24 +- tempest/lib/exceptions.py | 2 +- tempest/lib/services/compute/images_client.py | 10 +- .../lib/services/compute/quota_classes_client.py | 5 +- tempest/lib/services/compute/quotas_client.py | 14 +- tempest/lib/services/compute/servers_client.py | 16 +- .../network/metering_label_rules_client.py | 12 + tempest/lib/services/network/quotas_client.py | 16 ++ .../services/network/service_providers_client.py | 7 +- .../lib/services/object_storage/account_client.py | 2 +- .../services/object_storage/container_client.py | 2 +- .../services/volume/v2/encryption_types_client.py | 8 + .../lib/services/volume/v2/quota_classes_client.py | 10 +- .../lib/services/volume/v3/group_types_client.py | 64 ++++++ tempest/scenario/README.rst | 1 + tempest/scenario/manager.py | 6 +- tempest/scenario/test_minimum_basic.py | 71 +++--- .../scenario/test_network_advanced_server_ops.py | 2 + tempest/scenario/test_network_basic_ops.py | 4 + tempest/scenario/test_server_basic_ops.py | 40 ++-- tempest/scenario/test_volume_boot_pattern.py | 14 +- tempest/scenario/test_volume_migrate_attached.py | 9 +- tempest/test.py | 2 +- .../lib/services/network/test_networks_client.py | 242 ++++++++++++++++++++ .../lib/services/network/test_quotas_client.py | 88 +++++++- .../volume/v2/test_encryption_types_client.py | 17 ++ .../services/volume/v3/test_group_types_client.py | 114 ++++++++++ .../services/object_storage/test_object_client.py | 108 --------- test-requirements.txt | 6 +- tools/check_logs.py | 6 +- tools/find_stack_traces.py | 160 ------------- tools/generate-tempest-plugins-list.py | 4 +- tools/tox_install.sh | 30 --- tox.ini | 58 +++-- 170 files changed, 2807 insertions(+), 1808 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 023148b..76db574 100644 --- a/requirements.txt +++ b/requirements.txt @@ -10,4 +10,4 @@ netaddr>=0.7.18 # BSD -testrepository>=0.0.18 # Apache-2.0/BSD -oslo.concurrency>=3.20.0 # Apache-2.0 -oslo.config>=4.6.0 # Apache-2.0 -oslo.log>=3.30.0 # Apache-2.0 +oslo.concurrency>=3.25.0 # Apache-2.0 +oslo.config>=5.1.0 # Apache-2.0 +oslo.log>=3.36.0 # Apache-2.0 +stestr>=1.0.0 # Apache-2.0 @@ -15 +15 @@ oslo.serialization!=2.19.1,>=2.18.0 # Apache-2.0 -oslo.utils>=3.31.0 # Apache-2.0 +oslo.utils>=3.33.0 # Apache-2.0 @@ -22 +21,0 @@ PrettyTable<0.8,>=0.7.1 # BSD -os-testr>=1.0.0 # Apache-2.0 diff --git a/test-requirements.txt b/test-requirements.txt index 37644d0..e33f207 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -5,4 +4,0 @@ hacking!=0.13.0,<0.14,>=0.12.0 # Apache-2.0 -# needed for doc build -sphinx>=1.6.2 # BSD -openstackdocstheme>=1.17.0 # Apache-2.0 -reno>=2.5.0 # Apache-2.0 @@ -11 +7 @@ coverage!=4.4,>=4.0 # Apache-2.0 -oslotest>=1.10.0 # Apache-2.0 +oslotest>=3.2.0 # Apache-2.0