We jubilantly announce the release of:
tempest 16.1.0: OpenStack Integration Testing
This release is part of the pike release series.
Download the package from:
https://pypi.python.org/pypi/tempest
For more details, please see below.
16.1.0 ^^^^^^
This is an intermediate release during the Pike development cycle to make new functionality available to plugins and other consumers.
New Features ************
* Add a new client to handle the OAUTH token feature from the identity API.
* A new boolean config option "serial_console" is added to the section "compute-feature-enabled". If enabled, tests, which validate the behavior of Nova's *serial console* feature (an alternative to VNC, RDP, SPICE) can be executed.
* Add a new client to handle the domain configuration feature from the identity v3 API.
* A new config option in the network-feature-enabled section, floating_ips, to specify whether floating ips are available in the cloud under test. By default this is set to True.
* Add force detach volume feature API to v2 volumes_client library. This feature enables the possibility to force a volume to detach, and roll back an unsuccessful detach operation after you disconnect the volume.
* Defines the identity v3 OS-EP-FILTER EndPoint Groups API client. This client manages Create, Get, Update, Check, List, and Delete of EndPoint Group.
* Defines the identity v3 OS-EP-FILTER extension API client. This client manages associations between endpoints, projects along with groups.
* The "delete_volume" method of the "VolumesClient" class now has an additional "**params" argument that enables passing additional information in the query string of the HTTP request.
* Add list volume transfers with details API to v2 transfers_client library. This feature enables the possibility to list volume transfers with details.
* A new config option 'manage_snapshot_ref' is added in the volume section, to specify snapshot ref parameter for different storage backend drivers when managing an existing snapshot. By default it is set to fit the LVM driver.
* The "list_endpoints" method of the v3 "EndPointsClient" class now has an additional "**params" argument that enables passing additional information in the query string of the HTTP request.
* Add --save-state option to allow saving state of cloud before tempest run.
* Add validation schema for Nova server diagnostics API
* Add show host API to the volume v2 hosts_client library. This feature enables the possibility to show details for a host.
* Add show snapshot metadata item API to v2 snapshots_client library. This feature enables the possibility to show a snapshot's metadata for a specific key.
* Add show volume metadata item API to v2 volumes_client library. This feature enables the possibility to show a volume's metadata for a specific key.
* Define v3 volumes_client for the volume service as a library interface, allowing other projects to use this module as a stable library without maintenance changes. Add show volume summary API to v3 volumes_client library, min_microversion of this API is 3.12.
* volumes_client(v3)
* Define v3 backups_client for the volume service as a library interface, allowing other projects to use this module as a stable library without maintenance changes. Add update backup API to v3 backups_client library, min_microversion of this API is 3.9.
* backups_client(v3)
* Add groups and group_types clients for the volume service as library. Add tempest tests for create group, delete group, show group, and list group volume APIs.
* Define v2 quota_classes_client for the volume service as library interfaces, allowing other projects to use this module as stable libraries without maintenance changes.
* quota_classes_client(v2)
* A new configuration flag api_v2_admin is introduced in the identity feature flag group to allow for enabling/disabling all identity v2 admin tests. The new flag only applies when the existing api_v2 flag is set to True
* Add additional API endpoints to the identity v2 client token API: - list_endpoints_for_token - check_token_existence
* A new "attr" decorator has been added in the "tempest.lib.decorators" module. For example, use it to tag specific tests, which could be leveraged by test runners to run only a subset of Tempest tests.
* A new "related_bug" decorator has been added to "tempest.lib.decorators". Use it to decorate and tag a test that was added in relation to a launchpad bug report.
* Move base_client from tempest.lib.services.volume.v3 to tempest.lib.services.volume, so if we want to add new interfaces based on a v2 client, we can make that v2 client inherit from volume.base_client.BaseClient to get microversion support, and then to make the new v3 client inherit from the v2 client, thus to avoid the multiple inheritance.
* Define v2.0 "tags_client" for the network service as a library interface, allowing other projects to use this module as a stable library without maintenance changes.
* tags_client(v2.0)
* Pause teardown When pause_teardown flag in tempest.conf is set to True a pdb breakpoint is added to tearDown and tearDownClass methods in test.py. This allows to pause cleaning resources process, so that used resources can be examined. Closer examination of used resources may lead to faster debugging.
* Added tempest workspace remove --name <workspace_name> --rmdir feature to delete the workspace directory as well as entry.
Upgrade Notes *************
* Switch to use Glance v2 APIs in volume tests, by adding the Glance v2 client images_client.
* The Heat API tests have been removed from tempest, they were unmaintained. The future direction of api test for heat is their in- tree Gabbi tests
* The volume config option 'api_v3' default is changed to "True" because the volume v3 API is CURRENT.
* verify_tempest_config command starts using extension_client of cinder v2 API only, because cinder v3 API is current and v2 and v1 are deprecated and v3 extension API is the same as v2. Then we can reuse the v2 client for v3 API also.
Deprecation Notes *****************
* Image APIs in compute are deprecated, Image native APIs are recommended. And Glance v1 APIs are deprecated and v2 APIs are current. Image client compute_images_client and Glance v1 APIs are removed in volume tests.
* The config option "forbid_global_implied_dsr" from the "IdentityFeature" group is now deprecated. This feature flag was introduced to support testing of old OpenStack versions which are not supported anymore.
* Deprecate default value for configuration parameter v3_endpoint_type of identity section in OpenStack Pike and modify the default value to publicURL in OpenStack Q release.
* Deprecate class BaseClient from volume.v3.base_client and move it to volume.base_client. "tempest.lib.services.volume.v3.base_client.BaseClient" (new "tempest.lib.services.volume.base_client.BaseClient")
Bug Fixes *********
* When receiving nullable list as a response body, tempest.lib rest_client module raised an exception without valid json deserialization. A new release fixes this bug.
Changes in tempest 16.0.0..16.1.0 ---------------------------------
87226e3 Add release notes for an intermediate release. 088fb83 Updated from global requirements f4ce417 Add test case for update volume backup 6971907 Add test for showing snapshot metadata item 25dc03c Fix check_service_client_function doc typo b40cb19 Add test for showing volume metadata item 0273652 Move base_client from volume.v3 to volume 29b2230 Switch from oslosphinx to openstackdocstheme 3023e75 Boot server from encrypted volume 3e1f330 Fix import error "No module named six.moves" for plugin sanity job 030dd17 Update volume-status waiter for new cinder attach e280f66 Add response assertions and remove unused variables in volume tests b78e257 Added --rmdir flag to workspace remove ae155b7 Pause resource cleanup 43a4065 Fix image deletion checks after unshelve server d9594f5 Add token related API to Keystone v3-ext/OS-OAUTH1 client 73c3345 Fix no attribute 'urlopen' error in python3 63444d6 Adds **params to v3 list_endpoints 4f678e7 Enhance v3 list_services to test filtering by service type 09f9263 Updated from global requirements 9da5f97 Add Scheduler Stats client unit test d22514a Handle the case that RFP negotiation message arrived early. 31c75ac Add api-ref url for compute list_baremetal_nodes 795ef2d Adding and removing bytes_body param from unit tests d6833c9 Add network tags-ext API tests 0cb4f22 Tests for Nova instance diagnostics (microversion v2.48) aa8c931 Fix skip checks in volume and snapshot manage tests 83b3e55 Add api-ref urls for some volume v2 APIs 9dd7b04 Xenapi: Fix tempest for xenserver device tagging dba121e Fix check_service_client_function mock_args bug 66861a0 Add Quotas client unit tests cc38342 Fix auto allocate network cleanup 9b3f42b Wait for server deletion if validation setup fails 8654e5b Added tests for Nova microversion v2.47 1c79628 Implement tempest client for keystone v2 token APIs a028de1 Save state of cloud before running tempest 3a34df7 create a non-admin token validation test 8aa94cb Add expect_response_code to base network client 9984420 Add Capabilities Client unit tests d1fb3e8 Updated from global requirements 1df940e Fix test_volume_migrate_attached to retype as admin 9305dbe Add Floating IPs client unit tests 3c81b41 Fixed project lists for retrieving tempest plugins e52bee7 Remove unused variables from api tests fb29b9c Add Metering Label Rules client unit tests bf8ada7 Add Metering Labels client unit tests 06d1a30 Add Extensions client unit tests eb058af Add Ports client unit tests b24f904 Verify config support cinder on subpath ebc752b Fix for unmanage and manage snapshot 1177942 Add network tags client 1b1e6df Use urllib2 instead of requests in tempest generate plugin list 0601be1 ported tools/*.py scripts to python3 da810bb Update "test_unmanage_manage_snapshot" test to support some params 63f059e Do not use any() to check whether a list is empty 9d2ebc4 Replace assertGreaterEqual with assertNotEmpty d4ecd0e Update "create_server" to wait until status is 'ACTIVE' a936627 Update server nic handling for LXD style nic names 60dc7a3 Fix html_last_updated_fmt for Python3. 268ccce Replace assertions with more specific ones 9012ba3 Add notes for network and identity in get_service_list b9b0348 Remove unnecessary wait_for_volume_resource_status 00c4487 Fix 4 bytes utf8 char test comment for create images 798b925 Use @test.services instead of skipUnless 6fd9335 Test coverage for network v2 security group rules client 063512e Test coverage for network v2 security groups client dfbefae Use Neutron (if available) to create floating IPs for validation bce1fcf Remove unused variables from scenario tests 9a173b1 Remove TODO comment e88caa8 Fix minor header wordings f9009b4 Use assert(Not)Empty,IsNotNone instead of assert(Not)Equal,GreaterEqual(0, len(.. a4dadd3 Unit test for asserting correct url in list_services 2b96098 Fix DeprecationWarning in test_volume_pools.py b44452e Add server group API schema for microversion 2.13 0e873a2 Updated from global requirements 85e5952 Use create_volume classmethod 7e3047a Fix broken api urls in TypesClient f52c759 Add server param in manager.get_remote_client 1457006 Fix url in list_services 8b876dd Reuse v2 extension client for cinder v3 69a8edc Prevent error in _parse_resp when nullable list 2c0ae15 Fix ssh proxy regression 0343be5 Replace the usage of 'manager' with 'os_primary' cc0c048 Remove the exception description of auth_version in doc dc407d6 Fix for volume quota class test c0f9556 Use data_utils in tempest.lib.common.utils 5870ff1 Replace the usage of 'admin_manager' with 'os_admin' 7ae56bb Stop warning on client_parameters d2c0394 Replace the use of 'os_adm' with 'os_admin' cf999a8 cls.os is deprecated use cls.os_primary 8fdfb1c Check image after unshelve ead5bf6 Remove undefined variable in exception message 18bc991 Test coverage for network v2 subnetpools_client f741d0b Add T115 for admin test path 3312de3 Add option for whether the cloud supports floating ips 0419088 Replace the usage of 'os' with 'os_primary' 3204a0e Separate admin tests from test_routers a40e9b7 Add max_microversion for compute volumes_extensions_client tests a81e135 Test coverage for network v2 subnets_client 406e913 Create deleted server in resource_setup e7eaab4 Updated from global requirements 7a547df Fix volume attach tests failing when using FIP as ssh method 48b235f Replace assertEqual([], items) with assertEmpty(items) b4c0c82 Deprecate default value for v3_endpoint_type 93d7011 Add check after unset_flavor_extra_spec 768b334 Add additional assertions for volume transfer test 529eefa Merge test_snapshot_create_with_volume_in_use 33710b3 Using fixtures instead of deprecated mockpatch module 0c042a5 Reduce server rescue/unrescue times d2a657d Update document theme and section headers 5376ab3 Correct invalid client in ServersNegativeTestMultiTenantJSON 2ae6b90 Merge test_update_server_name_in_stop_state c5043f2 Remove duplicated test_list_servers_by_limits 79a1cbf Add test case for show volume summary 54555c1 Using fixtures instead of deprecated mockpatch module 0fe1c03 Skip test_volume_extend_when_volume_has_snapshot until bug 1687044 is fixed 517a807 Share a server in ServersOnMultiNodesTest 3fd5f0e Remove base_routers module 7e5773c Switch BaseAdminNetworkTest on test_routers_dvr 7e74d98 Move methods related to metering_label b926695 Use a helper to create aggregate 6627827 Separate admin test from RoutersNegativeTest ea2a31a Remove _remove_router_interface_with_port_id() 7835ca1 Fix deprecation warnings ef5d4e9 Fix for implied roles test 5380bdb Move logging extensions list before assert checks 632221b Remove unused _delete_router() 16dd62c Merge test_list_get_two_volume_attachments 5db146f Correct skip condition for migrate_server 909891d Remove the heat tests 0f107bc Change Cinder api_v3 config True on default 0ddf83e Add Tests for Groups Volume APIs - Part 1 b0d15bf Revert "cinder backup force-delete when backup is error" 321fbbc Move network admin test classes under admin path 59fdd2a Fix recent releasenotes typo 2ae8a9e Use min_count to create servers in ListServersNegativeTestJSON 4c79608 Create a server in resource_setup in ImagesOneServerTestJSON bd36c53 Updated from global requirements dcf6b56 Remove redundant api client in volume tests 44c2e0f Add test case for showing host details cdcf53c Volume snapshot backup 6c01199 Create a snapshot from a in-use volume with force=False 8160d31 test.py: stop using aliases for creds manager 3cf1627 Move compute admin test classes under admin path d526240 Log output of lsblk cmd if test_device_tagging failed 1189dd1 Add test for cinder volume extend when volume has snapshot 153df15 Update 'test_volume_upload' test docstring 5092b81 Fix bugs about 'default' domain. 69d58b8 Nova: test live migration with serial console 5d52d3a Keystone v3 extension os-ep-filter api testcases 8e5b324 Add python3-dev(el) to bindep.txt ebd5e41 Skip when force_tenant_isolation=true and no admin credentials 13736ae [Negative] Create a volume from deactivated image 7b0eaf8 Add test case for force detach volume 8cf5f90 Update help text for block_migrate_cinder_iscsi option cb0dd58 Use image native api in volume tests fb89271 Replace oslo_utils.timeutils.isotime 225d9ef Replace deprecated function 1413ba9 Add api_v2_admin flag f1c6825 Add docstring example for get_opt_lists 189b92f Remove deprecated TYPE in rest_client 2b33c1a Use base.create_domain to create domain in testcases 20c9756 Update .mailmap 343ca19 Add test about associate floating_ip to VM d0ed8b0 Identity V3 - Endpoint Groups Client 6be33a4 test_l3_agent_scheduler: remove workaround code for Liberty 9b198e6 Identity V3: create_domain() must return a description field bd391dc Deprecate the forbid_global_implied_dsr cfg option 0a42716 [Negative] Create volume from image with decreasing size 9e01dbb Always provision accounts with auth_version 2e644e6 Fix the position of the dots in write_tests.rst 32b53c3 Move 'test_admin_deactivate_reactivate_image' test under non-admin directory 4a3d587 Remove unused compute networks client in volume tests 3bc7395 Remove duplication of skip_tracker 9604aa0 Fix create, update or delete account metadata method cb5f650 Make delete_volume in volumes_client.py use **params f2f4705 Use base.setup_test_project to create project 963d2c3 Use base.setup_test_tenant to create test tenant 2f4d1d0 Rename 16.0.0 release notes c2618d9 Add support to list volume transfers with detail 94d8577 Identity v3 Domain Configuration Client 7ecc0ec Remove some debug print statements 52c9bf2 Remove usage of deprecated get_ipv6_addr_by_EUI64 function 07acf26 Move InvalidServiceTag 49e2011 Add a page for release 16 to release notes 933d982 Improvement in API Microversion testing doc ecb31f0 Tag test_create_server_invalid_bdm_in_2nd_dict as needing cinder d4d8346 Enhancement of tempest cleanup documentation 19b7ba4 Add write tests docs for using client managers and credentials 0c0f014 Use sequence directly instead of using len() 3b46d27 Move the `attr` decorator from test.py to tempest/lib 403eecf Deprecate Heat/Orchestration configuration options 683abe2 Improve docstring examples c5665a6 Move the `related_bug` decorator from test.py to tempest/lib 75d51a9 Remove leading underscore in _create_test_user 644b01d Add test cases for volume quota class d91094d Remove skip condition when resize is enable 00effc9 Use cliff.lister for tempest workspace command 31169e2 cinder backup force-delete when backup is error 77d3f0f Refactor resource cleanup methods in compute.base 39d8699 Add judgie condition after live-migration with volume 8eb9098 Support testing nova-novncproxy on SSL fd8d315 Add test of remove all security groups in test_server_actions.py a9dca2b TestSecurityGroupsBasicOps: log console from access point instance 0a5d606 Raise exception when get_disks() cannot get 'TYPE' column 3608544 Separate object-storage bulk operation service clients 0d1b63f Add missing IPs validation to test_rebuild_server
Diffstat (except docs and test files) -------------------------------------
.mailmap | 42 +- HACKING.rst | 7 +- bindep.txt | 2 + ...s-to-roles-client-library-178d4a6000dec72d.yaml | 7 + ...rameter-to-volumes-client-ff4f7f12795003a4.yaml | 7 + ...ulate-client-as-a-library-ed76baf25f02c3ca.yaml | 3 + ...ntent-type-without-spaces-b2c9b91b257814f3.yaml | 9 + ...-list-auth-project-client-5905076d914a3943.yaml | 6 + ...-list-glance-api-versions-ec5fc8081fc8a0ae.yaml | 6 + ...to-servers-client-library-088df48f6d81f4be.yaml | 6 + ...ersion-to-identity-client-944cb7396088a575.yaml | 6 + ...-version-to-volume-client-4769dd1bd4ab9c5e.yaml | 6 + ...d-quota-sets-detail-kwarg-74b72183295b3ce7.yaml | 6 + ...tempest-lib-remote-client-adbeb3f42a36910b.yaml | 11 + ...empest-run-combine-option-e94c1049ba8985d5.yaml | 6 + ...o-encryption-types-client-f3093532a0bcf9a1.yaml | 6 + ...-manage-client-as-library-78ab198a1dc1bd41.yaml | 9 + ...create-server-tags-client-8c0042a77e859af6.yaml | 8 + ...e-deactivate_image-config-7a282c471937bbcb.yaml | 6 + ...vr_extra_resources-config-8c319d6dab7f7e5c.yaml | 7 + ...pi-version-config-options-8370b63aea8e14cf.yaml | 10 + ...e-resources-prefix-option-ad490c0a30a0266b.yaml | 10 + ...kip_unless_attr-decorator-450a1ed727494724.yaml | 5 + ...p_unless_config-decorator-64c32d588043ab12.yaml | 5 + ...ated-cinder-api-v1-option-df7d5a54d93db5cf.yaml | 12 + ...precate_client_parameters-cb8d069e62957f7e.yaml | 6 + ...ce-clients-bugfix-1667354-73d2c3c8fedc08bf.yaml | 7 + ...e-call_until_true-of-test-de9c13bc8f969921.yaml | 6 + ..._security_disabled-option-d0ffaeb2e7817707.yaml | 5 + ...ion-config-options-part-2-5cd17b6e0e6cb8a3.yaml | 11 + ...vr_extra_resources-option-e8c441c38eab7ddd.yaml | 8 + ...-identity-reseller-option-4411c7e3951f1094.yaml | 4 + ...-sahara-service-available-44a642aa9c634ab4.yaml | 7 + ...e_enabled.volume_services-c6aa142cc1021297.yaml | 5 + ...6.0.0-use-keystone-v3-api-935860d30ddbb8e9.yaml | 5 + ...0-volume-transfers-client-e5ed3f5464c0cdc0.yaml | 18 + ...s-to-roles-client-library-178d4a6000dec72d.yaml | 7 - ...rameter-to-volumes-client-ff4f7f12795003a4.yaml | 7 - ...te-feature-serial-console-45583c4341e34fc9.yaml | 7 + ...ulate-client-as-a-library-ed76baf25f02c3ca.yaml | 3 - ...ntent-type-without-spaces-b2c9b91b257814f3.yaml | 9 - ...floating-ip-config-option-e5774bf77702ce9f.yaml | 5 + ...to-volumes-client-library-b2071f2954f8e8b1.yaml | 6 + ...ilter-api-endpoint-groups-3518a90bbb731d0f.yaml | 7 + ...-ep-filter-api-extensions-9cfd217fd2c6a61f.yaml | 6 + ...-delete-vol-of-vol-client-1ecde75beb62933c.yaml | 7 + ...-list-auth-project-client-5905076d914a3943.yaml | 6 - ...-list-glance-api-versions-ec5fc8081fc8a0ae.yaml | 6 - ...to-servers-client-library-088df48f6d81f4be.yaml | 6 - ...ersion-to-identity-client-944cb7396088a575.yaml | 6 - ...-version-to-volume-client-4769dd1bd4ab9c5e.yaml | 6 - ...etail-to-transfers-client-80169bf78cf4fa66.yaml | 5 + ...napshot-ref-config-option-67efd04897335b67.yaml | 7 + ...dentity-v3-list-endpoints-958a155be4e17e5b.yaml | 6 + ...d-quota-sets-detail-kwarg-74b72183295b3ce7.yaml | 6 - .../add-save-state-option-5ea67858cbaca969.yaml | 4 + ...nostics-validation-schema-b5a3c55b45aa718a.yaml | 4 + ...t-to-hosts-client-library-c60c4eb49d139480.yaml | 5 + ...pi-to-v2-snapshots-client-bd3cbab3c7f0e0b3.yaml | 6 + ...-api-to-v2-volumes-client-47d59ecd999ca9df.yaml | 6 + ...-api-to-v3-volumes-client-96e7b01abdb5c9c3.yaml | 10 + ...tempest-lib-remote-client-adbeb3f42a36910b.yaml | 11 - ...empest-run-combine-option-e94c1049ba8985d5.yaml | 6 - ...-api-to-v3-backups-client-e8465b2b66617dc0.yaml | 10 + ...o-encryption-types-client-f3093532a0bcf9a1.yaml | 6 - ...-manage-client-as-library-78ab198a1dc1bd41.yaml | 9 - ...a-class-client-as-library-c4c2b22c36ff807e.yaml | 8 + .../notes/api_v2_admin_flag-dea5ca9bc2ce63bc.yaml | 7 + ...create-server-tags-client-8c0042a77e859af6.yaml | 8 - ...forbid_global_implied_dsr-e64cfa66e6e3ded5.yaml | 5 + ...e-deactivate_image-config-7a282c471937bbcb.yaml | 6 - ...alue-for-v3_endpoint_type-fb9e47c5ba1c719d.yaml | 6 + ...vr_extra_resources-config-8c319d6dab7f7e5c.yaml | 7 - ...pi-version-config-options-8370b63aea8e14cf.yaml | 10 - ...e-resources-prefix-option-ad490c0a30a0266b.yaml | 10 - ...kip_unless_attr-decorator-450a1ed727494724.yaml | 5 - ...p_unless_config-decorator-64c32d588043ab12.yaml | 5 - ...ated-cinder-api-v1-option-df7d5a54d93db5cf.yaml | 12 - ...precate_client_parameters-cb8d069e62957f7e.yaml | 6 - ...ce-clients-bugfix-1667354-73d2c3c8fedc08bf.yaml | 7 - .../identity-token-client-8aaef74b1d61090a.yaml | 6 + ...intermediate-pike-release-2ce492432ff8f012.yaml | 4 + ...ove-attr-decorator-to-lib-a1e80c42ba9c5392.yaml | 6 + ...ated_bug-decorator-to-lib-dbfd5c543bbb2805.yaml | 6 + ...-v3-base_client-to-volume-1edbz0f207c3b283.yaml | 15 + .../notes/network-tag-client-f4614029af7927f0.yaml | 8 + .../notes/pause_teardown-45c9d60ffa889f7f.yaml | 9 + ...e-resp-when-nullable-list-9898cd0f22180986.yaml | 6 + ...e-call_until_true-of-test-de9c13bc8f969921.yaml | 6 - ..._security_disabled-option-d0ffaeb2e7817707.yaml | 5 - ...ion-config-options-part-2-5cd17b6e0e6cb8a3.yaml | 11 - ...vr_extra_resources-option-e8c441c38eab7ddd.yaml | 8 - ...-identity-reseller-option-4411c7e3951f1094.yaml | 4 - ...-sahara-service-available-44a642aa9c634ab4.yaml | 7 - ...e_enabled.volume_services-c6aa142cc1021297.yaml | 5 - ...cinder-api-v3-option-true-1b3e61e3129b7c00.yaml | 5 + ...-delete-directory-feature-74d6d157a5a05561.yaml | 5 + ...for-verify_tempest_config-2bf3d817b0070064.yaml | 6 + .../use-keystone-v3-api-935860d30ddbb8e9.yaml | 5 - .../volume-transfers-client-e5ed3f5464c0cdc0.yaml | 18 - releasenotes/source/conf.py | 13 +- releasenotes/source/index.rst | 3 +- releasenotes/source/v16.0.0.rst | 6 + requirements.txt | 6 +- setup.cfg | 6 +- tempest/api/compute/admin/test_agents.py | 8 +- tempest/api/compute/admin/test_aggregates.py | 68 +- .../api/compute/admin/test_aggregates_negative.py | 35 +- .../compute/admin/test_auto_allocate_network.py | 22 +- .../api/compute/admin/test_availability_zone.py | 4 +- .../admin/test_availability_zone_negative.py | 3 +- tempest/api/compute/admin/test_create_server.py | 109 ++ tempest/api/compute/admin/test_delete_server.py | 52 + tempest/api/compute/admin/test_fixed_ips.py | 2 +- .../api/compute/admin/test_fixed_ips_negative.py | 12 +- tempest/api/compute/admin/test_flavors.py | 2 +- tempest/api/compute/admin/test_flavors_access.py | 2 +- .../compute/admin/test_flavors_access_negative.py | 10 +- .../api/compute/admin/test_flavors_extra_specs.py | 7 +- .../admin/test_flavors_extra_specs_negative.py | 14 +- .../api/compute/admin/test_floating_ips_bulk.py | 4 +- tempest/api/compute/admin/test_hosts.py | 12 +- tempest/api/compute/admin/test_hosts_negative.py | 33 +- tempest/api/compute/admin/test_hypervisor.py | 12 +- .../api/compute/admin/test_hypervisor_negative.py | 33 +- .../compute/admin/test_instance_usage_audit_log.py | 2 +- .../test_instance_usage_audit_log_negative.py | 7 +- tempest/api/compute/admin/test_keypairs_v210.py | 4 +- .../admin/test_live_block_migration_negative.py | 49 + tempest/api/compute/admin/test_live_migration.py | 89 +- tempest/api/compute/admin/test_migrations.py | 2 +- tempest/api/compute/admin/test_networks.py | 4 +- tempest/api/compute/admin/test_quotas.py | 4 +- tempest/api/compute/admin/test_quotas_negative.py | 16 +- .../admin/test_security_group_default_rules.py | 4 +- tempest/api/compute/admin/test_security_groups.py | 2 +- .../api/compute/admin/test_server_diagnostics.py | 76 ++ .../admin/test_server_diagnostics_negative.py | 40 + tempest/api/compute/admin/test_servers.py | 20 +- tempest/api/compute/admin/test_servers_negative.py | 31 +- .../compute/admin/test_servers_on_multinodes.py | 32 +- tempest/api/compute/admin/test_services.py | 6 +- .../api/compute/admin/test_services_negative.py | 15 +- .../api/compute/admin/test_simple_tenant_usage.py | 4 +- .../admin/test_simple_tenant_usage_negative.py | 11 +- tempest/api/compute/admin/test_volumes_negative.py | 7 +- tempest/api/compute/base.py | 139 +-- tempest/api/compute/flavors/test_flavors.py | 17 +- .../api/compute/flavors/test_flavors_negative.py | 18 +- .../floating_ips/test_floating_ips_actions.py | 6 + .../test_floating_ips_actions_negative.py | 42 +- .../compute/floating_ips/test_list_floating_ips.py | 10 +- .../test_list_floating_ips_negative.py | 8 +- tempest/api/compute/images/test_image_metadata.py | 4 +- .../compute/images/test_image_metadata_negative.py | 13 +- tempest/api/compute/images/test_images_negative.py | 21 +- .../api/compute/images/test_images_oneserver.py | 21 +- .../images/test_images_oneserver_negative.py | 11 +- .../api/compute/images/test_list_image_filters.py | 103 +- .../images/test_list_image_filters_negative.py | 3 +- tempest/api/compute/images/test_list_images.py | 8 +- .../api/compute/keypairs/test_keypairs_negative.py | 17 +- tempest/api/compute/limits/test_absolute_limits.py | 2 +- .../limits/test_absolute_limits_negative.py | 3 +- .../security_groups/test_security_group_rules.py | 10 +- .../test_security_group_rules_negative.py | 16 +- .../security_groups/test_security_groups.py | 2 +- .../test_security_groups_negative.py | 22 +- .../api/compute/servers/test_attach_interfaces.py | 12 +- .../api/compute/servers/test_availability_zone.py | 2 +- tempest/api/compute/servers/test_create_server.py | 101 +- tempest/api/compute/servers/test_delete_server.py | 31 - tempest/api/compute/servers/test_device_tagging.py | 32 +- tempest/api/compute/servers/test_disk_config.py | 2 +- .../servers/test_instance_actions_negative.py | 5 +- .../compute/servers/test_list_server_filters.py | 2 +- .../compute/servers/test_list_servers_negative.py | 60 +- .../api/compute/servers/test_multiple_create.py | 2 +- .../servers/test_multiple_create_negative.py | 11 +- tempest/api/compute/servers/test_novnc.py | 4 +- tempest/api/compute/servers/test_server_actions.py | 38 +- .../api/compute/servers/test_server_addresses.py | 8 +- .../servers/test_server_addresses_negative.py | 4 +- .../servers/test_server_metadata_negative.py | 25 +- tempest/api/compute/servers/test_server_rescue.py | 87 +- .../compute/servers/test_server_rescue_negative.py | 12 +- tempest/api/compute/servers/test_servers.py | 17 +- .../api/compute/servers/test_servers_negative.py | 118 +- .../api/compute/servers/test_virtual_interfaces.py | 2 +- .../servers/test_virtual_interfaces_negative.py | 2 +- tempest/api/compute/test_extensions.py | 12 +- .../compute/test_live_block_migration_negative.py | 50 - tempest/api/compute/test_networks.py | 2 +- tempest/api/compute/test_tenant_networks.py | 2 +- tempest/api/compute/test_versions.py | 5 +- tempest/api/compute/volumes/test_attach_volume.py | 53 +- .../compute/volumes/test_attach_volume_negative.py | 5 +- .../api/compute/volumes/test_volume_snapshots.py | 5 + tempest/api/compute/volumes/test_volumes_get.py | 21 +- tempest/api/compute/volumes/test_volumes_list.py | 5 + .../api/compute/volumes/test_volumes_negative.py | 22 +- tempest/api/identity/admin/v2/test_endpoints.py | 2 +- tempest/api/identity/admin/v2/test_roles.py | 6 +- .../api/identity/admin/v2/test_roles_negative.py | 47 +- tempest/api/identity/admin/v2/test_services.py | 3 +- .../api/identity/admin/v2/test_tenant_negative.py | 48 +- tempest/api/identity/admin/v2/test_tenants.py | 62 +- tempest/api/identity/admin/v2/test_tokens.py | 68 +- .../api/identity/admin/v2/test_tokens_negative.py | 38 + tempest/api/identity/admin/v2/test_users.py | 73 +- .../api/identity/admin/v2/test_users_negative.py | 43 +- tempest/api/identity/admin/v3/test_credentials.py | 5 +- .../identity/admin/v3/test_default_project_id.py | 9 +- .../identity/admin/v3/test_domain_configuration.py | 184 ++++ tempest/api/identity/admin/v3/test_domains.py | 18 +- .../api/identity/admin/v3/test_domains_negative.py | 17 +- .../api/identity/admin/v3/test_endpoint_groups.py | 157 +++ tempest/api/identity/admin/v3/test_endpoints.py | 99 +- .../identity/admin/v3/test_endpoints_negative.py | 9 +- tempest/api/identity/admin/v3/test_groups.py | 16 +- tempest/api/identity/admin/v3/test_inherits.py | 20 +- tempest/api/identity/admin/v3/test_list_users.py | 2 +- .../api/identity/admin/v3/test_oauth_consumers.py | 2 +- tempest/api/identity/admin/v3/test_policies.py | 5 +- tempest/api/identity/admin/v3/test_projects.py | 54 +- .../identity/admin/v3/test_projects_negative.py | 22 +- tempest/api/identity/admin/v3/test_regions.py | 5 +- tempest/api/identity/admin/v3/test_roles.py | 21 +- tempest/api/identity/admin/v3/test_services.py | 16 +- tempest/api/identity/admin/v3/test_tokens.py | 49 +- tempest/api/identity/admin/v3/test_trusts.py | 30 +- tempest/api/identity/admin/v3/test_users.py | 25 +- .../api/identity/admin/v3/test_users_negative.py | 10 +- tempest/api/identity/base.py | 142 +-- tempest/api/identity/v2/test_api_discovery.py | 7 +- tempest/api/identity/v2/test_ec2_credentials.py | 2 +- tempest/api/identity/v2/test_tenants.py | 13 +- tempest/api/identity/v2/test_tokens.py | 2 +- tempest/api/identity/v2/test_users.py | 2 +- tempest/api/identity/v3/test_api_discovery.py | 9 +- tempest/api/identity/v3/test_projects.py | 19 +- tempest/api/identity/v3/test_tokens.py | 35 +- tempest/api/identity/v3/test_users.py | 2 +- tempest/api/image/admin/__init__.py | 0 tempest/api/image/admin/v2/__init__.py | 0 tempest/api/image/admin/v2/test_images.py | 58 - tempest/api/image/base.py | 31 +- tempest/api/image/v1/test_image_members.py | 2 +- .../api/image/v1/test_image_members_negative.py | 9 +- tempest/api/image/v1/test_images.py | 3 +- tempest/api/image/v1/test_images_negative.py | 15 +- tempest/api/image/v2/test_images.py | 46 +- .../api/image/v2/test_images_member_negative.py | 5 +- .../v2/test_images_metadefs_namespace_tags.py | 2 +- tempest/api/image/v2/test_images_negative.py | 17 +- tempest/api/image/v2/test_images_tags_negative.py | 5 +- tempest/api/image/v2/test_versions.py | 3 +- tempest/api/network/admin/test_agent_management.py | 2 +- .../admin/test_external_network_extension.py | 7 + .../admin/test_external_networks_negative.py | 3 +- .../admin/test_floating_ips_admin_actions.py | 4 +- .../api/network/admin/test_l3_agent_scheduler.py | 16 +- .../api/network/admin/test_metering_extensions.py | 168 +++ tempest/api/network/admin/test_negative_quotas.py | 2 +- tempest/api/network/admin/test_ports.py | 99 ++ tempest/api/network/admin/test_routers.py | 231 ++++ tempest/api/network/admin/test_routers_dvr.py | 4 +- tempest/api/network/admin/test_routers_negative.py | 63 ++ tempest/api/network/base.py | 70 +- tempest/api/network/base_routers.py | 61 -- tempest/api/network/test_dhcp_ipv6.py | 17 +- tempest/api/network/test_extensions.py | 2 +- tempest/api/network/test_floating_ips.py | 6 +- tempest/api/network/test_floating_ips_negative.py | 8 +- tempest/api/network/test_metering_extensions.py | 146 --- tempest/api/network/test_networks.py | 18 +- tempest/api/network/test_networks_negative.py | 21 +- tempest/api/network/test_ports.py | 90 +- tempest/api/network/test_routers.py | 174 +-- tempest/api/network/test_routers_negative.py | 59 +- tempest/api/network/test_security_groups.py | 8 +- .../api/network/test_security_groups_negative.py | 28 +- tempest/api/network/test_subnetpools_extensions.py | 2 +- tempest/api/network/test_tags.py | 202 ++++ tempest/api/network/test_versions.py | 3 +- tempest/api/object_storage/base.py | 12 +- tempest/api/object_storage/test_account_bulk.py | 13 +- tempest/api/object_storage/test_account_quotas.py | 17 +- .../object_storage/test_account_quotas_negative.py | 24 +- .../api/object_storage/test_account_services.py | 55 +- .../test_account_services_negative.py | 3 +- tempest/api/object_storage/test_container_acl.py | 14 +- .../object_storage/test_container_acl_negative.py | 31 +- .../api/object_storage/test_container_quotas.py | 6 +- .../api/object_storage/test_container_services.py | 9 +- .../test_container_services_negative.py | 21 +- .../api/object_storage/test_container_staticweb.py | 5 +- tempest/api/object_storage/test_container_sync.py | 6 +- .../test_container_sync_middleware.py | 2 +- tempest/api/object_storage/test_object_expiry.py | 4 +- tempest/api/object_storage/test_object_formpost.py | 6 +- .../test_object_formpost_negative.py | 10 +- tempest/api/object_storage/test_object_services.py | 33 +- tempest/api/object_storage/test_object_slo.py | 4 +- tempest/api/object_storage/test_object_temp_url.py | 14 +- .../test_object_temp_url_negative.py | 9 +- tempest/api/orchestration/__init__.py | 0 tempest/api/orchestration/base.py | 169 --- tempest/api/orchestration/stacks/__init__.py | 0 .../stacks/templates/cinder_basic.yaml | 33 - .../templates/cinder_basic_delete_retain.yaml | 34 - .../stacks/templates/neutron_basic.yaml | 72 -- .../stacks/templates/non_empty_stack.yaml | 36 - .../stacks/templates/nova_keypair.json | 48 - .../stacks/templates/nova_keypair.yaml | 43 - .../stacks/templates/random_string.yaml | 18 - .../stacks/templates/swift_basic.yaml | 23 - .../api/orchestration/stacks/test_environment.py | 92 -- tempest/api/orchestration/stacks/test_limits.py | 49 - .../orchestration/stacks/test_neutron_resources.py | 196 ---- .../orchestration/stacks/test_non_empty_stack.py | 150 --- .../stacks/test_nova_keypair_resources.py | 92 -- .../orchestration/stacks/test_resource_types.py | 50 - tempest/api/orchestration/stacks/test_soft_conf.py | 170 --- tempest/api/orchestration/stacks/test_stacks.py | 60 - .../orchestration/stacks/test_swift_resources.py | 120 -- tempest/api/orchestration/stacks/test_templates.py | 61 -- .../stacks/test_templates_negative.py | 61 -- tempest/api/orchestration/stacks/test_volumes.py | 116 -- tempest/api/volume/admin/test_groups.py | 109 ++ tempest/api/volume/admin/test_snapshot_manage.py | 57 +- tempest/api/volume/admin/test_volume_hosts.py | 39 +- tempest/api/volume/admin/test_volume_manage.py | 6 +- tempest/api/volume/admin/test_volume_pools.py | 2 +- .../api/volume/admin/test_volume_quota_classes.py | 101 ++ tempest/api/volume/admin/test_volume_quotas.py | 14 +- .../volume/admin/test_volume_quotas_negative.py | 7 +- tempest/api/volume/admin/test_volume_services.py | 8 +- .../admin/test_volume_snapshot_quotas_negative.py | 7 +- tempest/api/volume/admin/test_volume_types.py | 2 - .../test_volume_types_extra_specs_negative.py | 23 +- .../api/volume/admin/test_volume_types_negative.py | 11 +- tempest/api/volume/admin/test_volumes_actions.py | 38 + tempest/api/volume/api_microversion_fixture.py | 2 +- tempest/api/volume/base.py | 89 +- tempest/api/volume/test_availability_zone.py | 2 +- tempest/api/volume/test_extensions.py | 2 +- tempest/api/volume/test_snapshot_metadata.py | 15 +- tempest/api/volume/test_versions.py | 3 +- tempest/api/volume/test_volume_absolute_limits.py | 5 +- tempest/api/volume/test_volume_metadata.py | 11 +- tempest/api/volume/test_volume_transfers.py | 37 +- tempest/api/volume/test_volumes_actions.py | 31 +- tempest/api/volume/test_volumes_backup.py | 41 +- tempest/api/volume/test_volumes_clone.py | 6 +- tempest/api/volume/test_volumes_clone_negative.py | 3 +- tempest/api/volume/test_volumes_extend.py | 22 + tempest/api/volume/test_volumes_get.py | 23 +- tempest/api/volume/test_volumes_list.py | 7 +- tempest/api/volume/test_volumes_negative.py | 127 ++- tempest/api/volume/test_volumes_snapshots.py | 39 +- .../api/volume/test_volumes_snapshots_negative.py | 13 +- tempest/clients.py | 55 +- tempest/cmd/cleanup.py | 82 +- tempest/cmd/cleanup_service.py | 2 +- tempest/cmd/run.py | 32 + tempest/cmd/verify_tempest_config.py | 15 +- tempest/cmd/workspace.py | 163 +-- tempest/common/compute.py | 55 +- tempest/common/preprov_creds.py | 2 +- tempest/common/utils/linux/remote_client.py | 12 +- tempest/common/validation_resources.py | 21 +- tempest/common/waiters.py | 14 +- tempest/config.py | 94 +- tempest/exceptions.py | 4 - tempest/hacking/checks.py | 22 + .../api_schema/response/compute/v2_1/servers.py | 7 + .../api_schema/response/compute/v2_13/__init__.py | 0 .../api_schema/response/compute/v2_13/servers.py | 34 + .../api_schema/response/compute/v2_47/__init__.py | 0 .../api_schema/response/compute/v2_47/servers.py | 39 + .../api_schema/response/compute/v2_48/__init__.py | 0 .../api_schema/response/compute/v2_48/servers.py | 115 ++ .../api_schema/response/compute/v2_6/__init__.py | 0 .../api_schema/response/compute/v2_6/servers.py | 48 + .../api_schema/response/compute/v2_9/servers.py | 2 +- tempest/lib/auth.py | 10 +- tempest/lib/cli/output_parser.py | 2 +- tempest/lib/cmd/skip_tracker.py | 6 +- tempest/lib/common/rest_client.py | 13 +- tempest/lib/common/ssh.py | 8 +- tempest/lib/common/utils/linux/remote_client.py | 55 +- tempest/lib/decorators.py | 43 + tempest/lib/exceptions.py | 2 +- tempest/lib/services/clients.py | 77 +- .../lib/services/compute/baremetal_nodes_client.py | 13 +- .../lib/services/compute/server_groups_client.py | 9 + tempest/lib/services/compute/servers_client.py | 38 +- .../lib/services/identity/v2/identity_client.py | 22 + tempest/lib/services/identity/v3/__init__.py | 19 +- .../identity/v3/domain_configuration_client.py | 188 ++++ .../services/identity/v3/endpoint_filter_client.py | 68 ++ .../services/identity/v3/endpoint_groups_client.py | 78 ++ .../lib/services/identity/v3/endpoints_client.py | 15 +- .../lib/services/identity/v3/oauth_token_client.py | 236 ++++ .../lib/services/identity/v3/services_client.py | 2 +- tempest/lib/services/image/v1/images_client.py | 2 +- tempest/lib/services/network/__init__.py | 9 +- tempest/lib/services/network/base.py | 10 +- tempest/lib/services/network/tags_client.py | 84 ++ tempest/lib/services/volume/base_client.py | 45 + tempest/lib/services/volume/v2/__init__.py | 5 +- tempest/lib/services/volume/v2/backups_client.py | 3 +- .../services/volume/v2/encryption_types_client.py | 12 +- tempest/lib/services/volume/v2/hosts_client.py | 14 +- .../lib/services/volume/v2/quota_classes_client.py | 49 + tempest/lib/services/volume/v2/snapshots_client.py | 15 +- tempest/lib/services/volume/v2/transfers_client.py | 5 +- tempest/lib/services/volume/v2/types_client.py | 6 +- tempest/lib/services/volume/v2/volumes_client.py | 71 +- tempest/lib/services/volume/v3/__init__.py | 7 +- tempest/lib/services/volume/v3/backups_client.py | 37 + tempest/lib/services/volume/v3/base_client.py | 33 +- .../lib/services/volume/v3/group_types_client.py | 48 + tempest/lib/services/volume/v3/groups_client.py | 97 ++ tempest/lib/services/volume/v3/messages_client.py | 3 +- tempest/lib/services/volume/v3/versions_client.py | 3 +- tempest/lib/services/volume/v3/volumes_client.py | 40 + tempest/scenario/manager.py | 100 +- tempest/scenario/test_aggregates_basic_ops.py | 8 +- tempest/scenario/test_encrypted_cinder_volumes.py | 13 +- tempest/scenario/test_minimum_basic.py | 8 +- .../scenario/test_network_advanced_server_ops.py | 23 +- tempest/scenario/test_network_basic_ops.py | 73 +- tempest/scenario/test_network_v6.py | 22 +- tempest/scenario/test_object_storage_basic_ops.py | 2 +- tempest/scenario/test_security_groups_basic_ops.py | 51 +- tempest/scenario/test_server_advanced_ops.py | 9 +- tempest/scenario/test_server_basic_ops.py | 11 +- tempest/scenario/test_server_multinode.py | 20 +- tempest/scenario/test_shelve_instance.py | 4 +- tempest/scenario/test_snapshot_pattern.py | 2 +- tempest/scenario/test_stamp_pattern.py | 2 +- tempest/scenario/test_volume_boot_pattern.py | 27 +- tempest/scenario/test_volume_migrate_attached.py | 7 +- tempest/services/object_storage/__init__.py | 6 +- tempest/services/object_storage/account_client.py | 96 +- .../object_storage/bulk_middleware_client.py | 62 ++ tempest/test.py | 127 ++- tempest/test_discover/plugins.py | 152 +-- .../lib/services/compute/test_flavors_client.py | 4 +- .../services/compute/test_floating_ips_client.py | 6 +- .../lib/services/compute/test_images_client.py | 11 +- .../compute/test_security_groups_client.py | 6 +- .../services/compute/test_server_groups_client.py | 30 +- .../lib/services/compute/test_servers_client.py | 266 ++--- .../lib/services/compute/test_snapshots_client.py | 6 +- .../lib/services/compute/test_versions_client.py | 4 +- .../lib/services/compute/test_volumes_client.py | 6 +- .../services/identity/v2/test_identity_client.py | 55 + .../v3/test_domain_configuration_client.py | 217 ++++ .../identity/v3/test_endpoint_filter_client.py | 165 +++ .../identity/v3/test_endpoint_groups_client.py | 162 +++ .../services/identity/v3/test_endpoints_client.py | 19 +- .../identity/v3/test_oauth_token_client.py | 215 ++++ .../services/identity/v3/test_services_client.py | 11 +- .../services/network/test_base_network_client.py | 96 ++ .../lib/services/network/test_extensions_client.py | 201 ++++ .../services/network/test_floating_ips_client.py | 145 +++ .../network/test_metering_label_rules_client.py | 110 ++ .../network/test_metering_labels_client.py | 107 ++ .../lib/services/network/test_ports_client.py | 198 ++++ .../lib/services/network/test_quotas_client.py | 99 ++ .../network/test_security_group_rules_client.py | 138 +++ .../network/test_security_groups_client.py | 174 +++ .../services/network/test_subnetpools_client.py | 163 +++ .../lib/services/network/test_subnets_client.py | 250 +++++ .../services/volume/v2/test_capabilities_client.py | 77 ++ .../lib/services/volume/v2/test_hosts_client.py | 97 ++ .../volume/v2/test_quota_classes_client.py | 71 ++ .../volume/v2/test_scheduler_stats_client.py | 83 ++ .../services/volume/v2/test_snapshots_client.py | 21 + .../services/volume/v2/test_transfers_client.py | 61 ++ .../lib/services/volume/v2/test_volumes_client.py | 73 ++ .../lib/services/volume/v3/test_backups_client.py | 50 + .../services/volume/v3/test_group_types_client.py | 57 + .../lib/services/volume/v3/test_groups_client.py | 136 +++ .../lib/services/volume/v3/test_volumes_client.py | 48 + test-requirements.txt | 8 +- tools/check_logs.py | 3 +- tools/find_stack_traces.py | 4 +- tools/generate-tempest-plugins-list.py | 34 +- tools/skip_tracker.py | 125 +-- 524 files changed, 12109 insertions(+), 6938 deletions(-)
Requirements updates --------------------
diff --git a/requirements.txt b/requirements.txt index 14c42b3..259a4cf 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5 +5 @@ pbr!=2.1.0,>=2.0.0 # Apache-2.0 -cliff>=2.3.0 # Apache-2.0 +cliff>=2.6.0 # Apache-2.0 @@ -12 +12 @@ oslo.concurrency>=3.8.0 # Apache-2.0 -oslo.config>=3.22.0 # Apache-2.0 +oslo.config!=4.3.0,!=4.4.0,>=4.0.0 # Apache-2.0 @@ -23 +23 @@ os-testr>=0.8.0 # Apache-2.0 -urllib3>=1.15.1 # MIT +urllib3>=1.21.1 # MIT diff --git a/test-requirements.txt b/test-requirements.txt index 13950bd..6a5ea03 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -6,3 +6,3 @@ hacking!=0.13.0,<0.14,>=0.12.0 # Apache-2.0 -sphinx>=1.5.1 # BSD -oslosphinx>=4.7.0 # Apache-2.0 -reno>=1.8.0 # Apache-2.0 +sphinx>=1.6.2 # BSD +openstackdocstheme>=1.11.0 # Apache-2.0 +reno!=2.3.1,>=1.8.0 # Apache-2.0 @@ -10 +10 @@ mock>=2.0 # BSD -coverage>=4.0 # Apache-2.0 +coverage!=4.4,>=4.0 # Apache-2.0