We are tickled pink to announce the release of: patrole 0.5.0: Patrole is a tool for verifying that Role-Based Access Control is being enforced across OpenStack deployments. This release is part of the stein release series. The source is available from: https://git.openstack.org/cgit/openstack/patrole Download the package from: https://tarballs.openstack.org/patrole/ Please report issues through launchpad: https://storyboard.openstack.org/#!/project/openstack/patrole For more details, please see below. 0.5.0 ^^^^^ Prelude ******* This release is to tag the Patrole for OpenStack Stein release. After this release, Patrole will support below OpenStack Releases: * Stein * Rocky * Queens * Pike Current development of Patrole is for OpenStack Train development cycle. Every Patrole commit is also tested against master during the Train cycle. However, this does not necessarily mean that using Patrole as of this tag will work against a Train (or future release) cloud. New Features ************ * The exception class "RbacMalformedException" has been broken up into the following discrete exceptions: * "RbacMissingAttributeResponseBody" - incomplete means that the response body (for show or list) is missing certain attributes * "RbacPartialResponseBody" - partial means that a list response only returned a subset of the possible results available. * "RbacEmptyResponseBody" - empty means that the show or list response body is entirely empty Each of the exception classes above deals with a different type of failure related to a soft authorization failure. This means that, rather than a 403 error code getting returned by the server, the response body is incomplete in some way. * Add new exception called "RbacOverrideRoleException". Used for safeguarding against false positives that might occur when the expected exception isn't raised inside the "override_role" context. Specifically, when: * "override_role" isn't called * an exception is raised before "override_role" context * an exception is raised after "override_role" context * Supporting the role inference rules API gives Patrole an ability of testing role chains, when one role implies the second which can also imply the third: "admin" implies "member" implies "reader" Now in a case of testing against an "admin" role ("[patole] rbac_test_roles" = "admin") the "rbac_rule_validation.action" calls the "rbac_utils.get_all_needed_roles" function to extend the roles and validates a policy rule against the full list of possible roles: ["admin", "member", "reader"] Here is few examples: ["admin"] >> ["admin", "member", "reader"] ["member"] >> ["member", "reader"] ["reader"] >> ["reader"] ["custom_role"] >> ["custom_role"] ["custom_role", "member"] >> ["custom_role", "member", "reader"] * We have replaced CONF.patrole.rbac_test_role with CONF.patrole.rbac_test_roles, where instead of single role we can specify list of roles to be assigned to test user. This way we may run rbac tests for scenarios that requires user to have more that a single role. * In order to implement the tests for plugins which do not maintain the "policy.json" with full list of the policy rules and provide policy file with only their own policy rules, the Patrole should be able to load and merge multiple policy files for any of the services. * Discovery all policy files for each of the services. The updated "discover_policy_files" function picks all candidate paths found out of the potential paths in the "[patrole].custom_policy_files" config option. Using "glob.glob()" function makes it possible to use the patterns like '*.json' to discover the policy files. * Loading and merging a data from multiple policy files. Patrole loads a data from each of the discovered policy files for a service and merge the data from all files. * In order to test the list actions which doesn't have its own policy, implemented the "override_role_and_validate_list" function. The function has two modes: * Validating the number of the resources in a "ResponseBody" before calling the "override_role" and after. # make sure at least one resource is available self.ntp_client.create_policy_dscp_marking_rule() # the list of resources available for a user with admin role admin_resources = self.ntp_client.list_dscp_marking_rules( policy_id=self.policy_id)["dscp_marking_rules"] with self.rbac_utils.override_role_and_validate_list( self, admin_resources=admin_resources) as ctx: # the list of resources available for a user with member role ctx.resources = self.ntp_client.list_dscp_marking_rules( policy_id=self.policy_id)["dscp_marking_rules"] * Validating that a resource, created before "override_role", is not present in a "ResponseBody". # the resource created by a user with admin role admin_resource_id = ( self.ntp_client.create_dscp_marking_rule() ["dscp_marking_rule"]["id']) with self.rbac_utils.override_role_and_validate_list( self, admin_resource_id=admin_resource_id) as ctx: # the list of resources available for a user wirh member role ctx.resources = self.ntp_client.list_dscp_marking_rules( policy_id=self.policy_id)["dscp_marking_rules"] * Merged "RbacUtils" and "RbacUtilsMixin" classes. Now there is only "RbacUtilsMixin" class. The new class still provides all functionality of the original "RbacUtils" class. New implementation simplifies the usage of the rbac utils: * there is no need in calling "cls.setup_rbac_utils()" function, because it happens automatically at the "setup_clients" step. * there is no "rbac_utils" variable, so if you need to call a "override_role" function, just do it using "self": with self.override_role(): ... * there is no need in "test_obj" variable for "override_role" function, because it can use "self". * A new policy feature flag called "[policy_feature_flag].removed_nova_policies_stein" has been added to Patrole's config to handle Nova API extension policies removed in Stein. The policy feature flag is applied to tests that validate response bodies for expected attributes previously returned for the following policies that passed authorization: * os_compute_api:os-config-drive * os_compute_api:os-extended-availability-zone * os_compute_api:os-extended-status * os_compute_api:os-extended-volumes * os_compute_api:os-keypairs * os_compute_api:os-server-usage * os_compute_api:os-flavor-rxtx * os_compute_api:os-flavor-access (only from /flavors APIs) * os_compute_api:image-size Note that not all removed policies are included above because test coverage is missing for them (like os_compute_api:os-security- groups). * Added new feature flag called "removed_keystone_policies_stein" under the configuration group "[policy-feature-enabled]" for skipping Keystone tests whose policies were removed in Stein. This feature flag is currently applied to credentials-related policies, e.g.: identity:[create|update|get|delete]_credential * The "requirements_authority" module now supports the following 3 cases: * logical or operation of roles (existing functionality) * logical and operation of roles (new functionality) * logical not operation of roles (new functionality) <service_foo>: <logical_or_example>: - <allowed_role_1> - <allowed_role_2> <logical_and_example>: - <allowed_role_3>, <allowed_role_4> <service_bar>: <logical_not_example>: - <!disallowed_role_5> Each item under "logical_or_example" is "logical OR"-ed together. Each role in the comma-separated string under "logical_and_example" is "logical AND"-ed together. And each item prefixed with "!" under "logical_not_example" is "logical negated". This allows for expressing many more complex cases using the "requirements_authority" YAML syntax. For example, the policy rule (i.e. what may exist in a "policy.yaml" file): "foo_rule: (role:a and not role:b) or role:c" May now be expressed using the YAML syntax as: foo_rule: - a, !b - c * Patrole will validate the deprecated policy rules (if applicable) alongside the current policy rule. Add "[patrole] validate_deprecated_rules" enabled by default to validate the deprecated rules. * Added new Cinder feature flag ("CONF.policy_feature_enabled.added_cinder_policies_stein") for the following newly introduced granular Cinder policies: * "volume_extension:volume_type_encryption:create" * "volume_extension:volume_type_encryption:get" * "volume_extension:volume_type_encryption:update" * "volume_extension:volume_type_encryption:delete" The corresponding Patrole test cases are modified to support the granularity. The test cases also support backward compatibility with the old single rule: "volume_extension:volume_type_encryption" The "rules" parameter in "rbac_rule_validation.action" decorator now also accepts a list of callables; each callable should return a policy action (str). * Patrole now supports parsing custom YAML policy files, the new policy file extension since Ocata. The function "_get_policy_data" has been renamed to "get_rules" and been changed to re-use "oslo_policy.policy.Rules.load" function. Upgrade Notes ************* * The exception class "RbacMalformedException" has been removed. Use one of the following exception classes instead: * "RbacMissingAttributeResponseBody" * "RbacPartialResponseBody" * "RbacEmptyResponseBody" * Remove usage of "cls.setup_rbac_utils()" function. * Remove usage of "self.rbac_utils" variable: with self.rbac_utils.override_role(self): convert to with self.override_role(): * Remove "test_obj" in usage of "override_role" context manager: with self.override_role(self): convert to with self.override_role(): * Remove deprecated "[patrole].enable_rbac" configuration option. To skip Patrole tests going forward, use an appropriate regex. * The following deprecated parameters in "rbac_rule_validation.action" decorator: * "rule" * "expected_error_code" have been removed. Use the non-deprecated versions instead: * "rules" * "expected_error_codes" Deprecation Notes ***************** * Patrole will only support the v3 Tempest roles client for role overriding operations. Support for the v2 version has been dropped because the Keystone v2 API is slated for removal. * Config parameter CONF.rbac_test_role is deprecated in favor of CONF.rbac_test_roles that implements a list of roles instead of single role. Bug Fixes ********* * Previously, the "rbac_rule_validation.action" decorator could catch expected exceptions with no regard to where the error happened. Such behavior could cause false-positive results. To prevent this from happening from now on, if an exception happens outside of the "override_role" context, it will cause "rbac_exceptions.RbacOverrideRoleException" to be raised. Changes in patrole 0.4.0..0.5.0 ------------------------------- 9cd0a43 Fix README for Duplicate implicit target name: "storyboard" 87d83ea Add releasenote to tag the Patrole for Stein release b2ebe49 Updates Launchpad references to Storyboard 1a53003 Compute test are failing due not explicitly passing Network information 2db8338 Do not use self in classmethod 4e79004 Add py36 and py37 tox envs 198ac02 add python 3.7 unit test job 78e7f57 Refactoring RbacUtils part 3 - documentation d3d77ef Refactoring RbacUtils part 2 api tests ace8ea3 Refactoring RbacUtils 11e0c66 Enable checks and gate for reader role 7a308a0 Fix Policy action "get_flavors" not found 2a0fb1f Fix Policy action "get_network_ip_availabilities" not found 19e3bec Support implied rules d16ccfb Fix Doc mistakes and add log to gitignore. 588e806 Fix instability with volume attachment RBAC tests 55e5dfe Fix OverPermission exception for keystone tests 427c74c Fix the misspelling of "available" cd2c5fd Requirements yaml expected service names to be lowercase 6da06ed Use the canonical URL for repositories (git.openstack.org) 0868ded dict_object.keys() is not required for *in* operator 89d5d18 Change openstack-dev to openstack-discuss 0a82474 Rbac tests for Neutron list actions 22e2971 Add tests to cover trunks subports 02f6606 Add List Available Zones test cases for RBAC. a78dcae Fix the branches variant for stable branch job f5c0dfb Migrate patrol jobs to bionic(Ubuntu LTS 18.04) e36a973 docs: Include information about the list of supported projects b5d01cc Add missing ws seperator between words a261a2f Add tests to cover network ip availability API b68763c Add tests to cover service_profile bab9e94 Helper for validating RBAC list actions 47c43cb docs: Use sphinx-apidoc library for autodoc generation 0710e5d Validate omission of expected_error_codes defaults to 403 825d794 trivial: Fix irrelevant-files regexes for unit tests 0464e81 requirements authority: Use better exception/return code 4d4cb1e Add documentation about white box/black box testing to HACKING d02a8d8 RequirementsAuthority multi role support enhancement 2a5f41e Fix error codes for test_delete_flavor_service_profile 74f8e7d refactor: Break up RbacMalformedException into discrete exceptions 596bebd Replace all volume client aliases with _latest suffix 0d3c743 zuul: Use all rather than all-plugin for tox_envlist bbbdd93 refactor: Rename PluginRbacTest => ExtRbacTest b688823 Define common irrelevant-files d771e34 Update min tox version to 2.0 d1a87c5 trivial: Add hacking checks to irrelevant-files in .zuul.yaml 1b49965 trivial: Correct base class name in hacking check e0f3550 Multi role RBAC validation c38aca7 Add feature flag for Keystone policies removed in Stein ecfbb57 pbr: Remove unused translation sections from setup.cfg 904a02b hacking: Add hacking rule for plugin rbac test class names cacbd21 fix: Rename test classes causing accidental skip 2238c69 Fix test_update_address_scope_shared 42f7e1c docs: Add multi-policy validation documentation bf524fb Deprecate use of v2 roles client in rbac_utils.py 28c9c3a Fix create_rbac_policy tenant_id and network_id usage 1daa06a Use tempest.common.identity.identity_utils for project management 6925095 Add volume create from image test 8dd5f19 Remove invalid exception RbacConflictingPolicies 0170c99 docs: Add sections about context_is_admin/custom policy checks 59f538f Replace rule/expected_error_code with non-deprecated versions 2a6d329 Add tests to cover auto_allocated_topology 5f25db5 Add tests to cover neutron-agents dcd153a Remove extra_attr kwarg from RbacMalformedResponse 26b7e09 Add developer test writing guide for Patrole tests b485953 Include README for neutron + plugin tests 433bf50 Add tests to cover policy_minimum_bandwidth_rule d646a4c Add tests to cover policy_bandwidth_limit_rule 8c04bd8 Add granularity for volume_extension:volume_type_encryption ef7047d Use oslo_policy.policy.Rules.load to load rules 9358f74 Add :special-members: directive to automodule in docs e9a1355 Include README in patrole_tempest_plugin/tests/api via symlink 062fb15 Add support for multiple policy files 194752f Remove deprecated patrole.enable_rbac configuration option a3c15da Use templates for cover and lower-constraints d720bad switch documentation job to new PTI 4635c6a import zuul job settings from project-config a3d7311 Remove unused config.CONF 24961a8 Add test for create_subnetpool:is_default 11376ab Limit exception handling to calls within override_role e3b2527 Adds tests to cover QOS policy 849acef Adds tests to cover address scopes 1bee142 Add periodic-stable entry to .zuul.yaml 22bb9b3 Add Patrole gate job for stable/rocky 56bb731 Add tests to cover policy_dscp_marking_rule f4cb74c add python 3.6 unit test job 0f73e7c Add bandit python security scanning to pep8 98437d4 Remove override of 'expected_error_codes' with defaults 0f45285 Remove the usage of deprecated arg 'expected_error_code' 031b182 Add tests to cover RBAC policies 6bffc5c Skip the deprecated API extensions policy tests bd15460 Add release notes page for v0.4.0 63d8602 Add tests to cover flavor_service_profile 04b2628 Add tests to cover trunks fcd6fcf Add waiters after creating volume transfer for related tests Diffstat (except docs and test files) ------------------------------------- .gitignore | 2 + .zuul.yaml | 110 ++-- HACKING.rst | 55 +- README.rst | 41 +- REVIEWING.rst | 54 +- devstack/README.rst | 2 +- devstack/plugin.sh | 43 +- etc/patrole.conf.sample | 40 +- patrole_tempest_plugin/config.py | 52 +- patrole_tempest_plugin/hacking/checks.py | 41 ++ patrole_tempest_plugin/policy_authority.py | 181 +++--- patrole_tempest_plugin/rbac_exceptions.py | 84 ++- patrole_tempest_plugin/rbac_rule_validation.py | 185 ++++-- patrole_tempest_plugin/rbac_utils.py | 431 ++++++++++--- patrole_tempest_plugin/requirements_authority.py | 62 +- .../api/compute/test_availability_zone_rbac.py | 14 +- .../api/compute/test_flavor_extra_specs_rbac.py | 20 +- .../api/compute/test_floating_ip_pools_rbac.py | 4 +- .../api/compute/test_floating_ips_bulk_rbac.py | 12 +- .../compute/test_instance_usages_audit_log_rbac.py | 8 +- .../api/compute/test_quota_class_sets_rbac.py | 22 +- .../api/compute/test_server_migrations_rbac.py | 8 +- .../test_server_misc_policy_actions_rbac.py | 212 ++++--- .../compute/test_server_volume_attachments_rbac.py | 140 ++++- .../api/compute/test_virtual_interfaces_rbac.py | 4 +- .../v3/test_application_credentials_rbac.py | 28 +- .../identity/v3/test_domain_configuration_rbac.py | 58 +- .../api/identity/v3/test_ep_filter_groups_rbac.py | 24 +- .../identity/v3/test_ep_filter_projects_rbac.py | 20 +- .../api/identity/v3/test_oauth_consumers_rbac.py | 20 +- .../api/identity/v3/test_oauth_tokens_rbac.py | 24 +- .../identity/v3/test_policy_association_rbac.py | 36 +- .../api/identity/v3/test_project_tags_rbac.py | 24 +- .../api/identity/v3/test_role_assignments_rbac.py | 8 +- .../api/identity/v3/test_tokens_negative_rbac.py | 15 +- .../api/image/test_image_namespace_objects_rbac.py | 16 +- .../image/test_image_namespace_property_rbac.py | 16 +- .../api/image/test_image_namespace_tags_rbac.py | 20 +- .../api/image/test_image_resource_types_rbac.py | 15 +- .../network/test_auto_allocated_topology_rbac.py | 42 +- .../api/network/test_availability_zones_rbac.py | 48 ++ .../api/network/test_dscp_marking_rule_rbac.py | 121 ++++ .../network/test_flavor_service_profile_rbac.py | 77 +++ .../api/network/test_metering_label_rules_rbac.py | 29 +- .../network/test_network_ip_availability_rbac.py | 64 ++ .../api/network/test_network_segments_rbac.py | 11 +- .../test_policy_bandwidth_limit_rule_rbac.py | 109 ++++ .../test_policy_minimum_bandwidth_rule_rbac.py | 112 ++++ .../api/network/test_service_providers_rbac.py | 4 +- .../api/volume/test_snapshots_actions_rbac.py | 12 +- .../api/volume/test_snapshots_metadata_rbac.py | 30 +- .../api/volume/test_volume_basic_crud_rbac.py | 41 +- .../api/volume/test_volume_types_access_rbac.py | 12 +- .../volume/test_volume_types_extra_specs_rbac.py | 20 +- .../api/volume/test_volumes_snapshots_rbac.py | 36 +- ...-into-discrete-exceptions-92aedb99d0a13f58.yaml | 25 + ...ors-only-in-override-role-f7109a73f5ff70e2.yaml | 19 + ...oles-client-in-rbac-utils-087eda0658d18fa9.yaml | 6 + .../notes/implied-roles-96a307a2b9fa2a40.yaml | 22 + .../notes/multi-role-rbac-7f597c004a558956.yaml | 11 + .../multiple-policy-files-9aa7f7583283739e.yaml | 17 + ...de-role-and-validate-list-d3b80f773674a652.yaml | 37 ++ .../patrole-stein-release-874b36f2fedcd2fb.yaml | 15 + .../rbac-utils-refactoring-2f4f1e3b52fcae14.yaml | 49 ++ ...d-api-extensions-policies-fca3d31c7f5f1f6c.yaml | 23 + ...enable-rbac-config-option-a5e46ce1053b7dea.yaml | 5 + ...pected-error-codes-params-52071a83113934fd.yaml | 13 + ...licies-stein-feature-flag-6cfebbf64ed525d7.yaml | 8 + ...hority-multi-role-support-0fe53fc49567e595.yaml | 37 ++ .../support-deprecated-roles-eae9dc742cb4fa33.yaml | 7 + ...yption-policy-granularity-141ac283b9c0778e.yaml | 19 + .../yaml-policy-file-support-278d3edf64f98d69.yaml | 7 + releasenotes/source/index.rst | 1 + releasenotes/source/v0.4.0.rst | 6 + setup.cfg | 16 +- test-requirements.txt | 1 + tox.ini | 14 +- 184 files changed, 6243 insertions(+), 2600 deletions(-) Requirements updates -------------------- diff --git a/test-requirements.txt b/test-requirements.txt index 9085c07..a08c27a 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -10,0 +11 @@ oslotest>=3.2.0 # Apache-2.0 +bandit>=1.5 # Apache-2.0