Hi,

In Neutron we are merging more and more patches related to the secure rbac
policies [1]. Recently we noticed that our neutron-rally-task job is failing
very often [2].

I did some tests and here is what were my results:

* on latest neutron, with about 12 patches related to secure rbac merged - list
  of 4k ports took about 1m 40s:

  $ time neutron port-list | wc -l
  neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
  4044

  real    1m41,644s
  user    0m0,926s
  sys     0m0,087s

* same test with all those patches reverted - and it took 8 seconds:

  $ sudo systemctl restart devstack@q-svc; sleep 5; time neutron port-list | wc -l
  neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
  4044

  real    0m8,131s
  user    0m0,907s
  sys     0m0,120s


After a bit of digging I found out that most of the slowdown came from calls of
the method self._handle_deprecated_rule(default) in [3].
When I commented that one line I had results same like without secure-rbac patches:

  $ sudo systemctl restart devstack@q-svc; sleep 5; time neutron port-list | wc -l
  neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
  4044

  real    0m7,875s
  user    0m0,931s
  sys     0m0,087s


@Lance, can You maybe take a look at it and help me understand if we are doing
something wrong with those new rules in Neutron? Or maybe there is a bug in
oslo_policy?

[1] https://review.opendev.org/q/topic:secure-rbac+project:openstack/neutron
[2] https://bugs.launchpad.net/neutron/+bug/1913718
[3] https://github.com/openstack/oslo.policy/blob/e103baa002e54303b08630c436dfc7b0b8a013de/oslo_policy/policy.py#L641

--
Slawek Kaplonski
Principal Software Engineer
Red Hat