<div dir="ltr">Just for anyone in the future, we we're able to narrow down the cause. It looks like there is kind of caching for the auth tokens (either locally or at the Openstack level). I had to re-establish the Openstack session object from my script before trying to create the privileged object.<div><br></div><div>Oh well, lesson learned...<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Apr 20, 2020 at 11:08 AM Laurent Dumont <<a href="mailto:laurentfdumont@gmail.com">laurentfdumont@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi!<div><br></div><div>Here is what Neutron is complaining about.</div><div><br></div><div>DEBUG neutron.api.v2.base [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] Request body: {u'port': {u'network_id': u'4ba08203-4328-418b-89e6-b82d474275d5', u'fixed_ips': [{u'subnet_id': u'77c3fad2-bf78-4b4e-bcfb-8114f95d7884', u'ip_address': u'1.1.1.100'}], u'name': u'neutronpolicy_portb-fixed-ip'}} prepare_request_body /usr/lib/python2.7/site-packages/neutron/api/v2/base.py:690<br>DEBUG neutron.db.quota.driver [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] Resources subnetpool,trunk have unlimited quota limit. It is not required to calculate headroom make_reservation /usr/lib/python2.7/site-packages/neutron/db/quota/driver.py:223<br>DEBUG neutron.quota.resource [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] Usage tracker for resource:port and tenant:4bae655884b44e8e9c8dbb5983fb3cd8 is out of sync, need to count used quota count_used /usr/lib/python2.7/site-packages/neutron/quota/resource.py:272<br>DEBUG neutron.quota.resource [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] Quota usage for port was recalculated. Used quota:0. count_used /usr/lib/python2.7/site-packages/neutron/quota/resource.py:292<br>DEBUG neutron.db.quota.driver [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 make_reservation /usr/lib/python2.7/site-packages/neutron/db/quota/driver.py:255<br>DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] Made reservation on behalf of 4bae655884b44e8e9c8dbb5983fb3cd8 for: {'port': 1} before /usr/lib/python2.7/site-packages/neutron/pecan_wsgi/hooks/quota_enforcement.py:55<br>DEBUG neutron.policy [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] Unable to find requested field: shared in target: {'binding:host_id': <neutron_lib.constants.Sentinel object at 0x7f2a3b214d50>, u'name': u'neutronpolicy_portb-fixed-ip', 'allowed_address_pairs': <neutron_lib.constants.Sentinel object at 0x7f2a3b214d50>, 'admin_state_up': True, u'network_id': u'4ba08203-4328-418b-89e6-b82d474275d5', 'tenant_id': u'4bae655884b44e8e9c8dbb5983fb3cd8', 'extra_dhcp_opts': None, 'mac_address': <neutron_lib.constants.Sentinel object at 0x7f2a3b214d50>, 'binding:vnic_type': 'normal', 'device_owner': '', 'qos_policy_id': None, 'device_id': '', 'binding:profile': <neutron_lib.constants.Sentinel object at 0x7f2a3b214d50>, 'port_security_enabled': <neutron_lib.constants.Sentinel object at 0x7f2a3b214d50>, 'project_id': u'4bae655884b44e8e9c8dbb5983fb3cd8', u'fixed_ips': [{u'subnet_id': u'77c3fad2-bf78-4b4e-bcfb-8114f95d7884', u'ip_address': u'1.1.1.100'}], u'network:tenant_id': u'472631dbfc5f4dfda9293fb93b5e493f', 'security_groups': <neutron_lib.constants.Sentinel object at 0x7f2a3b214d50>, 'description': ''} __call__ /usr/lib/python2.7/site-packages/neutron/policy.py:314<br>DEBUG neutron.policy [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] Enforcing rules: ['create_port', 'create_port:fixed_ips'] log_rule_list /usr/lib/python2.7/site-packages/neutron/policy.py:334<br>DEBUG neutron.policy [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] Failed policy check for 'create_port' enforce /usr/lib/python2.7/site-packages/neutron/policy.py:405<br>INFO neutron.pecan_wsgi.hooks.translation [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] POST failed (client error): Access was denied to this resource.<br>DEBUG neutron.pecan_wsgi.hooks.notifier [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] No notification will be sent due to unsuccessful status code: 403 after /usr/lib/python2.7/site-packages/neutron/pecan_wsgi/hooks/notifier.py:79<br>INFO neutron.wsgi [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] 192.168.20.29 "POST /v2.0/ports HTTP/1.1" status: 403 len: 333 time: 0.3625510</div><div><br></div><div>I've tried removing each controller one by one (to validate if one was out of sync) - and I had the issue in all three cases.</div><div><br></div><div>Very weird. I'll keep digging.</div><div><br></div><div> <br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Apr 19, 2020 at 5:27 AM Slawek Kaplonski <<a href="mailto:skaplons@redhat.com" target="_blank">skaplons@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
<br>
If it fails only in some cases, it seems for me like maybe You have different<br>
policy.json configured (or different neutron version installed) on controllers.<br>
Can You check if all is the same in all cases?<br>
Also please send logs from neutron-server from when this failed.<br>
<br>
On Sun, Apr 19, 2020 at 02:08:09AM -0400, Laurent Dumont wrote:<br>
> Hi!<br>
> <br>
> I'm currently troubleshooting some transient Keystone/Neutron 403 issues<br>
> when creating ports in RBAC-ed networks.<br>
> <br>
> - As part of our CI process,we are running a few sanity tests in virtual<br>
> Openstack setups with 3 controllers and 2 computes.<br>
> - One of those test validates the following scenarios (everything is<br>
> done through the Openstack python sdk) :<br>
> - Create projectA / UserA / ProjectB / UserB<br>
> - Create Network in project A<br>
> - RBAC Network in projectA to ProjectB<br>
> - With a token issued to a user in projectB, try to create the<br>
> following ports in project.<br>
> - regular port<br>
> - update regular port with a fixed IP<br>
> - update regular port with an allowed_address_pair<br>
> - create port with a fixed IP<br>
> - create port with an allowed address pair.<br>
> - The behavior that I'm seeing the most is that the port creation<br>
> will fail with the following error (the logs are from the sdk with<br>
> logging/debug enabled.<br>
> - It fails maybe 1/3 - and I'm unable to see a common pattern.<br>
> - This is testing against the same Openstack env and the<br>
> failures/success can happen within a few seconds of each others.<br>
> <br>
> Sun Apr 19 01:54:06 2020 -- From ProjectB - create a port with a fixed<br>
> > address in the network shared from ProjectA<br>
> > REQ: curl -g -i -X GET<br>
> > <a href="https://OPENSTACK_URL_HERE:13696/v2.0/subnets/77c3fad2-bf78-4b4e-bcfb-8114f95d7884" rel="noreferrer" target="_blank">https://OPENSTACK_URL_HERE:13696/v2.0/subnets/77c3fad2-bf78-4b4e-bcfb-8114f95d7884</a><br>
> > -H "User-Agent: openstacksdk/0.37.0 keystoneauth1/3.18.0<br>
> > python-requests/2.22.0 CPython/3.7.5" -H "X-Auth-Token:<br>
> > {SHA256}d80fe300742786033376f0591f98d3b4c342103cda1f802087809963694a7935"<br>
> > RESP: [200] Content-Length: 619 Content-Type: application/json Date: Sun,<br>
> > 19 Apr 2020 01:54:06 GMT X-Openstack-Request-Id:<br>
> > req-fcee8345-7daa-499e-a985-92d59a31d0dd<br>
> > RESP BODY:<br>
> > {"subnet":{"service_types":[],"description":"","enable_dhcp":true,"tags":[],"network_id":"4ba08203-4328-418b-89e6-b82d474275d5","tenant_id":"472631dbfc5f4dfda9293fb93b5e493f","created_at":"2020-04-19T01:54:03Z","dns_nameservers":[],"updated_at":"2020-04-19T01:54:03Z","gateway_ip":"1.1.1.1","ipv6_ra_mode":null,"allocation_pools":[{"start":"1.1.1.2","end":"1.1.1.254"}],"host_routes":[],"revision_number":0,"ip_version":4,"ipv6_address_mode":null,"cidr":"<br>
> > <a href="http://1.1.1.0/24" rel="noreferrer" target="_blank">1.1.1.0/24</a><br>
> > ","project_id":"472631dbfc5f4dfda9293fb93b5e493f","id":"77c3fad2-bf78-4b4e-bcfb-8114f95d7884","subnetpool_id":null,"name":"neutronpolicy_shared_network"}}<br>
> > GET call to network for<br>
> > <a href="https://OPENSTACK_URL_HERE:13696/v2.0/subnets/77c3fad2-bf78-4b4e-bcfb-8114f95d7884" rel="noreferrer" target="_blank">https://OPENSTACK_URL_HERE:13696/v2.0/subnets/77c3fad2-bf78-4b4e-bcfb-8114f95d7884</a><br>
> > used request id req-fcee8345-7daa-499e-a985-92d59a31d0dd<br>
> > REQ: curl -g -i -X POST <a href="https://OPENSTACK_URL_HERE:13696/v2.0/ports" rel="noreferrer" target="_blank">https://OPENSTACK_URL_HERE:13696/v2.0/ports</a> -H<br>
> > "Content-Type: application/json" -H "User-Agent: openstacksdk/0.37.0<br>
> > keystoneauth1/3.18.0 python-requests/2.22.0 CPython/3.7.5" -H<br>
> > "X-Auth-Token:<br>
> > {SHA256}d80fe300742786033376f0591f98d3b4c342103cda1f802087809963694a7935"<br>
> > -d '{"port": {"name": "neutronpolicy_portb-fixed-ip", "network_id":<br>
> > "4ba08203-4328-418b-89e6-b82d474275d5", "fixed_ips": [{"subnet_id":<br>
> > "77c3fad2-bf78-4b4e-bcfb-8114f95d7884", "ip_address": "1.1.1.100"}]}}'<br>
> > RESP: [403] Content-Length: 151 Content-Type: application/json Date: Sun,<br>
> > 19 Apr 2020 01:54:07 GMT X-Openstack-Request-Id:<br>
> > req-00277dbc-faca-479f-8736-563595d2729b<br>
> > RESP BODY: {"NeutronError": {"message": "(rule:create_port and<br>
> > rule:create_port:fixed_ips) is disallowed by policy", "type":<br>
> > "PolicyNotAuthorized", "detail": ""}}<br>
> > POST call to network for <a href="https://OPENSTACK_URL_HERE:13696/v2.0/ports" rel="noreferrer" target="_blank">https://OPENSTACK_URL_HERE:13696/v2.0/ports</a> used<br>
> > request id req-00277dbc-faca-479f-8736-563595d2729b<br>
> <br>
> <br>
> When it works, it looks the same, just without the 403 during the port<br>
> creation.<br>
> <br>
> Sun Apr 19 02:29:54 2020 -- From ProjectB - create a port with a fixed<br>
> > address in the network shared from ProjectA<br>
> > REQ: curl -g -i -X GET<br>
> > <a href="https://OPENSTACK_URL_HERE:13696/v2.0/subnets/6d2231b6-3ce2-4121-a9a6-7284823c48ab" rel="noreferrer" target="_blank">https://OPENSTACK_URL_HERE:13696/v2.0/subnets/6d2231b6-3ce2-4121-a9a6-7284823c48ab</a><br>
> > -H "User-Agent: openstacksdk/0.37.0 keystoneauth1/3.18.0<br>
> > python-requests/2.22.0 CPython/3.7.5" -H "X-Auth-Token:<br>
> > {SHA256}d39995434f6a0f106cae7408b07eee93846230f2bb5c922a8e16cfc7c8b26599"<br>
> > RESP: [200] Content-Length: 619 Content-Type: application/json Date: Sun,<br>
> > 19 Apr 2020 02:29:54 GMT X-Openstack-Request-Id:<br>
> > req-bd41b51f-b6e9-454c-8c9a-ee444140459e<br>
> > RESP BODY:<br>
> > {"subnet":{"service_types":[],"description":"","enable_dhcp":true,"tags":[],"network_id":"e0d7e7a1-2863-45ec-bd7b-1767a1d02822","tenant_id":"4530b70143ec4cd0ad929a4ca571bdd8","created_at":"2020-04-19T02:29:51Z","dns_nameservers":[],"updated_at":"2020-04-19T02:29:51Z","gateway_ip":"1.1.1.1","ipv6_ra_mode":null,"allocation_pools":[{"start":"1.1.1.2","end":"1.1.1.254"}],"host_routes":[],"revision_number":0,"ip_version":4,"ipv6_address_mode":null,"cidr":"<br>
> > <a href="http://1.1.1.0/24" rel="noreferrer" target="_blank">1.1.1.0/24</a><br>
> > ","project_id":"4530b70143ec4cd0ad929a4ca571bdd8","id":"6d2231b6-3ce2-4121-a9a6-7284823c48ab","subnetpool_id":null,"name":"neutronpolicy_shared_network"}}<br>
> > GET call to network for<br>
> > https:/OPENSTACK_URL_HERE:13696/v2.0/subnets/6d2231b6-3ce2-4121-a9a6-7284823c48ab<br>
> > used request id req-bd41b51f-b6e9-454c-8c9a-ee444140459e<br>
> > REQ: curl -g -i -X POST <a href="https://OPENSTACK_URL_HERE:13696/v2.0/ports" rel="noreferrer" target="_blank">https://OPENSTACK_URL_HERE:13696/v2.0/ports</a> -H<br>
> > "Content-Type: application/json" -H "User-Agent: openstacksdk/0.37.0<br>
> > keystoneauth1/3.18.0 python-requests/2.22.0 CPython/3.7.5" -H<br>
> > "X-Auth-Token:<br>
> > {SHA256}d39995434f6a0f106cae7408b07eee93846230f2bb5c922a8e16cfc7c8b26599"<br>
> > -d '{"port": {"name": "neutronpolicy_portb-fixed-ip", "network_id":<br>
> > "e0d7e7a1-2863-45ec-bd7b-1767a1d02822", "fixed_ips": [{"subnet_id":<br>
> > "6d2231b6-3ce2-4121-a9a6-7284823c48ab", "ip_address": "1.1.1.100"}]}}'<br>
> > RESP: [201] Content-Length: 769 Content-Type: application/json Date: Sun,<br>
> > 19 Apr 2020 02:29:56 GMT X-Openstack-Request-Id:<br>
> > req-898dadfb-366d-4a78-8e89-c9d5e9a512f6<br>
> > RESP BODY:<br>
> > {"port":{"allowed_address_pairs":[],"extra_dhcp_opts":[],"updated_at":"2020-04-19T02:29:55Z","device_owner":"","revision_number":6,"port_security_enabled":true,"fixed_ips":[{"subnet_id":"6d2231b6-3ce2-4121-a9a6-7284823c48ab","ip_address":"1.1.1.100"}],"id":"9999784b-c38d-4cf2-b951-e8255d3483f4","security_groups":["4c022e69-b575-435b-bfb7-9cc81f50d49e"],"mac_address":"fa:16:3e:3e:22:fb","device_id":"","status":"DOWN","description":"","tags":[],"name":"neutronpolicy_portb-fixed-ip","project_id":"253ee9b6b5964185ab0818580129c011","qos_policy_id":null,"admin_state_up":true,"network_id":"e0d7e7a1-2863-45ec-bd7b-1767a1d02822","tenant_id":"253ee9b6b5964185ab0818580129c011","created_at":"2020-04-19T02:29:55Z","binding:vnic_type":"normal","ip_allocation":"immediate"}}<br>
> > POST call to network for <a href="https://OPENSTACK_URL_HERE:13696/v2.0/ports" rel="noreferrer" target="_blank">https://OPENSTACK_URL_HERE:13696/v2.0/ports</a> used<br>
> > request id req-898dadfb-366d-4a78-8e89-c9d5e9a512f6<br>
> > Sun Apr 19 02:29:56 2020 -- PORT-OK<br>
> <br>
> <br>
> This is what I see in the Neutron server logs (with debug enabled)<br>
> <br>
> DEBUG neutron.api.v2.base [req-00277dbc-faca-479f-8736-563595d2729b<br>
> > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default<br>
> > default] Request body: {u'port': {u'network_id':<br>
> > u'4ba08203-4328-418b-89e6-b82d474275d5', u'fixed_ips': [{u'subnet_id':<br>
> > u'77c3fad2-bf78-4b4e-bcfb-8114f95d7884', u'ip_address': u'1.1.1.100'}],<br>
> > u'name': u'neutronpolicy_portb-fixed-ip'}} prepare_request_body<br>
> > /usr/lib/python2.7/site-packages/neutron/api/v2/base.py:690<br>
> > DEBUG neutron.db.quota.driver [req-00277dbc-faca-479f-8736-563595d2729b<br>
> > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default<br>
> > default] Resources subnetpool,trunk have unlimited quota limit. It is not<br>
> > required to calculate headroom make_reservation<br>
> > /usr/lib/python2.7/site-packages/neutron/db/quota/driver.py:223<br>
> > DEBUG neutron.quota.resource [req-00277dbc-faca-479f-8736-563595d2729b<br>
> > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default<br>
> > default] Usage tracker for resource:port and<br>
> > tenant:4bae655884b44e8e9c8dbb5983fb3cd8 is out of sync, need to count used<br>
> > quota count_used<br>
> > /usr/lib/python2.7/site-packages/neutron/quota/resource.py:272<br>
> > DEBUG neutron.quota.resource [req-00277dbc-faca-479f-8736-563595d2729b<br>
> > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default<br>
> > default] Quota usage for port was recalculated. Used quota:0. count_used<br>
> > /usr/lib/python2.7/site-packages/neutron/quota/resource.py:292<br>
> > DEBUG neutron.db.quota.driver [req-00277dbc-faca-479f-8736-563595d2729b<br>
> > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default<br>
> > default] Attempting to reserve 1 items for resource port. Total usage: 0;<br>
> > quota limit: 500; headroom:500 make_reservation<br>
> > /usr/lib/python2.7/site-packages/neutron/db/quota/driver.py:255<br>
> > DEBUG neutron.pecan_wsgi.hooks.quota_enforcement<br>
> > [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989<br>
> > 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] Made reservation on<br>
> > behalf of 4bae655884b44e8e9c8dbb5983fb3cd8 for: {'port': 1} before<br>
> > /usr/lib/python2.7/site-packages/neutron/pecan_wsgi/hooks/quota_enforcement.py:55<br>
> > DEBUG neutron.policy [req-00277dbc-faca-479f-8736-563595d2729b<br>
> > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default<br>
> > default] Unable to find requested field: shared in target:<br>
> > {'binding:host_id': <neutron_lib.constants.Sentinel object at<br>
> > 0x7f2a3b214d50>, u'name': u'neutronpolicy_portb-fixed-ip',<br>
> > 'allowed_address_pairs': <neutron_lib.constants.Sentinel object at<br>
> > 0x7f2a3b214d50>, 'admin_state_up': True, u'network_id':<br>
> > u'4ba08203-4328-418b-89e6-b82d474275d5', 'tenant_id':<br>
> > u'4bae655884b44e8e9c8dbb5983fb3cd8', 'extra_dhcp_opts': None,<br>
> > 'mac_address': <neutron_lib.constants.Sentinel object at 0x7f2a3b214d50>,<br>
> > 'binding:vnic_type': 'normal', 'device_owner': '', 'qos_policy_id': None,<br>
> > 'device_id': '', 'binding:profile': <neutron_lib.constants.Sentinel object<br>
> > at 0x7f2a3b214d50>, 'port_security_enabled':<br>
> > <neutron_lib.constants.Sentinel object at 0x7f2a3b214d50>, 'project_id':<br>
> > u'4bae655884b44e8e9c8dbb5983fb3cd8', u'fixed_ips': [{u'subnet_id':<br>
> > u'77c3fad2-bf78-4b4e-bcfb-8114f95d7884', u'ip_address': u'1.1.1.100'}],<br>
> > u'network:tenant_id': u'472631dbfc5f4dfda9293fb93b5e493f',<br>
> > 'security_groups': <neutron_lib.constants.Sentinel object at<br>
> > 0x7f2a3b214d50>, 'description': ''} __call__<br>
> > /usr/lib/python2.7/site-packages/neutron/policy.py:314<br>
> > DEBUG neutron.policy [req-00277dbc-faca-479f-8736-563595d2729b<br>
> > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default<br>
> > default] Enforcing rules: ['create_port', 'create_port:fixed_ips']<br>
> > log_rule_list /usr/lib/python2.7/site-packages/neutron/policy.py:334<br>
> > DEBUG neutron.policy [req-00277dbc-faca-479f-8736-563595d2729b<br>
> > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default<br>
> > default] Failed policy check for 'create_port' enforce<br>
> > /usr/lib/python2.7/site-packages/neutron/policy.py:405<br>
> > INFO neutron.pecan_wsgi.hooks.translation<br>
> > [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989<br>
> > 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] POST failed (client<br>
> > error): Access was denied to this resource.<br>
> > DEBUG neutron.pecan_wsgi.hooks.notifier<br>
> > [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989<br>
> > 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] No notification will be<br>
> > sent due to unsuccessful status code: 403 after<br>
> > /usr/lib/python2.7/site-packages/neutron/pecan_wsgi/hooks/notifier.py:79<br>
> > INFO neutron.wsgi [req-00277dbc-faca-479f-8736-563595d2729b<br>
> > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default<br>
> > default] 192.168.20.29 "POST /v2.0/ports HTTP/1.1" status: 403 len: 333<br>
> > time: 0.3625510<br>
> ><br>
> <br>
> <br>
> I haven't found a whole lot online - just a couple of possibly related bugs<br>
> <a href="https://bugs.launchpad.net/neutron/+bug/1808112" rel="noreferrer" target="_blank">https://bugs.launchpad.net/neutron/+bug/1808112</a><br>
> <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1683385" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1683385</a><br>
> <br>
> Does anyone have any good tips/debugging methods?<br>
> <br>
> Thanks!<br>
<br>
-- <br>
Slawek Kaplonski<br>
Senior software engineer<br>
Red Hat<br>
<br>
</blockquote></div>
</blockquote></div>