[openstack-dev] [nova][placement][oslo] Excessive WARNING level log messages in placement-api

Doug Hellmann doug at doughellmann.com
Thu Aug 9 18:05:01 UTC 2018


Excerpts from Lance Bragstad's message of 2018-08-09 12:55:52 -0500:
> 
> On 08/09/2018 12:48 PM, Doug Hellmann wrote:
> > Excerpts from Matt Riedemann's message of 2018-08-09 12:18:14 -0500:
> >> On 8/9/2018 11:47 AM, Doug Hellmann wrote:
> >>> Excerpts from Jay Pipes's message of 2018-08-08 22:53:54 -0400:
> >>>> For evidence, see:
> >>>>
> >>>> http://logs.openstack.org/41/590041/1/check/tempest-full-py3/db08dec/controller/logs/screen-placement-api.txt.gz?level=WARNING
> >>>>
> >>>> thousands of these are filling the logs with WARNING-level log messages,
> >>>> making it difficult to find anything:
> >>>>
> >>>> Aug 08 22:17:30.837557 ubuntu-xenial-inap-mtl01-0001226060
> >>>> devstack at placement-api.service[14403]: WARNING py.warnings
> >>>> [req-a809b022-59af-4628-be73-488cfec3187d
> >>>> req-d46cb1f0-431f-490f-955b-b9c2cd9f6437 service placement]
> >>>> /usr/local/lib/python3.5/dist-packages/oslo_policy/policy.py:896:
> >>>> UserWarning: Policy placement:resource_providers:list failed scope
> >>>> check. The token used to make the request was project scoped but the
> >>>> policy requires ['system'] scope. This behavior may change in the future
> >>>> where using the intended scope is required
> >>>> Aug 08 22:17:30.837800 ubuntu-xenial-inap-mtl01-0001226060
> >>>> devstack at placement-api.service[14403]:   warnings.warn(msg)
> >>>> Aug 08 22:17:30.838067 ubuntu-xenial-inap-mtl01-0001226060
> >>>> devstack at placement-api.service[14403]:
> >>>>
> >>>> Is there any way we can get rid of these?
> >>>>
> >>>> Thanks,
> >>>> -jay
> >>>>
> >>> It looks like those are coming out of the policy library? Maybe file a
> >>> bug there. I added "oslo" to the subject line to get the team's
> >>> attention.
> >>>
> >>> This feels like something we could fix and backport to rocky.
> >>>
> >>> Doug
> >> I could have sworn I created a bug in oslo.policy for this at one point 
> >> for the same reason Jay mentions it, but I guess not.
> >>
> >> We could simply, on the nova side, add a warnings filter to only log 
> >> this once.
> >>
> > What level should it be logged at in the policy library? Should it be
> > logged there at all?
> 
> The initial intent behind logging was to make sure operators knew that
> they needed to make a role assignment adjustment in order to be
> compatible moving forward. I can investigate a way to log things at
> least once in oslo.policy though. I fear not logging it at all would
> cause failures in upgrade since operators wouldn't know they need to
> make that adjustment.

That sounds like a good check to add to the upgrade test tools as part
of the goal for Stein.

Doug



More information about the OpenStack-dev mailing list