<html><body>
<p><font size="2" face="sans-serif">I tracked down the cause of the check-grenade-dsvm failure on</font><font size="2" face="sans-serif"> </font><a href="https://review.openstack.org/#/c/167370"><font size="2" face="sans-serif">https://review.openstack.org/#/c/167370</font></a><font size="2" face="sans-serif"> . As I understand it, grenade is taking the previous stable release, deploying it, then upgrading to the current master (plus the proposed changeset) without changing any of the config from the stable deployment. Thus the policy.json file used in that test is the file from stable-juno. Then if we look at oslo_policy/policy.py we see that if the rule being looked for is missing then the default rule will be used, but then if that default rule is also missing a KeyError is thrown. Since the default rule was missing with ceilometer's policy.json file in Juno, that's what would happen here. I assume that KeyError then gets turned into the 403 Forbidden that is causing check-grenade-dsvm failure.</font><br>
<br>
<font size="2" face="sans-serif">I suspect the author of the already-merged</font><font size="2" face="sans-serif"> </font><a href="https://review.openstack.org/#/c/115717"><font size="2" face="sans-serif">https://review.openstack.org/#/c/115717</font></a><font size="2" face="sans-serif"> </font><font size="2" face="sans-serif">did what they did in ceilometer/api/rbac.py rather than what is proposed in</font><font size="2" face="sans-serif"> </font><a href="https://review.openstack.org/#/c/167370"><font size="2" face="sans-serif">https://review.openstack.org/#/c/167370</font></a><font size="2" face="sans-serif"> </font><font size="2" face="sans-serif">just to get the grenade tests to pass. I think they got lucky (unlucky for us), too, because I think they actually did break what the grenade tests are meant to catch. The patch set which was merged under </font><a href="https://review.openstack.org/#/c/115717"><font size="2" face="sans-serif">https://review.openstack.org/#/c/115717</font></a><font size="2" face="sans-serif"> </font><font size="2" face="sans-serif">changed the rule that is checked in get_limited_to() from "context_is_admin" to "segregation". But the "segregation" rule didn't exist in the Juno version of ceilometer's policy.json, so if a method that calls get_limited_to() was tested after an upgrade, I believe it would fail </font><font size="2" face="sans-serif">with a 403 Forbidden tracing back to a KeyError looking for the "segregation" rule... very similar to what we're seeing in </font><font size="2" face="sans-serif"><a href="https://review.openstack.org/#/c/167370">https://review.openstack.org/#/c/167370</a></font><br>
<br>
<font size="2" face="sans-serif">Am I on the right track here? How should we handle this? Is there a way to maintain backward compatibility while fixing what is currently broken (as a result of </font><a href="https://review.openstack.org/#/c/115717"><font size="2" face="sans-serif">https://review.openstack.org/#/c/115717</font></a><font size="2" face="sans-serif"> )</font><font size="2" face="sans-serif"> </font><font size="2" face="sans-serif"><u>and</u></font><font size="2" face="sans-serif"> allowing for a fix for </font><a href="https://bugs.launchpad.net/ceilometer/+bug/1435855"><font size="2" face="sans-serif">https://bugs.launchpad.net/ceilometer/+bug/1435855</font></a><font size="2" face="sans-serif"> (the goal of </font><a href="https://review.openstack.org/#/c/167370"><font size="2" face="sans-serif">https://review.openstack.org/#/c/167370</font></a><font size="2" face="sans-serif">)? Or will we need to document in the release notes that the manual step of modifying ceilometer's policy.json is required when upgrading from Juno, and then correspondingly modify grenade's upgrade_ceilometer file?</font><br>
<br>
<font size="2" face="sans-serif"><br>
W. Matthew Edmonds<br>
IBM Systems & Technology Group<br>
Email: edmondsw@us.ibm.com<br>
Phone: (919) 543-7538 / Tie-Line: 441-7538</font></body></html>