Open Stack

On 6/21/2016 10:12 PM, Angus Lees wrote:
> On Wed, 22 Jun 2016 at 05:59 Matt Riedemann <mriedem at linux.vnet.ibm.com
> <mailto:mriedem at linux.vnet.ibm.com>> wrote:
>
>     Angus, what should we be looking at from the privsep side for debugging
>     this?
>
>
> The line above the screen-n-cpu.txt.gz failure you linked to is:
> 2016-06-21 16:21:30.994
> <http://logs.openstack.org/85/331885/2/check/gate-grenade-dsvm-multinode/415e1bc/logs/new/screen-n-cpu.txt.gz?level=TRACE#_2016-06-21_16_21_30_994>1840
> WARNING oslo.privsep.daemon [-] privsep log:
> /usr/local/bin/nova-rootwrap: Unauthorized command: privsep-helper
> --config-file /etc/nova/nova.conf --privsep_context
> os_brick.privileged.default --privsep_sock_path
> /tmp/tmpV5w2VC/privsep.sock (no filter matched)
>
>  .. so nova-rootwrap is rejecting the privsep-helper command line
> because no filter matched.  This indicates the nova compute.filters file
> has not been updated, or is incorrect.
>
>
> As was later pointed out by mtreinish, grenade is attempting to run the
> newton code against mitaka configs, and this includes using mitaka
> rootwrap filters.   Unfortunately, the change to add privsep to nova's
> rootwrap filters wasn't approved until the newton cycle (so that all the
> os-brick privsep-related changes could be approved together), and so
> this doesn't Just Work.
>
> Digging in further, it appears that there *is* a mechanism in grenade to
> upgrade rootwrap filters between major releases, but this needs to be
> explicitly updated for each project+release and hasn't been for
> nova+mitaka->newton.  I'm not sure how this is *meant* to work, since
> the grenade "theory of upgrade" doesn't mention when configs should be
> updated - the only mechanism provided is an "exception ... used sparingly."

As noted in the review, my understanding of the config changes is 
deprecation of options across release boundaries so that you can't drop 
a config option that would break someone from release to release without 
it being deprecated first. So deprecate option foo in mitaka, people 
upgrading from liberty to mitaka aren't broken, but they get warnings in 
mitaka so that when you drop the option in newton it's not a surprise 
and consumers should have adjusted during mitaka.

For rootwrap filters I agree this is more complicated.

>
> Anyway, I added an upgrade step for nova mitaka->newton that updates
> rootwrap filters appropriately(*).  Again, I'm not sure what this
> communicates to deployers compared to cinder (which *did* have the
> updated rootwrap filter merged in mitaka, but of course that update
> still needs to be installed at some point).
> (*) https://review.openstack.org/#/c/332610
>
>  - Gus
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Alternatively Walter had a potential workaround to fallback to rootwrap 
for os-brick:

https://review.openstack.org/#/c/329586/

So we could maybe use that for newton. But os-vif doesn't have anything 
like that, so we'd have to see what kind of (immediately deprecated) 
workaround could happen for os-vif in newton and then drop that in ocata.

I'm told danpb is out until tomorrow though so we'll probably need to 
wait to talk to him about options there.

-- 

Thanks,

Matt Riedemann