[openstack-dev] [cinder] [nova] os-brick privsep failures and an upgrade strategy?

Li, Xiaoyan xiaoyan.li at intel.com
Fri Jul 22 08:05:54 UTC 2016


Hi,

What is the discussion result of privsep issue?
When can we release next os-brick?

Best wishes
Lisa

From: Ivan Kolodyazhny [mailto:e0ne at e0ne.info]
Sent: Wednesday, July 13, 2016 9:55 PM
To: OpenStack Development Mailing List (not for usage questions) <openstack-dev at lists.openstack.org>
Subject: Re: [openstack-dev] [cinder] [nova] os-brick privsep failures and an upgrade strategy?

Thanks for the update, Matt.

I will join our meeting next week.

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

On Tue, Jul 12, 2016 at 4:25 PM, Matt Riedemann <mriedem at linux.vnet.ibm.com<mailto:mriedem at linux.vnet.ibm.com>> wrote:
On 7/12/2016 6:29 AM, Ivan Kolodyazhny wrote:
Hi team,

Do we have any decision on this issue? I've found few patches but both
of them are -1'ed.

From Cinder perspective, it blocks us to release new os-brick with
features, which are needed for other projects like Cinder and
python-brick-cinderclient-ext.

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

On Wed, Jun 22, 2016 at 5:47 PM, Matt Riedemann
<mriedem at linux.vnet.ibm.com<mailto:mriedem at linux.vnet.ibm.com> <mailto:mriedem at linux.vnet.ibm.com<mailto:mriedem at linux.vnet.ibm.com>>> wrote:

    On 6/21/2016 10:12 PM, Angus Lees wrote:

        On Wed, 22 Jun 2016 at 05:59 Matt Riedemann
        <mriedem at linux.vnet.ibm.com<mailto:mriedem at linux.vnet.ibm.com> <mailto:mriedem at linux.vnet.ibm.com<mailto:mriedem at linux.vnet.ibm.com>>
        <mailto:mriedem at linux.vnet.ibm.com<mailto:mriedem at linux.vnet.ibm.com>

        <mailto:mriedem at linux.vnet.ibm.com<mailto:mriedem at linux.vnet.ibm.com>>>> wrote:

            Angus, what should we be looking at from the privsep side
        for debugging
            this?


        The line above the screen-n-cpu.txt.gz failure you linked to is:
        2016-06-21 16:21:30.994
        <http://logs.openstack.org/85/331885/2/check/gate-grenade-dsvm-multinode/415e1bc/logs/new/screen-n-cpu.txt.gz?level=TRACE#_2016-06-21_16_21_30_994>1840
        WARNING oslo.privsep.daemon [-] privsep log:
        /usr/local/bin/nova-rootwrap: Unauthorized command: privsep-helper
        --config-file /etc/nova/nova.conf --privsep_context
        os_brick.privileged.default --privsep_sock_path
        /tmp/tmpV5w2VC/privsep.sock (no filter matched)

         .. so nova-rootwrap is rejecting the privsep-helper command line
        because no filter matched.  This indicates the nova
        compute.filters file
        has not been updated, or is incorrect.


        As was later pointed out by mtreinish, grenade is attempting to
        run the
        newton code against mitaka configs, and this includes using mitaka
        rootwrap filters.   Unfortunately, the change to add privsep to
        nova's
        rootwrap filters wasn't approved until the newton cycle (so that
        all the
        os-brick privsep-related changes could be approved together), and so
        this doesn't Just Work.

        Digging in further, it appears that there *is* a mechanism in
        grenade to
        upgrade rootwrap filters between major releases, but this needs
        to be
        explicitly updated for each project+release and hasn't been for
        nova+mitaka->newton.  I'm not sure how this is *meant* to work,
        since
        the grenade "theory of upgrade" doesn't mention when configs
        should be
        updated - the only mechanism provided is an "exception ... used
        sparingly."


    As noted in the review, my understanding of the config changes is
    deprecation of options across release boundaries so that you can't
    drop a config option that would break someone from release to
    release without it being deprecated first. So deprecate option foo
    in mitaka, people upgrading from liberty to mitaka aren't broken,
    but they get warnings in mitaka so that when you drop the option in
    newton it's not a surprise and consumers should have adjusted during
    mitaka.

    For rootwrap filters I agree this is more complicated.


        Anyway, I added an upgrade step for nova mitaka->newton that updates
        rootwrap filters appropriately(*).  Again, I'm not sure what this
        communicates to deployers compared to cinder (which *did* have the
        updated rootwrap filter merged in mitaka, but of course that update
        still needs to be installed at some point).
        (*) https://review.openstack.org/#/c/332610

         - Gus


        __________________________________________________________________________
        OpenStack Development Mailing List (not for usage questions)
        Unsubscribe:
        OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
        <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
        http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


    Alternatively Walter had a potential workaround to fallback to
    rootwrap for os-brick:

    https://review.openstack.org/#/c/329586/

    So we could maybe use that for newton. But os-vif doesn't have
    anything like that, so we'd have to see what kind of (immediately
    deprecated) workaround could happen for os-vif in newton and then
    drop that in ocata.

    I'm told danpb is out until tomorrow though so we'll probably need
    to wait to talk to him about options there.


    --

    Thanks,

    Matt Riedemann


    __________________________________________________________________________
    OpenStack Development Mailing List (not for usage questions)
    Unsubscribe:
    OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
    <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

We probably aren't doing anything while Sean Dague is on vacation. He's back next week and we have the nova/cinder meetups, so I'm planning on talking about the grenade issue in person and hopefully we'll have a plan by the end of next week to move forward.


--

Thanks,

Matt Riedemann


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160722/6a76471c/attachment.html>


More information about the OpenStack-dev mailing list