<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hi,
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">What is the discussion result of privsep issue?
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">When can we release next os-brick?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Best wishes<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Lisa<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Ivan Kolodyazhny [mailto:e0ne@e0ne.info]
<br>
<b>Sent:</b> Wednesday, July 13, 2016 9:55 PM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org><br>
<b>Subject:</b> Re: [openstack-dev] [cinder] [nova] os-brick privsep failures and an upgrade strategy?<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Thanks for the update, Matt.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I will join our meeting next week.<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal">Regards,<br>
Ivan Kolodyazhny,<br>
<a href="http://blog.e0ne.info/" target="_blank">http://blog.e0ne.info/</a><o:p></o:p></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Tue, Jul 12, 2016 at 4:25 PM, Matt Riedemann <<a href="mailto:mriedem@linux.vnet.ibm.com" target="_blank">mriedem@linux.vnet.ibm.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">On 7/12/2016 6:29 AM, Ivan Kolodyazhny wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">Hi team,<br>
<br>
Do we have any decision on this issue? I've found few patches but both<br>
of them are -1'ed.<br>
<br>
From Cinder perspective, it blocks us to release new os-brick with<br>
features, which are needed for other projects like Cinder and<br>
python-brick-cinderclient-ext.<br>
<br>
Regards,<br>
Ivan Kolodyazhny,<br>
<a href="http://blog.e0ne.info/" target="_blank">http://blog.e0ne.info/</a><br>
<br>
On Wed, Jun 22, 2016 at 5:47 PM, Matt Riedemann<br>
<<a href="mailto:mriedem@linux.vnet.ibm.com" target="_blank">mriedem@linux.vnet.ibm.com</a> <mailto:<a href="mailto:mriedem@linux.vnet.ibm.com" target="_blank">mriedem@linux.vnet.ibm.com</a>>> wrote:<br>
<br>
On 6/21/2016 10:12 PM, Angus Lees wrote:<br>
<br>
On Wed, 22 Jun 2016 at 05:59 Matt Riedemann<br>
<<a href="mailto:mriedem@linux.vnet.ibm.com" target="_blank">mriedem@linux.vnet.ibm.com</a> <mailto:<a href="mailto:mriedem@linux.vnet.ibm.com" target="_blank">mriedem@linux.vnet.ibm.com</a>><br>
<mailto:<a href="mailto:mriedem@linux.vnet.ibm.com" target="_blank">mriedem@linux.vnet.ibm.com</a><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"><br>
<mailto:<a href="mailto:mriedem@linux.vnet.ibm.com" target="_blank">mriedem@linux.vnet.ibm.com</a>>>> wrote:<br>
<br>
Angus, what should we be looking at from the privsep side<br>
for debugging<br>
this?<br>
<br>
<br>
The line above the screen-n-cpu.txt.gz failure you linked to is:<br>
2016-06-21 16:21:30.994<br>
<<a href="http://logs.openstack.org/85/331885/2/check/gate-grenade-dsvm-multinode/415e1bc/logs/new/screen-n-cpu.txt.gz?level=TRACE#_2016-06-21_16_21_30_994" target="_blank">http://logs.openstack.org/85/331885/2/check/gate-grenade-dsvm-multinode/415e1bc/logs/new/screen-n-cpu.txt.gz?level=TRACE#_2016-06-21_16_21_30_994</a>>1840<br>
WARNING oslo.privsep.daemon [-] privsep log:<br>
/usr/local/bin/nova-rootwrap: Unauthorized command: privsep-helper<br>
--config-file /etc/nova/nova.conf --privsep_context<br>
os_brick.privileged.default --privsep_sock_path<br>
/tmp/tmpV5w2VC/privsep.sock (no filter matched)<br>
<br>
.. so nova-rootwrap is rejecting the privsep-helper command line<br>
because no filter matched. This indicates the nova<br>
compute.filters file<br>
has not been updated, or is incorrect.<br>
<br>
<br>
As was later pointed out by mtreinish, grenade is attempting to<br>
run the<br>
newton code against mitaka configs, and this includes using mitaka<br>
rootwrap filters. Unfortunately, the change to add privsep to<br>
nova's<br>
rootwrap filters wasn't approved until the newton cycle (so that<br>
all the<br>
os-brick privsep-related changes could be approved together), and so<br>
this doesn't Just Work.<br>
<br>
Digging in further, it appears that there *is* a mechanism in<br>
grenade to<br>
upgrade rootwrap filters between major releases, but this needs<br>
to be<br>
explicitly updated for each project+release and hasn't been for<br>
nova+mitaka->newton. I'm not sure how this is *meant* to work,<br>
since<br>
the grenade "theory of upgrade" doesn't mention when configs<br>
should be<br>
updated - the only mechanism provided is an "exception ... used<br>
sparingly."<br>
<br>
<br>
As noted in the review, my understanding of the config changes is<br>
deprecation of options across release boundaries so that you can't<br>
drop a config option that would break someone from release to<br>
release without it being deprecated first. So deprecate option foo<br>
in mitaka, people upgrading from liberty to mitaka aren't broken,<br>
but they get warnings in mitaka so that when you drop the option in<br>
newton it's not a surprise and consumers should have adjusted during<br>
mitaka.<br>
<br>
For rootwrap filters I agree this is more complicated.<br>
<br>
<br>
Anyway, I added an upgrade step for nova mitaka->newton that updates<br>
rootwrap filters appropriately(*). Again, I'm not sure what this<br>
communicates to deployers compared to cinder (which *did* have the<br>
updated rootwrap filter merged in mitaka, but of course that update<br>
still needs to be installed at some point).<br>
(*) <a href="https://review.openstack.org/#/c/332610" target="_blank">https://review.openstack.org/#/c/332610</a><br>
<br>
- Gus<br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe:<br>
<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
<br>
Alternatively Walter had a potential workaround to fallback to<br>
rootwrap for os-brick:<br>
<br>
<a href="https://review.openstack.org/#/c/329586/" target="_blank">https://review.openstack.org/#/c/329586/</a><br>
<br>
So we could maybe use that for newton. But os-vif doesn't have<br>
anything like that, so we'd have to see what kind of (immediately<br>
deprecated) workaround could happen for os-vif in newton and then<br>
drop that in ocata.<br>
<br>
I'm told danpb is out until tomorrow though so we'll probably need<br>
to wait to talk to him about options there.<br>
<br>
<br>
--<br>
<br>
Thanks,<br>
<br>
Matt Riedemann<br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe:<br>
<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
<br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><br>
We probably aren't doing anything while Sean Dague is on vacation. He's back next week and we have the nova/cinder meetups, so I'm planning on talking about the grenade issue in person and hopefully we'll have a plan by the end of next week to move forward.<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"><br>
<br>
-- <br>
<br>
Thanks,<br>
<br>
Matt Riedemann<br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>