<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-CA" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-left:36.0pt"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Corey O'Brien [mailto:coreypobrien@gmail.com]
<br>
<b>Sent:</b> March-07-16 8:11 AM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-left:36.0pt">Hongbin, I think the offer to support different OS options is a perfect example both of what we want and what we don't want. We definitely want to allow for someone like yourself to maintain templates for whatever
OS they want and to have that option be easily integrated in to a Magnum deployment. However, when developing features or bug fixes, we can't wait for you to have time to add it for whatever OS you are promising to maintain.<span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">It might be true that supporting additional OS could slow down the development speed, but the key question is how much the impact will be. Does it outweigh
the benefits? IMO, the impact doesn’t seem to be significant, given the fact that most features and bug fixes are OS agnostic. Also, keep in mind that every features we introduced (variety of COEs, variety of Nova virt-driver, variety of network driver, variety
of volume driver, variety of …) incurs a maintenance overhead. If you want an optimal development speed, we will be limited to support a single COE/virt driver/network driver/volume driver. I guess that is not the direction we like to be?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt">Instead, we would all be forced to develop the feature for that OS as well. If every member of the team had a special OS like that we'd all have to maintain all of them.<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">To be clear, I don’t have a special OS, I guess neither do others who disagreed in this thread.<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt">Alternatively, what was agreed on by most at the midcycle was that if someone like yourself wanted to support a specific OS option, we would have an easy place for those contributions to go without impacting the
rest of the team. The team as a whole would agree to develop all features for at least the reference OS.<span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Could we re-confirm that this is a team agreement? There is no harm to re-confirm it in the design summit/ML/team meeting. Frankly, it doesn’t seem to be.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt">Then individuals or companies who are passionate about an alternative OS can develop the features for that OS.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt">Corey<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt">On Sat, Mar 5, 2016 at 12:30 AM Hongbin Lu <<a href="mailto:hongbin.lu@huawei.com">hongbin.lu@huawei.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Adrian Otto [mailto:<a href="mailto:adrian.otto@rackspace.com" target="_blank">adrian.otto@rackspace.com</a>]
<br>
<b>Sent:</b> March-04-16 6:31 PM</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""><br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">Subject:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
Steve, <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
On Mar 4, 2016, at 2:41 PM, Steven Dake (stdake) <<a href="mailto:stdake@cisco.com" target="_blank">stdake@cisco.com</a>> wrote:<o:p></o:p></p>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From: </span>
</b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Adrian Otto <<a href="mailto:adrian.otto@rackspace.com" target="_blank">adrian.otto@rackspace.com</a>><br>
<b>Reply-To: </b>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<b>Date: </b>Friday, March 4, 2016 at 12:48 PM<br>
<b>To: </b>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<b>Subject: </b>Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Hongbin, </span>
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">To be clear, this pursuit is not about what OS options cloud operators can select. We will be offering a method of choice. It has to do with what we plan to build comprehensive testing for,</span><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">This is easy. Once we build comprehensive tests for the first OS, just re-run it for other OS(s).</span><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">and the implications that has on our pace of feature development. My guidance here is that we resist the temptation to create a system with more permutations than we can possibly support. The
relation between bay node OS, Heat Template, Heat Template parameters, COE, and COE dependencies (could-init, docker, flannel, etcd, etc.) are multiplicative in nature. From the mid cycle, it was clear to me that:</span><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">1) We want to test at least one OS per COE from end-to-end with comprehensive functional tests.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">2) We want to offer clear and precise integration points to allow cloud operators to substitute their own OS in place of whatever one is the default for the given COE.</span><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">A COE shouldn’t have a default necessarily that locks out other defaults. Magnum devs are the experts in how these systems operate, and as such need to take on the responsibility of the implementation
for multi-os support.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">3) We want to control the total number of configuration permutations to simplify our efforts as a project. We agreed that gate testing all possible permutations is intractable.</span><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">I disagree with this point, but I don't have the bandwidth available to prove it ;) </span><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
That’s exactly my point. It takes a chunk of human bandwidth to carry that responsibility. If we had a system engineer assigned from each of the various upstream OS distros working with Magnum, this would not be a big deal. Expecting our current contributors
to support a variety of OS variants is not realistic.<o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">You have my promise to support an additional OS for 1 or 2 popular COEs.</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
Change velocity among all the components we rely on has been very high. We see some of our best contributors frequently sidetracked in the details of the distros releasing versions of code that won’t work with ours. We want to upgrade a component to add a new
feature, but struggle to because the new release of the distro that offers that component is otherwise incompatible. Multiply this by more distros, and we expect a real problem.<o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">At Magnum upstream, the overhead doesn’t seem to come from the OS. Perhaps, that is specific to your downstream?</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt;margin-left:72.0pt">
<o:p> </o:p></p>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">There is no harm if you have 30 gates running the various combinations. Infrastructure can handle the load. Whether devs have the cycles to make a fully bulletproof gate is the question I think
you answered with the word intractable.</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
Actually, our existing gate tests are really stressing out our CI infra. At least one of the new infrastructure providers that replaced HP have equipment that runs considerably slower. For example, our swam functional gate now frequently fails because it can’t
finish before the allowed time limit of 2 hours where it could finish substantially faster before. If we expanded the workload considerably, we might quickly work to the detriment of other projects by perpetually clogging the CI pipelines. We want to be a
good citizen of the openstack CI community. Testing configuration of third party software should be done with third party CI setups. That’s one of the reasons those exist. Ideally, each would be maintained by those who have a strategic (commercial?) interest
in support for that particular OS.<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">I can tell you in Kolla we spend a lot of cycles just getting basic gating going of building containers and then deploying them. We have even made inroads into testing the deployment. We do
CentOS, Ubuntu, and soon Oracle Linux, for both source and binary and build and deploy. Lots of gates and if they aren't green we know the patch is wrong.</span><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
Remember that COE’s are tested on nova instances within heat stacks. Starting lots of nova instances within devstack in the gates is problematic. We are looking into using a libvirt-lxc instance type from nova instead of a libvirt-kvm instance to help alleviate
this. Until then, limiting the scope of our gate tests is appropriate. We will continue our efforts to make them reasonably efficient.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
Thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
Adrian<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
</div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Regards</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">-steve</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Note that it will take a thoughtful approach (subject to discussion) to balance these interests. Please take a moment to review the interest above. Do you or others disagree with these? If so,
why?</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Adrian</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">On Mar 4, 2016, at 9:09 AM, Hongbin Lu <<a href="mailto:hongbin.lu@huawei.com" target="_blank">hongbin.lu@huawei.com</a>> wrote:</span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I don’t think there is any consensus on supporting single distro. There are multiple disagreements on this thread, including several senior team members and a project co-founder.
This topic should be re-discussed (possibly at the design summit).</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Best regards,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hongbin</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Corey O'Brien [<a href="mailto:coreypobrien@gmail.com" target="_blank">mailto:coreypobrien@gmail.com</a>] <br>
<b>Sent:</b> March-04-16 11:37 AM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro</span><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
I don't think anyone is saying that code should somehow block support for multiple distros. The discussion at midcycle was about what the we should gate on and ensure feature parity for as a team. Ideally, we'd like to get support for every distro, I think,
but no one wants to have that many gates. Instead, the consensus at the midcycle was to have 1 reference distro for each COE, gate on those and develop features there, and then have any other distros be maintained by those in the community that are passionate
about them.<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
The issue also isn't about how difficult or not it is. The problem we want to avoid is spending precious time guaranteeing that new features and bug fixes make it through multiple distros. <o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
Corey<o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
On Fri, Mar 4, 2016 at 11:18 AM Steven Dake (stdake) <<a href="mailto:stdake@cisco.com" target="_blank"><span style="color:purple">stdake@cisco.com</span></a>> wrote:<o:p></o:p></p>
</div>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">My position on this is simple.</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Operators are used to using specific distros because that is what they used in the 90s,and the 00s, and the 10s. Yes, 25 years of using a distro, and you learn it inside and out. This means
you don't want to relearn a new distro, especially if your an RPM user going to DEB or a DEB user going to RPM. These are non-starter options for operators, and as a result, mean that distro choice is a must. Since CoreOS is a new OS in the marketplace,
it may make sense to consider placing it in "third" position in terms of support.</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Besides that problem, various distribution companies will only support distros running in Vms if it matches the host kernel, which makes total sense to me. This means on an Ubuntu host if I
want support I need to run Ubuntu vms, on a RHEL host I want to run RHEL vms, because, hey, I want my issues supported.</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">For these reasons and these reasons alone, there is no good rationale to remove multi-distro support from Magnum. All I've heard in this thread so far is "its too hard". Its not too hard,
especially with Heat conditionals making their way into Mitaka.</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">Regards</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif"">-steve</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From: </span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Hongbin Lu <<a href="mailto:hongbin.lu@huawei.com" target="_blank"><span style="color:purple">hongbin.lu@huawei.com</span></a>><br>
<b>Reply-To: </b>"<a href="mailto:openstack-dev@lists.openstack.org" target="_blank"><span style="color:purple">openstack-dev@lists.openstack.org</span></a>" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank"><span style="color:purple">openstack-dev@lists.openstack.org</span></a>><br>
<b>Date: </b>Monday, February 29, 2016 at 9:40 AM<br>
<b>To: </b>"<a href="mailto:openstack-dev@lists.openstack.org" target="_blank"><span style="color:purple">openstack-dev@lists.openstack.org</span></a>" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank"><span style="color:purple">openstack-dev@lists.openstack.org</span></a>><br>
<b>Subject: </b>[openstack-dev] [magnum] Discussion of supporting single/multiple OS distro</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="color:#1F497D">Hi team,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="color:#1F497D">This is a continued discussion from a review [1]. Corey O'Brien suggested to have Magnum support a single OS distro (Atomic). I disagreed. I think we should bring the discussion to here to get broader set of inputs. </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i>Corey O'Brien</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i>From the midcycle, we decided we weren't going to continue to support 2 different versions of the k8s template. Instead, we were going to maintain the Fedora Atomic version of k8s and remove the coreos templates from the tree. I don't think we should continue
to develop features for coreos k8s if that is true.</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i>In addition, I don't think we should break the coreos template by adding the trust token as a heat parameter.</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i> </i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i>Hongbin Lu</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i>I was on the midcycle and I don't remember any decision to remove CoreOS support. Why you want to remove CoreOS templates from the tree. Please note that this is a very big decision and please discuss it with the team thoughtfully and make sure everyone
agree.</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i> </i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i>Corey O'Brien</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i>Removing the coreos templates was a part of the COE drivers decision. Since each COE driver will only support 1 distro+version+coe we discussed which ones to support in tree. The decision was that instead of trying to support every distro and every version
for every coe, the magnum tree would only have support for 1 version of 1 distro for each of the 3 COEs (swarm/docker/mesos). Since we already are going to support Atomic for swarm, removing coreos and keeping Atomic for kubernetes was the favored choice.</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i> </i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i>Hongbin Lu</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i>Strongly disagree. It is a huge risk to support a single distro. The selected distro could die in the future. Who knows. Why make Magnum take this huge risk? Again, the decision of supporting single distro is a very big decision. Please bring it up to the
team and have it discuss thoughtfully before making any decision. Also, Magnum doesn't have to support every distro and every version for every coe, but should support *more than one* popular distro for some COEs (especially for the popular COEs).</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i> </i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i>Corey O'Brien</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i>The discussion at the midcycle started from the idea of adding support for RHEL and CentOS. We all discussed and decided that we wouldn't try to support everything in tree. Magnum would provide support in-tree for 1 per COE and the COE driver interface would
allow others to add support for their preferred distro out of tree.</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i> </i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i>Hongbin Lu</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<i>I agreed the part that "we wouldn't try to support everything in tree". That doesn't imply the decision to support single distro. Again, support single distro is a huge risk. Why make Magnum take this huge risk?</i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="color:#1F497D">[1] <a href="https://review.openstack.org/#/c/277284/" target="_blank"><span style="color:purple">https://review.openstack.org/#/c/277284/</span></a></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="color:#1F497D">Best regards,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="color:#1F497D">Hongbin</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe" target="_blank"><span style="color:purple">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</span></a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank"><span style="color:purple">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</span></a><o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:9.0pt;font-family:"Helvetica","sans-serif"">__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org" target="_blank">
OpenStack-dev-request@lists.openstack.org</a>?subject:unsubscribe<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></span><o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<span style="font-size:10.5pt;font-family:"Calibri","sans-serif""> </span><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org" target="_blank">
OpenStack-dev-request@lists.openstack.org</a>?subject:unsubscribe<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt">
<o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-left:36.0pt">__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</body>
</html>