<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div><br>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Adrian Otto <<a href="mailto:adrian.otto@rackspace.com">adrian.otto@rackspace.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Friday, March 4, 2016 at 12:48 PM<br>
<span style="font-weight:bold">To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro<br>
</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Hongbin,
<div class=""><br class="">
</div>
<div class="">To be clear, this pursuit is not about what OS options cloud operators can select. We will be offering a method of choice. It has to do with what we plan to build comprehensive testing for, and the implications that has on our pace of feature
development. My guidance here is that we resist the temptation to create a system with more permutations than we can possibly support. The relation between bay node OS, Heat Template, Heat Template parameters, COE, and COE dependencies (could-init, docker,
flannel, etcd, etc.) are multiplicative in nature. From the mid cycle, it was clear to me that:</div>
<div class=""><br class="">
</div>
<div class="">1) We want to test at least one OS per COE from end-to-end with comprehensive functional tests.</div>
<div class="">2) We want to offer clear and precise integration points to allow cloud operators to substitute their own OS in place of whatever one is the default for the given COE.</div>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
<div>A COE shouldn’t have a default necessarily that locks out other defaults. Magnum devs are the experts in how these systems operate, and as such need to take on the responsibility of the implementation for multi-os support.</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div class="">3) We want to control the total number of configuration permutations to simplify our efforts as a project. We agreed that gate testing all possible permutations is intractable.</div>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
<div>I disagree with this point, but I don't have the bandwidth available to prove it ;) There is no harm if you have 30 gates running the various combinations. Infrastructure can handle the load. Whether devs have the cycles to make a fully bulletproof
gate is the question I think you answered with the word intractable.</div>
<div><br>
</div>
<div>I can tell you in Kolla we spend a lot of cycles just getting basic gating going of building containers and then deploying them. We have even made inroads into testing the deployment. We do CentOS, Ubuntu, and soon Oracle Linux, for both source and
binary and build and deploy. Lots of gates and if they aren't green we know the patch is wrong.</div>
<div><br>
</div>
<div>Regards</div>
<div>-steve</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div class=""><br class="">
</div>
<div class="">Note that it will take a thoughtful approach (subject to discussion) to balance these interests. Please take a moment to review the interest above. Do you or others disagree with these? If so, why?</div>
<div class=""><br class="">
</div>
<div class="">Adrian</div>
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Mar 4, 2016, at 9:09 AM, Hongbin Lu <<a href="mailto:hongbin.lu@huawei.com" class="">hongbin.lu@huawei.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="WordSection1" style="page: WordSection1; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">I don’t think there is any consensus on supporting single distro. There are multiple disagreements on this thread, including several senior team members and a
project co-founder. This topic should be re-discussed (possibly at the design summit).<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Best regards,<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Hongbin<o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""> </span></div>
<div style="border-style: solid none none; border-top-color: rgb(181, 196, 223); border-top-width: 1pt; padding: 3pt 0cm 0cm;" class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<b class=""><span lang="EN-US" style="font-size: 10pt; font-family: Tahoma, sans-serif;" class="">From:</span></b><span lang="EN-US" style="font-size: 10pt; font-family: Tahoma, sans-serif;" class=""><span class="Apple-converted-space"> </span>Corey O'Brien
[<a href="mailto:coreypobrien@gmail.com" class="">mailto:coreypobrien@gmail.com</a>]<span class="Apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="Apple-converted-space"> </span>March-04-16 11:37 AM<br class="">
<b class="">To:</b><span class="Apple-converted-space"> </span>OpenStack Development Mailing List (not for usage questions)<br class="">
<b class="">Subject:</b><span class="Apple-converted-space"> </span>Re: [openstack-dev] [magnum] Discussion of supporting single/multiple OS distro<o:p class=""></o:p></span></div>
</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
I don't think anyone is saying that code should somehow block support for multiple distros. The discussion at midcycle was about what the we should gate on and ensure feature parity for as a team. Ideally, we'd like to get support for every distro, I think,
but no one wants to have that many gates. Instead, the consensus at the midcycle was to have 1 reference distro for each COE, gate on those and develop features there, and then have any other distros be maintained by those in the community that are passionate
about them.<o:p class=""></o:p></div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
The issue also isn't about how difficult or not it is. The problem we want to avoid is spending precious time guaranteeing that new features and bug fixes make it through multiple distros. <o:p class=""></o:p></div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
Corey<o:p class=""></o:p></div>
</div>
</div>
</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
On Fri, Mar 4, 2016 at 11:18 AM Steven Dake (stdake) <<a href="mailto:stdake@cisco.com" style="color: purple; text-decoration: underline;" class="">stdake@cisco.com</a>> wrote:<o:p class=""></o:p></div>
</div>
<blockquote style="border-style: none none none solid; border-left-color: rgb(204, 204, 204); border-left-width: 1pt; padding: 0cm 0cm 0cm 6pt; margin-left: 4.8pt; margin-right: 0cm;" class="">
<div class="">
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 10.5pt; font-family: Calibri, sans-serif;" class="">My position on this is simple.<o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 10.5pt; font-family: Calibri, sans-serif;" class=""> </span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 10.5pt; font-family: Calibri, sans-serif;" class="">Operators are used to using specific distros because that is what they used in the 90s,and the 00s, and the 10s. Yes, 25 years of using a distro, and you learn it inside and out.
This means you don't want to relearn a new distro, especially if your an RPM user going to DEB or a DEB user going to RPM. These are non-starter options for operators, and as a result, mean that distro choice is a must. Since CoreOS is a new OS in the marketplace,
it may make sense to consider placing it in "third" position in terms of support.<o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 10.5pt; font-family: Calibri, sans-serif;" class=""> </span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 10.5pt; font-family: Calibri, sans-serif;" class="">Besides that problem, various distribution companies will only support distros running in Vms if it matches the host kernel, which makes total sense to me. This means on an Ubuntu
host if I want support I need to run Ubuntu vms, on a RHEL host I want to run RHEL vms, because, hey, I want my issues supported.<o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 10.5pt; font-family: Calibri, sans-serif;" class=""> </span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 10.5pt; font-family: Calibri, sans-serif;" class="">For these reasons and these reasons alone, there is no good rationale to remove multi-distro support from Magnum. All I've heard in this thread so far is "its too hard". Its not
too hard, especially with Heat conditionals making their way into Mitaka.<o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 10.5pt; font-family: Calibri, sans-serif;" class=""> </span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 10.5pt; font-family: Calibri, sans-serif;" class="">Regards<o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 10.5pt; font-family: Calibri, sans-serif;" class="">-steve<o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 10.5pt; font-family: Calibri, sans-serif;" class=""> </span></div>
</div>
<div style="border-style: solid none none; border-top-color: rgb(181, 196, 223); border-top-width: 1pt; padding: 3pt 0cm 0cm;" class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<b class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">From:<span class="Apple-converted-space"> </span></span></b><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Hongbin Lu <<a href="mailto:hongbin.lu@huawei.com" target="_blank" style="color: purple; text-decoration: underline;" class="">hongbin.lu@huawei.com</a>><br class="">
<b class="">Reply-To:<span class="Apple-converted-space"> </span></b>"<a href="mailto:openstack-dev@lists.openstack.org" target="_blank" style="color: purple; text-decoration: underline;" class="">openstack-dev@lists.openstack.org</a>" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank" style="color: purple; text-decoration: underline;" class="">openstack-dev@lists.openstack.org</a>><br class="">
<b class="">Date:<span class="Apple-converted-space"> </span></b>Monday, February 29, 2016 at 9:40 AM<br class="">
<b class="">To:<span class="Apple-converted-space"> </span></b>"<a href="mailto:openstack-dev@lists.openstack.org" target="_blank" style="color: purple; text-decoration: underline;" class="">openstack-dev@lists.openstack.org</a>" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank" style="color: purple; text-decoration: underline;" class="">openstack-dev@lists.openstack.org</a>><br class="">
<b class="">Subject:<span class="Apple-converted-space"> </span></b>[openstack-dev] [magnum] Discussion of supporting single/multiple OS distro<o:p class=""></o:p></span></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="font-size: 10.5pt; font-family: Calibri, sans-serif;" class=""> </span></div>
</div>
<blockquote style="border-style: none none none solid; border-left-color: rgb(181, 196, 223); border-left-width: 4.5pt; padding: 0cm 0cm 0cm 4pt; margin-left: 3.75pt; margin-right: 0cm;" class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="color: rgb(31, 73, 125);" class="">Hi team,</span><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="color: rgb(31, 73, 125);" class=""> </span><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="color: rgb(31, 73, 125);" class="">This is a continued discussion from a review [1]. Corey O'Brien suggested to have Magnum support a single OS distro (Atomic). I disagreed. I think we should bring the discussion to here to get broader set of inputs.<span class="Apple-converted-space"> </span></span><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="color: rgb(31, 73, 125);" class=""> </span><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class="">Corey O'Brien</i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class="">From the midcycle, we decided we weren't going to continue to support 2 different versions of the k8s template. Instead, we were going to maintain the Fedora Atomic version of k8s and remove the coreos templates from the tree. I don't think we should
continue to develop features for coreos k8s if that is true.</i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class="">In addition, I don't think we should break the coreos template by adding the trust token as a heat parameter.</i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class=""> </i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class="">Hongbin Lu</i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class="">I was on the midcycle and I don't remember any decision to remove CoreOS support. Why you want to remove CoreOS templates from the tree. Please note that this is a very big decision and please discuss it with the team thoughtfully and make sure
everyone agree.</i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class=""> </i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class="">Corey O'Brien</i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class="">Removing the coreos templates was a part of the COE drivers decision. Since each COE driver will only support 1 distro+version+coe we discussed which ones to support in tree. The decision was that instead of trying to support every distro and every
version for every coe, the magnum tree would only have support for 1 version of 1 distro for each of the 3 COEs (swarm/docker/mesos). Since we already are going to support Atomic for swarm, removing coreos and keeping Atomic for kubernetes was the favored
choice.</i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class=""> </i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class="">Hongbin Lu</i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class="">Strongly disagree. It is a huge risk to support a single distro. The selected distro could die in the future. Who knows. Why make Magnum take this huge risk? Again, the decision of supporting single distro is a very big decision. Please bring it
up to the team and have it discuss thoughtfully before making any decision. Also, Magnum doesn't have to support every distro and every version for every coe, but should support *more than one* popular distro for some COEs (especially for the popular COEs).</i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class=""> </i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class="">Corey O'Brien</i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class="">The discussion at the midcycle started from the idea of adding support for RHEL and CentOS. We all discussed and decided that we wouldn't try to support everything in tree. Magnum would provide support in-tree for 1 per COE and the COE driver interface
would allow others to add support for their preferred distro out of tree.</i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class=""> </i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class="">Hongbin Lu</i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<i class="">I agreed the part that "we wouldn't try to support everything in tree". That doesn't imply the decision to support single distro. Again, support single distro is a huge risk. Why make Magnum take this huge risk?</i><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="color: rgb(31, 73, 125);" class=""> </span><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="color: rgb(31, 73, 125);" class="">[1]<span class="Apple-converted-space"> </span><a href="https://review.openstack.org/#/c/277284/" target="_blank" style="color: purple; text-decoration: underline;" class="">https://review.openstack.org/#/c/277284/</a></span><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="color: rgb(31, 73, 125);" class=""> </span><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="color: rgb(31, 73, 125);" class="">Best regards,</span><span style="" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
<span style="color: rgb(31, 73, 125);" class="">Hongbin</span><span style="" class=""><o:p class=""></o:p></span></div>
</div>
</div>
</div>
</blockquote>
</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">
__________________________________________________________________________<br class="">
OpenStack Development Mailing List (not for usage questions)<br class="">
Unsubscribe:<span class="Apple-converted-space"> </span><a href="http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe" target="_blank" style="color: purple; text-decoration: underline;" class="">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br class="">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank" style="color: purple; text-decoration: underline;" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p class=""></o:p></div>
</blockquote>
</div>
</div>
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">__________________________________________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">OpenStack
Development Mailing List (not for usage questions)</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Unsubscribe:
<a href="mailto:OpenStack-dev-request@lists.openstack.org" class="">OpenStack-dev-request@lists.openstack.org</a>?subject:unsubscribe</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></span></div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</blockquote>
</span>
</body>
</html>