<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:宋体;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@宋体";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:宋体;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="ZH-CN" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D">I agree this is reasonable to support all these cases in “cold upgrades” but in supports-rolling-upgrade (live upgrade in another word) case it is
different and complicated and not necessary, <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D">During rolling upgrade, we will have old/new services co-existed, and we need to make services compatible which need some extra code work and this
is the main purpose of spec [1]. And as far as I can see, we are not allowed to skip over releases when rolling upgrading. So my point is support name release is enough.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D">1. Because even if we want to support major number release, admins have to upgrade from 5.0 -> 6.0 then 6.0 -> 7.0 in Ruby’s case of 5.0.0, 5.1.0
== Mitaka, 5.2.0, 6.0.0, 6.1.0, 7.0.0, 7.1.0, 7.2.0 == Newton. And we might have a higher release frequency in the future. So it’s too much work for upgrade a service every six months.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D">2. As we usually rolling upgrade the whole cloud, not for ironic only. For example, other projects will upgrade from Mitaka to Netwon, there is not
much sense to upgrade Ironic from 5.0 -> 6.0 only. <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D">At last, we should add a multi-node grenade CI to test the rolling upgrade mechanism was not broken. Here is my suggestion, we will have two nodes,
Node A and Node B. Node A will run both of last named release of ironic-api and ironic-conductor and node B will run last named release ironic-conductor only. Multi-node grenade CI will only upgrade Node A and then we can test the interaction between new SHA
of ironic-api and new SHA/last named release of ironic-conductor still works.<a name="_MailEndCompose"> This should also apply to stable branch.<o:p></o:p></a></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D">B.R<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D">Tan<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[1] </span><a href="https://review.openstack.org/299245" target="_blank"><span lang="EN-US">https://review.openstack.org/299245</span></a><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Devananda van der Veen [mailto:devananda.vdv@gmail.com]
<br>
<b>Sent:</b> Wednesday, April 20, 2016 5:12 AM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org><br>
<b>Subject:</b> Re: [openstack-dev] [ironic] upgrade support between which versions of ironic?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">Thanks for starting the thread, Ruby.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US">We need to first establish a grenade job to test "cold upgrades" and assert the supports-upgrade tag. I believe Ironic meets all the criteria for that tag except:<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span lang="EN-US">- having a job that tests it (so, you know, it might be broken and I might be wrong)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">- having operator documentation describing the process (it should be here: <a href="http://docs.openstack.org/developer/ironic/deploy/upgrade-guide.html">http://docs.openstack.org/developer/ironic/deploy/upgrade-guide.html</a>
) but all we have are release-specific upgrade notes.<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">I think all of the scenario you outline are valid upgrade paths for an operator, and we should try to allow all of them to work. However, some of them can be covered by one test case, and you also missed some things I
think need to be covered. Also, I'm interpreting the word "master" in your scenarios to indicate the proposed change to our master branch, since we do pre-merge testing....<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">So, here are the test cases I think we need to cover:<o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US"><br>
=== run on proposed changes to master ===<o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">F. current master to new SHA<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">We need to ensure that we can upgrade master to the code being proposed. You listed this last, but I think it's actually the most important one.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">D. last named release to new SHA<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">E. last numbered release to new SHA<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">Because we cut new releases from master, this is the basis of testing the upgrade between sequential (named or numbered) releases before we cut a new (named or numbered) release, and is our most important test to ensure
that we don't break most operators. Based on the user survey, most operators are using named releases, so if we are resource constrained, I would prefer to cover (D) before (E)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">=== run on proposed changes to a stable branch ===<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">A. stable/N-1 -> new SHA -> [ stable/N+1 or current master]<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">We don't need to test upgrades between two named releases (eg. Liberty -> Mitaka) every time we land a new patch on the master branch, but we do need to test any time we land a change on a stable branch. Changes to the
most recent stable branch should be upgrade-tested to current master, whereas changes to any stable branch prior to that should get tested to the subsequent sequential release.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">Eg, a backport to stable/liberty should trigger an upgrade test for both (stable/kilo -> newpatch) and (newpatch -> stable/mitaka), whereas a backport to stable/mitaka should trigger a test for (stable/liberty -> newpatch)
and (newpatch -> master)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US"><br>
Once we've done that, then yes, I agree we should also work towards asserting supports-rolling-upgrade. That will require a partial upgrade job, eg. where we run >1 instance of the API and Conductor services and upgrade some of them.<o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">Regards,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">Devananda<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span lang="EN-US">On Tue, Apr 19, 2016 at 12:52 PM, Ruby Loo <<a href="mailto:opensrloo@gmail.com" target="_blank">opensrloo@gmail.com</a>> wrote:<o:p></o:p></span></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span lang="EN-US">Hi,<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span lang="EN-US">Currently, ironic doesn't support ("live", "online", "rolling", or minimal-downtime) upgrades between named versions of ironic. (Where "named version" is the final release or stable release
that is associated with a development cycle). So for example, Liberty -> Mitaka release.<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US">We've been working towards that, and have a spec [1] and a design session [2] at the Austin Summit. Upon reading the spec, I started to wonder -- what do we mean/want, when we talk about supporting upgrades. Do we want
to support:<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US">A. sequential named releases, eg. Mitaka -> Newton<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US">B. sequential major releases, eg 4.x -> 5.0; 5.0 -> 6.1<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">C. sequential minor releases, eg 5.1 -> 5.2<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">D. last named release to master<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">E. last release (major or minor) to master<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">F. some-SHA-more-recent-than-last-named to master. This could be some numbered (major or minor) release.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US">Keep in mind that ironic may release any number of numbered releases between two named releases, so eg. 4.3.0, 5.0.0, 5.1.0 == Mitaka, 5.2.0, 6.0.0, 6.1.0, 7.0.0, 7.1.0, 7.2.0 == Newton.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US">Note that there are two governance tags: supports-upgrade[3] and supports-rolling-upgrade[4], and I believe our goal is for supports-rolling-upgrade.<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span lang="EN-US">I think and hope that we can all agree that A. is a must :)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">What constitutes a major release versus a minor release may have implications in this, but that should be in a separate discussion.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span lang="EN-US"><br>
What do people think?<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US">--ruby<o:p></o:p></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US"><br>
[1] <a href="https://review.openstack.org/299245" target="_blank">https://review.openstack.org/299245</a><br>
[2] <a href="https://www.openstack.org/summit/austin-2016/summit-schedule/events/9267" target="_blank">
https://www.openstack.org/summit/austin-2016/summit-schedule/events/9267</a><br>
[3] <a href="https://github.com/openstack/governance/blob/master/reference/tags/assert_supports-upgrade.rst" target="_blank">
https://github.com/openstack/governance/blob/master/reference/tags/assert_supports-upgrade.rst</a><br>
[4] <a href="https://github.com/openstack/governance/blob/master/reference/tags/assert_supports-rolling-upgrade.rst" target="_blank">
https://github.com/openstack/governance/blob/master/reference/tags/assert_supports-rolling-upgrade.rst</a><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span lang="EN-US"><br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></span></p>
</blockquote>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
</div>
</body>
</html>