<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 9, 2017 at 11:46 AM, mathieu bultel <span dir="ltr"><<a href="mailto:mbultel@redhat.com" target="_blank">mbultel@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span>
<div class="m_4456247069834977178m_2173774349036483790moz-cite-prefix">On 05/08/2017 01:45 PM, Marios Andreou
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi folks, after some discussion locally with
colleagues about improving the upgrades experience, one of the
items that came up was pre-upgrade and update validations. I
took an AI to look at the current status of tripleo-validations
[0] and posted a simple WIP [1] intended to be run before an
undercloud update/upgrade and which just checks service status.
It was pointed out by shardy that for such checks it is better
to instead continue to use the per-service manifests where
possible like [2] <span style="font-size:12.8px">for example
where we check status before N..O major upgrade. There may
still be some undercloud specific validations that we can land
into the tripleo-validations repo (thinking about things like
the neutron networks/ports, validating the current nova nodes
state etc?).</span></div>
</blockquote></span>
Yes, I think a bunch of validation: db states, services states,
network connectivity (external, internal)<span><br>
<blockquote type="cite">
<div dir="ltr">
<div><span style="font-size:12.8px"><br>
</span></div>
<div><span style="font-size:12.8px">So do folks have any
thoughts about this subject - for example the kinds of
things we should be checking - Steve said he had some
reviews in progress for collecting the overcloud ansible
puppet/docker config into an ansible playbook that the
operator can invoke for upgrade of the 'manual' nodes (for
example compute in the N..O workflow) - the point being that
we can add more per-service ansible validation tasks into
the service manifests for execution when the play is run by
the operator - but I'll let Steve point at and talk about
those. </span></div>
<div><br>
</div>
</div>
</blockquote></span>
I have a WIP review about that [1], but i need to revisit it a bit,
to add a part into the mistral workflow (Im also writing a POC to
create a mistral workbook for major upgrade and validate minor/major
upgrade before starting [2], I have a third one in progress, not
pushed yet, to implement the major upgrade option in the cli):<br>
<pre><a class="m_4456247069834977178m_2173774349036483790moz-txt-link-freetext" href="https://review.openstack.org/444224" target="_blank">[1] https://review.openstack.org/4<wbr>44224</a>
[2] <a class="m_4456247069834977178m_2173774349036483790moz-txt-link-freetext" href="https://review.openstack.org/#/c/462961" target="_blank">https://review.openstack.org/#<wbr>/c/462961</a>
</pre>
<br></div></blockquote><div><br></div><div>ack I had a pass at those when you first sent this - will look again. I think our discussion have highlighted a few things </div><div><br></div><div>* lack of tripleo-common/client support for upgrades workflow (e.g. so we can kill the -e for every invocation)</div><div><br></div><div>* better - more pre-upgrade/update validations - especially undercloud doesn't have much/any (already have some pre-upgrade validations) also minor update doesn't have much/any converage .</div><div><br></div><div>* improving the 'manual node upgrade' - instead of running upgrade-non-controller.sh use a playbook. that can be run by the operator (or even automated?)</div><div><br></div><div>* better logging tracking of current upgrades step/progress - when failures happen</div><div><br></div><div>* related to ^^^ when upgrade/update completes on current node writing a /etc/tripleorelease or somesuch to say 'mitaka' or 'ocata' or whatever the version just been upgraded to, is. </div><div><br></div><div>I recall Emilien suggesting blueprint for tracking we should discuss some more and get these down as one or more blueprints probably - assuming folks agree with that list and what we can add/remove/change - lets work together to split the blueprints if you like but lets discuss them a bit here first/or not more comments lets see :)</div><div><br></div><div>thanks</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
<blockquote type="cite"><span>
<div dir="ltr">
<div>cheers, marios</div>
<div><br>
</div>
<div>
<div>
<div>
<div>[0] <a href="https://github.com/openstack/tripleo-validations" target="_blank">https://github.com/openstack/t<wbr>ripleo-validations</a> </div>
<div>[1] <a href="https://review.openstack.org/#/c/462918/" target="_blank">https://review.openstack.o<wbr>rg/#/c/462918/</a></div>
</div>
<div>[2] <span style="font-size:12.8px"> </span><a href="https://github.com/openstack/" target="_blank"></a><a class="m_4456247069834977178m_2173774349036483790moz-txt-link-freetext" href="https://github.com/openstack/" target="_blank">https://github.com/openst<wbr>ack/</a>tripleo-heat-templates/<wbr>blob/stable/ocata/puppet/<wbr>services/neutron-api.yaml#<wbr>L197 </div>
</div>
</div>
</div>
<br>
<fieldset class="m_4456247069834977178m_2173774349036483790mimeAttachmentHeader"></fieldset>
<br>
</span><pre>______________________________<wbr>______________________________<wbr>______________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="m_4456247069834977178m_2173774349036483790moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.op<wbr>enstack.org?subject:unsubscrib<wbr>e</a>
<a class="m_4456247069834977178m_2173774349036483790moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k-dev</a>
</pre>
</blockquote>
<p><br>
</p>
</div>
<br>______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.op<wbr>enstack.org?subject:unsubscrib<wbr>e</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k-dev</a><br>
<br></blockquote></div><br></div></div>