<div dir="ltr"><div dir="auto">Okay, let's say you found a critical bug and you have already sent a patch for review.</div><div dir="auto"> This - as you know can take quite a long time - gate broken, unit tests are not working etc etc but this is still regular fix and merged in other versions. </div><div dir="auto">Moreover upstream guys need to release a new pip package and amend upper constraints.<div dir="auto"><br></div><div dir="auto">But you need to fix your production now ..how can you do it in kolla now ? You cannot. </div><div dir="auto"><br></div><div dir="auto">As I said before as an example .. Oslo messaging is broken from xena to antelope, if your rabbitmq will go down on this versions ..your clients will not connect again to cluster - this is critical bug.</div><div>So, let's check how upstream guys dealing with it : </div><div><br></div><div>1, Patch applied upstream ? </div><div> - master - OK merged - <a href="https://review.opendev.org/c/openstack/oslo.messaging/+/866617">https://review.opendev.org/c/openstack/oslo.messaging/+/866617</a></div><div> - antelope - OK merged - <a href="https://review.opendev.org/c/openstack/oslo.messaging/+/883533">https://review.opendev.org/c/openstack/oslo.messaging/+/883533</a></div><div> - zed - they don't care - <a href="https://review.opendev.org/c/openstack/oslo.messaging/+/883537">https://review.opendev.org/c/openstack/oslo.messaging/+/883537</a></div><div> - xena - they don't care - <a href="https://review.opendev.org/c/openstack/oslo.messaging/+/883539">https://review.opendev.org/c/openstack/oslo.messaging/+/883539</a></div><div> - yoga - they don't care - <a href="https://review.opendev.org/c/openstack/oslo.messaging/+/883538">https://review.opendev.org/c/openstack/oslo.messaging/+/883538</a></div><div><br></div><div>2, Okay, antelope merged , is the new version released as this is a critical bug ?</div><div> - no , they again don't care -</div><div><br></div><div> * 0602d1a1 (HEAD -> master, origin/master, origin/HEAD) Increase ACK_REQUEUE_EVERY_SECONDS_MAX to exceed default kombu_reconnect_delay (Andrew Bogott, 5 weeks ago - 2023-04-20 15:27:58 -0500)</div>* fd2381c7 (tag: 14.3.0) Disable greenthreads for RabbitDriver "listen" connections (Arnaud Morin, 3 months ago - 2023-03-03 11:24:27 +0100)</div><div dir="auto"><br></div><div dir="auto">Last version is 14.3.0 and fix is still not released in pypi repo.</div><div dir="auto"><br></div><div> Other versions ? check 1.</div><div><br></div><div>Let's check how ubuntu handled this problem : </div><div><br></div><div><pre style="color:rgb(0,0,0);white-space:pre-wrap">python-oslo.messaging (12.13.0-0ubuntu1.1) jammy; urgency=medium
* d/gbp.conf: Create stable/yoga branch.
* d/p/revert-limit-maximum-timeout-in-the-poll-loop.patch: This reverts
an upstream patch that is preventing active/active rabbitmq from
failing over when a node goes down (LP: #1993149).
-- Corey Bryant <<a href="mailto:corey.bryant@canonical.com">corey.bryant@canonical.com</a>> Thu, 20 Oct 2022 15:48:16 -0400</pre><pre style="color:rgb(0,0,0);white-space:pre-wrap"><br></pre><pre style="color:rgb(0,0,0);white-space:pre-wrap">They patched the buggy version !! Kolla dropped binary builds ...so you can't install dependencies from apt repository where it is patched, and you don't have a way how to patch </pre><pre style="color:rgb(0,0,0);white-space:pre-wrap">your python library.</pre><pre style="color:rgb(0,0,0);white-space:pre-wrap"><br></pre><pre style="color:rgb(0,0,0);white-space:pre-wrap">Patching is normal way how to fix a problem, you don't have always option to bump version, you need patch code and kolla just don't have this option.</pre><pre style="color:rgb(0,0,0);white-space:pre-wrap"><br></pre><pre style="color:rgb(0,0,0);white-space:pre-wrap"><br></pre></div><div dir="auto"><br><div> - </div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, May 24, 2023, 08:41 Maksim Malchuk <<a href="mailto:maksim.malchuk@gmail.com" target="_blank">maksim.malchuk@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">The correct and only way to apply patches on images - is build them from source. You should patch the code, not binary files.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, May 24, 2023 at 3:32 AM Satish Patel <<a href="mailto:satish.txt@gmail.com" rel="noreferrer" target="_blank">satish.txt@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">This is very interesting that there is no standard or best way to patch images. Everyone uses their own way to handle it. Now I am very curious to see how folks running it in production with patching and building images. I am about to deploy kolla on production and trying to learn all best practices from experts. </div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, May 23, 2023 at 7:47 PM Michal Arbet <<a href="mailto:michal.arbet@ultimum.io" rel="noreferrer" target="_blank">michal.arbet@ultimum.io</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">How ? <div dir="auto"><br></div><div dir="auto">Say to me please ? How you can patch oslo.messaging except template override </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, May 23, 2023, 16:17 Danny Webb <<a href="mailto:Danny.Webb@thehutgroup.com" rel="noreferrer" target="_blank">Danny.Webb@thehutgroup.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
You can already do this with the kolla image builder which seems to me to be a much better solution than patching containers post creation. </div>
<div id="m_266455687142931101m_4599015773162883858m_5979146071302695043m_-8614601304107976373m_1401864542583147811appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="m_266455687142931101m_4599015773162883858m_5979146071302695043m_-8614601304107976373m_1401864542583147811divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Michal Arbet <<a href="mailto:michal.arbet@ultimum.io" rel="noreferrer noreferrer" target="_blank">michal.arbet@ultimum.io</a>><br>
<b>Sent:</b> 23 May 2023 13:01<br>
<b>To:</b> <a href="mailto:openstack-discuss@lists.openstack.org" rel="noreferrer noreferrer" target="_blank">openstack-discuss@lists.openstack.org</a> <<a href="mailto:openstack-discuss@lists.openstack.org" rel="noreferrer noreferrer" target="_blank">openstack-discuss@lists.openstack.org</a>><br>
<b>Subject:</b> Re: [kolla] How to patch images during build</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>
<div style="color:rgb(0,0,0);font-size:12px;text-align:left;font-family:Helvetica,Arial,sans-serif">
<strong><br>
<div style="background-color:rgb(244,127,83);font-family:sans-serif"><span style="font-size:14pt"><strong>CAUTION:</strong></span>
<span style="font-size:11pt;font-weight:400">This email originates from outside THG</span>
</div>
</strong><br>
</div>
<hr>
</div>
I am very glad that someone asked for an option to patch kolla images.
<div>I've already proposed patches for kolla here [1] and here [2]. </div>
<div>But unfortunately I didn't get that many votes to merge into master and I abandoned this.</div>
<div><br>
</div>
<div>[1] <a href="https://review.opendev.org/c/openstack/kolla/+/829296" rel="noreferrer noreferrer" target="_blank">https://review.opendev.org/c/openstack/kolla/+/829296</a></div>
<div>[2] <a href="https://review.opendev.org/c/openstack/kolla/+/829295" rel="noreferrer noreferrer" target="_blank">https://review.opendev.org/c/openstack/kolla/+/829295</a></div>
<div><br>
</div>
<div>With these above patches you can patch files inside every container.</div>
<div>Maybe we can discuss this again ??</div>
<div><br>
</div>
<div>For example now xena, yoga, zed, antelope has oslo.messaging broken :</div>
<div><br>
</div>
<div><a href="https://bugs.launchpad.net/oslo.messaging/+bug/2019978" rel="noreferrer noreferrer" target="_blank">https://bugs.launchpad.net/oslo.messaging/+bug/2019978</a><br>
</div>
<div>fixed by</div>
<div><a href="https://review.opendev.org/c/openstack/oslo.messaging/+/866617" rel="noreferrer noreferrer" target="_blank">https://review.opendev.org/c/openstack/oslo.messaging/+/866617</a><br>
</div>
<div><br>
</div>
<div>As I am using my kolla patches in my downstream kolla git repo i've only created patches/ directory and place fix for openstack-base container :)</div>
<div><br>
</div>
<div>patches/<br>
patches/openstack-base<br>
patches/openstack-base/series<br>
patches/openstack-base/fix-rabbitmq-issue-opendev-883538.patch<br>
</div>
<div><br>
</div>
<div>But, you still can use template-override <a href="https://docs.openstack.org/kolla/latest/admin/image-building.html" rel="noreferrer noreferrer" target="_blank">https://docs.openstack.org/kolla/latest/admin/image-building.html</a> .</div>
<div><br>
</div>
<div>Thanks</div>
<div><br>
</div>
<div>
<div>
<div>
<div dir="ltr">
<div dir="ltr">
<div style="margin:0px;padding:0px 0px 20px;width:1503px;color:rgb(34,34,34);font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:medium">
<div style="margin:8px 0px 0px;padding:0px">
<div dir="ltr"><span>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr" style="font-family:Arial,Helvetica,sans-serif;font-size:small"><span style="color:rgb(136,136,136);vertical-align:top;line-height:18px;font-size:11pt;font-family:Arial,Helvetica,Geneva;font-weight:bold">Michal Arbet<br>
</span><span style="color:rgb(136,136,136);font-size:10pt;vertical-align:top;line-height:18px;font-family:Arial,Helvetica,Geneva">Openstack Engineer<br>
<img width="200" height="81" src="https://www.google.com/a/ultimum.io/images/logo.gif"><br>
Ultimum Technologies a.s.<br>
Na Poříčí 1047/26, 11000 Praha 1<br>
Czech Republic<br>
<br>
<a value="+420608314961" style="color:rgb(17,85,204)" rel="noreferrer noreferrer">+420 604 228 897</a> <br>
<a href="mailto:michal.arbet@ultimum.io" style="color:rgb(17,85,204)" rel="noreferrer noreferrer" target="_blank">michal.arbet@ultimum.io</a><br>
</span><font color="#1155cc" face="Arial, Helvetica, Geneva" style="font-size:12.8px"><span style="font-size:13.3333px;line-height:18px"><u><a href="https://ultimum.io/" style="color:rgb(17,85,204)" rel="noreferrer noreferrer" target="_blank">https://ultimum.io</a></u></span></font><br>
</div>
<div dir="ltr" style="font-family:Arial,Helvetica,sans-serif;font-size:small"><font color="#1155cc" face="Arial, Helvetica, Geneva" style="font-size:12.8px"><span style="font-size:13.3333px;line-height:18px"><br>
</span></font></div>
<div style="font-family:Arial,Helvetica,sans-serif;font-size:small"><font color="#666666"><a href="https://www.linkedin.com/company/ultimum-technologies" style="color:rgb(17,85,204)" rel="noreferrer noreferrer" target="_blank">LinkedIn</a> | <a href="https://twitter.com/ultimumtech" style="color:rgb(17,85,204)" rel="noreferrer noreferrer" target="_blank">Twitter</a> | <a href="https://www.facebook.com/ultimumtechnologies/timeline" style="color:rgb(17,85,204)" rel="noreferrer noreferrer" target="_blank">Facebook</a></font></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</span></div>
</div>
<div>
<div></div>
<div>
<div></div>
</div>
</div>
<div></div>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
</div>
<br>
<div>
<div dir="ltr">st 17. 5. 2023 v 20:19 odesílatel Jeremy Stanley <<a href="mailto:fungi@yuggoth.org" rel="noreferrer noreferrer" target="_blank">fungi@yuggoth.org</a>> napsal:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
On 2023-05-17 21:02:02 +0300 (+0300), Maksim Malchuk wrote:<br>
> Yes, you can do that, but note bene mitaka not supported.<br>
[...]<br>
<br>
Not only unsupported, but the stable/mitaka branch of<br>
openstack/keystone was deleted when it reached EOL in 2017. You may<br>
instead want to specify `reference = mitaka-eol` (assuming Git tags<br>
also work there). That should get you the final state of the<br>
stable/mitaka branch prior to its deletion.<br>
-- <br>
Jeremy Stanley<br>
</blockquote>
</div>
</div>
</div>
</blockquote></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div><span>-- </span><br><div dir="ltr"><div dir="ltr"><div style="font-family:arial;font-size:small">Regards,<br></div><div style="font-family:arial;font-size:small">Maksim Malchuk</div><div style="font-family:arial;font-size:small"><br></div></div></div>
</blockquote></div>