Open Stack

Okay, let's say you found a critical bug and you have already sent a patch
for review.
 This - as you know can take quite a long time - gate broken, unit tests
are not working etc etc but this is still regular fix and merged in other
versions.
Moreover upstream guys need to release a new pip package and amend upper
constraints.

But you need to fix your production now ..how can you do it in kolla now ?
You cannot.

As I said before as an example .. Oslo messaging is broken from xena to
antelope, if your rabbitmq will go down on this versions ..your clients
will not connect again to cluster - this is critical bug.
So, let's check how upstream guys dealing with it :

1, Patch applied upstream ?
    - master - OK merged -
https://review.opendev.org/c/openstack/oslo.messaging/+/866617
    - antelope - OK merged -
https://review.opendev.org/c/openstack/oslo.messaging/+/883533
    - zed - they don't care -
https://review.opendev.org/c/openstack/oslo.messaging/+/883537
    - xena - they don't care -
https://review.opendev.org/c/openstack/oslo.messaging/+/883539
    - yoga - they don't care -
https://review.opendev.org/c/openstack/oslo.messaging/+/883538

2, Okay, antelope merged , is the new version released as this is a
critical bug ?
    - no , they again don't care -

 * 0602d1a1 (HEAD -> master, origin/master, origin/HEAD) Increase
ACK_REQUEUE_EVERY_SECONDS_MAX to exceed default kombu_reconnect_delay
(Andrew Bogott, 5 weeks ago - 2023-04-20 15:27:58 -0500)
* fd2381c7 (tag: 14.3.0) Disable greenthreads for RabbitDriver "listen"
connections (Arnaud Morin, 3 months ago - 2023-03-03 11:24:27 +0100)

Last version is 14.3.0 and fix is still not released in pypi repo.

   Other versions ? check 1.

Let's check how ubuntu handled this problem :

python-oslo.messaging (12.13.0-0ubuntu1.1) jammy; urgency=medium

  * d/gbp.conf: Create stable/yoga branch.
  * d/p/revert-limit-maximum-timeout-in-the-poll-loop.patch: This reverts
    an upstream patch that is preventing active/active rabbitmq from
    failing over when a node goes down (LP: #1993149).

 -- Corey Bryant <corey.bryant at canonical.com>  Thu, 20 Oct 2022 15:48:16 -0400


They patched the buggy version !! Kolla dropped binary builds ...so
you can't install dependencies from apt repository where it is
patched, and you don't have a way how to patch

your python library.


Patching is normal way how to fix a problem, you don't have always
option to bump version, you need patch code and kolla just don't have
this option.




    -

On Wed, May 24, 2023, 08:41 Maksim Malchuk <maksim.malchuk at gmail.com> wrote:

> The correct and only way to apply patches on images - is build them from
> source. You should patch the code, not binary files.
>
> On Wed, May 24, 2023 at 3:32 AM Satish Patel <satish.txt at gmail.com> wrote:
>
>> This is very interesting that there is no standard or best way to patch
>> images. Everyone uses their own way to handle it. Now I am very curious to
>> see how folks running it in production with patching and building images. I
>> am about to deploy kolla on production and trying to learn all best
>> practices from experts.
>>
>> On Tue, May 23, 2023 at 7:47 PM Michal Arbet <michal.arbet at ultimum.io>
>> wrote:
>>
>>> How ?
>>>
>>> Say to me please ? How you can patch oslo.messaging except template
>>> override
>>>
>>> On Tue, May 23, 2023, 16:17 Danny Webb <Danny.Webb at thehutgroup.com>
>>> wrote:
>>>
>>>> You can already do this with the kolla image builder which seems to me
>>>> to be a much better solution than patching containers post creation.
>>>> ------------------------------
>>>> *From:* Michal Arbet <michal.arbet at ultimum.io>
>>>> *Sent:* 23 May 2023 13:01
>>>> *To:* openstack-discuss at lists.openstack.org <
>>>> openstack-discuss at lists.openstack.org>
>>>> *Subject:* Re: [kolla] How to patch images during build
>>>>
>>>>
>>>> * CAUTION: This email originates from outside THG *
>>>> ------------------------------
>>>> I am very glad that someone asked for an option to patch kolla images.
>>>> I've already proposed patches for kolla here [1] and here [2].
>>>> But unfortunately I didn't get that many votes to merge into master and
>>>> I abandoned this.
>>>>
>>>> [1] https://review.opendev.org/c/openstack/kolla/+/829296
>>>> [2] https://review.opendev.org/c/openstack/kolla/+/829295
>>>>
>>>> With these above patches you can patch files inside every container.
>>>> Maybe we can discuss this again ??
>>>>
>>>> For example now xena, yoga, zed, antelope has oslo.messaging broken :
>>>>
>>>> https://bugs.launchpad.net/oslo.messaging/+bug/2019978
>>>> fixed by
>>>> https://review.opendev.org/c/openstack/oslo.messaging/+/866617
>>>>
>>>> As I am using my kolla patches in my downstream kolla git repo i've
>>>> only created patches/ directory and place fix for openstack-base container
>>>> :)
>>>>
>>>> patches/
>>>> patches/openstack-base
>>>> patches/openstack-base/series
>>>> patches/openstack-base/fix-rabbitmq-issue-opendev-883538.patch
>>>>
>>>> But, you still can use template-override
>>>> https://docs.openstack.org/kolla/latest/admin/image-building.html .
>>>>
>>>> Thanks
>>>>
>>>> Michal Arbet
>>>> Openstack Engineer
>>>>
>>>> Ultimum Technologies a.s.
>>>> Na Poříčí 1047/26, 11000 Praha 1
>>>> Czech Republic
>>>>
>>>> +420 604 228 897
>>>> michal.arbet at ultimum.io
>>>> *https://ultimum.io <https://ultimum.io/>*
>>>>
>>>> LinkedIn <https://www.linkedin.com/company/ultimum-technologies> |
>>>> Twitter <https://twitter.com/ultimumtech> | Facebook
>>>> <https://www.facebook.com/ultimumtechnologies/timeline>
>>>>
>>>>
>>>> st 17. 5. 2023 v 20:19 odesílatel Jeremy Stanley <fungi at yuggoth.org>
>>>> napsal:
>>>>
>>>> On 2023-05-17 21:02:02 +0300 (+0300), Maksim Malchuk wrote:
>>>> > Yes, you can do that, but note bene mitaka not supported.
>>>> [...]
>>>>
>>>> Not only unsupported, but the stable/mitaka branch of
>>>> openstack/keystone was deleted when it reached EOL in 2017. You may
>>>> instead want to specify `reference = mitaka-eol` (assuming Git tags
>>>> also work there). That should get you the final state of the
>>>> stable/mitaka branch prior to its deletion.
>>>> --
>>>> Jeremy Stanley
>>>>
>>>>
>
> --
> Regards,
> Maksim Malchuk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230524/98c65eb5/attachment.htm>