Open Stack

Hello,

I first would like to apologize for not keeping everybody in the loop but I’ve
been very busy with a lot of work.

I just wanted to give a quick update on some potential progress.

First off, the Oslo meeting on the 19th of September ([1] log here) kicked us into a direction
of how we could use the official nats.py python client [2] to talk to NATS as we quickly ruled
out the nats-python client [3] as it’s pretty unmaintained and incomplete.

We did however get a working POC driver [4] (see patchset 14) with [3] that confirmed we could get
a Devstack job working even though the driver missed a lot of critical functionality such as error handling, retries, TLS etc etc.

Since the nats.py client [2] is asyncio based we investigated a path forward that meant where we could utilize
asyncio in the oslo.messaging library and expose an API up to projects consuming oslo.messaging.

We decided to do some POC patches into oslo.messaging and using the Blazar project as a testing ground [4]. We very quickly
found ourselves digging into oslo.service to implement asyncio, checking SQLAlchemy for support, API frameworks would need
asyncio support, moving from WSGI to an ASGI compatible server, duplicating code for supporting sync and async code paths and more.

We came to the conclussion that introducing a NATS oslo.messaging driver meant requiring us to implement asyncio across our ecosystem
which would take an massive effort. Some good came out of it in form of a patch [5] further improving the consistency of the oslo.messaging API, we
also gained a lot of insight in the future if we would like to start introducing asyncio functionality into oslo projects to support it for the future.

Now moving forward to today.

I’ve talked with the maintainer of nats.py and the company behind him to investigate if we can include a synchronous code path into
that Python library to be used in a future revision of a oslo.messaging NATS driver. Hopefully we’ll have some more progress to report
on that in the beginning of next year.

Feel free to reach out to me (here or on IRC, my nick is tobias-urdin) if you want to know more, have any questions or
want to get involved in anything I’ve mentioned above!

Best regards
Tobias

[1] https://meetings.opendev.org/meetings/oslo/2022/oslo.2022-09-19-15.12.log.txt
[2] https://github.com/nats-io/nats.py
[3] https://github.com/Gr1N/nats-python
[4] https://review.opendev.org/q/topic:asyncio-nats
[5] https://review.opendev.org/c/openstack/oslo.messaging/+/862419

On 12 Sep 2022, at 09:05, Tobias Urdin <tobias.urdin at binero.com> wrote:

Hello,

Since I haven’t heard any objections we’ll move the meeting to the Oslo
meeting on the 19th of September. See you there! [1]

[1] https://wiki.openstack.org/wiki/Meetings/Oslo

Best regards
Tobias

On 8 Sept 2022, at 15:54, Herve Beraud <hberaud at redhat.com<mailto:hberaud at redhat.com>> wrote:

WFM

Le jeu. 8 sept. 2022 à 15:16, Tobias Urdin <tobias.urdin at binero.com<mailto:tobias.urdin at binero.com>> a écrit :
Hello,

I don’t have anything against moving to the 19th of September meeting if that’s OK
with everybody else, I propose we do that.

Let’s see if we get some feedback against the proposed new date otherwise we’ll move it.

Best regards

On 8 Sept 2022, at 08:41, Radosław Piliszek <radoslaw.piliszek at gmail.com<mailto:radoslaw.piliszek at gmail.com>> wrote:

Hi Tobias et al,

That Monday I cannot attend due to other work commitments. Could we schedule it for the following week?

Cheers,
Radek

On Tue, Sep 6, 2022, 18:33 Tobias Urdin <tobias.urdin at binero.com<mailto:tobias.urdin at binero.com>> wrote:
Great! I’ve updated the agenda with NATS driver and eventlet future topic https://wiki.openstack.org/wiki/Meetings/Oslo#Agenda_for_Next_Meeting

See you in the next meeting on 12th of September 15:00 UTC hopefully that
will give us all some time to read through the following material:

- Current POC code https://review.opendev.org/c/openstack/oslo.messaging/+/848338
- Devstack plugin https://github.com/tobias-urdin/devstack-plugin-nats
- Discussion on IRC on 2nd September about below links https://meetings.opendev.org/irclogs/%23openstack-oslo/%23openstack-oslo.2022-09-02.log.html<https://meetings.opendev.org/irclogs/#openstack-oslo/%23openstack-oslo.2022-09-02.log.html>

Links to old material:
- The old spec that we can work upon https://review.opendev.org/c/openstack/oslo-specs/+/692784
- Old driver implementation for above old spec https://review.opendev.org/c/openstack/oslo.messaging/+/680629
- Old asyncio planning https://wiki.openstack.org/wiki/Oslo/blueprints/asyncio
- Old asyncio executor in oslo.messaging https://blueprints.launchpad.net/oslo.messaging/+spec/asyncio-executor
- Old messaging API mail thread https://lists.openstack.org/pipermail/openstack-dev/2013-May/009784.html

Looking forward to talk to you all next monday!

Best regards
Tobias

On 5 Sept 2022, at 12:48, Herve Beraud <hberaud at redhat.com<mailto:hberaud at redhat.com>> wrote:



Le lun. 5 sept. 2022 à 10:00, Tobias Urdin <tobias.urdin at binero.com<mailto:tobias.urdin at binero.com>> a écrit :
Thanks! It seems that we have some traction here to get started in a more structured manner.

I’m on PTO today but back tomorrow, what do people feel about scheduling a recurring meeting about this topic?

The weekly Oslo meeting could host the conversation through a dedicated topic.
https://wiki.openstack.org/wiki/Meetings/Oslo

You could modify the meeting agenda to add this topic.
Daniel Bengtsson (damani) is our meeting facilitator. Do not hesitate to ping him to discuss the meeting management.

I still
need to read through the old spec and old implementation of NATS, and I am a bit restricted on time currently but we
could brainstorm in that meeting and get us started on a new/updated the other spec.

Should also note that, to anybody interested, that the topic is a little bit more broader than just a new messaging driver
as it includes planning and/or working around the usage of eventlet etc, if you are interested in that topic please also
reach out to us!

Best regards
Tobias

> On 2 Sept 2022, at 10:35, Felix Hüttner <felix.huettner at mail.schwarz<mailto:felix.huettner at mail.schwarz>> wrote:
>
> +1, we would also be interested in helping out. Let me know where we can help out with reviews, writing code, testing, etc.
>
> --
> Felix Huettner
>
>> +1, I'm also interested in this topic. I did some tests with nats-py to see how it works and I'm volunteering to go further with this technology.
>> Let me know if you need reviews, help etc...
>>
>> Le jeu. 1 sept. 2022 à 15:50, Tobias Urdin <mailto:tobias.urdin at binero.com<mailto:tobias.urdin at binero.com>> a écrit :
>> Hello Kai,
>>
>> Great to hear that you're interested!
>> I'm currently just testing this as a POC directly in devstack so there is real-world testing (not should it yet I think, needs more development first).
>>
>> Best regards
>> Tobias
>>
>>> On 30 Aug 2022, at 18:32, Kai Bojens <mailto:kb at kbojens.de<mailto:kb at kbojens.de>> wrote:
>>>
>>> Am 29.08.22 um 15:46 schrieb Tobias Urdin:
>>>
>>>> If anybody, or any company, out there would be interested in collaborating in a project to bring this support and maintain it feel free to
>>>> reach out. I'm hoping somebody will bite but atleast I've put it out there for all of you.
>>>
>>> Hi,
>>> I am very much interested to take a closer look at your work and maybe contribute to it. Although I'm working with OpenStack for my employer at the moment, I'd do this in my spare time as I'm not sure that I can convince him to add another staging system with a full OpenStack installation just for developing a RabbitMQ replacement. That's not on our agenda as we are mostly users and not developers of OpenStack.
>>>
>>> As I'm pretty new to the messaging topic and had just heard of NATS and your idea, I'll first would have to dive into NATS and then I would take a closer look at your code and maybe try to create some documentation or tests. So, I can't promise anything but as I said I'm very interested in your approach as I also see the massive load from RabbitMQ.
>>>
>>> This brings me to the most important question: Where do you run and test your code?
>>>
>>> Greetings,
>>> Kai
>
> Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für die Verwertung durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht der vorgesehene Empfänger sein, setzen Sie den Absender bitte unverzüglich in Kenntnis und löschen diese E Mail. Hinweise zum Datenschutz finden Sie hier<https://www.datenschutz.schwarz<https://www.datenschutz.schwarz/>>.



--
Hervé Beraud
Senior Software Engineer at Red Hat
irc: hberaud
https://github.com/4383/
https://twitter.com/4383hberaud




--
Hervé Beraud
Senior Software Engineer at Red Hat
irc: hberaud
https://github.com/4383/
https://twitter.com/4383hberaud



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20221208/17695afc/attachment-0001.htm>