Hi everyone, yes we also thought that, but it seems to be different. I'm unsure about the performance needs, but I think with quorum queues they might actually be possible. We did not yet try this out on our side, since we first wanted to gather ideas why it is implemented the current way. -- Felix Huettner
-----Original Message----- From: Arnaud Morin <arnaud.morin@gmail.com> Sent: Monday, September 5, 2022 8:04 PM To: Felix Hüttner <felix.huettner@mail.schwarz> Cc: openstack-discuss <openstack-discuss@lists.openstack.org> Subject: Re: [ops][largescale-sig] rabbitmq and the durability of reply_ queues
Hello,
Thank you for bringing that up. I was thinking (but it seems I was wrong) that the fanout/reply queues were used for a one shot message/reply.
Besides that, I am also wondering how much cpu power we will need to replicate such queues?
I guess I can can do the math from what we are currently running in production :)
Have you tried on a big cluster on your side?
Cheers,
Arnaud
Hi everyone,
the largescale-sig currently recommends the following rabbitmq policy [1]: '^(?!(amq\.)|(.*_fanout_)|(reply_)).*'.
If I found the history of this setting correctly, it resulted from this mailinglist
On 02.09.22 - 08:19, Felix Hüttner wrote: thread [2] which came to the conclusion:
* use durable-queue and replication for long-running queues/exchanges * use non-durable-queue without replication for short (fanout, reply_) queues
However I am unsure if reply_ queues are actually shorted lived than the
"long-running" ones.
If we take the example of the cinder-scheduler we have the following queues:
* cinder-scheduler: Tied to the lifetime of the whole cinder-scheduler cluster; currently durable + replicated * cinder-scheduler.myhostname: Tied to the lifetime of the cinder-scheduler service, but not removed when the service stops; currently durable + replicated * cinder-scheduler_fanout_somerandomid: Tied to the lifetime of the cinder-scheduler service, but removed when the service stops; currently neither durable nor replicated * reply_somerandomid: Tied to the lifetime of the cinder-scheduler service, but removed when the service stops; currently neither durable nor replicated
If I read got all of the above correctly then reply_ and fanout queues are actually not shorter lived than the normal queue to send requests to that service. The only difference is that we remove them when the service itself stops.
Therefor I want to propose the question if we should not also make these queues durable and replicated.
Thank you
[1]:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs
.openstack.org%2Flarge- scale%2Fother%2Frabbitmq.html%23pattern&dat
a=05%7C01%7C%7C1ae015a4eafb4646f9c108da8f690c71%7Cd04f47175a6e4b 98b3f9
6918e0385f4c%7C0%7C0%7C637979978586225879%7CUnknown%7CTWFpbG Zsb3d8eyJW
IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C 3000%
7C%7C%7C&sdata=FjBYOdVXeQABeSUxpg%2B0a6YkXJWN%2B0dBebbh 4CPwuok%3D&
amp;reserved=0 [2]: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist s.openstack.org%2Fpipermail%2Fopenstack-discuss%2F2020- August%2F016562
.html&data=05%7C01%7C%7C1ae015a4eafb4646f9c108da8f690c71%7Cd 04f471
75a6e4b98b3f96918e0385f4c%7C0%7C0%7C637979978586225879%7CUnknow n%7CTWF
pbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV CI6M
n0%3D%7C3000%7C%7C%7C&sdata=%2FrJpfgfuTTyglQS1AH3pz8mZF3IT cetAv78%
2BQ1%2BORNk%3D&reserved=0
-- Felix Huettner
Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für die Verwertung durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht der vorgesehene Empfänger sein, setzen Sie den Absender bitte unverzüglich in Kenntnis und löschen diese E Mail. Hinweise zum Datenschutz finden Sie hier<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2F www.datenschutz.schwarz%2F&data=05%7C01%7C%7C1ae015a4eafb4 646f9c108da8f690c71%7Cd04f47175a6e4b98b3f96918e0385f4c%7C0%7C0%7C 637979978586225879%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM DAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7 C&sdata=%2BTyQF5u6KtsGThSfYJEHV333WFxgnXXZjDD0dIAo%2B4E%3 D&reserved=0>.
Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für die Verwertung durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht der vorgesehene Empfänger sein, setzen Sie den Absender bitte unverzüglich in Kenntnis und löschen diese E Mail. Hinweise zum Datenschutz finden Sie hier<https://www.datenschutz.schwarz>.