December 2021 - openstack-discuss

[Interop] No WG meeting on Dec 23 and 30.
by Arkady Kanevsky 11 Dec '21

11 Dec '21

Happy holidays. -- Arkady Kanevsky, Ph.D. Phone: 972 707-6456 Corporate Phone: 919 729-5744 ext. 8176456

3 2

[all][tc] What's happening in Technical Committee: summary 15th Oct, 21: Reading: 5 min
by Ghanshyam Mann 10 Dec '21

10 Dec '21

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * TC this week IRC meeting held on Oct 14th Thursday. * Most of the meeting discussions are summarized below (Completed or in-progress activities section). Meeting full logs are available @ - https://meetings.opendev.org/meetings/tc/2021/tc.2021-10-14-15.00.log.html * Next week's meeting is cancelled as we are meeting in PTG. We will have the next IRC meeting on Oct 28th, Thursday 15:00 UTC, feel free the topic on agenda[1] by Oct 27th. 2. What we completed this week: ========================= * None in this week. 3. Activities In progress: ================== TC Tracker for Xena cycle ------------------------------ * TC is using the etherpad[2] for Xena cycle working item. We will be checking it in PTG * Current status is: 9 completed, 3 to-be-discussed in PTG, 1 in-progress Open Reviews ----------------- * Six open reviews for ongoing activities[3]. New project 'Skyline' proposal ------------------------------------ * You might be aware of this new dashboard proposal in the previous month's discussion. * A new project 'Skyline: an OpenStack dashboard optimized by UI and UE' is now proposed in governance to be an official OpenStack project[4]. * Skyline team is planning to meet in PTG on Tue, Wed and Thu at 5UTC, please ask your queries or have feedback/discussion with the team next week. Place to maintain the external hosted ELK, E-R, O-H services ------------------------------------------------------------------------- * We had a final discussion or I will say just a status update on this which was mentioned in last week's email summary[5]. * Now onwards, discussion and migration work will be done in TACT SIG (#openstack-infra IRC channel). Add project health check tool ----------------------------------- * No updates on this, we will continue discussing it in PTG for the next steps on this and what to do with TC liaison things. * Meanwhile, we are reviewing Rico proposal on collecting stats tools [6]. Stable Core team process change --------------------------------------- * Current proposal is under review[7]. Feel free to provide early feedback if you have any. Call for 'Technical Writing' SIG Chair/Maintainers ---------------------------------------------------------- * As agreed in last week's TC meeting, we will be moving this SIG work towards TC. * TC members have been added to the core members list in the SIG repos. * We will be discussing where to move the training repos/work in PTG. TC tags analysis ------------------- * Operator feedback is asked on open infra newsletter too, and we will continue the discussion in PTG and will take the final decision based on feedback we receive, if any[9]. Complete the policy pop up team ---------------------------------------- * Policy pop team has served its purpose and we have new RBAC as one of the the community-wide goal for the Yoga cycle. * We are marking this popup team as completed[10]. Project updates ------------------- * Retiring js-openstack-lib [11] Yoga release community-wide goal ----------------------------------------- * Please add the possible candidates in this etherpad [12]. * Current status: "Secure RBAC" is selected for Yoga cycle[13]. PTG planning ---------------- * We will be meeting in PTG next week, please check the details in this etherpad [14] * Do not forget to join the TC+community leaders sessions on Monday, Oct 18 15 UTC - 17 UTC. Test support for TLS default: ---------------------------------- * Rico has started a separate email thread over testing with tls-proxy enabled[15], we encourage projects to participate in that testing and help to enable the tls-proxy in gate testing. 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[16]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [17] 3. Office hours: The Technical Committee offers a weekly office hour every Tuesday at 0100 UTC [18] 4. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://etherpad.opendev.org/p/tc-xena-tracke [3] https://review.opendev.org/q/projects:openstack/governance+status:open [4] https://review.opendev.org/c/openstack/governance/+/814037 [5]http://lists.openstack.org/pipermail/openstack-discuss/2021-October/02525… [6] https://review.opendev.org/c/openstack/governance/+/810037 [7] https://review.opendev.org/c/openstack/governance/+/810721 [9] https://governance.openstack.org/tc/reference/tags/index.html [10] https://review.opendev.org/c/openstack/governance/+/814186 [11] https://review.opendev.org/c/openstack/governance/+/798540 [12] https://review.opendev.org/c/openstack/governance/+/807163 [13] https://etherpad.opendev.org/p/y-series-goals [14] https://etherpad.opendev.org/p/tc-yoga-ptg [15] http://lists.openstack.org/pipermail/openstack-discuss/2021-June/023000.html [16] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [17] http://eavesdrop.openstack.org/#Technical_Committee_Meeting [18] http://eavesdrop.openstack.org/#Technical_Committee_Office_hours -gmann

9 25

[all][tc] What's happening in Technical Committee: summary 10th Dec, 21: Reading: 10 min
by Ghanshyam Mann 10 Dec '21

10 Dec '21

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * Most of the meeting discussions are summarized below (Completed or in-progress activities section). Meeting full logs are available @ https://meetings.opendev.org/meetings/tc/2021/tc.2021-12-09-15.00.log.html * Next week's meeting is on IRC on 16th Dec, Thursday 15:00 UTC, feel free the topic on the agenda[1] by 15th Dec. 2. What we completed this week: ========================= * 'Consistent and Secure Default RBAC' is selected as community-wide goal [2] * Add NVidia vGPU plugin charm to OpenStack charms[3] 3. Activities In progress: ================== TC Tracker for Yoga cycle ------------------------------ * This etherpad includes the Yoga cycle targets/working items for TC[4]. Open Reviews ----------------- * 5 open reviews for ongoing activities[5]. OpenStack Pain points discussion ---------------------------------------- We had a separate video call on Wed 8th Dec and discussed four projects' pain points[6] and also added the tag to categorize them. Details are sent by Rico in a separate email[7] and we will continue the discussion in Jan. Stay tuned on that ML thread to know the next meeting details. Skyline as an official project --------------------------------- Discussion going on in ML thread[8] regarding the things/improvement to do in Skyline project for various things to make it consistent with other OpenStack projects. Skyline team has confirmed to work on all those points on the governance patch[9] which is under review by TC. TC position on release cadence ------------------------------------- Discussion is in-progress in ML thread[10]. No other updates from TC on this and we will set up a separate call to continue the discussion sometime in Jan (after the Christmas holidays). Yoga testing runtime ------------------------- Governance patch is up for review on keeping python3.6 support in Yoga[11] Fixing Zuul config error -------------------------- If your project is listed in zuul config error, please start planning to fix those[12] Community-wide goal updates ------------------------------------ You might have seen my email[13] about goal status and planning the work, below is the current status: * Selected (active) goals[14]: ** Migrate from oslo.rootwrap to oslo.privsep ** Consistent and Secure Default RBAC * Goal proposal under review: ** FIPS compatibility and compliance [15]. Adjutant need maintainers and PTLs ------------------------------------------- We are still waiting to hear from Braden, Albert on permission to work on this project[16]. Project updates ------------------- * Retire js-openstack-lib [17] 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[18]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [19] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rb… [3] https://review.opendev.org/c/openstack/governance/+/819856 [4] https://etherpad.opendev.org/p/tc-yoga-tracker [5] https://review.opendev.org/q/projects:openstack/governance+status:open [6] https://etherpad.opendev.org/p/pain-point-elimination [7] http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026245… [8] http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026206… [9] https://review.opendev.org/c/openstack/governance/+/814037 [10] http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025684… [11] https://review.opendev.org/c/openstack/governance/+/820195 [12] https://etherpad.opendev.org/p/zuul-config-error-openstack [13] http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026218… [14] https://governance.openstack.org/tc/goals/selected/index.html [15] https://review.opendev.org/c/openstack/governance/+/816587 [16] http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025786… [17] https://review.opendev.org/c/openstack/governance/+/798540 [18] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [19] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann

1 0

Force specific Region for resource creation
by Ryan Bannon 09 Dec '21

09 Dec '21

Hello all, Question: is there any way to associate a Region to a Project? Would this require custom code/development in OpenStack? (By my understanding, a Project lives outside the concept of a Region, and resources associated with a Project can target any Region.) Tx, Ryan

1 0

[neutron] Drivers meeting - Friday 10.12.2021 - cancelled
by Lajos Katona 09 Dec '21

09 Dec '21

Hi Neutron Drivers! Due to the lack of agenda, let's cancel tomorrow's drivers meeting. We have only one RFE from last week: https://bugs.launchpad.net/neutron/+bug/1953170 ([RFE] Unify quota engine API) but we discussed that last week (see [0]). See You on the meeting next week. [0]: https://meetings.opendev.org/meetings/neutron_drivers/2021/neutron_drivers.… Lajos Katona (lajoskatona)

1 0

[mistral] How to import openstack actions from mistral-extra
by Oleksandr Mykhalskyi 09 Dec '21

09 Dec '21

Hello, In release Pike, I was able to import openstack actions to mistral DB using "mistral-db-manage --config-file /etc/mistral/mistral.conf populate" As I understood, from release Ussuri, all openstack actions were moved to separate mistral-extra library. But it`s unclear, how to import openstack actions to DB now. Command "mistral-db-manage populate" creates only few std* actions. I didn`t find any info in mistral documentation https://docs.openstack.org/mistral/latest/user/index.html about these changes. In https://github.com/openstack/mistral-extra also there is no useful readme. Can you please clarify, how to work with mistral-extra library and openstack actions in latest openstack releases? Thank you! Oleksandr Mykhalskyi, System Engineer, IT Servers and Services, LAUT Netcracker Technology +38 (044) 238-8727 ext 6404 | office ________________________________ The information transmitted herein is intended only for the person or entity to which it is addressed and may contain confidential, proprietary and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.

1 0

RE: [ops] [kolla] RabbitMQ High Availability
by Braden, Albert 09 Dec '21

09 Dec '21

I read this with great interest because we are seeing this issue. Questions: 1. We are running kola-ansible Train, and our RMQ version is 3.7.23. Should we be upgrading our Train clusters to use 3.8.x? 2. Document [2] recommends policy '^(?!(amq\.)|(.*_fanout_)|(reply_)).*'. I don't see this in our ansible playbooks, nor in any of the config files in the RMQ container. What would this look like in Ansible, and what should the resulting container config look like? 3. It appears that we are not setting "amqp_durable_queues = True". What does this setting look like in Ansible, and what file does it go into? Does anyone have a sample set of RMQ config files that they can share? It looks like my Outlook has ruined the link; reposting: [2] https://wiki.openstack.org/wiki/Large_Scale_Configuration_Rabbit -----Original Message----- From: Arnaud Morin <arnaud.morin(a)gmail.com> Sent: Monday, November 29, 2021 2:04 PM To: Bogdan Dobrelya <bdobreli(a)redhat.com> Cc: DHilsbos(a)performair.com; openstack-discuss(a)lists.openstack.org Subject: [EXTERNAL] Re: [ops]RabbitMQ High Availability Caution: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Hi, After a talk on this ml (which starts at [1]), we endup building a documentation with Large Scale group. The doc is accessible at [2]. Hope this will help. [1] http://secure-web.cisco.com/1gFccuTyEVGnFd9aBOZ-RTPG0hbVIPGAbuLBNnoXP4onSZG… [2] https://secure-web.cisco.com/1OtQ3pcnPPBNwevAFxS8yOS2xFlkHo0tY4SmkFE-wpAU_Y… On 24.11.21 - 11:31, Bogdan Dobrelya wrote: > On 11/24/21 12:34 AM, DHilsbos(a)performair.com wrote: > > All; > > > > In the time I've been part of this mailing list, the subject of RabbitMQ high availability has come up several times, and each time specific recommendations for both Rabbit and Open Stack are provided. I remember it being an A or B kind of recommendation (i.e. configure Rabbit like A1, and Open Stack like A2, OR configure Rabbit like B1, and Open Stack like B2). > > There is no special recommendations for rabbitmq setup for openstack, > but probably a few, like instead of putting it behind a haproxy, or the > like, list the rabbit cluster nodes in the oslo messaging config > settings directly. Also, it seems that durable queues makes a very > little sense for highly ephemeral RPC calls, just by design. I would > also add that the raft quorum queues feature of rabbitmq >=3.18 does > neither fit well into the oslo messaging design for RPC calls. > > A discussable and highly opinionated thing is also configuring > ha/mirror queue policy params for queues used for RPC calls vs > broad-casted notifications. > > And my biased personal humble recommendation is: use the upstream OCF RA > [0][1], if configuring rabbitmq cluster by pacemaker. > > [0] https://secure-web.cisco.com/1N1wD9gW7NZho0LdTVNuiU2ZIB7NW-eJMfDgVzBH3D3E6U… > > [1] > https://secure-web.cisco.com/1iDK1NnL9JTkQqkpBda06xTQNrWY2W0pVOTDwUoadfQbSX… > > > > > Unfortunately, I can't find the previous threads on this topic. > > > > Does anyone have this information, that they would care to share with me? > > > > Thank you, > > > > Dominic L. Hilsbos, MBA > > Vice President - Information Technology > > Perform Air International Inc. > > DHilsbos(a)PerformAir.com > > www.PerformAir.com > > > > > > > > > -- > Best regards, > Bogdan Dobrelya, > Irc #bogdando > >

5 4

Re: [ops] [kolla] RabbitMQ High Availability
by Bogdan Dobrelya 09 Dec '21

09 Dec '21

Please see inline >> I read this with great interest because we are seeing this issue. Questions: >> >> 1. We are running kola-ansible Train, and our RMQ version is 3.7.23. Should we be upgrading our Train clusters to use 3.8.x? >> 2. Document [2] recommends policy '^(?!(amq\.)|(.*_fanout_)|(reply_)).*'. I don't see this in our ansible playbooks, nor in any of the config files in the RMQ container. What would this look like in Ansible, and what should the resulting container config look like? >> 3. It appears that we are not setting "amqp_durable_queues = True". What does this setting look like in Ansible, and what file does it go into? > > Note that even having rabbit HA policies adjusted like that and its HA > replication factor [0] decreased (e.g. to a 2), there still might be > high churn caused by a large enough number of replicated durable RPC > topic queues. And that might cripple the cloud down with the incurred > I/O overhead because a durable queue requires all messages in it to be > persisted to a disk (for all the messaging cluster replicas) before they > are ack'ed by the broker. > > Given that said, Oslo messaging would likely require a more granular > control for topic exchanges and the durable queues flag - to tell it to > declare as durable only the most critical paths of a service. A single > config setting and a single control exchange per a service might be not > enough. Also note that therefore, amqp_durable_queue=True requires dedicated control exchanges configured for each service. Those that use 'openstack' as a default cannot turn the feature ON. Changing it to a service specific might also cause upgrade impact, as described in the topic [3]. [3] https://review.opendev.org/q/topic:scope-config-opts > > There are also race conditions with durable queues enabled, like [1]. A > solution could be where each service declare its own dedicated control > exchange with its own configuration. > > Finally, openstack components should add perhaps a *.next CI job to test > it with durable queues, like [2] > > [0] https://www.rabbitmq.com/ha.html#replication-factor > > [1] > https://zuul.opendev.org/t/openstack/build/aa514dd788f34cc1be3800e6d7dba0e8… > > [2] https://review.opendev.org/c/openstack/nova/+/820523 > >> >> Does anyone have a sample set of RMQ config files that they can share? >> >> It looks like my Outlook has ruined the link; reposting: >> [2] https://wiki.openstack.org/wiki/Large_Scale_Configuration_Rabbit > > > -- > Best regards, > Bogdan Dobrelya, > Irc #bogdando -- Best regards, Bogdan Dobrelya, Irc #bogdando

2 1

Swift Account Reaper Deletion Status
by Matthew Grinnell 09 Dec '21

09 Dec '21

Hello, I was curious about expected account-reaper behavior after deleting an account and if there is a way to be able to tell when all data for the account has been reaped. When I issue a delete on a swift account, I see subsequent HEAD requests on the account then return 410 Gone. Testing with debug logging I can see that after passing the delay_reaping time I see the reaper report my test object is deleted, HEAD requests against the account still return the same 410 Gone. Is there a way to tell when everything for an account has been purged, ie does the 410 return change at some point when that happens? Thanks, Matt

2 1

[all][tc] Technical Committee next weekly meeting on Dec 9th at 1500 UTC
by Ghanshyam Mann 09 Dec '21

09 Dec '21

Hello Everyone, Technical Committee's next weekly meeting is scheduled for Dec 9th at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, Dec 8th, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1