April 2023 - openstack-discuss - lists.openstack.org

Re: [openstack][sharing][kolla ansible]Problems when 1 of 3 controller was be down
by Matt Crees 24 Apr '23

24 Apr '23

Hi all, It seems worth noting here that there is a fix ongoing in oslo.messaging which will resolve the issues with HA failing when one node is down. See here: https://review.opendev.org/c/openstack/oslo.messaging/+/866617 In the meantime, we have also found that setting kombu_reconnect_delay = 0.5 does resolve this issue. As for why om_enable_rabbitmq_high_availability is currently defaulting to false, as Michal said enabling it in stable releases will impact users. This is because it enables durable queues, and the migration from transient to durable queues is not a seamless procedure. It requires that the state of RabbitMQ is reset and that the OpenStack services which use RabbitMQ are restarted to recreate the queues. I think that there is some merit in changing this default value. But if we did this, we should either add additional support to automate the migration from transient to durable queues, or at the very least provide some decent docs on the manual procedure. However, as classic queue mirroring is deprecated in RabbitMQ (to be removed in RabbitMQ 4.0) we should maybe consider switching to quorum queues soon. Then it may be beneficial to leave the classic queue mirroring + durable queues setup as False by default. This is because the migration between queue types (durable or quorum) can take several hours on larger deployments. So it might be worth making sure the default values only require one migration to quorum queues in the future, rather than two (durable queues now and then quorum queues in the future). We will need to make this switch eventually, but right now RabbitMQ 4.0 does not even have a set release date, so it's not the most urgent change. Cheers, Matt >Hi Michal, > >Feel free to propose change of default in master branch, but I don?t think we can change the default in stable branches without impacting users. > >Best regards, >Michal > >> On 11 Apr 2023, at 15:18, Michal Arbet <michal.arbet(a)ultimum.io> wrote: >> >> Hi, >> >> Btw, why we have such option set to false ? >> Michal Arbet >> Openstack Engineer >> >> Ultimum Technologies a.s. >> Na Po???? 1047/26, 11000 Praha 1 >> Czech Republic >> >> +420 604 228 897 <> >> michal.arbet(a)ultimum.io <mailto:michal.arbet@ultimum.io> >> https://ultimum.io <https://ultimum.io/> >> >> LinkedIn <https://www.linkedin.com/company/ultimum-technologies> | Twitter <https://twitter.com/ultimumtech> | Facebook <https://www.facebook.com/ultimumtechnologies/timeline> >> >> >> ?t 11. 4. 2023 v 14:48 odes?latel Micha? Nasiadka <mnasiadka(a)gmail.com <mailto:mnasiadka@gmail.com>> napsal: >>> Hello, >>> >>> RabbitMQ HA has been backported into stable releases, and it?s documented here: >>> https://docs.openstack.org/kolla-ansible/yoga/reference/message-queues/rabb… >>> >>> Best regards, >>> Michal >>> >>> W dniu wt., 11.04.2023 o 13:32 Nguy?n H?u Kh?i <nguyenhuukhoinw(a)gmail.com <mailto:nguyenhuukhoinw@gmail.com>> napisa?(a): >>>> Yes. >>>> But cluster cannot work properly without it. :( >>>> >>>> On Tue, Apr 11, 2023, 6:18 PM Danny Webb <Danny.Webb(a)thehutgroup.com <mailto:Danny.Webb@thehutgroup.com>> wrote: >>>>> This commit explains why they largely removed HA queue durability: >>>>> >>>>> https://opendev.org/openstack/kolla-ansible/commit/2764844ee2ff9393a4eebd90… >>>>> From: Satish Patel <satish.txt(a)gmail.com <mailto:satish.txt@gmail.com>> >>>>> Sent: 09 April 2023 04:16 >>>>> To: Nguy?n H?u Kh?i <nguyenhuukhoinw(a)gmail.com <mailto:nguyenhuukhoinw@gmail.com>> >>>>> Cc: OpenStack Discuss <openstack-discuss(a)lists.openstack.org <mailto:openstack-discuss@lists.openstack.org>> >>>>> Subject: Re: [openstack][sharing][kolla ansible]Problems when 1 of 3 controller was be down >>>>> >>>>> >>>>> CAUTION: This email originates from outside THG >>>>> >>>>> Are you proposing a solution or just raising an issue? >>>>> >>>>> I did find it strange that kolla-ansible doesn't support HA queue by default. That is a disaster because when one of the nodes goes down it will make the whole rabbitMQ unacceptable. Whenever i deploy kolla i have to add HA policy to make queue HA otherwise you will endup in problem. >>>>> >>>>> On Sat, Apr 8, 2023 at 6:40?AM Nguy?n H?u Kh?i <nguyenhuukhoinw(a)gmail.com <mailto:nguyenhuukhoinw@gmail.com>> wrote: >>>>> Hello everyone. >>>>> >>>>> I want to summary for who meets problems with Openstack when deploy cluster with 3 controller using Kolla Ansible >>>>> >>>>> Scenario: 1 of 3 controller is down >>>>> >>>>> 1. Login horizon and use API such as nova, cinder will be very slow >>>>> >>>>> fix by: >>>>> >>>>> nano: >>>>> kolla-ansible/ansible/roles/heat/templates/heat.conf.j2 >>>>> kolla-ansible/ansible/roles/nova/templates/nova.conf.j2 >>>>> kolla-ansible/ansible/roles/keystone/templates/keystone.conf.j2 >>>>> kolla-ansible/ansible/roles/neutron/templates/neutron.conf.j2 >>>>> kolla-ansible/ansible/roles/cinder/templates/cinder.conf.j2 >>>>> >>>>> or which service need caches >>>>> >>>>> add as below >>>>> >>>>> [cache] >>>>> backend = oslo_cache.memcache_pool >>>>> enabled = True >>>>> memcache_servers = {{ kolla_internal_vip_address }}:{{ memcached_port }} >>>>> memcache_dead_retry = 0.25 >>>>> memcache_socket_timeout = 900 >>>>> >>>>> https://review.opendev.org/c/openstack/kolla-ansible/+/849487 >>>>> >>>>> but it is not the end >>>>> >>>>> 2. Cannot launch instance or mapping block device(stuck at this step) >>>>> >>>>> nano kolla-ansible/ansible/roles/rabbitmq/templates/definitions.json.j2 >>>>> >>>>> "policies":[ >>>>> {"vhost": "/", "name": "ha-all", "pattern": "^(?!(amq\.)|(.*_fanout_)|(reply_)).*", "apply-to": "all", "definition": {"ha-mode":"all"}, "priority":0}{% if project_name == 'outward_rabbitmq' %}, >>>>> {"vhost": "{{ murano_agent_rabbitmq_vhost }}", "name": "ha-all", "pattern": ".*", "apply-to": "all", "definition": {"ha-mode":"all"}, "priority":0} >>>>> {% endif %} >>>>> ] >>>>> >>>>> nano /etc/kollla/global.conf >>>>> >>>>> [oslo_messaging_rabbit] >>>>> kombu_reconnect_delay=0.5 >>>>> >>>>> >>>>> https://bugs.launchpad.net/oslo.messaging/+bug/1993149 >>>>> https://docs.openstack.org/large-scale/journey/configure/rabbitmq.html >>>>> >>>>> I used Xena 13.4 and Yoga 14.8.1. >>>>> >>>>> Above bugs are critical, but I see that it was not fixed. I am just an operator and I want to share what I encountered for new people who come to Openstack >>>>> >>>>> >>>>> Nguyen Huu Khoi >>> -- >>> Micha? Nasiadka >>> mnasiadka(a)gmail.com <mailto:mnasiadka@gmail.com>

6 18

Query about RBAC for subnets
by Aditya Sathish 24 Apr '23

24 Apr '23

Hello, We currently have an external network that is being interfaced from the physical interface of the compute server to an L3 switch with different subnets to different services. These services (and, by extension, the subnets) must be accessed and controlled by the operator to allow a project only a subset of these services. RBAC can only set rules for the network as a whole. Can this network be made external but only share a few subnets with each project? If not the RBAC way, is there another alternative to achieve the same? Regards, Aditya

2 1

[openstack-helm] Manila chart merged
by Vladimir Kozhukalov 24 Apr '23

24 Apr '23

Hi, everyone I would like to announce that last week we eventually merged the PR that adds the Manila chart. Thanks everyone for their contribution. -- Best regards, Kozhukalov Vladimir

1 0

[neutron] Bug Deputy Report April 17 - 23
by Lucas Alvares Gomes 24 Apr '23

24 Apr '23

Hi, This is the Neutron bug report from April 17th to 24th. *High:* * https://bugs.launchpad.net/neutron/+bug/2016704 - Listing ports creates inefficient database query - Assigned to: Ihtisham ul Haq * https://bugs.launchpad.net/neutron/+bug/2017145 - [OVN] - Logging configuration doesn't change after neutron restart - Assigned to: Elvira García Ruiz * https://bugs.launchpad.net/neutron/+bug/2017216 - [ovn-octavia-provider] batch member update generates out of sync with Octavia DB - Assigned to: Fernando Royo *Medium:* * https://bugs.launchpad.net/neutron/+bug/2016414 - Message when creating new neutron bugs has outdated links - Unassigned * https://bugs.launchpad.net/neutron/+bug/2016862 - [ovn-octavia-provider] Admin_state_up ignored on create a new member - Assigned to: Fernando Royo * https://bugs.launchpad.net/neutron/+bug/2017037 - Improve keepalived test pid termination - Assigned to: Miro Tomaska * https://bugs.launchpad.net/neutron/+bug/2017127 - [ovn-octavia-provider] Fix member update according to the Octavia API definition - Assigned to: Fernando Royo *Low:* * https://bugs.launchpad.net/neutron/+bug/2016413 - api-ref: missing documentation for subnet-onboard - Unassigned *Wishlist:* * https://bugs.launchpad.net/neutron/+bug/2016504 - [rfe]Support specify fixed_ip_address for DHCP or Metadata port - Unassigned * https://bugs.launchpad.net/neutron/+bug/2016960 - neutron fwaas support l2 firewall for ovn driver - Unassigned *Needs further triage:* * https://bugs.launchpad.net/neutron/+bug/2017023 - Tempest: remove test duplication for Compute legacy networking API and Neutron API calls - Unassigned * https://bugs.launchpad.net/neutron/+bug/2017131 - neutron-loki job fails tests with ORM session: SQL execution without transaction in progress - Unassigned Cheers, Lucas

1 0

[Networking-vSphere]
by SCAGLIA Emanuele (NPO) 23 Apr '23

23 Apr '23

Hello, I'd like to know if there is in your opinion any chance that the OVSvApp solution works with Openstack zed release. Regarding how to use it, I found this page: https://github.com/nephoscale-div/OVSvAPP Is it still valid? Thank you in advance, Emanuele Scaglia Emanuele Scaglia Senior System Engineer NPO TORINO s.r.l. [cid:image001.png@01D97443.46C8DCE0]<https://www.nposervices.com/> [cid:image002.png@01D97443.46C8DCE0] emanuele.scaglia(a)nposervices.com<mailto:emanuele.scaglia@nposervices.com> [cid:image003.png@01D97443.46C8DCE0] +39 3666180721<tel:3425965928> [cid:image004.png@01D97443.46C8DCE0] Via Vincenzo Lancia 27, 10141 TORINO<https://www.google.com/maps/place/Via+Vincenzo+Lancia,+27,+10141+Torino+TO/…> [cid:image005.png@01D97443.46C8DCE0] nposervices.com<https://www.nposervices.com/> [cid:image006.png@01D97443.46C8DCE0]<https://www.facebook.com/> [cid:image007.png@01D97443.46C8DCE0]<https://www.twitter.com/> [cid:image008.png@01D97443.46C8DCE0]<https://www.linkedin.com/> [cid:image009.png@01D97443.46C8DCE0]<https://www.instagram.com/> [cid:image010.png@01D97443.46C8DCE0]<https://www.youtube.com/> Sede legale: L.go Dora P. Colletta, 81, 10153- Torino Tel. +39 011 19701225 Fax. +39 011 19781413 P. IVA: 10041060012 The content of this email is confidential and intended for the recipient specified in message only. It is strictly forbidden to share any part of this message with any third party, without a written consent of the sender. If you received this message by mistake, please reply to this message and follow with its deletion, so that we can ensure such a mistake does not occur in the future. AVVERTENZE AI SENSI DEL D.LGS. 196/2003 . Il contenuto di questo messaggio (ed eventuali allegati) e' strettamente confidenziale. L'utilizzo del contenuto del messaggio e' riservato esclusivamente al destinatario. La modifica, distribuzione, copia del messaggio da parte di altri e' severamente proibita. Se non siete i destinatari Vi invitiamo ad informare il mittente ed eliminare tutte le copie del suddetto messaggio. The contents of this message (and all attachments) are strictly confidential. The message is intended exclusively to the addressee. The modification, distribution, or replication of the message from others is strictly forbidden. If you are not the intended addressee, please notify the sender and eliminate all the copies of the message,

2 1

[tc][all] Technical Committee Weekly Summary - April 22, 2023
by Nikolla, Kristi 22 Apr '23

22 Apr '23

Hi all, Here’s another edition of “What’s happening on the Technical Committee.” Meeting ======= The Technical Committee met on April 18, 2023 on the #openstack-tc channel on OFTC IRC. The meeting was chaired by Jay Faulkner. Thanks Jay! The next meeting will be held on April 25, 2023 at 18:00UTC. For more information visit https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting Happenings ========== Leaderless projects ------------------- There's only 3 projects now remaining without a leader[0]. Monasca[1], Vitrage[2] and Winstackers[3]. We're actively looking for people with the time and desire to maintain these projects. A retirement of Winstackers would cause Windows (and Hyper-V) host support to be removed. Meanwhile, TripleO has switched to a distributed project leadership model, and is initiating the process of deprecation.[4] [0]. https://etherpad.opendev.org/p/2023.2-leaderless [1]. https://lists.openstack.org/pipermail/openstack-discuss/2023-April/033112.h… [2]. https://lists.openstack.org/pipermail/openstack-discuss/2023-April/033348.h… [3]. https://lists.openstack.org/pipermail/openstack-discuss/2023-April/033342.h… [4]. https://review.opendev.org/c/openstack/governance/+/877132 Issues with docs since 2023.1 release ------------------------------------- 2023.1 is the first release that uses numeric versioning, rather than the old naming scheme. This confuses our docs build process, which produces incorrect results. For one example the Administrator Guides list for Zed lists all the projects correctly[0], but the one for 2023.1 is empty[1]. Please use the Etherpad [2] to report issues, and the same Etherpad for tracking work. We're looking for help in getting this resolved quickly. [0]. https://docs.openstack.org/zed/admin/index.html [1]. https://docs.openstack.org/2023.1.antelope/admin/index.html [2]. https://etherpad.opendev.org/p/docs-issues-2023 Weekly Gate Health Report ------------------------- The gate continues to experience failures regarding volume detach tests[0]. There's also some issues with Ironic grenade jobs. Helping out with CI is an amazing way to get involved in the community, reach out to the TC (see below how) to know more! [0]. https://meetings.opendev.org/meetings/tc/2023/tc.2023-04-18-18.00.log.html#… Changes ======= Merged Appoint Tim Burke as Swift PTL (formal-vote) | https://review.opendev.org/c/openstack/governance/+/878286 TripleO: switch to distributed project leadership (formal-vote) | https://review.opendev.org/c/openstack/governance/+/878799 Appoint Andriy Kurilin as Rally PTL (formal-vote) | https://review.opendev.org/c/openstack/governance/+/878065 Add missing governance-sigs repo in technical-committee-repos.yaml (project-update) | https://review.opendev.org/c/openstack/governance/+/880237 Consider release version as string in projects data (code-change) | https://review.opendev.org/c/openstack/governance/+/877143 New Open Update to hacking v6 (code-change) | https://review.opendev.org/c/openstack/governance/+/881266 Switch to 2023.2 testing runtime py version (code-change) | https://review.opendev.org/c/openstack/governance/+/881137 Abandoned Switch project and OpenStack versions in releases (formal-vote) | https://review.opendev.org/c/openstack/governance/+/875942 All Open https://review.opendev.org/q/project:openstack/governance+status:open How to contact the TC ===================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send an email with the tag [tc] on the openstack-discuss mailing list. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 16:00 UTC 3. IRC: Ping us using the 'tc-members' keyword on the #openstack-tc IRC channel on OFTC.

1 0

[nova]qemu-guest-agent Used in openstack
by 啧啧啧 20 Apr '23

20 Apr '23

Hi, We discovered the use of qemu-guest-agent in openstack,Set hw_qemu_guest_agent=yes to the metadata parameter on the image. https://docs.openstack.org/mitaka/config-reference/compute/hypervisor-kvm.h… Meanwhile, the application of qemu-guest-agent in cloud scenarios is investigated. It is found that Qemu-guest-Agent can be used for guest os performance monitoring, snapshot backup, and guest OS-level failover. But these things can be done in openstack with prometheus and  instance backup,There may be some differences between them. Sorry, I am not familiar with how hw_qemu_guest_agent is used in openstack, and I do not know the operations that can be performed on guest os in openstack by configuring hw_qemu_guest_agent. Only through qemu official know some of the effects.I want to clarify the use of hw_qemu_guest_agent. Other components of openstack, such as the prometheus service, can be used instead.

2 2

[qa][all] New hacking 6.0.0 is available
by Ghanshyam Mann 20 Apr '23

20 Apr '23

Hello Everyone, Hacking 6.0.0 is available to use now which includes flake8 5.0 changes. There are no other changes than bumping flake8 version to 5.0.1, you can refer to the changes done in flake8 5.0.0 (or 5.0.1) directly in flake8 release notes - https://flake8.pycqa.org/en/latest/release-notes/5.0.0.html Note: hacking is pinned in test-requirements.txt (for almost all the repo), do not forget to bump the version there while moving your code to new hacking. -gmann

1 0

[Glance] Bobcat PTG summary
by Abhishek Kekane 20 Apr '23

20 Apr '23

Hi Team, We had our seventh virtual PTG between 27th March to 31st March 2023. Thanks to everyone who joined the virtual PTG sessions. Using bluejeans app we had lots of discussion around different topics for glance, glance + cinder, fips and Secure RBAC. You can find the topics discussed in etherpad [1] with Notes from the session and which also includes the recordings of each discussion. Here I am going to highlight a few important topics which we are going to target this cycle. # Glance Image Direct URL access - OSSN-0090 This is the most important and high priority work item for us in this cycle. We are going to move the spec to the current cycle and implement glance, client and osc side work this cycle. We are also going to implement tempest side work for the same. Once it is completed, Rajat (current cinder PTL) is going to write consumer side spec for nova and cinder. https://bluejeans.com/s/dKF1IG2h8PC chapter 2 (from 22 minutes onward) # Bridge gap between Glance client and OSC Last cycle we have prepared the list of missing commands [2] and started submitting patches to implement the same. Our biggest hurdle at the moment is lack of reviews from the osc team (which has only one/two cores). Glance PTL is going to communicate with TC/existing core(s) of OSC whether they can add any glance core to the team so that we can prioritize our work and get reviews immediately. This cycle we are planning to complete this work. Recording: https://bluejeans.com/s/XZoRGceEhW3/ - Chapter 4 # Default glance to configure multistore Efforts has been started from last cycle but this work needs to be divided in small parts such as removing unnecessary tests, migrating existing tests to support multistore and so on. This cycle we are going to ensure that existing tests are supporting multistore configs. Recording: https://bluejeans.com/s/XZoRGceEhW3/ - Chapter 3 # Secure RBAC So far we are good with the goals decided by the community. This cycle PTL is going to discuss with the QE/TC team regarding deprecation of policies (migrating policies to use manager role) and decide further action. Recording: https://bluejeans.com/s/RHgqQi28_Hk/ - chapter 1 # Fips Overview ade_lee is going to submit some patches for us and we are going to help him in debugging failures if there are any. Recording: https://bluejeans.com/s/RHgqQi28_Hk/ - chapter 5 # Native Image encryption Finally the Barbican work is complete and we can move ahead with glance side work in this cycle. We need to move glance side spec to Bobcat and check whether there will be any changes in the implementation. Luzi is going to work on glance changes and the team is going to assist her in reviews and testing. Recording: https://bluejeans.com/s/RHgqQi28_Hk/ - chapter 3 # RBD deferred deletion Eric from cinder team is working on PoC/code for glance-store side changes for the same. Once we have PoC ready, the glance team will assist him to create a detailed spec and start working on tests for the same. https://bluejeans.com/s/dKF1IG2h8PC - chapter 2 Apart from above topics you can find other miscellaneous topics discussed in PTG etherpad [1] along with milestone wise priorities at the end. If you have any questions/suggestions then please join us in our weekly meeting (each Thursday #openstack-meeting irc channel at 1400 UTC). [1] https://etherpad.opendev.org/p/glance-bobcat-ptg [2] https://docs.google.com/spreadsheets/d/1tlipscCDeWmYGz3Hfp74czPW7BBJb4ST84q… Thanks and Regards, Abhishek Kekane

5 10

[cinder] festival of feature reviews 21 April 2023
by Rajat Dhasmana 20 Apr '23

20 Apr '23

Hello Argonauts, We will be having our monthly festival of reviews tomorrow i.e. 21st April (Friday) from 1400-1600 UTC. Our priorities for M-1 are DB changes for compatibility with sqlalchemy 2.0 and cinder-specs. Following are some additional details: Date: 21st April, 2023 Time: 1400-1600 UTC Meeting link: will be shared before meeting etherpad: https://etherpad.opendev.org/p/cinder-festival-of-reviews See you there! Thanks Rajat Dhasmana

1 0