April 2021 - openstack-discuss - lists.openstack.org

[all][tc] Technical Committee next weekly meeting on April 29th at 1500 UTC
by Ghanshyam Mann 29 Apr '21

29 Apr '21

Hello Everyone, Below is the agenda for tomorrow's TC meeting schedule on April 29th at 1500 UTC in #openstack-tc IRC channel. == Agenda for tomorrow's TC meeting == * Roll call * Follow up on past action items * PTG ** https://etherpad.opendev.org/p/tc-xena-ptg * TC tracker for Xena cycle (gmann) ** https://etherpad.opendev.org/p/tc-xena-tracker * Gate performance and heavy job configs (dansmith) ** http://paste.openstack.org/show/jD6kAP9tHk7PZr2nhv8h/ * TC's context, name, and documenting formal responsibilities (TheJulia) * Open Reviews ** https://review.opendev.org/q/project:openstack/governance+is:open -gmann

1 0

Xena PTG Policy Summary
by Lance Bragstad 29 Apr '21

29 Apr '21

Hey all, Last week we spent a lot of time discussing RBAC, where everything stands, and where we need to go to offer a consistent experience for operators and users. We kept track of all the sessions in a single etherpad, which also served as a place for daily summaries [0]. There is a lot of information in there, but we started working through the action items and correlating them to bugs or opening new bugs. Hopefully this helps us track progress through Xena. One of the biggest outcomes from last week was the discussion about how system users should interact with project-owned resources. For context, administrators have always been able to do things for project users because they both have project-scoped tokens. That's no longer going to be the case as services adopt system-scope. We came up with an interesting way to solve the problem and we compared it to other approaches. This all starts at about line 136 in the etherpad [0]. Ultimately, we think it will be the least invasive approach, we have a specification up for review [1], and a PoC in flight [2]. Please look over the summary and links to any actionables for your project. We can use this thread to discuss any questions if you have them. Thanks again for all the dedication and focus on policy last week. I know the discussions aren't easy and it's a tough problem to work through, but landing something this big across OpenStack services will be a huge win for operators and users. Lance [0] https://etherpad.opendev.org/p/policy-popup-xena-ptg [1] https://review.opendev.org/c/openstack/keystone-specs/+/787640 [2] https://review.opendev.org/c/openstack/keystonemiddleware/+/787822

1 0

[all][requirements] sqlalchemy 1.4.x release/upgrade
by Matthew Thode 28 Apr '21

28 Apr '21

Looks like a new release of sqlalchemy is upon us and is breaking tests in openstack (1.4.11 for now). Please test against https://review.opendev.org/788339 to get your project working against the newest version. currently failing are cinder ironic keystone masakari neutron and nova cross gates Thanks, -- Matthew Thode

6 8

[openstack-ansible-modules] Openstack Ansible collections PTG session summary
by Sagi Shnaidman 28 Apr '21

28 Apr '21

Hi, all Highlights from Openstack Ansible collection/modules PTG session[1] In the last cycle we had 68 commits, 3 released versions, 5 new contributors, 14 solved issues. The roadmap for next cycle: All modules in Xena should convert SDK output to a dictionary to match last SDK changes. CI tasks: 1. Automatic release jobs 2. Docs generation and publishing jobs - docs.openstack.org and readthedocs will be evaluated. 3. Gating jobs for projects starting to use the collections, like TripleO. Bugtracker - to evaluate Launchpad for bugs and issues submission. We continue to convert modules to use a standard OpenstackModule class. Features coverage for Openstack Ansible modules would help us to see how good we cover the current functionality. It would be useful to have roles in collection which will run specific and common tasks, which can be used by OPs easily. (Like Terraform "modules" concept) For fetching Openstack SDK logs during the tasks execution we may have a specific role or/and OpenstackModule class modifications. Ansible-test is used now for sanity testing only, we'll evaluate it for integration tests next cycle. Thanks [1] https://etherpad.opendev.org/p/xena-ptg-os-ansible-collections -- Best regards Sagi Shnaidman

1 0

[openstack-ansible][osa][PTG] OpenStack-Ansible Xena PTG summary
by Dmitriy Rabotyagov 28 Apr '21

28 Apr '21

Hi everyone! First of all I would like to thank everyone for attendance. We have discussed a lot of topics both regarding things that we need to merge before releasing Wallaby (we are trailing behind and may release in up to 3 month after official release) and regarding plans for the Xena. First comes list of things that we agreed to land for W in descending priority: 1. ansible-role-pki. Gerrit topic [1] 2. Centos8-Stream. Gerrit topic [2] 3. Move neutron-server to standalone group in env.d 4. Debian Bullseye. We're currently blocked with infra [3] 5. Senlin tempest test [4] 6. Fix broken roles 7. Implement send_service_user_token and service_token_roles_required Also we agreed to keep ansible-base 2.11.* release for Wallaby, and not switch to the ansible-core yet. List of descisions that were taken for the Xena release: 1. Deprecate Ubuntu Bionic at the beggining of X cycle 2. Deprecate Centos-8 (classic) at the beginning of X cycle 3. Switch MariaDB balancing frontend from HAproxy to MaxScale 4. Drop Nginx for the Keystone and use Apache is all scenarios as most unified solution. 5. Add option to roles to configure systemd timers for cleaning up soft-deleted records in DB 6. Create operating systems support table to clear out possible upgrade paths between releases 7. Replace dashes with underscores in group names and dynamic inventory and stop ignoring TRANSFORM_INVALID_GROUP_CHARS. 8. Adding ability to enable prometheus support for MariaDB and HAproxy 9. Revise possibility to support deployments on top of arm64 10. Look into creation common roles for Ansible OpenStack collection. At least write a blueprint/spec regarding scope of these roles. 11. Deploy ARA on localhost for OSA logging, configured by variables [1] https://review.opendev.org/q/topic:%22osa%252Fpki%22+(status:open) [2] https://review.opendev.org/q/topic:%22osa-stream%22+(status:open) [3] https://review.opendev.org/q/topic:%22debian_bullseye%22+(status:open) [4] https://review.opendev.org/c/openstack/openstack-ansible-os_senlin/+/754045 -- Kind Regards, Dmitriy Rabotyagov

3 4

internet not working in instance
by cloud learner 28 Apr '21

28 Apr '21

Hi All, Have installed rocky on centos 7 with 2 node controller and compute, used linuxbridge as mentioned in the document used self service network. My question is, as we have to create the br-ex and attach it to eth when we use OVS, so as I am using linuxbridge there is need to create the bridge. If there are any doc kindly suggest. Thanks

1 0

[wallaby][neutron] Neuton Server Upgrade
by Ammad Syed 28 Apr '21

28 Apr '21

Hi, I have upgraded neutron server from victoria to wallaby. Everytime I restart neutron server, I see below errors in neutron-server.log. Can you guys please advise on it. 2021-04-28 06:34:12.144 1767 INFO ovsdbapp.backend.ovs_idl.vlog [req-cc4f309f-1851-47cd-bab1-de0445964d56 - - - - -] tcp:172.16.30.51:6641: connecting... 2021-04-28 06:34:12.145 1767 INFO ovsdbapp.backend.ovs_idl.vlog [req-cc4f309f-1851-47cd-bab1-de0445964d56 - - - - -] tcp:172.16.30.51:6641: connected 2021-04-28 06:34:12.214 1770 INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbNbOvnIdl for MaintenanceWorker with retry 2021-04-28 06:34:12.238 1770 INFO ovsdbapp.backend.ovs_idl.vlog [-] tcp: 172.16.30.51:6641: connecting... 2021-04-28 06:34:12.238 1770 INFO ovsdbapp.backend.ovs_idl.vlog [-] tcp: 172.16.30.51:6641: connected 2021-04-28 06:34:12.248 1767 ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: AttributeError: 'NoneType' object has no attribute 'db_find_rows' 2021-04-28 06:34:12.248 1767 ERROR ovsdbapp.event Traceback (most recent call last): 2021-04-28 06:34:12.248 1767 ERROR ovsdbapp.event File "/usr/lib/python3/dist-packages/ovsdbapp/event.py", line 159, in notify_loop 2021-04-28 06:34:12.248 1767 ERROR ovsdbapp.event match.run(event, row, updates) 2021-04-28 06:34:12.248 1767 ERROR ovsdbapp.event File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py", line 443, in run 2021-04-28 06:34:12.248 1767 ERROR ovsdbapp.event self.driver.delete_mac_binding_entries(row.external_ip) 2021-04-28 06:34:12.248 1767 ERROR ovsdbapp.event File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 1058, in delete_mac_binding_entries 2021-04-28 06:34:12.248 1767 ERROR ovsdbapp.event mac_binds = self._sb_ovn.db_find_rows( 2021-04-28 06:34:12.248 1767 ERROR ovsdbapp.event AttributeError: 'NoneType' object has no attribute 'db_find_rows' 2021-04-28 06:34:12.248 1767 ERROR ovsdbapp.event 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: AttributeError: 'NoneType' object has no attribute 'db_find_rows' 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event Traceback (most recent call last): 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event File "/usr/lib/python3/dist-packages/ovsdbapp/event.py", line 159, in notify_loop 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event match.run(event, row, updates) 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py", line 443, in run 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event self.driver.delete_mac_binding_entries(row.external_ip) 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 1058, in delete_mac_binding_entries 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event mac_binds = self._sb_ovn.db_find_rows( 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event AttributeError: 'NoneType' object has no attribute 'db_find_rows' 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: AttributeError: 'NoneType' object has no attribute 'db_find_rows' 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event Traceback (most recent call last): 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event File "/usr/lib/python3/dist-packages/ovsdbapp/event.py", line 159, in notify_loop 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event match.run(event, row, updates) 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py", line 443, in run 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event self.driver.delete_mac_binding_entries(row.external_ip) 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 1058, in delete_mac_binding_entries 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event mac_binds = self._sb_ovn.db_find_rows( 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event AttributeError: 'NoneType' object has no attribute 'db_find_rows' 2021-04-28 06:34:12.250 1767 ERROR ovsdbapp.event 2021-04-28 06:34:12.251 1767 ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: AttributeError: 'NoneType' object has no attribute 'db_find_rows' 2021-04-28 06:34:12.251 1767 ERROR ovsdbapp.event Traceback (most recent call last): 2021-04-28 06:34:12.251 1767 ERROR ovsdbapp.event File "/usr/lib/python3/dist-packages/ovsdbapp/event.py", line 159, in notify_loop 2021-04-28 06:34:12.251 1767 ERROR ovsdbapp.event match.run(event, row, updates) 2021-04-28 06:34:12.251 1767 ERROR ovsdbapp.event File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py", line 443, in run 2021-04-28 06:34:12.251 1767 ERROR ovsdbapp.event self.driver.delete_mac_binding_entries(row.external_ip) 2021-04-28 06:34:12.251 1767 ERROR ovsdbapp.event File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 1058, in delete_mac_binding_entries 2021-04-28 06:34:12.251 1767 ERROR ovsdbapp.event mac_binds = self._sb_ovn.db_find_rows( 2021-04-28 06:34:12.251 1767 ERROR ovsdbapp.event AttributeError: 'NoneType' object has no attribute 'db_find_rows' 2021-04-28 06:34:12.251 1767 ERROR ovsdbapp.event 2021-04-28 06:34:12.251 1767 INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] OVN reports status down for port: 3d040d1d-c3d5-49da-9103-8ab4f9a8ab01 2021-04-28 06:34:12.252 1767 INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [req-cc4f309f-1851-47cd-bab1-de0445964d56 - - - - -] Getting OvsdbSbOvnIdl for WorkerService with retry Secondly after upgrade, I can only see upgraded network agents in openstack network agent list. The other agents that are on victoria release are not showing up. Need your help. -- Regards, Syed Ammad Ali

1 0

[victoria][ha]Rabbitmq HA configuration in victoria
by Shalabh Goel 28 Apr '21

28 Apr '21

Hi, I have been trying to configure HA for rabbitmq service for my Openstack installation. I found varying options on the Internet. I am facing some issues with rabbitmq with HAproxy. Option 1. I want to know if victoria supports the following configuration directive for openstack services. rabbit_hosts=rabbit1:5672,rabbit2:5672,rabbit3:5672 as told in this guide https://docs.openstack.org/ha-guide/control-plane-stateful.html Option 2. Can we configure as follows: transport_url = rabbit://RABBIT_USER:RABBIT_PASS@rabbit1:5672,RABBIT_USER:RABBIT_PASS@rabbit2:5672,RABBIT_USER:RABBIT_PASS@rabbit3:5672 Option 3. I have been using this configuration in haproxy listen rabbitmq_cluster_openstack bind controllers:5672 mode tcp balance roundrobin server controller1 controller1:5672 check inter 2000 rise 2 fall 5 server controller2 controller2:5672 backup check inter 2000 rise 2 fall 5 I am getting the following error in my log files for all the services. 2021-04-28 09:59:05.857 129188 INFO oslo.messaging._drivers.impl_rabbit [-] [bbf40d1a-e095-47ad-8629-931af485d4cf] Reconnected to AMQP server on controllers:5672 via [amqp] client with port 44818. I have set transport_url as follows: > transport_url = rabbit://openstack:rabbit123@controllers:5672 > The error goes away if I change host to controller1 or controller. Please suggest which is the recommended one of the 3. Thanks in advance -- Shalabh Goel

2 1

cyborg launchpad suppport
by Alex Song (宋文平) 28 Apr '21

28 Apr '21

Hi, We change to use launchpad to track cyborg bugs and features, with this patch merged: https://review.opendev.org/c/openstack/project-config/+/787306. But we cannot count on the website: https://www.stackalytics.io/ Please help us with this problem. Thanks.

3 4

[Interop][PTG] highlights from Interop PTG
by Kanevsky, Arkady 28 Apr '21

28 Apr '21

Fellow stackers, Thank you for successful Xena PTG. The PTG agenda and notes are at https://etherpad.opendev.org/p/april2021-ptg-interop-refstack Highlights: 1. Ghanshayam and Goutham are now co-chair of Interop along with Arkady 2. Superuser article for Interop is in progress highlighting major changes to Interop program including Rfestack transitioning to Python3, support of refstack of add-on projects, and adding share file system as add-on program. 3. Working on updated to Interop processes, and specifically no need for Board to approve new guidelines, only process changes. * See https://review.opendev.org/c/osf/interop/+/787646 * Discuss updates to all wiki pages starting from https://wiki.openstack.org/wiki/Governance/InteropWG - expect changes in Xena cycle 4. Agreed on how we will list add-on Logo to marketplace listing 5. Working on how to handle in automated matter in refstack admin privileges for setting up for tempest user testing, Manila in particular. 6. Starting discussion on how to handle microiservice versions of APIs. 7. Interlocks with all projects under interop guidelines, except Swift and Keystone. 8. Discussed several possible add-on projects Happy to provide more details. Thanks, Arkady Kanevsky, Ph.D. SP Chief Technologist & DE Dell Technologies office of CTO Dell Inc. One Dell Way, MS PS2-91 Round Rock, TX 78682, USA Phone: 512 7204955

1 0