July 2021 - openstack-discuss - lists.openstack.org

[neutron] Drivers meeting - agenda for 23.07.2021
by Slawek Kaplonski 22 Jul '21

22 Jul '21

Hi, Agenda for tomorrow's drivers meeting is available at https:// wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda We have 3 RFEs to discuss: * https://bugs.launchpad.net/neutron/+bug/1933222 - [RFE] Add distributed datapath for metadata - This is continuation from the last meeting 2 weeks ago. * https://bugs.launchpad.net/neutron/+bug/1933517 - [RFE][OVN] Create an intermediate OVS bridge between VM and intergration bridge to improve the live-migration process * https://bugs.launchpad.net/neutron/+bug/1935847 - [RFE] Basic Authentication Support for Standalone Neutron Please check those RFEs, ask questions if You have any and see You tomorrow on the meeting. -- Slawek Kaplonski Principal Software Engineer Red Hat

2 2

[octavia] PTL on vacation - Meetings cancelled
by Gregory Thiemonge 22 Jul '21

22 Jul '21

Hi, I'll be on PTO for 2 weeks, I'll be back on August 9th. As discussed during the previous weekly meeting, the next two Octavia meetings are cancelled. The next meeting is on August 11th. Greg

1 0

]ops] Something wrong with rabbit settings
by Massimo Sgaravatto 22 Jul '21

22 Jul '21

Dear all I am afraid I have something wrong in the setup of rabbit, and I would appreciate your help. We have a single rabbit cluster for all the OpenStack services. It is a cluster composed by 3 instances (now running rabbitmq-server-3.8.16 and erlang-24.0.2). We are using this setting: cluster_partition_handling = pause_minority to handle partitions, and we are using this policy: [root@cld-rbt-01 ~]# rabbitmqctl list_policies Listing policies for vhost "/" ... vhost name pattern apply-to definition priority / ha-all ^(?!amq\.).* all {"ha-mode":"all"} 0 In the conf files of the OpenStack services we have these settings related to Rabbit: transport_url = rabbit://openstack_prod:xxx@192.168.60.220:5672, openstack_prod:xxx@192.168.60.221:5672, openstack_prod:xxx@192.168.60.222:5672 rabbit_ha_queues = true >From time to time rabbit complains about some network partitions (still not clear why): 2021-07-02 08:12:55.715 [error] <0.463.0> Partial partition detected: * We saw DOWN from rabbit@cld-rbt-02 * We can still see rabbit@cld-rbt-03 which can see rabbit@cld-rbt-02 * pause_minority mode enabled We will therefore pause until the *entire* cluster recovers and when this happens Openstack services are of course impacted. As soon as I can check the cluster status, rabbitmqctl cluster_status doesn't complain about any problems (i.e. it doesn't report any network partitions and it reports all nodes running) but the problems on the OpenStack services are still there (e.g. "neutron agent-list" report many agents down). I need to restart the rabbit cluster in order to have OpenStack services working again Any hints ? Thanks, Massimo

4 5

[nova][placement] Yoga PTG
by Balazs Gibizer 22 Jul '21

22 Jul '21

Hi, It is time again to discuss and agree on what time we will meet during the PTG week (18 - 22 Oct). I created a doodle [1]. Please make your votes there about the timing. The final date to register our time slots is 21th of July so I ask you to make your vote not later than 20 of July, so I can make the registration in time. Also I think we should discuss how many slots we need. In the last two PTGs we tried finish our etherpad in 3 days x 4 hour slots. But in both cases some overtime was needed. How do you feel, should we book for 4 days x 4 hours slots this time instead? I created the PTG etherpad so we can start collecting the topics[2]. If you have cross project topics, then please either agree with the other team about the timing of the discussion or let me know and I will arrange for the slot. Cheers, gibi [1] https://doodle.com/poll/yetctyhufnn6g8hn [2] https://etherpad.opendev.org/p/nova-yoga-ptg

2 4

[Kolla] [ Kolla-Ansible] Kolla Ansible Downgrade Issue
by Richa Gupta 22 Jul '21

22 Jul '21

Hi Team, I installed Kolla-Ansible release 12.0.0, i.e. wallaby on multinode setup, and then downgraded Kolla-Ansible to release 11.0.0, i.e. Victoria, following the below-mentioned link: https://docs.openstack.org/kolla-ansible/latest/user/operating-kolla.html I downgraded Kolla-Ansible by running the command "*pip install --upgrade kolla-ansible==6.0.0*" and, changing "openstack_release: victoria" in globals.yml. But, "*kolla-ansible upgrade*" failed with the below error (Error also attached in the mail) : "alembic.util.exc.CommandError: Can't locate revision identified by 'wallaby_contract01'" Could you please suggest if the downgrade is supported in Kolla-Ansible? Best Regards, Richa Gupta

3 3

[all][neutron][qa] Gate status: tempest-full-py3, tempest-slow-py3 or few more jobs are broken: "Donot recheck"
by Ghanshyam Mann 22 Jul '21

22 Jul '21

Hello Everyone, Since 2-3 hrs before, tempest-full-py3, tempest-slow-py3 and few more jobs started failing consistently in create_neutron_initial_network() method: "++ lib/neutron_plugins/services/l3:create_neutron_initial_network:214 : oscwrap --os-cloud devstack-admin --os-region RegionOne network create --project a6b71b541987471595bf5e38f5fbe264 private" - https://zuul.openstack.org/builds?job_name=tempest-full-py3 - https://zuul.openstack.org/builds?job_name=tempest-slow-py3 Strange is that 'tempest-integrated-storage'|'compute' jobs are passing even the configuration is almost the same. Slaweq reported the below bug and trying some neutron revert to know the root cause. He will check it tomorrow morning. - https://bugs.launchpad.net/neutron/+bug/1936983 Until we find the root cause/fix, do not recheck on these failures. -gmann

5 8

[kolla][glance][cinder] issues with show_image_direct_url and show_multiple_locations in wallaby
by J-P Methot 22 Jul '21

22 Jul '21

Hi, As you might already know if you read this mailing list, I've been trying to push Kolla into production for a while. This issue however might be more Glance and Cinder related than Kolla. We're trying to provision an Openstack Wallaby cluster on Ubuntu using Kolla and Ceph pacific. The issue I now have is that, if I try to provision a Ceph volume from an image stored in Ceph, I get the following error : https://pastebin.com/59EZbxKu By looking around a bit, I figured out that this is linked to the following mailing list email : https://www.spinics.net/lists/ceph-users/msg66016.html Ceph copy on write cloning breaking because Cinder can't work ifshow_image_direct_url is set to true is a pretty big issue, I believe. It's possible that this has been talked about before, but there's no other trace I could find outside of that Ceph mailing list email. Additionally, in Kolla, setting show_image_direct_url to false doesn't solve this issue because by default, Kolla appears to also set show_multiple_locations to true. By setting show_multiple_locations to false through a custom configuration, I am able to create volumes from glance Ceph images again (images have to be re-uploaded for this to work though). Of course, these volumes are not created through copy-on-write, because that's broken. I couldn't find any bug report pertaining to these issues. If they are not known, then I would be happy to make a bug report. I would just like to verify if this is all already known or not. -- Jean-Philippe Méthot Senior Openstack system administrator Administrateur système Openstack sénior PlanetHoster inc.

2 1

[all][tc] Technical Committee next weekly meeting on July 22th at 1500 UTC
by Ghanshyam Mann 22 Jul '21

22 Jul '21

Hello Everyone, Technical Committee's next weekly meeting is scheduled for July 22th at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, June 21th, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

OpenStack network tap interface random rxdrops for some VMs
by hai wu 21 Jul '21

21 Jul '21

There are random very slow rxdrops for certain OpenStack VMs for their network tap interfaces. Please note that this is rxdrop, NOT txdrop. I know we could tune txqueuelen and multi-queue for tap network interface txdrop issue, but is there any known way to tune for this tap network interface rxdrop issue? Thanks, Hai

3 12

[kolla][wallaby][CEPH] bluestore partions names
by Ignazio Cassano 21 Jul '21

21 Jul '21

Hello, I am going to installa ceph under kolla wallaby and I have some questions about how I must prepare partitions for bluestore on may ceph servers. Suppose I have 6 spinning disk: sdb sdc sdd sde sdf sdf And I have 1 nvme disk: nvme1 I am labelling each spinning disk as follows: sdb KOLLA_CEPH_OSD_BOOTSTRAP_1 sdc KOLLA_CEPH_OSD_BOOTSTRAP_2 and so on On nvme I create 6 partitions with the following names: KOLLA_CEPH_OSD_BOOTSTRAP_1_BS KOLLA_CEPH_OSD_BOOTSTRAP_2_BS and so on Is the above correct ? Thanks Ignazio

2 2