August 2023 - openstack-discuss - lists.openstack.org

[OSA] CEPH libvirt secrets
by Murilo Morais 11 Aug '23

11 Aug '23

Good evening everyone! Guys, I'm having trouble adding a second CEPH cluster for Nova/Cincer to consume. I'm using the following configuration: cinder_backends: ceph1: volume_driver: cinder.volume.drivers.rbd.RBDDriver rbd_pool: ceph1_vol rbd_ceph_conf: /etc/ceph/ceph1.conf rbd_store_chunk_size: 8 volume_backend_name: ceph1 rbd_user: ceph1_vol rbd_secret_uuid: "{{ cinder_ceph_client_uuid }}" report_discard_supported: true ceph2: volume_driver: cinder.volume.drivers.rbd.RBDDriver rbd_pool: ceph2_vol rbd_ceph_conf: /etc/ceph/ceph2.conf rbd_store_chunk_size: 8 volume_backend_name: ceph2 rbd_user: ceph2_vol rbd_secret_uuid: "{{ cinder_ceph_client_uuid2 }}" report_discard_supported: true ceph_extra_confs: - src: /etc/openstack_deploy/ceph/ceph1.conf dest: /etc/ceph/ceph1.conf client_name: ceph1_vol keyring_src: /etc/openstack_deploy/ceph/ceph1_vol.keyring keyring_dest: /etc/ceph/ceph1.client.ceph1_vol.keyring secret_uuid: '{{ cinder_ceph_client_uuid }}' - src: /etc/openstack_deploy/ceph/ceph2.conf dest: /etc/ceph/ceph2.conf client_name: ceph2_vol keyring_src: /etc/openstack_deploy/ceph/ceph2_vol.keyring keyring_dest: /etc/ceph/ceph2.client.ceph2_vol.keyring secret_uuid: '{{ cinder_ceph_client_uuid2 }}' But when executing the `virsh secret-list` command it only shows the UUID of "cinder_ceph_client_uuid". Both "cinder_ceph_client_uuid" and "cinder_ceph_client_uuid2" are defined in "user_secrets.yml". I have a slight impression that I didn't configure something, but I don't know what, because I didn't find anything else to be done, according to the documentation [1], or it went unnoticed by me. [1] https://docs.openstack.org/openstack-ansible-ceph_client/latest/configure-c… Thanks in advance!

2 5

[nova] VM gets stuck in BUILD status after SQL DBConnectionError error in nova-conductor
by Pavlo Bychikhin 10 Aug '23

10 Aug '23

Hi all, Sometimes when I create a bunch of VMs (40) at once, one of them gets stuck in BUILD status. From logs I found that this happens due to SQL error in nova-conductor: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query'). There is an attempt to schedule the VM: .... nova-scheduler"sending reply msg_id: 593a990393d5439580870d8412ab49f3 reply queue: reply_a0992fe3f9924cb1b45bfa8b27057580 time elapsed: 1.49198414385s" And right after that the SQL error happens: nova-conductor "Exception during message handling" "DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT instance_id_mappings.created_at AS instance_id_mappings_created ......" And after that the VM stays in BUILD status forever. My setup is Galera cluster behind Haproxy load balancer. I checked various DB server timeouts - they are totally fine. Ones in MySQL correspond ones in nova.conf and haproxy.cfg. In the MySQL logs there are messages like: [Warning] Aborted connection 17798924 to db: 'nova' user: 'nova' host: 'xxx.xxx.xxx.xxx' (Got an error reading communication packets) In the Haproxy logs there are messages like: <134>2023-08-05T08:44:34.325338+00:00 dbs01 haproxy[1275]: xxx.xxx.xxx.xxx:44880 [05/Aug/2023:08:42:44.441] mysql_cluster_nova_conn mysql_cluster_nova_conn/dbs02 1/0/109884 11438 CD 4125/1884/1884/1884/0 0/0 "CD" means "The client unexpectedly aborted during data transfer" So the Haproxy believes the connection is closed at client side. I checked the network interface counters - they are also fine (no errors, drops etc). I found some discussions in the Internet. People say, when the DB is behind load balancer - such 'Lost connection' events are pretty normal and not harmful to the Openstack components. I would agree with them. I think, Nova should be able to recover from accidental DB connection lost. Or to set the ERROR status to the VM if that's impossible. My Nova version is 17.0.13 (Queens). May be someone encountered such issue and knows how to fix? Or that's a bug in Nova and I need to upgrade it to a more recent version? Thanks in advance ---- Best regards, Pavlo Bychikhin

1 0

Kayobe VLAN & bond configuration
by Stefan Pinter 10 Aug '23

10 Aug '23

Hi! I am trying to set up Openstack with Kayobe and right now I’m very stuck at VLAN configuration with LACP bonds on my two network nodes. Last Friday the network even came to a hold when activating Openvswitch because it started a broadcast storm or something in the whole company even though I have 2 VLANs just for this project. So I am trying to do it like this: #bond configuration dummy_net_interface: bond0 dummy_net_bootproto: static dummy_net_bond_slaves: - eno12399np0 - eno12409np1 dummy_net_bond_mode: 802.3ad dummy_net_bond_miimon: 100 dummy_net_bond_lacp_rate: fast #configure mgmt interface mgmt_net_interface: brmgmt mgmt_net_bridge_ports: - bond0 #external net work interface and bridge configuration #ext_net_bridge_ports: ext_net_interface: "brbond{{ ext_net_vlan }}" ext_net_bridge_ports: - "bond0.{{ ext_net_vlan }}" Part of my networks.yml … external_net_names: - ext_net … ext_net_cidr: 172.31.246.0/24 ext_net_no_ip: true ext_net_vlan: 705 Last time this worked, when I ran a “kayobe overcloud host configure” (the node had network connection) but then “kayobe overcloud service deploy” killed the network when it started the Openvswitch. Now (today) this configuration doesn’t even allow to reach the network/(to be)neutron nodes (they lose network connection) when the playbook restarts system-networkd. It already creates OVS PHY bridges and lots of stuff even though I ran “overcloud service destroy” before that. Should it do that at this stage? Are there any examples to set up a network node with bonds and vlans and external neutron networks out there? Any suggestions? Thank you 😊 Kind regards Stefan ________________________________ BearingPoint GmbH Sitz: Wien Firmenbuchgericht: Handelsgericht Wien Firmenbuchnummer: FN 175524z The information in this email is confidential and may be legally privileged. If you are not the intended recipient of this message, any review, disclosure, copying, distribution, retention, or any action taken or omitted to be taken in reliance on it is prohibited and may be unlawful. If you are not the intended recipient, please reply to or forward a copy of this message to the sender and delete the message, any attachments, and any copies thereof from your system.

1 0

Kayobe VLAN & bond configuration
by BPt-FM BBS IaaS 10 Aug '23

10 Aug '23

Hi! I am trying to set up Openstack with Kayobe and right now I’m very stuck at VLAN configuration with LACP bonds on my two network nodes. Last Friday the network even came to a hold when activating Openvswitch because it started a broadcast storm or something in the whole company even though I have 2 VLANs just for this project. So I am trying to do it like this: #bond configuration dummy_net_interface: bond0 dummy_net_bootproto: static dummy_net_bond_slaves: - eno12399np0 - eno12409np1 dummy_net_bond_mode: 802.3ad dummy_net_bond_miimon: 100 dummy_net_bond_lacp_rate: fast #configure mgmt interface mgmt_net_interface: brmgmt mgmt_net_bridge_ports: - bond0 #external net work interface and bridge configuration #ext_net_bridge_ports: ext_net_interface: "brbond{{ ext_net_vlan }}" ext_net_bridge_ports: - "bond0.{{ ext_net_vlan }}" Part of my networks.yml … external_net_names: - ext_net … ext_net_cidr: 172.31.246.0/24 ext_net_no_ip: true ext_net_vlan: 705 Last time this worked, when I ran a “kayobe overcloud host configure” (the node had network connection) but then “kayobe overcloud service deploy” killed the network when it started the Openvswitch. Now (today) this configuration doesn’t even allow to reach the network/(to be)neutron nodes (they lose network connection) when the playbook restarts system-networkd. It already creates OVS PHY bridges and lots of stuff even though I ran “overcloud service destroy” before that. Should it do that at this stage? Are there any examples to set up a network node with bonds and vlans and external neutron networks out there? Any suggestions? Thank you 😊 The information in this email is confidential and may be legally privileged. If you are not the intended recipient of this message, any review, disclosure, copying, distribution, retention, or any action taken or omitted to be taken in reliance on it is prohibited and may be unlawful. If you are not the intended recipient, please reply to or forward a copy of this message to the sender and delete the message, any attachments, and any copies thereof from your system.

1 0

Exploring the Intersection of AI and Open Source: Aug. 10 at 15 UTC
by Kristin Barrientos 09 Aug '23

09 Aug '23

Hi everyone, This week’s OpenInfra Live episode is brought to you by open source community leaders. Episode: If you stand at the intersection of AI and open source you’ll find a very active discussion around how each concept can apply to the other but you’ll find few definitive answers. This panel of open source community leaders will discuss the current state of AI & open source, the collaborative efforts underway to improve global understanding of this emerging use case and how open source community members can participate in solving today’s unanswered questions. Date and time: Aug 10, 2023 at 10:00 CST (15:00 UTC) You can watch us live on: YouTube: https://www.youtube.com/watch?v=LLujjvEIbPk LinkedIn: https://www.linkedin.com/events/7094342351971598337/comments/ WeChat: The recording will be posted on OpenInfra Foundation WeChat after the live stream Speakers: Mark Collier, Pamela Chestek, Stefano Maffulli, Armstrong Foundjem, Stephen O'Grady Have an idea for a future episode? Share it now at ideas.openinfra.live. Thanks, Kristin Barrientos Marketing Coordinator OpenInfra Foundation

1 0

[operators][swift][s3][ceph] interoperability s3/swift protocol using radosgw
by Benedikt Trefzer 09 Aug '23

09 Aug '23

Hi Openstack list I'm stuck with a problem: We implemented object storage using ceph radosgw, using keystone authentication and rgw multi-tenancy. So far so good. If I create a container named 'test-swift' using swift protocol, radosgw-admin shows me a bucket created as: "*\/test-swift" (* is the uuid of the opensstack project) which seems to be the correct behaviour. If I create a bucket with s3cmd named 'test-s3', radosgw-admin shows the bucket as: "test-s3" which is not the expected behaviour since this is not multi tenancy ! Can anyone point me to the right direction to solve this ? Could the use of v2 signatures for S3 be a problem ? Further information: Ceph version: Jewel Openstack keystone: train Access to rgw through Apache fcgi (sitting behind haproxy nodes) Thanks for all the hints Benedikt PS: - I know that Jewel and train are old versions, but ... - I know that S3/Swift protocols are not fully supported using rgw

1 0

Cinder Bug Report 2023-08-09 - No new Bugs This Week!
by Rajat Dhasmana 09 Aug '23

09 Aug '23

1 0

[skyline] Username or password is incorrect
by Satish Patel 09 Aug '23

09 Aug '23

Folks, Try to install skyline UI to replace horizon using doc: https://docs.openstack.org/skyline-apiserver/latest/install/docker-install-… Everything went well and I got a login page on http://x.x.x.x:9999 also it pulled Region/Domains. When I am trying to login with my account, I get an error: Username or Password is incorrect. I am using sqlite DB for skyline as per documents. No errors in logs command $ docker logs skyline When I use Chrome Developer Tools then it was indicating an error in these URLs. http://openstack.example.com:9999/api/openstack/skyline/api/v1/profile http://openstack.example.com:9999/api/openstack/skyline/api/v1/policies 401 Unauthorized ( {"detail":"no such table: revoked_token"} ) Find attached screenshot

2 2

[cinder] prevent change volume type from different AZ
by Nguyễn Hữu Khôi 09 Aug '23

09 Aug '23

Hello guys. I have 2 AZ and 1 volume backend on each AZ. My goal is when 1 volume was created from the AZ then user cannot change it to another volume type on a different AZ. Thank you. Nguyen Huu Khoi

2 2

[neutron] User survey Neutron related questions
by Slawek Kaplonski 09 Aug '23

09 Aug '23

Hi, As we talked on the Neutron team meeting yesterday [1] I prepared etherpad with Neutron related questions and possible answers from the User Survey [2]. I already added there some of my comments. Please check it and comment on it too. We can then discuss it during next week's Neutron team meeting and finally maybe send some proposals to change something there. [1] https://meetings.opendev.org/meetings/networking/2023/networking.2023-08-08… [2] https://etherpad.opendev.org/p/neutron-user-survey-questions-2023 -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0