April 2024 - openstack-discuss - lists.openstack.org

[ironic][nova-scheduler] "host_state" in nova filters has stale aggregate information
by Michael Sherman 29 Apr '24

29 Apr '24

Hi all, We’re running into an issue, where for two sites with 150-250 ironic nodes on a single conductor and nova-compute instance, we’ve started to get “no hosts available” errors from nova scheduler. We’re using the blazar-nova filter to match on hosts in specifically tagged aggregates. After adding some debug logs, I found that the “host_state” object passed to the filter seems to have out-of-date aggregate information. Specifically, if I query the system with “openstack aggregate show …” or “openstack allocation candidate list”, I see the correct aggregate for the nodes in question, but the contents of “host_state” reflect a previous state. This “staleness” does not seem to correct itself over time, but is resolved by restarting the nova-scheduler process (actually restarting the kolla docker container, but the same effect). However the issues return over the course of a couple hours. We haven’t increased the number of nodes, or otherwise changed the hardware, so I’m not sure what could have triggered this issue. Any advice on further debugging steps would be greatly appreciated. Thank you! -- Michael Sherman Infrastructure Lead – Chameleon Computer Science, University of Chicago MCS, Argonne National Lab

3 13

[publiccloud-sig] Meeting on May 1st will be skipped
by Felix Kronlage-Dammers 29 Apr '24

29 Apr '24

Hi everyone, This Wednesday (May 1st) the next meeting of the publiccloud sig would be scheduled. However in a lot of the european countries this is a public holiday. As such Tobias and I decided that we will skip this call. A lot of us will be at the OpenInfra Day Berlin on the 15th of May - which would be the date for the next publiccloud sig, as such the one meeting on the 15th will happen during coffee break at the OID berlin :) (More infos on the OID Berlin here: https://oideurope2024.openinfra.dev/germany/#:~:text=15%20May%202024%20%7C%….) regards felix -- Felix Kronlage-Dammers Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[oslo] Rootwrap configuration issues (sudoers entries?)
by Francesco Di Nucci 29 Apr '24

29 Apr '24

Hello, I am configuring oslo rootwrap on my OpenStack instance, but I could only find documentation about nova-rootwrap like here <https://docs.openstack.org/oslo.rootwrap/latest/user/usage.html>. On the compute nodes it is now working, while I'm having troubles with eg. network nodes. I have added a sudoers entry on the like of the nova one: neutron ALL = (root) NOPASSWD: /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf * But I don't know if it is correct, is there any documentation about it? Regards Francesco Di Nucci

2 3

[kolla] Weekly meeting cancelled on 1st of May
by Michał Nasiadka 29 Apr '24

29 Apr '24

Hello Koalas, Due to 1st of May being public holiday in some European countries (including Poland) - Kolla weekly meeting is cancelled. See you next week - 8th May. Michal

1 0

[neutron] Bug deputy - week of Apr 22
by Sławek Kapłoński 29 Apr '24

29 Apr '24

Hi, I was bug deputy last week and below is summary of what new bugs opened for Neutron during that time: ## High - https://bugs.launchpad.net/neutron/+bug/2063043 - octavia/ovn: filedescriptor out of range in select() - assigned to Fernando Royo ## Medium - https://bugs.launchpad.net/neutron/+bug/2063081 - [ovn] connectivity to load balancer with FIP from a server connected to a router with external gateway - assigned to Fernando Royo - https://bugs.launchpad.net/neutron/+bug/2063459 - # DHCP agent might use default gateway of unrouted subnets - assigned to Sebastian Lohff, patch already proposed ## Low - https://bugs.launchpad.net/neutron/+bug/2063669 - DNS extensions reported as available even if not enabled - unassigned, seems like kind of low hanging fruit for someone who wants to start ## Others - https://bugs.launchpad.net/neutron/+bug/2063463 - [ovn-octavia-provider] hairpin_snat_ip not set - unassigned -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

Losing connection when connecting br-ex to bond
by sarasoltanm2000＠gmail.com 28 Apr '24

28 Apr '24

The problem started from where I realized I couldn't ping or ssh the instances in openstack then I realized that it was a networking problem and that br-ex was not connected to my interfaces. for some reason I had to create a bond with a static private ip address of the second interface. (172.16.8.2/24). so to solve this problem I tried to connect the bond to the br-ex but as soon as I connect the bond to br-ex I get disconnected from the server (server cannot ping its gateway) and br-ex state goes down, when I connect it to the bond by the following command: sudo ovs-vsctl add-port br-ex bond0 and when I try to bring the bridge br-ex back up, the bond is automatically deleted from the ports. this is the configuration of bridges before adding the bond port: #ovs-vsctl show Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge br-ex Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port phy-br-ex Interface phy-br-ex type: patch options: {peer=int-br-ex} Port br-ex Interface br-ex type: internal Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port br-int Interface br-int type: internal Port "tapf556c-c7" tag: 1 Interface "tapf556c-c7" type: internal Port "tapd2980-19" tag: 4095 Interface "tapd2980-19" type: internal Port "qr-8ea26-da" tag: 4095 Interface "qr-8ea26-da" type: internal Port "tap89cc3-c0" tag: 2 Interface "tap89cc3-c0" type: internal Port int-br-ex Interface int-br-ex type: patch options: {peer=phy-br-ex} ovs_version: "2.12.0" any idea why this happens and what is the solution? btw the instance is not connected to a router and it is basically connected to 2 networks (172.16.8.32/28 and 172.16.8.48/28)

2 1

[Neutron] Openvswitch forcing MTU 1450
by Karl Kloppenborg 28 Apr '24

28 Apr '24

Hey Neutron team. I have an issue where within the VXLAN network of tenancies the expected 1500 MTU is honoured but despite all MTU in the underlay being 9k and all networks and paths set to 1500, Openvswitch caps the MTU transiting north/south to 1450. This is causing considerable pain to some of our customers and despite trying to force the openvswitche bridges request_mtu to 1500, and all interfaces look right, it’s still caping at 1450. Note, TCP dumps show the pings set to 1500 getting to the network namespace on the openvswitch agent, so it appears something between the nat/snat process. VM1 private -> VM2 private (mtu 1500) = works VM1 private -> VM2 public (mtu 1500) = fails VM -> External (mtu 1500) = fails External Host -> VM (mtu 1500) = fails Adusting to 1422 on ping sizes works. Anyone experienced this? I cannot seem to make heads or tails of it. Thanks, Karl.

1 1

[nova] allow only specific instances on host aggregate
by Eugen Block 26 Apr '24

26 Apr '24

Hi, I'm playing around with host aggregates and the aggregate_instance_extra_specs for flavors. I successfully created a host aggregate, it allows me to schedule instances with a specific flavor on that aggregate only. But is there a way to prevent other VMs from starting on that aggregate (which were launched with different flavors)? The only thing coming to mind is to put that compute node(s) into a different availbility zone. Since the default is "nova", this would prevent anything else to start on that hypervisor. Is there anything I missed with aggregates? Any pointers are appreciated! Thanks! Eugen

3 7

how to set up configuration parameter for a datastore
by 龚永生 26 Apr '24

26 Apr '24

hi, chunyang I installed torve 2023.2, but failed to list the configuration parameter list: root@compute2:~# openstack datastore list +--------------------------------------+----------+ | ID | Name | +--------------------------------------+----------+ | ee74cbff-92d1-4f83-a06e-cd40fb45bac6 | dt_mysql | +--------------------------------------+----------+ root@compute2:~# openstack datastore version list dt_mysql +--------------------------------------+------------+---------+ | ID | Name | Version | +--------------------------------------+------------+---------+ | 82e71712-4e4d-4ff6-947f-8de4c1535d15 | ver_5.7.29 | 5.7.29 | | 8ba2ab5f-6dd1-4f25-b571-4031361a91cf | ver_8.3 | 8.3 | +--------------------------------------+------------+---------+ root@compute2:~# openstack database configuration parameter list 82e71712-4e4d-4ff6-947f-8de4c1535d15 and failed to create configruantion group as well: root@compute2:~# openstack database configuration create group1 '{"sync_binlog" : 1}' --datastore dt_mysql Configuration groups are not supported for this datastore: dt_mysql ver_8.3 (HTTP 422) could you tell me how to enable the configuration parameter list? regards. 龚永生浙江九州未来信息科技有限公司 99CLOUD Co. Ltd. 邮箱（Email）：gong.yongsheng(a)99cloud.net 地址：海淀区上地六街研华大厦5层南侧手机（Mobile)：+86-18618199879 公司网址（WebSite）：http://99cloud.net

1 0

Instance stuck in BUILD status
by 西岭 26 Apr '24

26 Apr '24

Dear openstack trove module developers, I was having trouble creating a trove instance and kept showing build status. My create command is: trove create mysql_zh-11 1 --size 5 --nic net-id=5f161d13-0cfb-4707-9bd6-f23850d86fbf --databases dbtest --users user:password --datastore mysql --datastore_version mysql-5.6 After entering the trove list command, the final state is: | bdcb298c-86d3-42c5-b7af-1cc0c7f93904 | mysql_zh-11 | mysql | mysql-5.6 | BUILD | 1 | 5 | RegionOne | The image I used was downloaded from the openstack website: trove-train-mysql-ubuntu-xenial Enter the openstack server list command and it will display: 2bf580eb-3c93-4689-9b53-e76307beb241 | mysql_zh-11 | ACTIVE | external-net=192.168.0.56 | trove-train-mysql-ubuntu-xenial | mysql-mini | I don't know the username and password of the image, so I can't see the specific information inside the cloud host. In addition, when I used the tool to change the login username and password of the above image, and created the trove instance with the image, after waiting 3600 seconds, it showed: trove.taskmanager.models [-] Service status: ERROR, service error description: guestagent error: TroveError: Failed to get server 0908ee4e-9f1d-444a-9c69-c8b8cc9d8938 for instance 03c2a10c-87f1-4169-9cf7-13735ff8756c, error: Failed to validate token (HTTP 404) (Request-ID: req-9604fb4c-4aae-4351-8996-5ee4dfbd9ad3) This time, I was able to log in to the cloud host generated during the creation of trove. After entering, I found that the guest-agent service failed to start. The reasons were: sudo chown -R ubuntu:root /etc/trove/conf.d cannot be executed. I wonder if it is because the password of the image has been changed? Then, I used the same image to create the trove instance again, and during the instance creation process, I went to the cloud host and started the guest-agent service manually. The result was the same. What else did I miss when creating the instance? How to solve this problem? See attachment for details of trove log. Thank you very much for your help. ----------------------------------------------------- Xi Ling, from Qingdao, China April 25, 2024

2 1