October 2022 - openstack-discuss - lists.openstack.org

OpenStack Ansible Service troubleshooting
by John Ratliff 04 Oct '22

04 Oct '22

We've started deploying new Xena clusters with openstack-ansible. We keep running into problems with some parts of openstack not working. A service will fail or need restarted, but it's not clear which one or why. Recently, one of our test clusters (2 hosts) stopped working. I could login to horizon, but I could not create instances. At first it told me that a message wasn't answered quick enough. I assumed the problem was rabbitmq and restarted the container, but this didn't help. I eventually restarted every container and the nova- compute and haproxy services on the host. But this didn't help either. I eventually rebooted both hosts, but this made things worse (I think I broke the galera cluster doing this). After bootstrapping the galera cluster, I can log back into horizon, but I still cannot create hosts. It tells me "Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance [UUID]" If I look at the journal for nova-compute, I see this error: "libvirt.libvirtError: Failed to activate service 'org.freedesktop.machine1': timed out " Looking at systemd-machined, it won't start due to "systemd- machined.service: Job systemd-machined.service/start failed with result 'dependency'." I'm not sure what "dependency" it's referring to. In the cluster that does work, this service is running. But on both hosts on the cluster that do not, this service is not running. What should I be looking at here to fix? -- John Ratliff Systems Automation Engineer GlobalNOC @ Indiana University

2 4

[cinder] cancelling this week's meeting
by Rajat Dhasmana 04 Oct '22

04 Oct '22

Hello Argonauts, I won't be around to take the cinder meeting tomorrow (i.e. 05th October, 2022) and we are close to the PTG so there aren't many things to discuss. The agenda as of now is empty[1] so it shouldn't be a problem to skip this week's meeting. Also I would like to take this opportunity to remind everyone to add topics to the PTG etherpad[2] and that would be a better utilization of the cinder upstream meeting time. [1] https://etherpad.opendev.org/p/cinder-zed-meetings#L102 [2] https://etherpad.opendev.org/p/antelope-ptg-cinder-planning Thanks and regards Rajat Dhasmana

1 0

PTG Schedule and Reminders
by Kendall Nelson 03 Oct '22

03 Oct '22

Hello Everyone! The October 2022 Project Teams Gathering is right around the corner and the schedule is being setup by your team leads! Slots are going fast, so make sure to get your time booked ASAP! You can find the schedule and available slots on the PTGbot website [1]. The PTGbot site is the during-event website to keep track of what's being discussed and any last-minute schedule changes. It is driven via commands in the #openinfra-events IRC channel (on the OFTC network) where the PTGbot listens. If you have questions about the commands that you can give the bot, check out the documentation here[2]. Also, if you haven’t connected to IRC before, here are some docs on how to get setup![3] Lastly, please don't forget to register[4] (it is free after all!). Please let us know if you have any questions via email to ptg(a)openinfra.dev. Thanks! -Kendall (diablo_rojo) [1] PTGbot Site: https://ptg.opendev.org/ptg.html [2] PTGbot Documentation: https://github.com/openstack/ptgbot#open-infrastructure-ptg-bot [3] Setup IRC: https://docs.openstack.org/contributors/common/irc.html [4] PTG Registration: https://openinfra-ptg.eventbrite.com

1 0

[neutron] PTG topics and scheduling
by Rodolfo Alonso Hernandez 03 Oct '22

03 Oct '22

Hello all: Based on the schedule polls results, I've booked the Neutron meetings from Tuesday to Thursday in Mitaka channel [1], from 13UTC to 16UTC. Of course, if that is not enough, we can always use Friday to continue any pending conversation. The operator hour (actually 2), will be on Friday (pending for reservation), from 13UTC to 15UTC. Please continue adding any topic you want to discuss in the Neutron etherpad [2]. There is a specific section for the Nova-Neutron cross-team meeting. If you have any doubt or question, do not hesitate to let me know (IRC: ralonsoh, mail: ralonsoh(a)redhat.com). You can also ping any core reviewer in #openstack-neutron channel. See you in a few weeks! [1]https://ptg.opendev.org/ptg.html [2]https://etherpad.opendev.org/p/neutron-antelope-ptg

3 3

[neutron] Bug deputy 26 September to 2 October
by Rodolfo Alonso Hernandez 03 Oct '22

03 Oct '22

Hello Neutrinos: This is the list of bugs of the past week: Medium: * https://bugs.launchpad.net/neutron/+bug/1991092: Retry port provisioning for "nova:xxx" device_owner ports only. Assigned * https://bugs.launchpad.net/neutron/+bug/1991222: neutron.provisioningblocks WSREP: referenced FK check fail. Assigned Low: * https://bugs.launchpad.net/neutron/+bug/1990999: Install and configure compute node in Neutron. Unassigned * https://bugs.launchpad.net/neutron/+bug/1991398: Update port with given IPv6 address on SLAAC/stateless_dhpc subnets fails always when IP address is given. Assigned Wishlist: * https://bugs.launchpad.net/neutron/+bug/1990842: [RFE] Expose Open vSwitch other_config column in the API. Assigned * https://bugs.launchpad.net/neutron/+bug/1991000: [tripleo] Provide a tag to the container that will be used to kill it. Assigned Regards.

1 0

suggestion goal for zed: getting swift and glance to support uwsgi
by Thomas Goirand 02 Oct '22

02 Oct '22

Hi, As you may know, almost all of OpenStack now supports running on UWSGI. However, 2 projects remain incompatible with it: Glance and Swift. I've heard that it's now fixed with Glance, but I haven't checked the fact for myself. Has anyone already run Glance (with Swift as backend) under uwsgi, instead of eventlet? What's the status? Has all of the remaining issues been tackled? As for Swift, while upstream has even examples on how to run Swift over uwsgi, experience (in heavy load production) demonstrated that there are many issues running under uwsgi. That's a shame, because the uwsgi server makes most services (proxy, object, container and account servers) run twice as fast. Currently, the proxy and object servers aren't following the RFCs, and are incompatible with uwsgi, especially when chunks are involved (SLO/DLO in the pipeline). Also, it doesn't look like the Swift servers are thread safe. Switching to more than one thread just fails under heavy load. As a result, in production, we had to run double the amount of servers to handle the load. That's a huge waste of resources, IMO. It'd be great if upstream took it seriously to propose a uwsgi binary by default, and if the CI was using it. Note that I've already proposed such a patch [1], but it received zero votes, and not really getting attention upstream. Your thoughts anyone? Cheers, Thomas Goirand (zigo) P.S: This is just a suggestion for the work in the next cycle, I don't think I have the bandwidth to work on each individual project, and that my time is better spent on what I do best: Debian packaging of OpenStack and cluster deployment integration. [1] https://review.opendev.org/c/openstack/swift/+/821192

2 2