- openstack-discuss - lists.openstack.org

[nova][ci][gate] Recent gate CI instability: an incomplete list of failures
by melanie witt 04 Dec '23

04 Dec '23

Hey all, Last week I spent some time parsing through an assortment of recent CI failures on Nova patches and made a list, if anyone else is interested in looking at issues: https://etherpad.opendev.org/p/nova-ci-failures-minimal I didn't link specific job runs logs because those can and will expire after ~10 days, so for each item I included an OpenSearch query term that can be used to find sample logs and get an idea of the frequency of a given failure. Note that some query terms will register more than one appearance in a Gerrit change -- this can be due to failure in multiple jobs on the same change or the query might not be unique enough. I tried to make the queries as unique as possible. You can add columns like "build_change" and "build_name" to the search result by clicking "+" on the left bar "Available fields" to see duplication or to make comparisons. To search for a list of build failures, go to the OpenSearch dashboard: https://opensearch.logs.openstack.org/_dashboards/app/discover?security_ten… and login with Username: openstack and Password: openstack: Then copy paste the OpenSearch query into the search box and adjust the time range (the default of the last 30 minutes will probably have no results): -melwitt

1 0

[tc] Weekly TC Meeting tomorrow, Dec 5, 1800 UTC
by Jay Faulkner 04 Dec '23

04 Dec '23

Hi all, The weekly meeting of the Technical Committee is tomorrow, December 5, at 1800 UTC. This is the first meeting of the month, and will be held in a zoom video meeting -- you can attend by unredacting the following link: https://us06web (dot) zoom (dot) us/j/87108541765?pwd=emlXVXg4QUxrUTlLNDZ2TTllWUM3Zz09 -- the link will also be pasted in OFTC #openstack-tc before the meeting. As always, minutes will be kept in #openstack-tc during the meeting. The proposed agenda is: - Roll call - Follow up on tracked action items: https://meetings.opendev.org/meetings/tc/2023/tc.2023-11-28-18.00.html - Rosmaita work with fungi on the gerrit meta-acls (for unmaintained branch statuses) - Gate health check - Leaderless projects (gmann) - Implementation of Unmaintained branch statuses - 2024.1 TC Tracker - https://etherpad.opendev.org/p/tc-2024.1-tracker - Open Discussion and Reviews Thanks, Jay Faulkner

1 0

A few questions about OpenStack projects archival strategy
by arabat50＠gmail.com 04 Dec '23

04 Dec '23

Dear OpenStack team, I hope this email finds you well. I am Ali Arabat, a PhD student in Software Engineering at École de Technologie Supérieure in Montreal, University of Quebec, Canada, conducting research studies on the evolution of multi-component systems, among which OpenStack plays a significant role. Through this email, we would like to ask a few questions related to our study on OpenStack. Our study revealed a significant increase in the number of cross-component changes over time. However, Such cross-component changes have decreased after archiving a large number of projects beyond 500. Therefore, we would like to have clarification and answers to a few questions as follows: What are the main factors in archiving OpenStack projects? Are there any policies established by OpenStack to archive such projects i.e., end of life? Who generally decides to abandon a given project? We highly value your input and we genuinely appreciate your time. Thank you for considering our request, and we look forward to your valuable input. Best regards, --------------------------------------- Ali Arabat Ph.D. student in Software Engineering Department of Software Engineering and IT École de Technologie Supérieure (ÉTS) | University of Quebec 1100, rue Notre-Dame Ouest, Montréal, QC H3C 1K3, Canada

2 1

[cinder]Need help for BP and Bug review
by hao wang 02 Dec '23

02 Dec '23

Hi, folkers There is some help needed for BP and Bug review, please see the links below. Those bps are introduced for enhancing the query ability of Cinder resources. We have finished and merged a part of it years ago. But there is still some work to do, I have been working on them for almost three years and updated them according to the reviewing comments many times. So, if there are no more updates required, please help me to finish those works, and it's very appreciated to merge them into Cinder project. Thank you very much. [1] bp: https://review.opendev.org/c/openstack/cinder/+/740146 [2] bp: https://review.opendev.org/c/openstack/cinder/+/733062 [3] bug: https://review.opendev.org/c/openstack/cinder/+/734415 -- Best Wishes For You!

1 0

[charms] OpenStack Charms 2023.2 (Bobcat) release is now available
by Peter Matulis 01 Dec '23

01 Dec '23

The 2023.2 (Bobcat) release of the OpenStack Charms is now available. Please see the release notes for full details: https://docs.openstack.org/charm-guide/latest/release-notes/2023.2-bobcat.h… == Highlights == * OpenStack 2023.2 (Bobcat) OpenStack Bobcat is now supported on Ubuntu 22.04 LTS (via UCA) and Ubuntu 23.10 natively. * Two charms promoted to stable Both the watcher and watcher-dashboard charms have been promoted to stable. * Keystone role cache configuration The keystone charm now supports configuring the Keystone '[role] cache_time' option using a new 'role-cache-expiration' charm configuration option. If not set, this defaults to the global expiration_time (configurable by the 'dogpile-cache-expiration' charm configuration option). * Designate-bind allow-transfer configuration The designate-bind charm now supports configuring the BIND9 allow-transfer option. If not set, the corresponding feature is disabled. * Improved Vault snap channel refresh The refresh mechanism for the Vault snap managed within the vault charm has been improved. It now properly stops the service before performing a snap refresh. * Glance custom image properties The glance charm now supports configuring automatic custom image properties for imported images. * magnum charm: new option to expose cluster_user_trust The magnum charm has gained a new configuration option (cluster-user-trust) to control whether to assign a trust to the cluster user or not (upstream cluster_user_trust option). * nova-cloud-controller charm: new relation to openstack-dashboard A new relation informs the cloud of what hosts and IP addresses that can potentially access the OpenStack Dashboard. The nova-cloud-controller units use this information to configure the nova-serialproxy daemon to allow dashboard requests from web browsers. * Documentation updates The Charm Guide and the Deploy Guide were updated for OpenStack 2023.2 and Juju 3.x. Regular improvements and bug fixes. == OpenStack Charms team == The OpenStack Charms team can be contacted via real time chat: https://chat.charmhub.io/charmhub/channels/openstack-charms The Juju user forum is also popular for diagnosing technical issues: https://discourse.charmhub.io/c/juju/ == Thank you == Lots of thanks to the 34 contributors below who squashed bugs, enabled new features, and improved the documentation! Alex Litvinov Alex Kavanagh Brett Milford Chi Wai, Chan Chris MacNaughton Corey Bryant Dmitrii Shcherbakov Dmytro Kazantsev Edward Hope-Morley Facundo Ciccioli Felipe Reyes Frode Nordahl Fulvio Galeazzi Jadon Naas James Lin James Page Jesper Schmitz Mouridsen Jorge Merlino Joshua Zhang Liam Young Luciano Lo Giudice Martin Kalcok Neil Campbell Nobuto Murata Peter Matulis Peter Sabaini Rafael Lopez Rodrigo Barbieri Samuel Walladge Utkarsh Bhatt Wong Hong Han Yamen Hatahet Yoshi Kadokawa -- OpenStack Charms Team

1 0

[ironic] multi-conductor architecture, standalone and DHCP
by Dmitry Tantsur 01 Dec '23

01 Dec '23

Hi folks, This is detailed context on the discussions we've had recently about multi-conductor in metal3. I'm sending it to potentially interested people and cc'ing the openstack ML. The metal3 side is https://github.com/dtantsur/ironic-operator/issues/3 (although that covers a few more different issues). In metal3 we have Ironic without Neutron (or really any other OpenStack components). So far, we have only supported a single Ironic instance in a cluster, but we want to move towards a multi-conductor setup where every Kubernetes control plane node will host an Ironic (so, 3 Ironics in a normal deployment). The biggest problem stems from DHCP for the provisioning network. With Neutron, Ironic can configure it to direct a node to boot from a specific iPXE server. With Metal3, we have a static DHCP configuration. If we keep it the way it is, it will always direct a node to boot from the iPXE server on the same machine as dnsmasq (regardless of how many dnsmasq replicas we run). This has to be changed. I see several options, none of which are straightforward. (1) iPXE-level fix: boot config API This is what https://bugs.launchpad.net/ironic/+bug/2044561 proposes. If we stop using static iPXE scripts and start going through the Ironic API instead, Ironic can reach out to the correct conductor internally, always serving the right script. (2) dnsmasq DHCP implementation Ironic now has a non-neutron managed DHCP implementation through DHCP host and options files: https://opendev.org/openstack/ironic/src/branch/master/ironic/dhcp/dnsmasq.…. This could be a half of the solution. Since Metal3 does not support new node auto-discovery, we can run 3 copies of dnsmasq with dhcp-ignore=!known and rely on this DHCP provider to configure only 1 dnsmasq for each node. The other 2 will ignore the requests. The problem is in the DHCP range. As far as I understand, the 3 dnsmasqs must be configured with disjoint ranges to prevent leases from clashing. This seems non-trivial with Kubernetes which expects to create identical pods. Adam has suggested using annotations from inside ironic-operator, which could work if we find a way to read them inside the pod after it has started (which seems to pose a chicken-and-egg problem). Other workarounds are possible if we go down this path. E.g. ironic-operator could split the DHCP range and creating a mapping hostIP:subrange,hostIP2:subrange2 and pass it to pods. Then each pod will pick its subrange based on its IP address. Huge downside: every time a Kubernetes control plane node is replaced or changes its IP, the whole Ironic installation will get restarted. (2.1) Pre-created leases Another Metal3 contributor has suggested requiring an IPAM implementation (metal3 actually has one) that will create leases centrally, allowing us to use the same DHCP range. The only problem: how to pass them to dnsmasq in runtime without restarting all pods? We could probably have a small sidecar container next to dnsmasq that watches the Kubernetes API for new leases in IPAM. Or... (2.2) Dynamic DHCP and Kea The same contributor is working on Kea support for their deployment: https://hackmd.io/@7RNIJtmvSIeNpSOMTW9oqA/SkWhyr7Hp. There seem to be limitations in the free version, which caused them to write a tiny downstream plugin: https://gitlab.com/Orange-OpenSource/kanod/kanod-kea/-/blob/controlled-leas…. Changing the DHCP server is a dramatic step, I'm not sure if our small community can afford that. It may also pose problems for downstreams like OpenShift, requiring Metal3 to support both servers in the long run. Any thoughts are welcome. Dmitry

3 3

[neutron] Neutron drivers meeting
by Brian Haley 30 Nov '23

30 Nov '23

Hello Neutrinos, Please remember tomorrow (Friday) we have a Neutron drivers meeting. This is the agenda: https://wiki.openstack.org/wiki/Meetings/NeutronDrivers There is currently one topic: * [RFE] Use call_monitor_timeout of oslo.messaging RPCClient instead of custom backoff mechanism and hardcoded timeouts https://bugs.launchpad.net/neutron/+bug/2045058 Thanks, -Brian

1 0

[OpenInfra Live]: PTG Recap, Dec 7 at 15 UTC
by Kristin Barrientos 30 Nov '23

30 Nov '23

Hi everyone, Thanks for watching this week's episode of OpenInfra Live, featuring Zvonko Kaiser from NVIDIA. In case you missed it, you can find it here: https://www.youtube.com/live/urHjDdtuIa4?feature=shared Tune in next week for another OpenInfra Live, brought to you by the OpenStack and StarlingX community! Episode: Join us for a Project Teams Gathering (PTG) recap! Project leaders from Ironic, Technical Committee, Nova, Sunbeam, Manila, StarlingX and Neutron will provide recaps from discussions held during the PTG. Date and time: Thursday, Dec 7, 15 UTC YouTube: https://www.youtube.com/live/thidlQGX29M?feature=shared LinkedIn: https://www.linkedin.com/events/7132754094531747840/comments/ WeChat: The recording will be posted on OpenInfra Foundation WeChat after the live stream Speakers: Kendall Nelson, Jay Faulkner, Sylvain Bauza, James Page, Carlos Eduardo, Steve Geary and Brian Haley Thanks, Kristin Barrientos Marketing Coordinator OpenInfra Foundation

1 0

"openstack usage list" doesn't return information about all projects
by Michel Jouvin 30 Nov '23

30 Nov '23

Hi, We are using "openstack usage list" to follow resource usage by projects at our cloud. We recently discovered that ~1/3 of the resource consumed (mostly "big projects") are missing from the JSON/CSV produced where the Admin Overview menu in the Horizon dashboard displays everything. Is there a trivial reason for this? We tried both with a pretty old openstackclient (5.8) and the last one after applying the patch in https://github.com/openstack/python-openstackclient/commit/30462f57d48162d1… to fix the problem with date conversion. The result is the same. Thank you in advance for any help. Cheers, Michel

5 15

openstack Vm shutoff by itself
by AJ_ sunny 30 Nov '23

30 Nov '23

Hi team, I am using openstack kolla ansible on wallaby version and currently I am facing issue with virtual machine, vm is shutoff by itself and and from log it seems libvirt lifecycle stop event triggering again and again Logs:- 2023-10-16 08:48:10.971 7 WARNING nova.compute.manager [req-c7b731db-2b61-400e-917f-8645c9984696 f226d81a45dd46488fb2e19515 848 316d215042914de190f5f9e1c8466bf0 default default] [instance: 4b04d3f1-1fbd-4b63-b693-a0ef316ecff3] Received unexpected - vent network-vif-plugged-f191f6c8-dff5-4c1b-94b3-8d91aa6ff5ac for instance with vm_state active and task_state None. 2023-10-21 22:42:44.589 7 INFO nova.compute.manager [-] [instance: 4b04d3f1-1fbd-4b63-b693-a0ef316ecff3] VM Stopped (Lifecyc Event) 2023-10-21 22:42:44.683 7 INFO nova.compute.manager [req-1d99b87b-7ff7-462d-ab18-fbdec6bda71d -] [instance: 4b04d3f1- fbd-4b63-b693-a0ef316ecff3] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from ti e hypervisor (4). Updating power_state in the DB to match the hypervisor. 2023-10-21 22:42:44.811 7 WARNING nova.compute.manager [req-1d99b87b-7ff7-462d-ab18-fbdec6bda71d ----] [instance: 4b04d3f 1-1fbd-4b63-b693-a0ef316ecff3] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state : None, original DB power_state: 1, current VM power_state: 4 2023-10-21 22:42:44.977 7 INFO nova.compute.manager [req-1d99b87b-7ff7-462d-ab18-fbdec6bda71d -] [instance: 4b04d3f1-1 fbd-4b63-b693-a0ef316ecff3] Instance is already powered off in the hypervisor when stop is called. Thanks & Regards Arihant Jain +91 8299719369

5 15