June 2021 - openstack-discuss - lists.openstack.org

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

Live migration fails
by Szabo, Istvan (Agoda) 07 Aug '24

07 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

[all] Dynamic Zuul results table in Gerrit 3
by Radosław Piliszek 04 Aug '22

04 Aug '22

Hello Fellow OpenStack and OpenDev Folks! TL;DR click on [3] and enjoy. I am starting this thread to not hijack the discussion happening on [1]. First of all, I would like to thank gibi (Balazs Gibizer) for hacking a way to get the place to render the table in the first place (pun intended). I have been a long-time-now user of [2]. I have improved and customised it for myself but never really got to share back the changes I made. The new Gerrit obviously broke the whole script so it was of no use to share at that particular state. However, inspired by gibi's work, I decided to finally sit down and fix it to work with Gerrit 3 and here it comes: [3]. Works well on Chrome with Tampermonkey. Not tested others. I hope you will enjoy this little helper (I do). I know the script looks super fugly but it generally boils down to a mix of styles of 3 people and Gerrit having funky UI rendering. Finally, I'd also like to thank hrw (Marcin Juszkiewicz) for linking me to the original Michel's script in 2019. [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/019051… [2] https://opendev.org/x/coats/src/commit/444c95738677593dcfed0cfd9667d4c4f0d5… [3] https://gist.github.com/yoctozepto/7ea1271c299d143388b7c1b1802ee75e Kind regards, -yoctozepto

5 7

Nova not updating to new size of an extended in-use / attached cinder volume (Ceph RBD) to guest
by Christian Rohmann 28 Jun '22

28 Jun '22

Hello Openstack-Discuss, after some digging I found out that cinder learned to resize attached (in-use) volumes quite a while ago with the introduction of the initial "extend 'in-use' volume" feature: https://review.opendev.org/c/openstack/cinder/+/454287/ The support was then extended to also cover Ceph RBD backed volumes with: https://review.opendev.org/c/openstack/nova/+/613039/ Since this is only about the cinder part, I was wondering if nova would ever find out and would actively rescan the device / volume and quintessentially present an increased new size to the guest. Apparently this is where a certain volume-extended event comes into play: https://review.opendev.org/c/openstack/nova/+/454322/ that is to be emitted by cinder when a volume had been extened. I then went head and tried this with the openstack cli (as Horizon does not seem to support / offer to resize in-use volumes. I run Openstack Train with Ceph RBD as storage. First I ran into an issue with the openstack cli (https://bugs.launchpad.net/cinder/+bug/1871759/comments/2 <https://bugs.launchpad.net/cinder/+bug/1871759/comments/2>) but using cinder extend $volumeid $newsize I was able to resize in-use volumes just fine. The only thing missing was the propagation to the guest. I played around with SCSI rescans, but that did not work, reboots also failed to do anyting in this regards. Stopping and starting the VM did work, but why having the capability to online-resize an attached volume when not being able to resize the guest volume as well? Apparently there seems to be an old and somewhat similar observation / bug with https://bugs.launchpad.net/nova/+bug/1369465 So I was simply wondering if this is expected to be working? Are there any special settings / options I need to set to enable this feature? Thanks and with kind regards Christian

4 9

[nova][dev] Revisiting qemu emulation where guest arch != host arch
by Apsey, Christopher 05 Jan '22

05 Jan '22

All, A few years ago I asked a question[1] about why nova, when given a hw_architecture property from glance for an image, would not end up using the correct qemu-system-xx binary when starting the guest process on a compute node if that compute nodes architecture did not match the proposed guest architecture. As an example, if we had all x86 hosts, but wanted to run an emulated ppc guest, we should be able to do that given that at least one compute node had qemu-system-ppc already installed and libvirt was successfully reporting that as a supported architecture to nova. It seemed like a heavy lift at the time, so it was put on the back burner. I am now in a position to fund a contract developer to make this happen, so the question is: would this be a useful blueprint that would potentially be accepted? Most of the time when people want to run an emulated guest they would just nest it inside of an already running guest of the native architecture, but that severely limits observability and the task of managing any more than a handful of instances in this manner quickly becomes a tangled nightmare of networking, etc. I see real benefit in allowing this scenario to run natively so all of the tooling that exists for fleet management 'just works'. This would also be a significant differentiator for OpenStack as a whole. Thoughts? [1] http://lists.openstack.org/pipermail/openstack-operators/2018-August/015653… Chris Apsey Director | Georgia Cyber Range GEORGIA CYBER CENTER 100 Grace Hopper Lane | Augusta, Georgia | 30901 https://www.gacybercenter.org

2 2

[docs] Double headings on every page
by Peter Matulis 04 Nov '21

04 Nov '21

Hi, I'm hitting an oddity in one of my projects where the titles of all pages show up twice. Example: https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/wall… Source file is here: https://opendev.org/openstack/charm-deployment-guide/src/branch/master/depl… Does anyone see what can be causing this? It appears to happen only for the current stable release ('wallaby') and 'latest'. Thanks, Peter

5 6

Help with eventlet 0.26.1 and dnspython >= 2
by Thomas Goirand 04 Oct '21

04 Oct '21

Hi, As you may know, eventlet is incompatible with dnspython >= 2.0.0.0rc1. See [1] for the details. However, Debian unstable has 2.0.0. Would there be some good soul willing to help me fix this situation? I would need a patch to fix this, but I'm really not sure how to start. Cheers, Thomas Goirand (zigo)

8 15

[all][tc][goals] Project Specific PTL and Contributor Documentation: Week R-20 Update
by Ghanshyam Mann 17 Sep '21

17 Sep '21

Hello Everyone, As you know, we did not select any community-wide goal for Xena cycle and in PTG, TC decided to spend more time on the previous cycle pending community-wide goal work[1] One of the previous cycle (Ussuri) goals is 'Project Specific PTL and Contributor Documentation' which still not completed by many projects. - https://governance.openstack.org/tc/goals/selected/ussuri/project-ptl-and-c… I am starting the work to finish this goal which is pretty straight forwards and sending the updates in this ML thread. Please review the patches and merge them or push the patches for your project's repo if not yet done. Gerrit Topic: https://review.opendev.org/q/topic:%22project-ptl-and-contrib-docs%22+(stat… Tracking: https://storyboard.openstack.org/#!/story/2007236 Progress Summary: =============== * Projects completed: 32 * Projects for which patches are up and required to merge those: 10 * Projects for which patches are not yet up: 12 [1] https://etherpad.opendev.org/p/tc-xena-tracker (Item#2) -gmann

2 2

[all]Introduction to venus which is the project of log management and has been contributed to the OpenStack community
by Liye Pang(逄立业) 10 Sep '21

10 Sep '21

Hello everyone, after feedback from a large number of operations and maintenance personnel in InCloud OpenStack, we developed the log management project “Venus” for the OpenStack projects and that has contributed to the OpenStack community. The following is an introduction to “Venus”. If there is interest in the community, we are interested in proposing it to become an official OpenStack project in the future. Background In the day-to-day operation and maintenance of large-scale cloud platform, the following problems are encountered： l Time-consuming for log querying while the server increasing to thousands. l Difficult to retrieve logs, since there are many modules in the platform, e.g. systems service, compute, storage, network and other platform services. l The large amount and dispersion of log make faults are difficult to be discovered. l Because of distributed and interaction between components of the cloud platform, and scattered logs between components, it will take more time to locate problems. About Venus According to the key requirements of OpenStack in log storage, retrieval, analysis and so on, we introduced Venus project, a unified log management module. This module can provide a one-stop solution to log collection, cleaning, indexing, analysis, alarm, visualization, report generation and other needs, which involves helping operator or maintainer to quickly solve retrieve problems, grasp the operational health of the platform, and improve the management capabilities of the cloud platform. Additionally, this module plans to use machine learning algorithms to quickly locate IT failures and root causes, and improve operation and maintenance efficiency. Application scenario Venus played a key role in the following scenarios： l Retrieval: Provide a simple and easy-to-use way to retrieve all log and the context. l Analysis: Realize log association, field value statistics, and provide multi-scene and multi-dimensional visual analysis reports. l Alerts：Convert retrieval into active alerts to realize the error finding in massive logs. l Issue location: Establish a chain relationship and knowledge graphs to quickly locate problems. Overall structure The architecture of log management system based on Venus and elastic search is as follows: Diagram 0: Architecture of Venus venus_api: API module，provide API、rest-api service. venus_manager: Internal timing task module to realize the core functions of the log system. Current progress The current progress of the Venus project is as follows: l Collection：Develop fluentd collection tasks based on collectd to read, filter, format and send plug-ins for OpenStack, operating systems, and platform services, etc. l Index：Dealing with multi-dimensional index data in elasticsearch, and provide more concise and comprehensive authentication interface to return query results. l Analysis：Analyzing and display the related module errors, Mariadb connection errors, and Rabbitmq connection errors. l Alerts：Develop alarm task code to set threshold for the number of error logs of different modules at different times, and provides alarm services and notification services. l Location：Develop the call chain analysis function based on global_requested series, which can show the execution sequence, time and error information, etc., and provide the export operation. l Management：Develop configuration management functions in the log system, such as alarm threshold setting, timing task management, and log saving time setting, etc. Application examples Two examples of Venus application scenarios are as follows. 1. The virtual machine creation operation was performed on the cloud platform and it was found that the virtual machine was not created successfully. First, we can find the request id of the operation and jump to the virtual machine creation call chain page. Then, we can query the calling process, view and download the details of the log of the call. 2. In the cloud platform, the error log of each module can be converted into alarms to remind the users. Further, we can retrieve the details of the error log and error log statistics. Next step The next step of the Venus project is as follows: * Collection：In addition to fluent, other collection plugins such as logstash will be integrated. * Analysis: Explore more operation and maintenance scenarios, and conduct statistical analysis and alarm on key data. * display: The configuration, analysis and alarm of Venus will be integrated into horizon in the form of plugin. * location: Form clustering log and construct knowledge map, and integrate algorithm class library to locate the root cause of the fault. Venus Project Registry Venus library: https://opendev.org/inspur/venus You can grab the source code using the following git command: git clone https://opendev.org/inspur/venus.git Venus Demo Youtu.be: <https://youtu.be/mE2MoEx3awM> https://youtu.be/mE2MoEx3awM

7 15

[masakari] Deprecating process monitor - querying the community in this thread
by Radosław Piliszek 23 Aug '21

23 Aug '21

Hello users of Masakari, As the Masakari core team, we have discussed the deprecation/retirement of one of the monitors - the process monitor. The reason is simple - we consider its mission is better handled by other software such as Kubernetes, Docker, or systemd, depending on how you deploy your OpenStack cluster. I believe the process monitor might have made sense in the pre-containerisation, pre-systemd world but it does not seem to any longer. Please let us know, by replying to this mail, if you have a use case for the process monitor that cannot be handled by the aforementioned software. -yoctozepto

1 1

[tripleo] Changing TripleO's release model
by Wesley Hayutin 20 Aug '21

20 Aug '21

Greetings TripleO community! At the most recent TripleO community meetings we have discussed formally changing the OpenStack release model for TripleO [1]. The previous released projects can be found here [2]. TripleO has previously released with release-type[‘trailing’, ‘cycle-with-intermediary’]. To quote the release model doc: ‘Trailing deliverables trail the release, so they cannot, by definition, be independent. They need to pick between cycle-with-rc <https://releases.openstack.org/reference/release_models.html#cycle-with-rc> or cycle-with-intermediary <https://releases.openstack.org/reference/release_models.html#cycle-with-int…> models.’ We are proposing to update the release-model to ‘independent’. This would give the TripleO community more flexibility in when we choose to cut a release. In turn this would mean less backporting, less upstream and 3rd party resources used by potentially some future releases. To quote the release model doc: ‘Some projects opt to completely bypass the 6-month cycle and release independently. For example, that is the case of projects that support the development infrastructure. The “independent” model describes such projects.’ The discussion here is to merely inform the greater community with regards to the proposal and conversations regarding the release model. This thread is NOT meant to discuss previous releases or their supported status, merely changing the release model here [3] [0] https://etherpad.opendev.org/p/tripleo-meeting-items [1] https://releases.openstack.org/reference/release_models.html [2] https://releases.openstack.org/teams/tripleo.html [3] https://opendev.org/openstack/releases/src/branch/master/deliverables/xena

9 17

It's time to submit your OpenStack User Survey
by Allison Price 19 Aug '21

19 Aug '21

Hi everyone, Here’s my annual email requesting you complete the OpenStack User Survey [1] and share with your local communities / community friends who are operating OpenStack. Anonymous reposes and aggregated data from the User Survey are shared with the OPenStack PTLs and overall community to help understand operator software requirements. If you have never taken it before, please do! We welcome your feedback and I am happy to answer any questions you may have. If you are one of the many who have, all of your previous information should be stored, so you will just need to provide updates where relevant and re-save. Thank you all! Cheers, Allison [1] https://www.openstack.org/user-survey/survey-2021 <https://www.openstack.org/user-survey/survey-2021> Allison Price Director of Marketing & Community OpenInfra Foundation e: allison(a)openstack.org <mailto:allison@openstack.org>

1 1

[masakari] Compute service with name XXXXX not found.
by at 19 Aug '21

19 Aug '21

Hi, I have some problem with masakari. I can create segment (from CLI and Horizon), but can't create host (the same result from Horizon and CLI). openstack segment host create XXXXX COMPUTE SSH segment_id returns BadRequest: Compute service with name XXXXX could not be found. XXXXX is the name which Horizon suggest, and it's a name of compute host. openstack compute service list returns proper list with state up/enabled on compute hosts (zone nova) Maybe I misunderstood some parameters of host create? As "type" I use COMPUTE, what value should it be? From "Binary" column of openstack compute service list? What is "control_attributes" field, because documentation lacks preceise information what value should be there and what is it use for. Tried to found some info on this error but I haven't found anything... Thanks in advance for any help. Best regards Adam Tomas

4 7

[interop][cinder] review of next.json (part 2)
by Brian Rosmaita 09 Aug '21

09 Aug '21

Hello Interop WG, The Cinder team completed its discussion of what capabilities should be added as advisory in next.json at yesterday's weekly cinder meeting [0]. The tldr; is that we don't have anything to propose for inclusion in next.json at the present time. The team does have some general questions that would help us determine the suitability of some proposed capabilities. We'd like to invite someone from the Interop WG to the cinder xena R-9 virtual midcycle (Wednesday 4 August 2021, 1400-1600 UTC) so we can discuss this "live". So if someone could put 1400 UTC 4 August on their schedule for a 20 minute discussion, that would be very helpful. cheers, brian [0] http://eavesdrop.openstack.org/meetings/cinder/2021/cinder.2021-05-12-14.00…

3 8

Re: [neutron-vpnaas][kolla-ansible][victoria] Growing memory consumption with only one VPN connection
by at 09 Aug '21

09 Aug '21

Hi, 1. There is only one, bidirectional vpn connection between project networks in two regions 2. there is no (or almost no) traffic on this vpn link 3. second region is on one host (kolla all-in-one) and situation looks the same- with vpn service enabled there is excessive Memory use (growing also about 5-6MB/ minute and forcing the system to use swapfile) there is no big cpu usage (ofcourse also due to no traffic inside VPN) It's the same case even if there's no vpn connection configured. Best regards Adam Tomas Wysłane z iPhone'a > Wiadomość napisana przez Vinh Nguyen Duc <vinhducnguyen1708(a)gmail.com> w dniu 29.06.2021, o godz. 11:04: > > > Hi Adam, > Do you think the VPN connection is use CPU and memory to encrypt the VPN traffic? > > On Mon, Jun 28, 2021 at 14:30 Adam Tomas <bkslash(a)poczta.onet.pl <mailto:bkslash@poczta.onet.pl>> wrote: > Hi, > I have problem with neutron vpnaas - after enabling vpnaas plugin everything was ok at first. I was able to create VPN connection and communication is correct. But after a week of running vpnaas (only one VPN connection created/working!) I’ve noticed, that neutron-vpnaas takes more and more memory. I have 5 processes on each controller (there should be always five? Or it is changing dynamically?): > > 42435 1545384 0.6 8.6 5802516 5712412 ? S Jun15 110:50 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf > 42435 1545389 0.6 8.5 5735832 5645856 ? S Jun15 112:16 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf > 42435 1545378 0.5 8.5 5734192 5643620 ? S Jun15 108:09 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf > 42435 1545372 0.5 8.5 5731128 5641436 ? S Jun15 109:07 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf > 42435 1545369 0.6 8.4 5637084 5547392 ? S Jun15 114:21 /var/lib/kolla/venv/bin/python3.8 /var/lib/kolla/venv/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_vpnaas.conf > > now neutron_server takes over 27G of RAM on each controller: > > neutron_server running 10.2 27.2G > > After stopping all neutron containers and starting it again neutron takes a lot less memory: > > neutron_server running 0.5 583M > > but memory usage keeps growing (about 5-6 M every minute). > What’s wrong? When I disable vpnaas globally there’s no problem with excessive memory usage, so it’s vpnaas for sure… > > Best regards > Adam Tomas

3 2

[nova][neutron][deployment-projects] Re secbug #1734320
by Radosław Piliszek 04 Aug '21

04 Aug '21

Hello Folks! I am writing this because a recent patch proposed to DevStack [1] mentioned "when using ml2/ovs vif isolation should always be used to prevent cross tenant traffic during a live migration" which is related to secbug #1734320 "Eavesdropping private traffic" [2]. However, I've found that none of the publicly-available deployment projects seem to be using ``isolate_vif``. [3] [4] Should this be corrected? PS: I used the deployment-projects tag as a collective tag to avoid mentioning all the projects (as it is too long to write :-) ). I hope that relevant people see this if need be or someone passes the information to them. For now, I am curious whether this should actually be enforced by default with ML2/OVS. [1] https://review.opendev.org/c/openstack/devstack/+/796826 [2] https://bugs.launchpad.net/neutron/+bug/1734320 [3] https://codesearch.opendev.org/?q=%5Cbisolate_vif%5Cb&i=nope&files=&exclude… [4] https://github.com/search?p=1&q=isolate_vif&type=Code -yoctozepto

3 2

[horizon][stable] [release] Proposal to make stable/queen as EOL
by vishal manchanda 16 Jul '21

16 Jul '21

Hi all, As discussed in the last horizon weekly meeting[1] on 2021-06-23, I would like to announce that the horizon team decided to make stable/queen branch as EOL. Consider this mail as an official announcement for that. I would like to know if anyone still would like to keep this branch to be open otherwise, I will purpose an EOL patch for this branch. Thanks & Regards, Vishal Manchanda [1] https://meetings.opendev.org/meetings/horizon/2021/horizon.2021-06-23-15.00…

3 4

[nova][stable] Adding nova-core to nova stable branches
by Stephen Finucane 13 Jul '21

13 Jul '21

Happy Friday (again), Following on from my earlier email regarding os-vif stable branch permissions, I'd like to once again propose that we add nova-core to the list of teams with +2 rights on nova stable branches. Unlike the os-vif change, this has been discussed extensively in the past, most recently during the Victoria PTG [1] (search for "Can we add nova-core to nova-stable-maint") and therefore carries a little more baggage with it. Last time we discussed this, there were concerns from some existing stable cores that not all of nova-core were sufficiently well acquainted with stable branch policy and it was suggested that review velocity was sufficient. However, the stable team has remained pretty stable (heh) over the last couple of cycles and is staffed with developers with ample experience of both nova itself and the backport policies of nova and OpenStack as a whole. In addition, we continue to have a long tail of open stable reviews [2] from various people. The stable branches are increasingly important for users and organizations alike, and a faster, smoother backport policy encourages people to have an upstream-first mentality when it comes to maintaining a product built on these branches. The alternative is to handle these backports in a downstream-first or downstream- only manner, which results in duplication of effort and, over the long-term, potential difficulties keeping downstream and upstream in-sync. I think the time has long since passed where we should remove the artificial barrier that exists here. I expect all nova cores will have the maturity and sensibility to ask questions where uncertainty exists and only approve backports that they are confident and knowledgeable about. I do not expect existing stable cores will be sidelined either. Far from it, in fact. Existing stable cores will continue to be valuable both for their reviews and knowledge of the intricacies of stable policy. I proposed a waiting period of one week before making changes to os-vif permissions, but given the history around changes to nova stable core, I think we would benefit from some additional time to assuage any concerns that may still exist. I propose that we come to a decision by 09-Aug, or two weeks from now. If necessary, we can discuss this in real-time during the nova team meeting next Tuesday. Cheers, Stephen [1] https://etherpad.opendev.org/p/nova-victoria-ptg [2] https://review.opendev.org/q/project:openstack/nova+NOT+branch:master+is:op…

2 2

placement service issue (placement-status upgrade check failure)
by Deepak Mohanty 12 Jul '21

12 Jul '21

I need your help with installation of placement service on Ubuntu 20.04. I have followed the steps in: https://docs.openstack.org/placement/wallaby/install/install-ubuntu.html I am facing the following issue when I try to verify the installation: $ placement-status upgrade check Error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/oslo_upgradecheck/upgradecheck.py", line 196, in run return conf.command.action_fn() File "/usr/lib/python3/dist-packages/oslo_upgradecheck/upgradecheck.py", line 104, in check result = func_name(self, **kwargs) File "/usr/lib/python3/dist-packages/oslo_upgradecheck/common_checks.py", line 41, in check_policy_json policy_path = conf.find_file(conf.oslo_policy.policy_file) File "/usr/lib/python3/dist-packages/oslo_config/cfg.py", line 2543, in find_file raise NotInitializedError() oslo_config.cfg.NotInitializedError: call expression on parser has not been invoked I did a bit of debugging and found that the policy file is: policy.json. I could not find any policy.json on my machine. I created an empty policy.json at: /etc/placement/policy.json (user and group = placement). That too did not work. I have followed the steps in the install guide twice, deleting the placement database and other resources and recreating them. I continue to see the same issue. Thank you for your time.

5 6

[ patrole ] Understanding RBAC Permission Exceptions
by darumaseye 12 Jul '21

12 Jul '21

Hello, I am a researcher studying security of the Openstack's Policies. I'm very interested in the Patrole project and I would like to ask you some questions. In the documentation it is stated that Patrole compares two different types of results: - an expected one ( derived from oslo.policy ) - an actual one ( derived from an actual request to the API ). Given that, Patrole's tests can return 3 different values: 1) "Success" if actual result and expected result are both True or both False; 2) "RbacOverPermissionException" if actual result is True but expected result is False; 3) "RbacUnderPermissionException" in the other case. I can't understand in which cases the tests can return a value different from "Success". As far as I know, an API call should always be validated internally by oslo.policy's rules, before being allowed. So, in order for an API call to be accepted, oslo.policy's rules must allow that API call. It seems to me that the "expected" result ( derived from oslo.policy ) is always included in the actual result. Hoping that everything I said is correct, I would like to ask you: What issue is allowing such a strange behavior in Openstack APIs ? Why the expected results can be different from the actual ones? Are there publicly available examples showing "Failure" values? In general, are there any publicly available test cases that i can study to understand this? I Would like to thank you very much in advance; Best Regards, Jacopo

3 3

[all][infra][qa] Retiring Logstash, Elasticsearch, subunit2sql, and Health
by Clark Boylan 09 Jul '21

09 Jul '21

Hello everyone, Xenial has recently reached the end of its life. Our logstash+kibana+elasticsearch and subunit2sql+health data crunching services all run on Xenial. Even without the distro platform EOL concerns these services are growing old and haven't received the care they need to keep running reliably. Additionally these services represent a large portion of our resource consumption: * 6 x 16 vcpu + 60GB RAM + 1TB disk Elasticsearch servers * 20 x 4 vcpu + 4GB RAM logstash-worker servers * 1 x 2 vcpu + 2GB RAM logstash/kibana central server * 2 x 8 vcpu + 8GB RAM subunit-worker servers * 64GB RAM + 500GB disk subunit2sql trove db server * 1 x 4 vcpu + 4GB RAM health server To put things in perspective, they account for more than a quarter of our control plane servers, occupying over a third of our block storage and in excess of half the total memory footprint. The OpenDev/OpenStack Infra team(s) don't seem to have the time available currently to do the major lifting required to bring these services up to date. I would like to propose that we simply turn them off. All of these services operate off of public data that will not be going away (specifically job log content). If others are interested in taking this on they can hook into this data and run their own processing pipelines. I am sure not everyone will be happy with this proposal. I get it. I came up with the idea for the elasticsearch job log processing way back at the San Diego summit. I spent many many many hours since working to get it up and running and to keep it running. But pragmatism means that my efforts and the team's efforts are better spent elsewhere. I am happy to hear feedback on this. Thank you for your time. Clark

10 13

[Octavia dashboard] Impossible to fill the subnet field when creating a load balancer
by Taltavull Jean-Francois 09 Jul '21

09 Jul '21

Hi All, On my OSA Victoria deployment (on Ubuntu 20.04), the "Create Load Balancer" form does not allow me to fill the mandatory "subnet" field and therefore I can't create a load balancer with Horizon. On the other hand, Octavia works fine and I can create load balancers with the CLI, Terraform, etc. Has one of you already faced such a situation ? Best regards, Jean-François

3 6

[Ironic] Vendor-neutral Disk names
by Mahnoor Asghar 06 Jul '21

06 Jul '21

Dear all, There is [a proposal][1] in the metal3-io/baremetal-operator repository to extend the hardware RAID configuration to support ‘physical_disks’ and ‘controller’ fields in the 'target_raid_config' section. The user should be able to specify the disks for RAID configuration, in a vendor-agnostic way. (This requirement comes from the Airship project.) The names of the disks should be indicative of the physical location of the disks within the server. An algorithm to construct disk names is therefore needed, for this purpose. One possible algorithm was found in the [inventory module][2] of the [Redfish Tacklebox scripts][3]. To construct a disk name, it uses Redfish properties, specifically the ‘Drive’ resource> ‘Physical Location’ object> ‘Part Location’ property> ‘Service Label’ property. ([Link][4] to code) ([Link][5] to Redfish ‘Drive’ resource) If this property is empty, the resource type (String uptil the first dot encountered in the @odata.type field), and the ‘Id’ properties of the Drive resource are used to construct the disk name. ([Link][6] to code) For example, if the 'Drive'.'Physical Location'.'Part Location'.'Service Label' field is ‘Disk.Bay1.Slot0’, this is what the disk name will be. If this field is empty, and the resource name is ‘Drive’ and the resource ‘Id’ is ‘Disk.Bay1.Slot0’, the disk name will be: ‘Drive: Disk.Bay1.Slot0’. We would like to understand the values different vendors give in: - ‘Drive’ resource> ‘Physical Location’ object> ‘Part Location’ property> ‘Service Label’ property - The resource type for a Drive (@odata.type field) - The ‘Id’ property of the Drive resource Also, it would be helpful to understand the existing logic used by vendors to construct the disk names, including the (Redfish or other) properties used. This is so that a consensus can be reached for an algorithm to construct disk names. Any suggestions are welcome, thank you so much! [1]: https://github.com/metal3-io/metal3-docs/pull/148 [2]: https://github.com/DMTF/Redfish-Tacklebox/blob/master/redfish_utilities/inv… [3]: https://github.com/DMTF/Redfish-Tacklebox [4]: https://github.com/DMTF/Redfish-Tacklebox/blob/e429f70a79cfe288756618498ce4… [5]: https://redfish.dmtf.org/schemas/v1/Drive.v1_12_1.json [6]: https://github.com/DMTF/Redfish-Tacklebox/blob/e429f70a79cfe288756618498ce4… Best regards, Mahnoor Asghar Software Design Engineer xFlow Research Inc. mahnoor.asghar(a)xflowresearch.com www.xflowresearch.com

4 6

[wallaby][nova] CPU topology and NUMA Nodes
by Ammad Syed 06 Jul '21

06 Jul '21

Hi, I am using openstack wallaby on ubuntu 20.04 and kvm. I am working to make optimized flavor properties that should provide optimal performance. I was reviewing the document below. https://docs.openstack.org/nova/wallaby/admin/cpu-topologies.html I have two socket AMD compute node. The workload running on nodes are mixed workload. My question is should I use default nova CPU topology and NUMA node that nova deploys instance by default OR should I use hw:cpu_sockets='2' and hw:numa_nodes='2'. Which one from above provide best instance performance ? or any other tuning should I do ? The note in the URL (CPU topology sesion) suggests that I should stay with default options that nova provides. Currently it also works with libvirt/QEMU driver but we don’t recommend it in production use cases. This is because vCPUs are actually running in one thread on host in qemu TCG (Tiny Code Generator), which is the backend for libvirt/QEMU driver. Work to enable full multi-threading support for TCG (a.k.a. MTTCG) is on going in QEMU community. Please see this MTTCG project <http://wiki.qemu.org/Features/tcg-multithread> page for detail. Ammad

5 9