November 2021 - openstack-discuss

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

Live migration fails
by Szabo, Istvan (Agoda) 06 Aug '24

06 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

[all] Dynamic Zuul results table in Gerrit 3
by Radosław Piliszek 04 Aug '22

04 Aug '22

Hello Fellow OpenStack and OpenDev Folks! TL;DR click on [3] and enjoy. I am starting this thread to not hijack the discussion happening on [1]. First of all, I would like to thank gibi (Balazs Gibizer) for hacking a way to get the place to render the table in the first place (pun intended). I have been a long-time-now user of [2]. I have improved and customised it for myself but never really got to share back the changes I made. The new Gerrit obviously broke the whole script so it was of no use to share at that particular state. However, inspired by gibi's work, I decided to finally sit down and fix it to work with Gerrit 3 and here it comes: [3]. Works well on Chrome with Tampermonkey. Not tested others. I hope you will enjoy this little helper (I do). I know the script looks super fugly but it generally boils down to a mix of styles of 3 people and Gerrit having funky UI rendering. Finally, I'd also like to thank hrw (Marcin Juszkiewicz) for linking me to the original Michel's script in 2019. [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/019051… [2] https://opendev.org/x/coats/src/commit/444c95738677593dcfed0cfd9667d4c4f0d5… [3] https://gist.github.com/yoctozepto/7ea1271c299d143388b7c1b1802ee75e Kind regards, -yoctozepto

5 7

[all] PyCharm Licenses Renewed till July 2021
by Swapnil Kulkarni 13 Jul '22

13 Jul '22

Hello, Happy to inform you the open source developer license for Pycharm has been renewed for 1 additional year till July 2021. Best Regards, Swapnil Kulkarni coolsvap at gmail dot com

8 13

Nova not updating to new size of an extended in-use / attached cinder volume (Ceph RBD) to guest
by Christian Rohmann 28 Jun '22

28 Jun '22

Hello Openstack-Discuss, after some digging I found out that cinder learned to resize attached (in-use) volumes quite a while ago with the introduction of the initial "extend 'in-use' volume" feature: https://review.opendev.org/c/openstack/cinder/+/454287/ The support was then extended to also cover Ceph RBD backed volumes with: https://review.opendev.org/c/openstack/nova/+/613039/ Since this is only about the cinder part, I was wondering if nova would ever find out and would actively rescan the device / volume and quintessentially present an increased new size to the guest. Apparently this is where a certain volume-extended event comes into play: https://review.opendev.org/c/openstack/nova/+/454322/ that is to be emitted by cinder when a volume had been extened. I then went head and tried this with the openstack cli (as Horizon does not seem to support / offer to resize in-use volumes. I run Openstack Train with Ceph RBD as storage. First I ran into an issue with the openstack cli (https://bugs.launchpad.net/cinder/+bug/1871759/comments/2 <https://bugs.launchpad.net/cinder/+bug/1871759/comments/2>) but using cinder extend $volumeid $newsize I was able to resize in-use volumes just fine. The only thing missing was the propagation to the guest. I played around with SCSI rescans, but that did not work, reboots also failed to do anyting in this regards. Stopping and starting the VM did work, but why having the capability to online-resize an attached volume when not being able to resize the guest volume as well? Apparently there seems to be an old and somewhat similar observation / bug with https://bugs.launchpad.net/nova/+bug/1369465 So I was simply wondering if this is expected to be working? Are there any special settings / options I need to set to enable this feature? Thanks and with kind regards Christian

4 9

[oslo] Propose to EOL stable/queens, stable/rocky on all the oslo scope
by Herve Beraud 24 Jun '22

24 Jun '22

Hi, On our last meeting of the oslo team we discussed the problem with broken stable branches (rocky and older) in oslo's projects [1]. Indeed, almost all these branches are broken. Előd Illés kindly generated a list of periodic-stable errors on Oslo's stable branches [2]. Given the lack of active maintainers on Oslo and given the current status of the CI in those branches, I propose to make them End Of Life. I will wait until the end of month for anyone who would like to maybe step up as maintainer of those branches and who would at least try to fix CI of them. If no one will volunteer for that, I'll EOLing those branches for all the projects under the oslo umbrella. Let us know your thoughts. Thank you for your attention. [1] https://meetings.opendev.org/meetings/oslo/2021/oslo.2021-10-04-15.00.log.t… [2] http://lists.openstack.org/pipermail/openstack-discuss/2021-July/023939.html -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud

5 5

[TripleO] Core team cleanup
by James Slagle 30 Mar '22

30 Mar '22

Hello, I took a look at our core team, "tripleo-core" in gerrit. We have a few individuals who I feel have moved on from TripleO in their focus. I looked at the reviews from stackalytics.io for the last 180 days[1]. These individuals have less than 6 reviews, which is about 1 review a month: Bob Fournier Dan Sneddon Dmitry Tantsur Jiří Stránský Juan Antonio Osorio Robles Marius Cornea These individuals have publicly expressed that they are moving on from TripleO: Michele Baldessari wes hayutin I'd like to propose we remove these folks from our core team, while thanking them for their contributions. I'll also note that I'd still value +1/-1 from these folks with a lot of significance, and encourage them to review their areas of expertise! If anyone on the list plans to start reviewing in TripleO again, then I also think we can postpone the removal for the time being and re-evaluate later. Please let me know if that's the case. Please reply and let me know any agreements or concerns with this change. Thank you! [1] https://www.stackalytics.io/report/contribution?module=tripleo-group&projec… -- -- James Slagle --

5 6

[all][tc] Relmgt team position on release cadence
by Thierry Carrez 14 Feb '22

14 Feb '22

Hi everyone, The (long) document below reflects the current position of the release management team on a popular question: should the OpenStack release cadence be changed? Please note that we only address the release management / stable branch management facet of the problem. There are other dimensions to take into account (governance, feature deprecation, supported distros...) to get a complete view of the debate. Introduction ------------ The subject of how often OpenStack should be released has been regularly debated in the OpenStack community. OpenStack started with a 3-month release cycle, then switched to 6-month release cycle starting with Diablo. It is often thought of a release management decision, but it is actually a much larger topic: a release cadence is a trade-off between pressure to release more often and pressure to release less often, coming in from a lot of different stakeholders. In OpenStack, it is ultimately a Technical Committee decision. But that decision is informed by the position of a number of stakeholders. This document gives historical context and describes the current release management team position. The current trade-off --------------------- The main pressure to release more often is to make features available to users faster. Developers get a faster feedback loop, hardware vendors ensure software is compatible with their latest products, and users get exciting new features. "Release early, release often" is a best practice in our industry -- we should generally aim at releasing as often as possible. But that is counterbalanced by pressure to release less often. From a development perspective, each release cycle comes with some process overhead. On the integrators side, a new release means packaging and validation work. On the users side, it means pressure to upgrade. To justify that cost, there needs to be enough user-visible benefit (like new features) in a given release. For the last 10 years for OpenStack, that balance has been around six months. Six months let us accumulate enough new development that it was worth upgrading to / integrating the new version, while giving enough time to actually do the work. It also aligned well with Foundation events cadence, allowing to synchronize in-person developer meetings date with start of cycles. What changed ------------ The major recent change affecting this trade-off is that the pace of new development in OpenStack slowed down. The rhythm of changes was divided by 3 between 2015 and 2021, reflecting that OpenStack is now a mature and stable solution, where accessing the latest features is no longer a major driver. That reduces some of the pressure for releasing more often. At the same time, we have more users every day, with larger and larger deployments, and keeping those clusters constantly up to date is an operational challenge. That increases the pressure to release less often. In essence, OpenStack is becoming much more like a LTS distribution than a web browser -- something users like moving slow. Over the past years, project teams also increasingly decoupled individual components from the "coordinated release". More and more components opted for an independent or intermediary-released model, where they can put out releases in the middle of a cycle, making new features available to their users. This increasingly opens up the possibility of a longer "coordinated release" which would still allow development teams to follow "release early, release often" best practices. All that recent evolution means it is (again) time to reconsider if the 6-month cadence is what serves our community best, and in particular if a longer release cadence would not suit us better. The release management team position on the debate -------------------------------------------------- While releasing less often would definitely reduce the load on the release management team, most of the team work being automated, we do not think it should be a major factor in motivating the decision. We should not adjust the cadence too often though, as there is a one-time cost in switching our processes. In terms of impact, we expect that a switch to a longer cycle will encourage more project teams to adopt a "with-intermediary" release model (rather than the traditional "with-rc" single release per cycle), which may lead to abandoning the latter, hence simplifying our processes. Longer cycles might also discourage people to commit to PTL or release liaison work. We'd probably need to manage expectations there, and encourage more frequent switches (or create alternate models). If the decision is made to switch to a longer cycle, the release management team recommends to switch to one year directly. That would avoid changing it again anytime soon, and synchronizing on a calendar year is much simpler to follow and communicate. We also recommend announcing the change well in advance. We currently have an opportunity of making the switch when we reach the end of the release naming alphabet, which would also greatly simplify the communications around the change. Finally, it is worth mentioning the impact on the stable branch work. Releasing less often would likely impact the number of stable branches that we keep on maintaining, so that we do not go too much in the past (and hit unmaintained distributions or long-gone dependencies). We currently maintain releases for 18 months before they switch to extended maintenance, which results in between 3 and 4 releases being maintained at the same time. We'd recommend switching to maintaining one-year releases for 24 months, which would result in between 2 and 3 releases being maintained at the same time. Such a change would lead to longer maintenance for our users while reducing backporting work for our developers. -- Thierry Carrez (ttx) On behalf of the OpenStack Release Management team

15 45

[nova][powervm][zvm] Time to mark powervm and zvm drivers as deprecated for removal?
by Stephen Finucane 28 Jan '22

28 Jan '22

I've had a PR open against the pypowervm library for over a fortnight now [1] with no activity. This has prompted me to go looking into the state of the powervm driver and what I've found isn't great. It doesn't seem there has been any feature development on the driver for many years, and the last change I can see that wasn't simply of a case of "I need to do this to get tests to pass" was over three years ago [2]. The CI also doesn't appear to have been touched in years. The situation for zvm isn't any better. The last functional change to that driver was nearly 2 years ago and I wasn't able to spot any CI running (though it's possible I just missed this). This means the powervm and zvm drivers stand out from the other non-libvirt drivers in-tree, each of which are getting at least some activity (albeit out- of-tree for Hyper-V, from what I recall). It also begs the question: are these drivers something we want to keep around? If we do, how are we going to maintain them long term? If not, how aggressive should we be in removing that now dead code? I'll open patches to mark both powervm and zvm as deprecated shortly so that, assuming nothing changes in the interim, we can look to remove it early in the Z cycle. Please respond here or on the reviews if you have concerns. Stephen [1] https://github.com/powervm/pypowervm/pull/17 [2] Change ID I89ad36f19672368a1f795e1f29c5af6368ccfeec

1 1

[ops][nova][victoria] Power State = Suspended?
by DHilsbos＠performair.com 24 Jan '22

24 Jan '22

All; I had something unusual happen this morning; one of my VMs was showing "Suspended" under the Power State in the Horizon dashboard. I've never seen that. What does it mean? Any search that I do points me to a bunch of resources for Status Suspended. Thank you, Dominic L. Hilsbos, MBA Vice President - Information Technology Perform Air International Inc. DHilsbos(a)PerformAir.com www.PerformAir.com

4 8

[nova][dev] Revisiting qemu emulation where guest arch != host arch
by Apsey, Christopher 05 Jan '22

05 Jan '22

All, A few years ago I asked a question[1] about why nova, when given a hw_architecture property from glance for an image, would not end up using the correct qemu-system-xx binary when starting the guest process on a compute node if that compute nodes architecture did not match the proposed guest architecture. As an example, if we had all x86 hosts, but wanted to run an emulated ppc guest, we should be able to do that given that at least one compute node had qemu-system-ppc already installed and libvirt was successfully reporting that as a supported architecture to nova. It seemed like a heavy lift at the time, so it was put on the back burner. I am now in a position to fund a contract developer to make this happen, so the question is: would this be a useful blueprint that would potentially be accepted? Most of the time when people want to run an emulated guest they would just nest it inside of an already running guest of the native architecture, but that severely limits observability and the task of managing any more than a handful of instances in this manner quickly becomes a tangled nightmare of networking, etc. I see real benefit in allowing this scenario to run natively so all of the tooling that exists for fleet management 'just works'. This would also be a significant differentiator for OpenStack as a whole. Thoughts? [1] http://lists.openstack.org/pipermail/openstack-operators/2018-August/015653… Chris Apsey Director | Georgia Cyber Range GEORGIA CYBER CENTER 100 Grace Hopper Lane | Augusta, Georgia | 30901 https://www.gacybercenter.org

2 2

[stable][requirements][zuul] unpinned setuptools dependency on stable
by Balazs Gibizer 15 Dec '21

15 Dec '21

Hi, The latest setuptools (58.0) removed support for "use_2to3" [1] (deprecated since 46.2). Many OpenStack modules defines decorator==3.4.0 as lower constraints[2]. However, decorator 3.4.0 cannot be installed anymore with the latest setuptool as it depends on "use_2to3". On master, this can be solved easily by bumping the dependency to decorator 4.0.0. On stable/xena we can still solve it the same way with a new RC. But on older stable branches such solution might be against the stable policy as it would require a major bump on our dependencies. This issue is not limited to lower-constraints testing it just hit us there first. A similar break could happen in our upper constraints as well on old stable branches. The root of the problem is that we always use the latest setuptools in our CI testing even on old stable branches. Zuul's ensure-tox task[4] installs tox which installs the virtualenv package which bundles the latest setuptools package. This happens without applying any constraints. Then tox is used to install the project's dependencies under lower or upper constraints with the unconstrained setuptools available. During and after yesterday's nova meeting [3] we discussed possible solutions. Option 1: Bump the major version of the decorator dependency on stable. Pros: * easy to implement Cons: * might against the policy / breaks downstream packagers * needs to be done in each affected project[3] separately * not future proof. After a future setuptools release we can see a similar break with another of our dependencies. @Stable Cores: how do you feel about such bump? Option 2: Pin the setuptools version during tox installation Pros: * single solution for all affected projects * future proof against breaks introduced by future setuptools releases Cons: * complex change as it probably requires to extend the base task in zuul itself Option 3: turn off lower-constraints testing Pros: * easy to implement * some of us want to get rid of lower constraints anyhow Cons: * needs per project changes * not future proof against similar break affecting our upper constraints on old stable branches. Option 4: utilize pyproject.toml[6] to specify build-time requirements Unfortunately, I'm not sure if it is possible to restrict the maximum setuptools version with it As a side note I tried applying [tox]requires in tox.ini to restrict setuptools version[7] but It does not seem to take effect in our case[8]. @Stable Cores: what do you think what could be the way forward? Cheers, gibi [1] https://setuptools.pypa.io/en/latest/history.html#v58-0-2 [2] https://codesearch.openstack.org/?q=decorator%3D%3D3.4.0&i=nope&literal=nop… [3] https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.20… [4] https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-tox/tasks… [5] https://tox.readthedocs.io/en/latest/config.html#conf-requires [6] https://www.python.org/dev/peps/pep-0518/ [7] https://review.opendev.org/c/openstack/placement/+/810293/1/tox.ini [8] https://zuul.opendev.org/t/openstack/build/f78a0300c8734e7c8475309af1d2e1a4…

11 28

[Openstack][FOSDEM][CFP] Virtualization & IaaS Devroom
by Piotr Kliczewski 15 Dec '21

15 Dec '21

We are excited to announce that the call for proposals is now open for the Virtualization & IaaS devroom at the upcoming FOSDEM 2022, to be hosted virtually on February 5th 2022. This year will mark FOSDEM’s 22nd anniversary as one of the longest-running free and open source software developer events, attracting thousands of developers and users from all over the world. Due to Covid-19, FOSDEM will be held virtually this year on February 5th & 6th, 2022. About the Devroom The Virtualization & IaaS devroom will feature session topics such as open source hypervisors and virtual machine managers such as Xen Project, KVM, bhyve, and VirtualBox, and Infrastructure-as-a-Service projects such as KubeVirt, Apache CloudStack, Foreman, OpenStack, oVirt, QEMU and OpenNebula. This devroom will host presentations that focus on topics of shared interest, such as KVM; libvirt; shared storage; virtualized networking; cloud security; clustering and high availability; interfacing with multiple hypervisors; hyperconverged deployments; and scaling across hundreds or thousands of servers. Presentations in this devroom will be aimed at users or developers working on these platforms who are looking to collaborate and improve shared infrastructure or solve common problems. We seek topics that encourage dialog between projects and continued work post-FOSDEM. Important Dates Submission deadline: 20th of December Acceptance notifications: 25th of December Final schedule announcement: 31st of December Recorded presentations upload deadline: 15th of January Devroom: 6th February 2022 Submit Your Proposal All submissions must be made via the Pentabarf event planning site[1]. If you have not used Pentabarf before, you will need to create an account. If you submitted proposals for FOSDEM in previous years, you can use your existing account. After creating the account, select Create Event to start the submission process. Make sure to select Virtualization and IaaS devroom from the Track list. Please fill out all the required fields, and provide a meaningful abstract and description of your proposed session. Submission Guidelines We expect more proposals than we can possibly accept, so it is vitally important that you submit your proposal on or before the deadline. Late submissions are unlikely to be considered. All presentation slots are 30 minutes, with 20 minutes planned for presentations, and 10 minutes for Q&A. All presentations will need to be pre-recorded and put into our system at least a couple of weeks before the event. The presentations should be uploaded by 15th of January and made available under Creative Commons licenses. In the Submission notes field, please indicate that you agree that your presentation will be licensed under the CC-By-SA-4.0 or CC-By-4.0 license and that you agree to have your presentation recorded. For example: "If my presentation is accepted for FOSDEM, I hereby agree to license all recordings, slides, and other associated materials under the Creative Commons Attribution Share-Alike 4.0 International License. Sincerely, <NAME>." In the Submission notes field, please also confirm that if your talk is accepted, you will be able to attend the virtual FOSDEM event for the Q&A. We will not consider proposals from prospective speakers who are unsure whether they will be able to attend the FOSDEM virtual event. If you are experiencing problems with Pentabarf, the proposal submission interface, or have other questions, you can email our devroom mailing list[2] and we will try to help you. Code of Conduct Following the release of the updated code of conduct for FOSDEM, we'd like to remind all speakers and attendees that all of the presentations and discussions in our devroom are held under the guidelines set in the CoC and we expect attendees, speakers, and volunteers to follow the CoC at all times. If you submit a proposal and it is accepted, you will be required to confirm that you accept the FOSDEM CoC. If you have any questions about the CoC or wish to have one of the devroom organizers review your presentation slides or any other content for CoC compliance, please email us and we will do our best to assist you. Call for Volunteers We are also looking for volunteers to help run the devroom. We need assistance with helping speakers to record the presentation as well as helping with streaming and chat moderation for the devroom. Please contact devroom mailing list [2] for more information. Questions? If you have any questions about this devroom, please send your questions to our devroom mailing list. You can also subscribe to the list to receive updates about important dates, session announcements, and to connect with other attendees. See you all at FOSDEM! [1] https://penta.fosdem.org/submission/FOSDEM22 [2] iaas-virt-devroom at lists.fosdem.org

1 1

[nova] Next spec review day on Tuesday Dec 14th
by Sylvain Bauza 13 Dec '21

13 Dec '21

Hola, As agreed during today's Nova meeting, we will have another spec review day on Dec 14th. Please sharpen your pens and write your specs in advance, and just be around during this day if you want to discuss with the reviewers. Thanks, -Sylvain

1 1

[CLOUDKITTY][VICTORIA] - policy prohibit report:get_summary to admin
by Gaël THEROND 13 Dec '21

13 Dec '21

Hi everyone! Today I faced a weird situation with one of our cloud platforms using victoria release. When trying to get a summary of projects rates would it be through Horizon or CLI using the admin user of the platform we've got the following error message: https://paste.opendev.org/show/bIgG6owrN9B2F3O7iqYG/ >From my understanding of the default policies of cloudkitty, this error seems to be a bit odd as the admin user profile actually match the default rules. At least as exposed in: https://opendev.org/openstack/cloudkitty/src/branch/stable/victoria/cloudki… and https://opendev.org/openstack/cloudkitty/src/branch/stable/victoria/cloudki… Unless I misunderstood something (please correct me if I'm wrong), it's supposed to at least be ok with the matching.

3 7

[all][tc] What's happening in Technical Committee: summary 15th Oct, 21: Reading: 5 min
by Ghanshyam Mann 10 Dec '21

10 Dec '21

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * TC this week IRC meeting held on Oct 14th Thursday. * Most of the meeting discussions are summarized below (Completed or in-progress activities section). Meeting full logs are available @ - https://meetings.opendev.org/meetings/tc/2021/tc.2021-10-14-15.00.log.html * Next week's meeting is cancelled as we are meeting in PTG. We will have the next IRC meeting on Oct 28th, Thursday 15:00 UTC, feel free the topic on agenda[1] by Oct 27th. 2. What we completed this week: ========================= * None in this week. 3. Activities In progress: ================== TC Tracker for Xena cycle ------------------------------ * TC is using the etherpad[2] for Xena cycle working item. We will be checking it in PTG * Current status is: 9 completed, 3 to-be-discussed in PTG, 1 in-progress Open Reviews ----------------- * Six open reviews for ongoing activities[3]. New project 'Skyline' proposal ------------------------------------ * You might be aware of this new dashboard proposal in the previous month's discussion. * A new project 'Skyline: an OpenStack dashboard optimized by UI and UE' is now proposed in governance to be an official OpenStack project[4]. * Skyline team is planning to meet in PTG on Tue, Wed and Thu at 5UTC, please ask your queries or have feedback/discussion with the team next week. Place to maintain the external hosted ELK, E-R, O-H services ------------------------------------------------------------------------- * We had a final discussion or I will say just a status update on this which was mentioned in last week's email summary[5]. * Now onwards, discussion and migration work will be done in TACT SIG (#openstack-infra IRC channel). Add project health check tool ----------------------------------- * No updates on this, we will continue discussing it in PTG for the next steps on this and what to do with TC liaison things. * Meanwhile, we are reviewing Rico proposal on collecting stats tools [6]. Stable Core team process change --------------------------------------- * Current proposal is under review[7]. Feel free to provide early feedback if you have any. Call for 'Technical Writing' SIG Chair/Maintainers ---------------------------------------------------------- * As agreed in last week's TC meeting, we will be moving this SIG work towards TC. * TC members have been added to the core members list in the SIG repos. * We will be discussing where to move the training repos/work in PTG. TC tags analysis ------------------- * Operator feedback is asked on open infra newsletter too, and we will continue the discussion in PTG and will take the final decision based on feedback we receive, if any[9]. Complete the policy pop up team ---------------------------------------- * Policy pop team has served its purpose and we have new RBAC as one of the the community-wide goal for the Yoga cycle. * We are marking this popup team as completed[10]. Project updates ------------------- * Retiring js-openstack-lib [11] Yoga release community-wide goal ----------------------------------------- * Please add the possible candidates in this etherpad [12]. * Current status: "Secure RBAC" is selected for Yoga cycle[13]. PTG planning ---------------- * We will be meeting in PTG next week, please check the details in this etherpad [14] * Do not forget to join the TC+community leaders sessions on Monday, Oct 18 15 UTC - 17 UTC. Test support for TLS default: ---------------------------------- * Rico has started a separate email thread over testing with tls-proxy enabled[15], we encourage projects to participate in that testing and help to enable the tls-proxy in gate testing. 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[16]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [17] 3. Office hours: The Technical Committee offers a weekly office hour every Tuesday at 0100 UTC [18] 4. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://etherpad.opendev.org/p/tc-xena-tracke [3] https://review.opendev.org/q/projects:openstack/governance+status:open [4] https://review.opendev.org/c/openstack/governance/+/814037 [5]http://lists.openstack.org/pipermail/openstack-discuss/2021-October/02525… [6] https://review.opendev.org/c/openstack/governance/+/810037 [7] https://review.opendev.org/c/openstack/governance/+/810721 [9] https://governance.openstack.org/tc/reference/tags/index.html [10] https://review.opendev.org/c/openstack/governance/+/814186 [11] https://review.opendev.org/c/openstack/governance/+/798540 [12] https://review.opendev.org/c/openstack/governance/+/807163 [13] https://etherpad.opendev.org/p/y-series-goals [14] https://etherpad.opendev.org/p/tc-yoga-ptg [15] http://lists.openstack.org/pipermail/openstack-discuss/2021-June/023000.html [16] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [17] http://eavesdrop.openstack.org/#Technical_Committee_Meeting [18] http://eavesdrop.openstack.org/#Technical_Committee_Office_hours -gmann

9 25

[neutron][CI] How to reduce number of rechecks - brainstorming
by Slawek Kaplonski 07 Dec '21

07 Dec '21

Hi, Recently I spent some time to check how many rechecks we need in Neutron to get patch merged and I compared it to some other OpenStack projects (see [1] for details). TL;DR - results aren't good for us and I think we really need to do something with that. Of course "easiest" thing to say is that we should fix issues which we are hitting in the CI to make jobs more stable. But it's not that easy. We are struggling with those jobs for very long time. We have CI related meeting every week and we are fixing what we can there. Unfortunately there is still bunch of issues which we can't fix so far because they are intermittent and hard to reproduce locally or in some cases the issues aren't realy related to the Neutron or there are new bugs which we need to investigate and fix :) So this is never ending battle for us. The problem is that we have to test various backends, drivers, etc. so as a result we have many jobs running on each patch - excluding UT, pep8 and docs jobs we have around 19 jobs in check and 14 jobs in gate queue. In the past we made a lot of improvements, like e.g. we improved irrelevant files lists for jobs to run less jobs on some of the patches, together with QA team we did "integrated-networking" template to run only Neutron and Nova related scenario tests in the Neutron queues, we removed and consolidated some of the jobs (there is still one patch in progress for that but it should just remove around 2 jobs from the check queue). All of that are good improvements but still not enough to make our CI really stable :/ Because of all of that, I would like to ask community about any other ideas how we can improve that. If You have any ideas, please send it in this email thread or reach out to me directly on irc. We want to discuss about them in the next video CI meeting which will be on November 30th. If You would have any idea and would like to join that discussion, You are more than welcome in that meeting of course :) [1] http://lists.openstack.org/pipermail/openstack-discuss/2021-November/ 025759.html -- Slawek Kaplonski Principal Software Engineer Red Hat

11 19

[ops]RabbitMQ High Availability
by DHilsbos＠performair.com 06 Dec '21

06 Dec '21

All; In the time I've been part of this mailing list, the subject of RabbitMQ high availability has come up several times, and each time specific recommendations for both Rabbit and Open Stack are provided. I remember it being an A or B kind of recommendation (i.e. configure Rabbit like A1, and Open Stack like A2, OR configure Rabbit like B1, and Open Stack like B2). Unfortunately, I can't find the previous threads on this topic. Does anyone have this information, that they would care to share with me? Thank you, Dominic L. Hilsbos, MBA Vice President - Information Technology Perform Air International Inc. DHilsbos(a)PerformAir.com www.PerformAir.com

3 3

[kolla] Plan to deprecate binary and unify on single distrubition
by Michał Nasiadka 06 Dec '21

06 Dec '21

Hello Koalas, On the PTG we have discussed two topics: 1) Deprecate and drop binary type of Kolla images 2) Use a common base (single Linux distribution) for Kolla images This is a call for feedback - for people that have not been attending the PTG. What this essentially mean for consumers: 1) In Yoga cycle we will deprecate binary type of Kolla images, and in Z cycle those will be dropped. 2) We are not going to support CentOS Stream 9 (cs9) as a base operating system, and the source type build will rely on CentOS Stream 8 in Z release. 3) Beginning from A release Kolla will build only Debian source images - but Kolla-Ansible will still support deployment of those images on CentOS/Ubuntu/Debian Host operating systems (and Rocky Linux to be added in Yoga to that mix). Justification: The Kolla project team is limited in numbers, therefore supporting current broad mix of operating systems (especially with CentOS Stream 9 ,,on the way’’) is a significant maintenance burden. By dropping binary type of images - users would be running more tested images (since Kolla/Kolla-Ansible CI runs source images jobs as voting). In Xena we’ve already changed the default image type Kolla-Ansible uses to source. We also feel that using a unified base OS for Kolla container images is a way to remove some of the maintenance burden (including CI cycles and Request for feedback: If any of those changes is a no go from your perspective - we’d like to hear your opinions. Best regards, Michal Nasiadka

9 17

[TripleO] IPV6 Support
by Lokendra Rathour 03 Dec '21

03 Dec '21

Hi, I am trying to install the undercloud using the ipv6 address on Centos 8 using Train release. It is seen that the deployment of undercloud is getting failed with error as mentioned below. Same deployment is working in the normal ipv4 case. So Questions: 1. is IPV6 supported in TripleO , if yes then please suggest how. Error: nied", " attempt(s): 2", "2021-11-25 07:25:31,535 WARNING: 65355 -- Retrying running container: zaqar", "2021-11-25 07:25:34,887 ERROR: 65355 -- ['/usr/bin/podman', 'run', '--user', '0', '--name', 'container-puppet-zaqar', '--env', 'PUPPET_TAGS=file,file_line,concat,augeas,cron,zaqar_config', '--env', 'NAME=zaqar', '--env', 'HOSTNAME=undercloud', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--env', 'DEBUG=False', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmp06nvxzzz:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--rm', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/container-puppet-zaqar.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', 'undercloud.ctlplane.localdomain:8787/tripleotraincentos8/centos-binary-zaqar-wsgi:current-tripleo'] run failed after Error: container_linux.go:370: starting container process caused: error adding seccomp filter rule for syscall bdflush: permission denied: OCI permission denied", " attempt(s): 3", "2021-11-25 07:25:34,888 WARNING: 65355 -- Retrying running container: zaqar", "2021-11-25 07:25:34,888 ERROR: 65355 -- Failed running container for zaqar", "2021-11-25 07:25:34,888 INFO: 65355 -- Finished processing puppet configs for zaqar", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring crond", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring glance_api", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring heat_api", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring heat", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring ironic_api", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring ironic", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring ironic_inspector", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring neutron", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring iscsid", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring keystone", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring memcached", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring mistral", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring mysql", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring nova", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring rabbitmq", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring placement", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring swift", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring swift_ringbuilder", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring zaqar" ], "stderr_lines": [], "_ansible_no_log": false, "attempts": 15 } ] ] Not cleaning working directory /home/stack/tripleo-heat-installer-templates Not cleaning ansible directory /home/stack/undercloud-ansible-mw4crw92 Install artifact is located at /home/stack/undercloud-install-20211125072536.tar.bzip2 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Deployment Failed! ERROR: Heat log files: /var/log/heat-launcher/undercloud_deploy-o_qf1b4w !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Deployment failed. Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1345, in _standalone_deploy raise exceptions.DeploymentError('Deployment failed') tripleoclient.exceptions.DeploymentError: Deployment failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1363, in _standalone_deploy raise exceptions.DeploymentError(six.text_type(e)) tripleoclient.exceptions.DeploymentError: Deployment failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/cliff/app.py", line 401, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1451, in take_action if self._standalone_deploy(parsed_args) != 0: File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1400, in _standalone_deploy raise exceptions.DeploymentError('Deployment failed.') tripleoclient.exceptions.DeploymentError: Deployment failed. clean_up Deploy: Deployment failed. Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1345, in _standalone_deploy raise exceptions.DeploymentError('Deployment failed') tripleoclient.exceptions.DeploymentError: Deployment failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1363, in _standalone_deploy raise exceptions.DeploymentError(six.text_type(e)) tripleoclient.exceptions.DeploymentError: Deployment failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/osc_lib/shell.py", line 136, in run ret_val = super(OpenStackShell, self).run(argv) File "/usr/lib/python3.6/site-packages/cliff/app.py", line 281, in run result = self.run_subcommand(remainder) File "/usr/lib/python3.6/site-packages/osc_lib/shell.py", line 176, in run_subcommand ret_value = super(OpenStackShell, self).run_subcommand(argv) File "/usr/lib/python3.6/site-packages/cliff/app.py", line 401, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1451, in take_action if self._standalone_deploy(parsed_args) != 0: File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1400, in _standalone_deploy raise exceptions.DeploymentError('Deployment failed.') tripleoclient.exceptions.DeploymentError: Deployment failed. END return value: 1 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! An error has occured while deploying the Undercloud. See the previous output for details about what went wrong. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Command '['sudo', '--preserve-env', 'openstack', 'tripleo', 'deploy', '--standalone', '--standalone-role', 'Undercloud', '--stack', 'undercloud', '--local-domain=localdomain', '--local-ip=abce:abce:abce::1/64', '--templates=/usr/share/openstack-tripleo-heat-templates/', '--networks-file=network_data_undercloud.yaml', '--heat-native', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/podman.yaml', '-e', '/home/stack/containers-prepare-parameter.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/zaqar-swift-backend.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/ssl/no-tls-endpoints-public-ip.yaml', '--deployment-user', 'stack', '--output-dir=/home/stack', '-e', '/home/stack/tripleo-config-generated-env-files/undercloud_parameters.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/tripleo-validations.yaml', '--debug', '--log-file=install-undercloud.log', '-e', '/usr/share/openstack-tripleo-heat-templates/undercloud-stack-vstate-dropin.yaml']' returned non-zero exit status 1. Command '['sudo', '--preserve-env', 'openstack', 'tripleo', 'deploy', '--standalone', '--standalone-role', 'Undercloud', '--stack', 'undercloud', '--local-domain=localdomain', '--local-ip=abce:abce:abce::1/64', '--templates=/usr/share/openstack-tripleo-heat-templates/', '--networks-file=network_data_undercloud.yaml', '--heat-native', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/podman.yaml', '-e', '/home/stack/containers-prepare-parameter.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/zaqar-swift-backend.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/ssl/no-tls-endpoints-public-ip.yaml', '--deployment-user', 'stack', '--output-dir=/home/stack', '-e', '/home/stack/tripleo-config-generated-env-files/undercloud_parameters.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/tripleo-validations.yaml', '--debug', '--log-file=install-undercloud.log', '-e', '/usr/share/openstack-tripleo-heat-templates/undercloud-stack-vstate-dropin.yaml']' returned non-zero exit status 1. END return value: 1 [stack@undercloud ~]$

2 5

[ironic][meeting] Postpone meeting time until end of March 2022
by Riccardo Pittau 03 Dec '21

03 Dec '21

Hello ironicers! Due to some overlapping time with other meetings that cause me to attend to 2 (or sometimes 3.....) meetings at the same time, I'd like to propose to move the time for the weekly ironic upstream meeting 1 hour later, so from the current 15:00-16:00 UTC to 16:00-17:00 UTC, until the next Daylight saving time change that should happen at the end of March 2022. Thank you! Ciao, Riccardo

4 5

python_requires >= 3.8 during Yoga
by Stephen Finucane 02 Dec '21

02 Dec '21

gmann has been helpfully proposing patches to change the versions of Python we're testing against in Yoga. I've suggested that we might want to bump 'python_requires' in 'setup.cfg' to indicate that we no longer support any version of Python before 3.8 [1]. As gmann has noted, doing so would mean nova would no longer be installable on Python 3.6 or 3.7 and there has been a small bit of back and forth on the pros and cons of this. I'm wondering what other people's thoughts on this are. Is this something we should be doing? Should we do it for libraries too or just services? Do we ever want to do this? Thoughts, please! Stephen [1] https://review.opendev.org/c/openstack/nova/+/819194/comment/72ecf24f_2bd29…

17 61

Windows imaging process
by Gaël THEROND 02 Dec '21

02 Dec '21

Hi everyone, On one of our Openstack platforms, we maintain windows based workloads. We currently have the following build process: 1°/- We download windows ISO image. 2°/- Use hyper-v to create a gold image that bring virtio drivers. 3°/- We upload this image on glance and publish it for our users. 4°/- Our users use packer to create their own windows custom image from our gold image. This workflow works pretty fine, it’s simple enough and as Microsoft isn’t releasing a new major Windows every two months it’s pretty ok not having all the steps automated until now. However, I’m wondering if there is a way to automate my first two steps using Openstack? So far, from my early tests, I didn’t managed to get all of appropriate gears to work together, did I missed something? I need to create a VM that at least can use the Windows ISO plus a virtio ISO as a second cdrom device and an additional user-data payload that will instruct the Windows installer to go automatically through the installation steps plus loads missing drivers from the virtio iso. This is perfectly working on a native kvm as you can add multiple cdrom device but I didn’t find a way to replicate that on Openstack. Starting from an unaltered Microsoft originated ISO image is a mandatory requirement for this project (Because of security constraints that I can’t have any impact on). Help from anyone that already had to deal with such situation would be very appreciated ! Thanks!

2 3

[kuryr] Propose to EOL stable branches (Stein, Rocky and Queens) for all the kuryr scope
by Maysa De Macedo Souza 02 Dec '21

02 Dec '21

Hello, We have plenty of stable branches in Kuryr repositories still open with the oldest being stable/queens. We haven't seen many backports[0] proposed to Stein, Rocky and Queens, so I would like to propose retiring these branches. Let me know if anybody is interested in keeping any of those branches. [0] https://review.opendev.org/q/(project:%255Eopenstack/kuryr.*)AND((branch:st… Thank you, Maysa Macedo.

2 3

[all][tc] Technical Committee next weekly meeting on Dec 2nd at 1500 UTC
by Ghanshyam Mann 01 Dec '21

01 Dec '21

Hello Everyone, Technical Committee's next weekly meeting is scheduled for Dec 2nd at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, Dec 1st, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[cinder] : SAN migration
by Gauurav Sabharwal1 01 Dec '21

01 Dec '21

Hi Experts , I need some expert advise of one of the scenario, I have multiple isolated OpenStack cluster running with train & rocky edition. Each OpenStack cluster environment have it's own isolated infrastructure of SAN ( CISCO fabric ) & Storage ( HP, EMC & IBM). Now company planning to refresh their SAN infrastructure. By procuring new Brocade SAN switches. But there are some migration relevant challenges we have. As we understand under one cinder instance only one typer of FC zone manager is supported . Currently customer configured & managing CISCO . Is it possible to configure two different vendor FC Zone manager under one cinder instance. Migration of SAN zoning is supposedly going to be happen offline way from OpenStack point of view. We will be migrating all ports of each existing cisco fabric to Brocade with zone configuration using brocade CLI. Our main concern is that after migration How CINDER DB update new zone info & path via Brocade SAN. Regards Gauurav Sabharwal IBM India Pvt. Ltd. IBM towers Ground floor, Block -A , Plot number 26, Sector 62, Noida Gautam budhnagar UP-201307. Email:gauurav.sabharwal@in.ibm.com Mobile No.: +91-9910159277

3 3

[neutron][networking][ipv6][dns][ddi] Upcoming OpenInfra Edge Computing Group sessions
by Ildiko Vancsa 30 Nov '21

30 Nov '21

Hi, I’m reaching out to you to draw your attention to the amazing lineup of discussion topics for the OpenInfra Edge Computing Group weekly calls up until the end of this year with industry experts to present and participate in the discussions! I would like to invite and encourage you to join the working group sessions to discuss edge related challenges and solutions in the below areas and more! Some of the sessions to highlight will be continuing the discussions we started at the recent PTG: * November 29th - Networking and DNS discussion with Cricket Liu and Andrew Wertkin * December 6th - Networking and IPv6 discussion with Ed Horley Beyond the above topics we will have presentations from groups and projects such as Smart Edge and the Industry IoT Consortium (Formerly Industrial Internet Consortium). For the full schedule please see the edge WG’s wiki: https://wiki.openstack.org/wiki/Edge_Computing_Group#Upcoming_Topics Our calls happen every Monday at 6am Pacific Time / 1400 UTC on Zoom. You can find all the meeting details here: https://wiki.openstack.org/wiki/Edge_Computing_Group#Meetings Please let me know if you have any questions about the working group or any of the upcoming sessions. Thanks and Best Regards, Ildikó

1 3

[Neutron][OVN] nova-compute timeout while waiting for VIF activation confirmed by Neutron
by Tony Liu 30 Nov '21

30 Nov '21

Hi, I see such problem from time to time. It's not consistently reproduceable. ====================== 2021-11-23 22:16:28.532 7 INFO nova.compute.manager [req-50817b01-e7ae-4991-94fe-e4c5672c481b - - - - -] [instance: 7021bf37-348d-4dfa-b0c5-6a251fceb6cd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-11-23 22:21:28.342 7 WARNING nova.virt.libvirt.driver [req-814c98a4-3fd9-4607-9abb-5fbe5cef8650 3a4c320d64664d9cb6784b7ea52d618a 21384fe21ebb4339b1a5f2a8f9000ea3 - default default] [instance: 7021bf37-348d-4dfa-b0c5-6a251fceb6cd] Timeout waiting for [('network-vif-plugged', '735d97a3-db7c-42ed-b3be-1596d4cc7f4b')] for instance with vm_state building and task_state spawning.: eventlet.timeout.Timeout: 300 seconds ====================== The VIF/port is activated by OVN ovn-controller to ovn-sb. It seems that, either Neutron didn't capture the update or didn't send message back to nova-compute. Is there any known fix for this problem? Thanks! Tony

5 9

[neutron][nova-compute] Received unexpected event network-vif-plugged-<uuid>
by Tony Liu 30 Nov '21

30 Nov '21

Hi, With Ussuri, when launch VM, I see this from nova-compute. ======== Received unexpected event network-vif-plugged-32b7af8a-ef81-4785-bd79-3a5619823638 ======== That caused VM to be destroyed, rescheduled and relaunched. And that recreation worked fine. >From user POV, it just takes a bit longer to launch the VM. What's the event nova-compute expects from Neutron for vif-plug? Any hints where I can look into it? More logs here. ======== 2021-11-29 18:28:10.064 7 INFO os_vif [req-2a89f605-6972-41dc-83f1-1ef6a444e486 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f1:94:a0,bridge_name='br-int',has_traffic_filtering=True,id=32b7af8a-ef81-4785-bd79-3a5619823638,network=Network(27ea27c6-3d1e-4d37-ad4a-de2ae1578f1e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap32b7af8a-ef') 2021-11-29 18:28:10.515 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384-1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] VM Started (Lifecycle Event) 2021-11-29 18:28:10.544 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384 -1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] VM Paused (Lifecycle Event) 2021-11-29 18:28:10.604 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384 -1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] During sync_power_state the instance has a pending task (spawning). Skip. 2021-11-29 18:28:25.441 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384 -1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] VM Res umed (Lifecycle Event) 2021-11-29 18:28:25.445 7 INFO nova.virt.libvirt.driver [-] [instance: a8eb2a2a- 8c83-46b2-bfd4-fd3a7d00f2ce] Instance spawned successfully. 2021-11-29 18:28:25.445 7 INFO nova.compute.manager [req-2a89f605-6972-41dc-83f1-1ef6a444e486 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] Took 18.75 seconds to spawn the instance on the hypervisor. 2021-11-29 18:28:25.502 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384-1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] During sync_power_state the instance has a pending task (spawning). Skip. 2021-11-29 18:28:25.514 7 INFO nova.compute.manager [req-2a89f605-6972-41dc-83f1-1ef6a444e486 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] Took 19.46 seconds to build instance. 2021-11-29 18:28:27.466 7 WARNING nova.compute.manager [req-82314965-fe71-4993-bbf6-f0aa9e708529 f0c1375066214e24ab7942d72d829097 3ec2940fc10d4e57bcf929d1fe678c79 - default default] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] Received unexpected event network-vif-plugged-32b7af8a-ef81-4785-bd79-3a5619823638 for instance with vm_state active and task_state None. 2021-11-29 18:28:55.829 7 INFO nova.compute.manager [req-e36e4a81-4755-4d7f-9ee4-d43d00629765 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] Terminating instance ======== Thanks! Tony

2 1

[neutron] CI meeting 30.11.2021 - cancelled
by Slawek Kaplonski 29 Nov '21

29 Nov '21

Hi, Due to some personal reasons I will not be able to drive today's Neutron CI meeting. Let's cancel it this week. Remember, next week we will have CI meeting on Jitsi and we will discuss all ideas about how to improve stability of our CI. -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[Neutron][OVN] AttributeError: 'NoneType' object has no attribute 'get_lrouter'
by Tony Liu 29 Nov '21

29 Nov '21

Hi, It happens from time to time, not consistently, with the latest kolla/centos-binary-neutron-server:ussuri container image. >From the bottome of traceback, it seems self._ovn is None. Any clues how this can happen? I don't see other suspicious logging in Neutron and OVN. ===================== File "/usr/lib/python3.6/site-packages/neutron/services/ovn_l3/plugin.py", line 477, in get_router_availability_zones ERROR neutron.api.v2.resource lr = self._ovn.get_lrouter(router['id']) ERROR neutron.api.v2.resource AttributeError: 'NoneType' object has no attribute 'get_lrouter' ===================== Here is the full traceback. ===================== neutron.api.v2.resource [req-bf2230aa-0491-4caa-b5f7-169d37009672 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] index failed: No details.: AttributeError: 'NoneType' object has no attribute 'get_lrouter' neutron.api.v2.resource Traceback (most recent call last): neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/api/v2/resource.py", line 98, in resource neutron.api.v2.resource result = method(request=request, **args) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped neutron.api.v2.resource return f(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper neutron.api.v2.resource ectxt.value = e.inner_exc neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper neutron.api.v2.resource return f(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped neutron.api.v2.resource LOG.debug("Retry wrapper got retriable exception: %s", e) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped neutron.api.v2.resource return f(*dup_args, **dup_kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 369, in index neutron.api.v2.resource return self._items(request, True, parent_id) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 304, in _items neutron.api.v2.resource obj_list = obj_getter(request.context, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped neutron.api.v2.resource return method(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped neutron.api.v2.resource return f(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper neutron.api.v2.resource ectxt.value = e.inner_exc neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper neutron.api.v2.resource return f(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped neutron.api.v2.resource LOG.debug("Retry wrapper got retriable exception: %s", e) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped neutron.api.v2.resource return f(*dup_args, **dup_kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/db/l3_db.py", line 547, in get_routers neutron.api.v2.resource page_reverse=page_reverse) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/model_query.py", line 317, in get_collection neutron.api.v2.resource for c in query neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/model_query.py", line 317, in <listcomp> neutron.api.v2.resource for c in query neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/db/l3_db.py", line 221, in _make_router_dict neutron.api.v2.resource resource_extend.apply_funcs(l3_apidef.ROUTERS, res, router) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/resource_extend.py", line 84, in apply_funcs neutron.api.v2.resource resolved_func(response, db_object) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/db/availability_zone/router.py", line 40, in _add_az_to_response neutron.api.v2.resource l3_plugin.get_router_availability_zones(router_db)) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/services/ovn_l3/plugin.py", line 477, in get_router_availability_zones neutron.api.v2.resource lr = self._ovn.get_lrouter(router['id']) neutron.api.v2.resource AttributeError: 'NoneType' object has no attribute 'get_lrouter' neutron.api.v2.resource neutron.wsgi [req-bf2230aa-0491-4caa-b5f7-169d37009672 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] 10.91.1.7,10.250.20.26 "GET /v2.0/routers HTTP/1.1" status: 500 len: 344 time: 0.2660162 ===================== Thanks! Tony

1 0

RE: Nova Local Disk Storage (Seena Fallah)
by Steven Relf 29 Nov '21

29 Nov '21

Hi Seena, Do you mean passing an entire disk to an instance, or moving where nova stores its ephemeral disks? (e.g. /var/lib/nova/instances?) Rgds Steve. The future has already arrived. It's just not evenly distributed yet - William Gibson Hi, Is there any support for creating instances on a whole separate disk, not an LVM?

1 0

[glance][image-import] local disk space required by image import and conversion
by Tony Liu 29 Nov '21

29 Nov '21

Hi, The target is for user to import .qcow2 image from download site and automatically stored as raw image. Read through [1], this seems doable. I am not sure the workflow. Does it download the whole qcow2 and convert it to raw locally, or download, convert and upload in a streaming way, which only require small space for caching? The question is that, how much local disk space is required for such import and conversion? [1] https://docs.openstack.org/glance/latest/admin/interoperable-image-import.h… Thanks! Tony

2 2

[tripleo] puppet_config: immutable relationship between config_volume and config_image?
by Brent Eagles 29 Nov '21

29 Nov '21

Hi all, I've been working on https://review.opendev.org/c/openstack/tripleo-heat-templates/+/816895 and the current pep8 check error on the designate-api-container-puppet.yaml template appears to disallow using different container config images on the same config volume. I see where this makes sense but because the way the puppet_config sections seem to be structured I wonder if it was intentional. The templates happen to work in this patch's case because the api specific config image in this template only includes some additional binaries for apache+wsgi necessary for the puppet. However, I can see where this might cause some really unfortunate issues if the config images for a given config_volume were to have different versions of puppet. While I will alter the templates so the designate config volume use the same config image, it does beg the question if having the puppet_config config_volume definitions duplicated across the templates is a good pattern. Would it be a better choice to adopt a pattern where the immutable parts of the config_volumes (e.g. config_volume name, config_image) can only be defined once per config volume? Perhaps in a separate file and have the mutable parts in their respective template definitions? Cheer, Brent -- Brent Eagles Principal Software Engineer Red Hat Inc.

2 2

[ironic] End of year meeting schedule
by Julia Kreger 29 Nov '21

29 Nov '21

Greetings everyone, Today, during the Ironic team meeting, we reached consensus[0] that we would not hold meetings during the last half of December and for the first week of January. Much as we've done the past few years to give contributors time they can feel safe taking off without missing any meetings, coupled with end of year PTO usage and holidays. We will resume our regular meetings on January 10th, 2022. Dates of cancelled meetings: * December 20th, 2021 * December 27th, 2021 * January 3rd, 2022 Hopefully everyone will get plenty of rest, and be ready to review plenty of code when they get back! Thanks, -Julia [0]: https://meetings.opendev.org/meetings/ironic/2021/ironic.2021-11-29-15.00.l…

1 0

[neutron][nova] [kolla] vif plugged timeout
by Michal Arbet 29 Nov '21

29 Nov '21

Hi, Has anyone seen issue which I am currently facing ? When launching heat stack ( but it's same if I launch several of instances ) vif plugged in timeouts an I don't know why, sometimes it is OK ..sometimes is failing. Sometimes neutron reports vif plugged in < 10 sec ( test env ) sometimes it's 100 and more seconds, it seems there is some race condition but I can't find out where the problem is. But on the end every instance is spawned ok (retry mechanism worked). Another finding is that it has to do something with security group, if noop driver is used ..everything is working good. Firewall security setup is openvswitch . Test env is wallaby. I will attach some logs when I will be near PC .. Thank you, Michal Arbet (Kevko)

8 12

[wallaby][ovn][Open vSwitch] HA for OVN DB servers using pacemaker
by Faisal Sheikh 29 Nov '21

29 Nov '21

Hi, I am using Openstack Wallaby release with OVN on Ubuntu 20.04. My environment consists of 2 compute nodes and 1 controller node. ovs-vswitchd (Open vSwitch) 2.15.0 Ubuntu Kernel Version: 5.4.0-88-generic compute node1 172.16.30.1 compute node2 172.16.30.3 controller/Network node/Primary ovs-db IP 172.16.30.46 backup ovs-db 172.16.30.47 I added another node for backup ovs-db server (172.16.30.47 ) and installed openvswitch-ovn and networking-ovn packages on it for backup ovs-db server and using Pacemaker to manage the ovn-northd(Open vSwitch) service between primary and backup ovs-db server active/passive HA mode. Pacemaker cluster is successful and after creating a pacemaker cluster, I am using the following commands to create one active and one backup server for OVN databases. $ pcs resource create ovndb_servers ocf:ovn:ovndb-servers \ master_ip=172.16.30.46 \ ovn_ctl=<path of the ovn-ctl script> \ op monitor interval="10s" \ op monitor role=Master interval="15s" But i am getting below error: Error: Agent 'ocf:ovn:ovndb-servers' is not installed or does not provide valid metadata: Metadata query for ocf:ovn:ovndb-servers failed: Input/output error, use --force to override I would really appreciate any input in this regard. Best regards, Faisal Sheikh

1 0

Regarding Drop down of frequently used stacks in OpenStack Wallaby Version 23.1.2
by Adivya Singh 29 Nov '21

29 Nov '21

Dear Team, I want a Drop down of frequently used stack in my orchestration --> Launch Stack parametre in openstack version given above, Is there any Solution for this, How should i go ahead with this Regards Adivya Singh

2 2

[wallaby][ovn][Open vSwitch] HA for OVN DB servers using pacemaker
by Faisal Sheikh 29 Nov '21

29 Nov '21

Hi, I am using Openstack Wallaby release with OVN on Ubuntu 20.04. My environment consists of 2 compute nodes and 1 controller node. ovs-vswitchd (Open vSwitch) 2.15.0 Ubuntu Kernel Version: 5.4.0-88-generic compute node1 172.16.30.1 compute node2 172.16.30.3 controller/Network node/Primary ovs-db IP 172.16.30.46 backup ovs-db 172.16.30.47 I added another node for backup ovs-db server (172.16.30.47 ) and installed openvswitch-ovn and networking-ovn packages on it for backup ovs-db server and using Pacemaker to manage the ovn-northd(Open vSwitch) service between primary and backup ovs-db server active/passive HA mode. Pacemaker cluster is successful and after creating a pacemaker cluster, I am using the following commands to create one active and one backup server for OVN databases. $ pcs resource create ovndb_servers ocf:ovn:ovndb-servers \ master_ip=172.16.30.46 \ ovn_ctl=<path of the ovn-ctl script> \ op monitor interval="10s" \ op monitor role=Master interval="15s" But i am getting below error: Error: Agent 'ocf:ovn:ovndb-servers' is not installed or does not provide valid metadata: Metadata query for ocf:ovn:ovndb-servers failed: Input/output error, use --force to override I would really appreciate any input in this regard. Best regards, Faisal Sheikh

1 0

[neutron]Some floating IPs inaccessible after restart of L3 agent
by Kamil Madáč 29 Nov '21

29 Nov '21

Hello Everyone, We have openstack Victoria deployed since the beginning of the year with kolla/ansible in docker containers. Everything was running OK, but few weeks ago we noticed issues with networking. Our installation uses Openvswitch networking with DVR non HA routers. Everything is running smoothly until we restart L3 agent. After that, some floating ips of VMs running on the node where L3 agent is running becomes inaccessible. Workaround is to reassign floating IP to affected VM. Every restart affects same floating IPs and VMs. No errors/excpetions found in logs. I was able to find out that after restart there are missing routes for those particular floating IPs in fip- namespace, which causes that proxy arp responses are not working. After floating IP address is reassigned, routes are added by L3 agent and floating IP is working again. Looks like some sort of race condition in L3 agent, but I was not able to identify any possible existing bug. L3 agent is in version 17.0.1.dev44. Is anyone aware of any existing bug which could explain such behavior, or does anyone have idea how to solve the issue? Kamil Madáč Slovensko IT a.s.

2 3

[Image Encryption] No meeting today
by Josephine Seifert 29 Nov '21

29 Nov '21

Hi, due to a conflicting meeting today I will not be able to hold the popup team meeting. Next meeting will be next week. greetings Josphine (Luzi)

1 0

[neutron] Bug deputy, 22Nov - 28Nov
by Rodolfo Alonso Hernandez 29 Nov '21

29 Nov '21

Hello Neutrinos: This is the bug report from last week. Critical: https://bugs.launchpad.net/neutron/+bug/1952004: “ovn: Functional test may fail if northd hasn't initialized DBs yet”. Assigned. Patch: https://review.opendev.org/c/openstack/neutron/+/819032 https://bugs.launchpad.net/neutron/+bug/1952023: “Neutron functional tests don't properly clean up ovn-northd”. Assigned. Patch: https://review.opendev.org/c/openstack/neutron/+/819049 <https://bugs.launchpad.net/neutron/+bug/1952066> https://bugs.launchpad.net/neutron/+bug/1952066: “Scenario test test_mac_learning_vms_on_same_network fails intermittently in the ovn job”. Assigned. https://bugs.launchpad.net/neutron/+bug/1952357: “Functional tests job in the ovn-octavia-provider is broken“. Assigned. Patch: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/819377 <https://bugs.launchpad.net/neutron/+bug/1952393> https://bugs.launchpad.net/neutron/+bug/1952393: “[OVN] neutron-tempest-plugin-scenario-ovn broken with "ovn-northd did not start"“. Assigned. Patch: https://review.opendev.org/c/openstack/devstack/+/819402 <https://bugs.launchpad.net/neutron/+bug/1952508> https://bugs.launchpad.net/neutron/+bug/1952508: “[OVN] "TestAgentMonitor.test_agent_change_controller" failing randomly“. Assigned. Patch: https://review.opendev.org/c/openstack/neutron/+/819502 High: https://bugs.launchpad.net/neutron/+bug/1952567: “[ml2][ovs] ports tag are missing and flood on those”. Assigned. Patch: https://review.opendev.org/c/openstack/neutron/+/819567 <https://bugs.launchpad.net/neutron/+bug/1952550> https://bugs.launchpad.net/neutron/+bug/1952550: “[OVN] neutron_ovn_metadata_agent retrying on UUID errors infinitely”. *Unassigned* . Medium: https://bugs.launchpad.net/neutron/+bug/1951816: “[OVN] setting a IPv6 address in dns_servers is broken”. *Unassigned*. https://bugs.launchpad.net/neutron/+bug/1951872: “OVN: Missing reverse DNS for instances”. *Unassigned*. https://bugs.launchpad.net/neutron/+bug/1952055: “native firewall driver - conntrack marks too much traffic as invalid“. *Unassigned*. Low: https://bugs.launchpad.net/neutron/+bug/1951569: “[L3] L3 agent extension should always inherit from "L3AgentExtension"”. Assigned. Patch: https://review.opendev.org/c/openstack/neutron/+/818540 <https://bugs.launchpad.net/neutron/+bug/1952409> https://bugs.launchpad.net/neutron/+bug/1952409: “SQLAlchemy "ConnectionEvents.before_execute" signature has changed in 1.4“. Assigned. Patch: https://review.opendev.org/c/openstack/neutron/+/819420 Undecided/Opinion/Invalid: https://bugs.launchpad.net/neutron/+bug/1951720: "Virtual interface creation failed". More information is needed to debug this issue. https://bugs.launchpad.net/neutron/+bug/1952249: "[OVN] Created VM name overrides predefined port dns-name". The extension works as expected. Regards.

1 0

[all][tc] What's happening in Technical Committee: summary 26th Nov, 21: Reading: 10 min
by Ghanshyam Mann 26 Nov '21

26 Nov '21

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * Most of the meeting discussions are summarized below (Completed or in-progress activities section). Meeting full logs are available @ https://meetings.opendev.org/meetings/tc/2021/tc.2021-11-25-15.00.log.html * Next week's meeting is video call on 2nd Dec, Thursday 15:00 UTC, feel free the topic on the agenda[1] by 1st Dec. 2. What we completed this week: ========================= * Add ProxySQL repository for OpenStack-Ansible[2] * Mark 'Technical Writing' SIG 'complete' as it is merged into TC [3] * Rename ‘Extended Maintenance’ SIG to the ‘Stable Maintenance’ [4] * Retire training-labs repo [5] 3. Activities In progress: ================== TC Tracker for Yoga cycle ------------------------------ * This etherpad includes the Yoga cycle targets/working items for TC[6]. Open Reviews ----------------- * 8 open reviews for ongoing activities[7]. Yoga testing runtime ------------------------- We updated the Yoga testing runtime to move from CentOS stream 8 to CentOS stream 9 and so drop the py3.6[8]. But while updating projects to drop py3.6, and discussion on whether we should change 'python_requires'>= 3.8 in setup.cfg, another (but good) discussion started on ML about whether we should continue testing py3.6 and support CentOS stream 8 and 9 both in yoga which is what RDO community plan is[8]. After discussing on ML and then with Alfredo Moralejo on TC IRC channel, we have a potential way forward on this which I have summarized on ML thread[9]. I will keep that open to get feedback or any objection to the proposed plan and accordingly we will conclude it early next week. I hope I have summarized the discussion here if not please read the ML thread for this. Fixing Zuul config error -------------------------- If your project is listed in zuul config error, please start planing to fix those[10] RBAC discussion: continuing from PTG ---------------------------------------------- After last week meeting discussion, we are almost ready with this goal, if you would like to review the current direction as well as the schedule, please review it[11]. Next plan is to merge the proposed goal and then select it ASAP as m-1 is already passed for Yoga cycle. Community-wide goal updates ------------------------------------ * Two goals are proposed currently and under review. ** RBAC goal is pretty much in good shape now, feel free to review it[11] ** 'FIPS compatibility and compliance' [12]. Adjutant need maintainers and PTLs ------------------------------------------- We are waiting to hear from Braden, Albert on permission to work in this project[13]. New project 'Skyline' proposal ------------------------------------ * TC is still waiting for skyline team to respond on open queries[14]. TC tags analysis ------------------- * TC agreed to remove the framework and it is communicated in ML[15]. Project updates ------------------- * Add ansible-collection-kolla repo to Kolla project[16] * Retire js-openstack-lib [17] 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[18]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [19] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/817245 [3] https://review.opendev.org/c/openstack/governance-sigs/+/815868 [4] https://review.opendev.org/c/openstack/governance-sigs/+/817499 [5] https://review.opendev.org/c/openstack/governance/+/817511 [6] https://etherpad.opendev.org/p/tc-yoga-tracker [7] https://review.opendev.org/q/projects:openstack/governance+status:open [8] http://lists.openstack.org/pipermail/openstack-discuss/2021-November/026001… [9] http://lists.openstack.org/pipermail/openstack-discuss/2021-November/026024… [10] https://etherpad.opendev.org/p/zuul-config-error-openstack [11] https://review.opendev.org/c/openstack/governance/+/815158 [12] https://review.opendev.org/c/openstack/governance/+/816587 [13] http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025786… [14] https://review.opendev.org/c/openstack/governance/+/814037 [15] http://lists.openstack.org/pipermail/openstack-discuss/2021-October/025571.… [16] https://review.opendev.org/c/openstack/governance/+/819331 [17] https://review.opendev.org/c/openstack/governance/+/807163 [18] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [19] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann

1 0

Project-scoped app creds - Best practice
by Ryan Bannon 26 Nov '21

26 Nov '21

Hello all, Relatively new to OpenStack. To my understanding, application credentials are bound to users. Is there a way to bind them to Projects (I assume not) or, perhaps, Groups? My naive thought on a possible solution is that if a group has access to a Project, a "generic" user account that everybody has access to could be used for the application credentials. (The use case here is to not bind an app cred to an individual who might leave the organization, thus making the app cred secret lost.) Thanks, Ryan

2 5

Nova Local Disk Storage
by Seena Fallah 26 Nov '21

26 Nov '21

Hi, Is there any support for creating instances on a whole separate disk, not an LVM?

1 0

[Openstack-victoria] [IRONIC] error in network interface | baremetal node validate
by Swogat Pradhan 25 Nov '21

25 Nov '21

Hi, I am currently trying to setup openstack ironic using driver IPMI. I followed the official docs of openstack for setting everything up. When i run openstack baremetal node validate $NODE_UUID, i am getting the following error: * Unexpected exception, traceback saved into log by ironic conductor service that is running on controller: 'ServiceTokenAuthWrapper' object has no attribute '_discovery_cache' * in the network interface in command output. When i check the ironic conductor logs i see the following messages: <attached <https://mail.google.com/mail/u/0?ui=2&ik=7bb3543b01&attid=0.1&permmsgid=msg…> > Can anyone suggest a solution or a way forward. With regards Swogat Pradhan

2 4

[aodh][tacker] Support for pyparsing v 3.0.6 is required in python-aodhclient
by ueha.ayumu＠fujitsu.com 25 Nov '21

25 Nov '21

Hi aodh team, Hi I'm Ueha from tacker team. Tacker CI fails with the following error since a few days ago: ---from h-eng log--- Traceback (most recent call last): ... File "/usr/local/lib/python3.8/dist-packages/aodhclient/utils.py", line 43, in <module> expr = pp.operatorPrecedence(condition, [ AttributeError: module 'pyparsing' has no attribute 'operatorPrecedence' -------------------- As I have researched, the version of pyparsing has been updated to v3.0.6 in global requirements [1]. And operatorPrecedence was already deprecated in pyparsing v3.0.6. A patch to fix this appears to have already been posted to python-aodhclient [2]. Could you consider merging the fixes to avoid errors in the CI? For your information, it have been merged similar fixes in cinder [3] and manila [4]. [1] https://review.opendev.org/c/openstack/requirements/+/818614 [2] https://review.opendev.org/c/openstack/python-aodhclient/+/819054 [3] https://review.opendev.org/c/openstack/cinder/+/818834 [4] https://review.opendev.org/c/openstack/manila/+/818829 Regards, Ueha

1 0

[puppet] Propose retiring old stable branches(Stein, Rocky and Queens)
by Takashi Kajinami 25 Nov '21

25 Nov '21

Hello, In puppet repos we have plenty of stable branches still open, and the oldest is now stable/queens. However recently we haven't seen many backports proposed to stein, rocky and queens [1], so I'll propose retiring these three old branches now. Please let me know if anybody is interested in keeping any of these three. Note that currently CI jobs are broken in these three branches and it is likely we need to investigate the required additional pinning of dependent packages. If somebody is still interested in maintaining these old branches then these jobs should be fixed. [1] https://review.opendev.org/q/(project:%255Eopenstack/puppet-.*)AND(NOT+proj… Thank you, Takashi Kajinami

2 2

[neutron] Drivers meeting - Friday 26.11.2021 - cancelled
by Lajos Katona 25 Nov '21

25 Nov '21

Hi Neutron Drivers! Due to the lack of agenda, let's cancel tomorrow's drivers meeting. See You on the meeting next week. Lajos Katona (lajoskatona)

1 0