November 2021 - openstack-discuss

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

Live migration fails
by Szabo, Istvan (Agoda) 06 Aug '24

06 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

[all] Dynamic Zuul results table in Gerrit 3
by Radosław Piliszek 04 Aug '22

04 Aug '22

Hello Fellow OpenStack and OpenDev Folks! TL;DR click on [3] and enjoy. I am starting this thread to not hijack the discussion happening on [1]. First of all, I would like to thank gibi (Balazs Gibizer) for hacking a way to get the place to render the table in the first place (pun intended). I have been a long-time-now user of [2]. I have improved and customised it for myself but never really got to share back the changes I made. The new Gerrit obviously broke the whole script so it was of no use to share at that particular state. However, inspired by gibi's work, I decided to finally sit down and fix it to work with Gerrit 3 and here it comes: [3]. Works well on Chrome with Tampermonkey. Not tested others. I hope you will enjoy this little helper (I do). I know the script looks super fugly but it generally boils down to a mix of styles of 3 people and Gerrit having funky UI rendering. Finally, I'd also like to thank hrw (Marcin Juszkiewicz) for linking me to the original Michel's script in 2019. [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-November/019051… [2] https://opendev.org/x/coats/src/commit/444c95738677593dcfed0cfd9667d4c4f0d5… [3] https://gist.github.com/yoctozepto/7ea1271c299d143388b7c1b1802ee75e Kind regards, -yoctozepto

5 7

[all] PyCharm Licenses Renewed till July 2021
by Swapnil Kulkarni 13 Jul '22

13 Jul '22

Hello, Happy to inform you the open source developer license for Pycharm has been renewed for 1 additional year till July 2021. Best Regards, Swapnil Kulkarni coolsvap at gmail dot com

8 13

Nova not updating to new size of an extended in-use / attached cinder volume (Ceph RBD) to guest
by Christian Rohmann 28 Jun '22

28 Jun '22

Hello Openstack-Discuss, after some digging I found out that cinder learned to resize attached (in-use) volumes quite a while ago with the introduction of the initial "extend 'in-use' volume" feature: https://review.opendev.org/c/openstack/cinder/+/454287/ The support was then extended to also cover Ceph RBD backed volumes with: https://review.opendev.org/c/openstack/nova/+/613039/ Since this is only about the cinder part, I was wondering if nova would ever find out and would actively rescan the device / volume and quintessentially present an increased new size to the guest. Apparently this is where a certain volume-extended event comes into play: https://review.opendev.org/c/openstack/nova/+/454322/ that is to be emitted by cinder when a volume had been extened. I then went head and tried this with the openstack cli (as Horizon does not seem to support / offer to resize in-use volumes. I run Openstack Train with Ceph RBD as storage. First I ran into an issue with the openstack cli (https://bugs.launchpad.net/cinder/+bug/1871759/comments/2 <https://bugs.launchpad.net/cinder/+bug/1871759/comments/2>) but using cinder extend $volumeid $newsize I was able to resize in-use volumes just fine. The only thing missing was the propagation to the guest. I played around with SCSI rescans, but that did not work, reboots also failed to do anyting in this regards. Stopping and starting the VM did work, but why having the capability to online-resize an attached volume when not being able to resize the guest volume as well? Apparently there seems to be an old and somewhat similar observation / bug with https://bugs.launchpad.net/nova/+bug/1369465 So I was simply wondering if this is expected to be working? Are there any special settings / options I need to set to enable this feature? Thanks and with kind regards Christian

4 9

[oslo] Propose to EOL stable/queens, stable/rocky on all the oslo scope
by Herve Beraud 24 Jun '22

24 Jun '22

Hi, On our last meeting of the oslo team we discussed the problem with broken stable branches (rocky and older) in oslo's projects [1]. Indeed, almost all these branches are broken. Előd Illés kindly generated a list of periodic-stable errors on Oslo's stable branches [2]. Given the lack of active maintainers on Oslo and given the current status of the CI in those branches, I propose to make them End Of Life. I will wait until the end of month for anyone who would like to maybe step up as maintainer of those branches and who would at least try to fix CI of them. If no one will volunteer for that, I'll EOLing those branches for all the projects under the oslo umbrella. Let us know your thoughts. Thank you for your attention. [1] https://meetings.opendev.org/meetings/oslo/2021/oslo.2021-10-04-15.00.log.t… [2] http://lists.openstack.org/pipermail/openstack-discuss/2021-July/023939.html -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud

5 5

[TripleO] Core team cleanup
by James Slagle 30 Mar '22

30 Mar '22

Hello, I took a look at our core team, "tripleo-core" in gerrit. We have a few individuals who I feel have moved on from TripleO in their focus. I looked at the reviews from stackalytics.io for the last 180 days[1]. These individuals have less than 6 reviews, which is about 1 review a month: Bob Fournier Dan Sneddon Dmitry Tantsur Jiří Stránský Juan Antonio Osorio Robles Marius Cornea These individuals have publicly expressed that they are moving on from TripleO: Michele Baldessari wes hayutin I'd like to propose we remove these folks from our core team, while thanking them for their contributions. I'll also note that I'd still value +1/-1 from these folks with a lot of significance, and encourage them to review their areas of expertise! If anyone on the list plans to start reviewing in TripleO again, then I also think we can postpone the removal for the time being and re-evaluate later. Please let me know if that's the case. Please reply and let me know any agreements or concerns with this change. Thank you! [1] https://www.stackalytics.io/report/contribution?module=tripleo-group&projec… -- -- James Slagle --

5 6

[all][tc] Relmgt team position on release cadence
by Thierry Carrez 14 Feb '22

14 Feb '22

Hi everyone, The (long) document below reflects the current position of the release management team on a popular question: should the OpenStack release cadence be changed? Please note that we only address the release management / stable branch management facet of the problem. There are other dimensions to take into account (governance, feature deprecation, supported distros...) to get a complete view of the debate. Introduction ------------ The subject of how often OpenStack should be released has been regularly debated in the OpenStack community. OpenStack started with a 3-month release cycle, then switched to 6-month release cycle starting with Diablo. It is often thought of a release management decision, but it is actually a much larger topic: a release cadence is a trade-off between pressure to release more often and pressure to release less often, coming in from a lot of different stakeholders. In OpenStack, it is ultimately a Technical Committee decision. But that decision is informed by the position of a number of stakeholders. This document gives historical context and describes the current release management team position. The current trade-off --------------------- The main pressure to release more often is to make features available to users faster. Developers get a faster feedback loop, hardware vendors ensure software is compatible with their latest products, and users get exciting new features. "Release early, release often" is a best practice in our industry -- we should generally aim at releasing as often as possible. But that is counterbalanced by pressure to release less often. From a development perspective, each release cycle comes with some process overhead. On the integrators side, a new release means packaging and validation work. On the users side, it means pressure to upgrade. To justify that cost, there needs to be enough user-visible benefit (like new features) in a given release. For the last 10 years for OpenStack, that balance has been around six months. Six months let us accumulate enough new development that it was worth upgrading to / integrating the new version, while giving enough time to actually do the work. It also aligned well with Foundation events cadence, allowing to synchronize in-person developer meetings date with start of cycles. What changed ------------ The major recent change affecting this trade-off is that the pace of new development in OpenStack slowed down. The rhythm of changes was divided by 3 between 2015 and 2021, reflecting that OpenStack is now a mature and stable solution, where accessing the latest features is no longer a major driver. That reduces some of the pressure for releasing more often. At the same time, we have more users every day, with larger and larger deployments, and keeping those clusters constantly up to date is an operational challenge. That increases the pressure to release less often. In essence, OpenStack is becoming much more like a LTS distribution than a web browser -- something users like moving slow. Over the past years, project teams also increasingly decoupled individual components from the "coordinated release". More and more components opted for an independent or intermediary-released model, where they can put out releases in the middle of a cycle, making new features available to their users. This increasingly opens up the possibility of a longer "coordinated release" which would still allow development teams to follow "release early, release often" best practices. All that recent evolution means it is (again) time to reconsider if the 6-month cadence is what serves our community best, and in particular if a longer release cadence would not suit us better. The release management team position on the debate -------------------------------------------------- While releasing less often would definitely reduce the load on the release management team, most of the team work being automated, we do not think it should be a major factor in motivating the decision. We should not adjust the cadence too often though, as there is a one-time cost in switching our processes. In terms of impact, we expect that a switch to a longer cycle will encourage more project teams to adopt a "with-intermediary" release model (rather than the traditional "with-rc" single release per cycle), which may lead to abandoning the latter, hence simplifying our processes. Longer cycles might also discourage people to commit to PTL or release liaison work. We'd probably need to manage expectations there, and encourage more frequent switches (or create alternate models). If the decision is made to switch to a longer cycle, the release management team recommends to switch to one year directly. That would avoid changing it again anytime soon, and synchronizing on a calendar year is much simpler to follow and communicate. We also recommend announcing the change well in advance. We currently have an opportunity of making the switch when we reach the end of the release naming alphabet, which would also greatly simplify the communications around the change. Finally, it is worth mentioning the impact on the stable branch work. Releasing less often would likely impact the number of stable branches that we keep on maintaining, so that we do not go too much in the past (and hit unmaintained distributions or long-gone dependencies). We currently maintain releases for 18 months before they switch to extended maintenance, which results in between 3 and 4 releases being maintained at the same time. We'd recommend switching to maintaining one-year releases for 24 months, which would result in between 2 and 3 releases being maintained at the same time. Such a change would lead to longer maintenance for our users while reducing backporting work for our developers. -- Thierry Carrez (ttx) On behalf of the OpenStack Release Management team

15 45

[nova][powervm][zvm] Time to mark powervm and zvm drivers as deprecated for removal?
by Stephen Finucane 28 Jan '22

28 Jan '22

I've had a PR open against the pypowervm library for over a fortnight now [1] with no activity. This has prompted me to go looking into the state of the powervm driver and what I've found isn't great. It doesn't seem there has been any feature development on the driver for many years, and the last change I can see that wasn't simply of a case of "I need to do this to get tests to pass" was over three years ago [2]. The CI also doesn't appear to have been touched in years. The situation for zvm isn't any better. The last functional change to that driver was nearly 2 years ago and I wasn't able to spot any CI running (though it's possible I just missed this). This means the powervm and zvm drivers stand out from the other non-libvirt drivers in-tree, each of which are getting at least some activity (albeit out- of-tree for Hyper-V, from what I recall). It also begs the question: are these drivers something we want to keep around? If we do, how are we going to maintain them long term? If not, how aggressive should we be in removing that now dead code? I'll open patches to mark both powervm and zvm as deprecated shortly so that, assuming nothing changes in the interim, we can look to remove it early in the Z cycle. Please respond here or on the reviews if you have concerns. Stephen [1] https://github.com/powervm/pypowervm/pull/17 [2] Change ID I89ad36f19672368a1f795e1f29c5af6368ccfeec

1 1

[ops][nova][victoria] Power State = Suspended?
by DHilsbos＠performair.com 24 Jan '22

24 Jan '22

All; I had something unusual happen this morning; one of my VMs was showing "Suspended" under the Power State in the Horizon dashboard. I've never seen that. What does it mean? Any search that I do points me to a bunch of resources for Status Suspended. Thank you, Dominic L. Hilsbos, MBA Vice President - Information Technology Perform Air International Inc. DHilsbos(a)PerformAir.com www.PerformAir.com

4 8

[nova][dev] Revisiting qemu emulation where guest arch != host arch
by Apsey, Christopher 05 Jan '22

05 Jan '22

All, A few years ago I asked a question[1] about why nova, when given a hw_architecture property from glance for an image, would not end up using the correct qemu-system-xx binary when starting the guest process on a compute node if that compute nodes architecture did not match the proposed guest architecture. As an example, if we had all x86 hosts, but wanted to run an emulated ppc guest, we should be able to do that given that at least one compute node had qemu-system-ppc already installed and libvirt was successfully reporting that as a supported architecture to nova. It seemed like a heavy lift at the time, so it was put on the back burner. I am now in a position to fund a contract developer to make this happen, so the question is: would this be a useful blueprint that would potentially be accepted? Most of the time when people want to run an emulated guest they would just nest it inside of an already running guest of the native architecture, but that severely limits observability and the task of managing any more than a handful of instances in this manner quickly becomes a tangled nightmare of networking, etc. I see real benefit in allowing this scenario to run natively so all of the tooling that exists for fleet management 'just works'. This would also be a significant differentiator for OpenStack as a whole. Thoughts? [1] http://lists.openstack.org/pipermail/openstack-operators/2018-August/015653… Chris Apsey Director | Georgia Cyber Range GEORGIA CYBER CENTER 100 Grace Hopper Lane | Augusta, Georgia | 30901 https://www.gacybercenter.org

2 2

[stable][requirements][zuul] unpinned setuptools dependency on stable
by Balazs Gibizer 15 Dec '21

15 Dec '21

Hi, The latest setuptools (58.0) removed support for "use_2to3" [1] (deprecated since 46.2). Many OpenStack modules defines decorator==3.4.0 as lower constraints[2]. However, decorator 3.4.0 cannot be installed anymore with the latest setuptool as it depends on "use_2to3". On master, this can be solved easily by bumping the dependency to decorator 4.0.0. On stable/xena we can still solve it the same way with a new RC. But on older stable branches such solution might be against the stable policy as it would require a major bump on our dependencies. This issue is not limited to lower-constraints testing it just hit us there first. A similar break could happen in our upper constraints as well on old stable branches. The root of the problem is that we always use the latest setuptools in our CI testing even on old stable branches. Zuul's ensure-tox task[4] installs tox which installs the virtualenv package which bundles the latest setuptools package. This happens without applying any constraints. Then tox is used to install the project's dependencies under lower or upper constraints with the unconstrained setuptools available. During and after yesterday's nova meeting [3] we discussed possible solutions. Option 1: Bump the major version of the decorator dependency on stable. Pros: * easy to implement Cons: * might against the policy / breaks downstream packagers * needs to be done in each affected project[3] separately * not future proof. After a future setuptools release we can see a similar break with another of our dependencies. @Stable Cores: how do you feel about such bump? Option 2: Pin the setuptools version during tox installation Pros: * single solution for all affected projects * future proof against breaks introduced by future setuptools releases Cons: * complex change as it probably requires to extend the base task in zuul itself Option 3: turn off lower-constraints testing Pros: * easy to implement * some of us want to get rid of lower constraints anyhow Cons: * needs per project changes * not future proof against similar break affecting our upper constraints on old stable branches. Option 4: utilize pyproject.toml[6] to specify build-time requirements Unfortunately, I'm not sure if it is possible to restrict the maximum setuptools version with it As a side note I tried applying [tox]requires in tox.ini to restrict setuptools version[7] but It does not seem to take effect in our case[8]. @Stable Cores: what do you think what could be the way forward? Cheers, gibi [1] https://setuptools.pypa.io/en/latest/history.html#v58-0-2 [2] https://codesearch.openstack.org/?q=decorator%3D%3D3.4.0&i=nope&literal=nop… [3] https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.20… [4] https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-tox/tasks… [5] https://tox.readthedocs.io/en/latest/config.html#conf-requires [6] https://www.python.org/dev/peps/pep-0518/ [7] https://review.opendev.org/c/openstack/placement/+/810293/1/tox.ini [8] https://zuul.opendev.org/t/openstack/build/f78a0300c8734e7c8475309af1d2e1a4…

11 28

[Openstack][FOSDEM][CFP] Virtualization & IaaS Devroom
by Piotr Kliczewski 15 Dec '21

15 Dec '21

We are excited to announce that the call for proposals is now open for the Virtualization & IaaS devroom at the upcoming FOSDEM 2022, to be hosted virtually on February 5th 2022. This year will mark FOSDEM’s 22nd anniversary as one of the longest-running free and open source software developer events, attracting thousands of developers and users from all over the world. Due to Covid-19, FOSDEM will be held virtually this year on February 5th & 6th, 2022. About the Devroom The Virtualization & IaaS devroom will feature session topics such as open source hypervisors and virtual machine managers such as Xen Project, KVM, bhyve, and VirtualBox, and Infrastructure-as-a-Service projects such as KubeVirt, Apache CloudStack, Foreman, OpenStack, oVirt, QEMU and OpenNebula. This devroom will host presentations that focus on topics of shared interest, such as KVM; libvirt; shared storage; virtualized networking; cloud security; clustering and high availability; interfacing with multiple hypervisors; hyperconverged deployments; and scaling across hundreds or thousands of servers. Presentations in this devroom will be aimed at users or developers working on these platforms who are looking to collaborate and improve shared infrastructure or solve common problems. We seek topics that encourage dialog between projects and continued work post-FOSDEM. Important Dates Submission deadline: 20th of December Acceptance notifications: 25th of December Final schedule announcement: 31st of December Recorded presentations upload deadline: 15th of January Devroom: 6th February 2022 Submit Your Proposal All submissions must be made via the Pentabarf event planning site[1]. If you have not used Pentabarf before, you will need to create an account. If you submitted proposals for FOSDEM in previous years, you can use your existing account. After creating the account, select Create Event to start the submission process. Make sure to select Virtualization and IaaS devroom from the Track list. Please fill out all the required fields, and provide a meaningful abstract and description of your proposed session. Submission Guidelines We expect more proposals than we can possibly accept, so it is vitally important that you submit your proposal on or before the deadline. Late submissions are unlikely to be considered. All presentation slots are 30 minutes, with 20 minutes planned for presentations, and 10 minutes for Q&A. All presentations will need to be pre-recorded and put into our system at least a couple of weeks before the event. The presentations should be uploaded by 15th of January and made available under Creative Commons licenses. In the Submission notes field, please indicate that you agree that your presentation will be licensed under the CC-By-SA-4.0 or CC-By-4.0 license and that you agree to have your presentation recorded. For example: "If my presentation is accepted for FOSDEM, I hereby agree to license all recordings, slides, and other associated materials under the Creative Commons Attribution Share-Alike 4.0 International License. Sincerely, <NAME>." In the Submission notes field, please also confirm that if your talk is accepted, you will be able to attend the virtual FOSDEM event for the Q&A. We will not consider proposals from prospective speakers who are unsure whether they will be able to attend the FOSDEM virtual event. If you are experiencing problems with Pentabarf, the proposal submission interface, or have other questions, you can email our devroom mailing list[2] and we will try to help you. Code of Conduct Following the release of the updated code of conduct for FOSDEM, we'd like to remind all speakers and attendees that all of the presentations and discussions in our devroom are held under the guidelines set in the CoC and we expect attendees, speakers, and volunteers to follow the CoC at all times. If you submit a proposal and it is accepted, you will be required to confirm that you accept the FOSDEM CoC. If you have any questions about the CoC or wish to have one of the devroom organizers review your presentation slides or any other content for CoC compliance, please email us and we will do our best to assist you. Call for Volunteers We are also looking for volunteers to help run the devroom. We need assistance with helping speakers to record the presentation as well as helping with streaming and chat moderation for the devroom. Please contact devroom mailing list [2] for more information. Questions? If you have any questions about this devroom, please send your questions to our devroom mailing list. You can also subscribe to the list to receive updates about important dates, session announcements, and to connect with other attendees. See you all at FOSDEM! [1] https://penta.fosdem.org/submission/FOSDEM22 [2] iaas-virt-devroom at lists.fosdem.org

1 1

[nova] Next spec review day on Tuesday Dec 14th
by Sylvain Bauza 13 Dec '21

13 Dec '21

Hola, As agreed during today's Nova meeting, we will have another spec review day on Dec 14th. Please sharpen your pens and write your specs in advance, and just be around during this day if you want to discuss with the reviewers. Thanks, -Sylvain

1 1

[CLOUDKITTY][VICTORIA] - policy prohibit report:get_summary to admin
by Gaël THEROND 13 Dec '21

13 Dec '21

Hi everyone! Today I faced a weird situation with one of our cloud platforms using victoria release. When trying to get a summary of projects rates would it be through Horizon or CLI using the admin user of the platform we've got the following error message: https://paste.opendev.org/show/bIgG6owrN9B2F3O7iqYG/ >From my understanding of the default policies of cloudkitty, this error seems to be a bit odd as the admin user profile actually match the default rules. At least as exposed in: https://opendev.org/openstack/cloudkitty/src/branch/stable/victoria/cloudki… and https://opendev.org/openstack/cloudkitty/src/branch/stable/victoria/cloudki… Unless I misunderstood something (please correct me if I'm wrong), it's supposed to at least be ok with the matching.

3 7

[all][tc] What's happening in Technical Committee: summary 15th Oct, 21: Reading: 5 min
by Ghanshyam Mann 10 Dec '21

10 Dec '21

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * TC this week IRC meeting held on Oct 14th Thursday. * Most of the meeting discussions are summarized below (Completed or in-progress activities section). Meeting full logs are available @ - https://meetings.opendev.org/meetings/tc/2021/tc.2021-10-14-15.00.log.html * Next week's meeting is cancelled as we are meeting in PTG. We will have the next IRC meeting on Oct 28th, Thursday 15:00 UTC, feel free the topic on agenda[1] by Oct 27th. 2. What we completed this week: ========================= * None in this week. 3. Activities In progress: ================== TC Tracker for Xena cycle ------------------------------ * TC is using the etherpad[2] for Xena cycle working item. We will be checking it in PTG * Current status is: 9 completed, 3 to-be-discussed in PTG, 1 in-progress Open Reviews ----------------- * Six open reviews for ongoing activities[3]. New project 'Skyline' proposal ------------------------------------ * You might be aware of this new dashboard proposal in the previous month's discussion. * A new project 'Skyline: an OpenStack dashboard optimized by UI and UE' is now proposed in governance to be an official OpenStack project[4]. * Skyline team is planning to meet in PTG on Tue, Wed and Thu at 5UTC, please ask your queries or have feedback/discussion with the team next week. Place to maintain the external hosted ELK, E-R, O-H services ------------------------------------------------------------------------- * We had a final discussion or I will say just a status update on this which was mentioned in last week's email summary[5]. * Now onwards, discussion and migration work will be done in TACT SIG (#openstack-infra IRC channel). Add project health check tool ----------------------------------- * No updates on this, we will continue discussing it in PTG for the next steps on this and what to do with TC liaison things. * Meanwhile, we are reviewing Rico proposal on collecting stats tools [6]. Stable Core team process change --------------------------------------- * Current proposal is under review[7]. Feel free to provide early feedback if you have any. Call for 'Technical Writing' SIG Chair/Maintainers ---------------------------------------------------------- * As agreed in last week's TC meeting, we will be moving this SIG work towards TC. * TC members have been added to the core members list in the SIG repos. * We will be discussing where to move the training repos/work in PTG. TC tags analysis ------------------- * Operator feedback is asked on open infra newsletter too, and we will continue the discussion in PTG and will take the final decision based on feedback we receive, if any[9]. Complete the policy pop up team ---------------------------------------- * Policy pop team has served its purpose and we have new RBAC as one of the the community-wide goal for the Yoga cycle. * We are marking this popup team as completed[10]. Project updates ------------------- * Retiring js-openstack-lib [11] Yoga release community-wide goal ----------------------------------------- * Please add the possible candidates in this etherpad [12]. * Current status: "Secure RBAC" is selected for Yoga cycle[13]. PTG planning ---------------- * We will be meeting in PTG next week, please check the details in this etherpad [14] * Do not forget to join the TC+community leaders sessions on Monday, Oct 18 15 UTC - 17 UTC. Test support for TLS default: ---------------------------------- * Rico has started a separate email thread over testing with tls-proxy enabled[15], we encourage projects to participate in that testing and help to enable the tls-proxy in gate testing. 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[16]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [17] 3. Office hours: The Technical Committee offers a weekly office hour every Tuesday at 0100 UTC [18] 4. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://etherpad.opendev.org/p/tc-xena-tracke [3] https://review.opendev.org/q/projects:openstack/governance+status:open [4] https://review.opendev.org/c/openstack/governance/+/814037 [5]http://lists.openstack.org/pipermail/openstack-discuss/2021-October/02525… [6] https://review.opendev.org/c/openstack/governance/+/810037 [7] https://review.opendev.org/c/openstack/governance/+/810721 [9] https://governance.openstack.org/tc/reference/tags/index.html [10] https://review.opendev.org/c/openstack/governance/+/814186 [11] https://review.opendev.org/c/openstack/governance/+/798540 [12] https://review.opendev.org/c/openstack/governance/+/807163 [13] https://etherpad.opendev.org/p/y-series-goals [14] https://etherpad.opendev.org/p/tc-yoga-ptg [15] http://lists.openstack.org/pipermail/openstack-discuss/2021-June/023000.html [16] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [17] http://eavesdrop.openstack.org/#Technical_Committee_Meeting [18] http://eavesdrop.openstack.org/#Technical_Committee_Office_hours -gmann

9 25

[neutron][CI] How to reduce number of rechecks - brainstorming
by Slawek Kaplonski 07 Dec '21

07 Dec '21

Hi, Recently I spent some time to check how many rechecks we need in Neutron to get patch merged and I compared it to some other OpenStack projects (see [1] for details). TL;DR - results aren't good for us and I think we really need to do something with that. Of course "easiest" thing to say is that we should fix issues which we are hitting in the CI to make jobs more stable. But it's not that easy. We are struggling with those jobs for very long time. We have CI related meeting every week and we are fixing what we can there. Unfortunately there is still bunch of issues which we can't fix so far because they are intermittent and hard to reproduce locally or in some cases the issues aren't realy related to the Neutron or there are new bugs which we need to investigate and fix :) So this is never ending battle for us. The problem is that we have to test various backends, drivers, etc. so as a result we have many jobs running on each patch - excluding UT, pep8 and docs jobs we have around 19 jobs in check and 14 jobs in gate queue. In the past we made a lot of improvements, like e.g. we improved irrelevant files lists for jobs to run less jobs on some of the patches, together with QA team we did "integrated-networking" template to run only Neutron and Nova related scenario tests in the Neutron queues, we removed and consolidated some of the jobs (there is still one patch in progress for that but it should just remove around 2 jobs from the check queue). All of that are good improvements but still not enough to make our CI really stable :/ Because of all of that, I would like to ask community about any other ideas how we can improve that. If You have any ideas, please send it in this email thread or reach out to me directly on irc. We want to discuss about them in the next video CI meeting which will be on November 30th. If You would have any idea and would like to join that discussion, You are more than welcome in that meeting of course :) [1] http://lists.openstack.org/pipermail/openstack-discuss/2021-November/ 025759.html -- Slawek Kaplonski Principal Software Engineer Red Hat

11 19

[ops]RabbitMQ High Availability
by DHilsbos＠performair.com 06 Dec '21

06 Dec '21

All; In the time I've been part of this mailing list, the subject of RabbitMQ high availability has come up several times, and each time specific recommendations for both Rabbit and Open Stack are provided. I remember it being an A or B kind of recommendation (i.e. configure Rabbit like A1, and Open Stack like A2, OR configure Rabbit like B1, and Open Stack like B2). Unfortunately, I can't find the previous threads on this topic. Does anyone have this information, that they would care to share with me? Thank you, Dominic L. Hilsbos, MBA Vice President - Information Technology Perform Air International Inc. DHilsbos(a)PerformAir.com www.PerformAir.com

3 3

[kolla] Plan to deprecate binary and unify on single distrubition
by Michał Nasiadka 06 Dec '21

06 Dec '21

Hello Koalas, On the PTG we have discussed two topics: 1) Deprecate and drop binary type of Kolla images 2) Use a common base (single Linux distribution) for Kolla images This is a call for feedback - for people that have not been attending the PTG. What this essentially mean for consumers: 1) In Yoga cycle we will deprecate binary type of Kolla images, and in Z cycle those will be dropped. 2) We are not going to support CentOS Stream 9 (cs9) as a base operating system, and the source type build will rely on CentOS Stream 8 in Z release. 3) Beginning from A release Kolla will build only Debian source images - but Kolla-Ansible will still support deployment of those images on CentOS/Ubuntu/Debian Host operating systems (and Rocky Linux to be added in Yoga to that mix). Justification: The Kolla project team is limited in numbers, therefore supporting current broad mix of operating systems (especially with CentOS Stream 9 ,,on the way’’) is a significant maintenance burden. By dropping binary type of images - users would be running more tested images (since Kolla/Kolla-Ansible CI runs source images jobs as voting). In Xena we’ve already changed the default image type Kolla-Ansible uses to source. We also feel that using a unified base OS for Kolla container images is a way to remove some of the maintenance burden (including CI cycles and Request for feedback: If any of those changes is a no go from your perspective - we’d like to hear your opinions. Best regards, Michal Nasiadka

9 17

[TripleO] IPV6 Support
by Lokendra Rathour 03 Dec '21

03 Dec '21

Hi, I am trying to install the undercloud using the ipv6 address on Centos 8 using Train release. It is seen that the deployment of undercloud is getting failed with error as mentioned below. Same deployment is working in the normal ipv4 case. So Questions: 1. is IPV6 supported in TripleO , if yes then please suggest how. Error: nied", " attempt(s): 2", "2021-11-25 07:25:31,535 WARNING: 65355 -- Retrying running container: zaqar", "2021-11-25 07:25:34,887 ERROR: 65355 -- ['/usr/bin/podman', 'run', '--user', '0', '--name', 'container-puppet-zaqar', '--env', 'PUPPET_TAGS=file,file_line,concat,augeas,cron,zaqar_config', '--env', 'NAME=zaqar', '--env', 'HOSTNAME=undercloud', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--env', 'DEBUG=False', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmp06nvxzzz:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--rm', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/container-puppet-zaqar.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', 'undercloud.ctlplane.localdomain:8787/tripleotraincentos8/centos-binary-zaqar-wsgi:current-tripleo'] run failed after Error: container_linux.go:370: starting container process caused: error adding seccomp filter rule for syscall bdflush: permission denied: OCI permission denied", " attempt(s): 3", "2021-11-25 07:25:34,888 WARNING: 65355 -- Retrying running container: zaqar", "2021-11-25 07:25:34,888 ERROR: 65355 -- Failed running container for zaqar", "2021-11-25 07:25:34,888 INFO: 65355 -- Finished processing puppet configs for zaqar", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring crond", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring glance_api", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring heat_api", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring heat", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring ironic_api", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring ironic", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring ironic_inspector", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring neutron", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring iscsid", "2021-11-25 07:25:34,889 ERROR: 65345 -- ERROR configuring keystone", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring memcached", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring mistral", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring mysql", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring nova", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring rabbitmq", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring placement", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring swift", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring swift_ringbuilder", "2021-11-25 07:25:34,890 ERROR: 65345 -- ERROR configuring zaqar" ], "stderr_lines": [], "_ansible_no_log": false, "attempts": 15 } ] ] Not cleaning working directory /home/stack/tripleo-heat-installer-templates Not cleaning ansible directory /home/stack/undercloud-ansible-mw4crw92 Install artifact is located at /home/stack/undercloud-install-20211125072536.tar.bzip2 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Deployment Failed! ERROR: Heat log files: /var/log/heat-launcher/undercloud_deploy-o_qf1b4w !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Deployment failed. Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1345, in _standalone_deploy raise exceptions.DeploymentError('Deployment failed') tripleoclient.exceptions.DeploymentError: Deployment failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1363, in _standalone_deploy raise exceptions.DeploymentError(six.text_type(e)) tripleoclient.exceptions.DeploymentError: Deployment failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/cliff/app.py", line 401, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1451, in take_action if self._standalone_deploy(parsed_args) != 0: File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1400, in _standalone_deploy raise exceptions.DeploymentError('Deployment failed.') tripleoclient.exceptions.DeploymentError: Deployment failed. clean_up Deploy: Deployment failed. Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1345, in _standalone_deploy raise exceptions.DeploymentError('Deployment failed') tripleoclient.exceptions.DeploymentError: Deployment failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1363, in _standalone_deploy raise exceptions.DeploymentError(six.text_type(e)) tripleoclient.exceptions.DeploymentError: Deployment failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/osc_lib/shell.py", line 136, in run ret_val = super(OpenStackShell, self).run(argv) File "/usr/lib/python3.6/site-packages/cliff/app.py", line 281, in run result = self.run_subcommand(remainder) File "/usr/lib/python3.6/site-packages/osc_lib/shell.py", line 176, in run_subcommand ret_value = super(OpenStackShell, self).run_subcommand(argv) File "/usr/lib/python3.6/site-packages/cliff/app.py", line 401, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1451, in take_action if self._standalone_deploy(parsed_args) != 0: File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1400, in _standalone_deploy raise exceptions.DeploymentError('Deployment failed.') tripleoclient.exceptions.DeploymentError: Deployment failed. END return value: 1 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! An error has occured while deploying the Undercloud. See the previous output for details about what went wrong. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Command '['sudo', '--preserve-env', 'openstack', 'tripleo', 'deploy', '--standalone', '--standalone-role', 'Undercloud', '--stack', 'undercloud', '--local-domain=localdomain', '--local-ip=abce:abce:abce::1/64', '--templates=/usr/share/openstack-tripleo-heat-templates/', '--networks-file=network_data_undercloud.yaml', '--heat-native', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/podman.yaml', '-e', '/home/stack/containers-prepare-parameter.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/zaqar-swift-backend.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/ssl/no-tls-endpoints-public-ip.yaml', '--deployment-user', 'stack', '--output-dir=/home/stack', '-e', '/home/stack/tripleo-config-generated-env-files/undercloud_parameters.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/tripleo-validations.yaml', '--debug', '--log-file=install-undercloud.log', '-e', '/usr/share/openstack-tripleo-heat-templates/undercloud-stack-vstate-dropin.yaml']' returned non-zero exit status 1. Command '['sudo', '--preserve-env', 'openstack', 'tripleo', 'deploy', '--standalone', '--standalone-role', 'Undercloud', '--stack', 'undercloud', '--local-domain=localdomain', '--local-ip=abce:abce:abce::1/64', '--templates=/usr/share/openstack-tripleo-heat-templates/', '--networks-file=network_data_undercloud.yaml', '--heat-native', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/podman.yaml', '-e', '/home/stack/containers-prepare-parameter.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/zaqar-swift-backend.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/ssl/no-tls-endpoints-public-ip.yaml', '--deployment-user', 'stack', '--output-dir=/home/stack', '-e', '/home/stack/tripleo-config-generated-env-files/undercloud_parameters.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/tripleo-validations.yaml', '--debug', '--log-file=install-undercloud.log', '-e', '/usr/share/openstack-tripleo-heat-templates/undercloud-stack-vstate-dropin.yaml']' returned non-zero exit status 1. END return value: 1 [stack@undercloud ~]$

2 5

[ironic][meeting] Postpone meeting time until end of March 2022
by Riccardo Pittau 03 Dec '21

03 Dec '21

Hello ironicers! Due to some overlapping time with other meetings that cause me to attend to 2 (or sometimes 3.....) meetings at the same time, I'd like to propose to move the time for the weekly ironic upstream meeting 1 hour later, so from the current 15:00-16:00 UTC to 16:00-17:00 UTC, until the next Daylight saving time change that should happen at the end of March 2022. Thank you! Ciao, Riccardo

4 5

python_requires >= 3.8 during Yoga
by Stephen Finucane 02 Dec '21

02 Dec '21

gmann has been helpfully proposing patches to change the versions of Python we're testing against in Yoga. I've suggested that we might want to bump 'python_requires' in 'setup.cfg' to indicate that we no longer support any version of Python before 3.8 [1]. As gmann has noted, doing so would mean nova would no longer be installable on Python 3.6 or 3.7 and there has been a small bit of back and forth on the pros and cons of this. I'm wondering what other people's thoughts on this are. Is this something we should be doing? Should we do it for libraries too or just services? Do we ever want to do this? Thoughts, please! Stephen [1] https://review.opendev.org/c/openstack/nova/+/819194/comment/72ecf24f_2bd29…

17 61

Windows imaging process
by Gaël THEROND 02 Dec '21

02 Dec '21

Hi everyone, On one of our Openstack platforms, we maintain windows based workloads. We currently have the following build process: 1°/- We download windows ISO image. 2°/- Use hyper-v to create a gold image that bring virtio drivers. 3°/- We upload this image on glance and publish it for our users. 4°/- Our users use packer to create their own windows custom image from our gold image. This workflow works pretty fine, it’s simple enough and as Microsoft isn’t releasing a new major Windows every two months it’s pretty ok not having all the steps automated until now. However, I’m wondering if there is a way to automate my first two steps using Openstack? So far, from my early tests, I didn’t managed to get all of appropriate gears to work together, did I missed something? I need to create a VM that at least can use the Windows ISO plus a virtio ISO as a second cdrom device and an additional user-data payload that will instruct the Windows installer to go automatically through the installation steps plus loads missing drivers from the virtio iso. This is perfectly working on a native kvm as you can add multiple cdrom device but I didn’t find a way to replicate that on Openstack. Starting from an unaltered Microsoft originated ISO image is a mandatory requirement for this project (Because of security constraints that I can’t have any impact on). Help from anyone that already had to deal with such situation would be very appreciated ! Thanks!

2 3

[kuryr] Propose to EOL stable branches (Stein, Rocky and Queens) for all the kuryr scope
by Maysa De Macedo Souza 02 Dec '21

02 Dec '21

Hello, We have plenty of stable branches in Kuryr repositories still open with the oldest being stable/queens. We haven't seen many backports[0] proposed to Stein, Rocky and Queens, so I would like to propose retiring these branches. Let me know if anybody is interested in keeping any of those branches. [0] https://review.opendev.org/q/(project:%255Eopenstack/kuryr.*)AND((branch:st… Thank you, Maysa Macedo.

2 3

[all][tc] Technical Committee next weekly meeting on Dec 2nd at 1500 UTC
by Ghanshyam Mann 01 Dec '21

01 Dec '21

Hello Everyone, Technical Committee's next weekly meeting is scheduled for Dec 2nd at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, Dec 1st, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[cinder] : SAN migration
by Gauurav Sabharwal1 01 Dec '21

01 Dec '21

Hi Experts , I need some expert advise of one of the scenario, I have multiple isolated OpenStack cluster running with train & rocky edition. Each OpenStack cluster environment have it's own isolated infrastructure of SAN ( CISCO fabric ) & Storage ( HP, EMC & IBM). Now company planning to refresh their SAN infrastructure. By procuring new Brocade SAN switches. But there are some migration relevant challenges we have. As we understand under one cinder instance only one typer of FC zone manager is supported . Currently customer configured & managing CISCO . Is it possible to configure two different vendor FC Zone manager under one cinder instance. Migration of SAN zoning is supposedly going to be happen offline way from OpenStack point of view. We will be migrating all ports of each existing cisco fabric to Brocade with zone configuration using brocade CLI. Our main concern is that after migration How CINDER DB update new zone info & path via Brocade SAN. Regards Gauurav Sabharwal IBM India Pvt. Ltd. IBM towers Ground floor, Block -A , Plot number 26, Sector 62, Noida Gautam budhnagar UP-201307. Email:gauurav.sabharwal@in.ibm.com Mobile No.: +91-9910159277

3 3

[neutron][networking][ipv6][dns][ddi] Upcoming OpenInfra Edge Computing Group sessions
by Ildiko Vancsa 30 Nov '21

30 Nov '21

Hi, I’m reaching out to you to draw your attention to the amazing lineup of discussion topics for the OpenInfra Edge Computing Group weekly calls up until the end of this year with industry experts to present and participate in the discussions! I would like to invite and encourage you to join the working group sessions to discuss edge related challenges and solutions in the below areas and more! Some of the sessions to highlight will be continuing the discussions we started at the recent PTG: * November 29th - Networking and DNS discussion with Cricket Liu and Andrew Wertkin * December 6th - Networking and IPv6 discussion with Ed Horley Beyond the above topics we will have presentations from groups and projects such as Smart Edge and the Industry IoT Consortium (Formerly Industrial Internet Consortium). For the full schedule please see the edge WG’s wiki: https://wiki.openstack.org/wiki/Edge_Computing_Group#Upcoming_Topics Our calls happen every Monday at 6am Pacific Time / 1400 UTC on Zoom. You can find all the meeting details here: https://wiki.openstack.org/wiki/Edge_Computing_Group#Meetings Please let me know if you have any questions about the working group or any of the upcoming sessions. Thanks and Best Regards, Ildikó

1 3

[Neutron][OVN] nova-compute timeout while waiting for VIF activation confirmed by Neutron
by Tony Liu 30 Nov '21

30 Nov '21

Hi, I see such problem from time to time. It's not consistently reproduceable. ====================== 2021-11-23 22:16:28.532 7 INFO nova.compute.manager [req-50817b01-e7ae-4991-94fe-e4c5672c481b - - - - -] [instance: 7021bf37-348d-4dfa-b0c5-6a251fceb6cd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-11-23 22:21:28.342 7 WARNING nova.virt.libvirt.driver [req-814c98a4-3fd9-4607-9abb-5fbe5cef8650 3a4c320d64664d9cb6784b7ea52d618a 21384fe21ebb4339b1a5f2a8f9000ea3 - default default] [instance: 7021bf37-348d-4dfa-b0c5-6a251fceb6cd] Timeout waiting for [('network-vif-plugged', '735d97a3-db7c-42ed-b3be-1596d4cc7f4b')] for instance with vm_state building and task_state spawning.: eventlet.timeout.Timeout: 300 seconds ====================== The VIF/port is activated by OVN ovn-controller to ovn-sb. It seems that, either Neutron didn't capture the update or didn't send message back to nova-compute. Is there any known fix for this problem? Thanks! Tony

5 9

[neutron][nova-compute] Received unexpected event network-vif-plugged-<uuid>
by Tony Liu 30 Nov '21

30 Nov '21

Hi, With Ussuri, when launch VM, I see this from nova-compute. ======== Received unexpected event network-vif-plugged-32b7af8a-ef81-4785-bd79-3a5619823638 ======== That caused VM to be destroyed, rescheduled and relaunched. And that recreation worked fine. >From user POV, it just takes a bit longer to launch the VM. What's the event nova-compute expects from Neutron for vif-plug? Any hints where I can look into it? More logs here. ======== 2021-11-29 18:28:10.064 7 INFO os_vif [req-2a89f605-6972-41dc-83f1-1ef6a444e486 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f1:94:a0,bridge_name='br-int',has_traffic_filtering=True,id=32b7af8a-ef81-4785-bd79-3a5619823638,network=Network(27ea27c6-3d1e-4d37-ad4a-de2ae1578f1e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap32b7af8a-ef') 2021-11-29 18:28:10.515 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384-1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] VM Started (Lifecycle Event) 2021-11-29 18:28:10.544 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384 -1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] VM Paused (Lifecycle Event) 2021-11-29 18:28:10.604 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384 -1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] During sync_power_state the instance has a pending task (spawning). Skip. 2021-11-29 18:28:25.441 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384 -1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] VM Res umed (Lifecycle Event) 2021-11-29 18:28:25.445 7 INFO nova.virt.libvirt.driver [-] [instance: a8eb2a2a- 8c83-46b2-bfd4-fd3a7d00f2ce] Instance spawned successfully. 2021-11-29 18:28:25.445 7 INFO nova.compute.manager [req-2a89f605-6972-41dc-83f1-1ef6a444e486 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] Took 18.75 seconds to spawn the instance on the hypervisor. 2021-11-29 18:28:25.502 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384-1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] During sync_power_state the instance has a pending task (spawning). Skip. 2021-11-29 18:28:25.514 7 INFO nova.compute.manager [req-2a89f605-6972-41dc-83f1-1ef6a444e486 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] Took 19.46 seconds to build instance. 2021-11-29 18:28:27.466 7 WARNING nova.compute.manager [req-82314965-fe71-4993-bbf6-f0aa9e708529 f0c1375066214e24ab7942d72d829097 3ec2940fc10d4e57bcf929d1fe678c79 - default default] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] Received unexpected event network-vif-plugged-32b7af8a-ef81-4785-bd79-3a5619823638 for instance with vm_state active and task_state None. 2021-11-29 18:28:55.829 7 INFO nova.compute.manager [req-e36e4a81-4755-4d7f-9ee4-d43d00629765 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] Terminating instance ======== Thanks! Tony

2 1

[neutron] CI meeting 30.11.2021 - cancelled
by Slawek Kaplonski 29 Nov '21

29 Nov '21

Hi, Due to some personal reasons I will not be able to drive today's Neutron CI meeting. Let's cancel it this week. Remember, next week we will have CI meeting on Jitsi and we will discuss all ideas about how to improve stability of our CI. -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[Neutron][OVN] AttributeError: 'NoneType' object has no attribute 'get_lrouter'
by Tony Liu 29 Nov '21

29 Nov '21

Hi, It happens from time to time, not consistently, with the latest kolla/centos-binary-neutron-server:ussuri container image. >From the bottome of traceback, it seems self._ovn is None. Any clues how this can happen? I don't see other suspicious logging in Neutron and OVN. ===================== File "/usr/lib/python3.6/site-packages/neutron/services/ovn_l3/plugin.py", line 477, in get_router_availability_zones ERROR neutron.api.v2.resource lr = self._ovn.get_lrouter(router['id']) ERROR neutron.api.v2.resource AttributeError: 'NoneType' object has no attribute 'get_lrouter' ===================== Here is the full traceback. ===================== neutron.api.v2.resource [req-bf2230aa-0491-4caa-b5f7-169d37009672 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] index failed: No details.: AttributeError: 'NoneType' object has no attribute 'get_lrouter' neutron.api.v2.resource Traceback (most recent call last): neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/api/v2/resource.py", line 98, in resource neutron.api.v2.resource result = method(request=request, **args) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped neutron.api.v2.resource return f(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper neutron.api.v2.resource ectxt.value = e.inner_exc neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper neutron.api.v2.resource return f(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped neutron.api.v2.resource LOG.debug("Retry wrapper got retriable exception: %s", e) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped neutron.api.v2.resource return f(*dup_args, **dup_kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 369, in index neutron.api.v2.resource return self._items(request, True, parent_id) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 304, in _items neutron.api.v2.resource obj_list = obj_getter(request.context, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped neutron.api.v2.resource return method(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped neutron.api.v2.resource return f(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper neutron.api.v2.resource ectxt.value = e.inner_exc neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper neutron.api.v2.resource return f(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped neutron.api.v2.resource LOG.debug("Retry wrapper got retriable exception: %s", e) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped neutron.api.v2.resource return f(*dup_args, **dup_kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/db/l3_db.py", line 547, in get_routers neutron.api.v2.resource page_reverse=page_reverse) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/model_query.py", line 317, in get_collection neutron.api.v2.resource for c in query neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/model_query.py", line 317, in <listcomp> neutron.api.v2.resource for c in query neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/db/l3_db.py", line 221, in _make_router_dict neutron.api.v2.resource resource_extend.apply_funcs(l3_apidef.ROUTERS, res, router) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/resource_extend.py", line 84, in apply_funcs neutron.api.v2.resource resolved_func(response, db_object) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/db/availability_zone/router.py", line 40, in _add_az_to_response neutron.api.v2.resource l3_plugin.get_router_availability_zones(router_db)) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/services/ovn_l3/plugin.py", line 477, in get_router_availability_zones neutron.api.v2.resource lr = self._ovn.get_lrouter(router['id']) neutron.api.v2.resource AttributeError: 'NoneType' object has no attribute 'get_lrouter' neutron.api.v2.resource neutron.wsgi [req-bf2230aa-0491-4caa-b5f7-169d37009672 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] 10.91.1.7,10.250.20.26 "GET /v2.0/routers HTTP/1.1" status: 500 len: 344 time: 0.2660162 ===================== Thanks! Tony

1 0

RE: Nova Local Disk Storage (Seena Fallah)
by Steven Relf 29 Nov '21

29 Nov '21

Hi Seena, Do you mean passing an entire disk to an instance, or moving where nova stores its ephemeral disks? (e.g. /var/lib/nova/instances?) Rgds Steve. The future has already arrived. It's just not evenly distributed yet - William Gibson Hi, Is there any support for creating instances on a whole separate disk, not an LVM?

1 0

[glance][image-import] local disk space required by image import and conversion
by Tony Liu 29 Nov '21

29 Nov '21

Hi, The target is for user to import .qcow2 image from download site and automatically stored as raw image. Read through [1], this seems doable. I am not sure the workflow. Does it download the whole qcow2 and convert it to raw locally, or download, convert and upload in a streaming way, which only require small space for caching? The question is that, how much local disk space is required for such import and conversion? [1] https://docs.openstack.org/glance/latest/admin/interoperable-image-import.h… Thanks! Tony

2 2

[tripleo] puppet_config: immutable relationship between config_volume and config_image?
by Brent Eagles 29 Nov '21

29 Nov '21

Hi all, I've been working on https://review.opendev.org/c/openstack/tripleo-heat-templates/+/816895 and the current pep8 check error on the designate-api-container-puppet.yaml template appears to disallow using different container config images on the same config volume. I see where this makes sense but because the way the puppet_config sections seem to be structured I wonder if it was intentional. The templates happen to work in this patch's case because the api specific config image in this template only includes some additional binaries for apache+wsgi necessary for the puppet. However, I can see where this might cause some really unfortunate issues if the config images for a given config_volume were to have different versions of puppet. While I will alter the templates so the designate config volume use the same config image, it does beg the question if having the puppet_config config_volume definitions duplicated across the templates is a good pattern. Would it be a better choice to adopt a pattern where the immutable parts of the config_volumes (e.g. config_volume name, config_image) can only be defined once per config volume? Perhaps in a separate file and have the mutable parts in their respective template definitions? Cheer, Brent -- Brent Eagles Principal Software Engineer Red Hat Inc.

2 2

[ironic] End of year meeting schedule
by Julia Kreger 29 Nov '21

29 Nov '21

Greetings everyone, Today, during the Ironic team meeting, we reached consensus[0] that we would not hold meetings during the last half of December and for the first week of January. Much as we've done the past few years to give contributors time they can feel safe taking off without missing any meetings, coupled with end of year PTO usage and holidays. We will resume our regular meetings on January 10th, 2022. Dates of cancelled meetings: * December 20th, 2021 * December 27th, 2021 * January 3rd, 2022 Hopefully everyone will get plenty of rest, and be ready to review plenty of code when they get back! Thanks, -Julia [0]: https://meetings.opendev.org/meetings/ironic/2021/ironic.2021-11-29-15.00.l…

1 0

[neutron][nova] [kolla] vif plugged timeout
by Michal Arbet 29 Nov '21

29 Nov '21

Hi, Has anyone seen issue which I am currently facing ? When launching heat stack ( but it's same if I launch several of instances ) vif plugged in timeouts an I don't know why, sometimes it is OK ..sometimes is failing. Sometimes neutron reports vif plugged in < 10 sec ( test env ) sometimes it's 100 and more seconds, it seems there is some race condition but I can't find out where the problem is. But on the end every instance is spawned ok (retry mechanism worked). Another finding is that it has to do something with security group, if noop driver is used ..everything is working good. Firewall security setup is openvswitch . Test env is wallaby. I will attach some logs when I will be near PC .. Thank you, Michal Arbet (Kevko)

8 12

[wallaby][ovn][Open vSwitch] HA for OVN DB servers using pacemaker
by Faisal Sheikh 29 Nov '21

29 Nov '21

Hi, I am using Openstack Wallaby release with OVN on Ubuntu 20.04. My environment consists of 2 compute nodes and 1 controller node. ovs-vswitchd (Open vSwitch) 2.15.0 Ubuntu Kernel Version: 5.4.0-88-generic compute node1 172.16.30.1 compute node2 172.16.30.3 controller/Network node/Primary ovs-db IP 172.16.30.46 backup ovs-db 172.16.30.47 I added another node for backup ovs-db server (172.16.30.47 ) and installed openvswitch-ovn and networking-ovn packages on it for backup ovs-db server and using Pacemaker to manage the ovn-northd(Open vSwitch) service between primary and backup ovs-db server active/passive HA mode. Pacemaker cluster is successful and after creating a pacemaker cluster, I am using the following commands to create one active and one backup server for OVN databases. $ pcs resource create ovndb_servers ocf:ovn:ovndb-servers \ master_ip=172.16.30.46 \ ovn_ctl=<path of the ovn-ctl script> \ op monitor interval="10s" \ op monitor role=Master interval="15s" But i am getting below error: Error: Agent 'ocf:ovn:ovndb-servers' is not installed or does not provide valid metadata: Metadata query for ocf:ovn:ovndb-servers failed: Input/output error, use --force to override I would really appreciate any input in this regard. Best regards, Faisal Sheikh

1 0

Regarding Drop down of frequently used stacks in OpenStack Wallaby Version 23.1.2
by Adivya Singh 29 Nov '21

29 Nov '21

Dear Team, I want a Drop down of frequently used stack in my orchestration --> Launch Stack parametre in openstack version given above, Is there any Solution for this, How should i go ahead with this Regards Adivya Singh

2 2

[wallaby][ovn][Open vSwitch] HA for OVN DB servers using pacemaker
by Faisal Sheikh 29 Nov '21

29 Nov '21

Hi, I am using Openstack Wallaby release with OVN on Ubuntu 20.04. My environment consists of 2 compute nodes and 1 controller node. ovs-vswitchd (Open vSwitch) 2.15.0 Ubuntu Kernel Version: 5.4.0-88-generic compute node1 172.16.30.1 compute node2 172.16.30.3 controller/Network node/Primary ovs-db IP 172.16.30.46 backup ovs-db 172.16.30.47 I added another node for backup ovs-db server (172.16.30.47 ) and installed openvswitch-ovn and networking-ovn packages on it for backup ovs-db server and using Pacemaker to manage the ovn-northd(Open vSwitch) service between primary and backup ovs-db server active/passive HA mode. Pacemaker cluster is successful and after creating a pacemaker cluster, I am using the following commands to create one active and one backup server for OVN databases. $ pcs resource create ovndb_servers ocf:ovn:ovndb-servers \ master_ip=172.16.30.46 \ ovn_ctl=<path of the ovn-ctl script> \ op monitor interval="10s" \ op monitor role=Master interval="15s" But i am getting below error: Error: Agent 'ocf:ovn:ovndb-servers' is not installed or does not provide valid metadata: Metadata query for ocf:ovn:ovndb-servers failed: Input/output error, use --force to override I would really appreciate any input in this regard. Best regards, Faisal Sheikh

1 0

[neutron]Some floating IPs inaccessible after restart of L3 agent
by Kamil Madáč 29 Nov '21

29 Nov '21

Hello Everyone, We have openstack Victoria deployed since the beginning of the year with kolla/ansible in docker containers. Everything was running OK, but few weeks ago we noticed issues with networking. Our installation uses Openvswitch networking with DVR non HA routers. Everything is running smoothly until we restart L3 agent. After that, some floating ips of VMs running on the node where L3 agent is running becomes inaccessible. Workaround is to reassign floating IP to affected VM. Every restart affects same floating IPs and VMs. No errors/excpetions found in logs. I was able to find out that after restart there are missing routes for those particular floating IPs in fip- namespace, which causes that proxy arp responses are not working. After floating IP address is reassigned, routes are added by L3 agent and floating IP is working again. Looks like some sort of race condition in L3 agent, but I was not able to identify any possible existing bug. L3 agent is in version 17.0.1.dev44. Is anyone aware of any existing bug which could explain such behavior, or does anyone have idea how to solve the issue? Kamil Madáč Slovensko IT a.s.

2 3

[Image Encryption] No meeting today
by Josephine Seifert 29 Nov '21

29 Nov '21

Hi, due to a conflicting meeting today I will not be able to hold the popup team meeting. Next meeting will be next week. greetings Josphine (Luzi)

1 0

[neutron] Bug deputy, 22Nov - 28Nov
by Rodolfo Alonso Hernandez 29 Nov '21

29 Nov '21

Hello Neutrinos: This is the bug report from last week. Critical: https://bugs.launchpad.net/neutron/+bug/1952004: “ovn: Functional test may fail if northd hasn't initialized DBs yet”. Assigned. Patch: https://review.opendev.org/c/openstack/neutron/+/819032 https://bugs.launchpad.net/neutron/+bug/1952023: “Neutron functional tests don't properly clean up ovn-northd”. Assigned. Patch: https://review.opendev.org/c/openstack/neutron/+/819049 <https://bugs.launchpad.net/neutron/+bug/1952066> https://bugs.launchpad.net/neutron/+bug/1952066: “Scenario test test_mac_learning_vms_on_same_network fails intermittently in the ovn job”. Assigned. https://bugs.launchpad.net/neutron/+bug/1952357: “Functional tests job in the ovn-octavia-provider is broken“. Assigned. Patch: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/819377 <https://bugs.launchpad.net/neutron/+bug/1952393> https://bugs.launchpad.net/neutron/+bug/1952393: “[OVN] neutron-tempest-plugin-scenario-ovn broken with "ovn-northd did not start"“. Assigned. Patch: https://review.opendev.org/c/openstack/devstack/+/819402 <https://bugs.launchpad.net/neutron/+bug/1952508> https://bugs.launchpad.net/neutron/+bug/1952508: “[OVN] "TestAgentMonitor.test_agent_change_controller" failing randomly“. Assigned. Patch: https://review.opendev.org/c/openstack/neutron/+/819502 High: https://bugs.launchpad.net/neutron/+bug/1952567: “[ml2][ovs] ports tag are missing and flood on those”. Assigned. Patch: https://review.opendev.org/c/openstack/neutron/+/819567 <https://bugs.launchpad.net/neutron/+bug/1952550> https://bugs.launchpad.net/neutron/+bug/1952550: “[OVN] neutron_ovn_metadata_agent retrying on UUID errors infinitely”. *Unassigned* . Medium: https://bugs.launchpad.net/neutron/+bug/1951816: “[OVN] setting a IPv6 address in dns_servers is broken”. *Unassigned*. https://bugs.launchpad.net/neutron/+bug/1951872: “OVN: Missing reverse DNS for instances”. *Unassigned*. https://bugs.launchpad.net/neutron/+bug/1952055: “native firewall driver - conntrack marks too much traffic as invalid“. *Unassigned*. Low: https://bugs.launchpad.net/neutron/+bug/1951569: “[L3] L3 agent extension should always inherit from "L3AgentExtension"”. Assigned. Patch: https://review.opendev.org/c/openstack/neutron/+/818540 <https://bugs.launchpad.net/neutron/+bug/1952409> https://bugs.launchpad.net/neutron/+bug/1952409: “SQLAlchemy "ConnectionEvents.before_execute" signature has changed in 1.4“. Assigned. Patch: https://review.opendev.org/c/openstack/neutron/+/819420 Undecided/Opinion/Invalid: https://bugs.launchpad.net/neutron/+bug/1951720: "Virtual interface creation failed". More information is needed to debug this issue. https://bugs.launchpad.net/neutron/+bug/1952249: "[OVN] Created VM name overrides predefined port dns-name". The extension works as expected. Regards.

1 0

[all][tc] What's happening in Technical Committee: summary 26th Nov, 21: Reading: 10 min
by Ghanshyam Mann 26 Nov '21

26 Nov '21

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * Most of the meeting discussions are summarized below (Completed or in-progress activities section). Meeting full logs are available @ https://meetings.opendev.org/meetings/tc/2021/tc.2021-11-25-15.00.log.html * Next week's meeting is video call on 2nd Dec, Thursday 15:00 UTC, feel free the topic on the agenda[1] by 1st Dec. 2. What we completed this week: ========================= * Add ProxySQL repository for OpenStack-Ansible[2] * Mark 'Technical Writing' SIG 'complete' as it is merged into TC [3] * Rename ‘Extended Maintenance’ SIG to the ‘Stable Maintenance’ [4] * Retire training-labs repo [5] 3. Activities In progress: ================== TC Tracker for Yoga cycle ------------------------------ * This etherpad includes the Yoga cycle targets/working items for TC[6]. Open Reviews ----------------- * 8 open reviews for ongoing activities[7]. Yoga testing runtime ------------------------- We updated the Yoga testing runtime to move from CentOS stream 8 to CentOS stream 9 and so drop the py3.6[8]. But while updating projects to drop py3.6, and discussion on whether we should change 'python_requires'>= 3.8 in setup.cfg, another (but good) discussion started on ML about whether we should continue testing py3.6 and support CentOS stream 8 and 9 both in yoga which is what RDO community plan is[8]. After discussing on ML and then with Alfredo Moralejo on TC IRC channel, we have a potential way forward on this which I have summarized on ML thread[9]. I will keep that open to get feedback or any objection to the proposed plan and accordingly we will conclude it early next week. I hope I have summarized the discussion here if not please read the ML thread for this. Fixing Zuul config error -------------------------- If your project is listed in zuul config error, please start planing to fix those[10] RBAC discussion: continuing from PTG ---------------------------------------------- After last week meeting discussion, we are almost ready with this goal, if you would like to review the current direction as well as the schedule, please review it[11]. Next plan is to merge the proposed goal and then select it ASAP as m-1 is already passed for Yoga cycle. Community-wide goal updates ------------------------------------ * Two goals are proposed currently and under review. ** RBAC goal is pretty much in good shape now, feel free to review it[11] ** 'FIPS compatibility and compliance' [12]. Adjutant need maintainers and PTLs ------------------------------------------- We are waiting to hear from Braden, Albert on permission to work in this project[13]. New project 'Skyline' proposal ------------------------------------ * TC is still waiting for skyline team to respond on open queries[14]. TC tags analysis ------------------- * TC agreed to remove the framework and it is communicated in ML[15]. Project updates ------------------- * Add ansible-collection-kolla repo to Kolla project[16] * Retire js-openstack-lib [17] 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[18]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [19] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/817245 [3] https://review.opendev.org/c/openstack/governance-sigs/+/815868 [4] https://review.opendev.org/c/openstack/governance-sigs/+/817499 [5] https://review.opendev.org/c/openstack/governance/+/817511 [6] https://etherpad.opendev.org/p/tc-yoga-tracker [7] https://review.opendev.org/q/projects:openstack/governance+status:open [8] http://lists.openstack.org/pipermail/openstack-discuss/2021-November/026001… [9] http://lists.openstack.org/pipermail/openstack-discuss/2021-November/026024… [10] https://etherpad.opendev.org/p/zuul-config-error-openstack [11] https://review.opendev.org/c/openstack/governance/+/815158 [12] https://review.opendev.org/c/openstack/governance/+/816587 [13] http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025786… [14] https://review.opendev.org/c/openstack/governance/+/814037 [15] http://lists.openstack.org/pipermail/openstack-discuss/2021-October/025571.… [16] https://review.opendev.org/c/openstack/governance/+/819331 [17] https://review.opendev.org/c/openstack/governance/+/807163 [18] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [19] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann

1 0

Project-scoped app creds - Best practice
by Ryan Bannon 26 Nov '21

26 Nov '21

Hello all, Relatively new to OpenStack. To my understanding, application credentials are bound to users. Is there a way to bind them to Projects (I assume not) or, perhaps, Groups? My naive thought on a possible solution is that if a group has access to a Project, a "generic" user account that everybody has access to could be used for the application credentials. (The use case here is to not bind an app cred to an individual who might leave the organization, thus making the app cred secret lost.) Thanks, Ryan

2 5

Nova Local Disk Storage
by Seena Fallah 26 Nov '21

26 Nov '21

Hi, Is there any support for creating instances on a whole separate disk, not an LVM?

1 0

[Openstack-victoria] [IRONIC] error in network interface | baremetal node validate
by Swogat Pradhan 25 Nov '21

25 Nov '21

Hi, I am currently trying to setup openstack ironic using driver IPMI. I followed the official docs of openstack for setting everything up. When i run openstack baremetal node validate $NODE_UUID, i am getting the following error: * Unexpected exception, traceback saved into log by ironic conductor service that is running on controller: 'ServiceTokenAuthWrapper' object has no attribute '_discovery_cache' * in the network interface in command output. When i check the ironic conductor logs i see the following messages: <attached <https://mail.google.com/mail/u/0?ui=2&ik=7bb3543b01&attid=0.1&permmsgid=msg…> > Can anyone suggest a solution or a way forward. With regards Swogat Pradhan

2 4

[aodh][tacker] Support for pyparsing v 3.0.6 is required in python-aodhclient
by ueha.ayumu＠fujitsu.com 25 Nov '21

25 Nov '21

Hi aodh team, Hi I'm Ueha from tacker team. Tacker CI fails with the following error since a few days ago: ---from h-eng log--- Traceback (most recent call last): ... File "/usr/local/lib/python3.8/dist-packages/aodhclient/utils.py", line 43, in <module> expr = pp.operatorPrecedence(condition, [ AttributeError: module 'pyparsing' has no attribute 'operatorPrecedence' -------------------- As I have researched, the version of pyparsing has been updated to v3.0.6 in global requirements [1]. And operatorPrecedence was already deprecated in pyparsing v3.0.6. A patch to fix this appears to have already been posted to python-aodhclient [2]. Could you consider merging the fixes to avoid errors in the CI? For your information, it have been merged similar fixes in cinder [3] and manila [4]. [1] https://review.opendev.org/c/openstack/requirements/+/818614 [2] https://review.opendev.org/c/openstack/python-aodhclient/+/819054 [3] https://review.opendev.org/c/openstack/cinder/+/818834 [4] https://review.opendev.org/c/openstack/manila/+/818829 Regards, Ueha

1 0

[puppet] Propose retiring old stable branches(Stein, Rocky and Queens)
by Takashi Kajinami 25 Nov '21

25 Nov '21

Hello, In puppet repos we have plenty of stable branches still open, and the oldest is now stable/queens. However recently we haven't seen many backports proposed to stein, rocky and queens [1], so I'll propose retiring these three old branches now. Please let me know if anybody is interested in keeping any of these three. Note that currently CI jobs are broken in these three branches and it is likely we need to investigate the required additional pinning of dependent packages. If somebody is still interested in maintaining these old branches then these jobs should be fixed. [1] https://review.opendev.org/q/(project:%255Eopenstack/puppet-.*)AND(NOT+proj… Thank you, Takashi Kajinami

2 2

[neutron] Drivers meeting - Friday 26.11.2021 - cancelled
by Lajos Katona 25 Nov '21

25 Nov '21

Hi Neutron Drivers! Due to the lack of agenda, let's cancel tomorrow's drivers meeting. See You on the meeting next week. Lajos Katona (lajoskatona)

1 0

[release] Release countdown for week R-18, Nov 22-26
by Előd Illés 25 Nov '21

25 Nov '21

Development Focus ---------------------------- We are now past the Yoga-1 milestone. Teams should currentlybe focused on feature development! General Information ---------------------------- Our next milestone in this development cycle will be Yoga-2, on 06 January, 2022. This milestone is when we freeze the list of deliverables that will be included in the Yogafinal release, so if you plan to introduce new deliverables in this release, please propose a change to add an empty deliverable file in the deliverables/yogadirectory of the openstack/releases repository. Now is also generally a good time to look at bugfixes that were introduced in the master branch that might make sense to be backported and released in a stable release. If you have any question around the OpenStack release process, feel free to ask on this mailing-list or on the #openstack-release channel on IRC. Upcoming Deadlines & Dates ----------------------------------------- Yoga-2 Milestone:06 January, 2022 Yoga final release: March 30, 2022 Előd Illés irc: elodilles

1 0

[kolla] Proposing Michal Arbet as Kolla core
by Michał Nasiadka 25 Nov '21

25 Nov '21

Hi, I would like to propose adding Michal Arbet (kevko) to the kolla-core and kolla-ansible-core groups. Michal did some great work on ProxySQL, is a consistent maintainer of Debian related images and has provided some helpful reviews. Cores - please reply +1/-1 before the end of Friday 26th November. Thanks, Michal

5 5

[neutron] Averaga number of rechecks in Neutron comparing to other projects
by Slawek Kaplonski 25 Nov '21

25 Nov '21

Hi neutrinos, As we discussed during the last PTG, I spent some time today to get average number of rechecks which we need to do in the last PS of the change before it's merged. In theory that number should be close to 0 as patch should be merged with first CI run when it's approved by reviewers :) All data are only from the master branch. I didn't check the same for stable branches. File with graph and raw data in csv format are in the attachments. Basically my conclusion is that Neutron's CI is really bad in that. We have to recheck many, many times before patches will be merged. We really need to think about how to improve that in Neutron. -- Slawek Kaplonski Principal Software Engineer Red Hat

2 2

[Triple 0] Undercloud deployment Getting failed
by Lokendra Rathour 25 Nov '21

25 Nov '21

Hi Team getting strange error when installing triple O Train on Centos 8.4 '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--rm', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/container-puppet-zaqar.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', 'quay.io/tripleotraincentos8/centos-binary-zaqar-wsgi:current-tripleo'] run failed after Error: container_linux.go:370: starting container process caused: error adding seccomp filter rule for syscall bdflush: permission denied: OCI permission denied", " attempt(s): 3", "2021-11-23 11:00:38,384 WARNING: 58791 -- Retrying running container: zaqar", "2021-11-23 11:00:38,384 ERROR: 58791 -- Failed running container for zaqar", "2021-11-23 11:00:38,385 INFO: 58791 -- Finished processing puppet configs for zaqar", "2021-11-23 11:00:38,385 ERROR: 58782 -- ERROR configuring crond", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring glance_api", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring heat_api", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring heat", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring ironic_api", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring ironic", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring ironic_inspector", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring neutron", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring iscsid", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring keystone", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring memcached", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring mistral", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring mysql", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring nova", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring rabbitmq", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring placement", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring swift", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring swift_ringbuilder", "2021-11-23 11:00:38,386 ERROR: 58782 -- ERROR configuring zaqar" ], "stderr_lines": [], "_ansible_no_log": false, "attempts": 15 } ] ] Not cleaning working directory /home/stack/tripleo-heat-installer-templates Not cleaning ansible directory /home/stack/undercloud-ansible-mie5k51_ Install artifact is located at /home/stack/undercloud-install-20211123110040.tar.bzip2 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Deployment Failed! This issue is recurring multiple times please advise. -- ~ Lokendra

2 2

[all][tc] Technical Committee next weekly meeting on Nov 25th at 1500 UTC
by Ghanshyam Mann 24 Nov '21

24 Nov '21

Hello Everyone, Technical Committee's next weekly meeting is scheduled for Nov 25th at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, Nov 24th, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[ops][nova][victoria] QEMU/KVM libvirt Authentication Failed
by DHilsbos＠performair.com 24 Nov '21

24 Nov '21

All; I feel like I've dealt with this issue before, but I can't find any records of it. I've been swapping out the compute nodes in my cluster for newer and better hardware. We also decided to abandon CentOS. All the differences mean that we haven't been able to do live migrations. I now have 2 servers with the same CPUs, OS (Ubuntu), OS Version (20.10), etc., and would like to get live migration working again. I configured passwordless ssh access between the servers for the nova users to get cold migration working. I have also configured passwordless ssh for the root users in accordance with [1]. When I try to do a live migration, the origin server generates this error, in the nova-compute log: 2021-11-19 15:52:31.130 15610 ERROR nova.virt.libvirt.driver [-] [instance: 5935c07d-0c7f-48cc-a4b9-674504fc6005] Live Migration failure: operation failed: Failed to connect to remote libvirt URI qemu+tcp://<remote_server>/system: authentication failed: authentication failed: libvirt.libvirtError: operation failed: Failed to connect to remote libvirt URI qemu+tcp://<remote_server>/system: authentication failed: authentication failed At one point, I came across a tutorial on configuring live-migration for libvirt, which included a bunch of user configuration. I don't remember having to do that before, but is that what I need to be looking for? Thank you, Dominic L. Hilsbos, MBA Vice President - Information Technology Perform Air International Inc. DHilsbos(a)PerformAir.com www.PerformAir.com 1: https://docs.openstack.org/nova/victoria/admin/configuring-migrations.html#…

3 6

[all][tc] Yoga testing runtime update. Action needed if py3.9 job failing for your project
by Ghanshyam Mann 24 Nov '21

24 Nov '21

Hello Everyone. As discussed in TC PTG, we have updated the testing runtime for the Yoga cycle [1]. Changes are: * Add Debian 11 as the tested distro * Change centos stream 8 -> centos stream 9 * Bump lowest python version to test to 3.8 and highest to python 3.9 ** This removes python 3.6 from testing. I pushed the job template update[2] which will make py3.9 unit test job voting (which is non-voting currently). I do not see any projects failing consistently on py3.9[3] but still, I will keep it as -W until early next week (23rd Nov). If any project needs time to fix the failing py3.9 job, please do it before 23rd Nov. [1] https://governance.openstack.org/tc/reference/runtimes/yoga.html [2] https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/818609 [3] https://zuul.openstack.org/builds?job_name=openstack-tox-py39&branch=master… -gmann

1 1

[keystone][policy][ussuri] why I can create a domain
by Piotr Misiak 24 Nov '21

24 Nov '21

Hi, Maybe a stupid question but I'm really confused. In my Ussuri cloud Keystone has a following policy for create_domain action (this is a default policy from Keystone code): "identity:create_domain": "role:admin and system_scope:all" I have a user which has "admin" role assigned in project "admin" in domain "default" - AKA cloud admin. The user does not have any roles assigned on system scope. Could someone please explain why this user is able to create a domain in the cloud? Looking at the policy rule he shouldn't or maybe I'm reading it in a wrong way? Is there any "backward compatibility" casting "cloud admin" role to "system_scope:all"? Please help Thanks Piotr

2 1

[largescale-sig] Next meeting: Nov 24th, 15utc
by Thierry Carrez 24 Nov '21

24 Nov '21

Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 15UTC. You can doublecheck how that time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20211124T15 We will be discussing our upcoming OpenInfra Live! episode on Dec 9. Feel free to add topics to our agenda at: https://etherpad.openstack.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

1 1

[all][sphinx] "rST localisation for language "foo" not found" error
by Stephen Finucane 24 Nov '21

24 Nov '21

We've seen issues in some projects, whereby attempting to build translations as part of the 'openstack-tox-docs' zuul job [1] can result in the following warning: Warning, treated as error: ../doc/source/contributor/contributing.rst::rST localisation for language "id" not found. Based on [2], this issue appears to have been introduced by a recent version bump of docutils, which was in turn introduced by the introduction of Sphinx 4.x. We don't have a resolution for this bug yet but we'll update when we do. Hopefully this email will improve the "Google'ability" of this bug. Stephen [1] https://opendev.org/openstack/openstackdocstheme/src/commit/08461c5311aa692… [2] https://lists.freebsd.org/pipermail/freebsd-ports/2021-April/120848.html

2 2

[cinder] Bug deputy report for week of 11-24-2021
by Sofia Enriquez 24 Nov '21

24 Nov '21

This is a bug report from 11-17-2021 to 11-24-2021. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Medium The next 3 bugs are related to multipath config everywhere <https://review.opendev.org/c/openstack/cinder/+/818994>: - https://bugs.launchpad.net/cinder/+bug/1951982 "On encrypted volume cloning rekeying doesn't use multipathing". Assigned to Gorka Eguileor. - https://bugs.launchpad.net/cinder/+bug/1951981 "Kaminario not using multipath for creating volume". Assigned to Gorka Eguileor. - https://bugs.launchpad.net/cinder/+bug/1951977 "Cinder backup - The multipath flag is not set on a create_backup operation". Assigned to Gorka Eguileor. Low - https://bugs.launchpad.net/cinder/+bug/1951250 "[Storwize] Fix multiple lsiogrp,lsvdisk calls in Retype". Unassigned. Cheers, -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com> -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

[kolla] Transition kolla/kolla-ansible Rocky/Stein to EOL
by Michał Nasiadka 24 Nov '21

24 Nov '21

Hello, We've kept stable/rocky and stable/stein branches of kolla and kolla-ansible around under extended maintenance for a while now. The coordinated Rocky release was 30th Aug, 2018 (Kolla on 22nd Oct, 2018) and Stein was 10th Apr, 2019 (Kolla on 17th Jul, 2019). The last time a change was backported to these branches has been a while: - kolla stable/rocky: Sep 4, 2020 stable/stein: Mar 19, 2021 - kolla-ansible stable/rocky: Sep 13, 2020 stable/stein: May 28,2021 There is no CI running for those projects periodically, last CI jobs have been run: - kolla: stable/rocky: Jul 8th 2019 stable/stein: Dec 10th 2020 - kolla-ansible stable/rocky: Sep 14th 2019 stable/stein: Dec 10th 2020 All changes to those branches have been abandoned long time (and nothing new proposed for at least a year): https://review.opendev.org/q/branch:stable/rocky+project:openstack/kolla <https://review.opendev.org/q/branch:stable/rocky+project:openstack/kolla> https://review.opendev.org/q/branch:stable/stein+project:openstack/kolla <https://review.opendev.org/q/branch:stable/stein+project:openstack/kolla> https://review.opendev.org/q/branch:stable/rocky+project:openstack/kolla-an… <https://review.opendev.org/q/branch:stable/rocky+project:openstack/kolla-an…> https://review.opendev.org/q/branch:stable/stein+project:openstack/kolla-an… <https://review.opendev.org/q/branch:stable/stein+project:openstack/kolla-an…> In addition to that - kolla-ansible CI jobs have incorrect Zuul configurations (see http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025797… <http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025797…>) - and on one of the Kolla weekly meetings it was decided to mark those branches as EOL. So, I'd like to request that these branches be moved to end-of-life. Patch proposed to add EOL tags to both kolla and kolla-ansible: https://review.opendev.org/c/openstack/releases/+/818920 <https://review.opendev.org/c/openstack/releases/+/818920> Best regards, Michal

1 0

[kolla][release][Release-job-failures] Release of openstack/kolla for ref refs/tags/10.4.0 failed
by Előd Illés 24 Nov '21

24 Nov '21

Hi Kolla team, The last release of kolla (10.4.0 [1]) had release job failures (see below the attached, forwarded mail). The two job that failed are: - kolla-publish-centos8-source-dockerhub https://zuul.opendev.org/t/openstack/build/feb51c87f74c4dadb4984733d8b3793c : FAILURE in 2h 12m 37s - kolla-publish-centos8-binary-dockerhub https://zuul.opendev.org/t/openstack/build/ebce7219ff4b44368869bc1f3172d688 : FAILURE in 1h 46m 56s (non-voting) Could you please have a look at these? Thanks in advance, Előd (elodilles @ #openstack-release) [1] https://review.opendev.org/c/openstack/releases/+/818882 -------- Forwarded Message -------- Subject: [Release-job-failures] Release of openstack/kolla for ref refs/tags/10.4.0 failed Date: Tue, 23 Nov 2021 17:22:02 +0000 From: zuul(a)openstack.org Reply-To: openstack-discuss(a)lists.openstack.org To: release-job-failures(a)lists.openstack.org Build failed. - openstack-upload-github-mirror https://zuul.opendev.org/t/openstack/build/4834324fec464b309952930a2c221336 : SUCCESS in 1m 45s - release-openstack-python https://zuul.opendev.org/t/openstack/build/e3f27c5f3f9b4b289dc73570243507ff : SUCCESS in 3m 06s - announce-release https://zuul.opendev.org/t/openstack/build/1d759f048a1145af9b253b6e5d69b4f4 : SUCCESS in 4m 23s - propose-update-constraints https://zuul.opendev.org/t/openstack/build/3a600e6a4c494a829bfcd45c91931067 : SUCCESS in 3m 42s - kolla-publish-centos8-source-dockerhub https://zuul.opendev.org/t/openstack/build/feb51c87f74c4dadb4984733d8b3793c : FAILURE in 2h 12m 37s - kolla-publish-centos8-binary-dockerhub https://zuul.opendev.org/t/openstack/build/ebce7219ff4b44368869bc1f3172d688 : FAILURE in 1h 46m 56s (non-voting) - kolla-publish-debian-source-dockerhub https://zuul.opendev.org/t/openstack/build/e076910efb394683b98106d5648c2ab5 : SUCCESS in 1h 54m 47s (non-voting) - kolla-publish-debian-source-aarch64-dockerhub https://zuul.opendev.org/t/openstack/build/e201c6f797ea4248b933df8c53d65fa8 : SUCCESS in 2h 03m 51s (non-voting) - kolla-publish-debian-binary-dockerhub https://zuul.opendev.org/t/openstack/build/d0cf2b3aa48e49d8a5aa46719b84c926 : SUCCESS in 1h 18m 03s (non-voting) - kolla-publish-ubuntu-source-dockerhub https://zuul.opendev.org/t/openstack/build/17644441b45f4964b52a1e26e8ef3f19 : SUCCESS in 2h 01m 47s - kolla-publish-ubuntu-binary-dockerhub https://zuul.opendev.org/t/openstack/build/63d1810841da4d9c96d4755177761f4d : SUCCESS in 1h 34m 09s (non-voting) _______________________________________________ Release-job-failures mailing list Release-job-failures(a)lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/release-job-failures

1 0

[neutron] default QOS on L3 gateway
by Damien Rannou 24 Nov '21

24 Nov '21

Hello We are currently playing with QOS on L3 agent, mostly for SNAT, but it can apply also on FIP. Everything is working properly, but I’m wondering if there is a way to define a « default » QOS that would be applied on Router creation, but also when the user is setting « no_qos » on his router. On a public cloud environnement, we cannot let the customers without any QOS limitation. Thanks ! Damien

3 3

[kolla-ansible][wallaby][magnum][Kubernetes] Cannot auto-scale workers
by wodel youchi 24 Nov '21

24 Nov '21

Hi, I have a new kolla-ansible deployment with wallaby. I have created a kubernetes cluster using calico (flannel didn't work for me). I configured an autoscale test to see if it works. - pods autoscale is working. - worker nodes autoscale is not working. This is my deployment file :*cat php-apache.yaml* apiVersion: apps/v1 kind: Deployment metadata: name: php-apache-deployment spec: selector: matchLabels: app: php-apache replicas: 2 template: metadata: labels: app: php-apache spec: containers: - name: php-apache image: k8s.gcr.io/hpa-example ports: - containerPort: 80 resources: limits: cpu: 500m requests: cpu: 200m --- apiVersion: v1 kind: Service metadata: name: php-apache-service labels: app: php-apache spec: ports: - port: 80 targetPort: 80 protocol: TCP selector: app: php-apache type: LoadBalancer This is my HPA file :*cat php-apache-hpa.yaml* apiVersion: autoscaling/v2beta2 kind: HorizontalPodAutoscaler metadata: name: php-apache-hpa namespace: default labels: service: php-apache-service spec: minReplicas: 2 maxReplicas: 30 scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: php-apache-deployment metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 30 # en pourcentage This is my load program : kubectl run -i --tty load-generator-1 --rm --image=busybox --restart=Never -- /bin/sh -c "while sleep 0.01; do wget -q -O- http://ip_load_balancer; done" Here are the output of my kub cluster before the test : [kube8@cdndeployer ~]$ kubectl get pod NAME READY STATUS RESTARTS AGE php-apache-deployment-5b65bbc75c-95k6k 1/1 Running 0 24m php-apache-deployment-5b65bbc75c-mv5h6 1/1 Running 0 24m [kube8@cdndeployer ~]$ kubectl get hpa NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE php-apache-hpa Deployment/php-apache-deployment *0%/30%* 2 15 2 24m [kube8@cdndeployer ~]$ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.254.0.1 <none> 443/TCP 13h php-apache-service LoadBalancer 10.254.3.54 *xx.xx.xx.213* 80:31763/TCP 25m When I apply the load : - pods autoscale creates new pods, then some of them get in the state of : *pending * [kube8@cdndeployer ~]$ kubectl get hpa NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE php-apache-hpa Deployment/php-apache-deployment *155%/30%* 2 15 4 27m [kube8@cdndeployer ~]$ kubectl get pod NAME READY STATUS RESTARTS AGE load-generator-1 1/1 Running 0 97s load-generator-2 1/1 Running 0 94s php-apache-deployment-5b65bbc75c-95k6k 1/1 Running 0 28m *php-apache-deployment-5b65bbc75c-cjkwk 0/1 Pending 0 33s* *php-apache-deployment-5b65bbc75c-cn5rt 0/1 Pending 0 33s* *php-apache-deployment-5b65bbc75c-cxctx 0/1 Pending 0 48s* php-apache-deployment-5b65bbc75c-fffnc 1/1 Running 0 64s php-apache-deployment-5b65bbc75c-hbfw8 0/1 Pending 0 33s php-apache-deployment-5b65bbc75c-l8496 1/1 Running 0 48s php-apache-deployment-5b65bbc75c-mv5h6 1/1 Running 0 28m php-apache-deployment-5b65bbc75c-qddrb 1/1 Running 0 48s php-apache-deployment-5b65bbc75c-dd5r5 0/1 Pending 0 48s php-apache-deployment-5b65bbc75c-tr65j 1/1 Running 0 64s 2 - The cluster is unable to create more pods/workers and I get this error message from the pending pods kubectl describe pod php-apache-deployment-5b65bbc75c-dd5r5 Name: php-apache-deployment-5b65bbc75c-dd5r5 Namespace: default Priority: 0 Node: <none> Labels: app=php-apache pod-template-hash=5b65bbc75c Annotations: kubernetes.io/psp: magnum.privileged *Status: Pending* IP: IPs: <none> Controlled By: ReplicaSet/php-apache-deployment-5b65bbc75c Containers: php-apache: Image: k8s.gcr.io/hpa-example Port: 80/TCP Host Port: 0/TCP Limits: cpu: 500m Requests: cpu: 200m Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from default-token-4fsgh (ro) Conditions: Type Status PodScheduled False Volumes: default-token-4fsgh: Type: Secret (a volume populated by a Secret) SecretName: default-token-4fsgh Optional: false QoS Class: Burstable Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s node.kubernetes.io/unreachable:NoExecute for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- * Warning FailedScheduling 2m48s default-scheduler 0/4 nodes are available: 2 Insufficient cpu, 2 node(s) had taint {node-role.kubernetes.io/master <http://node-role.kubernetes.io/master>: }, that the pod didn't tolerate.* * Warning FailedScheduling 2m48s default-scheduler 0/4 nodes are available: 2 Insufficient cpu, 2 node(s) had taint {node-role.kubernetes.io/master <http://node-role.kubernetes.io/master>: }, that the pod didn't tolerate.* * Normal NotTriggerScaleUp 2m42s cluster-autoscaler pod didn't trigger scale-up (it wouldn't fit if a new node is added): 1 in backoff after failed scale-u**p* I have this error message from the autoscaller pod *cluster-autoscaler*-f4bd5f674-b9692 : I1123 00:50:27.714801 1 node_instances_cache.go:168] Refresh cloud provider node instances cache finished, refresh took 12.709µs I1123 00:51:34.181145 1 scale_up.go:658] Scale-up: setting group default-worker size to 3 *W1123 00:51:34.381953 1 clusterstate.go:281] Disabling scale-up for node group default-worker until 2021-11-23 00:56:34.180840351 +0000 UTC m=+47174.376164120; errorClass=Other; errorCode=cloudProviderError* *E1123 00:51:34.382081 1 static_autoscaler.go:415] Failed to scale up: failed to increase node group size: could not check current nodegroup size: could not get cluster: Get https://dash.cdn.domaine.tld:9511/v1/clusters/b4a6b3eb-fcf3-416f-b740-11a08… <https://dash.cdn.cerist.dz:9511/v1/clusters/b4a6b3eb-fcf3-416f-b740-11a083d…>: dial tcp: lookup dash.cdn.domaine. <http://dash.cdn.cerist.dz>tld on 10.254.0.10:53 <http://10.254.0.10:53>: no such host* W1123 00:51:44.392523 1 scale_up.go:383] Node group default-worker is not ready for scaleup - backoff W1123 00:51:54.410273 1 scale_up.go:383] Node group default-worker is not ready for scaleup - backoff W1123 00:52:04.422128 1 scale_up.go:383] Node group default-worker is not ready for scaleup - backoff W1123 00:52:14.434278 1 scale_up.go:383] Node group default-worker is not ready for scaleup - backoff W1123 00:52:24.442480 1 scale_up.go:383] Node group default-worker is not ready for scaleup - backoff I1123 00:52:27.715019 1 node_instances_cache.go:156] Start refreshing cloud provider node instances cache I did some tests on the DNS pod and : kubectl get svc -A NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default kubernetes ClusterIP 10.254.0.1 <none> 443/TCP 13h default php-apache-service LoadBalancer 10.254.3.54 xx.xx.xx.213 80:31763/TCP 19m kube-system dashboard-metrics-scraper ClusterIP 10.254.19.191 <none> 8000/TCP 13h *kube-system kube-dns ClusterIP 10.254.0.10 <none> 53/UDP,53/TCP,9153/TCP 13h* kube-system kubernetes-dashboard ClusterIP 10.254.132.17 <none> 443/TCP 13h kube-system magnum-metrics-server ClusterIP 10.254.235.147 <none> 443/TCP 13h I have noticed this behaviour about the horizon url, sometimes the dns pod responds sometimes it does not !!!!! [root@k8multiclustercalico-ve5t6uuoo245-master-0 ~]# *dig @10.254.0.10 <http://10.254.0.10> dash.cdn. <http://dash.cdn.cerist.dz>domaine.tld* ; <<>> DiG 9.11.28-RedHat-9.11.28-1.fc33 <<>> @10.254.0.10 dash.cdn. <http://dash.cdn.cerist.dz>domaine.tld ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 5646 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;dash.cd <http://dash.cdn.cerist.dz>n.domaine.tld. IN A ;; AUTHORITY SECTION: *cdn. <http://cdn.cerist.dz>domaine.tld. 30 IN SOA cdn. <http://cdn.cerist.dz>domaine.tld. root.cdn. <http://root.cdn.cerist.dz>domaine.tld. 2021100900 604800 86400 2419200 604800* ;; Query time: 84 msec ;; SERVER: 10.254.0.10#53(10.254.0.10) ;; WHEN: Tue Nov 23 01:08:03 UTC 2021 ;; MSG SIZE rcvd: 12 2 secondes later [root@k8multiclustercalico-ve5t6uuoo245-master-0 ~]# *dig @10.254.0.10 <http://10.254.0.10> dash.cdn. <http://dash.cdn.cerist.dz>domaine.tld* ; <<>> DiG 9.11.28-RedHat-9.11.28-1.fc33 <<>> @10.254.0.10 dash.cdn. <http://dash.cdn.cerist.dz>domaine.tld ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7653 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;dash.cdn. <http://dash.cdn.cerist.dz>domaine.tld. IN A ;; ANSWER SECTION: *dash.cdn. <http://dash.cdn.cerist.dz>domaine.tld. 30 IN A xx.xx.xx.129* ;; Query time: 2 msec ;; SERVER: 10.254.0.10#53(10.254.0.10) ;; WHEN: Tue Nov 23 01:08:21 UTC 2021 ;; MSG SIZE rcvd: 81 In the log of the dns pod I have this kubectl logs *kube-dns-autoscaler*-75859754fd-q8z4w -n kube-system *E1122 20:56:09.944449 1 autoscaler_server.go:120] Update failure: the server could not find the requested resource E1122 20:56:19.945294 1 autoscaler_server.go:120] Update failure: the server could not find the requested resource E1122 20:56:29.944245 1 autoscaler_server.go:120] Update failure: the server could not find the requested resource E1122 20:56:39.946346 1 autoscaler_server.go:120] Update failure: the server could not find the requested resource* *E1122 20:56:49.944693 1 autoscaler_server.go:120] Update failure: the server could not find the requested resource* I don't have experience on kubernetes yet, could someone help me debug this? Regards.

1 0

[horizon] Cancelling Today's weekly meeting
by vishal manchanda 23 Nov '21

23 Nov '21

Hello Team, Since there are no agenda items [1] to discuss for today's horizon weekly meeting. Also, Today is a holiday for me or maybe for others as well So let's cancel today's weekly meeting. Next weekly meeting will be on 01 December. Thanks & Regards, Vishal Manchanda [1] https://etherpad.opendev.org/p/horizon-release-priorities

1 0

[Glance] No weekly meeting
by Abhishek Kekane 23 Nov '21

23 Nov '21

Hello Team, As this is the holiday season and ThanksGiving is on our weekly meeting day, we are not meeting this week. Next meeting will be on 02 December. Thanks & Best Regards, Abhishek Kekane

1 0

[neutron] Neutron OVN+QoS Support
by Erlon Cruz 23 Nov '21

23 Nov '21

4 10

[Neutron][OVN] networking_ovn.common.maintenance.DBInconsistenciesPeriodics.check_for_mcast_flood_reports failed
by Tony Liu 23 Nov '21

23 Nov '21

Hi, Is this known issue? Any filed bug or fix to it? ======================================= 2021-11-23 23:04:20.943 40 ERROR futurist.periodics [-] Failed to call periodic 'networking_ovn.common.maintenance.DBInconsistenciesPeriodics.check_for_mcast_flood_reports' (it runs every 600.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it 2021-11-23 23:04:20.943 40 ERROR futurist.periodics Traceback (most recent call last): 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib/python3.6/site-packages/ovsdbapp/api.py", line 111, in transaction 2021-11-23 23:04:20.943 40 ERROR futurist.periodics yield self._nested_txns_map[cur_thread_id] 2021-11-23 23:04:20.943 40 ERROR futurist.periodics KeyError: 140347734591640 2021-11-23 23:04:20.943 40 ERROR futurist.periodics 2021-11-23 23:04:20.943 40 ERROR futurist.periodics During handling of the above exception, another exception occurred: 2021-11-23 23:04:20.943 40 ERROR futurist.periodics 2021-11-23 23:04:20.943 40 ERROR futurist.periodics Traceback (most recent call last): 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib/python3.6/site-packages/futurist/periodics.py", line 290, in run 2021-11-23 23:04:20.943 40 ERROR futurist.periodics work() 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib/python3.6/site-packages/futurist/periodics.py", line 64, in __call__ 2021-11-23 23:04:20.943 40 ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib/python3.6/site-packages/futurist/periodics.py", line 178, in decorator 2021-11-23 23:04:20.943 40 ERROR futurist.periodics return f(*args, **kwargs) 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib/python3.6/site-packages/networking_ovn/common/maintenance.py", line 667, in check_for_mcast_flood_reports 2021-11-23 23:04:20.943 40 ERROR futurist.periodics for port in self._nb_idl.lsp_list().execute(check_error=True): 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 42, in execute 2021-11-23 23:04:20.943 40 ERROR futurist.periodics t.add(self) 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__ 2021-11-23 23:04:20.943 40 ERROR futurist.periodics next(self.gen) 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib/python3.6/site-packages/networking_ovn/ovsdb/impl_idl_ovn.py", line 183, in transaction 2021-11-23 23:04:20.943 40 ERROR futurist.periodics yield t 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__ 2021-11-23 23:04:20.943 40 ERROR futurist.periodics next(self.gen) 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib/python3.6/site-packages/ovsdbapp/api.py", line 119, in transaction 2021-11-23 23:04:20.943 40 ERROR futurist.periodics del self._nested_txns_map[cur_thread_id] 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib/python3.6/site-packages/ovsdbapp/api.py", line 69, in __exit__ 2021-11-23 23:04:20.943 40 ERROR futurist.periodics self.result = self.commit() 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 62, in commit 2021-11-23 23:04:20.943 40 ERROR futurist.periodics raise result.ex 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 128, in run 2021-11-23 23:04:20.943 40 ERROR futurist.periodics txn.results.put(txn.do_commit()) 2021-11-23 23:04:20.943 40 ERROR futurist.periodics File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 115, in do_commit 2021-11-23 23:04:20.943 40 ERROR futurist.periodics raise RuntimeError(msg) 2021-11-23 23:04:20.943 40 ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it 2021-11-23 23:04:20.943 40 ERROR futurist.periodics ======================================= Thanks! Tony

2 2

[cinder] spec opportunities and deadline
by Brian Rosmaita 23 Nov '21

23 Nov '21

To anyone working on a cinder spec for Yoga: This is a reminder that all Cinder Specs for features to be implemented in Yoga must be approved by Friday 17 December 2021 (23:59 UTC). There are two upcoming opportunities to get feedback on your spec proposal and/or ask questions about your spec from the cinder team: 1. Tomorrow's cinder weekly meeting (Wednesday 24 November, 1400 UTC) is being held in videoconference; if you'd like a discussion, please put a topic on the weekly agenda (which also has connection details): https://etherpad.opendev.org/p/cinder-yoga-meetings 2. The cinder yoga R-17 virtual midcycle meeting is being held next week on Wednesday 1 December 1400-1600 UTC; if you'd like to discuss your proposal, please add it to the planning etherpad: https://etherpad.opendev.org/p/cinder-yoga-midcycles cheers, brian

1 0

[kolla-ansible][wallaby][glance] Problem with image list after reconfigure
by Franck VEDEL 23 Nov '21

23 Nov '21

Hello everyone I have a strange problem and I haven't found the solution yet. Following a certificate update I had to do a "kolla-ansible -t multinode reconfigure ». Well, after several attempts (it is not easy to use certificates with Kolla-ansible, and from my advice, not documented enough for beginners), I have my new functional certificates. Perfect ... well almost. I am trying to create a new instance to check general operation. ERROR. Okay, I look in the logs and I see that Cinder is having problems creating volumes with an error that I never had ("TypeError: 'NoneType' object is not iterable). I dig and then I wonder if it is not the Glance images which cannot be used, while they are present (openstack image list is OK). I create an empty volume: it works. I am creating a volume from an image: Failed. However, I have my list of ten images in glance. I create a new image and create a volume with this new image: it works. I create an instance with this new image: OK. What is the problem ? The images present before the "reconfigure" are listed, visible in horizon for example, but unusable. Is there a way to fix this, or do we have to reinstall them all? Thanks in advance for your help if this problem speaks to you. Franck VEDEL Dép. Réseaux Informatiques & Télécoms IUT1 - Univ GRENOBLE Alpes 0476824462 Stages, Alternance, Emploi. http://www.rtgrenoble.fr <http://www.rtgrenoble.fr/>

4 13

Cinder NFS Encryption
by Hitesh Mathur 23 Nov '21

23 Nov '21

Hi, I am not able to find whether Cinder support NFS data-in-transit encryption or not. Can you please provide the information on this and how to use it ? -- Regards Hitesh

2 1

[magnum] [victoria] [fedora-coreos]
by Namrata Sitlani 23 Nov '21

23 Nov '21

Hello All, We run release Victoria(11.1.1) and we deployed Kubernetes version 1.18.20 on Magnum with Fedora CoreOS version 33 and recently we ran into issues with Cinder CSI plugin. Multiple master nodes break the CSI plugin, but if I use a single master everything works fine. In my understanding it could be related to Fedora CoreOS version. It is not clear which versions are supported. Can somebody give me information on which version is supported? Thanks, Namrata Sitlani

1 0

[openstack-helm] No Meeting Tomorrow
by Gage Hugo 22 Nov '21

22 Nov '21

Hey team, Since there are no agenda items [0] for the IRC meeting tomorrow, the meeting is cancelled. Our next meeting will be November 30th. [0] https://etherpad.opendev.org/p/openstack-helm-weekly-meeting

1 0

[all][refstack][neutron][kolla][ironic][heat][trove][senlin][barbican][manila] Fixing Zuul Config Errors
by Clark Boylan 22 Nov '21

22 Nov '21

Hello, The OpenStack tenant in Zuul currently has 134 config errors. You can find these errors at https://zuul.opendev.org/t/openstack/config-errors or by clicking the blue bell icon in the top right of https://zuul.opendev.org/t/openstack/status. The vast majority of these errors appear related to project renames that have been requested of OpenDev or project retirements. Can you please look into fixing these as they can be an attractive nuisance when debugging Zuul problems (they also indicate that a number of your jobs are probably not working). Project renames creating issues: * openstack/python-tempestconf -> osf/python-tempestconf -> openinfra/python-tempestconf * openstack/refstack -> osf/refstack -> openinfra/refstack * x/tap-as-a-service -> openstack/tap-as-a-service * openstack/networking-l2gw -> x/networking-l2gw Project retirements creating issues: * openstack/neutron-lbaas * recordsansible/ara Projects whose configs have errors: * openinfra/python-tempestconf * openstack/heat * openstack/ironic * openstack/kolla-ansible * openstack/kuryr-kubernetes * openstack/murano-apps * openstack/networking-midonet * openstack/networking-odl * openstack/neutron * openstack/neutron-fwaas * openstack/python-troveclient * openstack/senlin * openstack/tap-as-a-service * openstack/zaqar * x/vmware-nsx * openinfra/openstackid * openstack/barbican * openstack/cookbook-openstack-application-catalog * openstack/heat-dashboard * openstack/manila-ui * openstack/python-manilaclient Let us know if we can help decipher any errors, Clark

6 8

[requirements][cinder][manila] pyparsing update needs handling
by Matthew Thode 22 Nov '21

22 Nov '21

https://review.opendev.org/818614 For cinder it looks like operatorPrecedence is gone now. For manila it looks like the same thing. -- Matthew Thode

2 1

[manila] First Yoga cycle bug squash - Nov 29th - 3rd Dec
by Carlos Silva 22 Nov '21

22 Nov '21

Greetings Zorillas and interested stackers! As mentioned in the previous weekly meetings, we will soon be meeting for the first bugsquash of the Yoga release! The event will be held from Nov 29th to 3rd December, 2021, providing an extended contribution window. We will start the event with a call on the first day (Nov 29th). There will be three calls (one of them using our Manila upstream meeting time slot). All of the three calls of the week will be held in a Jitsi room [1]. Nov 29th - 1500 to 1540 UTC - Event opening and bug assignments Dec 2nd - 1500 to 1600 UTC - Collab review session (no regular Manila meeting on this day) Dec 3rd - 1500 to 1540 UTC - Status update and event wrap up A list of selected bugs will be shared here [2]. Please feel free to add any additional bugs you would like to address during the event. [1] https://meetpad.opendev.org/ManilaYogaM1Bugsquash [2] https://ethercalc.openstack.org/wvb2oa23rxbb Thank you in advance! Hope to see you there :) carloss

1 0

[manila] No IRC meeting on 25th Nov
by Goutham Pacha Ravi 22 Nov '21

22 Nov '21

Hello Zorillas, In lieu of holidays this week, we'll be skipping the weekly meeting earlier scheduled at 1500 UTC on 25th Nov 2021. Please reach out here or on OFTC's #openstack-manila should you have any matters that need to be addressed. Thanks, Goutham

1 0

[cinder] reminder: this week's meeting in video+IRC
by Brian Rosmaita 22 Nov '21

22 Nov '21

Quick reminder that this week's Cinder team meeting on Wednesday 24 November, being the final meeting of the month, will be held in both videoconference and IRC at the regularly scheduled time of 1400 UTC. These are the video meeting rules we've agreed to: * Everyone will keep IRC open during the meeting. * We'll take notes in IRC to leave a record similar to what we have for our regular IRC meetings. * Some people are more comfortable communicating in written English. So at any point, any attendee may request that the discussion of the current topic be conducted entirely in IRC. * The meeting will be recorded. connection info: https://bluejeans.com/3228528973 meeting agenda: https://etherpad.opendev.org/p/cinder-yoga-meetings cheers, brian

1 0

Migration from Midonet to OVN
by Jake Yip 22 Nov '21

22 Nov '21

Hi all, We are planning a migration from Midonet to OVN. The general idea is to: - pause neutron - do a `neutron-ovn-db-sync-util` - change networks/ports to geneve/ovs - hard reboot the instances We are wondering if anyone has done a migration like this before, and will like to share their experiences. Any input will be greatly appreciated. Regards, Jake -- Jake Yip DevOps Engineer, ARDC Nectar Research Cloud

2 1

[kolla-ansible][wallaby][magnum] calico works, flannel does not work
by wodel youchi 22 Nov '21

22 Nov '21

Hi, I don't have any experience in kubernetes yet, my goal in the present is to test that the auto-sacele of a kubernetes cluster is working properly using magnum. I followed some tutorials to get the test done. I tried to configure the test but I had problems with magnum-metrics-server not being able to provide metrics, after digging and reading a lot of things on the web, I found out that kubernetes master couldn't connect to the metrics server (curl -k https://ip_metrics_server didn't work) So I created another template but this time using calico as a network provider, and this time it worked. How can I find why flannel does not work? where should I look? Regards.

2 1

[neutron] Bug deputy report (Nov 15 - 21)
by Hongbin Lu 21 Nov '21

21 Nov '21

Hi, I am the bug deputy this week. Please find the report below: High * https://bugs.launchpad.net/neutron/+bug/1951010 Restarting Neutron floods Nova with segment aggregates calls * https://bugs.launchpad.net/neutron/+bug/1951225 [OVN] Agent can't be found in functional test sometimes Medium * https://bugs.launchpad.net/neutron/+bug/1951149 [OVN] If "chassis" register is deleted, "chassis_private" can have 0 "chassis" associated * https://bugs.launchpad.net/neutron/+bug/1951559 [OVN] Router ports gateway_mtu option should not always be set * https://bugs.launchpad.net/neutron/+bug/1951564 snat random-fully supported with iptables 1.6.0 Low * https://bugs.launchpad.net/neutron/+bug/1951272 [OVN] OVS to OVN migration should be stopped if OVS agent firewall is "iptables_hybrid" * https://bugs.launchpad.net/neutron/+bug/1951429 Neutron API responses should not contain tracebacks * https://bugs.launchpad.net/neutron/+bug/1951569 Undecided (might need further traiging from domain experts) * https://bugs.launchpad.net/neutron/+bug/1951074 [OVN] default setting leak nameserver config from the host to instances * https://bugs.launchpad.net/neutron/+bug/1951493 OVS DPDK restart results in that tap interfaces in network namespaces can't be opened * https://bugs.launchpad.net/neutron/+bug/1951517 Segmentation ID should be lower or equal to 4095 Best regards, Hongbin

1 0

[all][tc] What's happening in Technical Committee: summary 19th Nov, 21: Reading: 10 min
by Ghanshyam Mann 19 Nov '21

19 Nov '21

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We cancelled this week's meeting due to Open Infra Keynotes at the same time. * Next week's meeting is on IRC on 25th Nov, Thursday 15:00 UTC, feel free the topic on the agenda[1] by Nov 24th. 2. What we completed this week: ========================= * Updated Yoga testing runtime [2] * Retire puppet-senlin[3] * Removed TC office hours in favour of weekly meetings [4] 3. Activities In progress: ================== TC Tracker for Yoga cycle ------------------------------ * This etherpad includes the Yoga cycle targets/working items for TC[5]. Open Reviews ----------------- * 8 open reviews for ongoing activities[6]. Fixing Zuul config error -------------------------- Clark sent a email tagging projects need action for zuul config error in their projects[7]. Please fix those config error which is not causing failure now but will fail on changes in those repo or at least is not testing what we think they are. Few projects are fixing those but still we need to clean it up completly. RBAC discussion: continuing from PTG ---------------------------------------------- We had another meeting on Wed and figured out the open question and schedule. Meeting notes are in this etherpad[8]. Final design and schedule is up in this goal rework patch[9]. Please review that and provide feedback from your project perspective. Community-wide goal updates ------------------------------------ * RBAC goal is pretty much in good shape now, feel free to review it[9] * There is one more goal proposal for 'FIPS compatibility and compliance' which needs feedback from community as well from TC[10]. * TC is working to prepare the final goal(s) soon, please wait for some more time. Adjutant need maintainers and PTLs ------------------------------------------- We received response from Braden, Albert on ML, so let's wait on the final call to help here[11]. New project 'Skyline' proposal ------------------------------------ * We discussed it in TC PTG, we are ok to have Skyline as official project but we have few open queries on gerrit and still waiting from skyline team to respond[12]. TC tags analysis ------------------- * TC agreed to remove the framework and it is communicated in ML[13]. Project updates ------------------- * Rename ‘Extended Maintenance’ SIG to the ‘Stable Maintenance’[14] * Retire training-labs repo[15] * Add ProxySQL repository for OpenStack-Ansible[16] * Retire js-openstack-lib [17] 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[18]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [19] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/815851 [3] https://review.opendev.org/c/openstack/governance/+/817329 [4] https://review.opendev.org/c/openstack/governance/+/817493 [5] https://etherpad.opendev.org/p/tc-yoga-tracker [6] https://review.opendev.org/q/projects:openstack/governance+status:open [7] http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025758… [8] https://etherpad.opendev.org/p/policy-popup-yoga-ptg [9] https://review.opendev.org/c/openstack/governance/+/815158 [10] https://review.opendev.org/c/openstack/governance/+/816587 [11] http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025786… [12] https://review.opendev.org/c/openstack/governance/+/814037 [13] http://lists.openstack.org/pipermail/openstack-discuss/2021-October/025571.… [14] https://review.opendev.org/c/openstack/governance-sigs/+/817499 [15] https://review.opendev.org/c/openstack/governance/+/817511 [16] https://review.opendev.org/c/openstack/governance/+/817245 [17] https://review.opendev.org/c/openstack/governance/+/807163 [18] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [19] http://eavesdrop.openstack.org/#Technical_Committee_Meeting-gmann

1 0

[keystone] No weekly meeting next week
by Douglas Mendizabal 19 Nov '21

19 Nov '21

Hi Keystone friends, I'll be out on PTO next week, so I'm canceling the Keystone weekly meeting for November 23. Meetings will resume the following week on November 30. Thanks, - Douglas Mendizábal (redrobot)

1 0

[requirements][keystone] pysaml2 and oslo.policy not able to be updated due to keystone test failures
by Matthew Thode 19 Nov '21

19 Nov '21

more or less as the title states the following reviews show the failures pysaml2: https://review.opendev.org/818612 oslo.policy: https://review.opendev.org/815820 -- Matthew Thode

2 1

[sdk][tc] A new home for Gophercloud
by Emilien Macchi 19 Nov '21

19 Nov '21

Hi everyone, The Gophercloud project is a stable OpenStack SDK in Golang, which has been widely used by many communities now, including Terraform and all the Kubernetes on OpenStack projects. The project was initiated by Rackspace in 2013 and has already successfully managed their departure as a principal contributor. Fast forward to 2021, Joe Topjian (major maintainer) who used to be an active contributor in Puppet modules and also a voice in the OpenStack operators space, has reached out to me so we can discuss a transition for maintenance: https://github.com/gophercloud/gophercloud/issues/2246 We have discussed this internally and here are our notes: https://github.com/gophercloud/gophercloud/issues/2246#issuecomment-9575894… . To make it short, we are figuring out whether it would make sense to find a new home for the project and if yes, where. The main reason we're reaching out to the opendev community first is because we think this is the most logical place to host the project, alongside OpenStack: Some ideas: - The project would potentially have more visibility in the community, it’s a SDK therefore strongly relying on OpenStack APIs stability. - Use (some) opendev tools, mainly Zuul & nodepool resources - integrate with other projects. - Governance: - Gerrit / not Gerrit: we don’t think we would move to Gerrit yet, as the existing contributors are probably more used to the Github workflow, and we clearly don’t want to lose anyone in the process). - IRC: we could have an IRC channel, potentially Slack as well. Many things to figure out and before we answer these questions, we would like to poll the community: what do you think? Have you contributed to the project? What's your feeling about the ideas here? We welcome any feedback and will take it in consideration in our discussions. Thanks everyone, -- Emilien Macchi, on behalf of the Gophercloud contributors

8 13

[nova][all] Adding /healthcheck support in Nova, and better healthcheck in every projects
by Thomas Goirand 19 Nov '21

19 Nov '21

Hi, About a year and a half ago, I attempted to add /healthcheck support by default in all projects. For Nova, this resulted in this patch: https://review.opendev.org/c/openstack/nova/+/724684 For other projects, it's been merged almost everywhere (I'd have to survey all project to see if that's the case, or if I still have debian specific patches somewhere). Though for Nova, this sparked a discussion where it's been said that the current implementation of /healthcheck wasn't good enough. This resulted in threads about how to better do it. Unfortunately, this blocked my patch from being merged in Nova. It is my point of view to recognize a failure here. The /healthcheck URL was added in oslo.middleware so one can use it with something like haproxy to verify that the API is up, and responds. It was never designed to check, for example, if nova-api has a valid connectivity to MySQL and RabbitMQ. Yes, this is welcome, but in the mean time, operators must tweak the default file to have a valid, useable /etc/nova/api-paste.ini. So I am hereby asking the nova team: Can we please move forward and agree that 1.5 years waiting for such a minor patch is too long, and that such patch should be approved, prior to having a better healtcheck mechanism? I don't think it's a good idea to ask Nova users to wait potentially more development cycles to have a good-by-default api-paste.ini file. At the same time, I am wondering: is anyone even working on a better healthcheck system? I haven't heard that anyone is working on this. Though it would be more than welcome. Currently, to check that a daemon is alive and well, operators are stuck with: - checking with ss if the daemon is correctly connected to a given port - check the logs for rabbitmq and mysql errors (with something like filebeat + elastic search and alarming) Clearly, this doesn't scale. When running many large OpenStack clusters, it is not trivial to have a monitoring system that works and scales. The effort to deploy such a monitoring system is also not trivial at all. So what's been discussed at the time for improving the monitoring would be very much welcome, though not only for the API service: something to check the health of other daemons would be very much welcome. I'd very much would like to participate in a Yoga effort to improve the current situation, and contribute the best I can, though I'm not sure I'd be the best person to drive this... Is there anyone else willing to work on this? Hoping this message is helpful, Cheers, Thomas Goirand (zigo)

6 9

[neutron] Drivers meeting - Friday 18.11.2021 - cancelled
by Lajos Katona 18 Nov '21

18 Nov '21

Hi Neutron Drivers! Due to the lack of agenda, let's cancel tomorrow's drivers meeting. See You on the meeting next week. Lajos Katona (lajoskatona)

1 0

[all] tempest-integrated-compute-centos-8-stream job is failing
by Balazs Gibizer 18 Nov '21

18 Nov '21

Hi, The centos 8 stream job is failing in 100% of the cases with mirror issues [2]. You probably need to hold you recheck until it is resolved. cheers, gibi [1] https://zuul.opendev.org/t/openstack/builds?job_name=tempest-integrated-com… [2] https://zuul.opendev.org/t/openstack/build/67dc088859d1435c9a5097e7469e5c12…

3 5

[all][sdk][neutronclient][neutron] List of projects that use python-neutronclient as "client"
by Lajos Katona 18 Nov '21

18 Nov '21

Hi, During the Yoga PTG we discussed the possibility of deprecating python-neutronclient (see [0]). The CLI part of python-neutronclient is already deprecated (see [1]), but we have python bindings for Neutron both in python-neutronclient and in openstacksdk. As python-neutronclient's binding code is widely used as "client" code (i.e.: Heat, Horizon, Nova), but python-openstackclient uses openstacksdk's binding code for "client" this means duplicated work and maintenance for all the bindings. If we have a new API feature the binding code must go both to python-neutronclient to make it available in Heat, and to openstacksdk to have openstackclient support for it. The best would be to have the binding code in openstacksdk and deprecate python-neutronclient, but before we plan anything we would like to have a list of projects that depend on python-neutronclient. We identified a few but for sure with python-neutronclient's long history there are a lot more: * Heat * Horizon * Nova * various Networking projects * ...... Please share Your thoughts about this topic and the projects which use python-neutronclient, it would be really helpful to see how we can move forward. [0]: https://etherpad.opendev.org/p/neutron-yoga-ptg#L372 [1]: https://review.opendev.org/c/openstack/python-neutronclient/+/795475 Regards Lajos Katona (lajoskatona)

1 0

[sdk]: Check instance error message
by dmeng 17 Nov '21

17 Nov '21

Hello there, Hope everything is going well. I'm wondering if there is any method that could check the error message of an instance whose status is "ERROR"? Like from openstack cli, "openstack server show server_name", if the server is in "ERROR" status, this will return a field "fault" with a message shows the error. I tried the compute service get_server and find_server, but neither of them show the error messages of an instance. Thanks and have a great day! Catherine

3 3

[cinder] yoga R-17 virtual midcycle on 1 december
by Brian Rosmaita 17 Nov '21

17 Nov '21

As decided at today's weekly meeting, the Cinder Yoga R-17 virtual midcycle will be held: DATE: Wednesday 1 December 2021 TIME: 1400-1600 UTC LOCATION: https://bluejeans.com/3228528973 The meeting will be recorded. Please add topics to the midcycle etherpad: https://etherpad.opendev.org/p/cinder-yoga-midcycles cheers, brian

1 0

[tacker] Skip next IRC meeting
by Yasufumi Ogawa 17 Nov '21

17 Nov '21

Hi team, Due to my absence from work, I would like to skip the next IRC meeting on November 23. Thanks, Yasufumi

1 0

[barbican] No weekly meeting next week
by Douglas Mendizabal 17 Nov '21

17 Nov '21

Hi Barbicaneers, I'll be out on PTO next week, so I'm canceling the Barbican weekly meeting for November 23. Meeting will resume the following week on November 30. Thanks, - Douglas Mendizábal (redrobot)

1 0

[all] git-review broken by git version 2.34.0
by Pierre Riteau 17 Nov '21

17 Nov '21

Hello, I recently upgraded to the latest git release 2.34.0 and noticed it breaks git-review (output below is with -v): Errors running git rebase -p -i remotes/gerrit/stable/xena fatal: --preserve-merges was replaced by --rebase-merges I submitted a patch [1], but since it would break compatibility with git versions older than 2.18 that don't support the --rebase-merges (-r) option, it may need to be refined before being merged. Cheers, Pierre [1] https://review.opendev.org/c/opendev/git-review/+/818219

2 2

[cinder] Bug deputy report for week of 11-17-2021
by Sofia Enriquez 17 Nov '21

17 Nov '21

No meeting today :( This is a bug report from 11-10-2021 to 11-17-2021. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- High - https://bugs.launchpad.net/cinder/+bug/1950474 "Xena accept transfer policy breaks volume transfer workflow". Unassigned. Medium - https://bugs.launchpad.net/cinder/+bug/1950291 "tempest-integrated-compute-centos-8-stream fails with version conflict in boto3". Unassigned. - https://bugs.launchpad.net/cinder/+bug/1951163 "Unable to import/manage an encrypted volume". Unassigned. - https://bugs.launchpad.net/cinder/+bug/1951046 "DS8000 driver terminates volume connection when there still has volume attached to instances - Ussuri". Unassigned. Cheers, -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com> -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

[nova] weird python stacktrace in nova-compute
by Arnaud Morin 17 Nov '21

17 Nov '21

Hey all, We have some python stacktrace in our nova-compute journalctl logs, which looks like that: $ journalctl -u nova-compute ... Nov 01 09:04:03 host1234 python3[161354]: Exception in thread tpool_thread_6: Nov 01 09:04:03 host1234 python3[161354]: Traceback (most recent call last): Nov 01 09:04:03 host1234 python3[161354]: File "/opt/openstack/nova/lib/python3.6/threading.py", line 916, in _bootstrap_inner Nov 01 09:04:03 host1234 python3[161354]: self.run() Nov 01 09:04:03 host1234 python3[161354]: File "/opt/openstack/nova/lib/python3.6/threading.py", line 864, in run Nov 01 09:04:03 host1234 python3[161354]: self._target(*self._args, **self._kwargs) Nov 01 09:04:03 host1234 python3[161354]: File "/opt/openstack/nova/lib/python3.6/site-packages/eventlet/tpool.py", line 96, in tworker Nov 01 09:04:03 host1234 python3[161354]: _wsock.sendall(_bytetosend) Nov 01 09:04:03 host1234 python3[161354]: TimeoutError: [Errno 110] Connection timed out Nov 01 09:04:03 host1234 python3[161354]: Traceback (most recent call last): Nov 01 09:04:03 host1234 python3[161354]: File "/opt/openstack/nova/lib/python3.6/site-packages/eventlet/hubs/poll.py", line 109, in wait Nov 01 09:04:03 host1234 python3[161354]: listener.cb(fileno) Nov 01 09:04:03 host1234 python3[161354]: File "/opt/openstack/nova/lib/python3.6/site-packages/eventlet/tpool.py", line 58, in tpool_trampoline Nov 01 09:04:03 host1234 python3[161354]: assert _c Nov 01 09:04:03 host1234 python3[161354]: AssertionError Nov 01 09:04:03 host1234 python3[161354]: Removing descriptor: 16 ... After this, nova-compute is "stuck" and does nothing more, but still continue to answer on RPC (it is still up in nova services) We am not experts of python threading / eventlet stuff, and we have no idea how to debug this. Our current solution is to restart nova-compute, but it's more a dirty workaround than a real fix. Does it ring a bell to someone in the community? Cheers, Arnaud

1 0

[edge][neutron][all] Edge related documentation
by Lajos Katona 16 Nov '21

16 Nov '21

Hi, During Yoga PTG the Neutron team had a very useful discussion together with some Designate folks, see the etherpad [1]. >From Neutron team's perspective the task is to document how to set up an edge site with AZ, DNS.... etc. This is a cross project effort (even if just do it for Neutron-Designate today), is there a place for such documentation? [1]: https://etherpad.opendev.org/p/octavia-designate-neutron-ptg#L16

1 0

[cinder][rbd] festival of RBD driver reviews 11 november 1500 utc
by Brian Rosmaita 16 Nov '21

16 Nov '21

By popular demand (really!), I'm scheduling a RBD driver review festival for next week. It's a community driver, and we've got a backlog of patches: https://review.opendev.org/q/project:openstack/cinder+status:open+file:cind… If your patch is currently in merge conflict, it would be helpful if you could get conflicts resolved before the festival. Also, if you have questions about comments that have been left on your patch, this would be a good time to get them answered. who: Everyone! what: The Cinder Festival of RBD Driver Reviews when: Thursday 11 November 2021 from 1500-1600 UTC where: https://meet.google.com/fsb-qkfc-qun etherpad: https://etherpad.opendev.org/p/cinder-festival-of-driver-reviews (Note that we're trying google meet for this session.)

3 3

[ Tacker ] Passing a shell script/parameters as a file in cloud config
by Lokendra Rathour 16 Nov '21

16 Nov '21

Hi, *In Tacker, while deploying VNFD can we pass a file ( parameter file) and keep it at a defined path using cloud-config way?* Like in *generic hot template*s, we have the below-mentioned way to pass a file directly as below: parameters: foo: default: bar resources: the_server: type: OS::Nova::Server properties: # flavor, image etc user_data: str_replace: template: {get_file: the_server_boot.sh} params: $FOO: {get_param: foo} *but when using this approach in Tacker BaseHOT it gives an error saying * "nstantiation wait failed for vnf 77693e61-c80e-41e0-af9a-a0f702f3a9a7, error: VNF Create Resource CREATE failed: resources.obsvrnnu62mb: resources.CAS_0_group.Property error: resources.soft_script.properties.config: No content found in the "files" section for get_file path: Files/scripts/install.py 2021-10-28 00:46:35.677 3853831 ERROR oslo_messaging.rpc.server " do we have a defined way to use the hot capability in TACKER? Defined Folder Structure for CSAR: . ├── BaseHOT │ └── default │ ├── RIN_vnf_hot.yaml │ └── nested │ ├── RIN_0.yaml │ └── RIN_1.yaml ├── Definitions │ ├── RIN_df_default.yaml │ ├── RIN_top_vnfd.yaml │ ├── RIN_types.yaml │ ├── etsi_nfv_sol001_common_types.yaml │ └── etsi_nfv_sol001_vnfd_types.yaml ├── Files │ ├── images │ └── scripts │ └── install.py ├── Scripts ├── TOSCA-Metadata │ └── TOSCA.meta ├── UserData │ ├── __init__.py │ └── lcm_user_data.py *Objective: * To pass a file at a defined path on the VDU after the VDU is instantiated/launched. -- ~ Lokendra skype: lokendrarathour

2 3