- openstack-discuss - lists.openstack.org

[patrole][qa] Proposing Soniya Vyas to Patrole core
by Martin Kopec 15 Sep '21

15 Sep '21

Hi all, Soniya Vyas (IRC: soniya29) is very enthusiastic about Patrole project and since she has been very helpful, I'd like to propose her for Patrole core. You can vote/feedback on this email. If no objections within a week, I'll add her to the core list. Regards, -- Martin Kopec Senior Software Quality Engineer Red Hat EMEA

6 6

[nova] Backporting test refactors to stable/train
by Artom Lifshitz 15 Sep '21

15 Sep '21

Hey Elod and Vlad (and the ML, cc'ed) This is continuing the conversation Elod and I had on IRC. The immediate context is [1], which is a test refactor backport that in turn enables conflict-free backporting of at least two fixes. >From Red Hat's point of view, we will have to care about stable/train for *very* long time, so we'd like to do everything we can to make our lives easier, including backportings things are aren't strictly necessary (like test refactors) in order to greatly facilitate future backports, which tend to be of the "func reproducer test + actual fix" variety. Elod made the excellent point that while this approach does make backports to train much easier to both write and review, it also "postpones" all of the manual conflict resolution work onto anyone doing stable/stein backports. I went and gathered data. I looked at the last 6 months of backports in Gerrit. At the time of this writing, stable/train 72 [2] had and stable/stein had 24 [3]. Out of the latter, Vlad Gusev was the main non-Red Hat contributor. So my proposal is simple: I'm willing to help resolve conflicts in stable/stein backport for folks like Vlad, if it means making backports to stable/train easier. I'm confident that it would still mean an overall reduction in man-hours for backports, even if I become personally on the hook to deal with the fallout of backporting test refactors to stable/train. Cheers! [1] https://review.opendev.org/c/openstack/nova/+/791481 [2] https://review.opendev.org/q/project:openstack/nova+branch:stable/train+-ag… [3] https://review.opendev.org/q/project:openstack/nova+branch:stable/stein+-ag…

3 3

[tc][neutron] Use Kea insted of Dibbler for IPv6 PD
by Lajos Katona 15 Sep '21

15 Sep '21

Hi, In Neutron we have a bug (see [1]) that states Dibbler is concluded (see [2]), and the developer of Dibbler suggests using Kea (see [3]) instead. Before we start checking technical things first thing is that Kea is MPL2.0 licensed (see [4]) and not sure if we can pull that as dependency to Openstack. Kea is an ISC project so looks safe, but I am not familiar with these legal things. How can we check and be sure? [1] https://bugs.launchpad.net/neutron/+bug/1916428 [2] https://github.com/tomaszmrugalski/dibbler#project-status [3] https://gitlab.isc.org/isc-projects/kea [4] https://www.isc.org/kea/ Thanks in advance Lajos Katona (lajoskatona)

3 3

[nova] "Migrating" ComputeNode within ComputeHost / vmwareapi ComputeNode per ESXi
by Wiesel, Fabian 15 Sep '21

15 Sep '21

Hi, I am looking at adding the option for the vmwareapi driver to expose the individual ESXi hosts. A compute-host would represent a cluster in vsphere, and each ESXi-host within a cluster would be a ComputeNode. The main question here is, how to handle a hypervisor initiated change of the compute-node and if the community is open to extending the compute-driver interface for such a use-case. The secondary one is, if there is interest in having the upstream vmwareapi driver getting this functionality. * What are we trying to solve by that? Our users are asking for that feature, because they want to know, if a payload of two VMs runs on the same physical host, as the performance characteristics are quite different. We also gain there from an operational and code perspective: - Nova would become aware of the actual resources per ESXi host instead of the aggregated ones. - Resources could also be more faithfully represented in the resource-tree (e.g., shared storage as an aggregate). - Capabilities, (cpu-flags, architecture,… ), Numa topology, and even small things like up-time can be faithfully reflected on a per-host basis. - The internal scheduling decisions in the driver for ephemeral storage and compute (for live-migration) can be handled by standard placement & nova scheduling. * Why do we need to change the driver interface? Main problem here is that VSphere might migrate a VM between two ESXi-hosts for various reasons. So, the driver would need a way to communicate such a change somehow. But even if we disable all kind of migrations on VMware side, we still need a way to migrate the VMs from the single compute-node for the whole cluster to the compute nodes for each individual ESXi-hosts. * Proposed Driver API Change The Compute-Manager is currently periodically polling the driver with ComputeDriver.get_info(self, instance, use_cache) which returns an InstanceInfo, which is essentially only the power-state. Adding two new optional fields to InstanceInfo could allow the driver to inform the manager about - the current compute-node - the compute-node the VM is migrated to This allows the Compute-Manager to take care of updating the node field and the resource-allocations accordingly. Additionally, we can notify the manager over Instance lifecycle events, allowing for a faster reflection of a compute-node change. The lifecycle events could be - EVENT_LIFECYCLE_COMPUTE_NODE_MIGRATION_STARTED - EVENT_LIFECYCLE_COMPUTE_NODE_MIGRATION_COMPLETED Or one could generalize the current life-migration events to include also such compute-node changes. I am looking forward for your feedback. Cheers, Fabian

1 0

[TripleO] Installation of Undercloud on CentOS 8.4
by Anirudh Gupta 14 Sep '21

14 Sep '21

Hi Team, I was trying to install Undercloud on CentOS 8.4 While installing the repos, I got the error "The host is not stream" So wanted to confirm if only CentOS 8 Stream is supported or is there still a way we can install TripleO on CentOS 8.4 or any specific version? Regards Anirudh Gupta

2 1

[tacker] Ayumu Ueha for Tacker Core
by Yasufumi Ogawa 14 Sep '21

14 Sep '21

Hi everyone, I'd like to propose Ayumu Ueha as a core reviewer. He has made huge contributions not only for reviewing actively but also developing key features of NFV Container support and helping us for fixing critical problems [1][2]. Assuming that there are no objections, we will add Ayumu to tacker core team next week. [1] https://review.opendev.org/q/owner:ayumu [2] https://www.stackalytics.io/?module=opendev.org%2fopenstack%2ftacker&user_i…

7 8

The code about BP Allow migrating PMEM's data
by Juntingqiu Qiujunting (邱军婷) 14 Sep '21

14 Sep '21

Hi all: Is there anything in the code logic that needs to be modified? If there is no problem with the implementation structure of the code, I start adding unit tests. https://review.opendev.org/c/openstack/nova/+/802225 Thank you Fossen. -----邮件原件----- 发件人: Balazs Gibizer [mailto:balazs.gibizer@est.tech] 发送时间: 2021年9月14日 16:45 收件人: Juntingqiu Qiujunting (邱军婷) <qiujunting(a)inspur.com> 抄送: smooney(a)redhat.com; Brin Zhang(张百林) <zhangbailin(a)inspur.com> 主题: Re: The code about BP Allow migrating PMEM's data Hi, Sorry I cannot read the content of your mail due to smime. Also please keep the discussion on openstack-discuss if possible. Cheers, gibi On Tue, Sep 14 2021 at 01:45:49 AM +0000, Juntingqiu Qiujunting (邱军婷) <qiujunting(a)inspur.com> wrote:

1 0

Question about vDPA support for live migration
by Jeffrey Zhang 14 Sep '21

14 Sep '21

Hey everyone, I am testing vDPA on OpenStack Wallaby. I have set it up successfully. But I found there are several features that are not implemented, including live/code migration[0], which I wanted. Anyone know what's the latest progress about this. when will the live migration feature land? I found no talk about this. [0] <https://signaldomn.online/click?redirect=https%3A%2F%2Freview.opendev.org%2…> https://review.opendev.org/c/openstack/nova/+/780866/3/ <https://signaldomn.online/click?redirect=https%3A%2F%2Freview.opendev.org%2…> releasenotes/notes/vdpa-cc2300d2c46c150b.yaml Thanks.

2 3

[puppet] Propose retiring puppet-freezer
by Takashi Kajinami 14 Sep '21

14 Sep '21

Hello, Today I looked into the current code of puppet-freezer and noticed this module is still very incomplete. Most of its implementations are inherited from the cookiecutter template, and the module doesn't support fundamental resources like packages, services and so on. Because we haven't seen any interest in fixing that feature gap for a long time, I'll propose retiring the module during this cycle. Please let me know if you have any concerns. If I don't hear any objections I'll submit a series of patches to retire the module. Thank you, Takashi Kajinami

2 3

[oslo] RFE requested for oslo-messaging
by Nikita Kalyanov 14 Sep '21

14 Sep '21

Dear Release Team, We experienced strange freezes and timeouts for a long time on our OpenStack deployment [1] and some duplicate messages [2]. After applying the fixes [3], [4] both bugs are gone. Is a freeze exception possible for these? Thanks! nkalyanov [1] https://bugs.launchpad.net/oslo.messaging/+bug/1935864 [2] https://bugs.launchpad.net/oslo.messaging/+bug/1935883 [3] https://review.opendev.org/c/openstack/oslo.messaging/+/800564 [4] https://review.opendev.org/c/openstack/oslo.messaging/+/800575

5 5

MTU configuration for instances
by shubjero 13 Sep '21

13 Sep '21

Hey Openstackers, I am trying to determine how to configure/manipulate an OpenStack instances MTU configuration upon deployment/creation. I know that in neutron.conf you can set the 'global_physnet_mtu = value' and 'path_mtu = value' in ml2_conf.ini. In our case we are setting these to 9000 as the physical interfaces of our control plane and compute nodes are all using 9000 mtu however the bonded interfaces for our various networks are 8900. With this mtu configured, we've noticed that our OpenStack neutron networks within a tenancy are auto-selecting a mtu of 8958 as from what I've read it will take the 9000 mtu and minus the approriate overhead to pass the underlying physical network properly but I think because our bonded interfaces are 8900 this has started to introduce some problems in instances with operating systems that hard code the MTU from Neutron networks. When an instance gets deployed it may (Ubuntu 20.04) or may not (Ubuntu 18.04) retrieve the network mtu from neutron metadata service and builds it in to the netplan configuration as provided below. My question is, where in Neutron could I configure a lower MTU for the instances? I am not sure if I should change my global_physnet_mtu and path_mtu to 8900 OR some other configuration option? Thanks all! ---- # curl --silent http://169.254.169.254/openstack/2018-08-27/network_data.json | jq . { "links": [ { "id": "tap4318b50a-76", "vif_id": "4318b50a-7685-4777-917f-eea947aa1cb6", "type": "ovs", "mtu": 8958, <--- Matches the MTU of the underlying OpenStack network attached to this instance "ethernet_mac_address": "fa:16:3e:a4:19:44" } ], "networks": [ { "id": "network0", "type": "ipv4_dhcp", "link": "tap4318b50a-76", "network_id": "4acb077c-0656-4731-ba42-031a22af7931" } ], "services": [] } ---- # cat /etc/netplan/50-cloud-init.yaml # This file is generated from information provided by the datasource. Changes # to it will not persist across an instance reboot. To disable cloud-init's # network configuration capabilities, write a file # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following: # network: {config: disabled} network: version: 2 ethernets: ens3: dhcp4: true match: macaddress: fa:16:3e:a4:19:44 mtu: 8958 <--- MTU gets hard-coded in the config for Ubuntu 20.04 but not Ubuntu 18.04. set-name: ens3

2 1

[TripleO] Reviving the TripleO review priorities etherpad
by James Slagle 13 Sep '21

13 Sep '21

I'd like to see about helping to focus our review priorities by reviving the tripleo-review-priorities etherpad: https://etherpad.opendev.org/p/tripleo-review-priorities I deleted all the old cruft, and created some new high level topics. There are likely others as well. I'll start by filling in some links to what I think the highest priority items are, but I could use help to create an accurate list of where the core team should focus. I'd like to get some feedback on the approach in the #tripleo meeting tomorrow as well. Let me know any thoughts. -- -- James Slagle --

1 0

[neutron] Bug deputy report for week of September 6
by Lajos Katona 13 Sep '21

13 Sep '21

Hi Neutron Team I was bug deputy in neutron last week. Summary ======= One bug needs more attention but that is for Rocky, and seems like ovs-agent ovs communication issue ( https://bugs.launchpad.net/neutron/+bug/1943112 ) And one question regarding Neutron standalone use with OVN perhaps ( https://bugs.launchpad.net/neutron/+bug/1943355 ) The employee of the week is ralonsoh :-) Needs attention ================ * openstack rocky version, neutron is throwing the following error : ovs|00002|ovsdb_idl|WARN|transaction error: {"details":"Transaction causes multiple rows in \"Manager\" table to have identi https://bugs.launchpad.net/neutron/+bug/1943112 * Can We Use Openstack Neutron as standalone component? https://bugs.launchpad.net/neutron/+bug/1943355 This one is more like a question, not an actual bug, or a bug for documentation. Bugs with assignees =================== * [OVN] "DHCP_Options" is not updated when the metadata port IPs are https://bugs.launchpad.net/neutron/+bug/1942794 Assigned to ralonsoh, patch: https://review.opendev.org/c/openstack/neutron/+/807692 * [OVN] Metadata ports owner is "network:distributed" only since Wallaby release https://bugs.launchpad.net/neutron/+bug/1942866 Assigned to ralonsoh, patch: https://review.opendev.org/c/openstack/neutron/+/807707 * [OVN] "neutron-ovn-tempest-ovs-master-fedora" CI job failing since 2021-08-24 https://bugs.launchpad.net/neutron/+bug/1942913 Assigned to ralonsoh, patches: https://review.opendev.org/c/openstack/neutron/+/808109 & https://review.opendev.org/c/openstack/tempest/+/808081 * [OVN][FT] During the OvnNorthd stop, the unix file could not exist https://bugs.launchpad.net/neutron/+bug/1943029 Assigned to ralonsoh, patch: https://review.opendev.org/c/openstack/neutron/+/807862 * Replace "Inspector.from_engine()" with "sqlalchemy.inspect()" https://bugs.launchpad.net/neutron/+bug/1943155 Assigned to ralonsoh, patch: https://review.opendev.org/c/openstack/neutron/+/808103 RFEs ==== No new RFE for this week.

1 0

OpenInfra Live - September 16 at 9am CT
by Ashlee Ferguson 13 Sep '21

13 Sep '21

Hi everyone, This week’s OpenInfra Live episode is brought to you by the OpenInfra Community. Since the Paris Summit in 2014, the OpenInfra Foundation has hosted our annual Superuser Awards to recognize organizations that have used open infrastructure to meaningfully improve their business while contributing back to the community. Past winners will join us for this episode to discuss where they are now, what is next, and tips for organizations applying for this year's awards. Episode: Superusers: Where are they now? Date and time: September 16 at 9am CT (1400 UTC) You can watch us live on: YouTube: https://www.youtube.com/watch?v=nUwgBdx0lSw <https://www.youtube.com/watch?v=nUwgBdx0lSw> LinkedIn: https://www.linkedin.com/feed/update/urn:li:ugcPost:6841804188478009344/ <https://www.linkedin.com/feed/update/urn:li:ugcPost:6841804188478009344/> Facebook: https://www.facebook.com/104139126308032/posts/4319519031436666/ <https://www.facebook.com/104139126308032/posts/4319519031436666/> WeChat: recording will be posted on OpenStack WeChat after the live stream Speakers: Jared Baker (The Ontario Institute for Cancer Research (OICR)) Belmiro Moreira (CERN) Mohammed Naser (VEXXHOST) Xiaoguang, Zhang and Zhiqiang Yu (China Mobile) Think your organization or someone you know could be a Superuser? Nominations <https://openinfrafoundation.formstack.com/forms/superuserawards_2021> for the 2021 Superuser Awards are open! Have an idea for a future episode? Share it now at ideas.openinfra.live <https://openinfrafoundation.formstack.com/forms/openinfralive>. Thanks, Ashlee

1 0

[Image Encryption] Meeting today cancelled
by Josephine Seifert 13 Sep '21

13 Sep '21

Hello, unfortunately I have to cancel the meeting today. Next meeting will be Monday 20th. greetings Josephine (Luzi)

1 0

[designate][interop] test_create_all_recordset_types ddt issue
by Martin Kopec 13 Sep '21

13 Sep '21

Hi designate team, I'm reaching out because we have encountered a problem with one of the tests [1]. We track the following name [2]: designate_tempest_plugin.tests.api.v2.test_recordset.RecordsetsTest.test_create_all_recordset_types That test name is actually incorrect due to the ddt and the real name of the test is created dynamically based on the test data passed via ddt. We have a consistency check which is supposed to verify that the test names tracked in our guidelines match the ones in Tempest/tempest plugins, however, there was a bug - we are fixing that by [3]. The question is, should interop track all the permutations of the test? See [4]. [1] https://opendev.org/openstack/designate-tempest-plugin/src/commit/da27a70ae… [2] https://opendev.org/osf/interop/src/commit/d8eac71fa507c68e9c1f83a19dd77af8… [3] https://review.opendev.org/c/osf/interop/+/806598 [4] https://review.opendev.org/c/osf/interop/+/807586 Thank you, -- Martin Kopec Senior Software Quality Engineer Red Hat EMEA

5 7

Instances cannot ping each other and cannot ping virtual-router
by wodel youchi 13 Sep '21

13 Sep '21

Hi, I deployed OpenStack Train using TripleO using this tutorial : https://kdjlab.com/deploying-rdo-in-a-cohesive-manner/ and the documentation of TripleO. I deployed it with DVR. In my deployment I am using virtual machines with nested-kvm. The deployment went well, I am using network isolation like this : - nic1 : provisioning - nic2 and nic3 (bond0) storage and storage mgmt networks, each one in it's VLAN - nic3 and nic5 (bond1) tenant, api and *external* (10.0.2.0/24 VLAN2100) networks, each one in it's VLAN In my physical host (the bare metal KVM) I created a bridge which handles the provisioning, tenant, api and external networks. I created a private tenant network (172.16.100.0/24). openstack network create private neutron subnet-create private 172.16.100.0/24 --name private-sub --dns-nameserver 172.16.0.252 I created a public network and I attached it to the external network using the same VLAN tag (10.0.2.0/24 VLAN 2100, pool: 10.0.2.100-10.0.2.120) : *openstack network create --provider-network-type vlan --provider-physical-network datacentre --provider-segment 2100 --external public* neutron subnet-create public 10.0.2.0/24 --name public-sub --disable-dhcp --allocation-pool=start=10.0.2.100,end=10.0.2.120 --gateway=10.0.2.1 --dns-nameserver 172.16.0.252 I created a vrouter, one port in the public network and the other in the private network. I created two cirrus instances, each one got it's ip address from the private network. I found : cirrus-1 : 172.16.100.81 cirrus-2 : 172.16.100.103 vrouter : 172.16.100.1 private : 10.0.2.101 external neutron:dhcp : 172.16.100.2 The problems : - The instances cannot ping each other. - The instances cannot ping the vrouter. - I cannot ping the public vrouter interface. But both instances can ping neutron:dhcp Could someone help me dig into this. Thanks in advance, Regards.

1 1

Openstack nova live migration pain
by hai wu 13 Sep '21

13 Sep '21

This is Openstack train release. Nova live migration is extremely painful to deal with. Everytime we complete a live migration, nova-compute service on the source host would still be up and runnning, but it would always fail to further increment the report_count database field in nova database, host heartbeat purpose, thus controller would think the hypervisor host failed via its is_up() function checking on report_count and updated_at fields iirc. So we end up having to manually migrate one vm at a time, then restart service, then manually migrate the next vm .. Any ideas? I already tried setting debug=True for nova.conf even for database tracing, but thus far I could not find any obvious error message. Each time the live migration (no shared storage) would succeed, but each time we will have to restart nova-compute service. This is so bad .. Any suggestions on this would be highly appreciated. Thanks, Hai

4 12

[largescale-sig] Next meeting: Sept 15th, 15utc
by Thierry Carrez 13 Sep '21

13 Sep '21

Hi everyone, The Large Scale SIG meeting is back this Wednesday in #openstack-operators on OFTC IRC, at 15UTC. You can doublecheck how that time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20210915T15 A number of topics have already been added to the agenda, including discussing the topic of our next OpenInfra.Live show! Feel free to add other topics to our agenda at: https://etherpad.openstack.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

1 0

[kolla][operators] Kolla Ansible will switch to use Quay.io by default (instead of Docker Hub)
by Radosław Piliszek 13 Sep '21

13 Sep '21

Dear Kolla Operators, This is a heads-up message that Kolla Ansible will switch to using Quay.io instead of Docker Hub by default. [1] This was agreed at the last meeting unless we find obstacles not to do it. But it seems it's even a better option for our users from China. [2] Therefore the patch [1] has been proposed and will likely be merged and probably backported to stable branches too. The main reasoning behind this change is that users suffer from Docker Hub's limits. [3] You may have noticed we started (a long time ago) publishing weekly, instead of daily, to Docker Hub to mitigate the issue to some extent. However, we had to switch our CI to Quay.io anyway to really be able to work on the project rather than pray for luck with Docker Hub. Quay.io has been great for us in general so we decided to make the same switch for users. Oh, and, by the way, we publish to Quay.io daily. ;-) If you use your own, local registry, then you are most likely unaffected but make sure you set both `kolla_registry` AND `kolla_namespace` as we have to override both. Finally, kind reminder that operators of multinode and production clusters are strongly advised to set up their own, local registries. The docs show a simple minimal example of that. [4] (use option 1 from the current page as possibility of option 2 will be gone with the switch to quay.io). Thank you for reading up to this point! [1] https://review.opendev.org/c/openstack/kolla-ansible/+/808486 [2] http://lists.openstack.org/pipermail/openstack-discuss/2021-September/02481… [3] https://bugs.launchpad.net/kolla-ansible/+bug/1942134 [4] https://docs.openstack.org/kolla-ansible/wallaby/user/multinode.html#deploy… -yoctozepto

2 2

Cannot create a swift container, mandatory "Storage Policy" dropdown field is empty
by Michel Niyoyita 13 Sep '21

13 Sep '21

Hello team , I am replacing swift by Ceph Radosgateway , and I am successful by creating containers through openstack and ceph CLI side . but once trying to create through the horizon dashboard I get errors: *Error: *Unable to fetch the policy details. , Unable to get the Swift service info and Unable to get the Swift container listing. Kindly help to solve the issue I deployed Openstack wallaby using kolla-ansible running on ubuntu 20.04 and ceph pacific using ansible running on ubuntu 20.04 Michel

1 0

goodbye openstack, welcome ansible
by Sorin Sbarnea 12 Sep '21

12 Sep '21

Today was my last day as a part of the OpenStack team at Red Hat. After almost 5 years, I've decided to use the opportunity to join the newly created Ansible DevTools team, a team that will improve and maintain development tools such as ansible-lint, molecule and the more recent Ansible extension for VS Code. If you are interested in finding out more, make sure to attend the next Ansible Contributor Summit <https://github.com/ansible/community/wiki/Contributor-Summit> which is happening at the end of this month. To those who will be missing me, don't worry I will not be far away. Cheers! Sorin Sbarnea

7 6

[all][infra] Mailing lists offline 2021-09-12 for server upgrade
by Jeremy Stanley 12 Sep '21

12 Sep '21

n Sunday, September 12, the OpenDev sysadmins will be performing operating system upgrades for the server hosting the lists.airshipit.org, lists.opendev.org, lists.openstack.org, lists.starlingx.io, and lists.zuul-ci.org sites. Expect extended outages for message delivery and access to archives between 15:00 UTC and 21:00 UTC, though messages sent to the mailing lists at these sites should queue normally and be delivered automatically after maintenance activities have concluded. If you have any questions, feel free to reply to this message, or find us in the #opendev channel on the OFTC IRC network. -- Jeremy Stanley

1 1

[all][tc] What's happening in Technical Committee: summary 10th Sept, 21: Reading: 5 min
by Ghanshyam Mann 10 Sep '21

10 Sep '21

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * TC held this week's IRC meeting on Sept 9th Thursday. * Most of the meeting discussions are summarized in the below sections (Completed or in-progress activities section). To know more details, you can check the complete logs @ https://meetings.opendev.org/meetings/tc/2021/tc.2021-09-09-15.00.log.html * We will have next week's meeting on Sept 16th, Thursday 15:00 UTC on IRC, feel free the topic in agenda [1] by Sept 15th. 2. What we completed this week: ========================= * Closeout Yoga PTL Election[2] * New project 'Venus' is an official project now[3] * Added the cinder-lvm charm to Openstack charms[4] * Fixed documentation for "assert:supports-standalone" tag[5] 3. Activities In progress: ================== TC Tracker for Xena cycle ------------------------------ TC is using the etherpad[6] for Xena cycle working item. We will be checking and updating the status biweekly in the same etherpad. Open Reviews ----------------- * Twelve open reviews for ongoing activities[7]. TC Chair/Vice-chair for Yoga ---------------------------------- * TC chair and vice-chair nomination are also up for review/vote[8][9]. TC tags analysis ------------------- * As discussed in last PTG, TC is working on an analysis of the usefulness of the Tags framework[10] or what all tags can be cleaned up. * You might have read the email from yoctozepto[11] about it, if you are operator, please respond to the email and based on the feedback we will continue the discussion on this. Leaderless projects ----------------------- * All the six leaderless projects[12] now have leaders identified and they are being proposed as PTL in governance[13]. Collecting Pain points from projects/SIGs/pop-up ----------------------------------------------------------- * We will continue discussing it in PTG[14], join us if you are interested to help/discuss this topic. Project updates ------------------- * Add openstack-loadbalancer charm and interfaces[15] * Retiring js-openstack-lib [16] Yoga release community-wide goal ----------------------------------------- * Please add the possible candidates in this etherpad [17]. * Current status: "Secure RBAC" is selected for Yoga cycle[18]. PTG planning ---------------- * We are collecting the PTG topics in etherpad[19], feel free to add any topic you would like to discuss. * We discussed the live stream of one of the TC PTG sessions like we did last time. Once we will have more topics in etherpad then we can select the appropriate one. Test support for TLS default: ---------------------------------- * Rico has started a separate email thread over testing with tls-proxy enabled[20], we encourage projects to participate in that testing and help to enable the tls-proxy in gate testing. 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[21]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [22] 3. Office hours: The Technical Committee offers a weekly office hour every Tuesday at 0100 UTC [23] 4. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] http://lists.openstack.org/pipermail/openstack-discuss/2021-September/02474… [3] https://review.opendev.org/c/openstack/governance/+/804824 [4] https://review.opendev.org/c/openstack/governance/+/807203 [5] https://review.opendev.org/c/openstack/governance/+/807728 [6] https://etherpad.opendev.org/p/tc-xena-tracker [7] https://review.opendev.org/q/project:openstack/governance+status:open [8] https://review.opendev.org/c/openstack/governance/+/807163 [9] https://review.opendev.org/c/openstack/governance/+/807178 [10] https://governance.openstack.org/tc/reference/tags/index.html [11] http://lists.openstack.org/pipermail/openstack-discuss/2021-September/02480… [12] https://etherpad.opendev.org/p/yoga-leaderless [13] https://review.opendev.org/q/topic:%2522formal-vote%2522+status:open+messag… [14] https://etherpad.opendev.org/p/pain-point-elimination [15] https://review.opendev.org/c/openstack/governance/+/807837 [16] https://review.opendev.org/c/openstack/governance/+/798540 [17] https://etherpad.opendev.org/p/y-series-goals [18] https://review.opendev.org/c/openstack/governance/+/803783 [19] https://etherpad.opendev.org/p/tc-yoga-ptg [20] http://lists.openstack.org/pipermail/openstack-discuss/2021-June/023000.html [21] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [22] http://eavesdrop.openstack.org/#Technical_Committee_Meeting [23] http://eavesdrop.openstack.org/#Technical_Committee_Office_hours -gmann

1 0

[designate][interop][qa] Yoga PTG - ddt topic
by Martin Kopec 10 Sep '21

10 Sep '21

Hi designate team, we (interop) would like to schedule a session on your PTG agenda to go through the latest changes in interop and more importantly through the ddt issue [1][2]. I put a topic proposal to your ptg agenda [3]. [1] http://lists.openstack.org/pipermail/openstack-discuss/2021-September/02475… [2] https://bugs.launchpad.net/designate/+bug/1943115 [3] https://etherpad.opendev.org/p/yoga-ptg-designate Thank you, -- Martin Kopec Senior Software Quality Engineer Red Hat EMEA

1 0

[neutron][interop] Yoga PTG session request
by Martin Kopec 10 Sep '21

10 Sep '21

Hi neutron team, we (interop) would like to schedule a 15-30 minute session on your PTG agenda to go through the latest changes in interop. We also detected a few tests from neutron-tempest-plugin which aren't tracked in interop yet and we need your help to find out whether they are good candidates or not. I put a topic proposal to your ptg agenda [1]. The best time seems to be on Wednesday or Thursday atm. [1] https://etherpad.opendev.org/p/neutron-yoga-ptg Thank you, -- Martin Kopec Senior Software Quality Engineer Red Hat EMEA

1 0

[horizon][dev] Multiple Login Sessions Enhancement Proposal
by Soares Sarto, Ricardo 10 Sep '21

10 Sep '21

Hi, Based on a new feature request about Horizon's login session, we would like to know if you're open to considering the improvement described below. If that is the case, we can submit a spec and upstream that implementation when the feature is done. Currently, a specific user is able to create multiple login sessions in Horizon, allowing him to log in to Horizon from multiple browsers/devices at the same time. The proposal is to have a parameterized solution and have a configuration to control how Horizon's login sessions are handled, the configuration options would be as below: 1 - Default option, allow the user to create multiple login sessions - this is the current behavior; 2 - Block subsequent login attempts if the user already has an active session; 3 - Invalidate the active session (if there is one) when the user creates a new one. Option number 2 obligates the user to log out of the system before opening a new one, on the other hand, in option number 3, the Horizon application is in charge of invalidating any previous active session and keeping only the latest one. This solution targets only Horizon login sessions, not affecting other kinds of sessions (e.g.: CLI).

3 2

[devstack][openstack-qa] DevStack on VirtualBox with Host-only Adapter
by os-user157 10 Sep '21

10 Sep '21

Hi all, I'm trying to deploy DevStack inside an Ubuntu Server 20.04.3 LTS virtual machine inside VirtualBox, but I'm struggling a lot, I got a lot of different errors. My use case: I want to test Zuul and DevStack, so I'll create a virtual machine for DevStack and one for Zuul, both on VirtualBox. I'm using a Host-only adapter with CIDR 192.168.200.0/24, and I need DevStack to be accessible on this network, and to use this network as the floating range for giving floating IP addresses to instances inside DevStack, so that Zuul can also access them. My local.conf: https://paste.openstack.org/show/809182/ Screenshots of the network configuration of my VirtualBox VM: First adapter, NAT (used for internet connectivity): https://i.imgur.com/C7XTiLJ.png Second adapter, host-only, used for communication with my Windows host and other VMs in the same network: https://i.imgur.com/Qeciuqx.png DevStack is erroring with the arping command on lib/neutron-legacy, arping gets 0 responses and exits the script with exit code 1 Does anyone have experience in dpeloying DevStack inside VirtualBox in such manner? I really need some help with this, if anyone could give me some ideas of waht could be wrong, it'd very much appreciated! Thanks.

3 3

[Interop] Proposal for new weekly meeting
by Arkady Kanevsky 10 Sep '21

10 Sep '21

Team, Friday meetings are a bit late for EMEA folks and too close to weekend plans. Propose that we move the meeting to Mondays (3pm UTC). Feedback, please. Thanks, -- Arkady Kanevsky, Ph.D. Phone: 972 707-6456 Corporate Phone: 919 729-5744 ext. 8176456

2 2

[release] Release countdown for week R-3, Sep 13 - Sep 17
by Herve Beraud 10 Sep '21

10 Sep '21

Development Focus ----------------- The Release Candidate (RC) deadline is next Thursday, September 16. Work should be focused on fixing any release-critical bugs. General Information ------------------- All deliverables released under a cycle-with-rc model should have a first release candidate by the end of the week, from which a stable/xena branch will be cut. This branch will track the Xena release. Once stable/xena has been created, the master branch will be ready to switch to Yoga development. While the master branch will no longer be feature-frozen, please prioritize any work necessary for completing Xena plans. Release-critical bugfixes will need to be merged in the master branch first, then backported to the stable/xena branch before a new release candidate can be proposed. Actions ------- Early in the week, the release team will be proposing RC1 patches for all cycle-with-rc projects, using the latest commit from the master branch. If your team is ready to go for cutting RC1, please let us know by leaving a +1 on these patches. If there are still a few more patches needed before RC1, you can -1 the patch and update it later in the week with the new commit hash you would like to use. Remember, stable/xena branches will be created with this, so you will want to make sure you have what you need included to avoid needing to backport changes from the master branch (which will technically then be Yoga) to this stable branch for any additional RCs before the final release. The release team will also be proposing releases for any deliverable following a cycle-with-intermediary model that has not produced any Xena release so far. Finally, now is a good time to finalize release highlights. Release highlights help shape the messaging around the release and make sure that your work is properly represented. Upcoming Deadlines & Dates -------------------------- RC1 deadline: September 16 (R-3 week) Final RC deadline: September 30 (R-1 week) Final Xena release: October 6 -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud

1 0

[cyborg] RFE requested for Cyborg
by Brin Zhang(张百林) 10 Sep '21

10 Sep '21

Dear Release Team, We experienced strange freezes and timeouts for a long time on our python-cyborgclient need to update the lost feature of device profile 'description' [1] And we have merged this at 7, Sep, that we can update [2]'s hash then to publish it. [1] https://review.opendev.org/c/openstack/python-cyborgclient/+/803886 [2] https://review.opendev.org/c/openstack/releases/+/807624 brinzhang

3 2

[all]Introduction to venus which is the project of log management and has been contributed to the OpenStack community
by Liye Pang(逄立业) 10 Sep '21

10 Sep '21

Hello everyone, after feedback from a large number of operations and maintenance personnel in InCloud OpenStack, we developed the log management project “Venus” for the OpenStack projects and that has contributed to the OpenStack community. The following is an introduction to “Venus”. If there is interest in the community, we are interested in proposing it to become an official OpenStack project in the future. Background In the day-to-day operation and maintenance of large-scale cloud platform, the following problems are encountered： l Time-consuming for log querying while the server increasing to thousands. l Difficult to retrieve logs, since there are many modules in the platform, e.g. systems service, compute, storage, network and other platform services. l The large amount and dispersion of log make faults are difficult to be discovered. l Because of distributed and interaction between components of the cloud platform, and scattered logs between components, it will take more time to locate problems. About Venus According to the key requirements of OpenStack in log storage, retrieval, analysis and so on, we introduced Venus project, a unified log management module. This module can provide a one-stop solution to log collection, cleaning, indexing, analysis, alarm, visualization, report generation and other needs, which involves helping operator or maintainer to quickly solve retrieve problems, grasp the operational health of the platform, and improve the management capabilities of the cloud platform. Additionally, this module plans to use machine learning algorithms to quickly locate IT failures and root causes, and improve operation and maintenance efficiency. Application scenario Venus played a key role in the following scenarios： l Retrieval: Provide a simple and easy-to-use way to retrieve all log and the context. l Analysis: Realize log association, field value statistics, and provide multi-scene and multi-dimensional visual analysis reports. l Alerts：Convert retrieval into active alerts to realize the error finding in massive logs. l Issue location: Establish a chain relationship and knowledge graphs to quickly locate problems. Overall structure The architecture of log management system based on Venus and elastic search is as follows: Diagram 0: Architecture of Venus venus_api: API module，provide API、rest-api service. venus_manager: Internal timing task module to realize the core functions of the log system. Current progress The current progress of the Venus project is as follows: l Collection：Develop fluentd collection tasks based on collectd to read, filter, format and send plug-ins for OpenStack, operating systems, and platform services, etc. l Index：Dealing with multi-dimensional index data in elasticsearch, and provide more concise and comprehensive authentication interface to return query results. l Analysis：Analyzing and display the related module errors, Mariadb connection errors, and Rabbitmq connection errors. l Alerts：Develop alarm task code to set threshold for the number of error logs of different modules at different times, and provides alarm services and notification services. l Location：Develop the call chain analysis function based on global_requested series, which can show the execution sequence, time and error information, etc., and provide the export operation. l Management：Develop configuration management functions in the log system, such as alarm threshold setting, timing task management, and log saving time setting, etc. Application examples Two examples of Venus application scenarios are as follows. 1. The virtual machine creation operation was performed on the cloud platform and it was found that the virtual machine was not created successfully. First, we can find the request id of the operation and jump to the virtual machine creation call chain page. Then, we can query the calling process, view and download the details of the log of the call. 2. In the cloud platform, the error log of each module can be converted into alarms to remind the users. Further, we can retrieve the details of the error log and error log statistics. Next step The next step of the Venus project is as follows: * Collection：In addition to fluent, other collection plugins such as logstash will be integrated. * Analysis: Explore more operation and maintenance scenarios, and conduct statistical analysis and alarm on key data. * display: The configuration, analysis and alarm of Venus will be integrated into horizon in the form of plugin. * location: Form clustering log and construct knowledge map, and integrate algorithm class library to locate the root cause of the fault. Venus Project Registry Venus library: https://opendev.org/inspur/venus You can grab the source code using the following git command: git clone https://opendev.org/inspur/venus.git Venus Demo Youtu.be: <https://youtu.be/mE2MoEx3awM> https://youtu.be/mE2MoEx3awM

7 15

Quay.io availability in China
by Radosław Piliszek 10 Sep '21

10 Sep '21

Dears, I have a question (mostly to our readers from China) whether https://quay.io/ is freely accessible from there. Asking because Kolla is planning to switch from Docker Hub (which has some local mirror in China as far as we know) to Quay.io for public images. -yoctozepto

3 3

Horizon connection errors from object store
by Michel Niyoyita 10 Sep '21

10 Sep '21

Hello team , I am facing an issue when I am trying to connect to the object store containers on the horizon dashboad . Once click on containers it automatically disconnect. please find below logs I am getting and help for further analysis. [Thu Sep 09 06:35:22.185771 2021] [wsgi:error] [pid 167:tid 139887608641280] [remote 10.10.29.150:55130] Attempted scope to domain Default failed, will attempt to scope to another domain. [Thu Sep 09 06:35:22.572522 2021] [wsgi:error] [pid 167:tid 139887608641280] [remote 10.10.29.150:55130] Login successful for user "admin" using domain "Default", remote address 10.10.29.150. [Thu Sep 09 06:35:51.494815 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] REQ: curl -i http://ceph-mon2:8080/swift/v1?format=json&limit=1001 -X GET -H "X-Auth-Token: gAAAAABhOasqHFyB..." -H "Accept-Encoding: gzip" [Thu Sep 09 06:35:51.495140 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP STATUS: 401 Unauthorized [Thu Sep 09 06:35:51.495541 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP HEADERS: {'Content-Length': '119', 'X-Trans-Id': 'tx00000000000000000000f-006139ab44-9fc1a-default', 'X-Openstack-Request-Id': 'tx00000000000000000000f-006139ab44-9fc1a-default', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Thu, 09 Sep 2021 06:35:51 GMT', 'Connection': 'Keep-Alive'} [Thu Sep 09 06:35:51.495792 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP BODY: b'{"Code":"AccessDenied","RequestId":"tx00000000000000000000f-006139ab44-9fc1a-default","HostId":"9fc1a-default-default"}' [Thu Sep 09 06:35:51.498743 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] Unauthorized: /api/swift/containers/ [Thu Sep 09 06:35:52.924169 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] REQ: curl -i http://ceph-mon2:8080/swift/v1?format=json&limit=1001 -X GET -H "X-Auth-Token: gAAAAABhOasqHFyB..." -H "Accept-Encoding: gzip" [Thu Sep 09 06:35:52.924520 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP STATUS: 401 Unauthorized [Thu Sep 09 06:35:52.924789 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP HEADERS: {'Content-Length': '119', 'X-Trans-Id': 'tx000000000000000000010-006139ab48-9fc1a-default', 'X-Openstack-Request-Id': 'tx000000000000000000010-006139ab48-9fc1a-default', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Thu, 09 Sep 2021 06:35:52 GMT', 'Connection': 'Keep-Alive'} [Thu Sep 09 06:35:52.925034 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP BODY: b'{"Code":"AccessDenied","RequestId":"tx000000000000000000010-006139ab48-9fc1a-default","HostId":"9fc1a-default-default"}' [Thu Sep 09 06:35:52.929398 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] Unauthorized: /api/swift/containers/ [Thu Sep 09 06:35:52.935799 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:56016] Logging out user "admin". [Thu Sep 09 06:35:53.061489 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] Logging out user "". [Thu Sep 09 06:35:54.541593 2021] [wsgi:error] [pid 165:tid 139887608641280] [remote 10.10.29.150:55852] The request's session was deleted before the request completed. The user may have logged out in a concurrent request, for example. [Thu Sep 09 06:35:54.542896 2021] [wsgi:error] [pid 165:tid 139887608641280] [remote 10.10.29.150:55852] Bad Request: /api/swift/policies/ [Thu Sep 09 06:35:54.566055 2021] [wsgi:error] [pid 167:tid 139887608641280] [remote 10.10.29.150:55860] The request's session was deleted before the request completed. The user may have logged out in a concurrent request, for example. [Thu Sep 09 06:35:54.567130 2021] [wsgi:error] [pid 167:tid 139887608641280] [remote 10.10.29.150:55860] Bad Request: /api/swift/info/ (kolla-open1) stack@kolla-open1 :/var/lib/docker/volumes/kolla_logs/_data/horizon$ Michel

2 6

Cinder API timeout on single-control node
by Eugen Block 10 Sep '21

10 Sep '21

Hi *, since your last responses were quite helpful regarding rabbitmq I would like to ask a different question for the same environment. It's an older openstack version (Pike) running with only one control node. There already were lots of volumes (way more than 1000) in that cloud, but after adding a bunch more (not sure how many exactly) in one project the whole cinder api became extremely slow. Both horizon and CLI run into timeouts: [Wed Sep 01 13:18:52.109178 2021] [wsgi:error] [pid 60440] [client <IP>:58474] Timeout when reading response headers from daemon process 'horizon': /srv/www/openstack-dashboard/openstack_dashboard/wsgi/django.wsgi, referer: http://<control>/project/volumes/ [Wed Sep 01 13:18:53.664714 2021] [wsgi:error] [pid 13007] Not Found: /favicon.ico Sometimes the volume creation succeeds if you just retry, but it often fails. The dashboard shows a "504 gateway timeout" after two minutes (also after four minutes after I increased the timeout for the apache dashboard config). The timeout also shows even if I try to get into the volumes tab of an empty project. A couple of weeks ago I already noticed some performance issues with cinder api if there are lots of attached volumes, if there are many "available" volumes it doesn't seem to slow things down. But since then the total number of volumes has doubled. At the moment there are more than 960 attached volumes across all projects and more than 750 detached volumes. I searched the cinder.conf for any helpful setting but I'm not sure which would actually help. And since it's a production cloud I would like to avoid restarting services all the time just to try something. Maybe some of you can point me in the right direction? I would appreciate any help! If there's more information I can provide just let me know. Thanks! Eugen

2 2

[tc][operators][all] Querying operators about their use of TC tags
by Radosław Piliszek 10 Sep '21

10 Sep '21

Dear OpenStack Operators, As part of Technical Committee's (TC's) efforts for this cycle (Xena) [1] we are working on an analysis of the usefulness of the Tags framework [2]. To this end, a summarising spreadsheet with tag distribution and applicable notes was started by me. [3] As the Tags framework is aimed at operators, we (TC) have a question to you (operators) whether you are aware of the existence of the Tags framework and whether it is something that you actually use to make deployment decisions. Please reply to this mail if you use the Tags framework and let us know how. [1] https://etherpad.opendev.org/p/tc-xena-tracker [2] https://governance.openstack.org/tc/reference/tags/index.html [3] https://docs.google.com/spreadsheets/d/18GXibtdQnSkIwA7DsBvX9dPwbw9JH76AhhR… Kind regards, Radosław Piliszek (yoctozepto) OpenStack TC vice-chair

1 0

[cinder] review patches with FFE first
by Brian Rosmaita 09 Sep '21

09 Sep '21

For anyone who missed yesterday's cinder meeting, we are conveniently tracking the patches that have FFEs on this etherpad: https://etherpad.opendev.org/p/cinder-xena-ffe Please give them the highest review priority. cheers, brian

1 0

[all][elections][ptl] Project Team Lead Election Conclusion and Results
by Ian Y. Choi 09 Sep '21

09 Sep '21

Thank you to the electorate, to all those who voted and to all candidates who put their name forward for Project Team Lead (PTL) in this election. A healthy, open process breeds trust in our decision making capability thank you to all those who make this process possible. Now for the results of the PTL election process, please join me in extending congratulations to the following PTLs: * Barbican : Douglas Mendizábal * Blazar : Pierre Riteau * Cinder : Brian Rosmaita * Cloudkitty : Rafael Weingartner * Cyborg : Bailin Zhang * Designate : Michael Johnson * Ec2 Api : Andrey Pavlov * Freezer : ge cong * Glance : Abhishek Kekane * Horizon : Vishal Manchanda * Ironic : Iury Gregory Melo Ferreira * Kolla : Michal Nasiadka * Kuryr : Maysa de Macedo Souza * Magnum : Feilong Wang * Manila : Goutham Pacha Ravi * Masakari : Radosław Piliszek * Murano : Rong Zhu * Neutron : Lajos Katona * Nova : Sylvain Bauza * Octavia : Gregory Thiemonge * OpenStack Charms : Alex Kavanagh * Openstack Chef : Lance Albertson * OpenStack Helm : Gage Hugo * OpenStackAnsible : Dmitriy Rabotyagov * OpenStackSDK : Artem Goncharov * Quality Assurance : Martin Kopec * Rally : Andrey Kurilin * Release Management : Elod Illes * Requirements : Matthew Thode * Senlin : XueFeng Liu * Solum : Rong Zhu * Storlets : Takashi Kajinami * Swift : Tim Burke * Tacker : Yasufumi Ogawa * Telemetry : Matthias Runge * Tripleo : James Slagle * Trove : Lingxian Kong * Vitrage : Eyal Bar-Ilan * Watcher : chen ke * Winstackers : Lucian Petrut * Zaqar : wang hao Elections: * Cyborg: https://civs1.civs.us/cgi-bin/results.pl?id=E_807146df7d6a1d3b Election process details and results are also available here: https://governance.openstack.org/election/ Thank you to all involved in the PTL election process, Ian Y. Choi, on behalf of the OpenStack Technical Election Officials

3 2

[all] Dangerous trend toward too many release-independent components
by Thomas Goirand 09 Sep '21

09 Sep '21

Hi, As I'm packaging Xena in Debian, I've seen more and more components switching to release-independent. This poses a number of problems which I'm really unsure how to address. Let me explain. First of all, once an OpenStack release reaches a distribution stable release, there will be no update of it (ie: no new upstream release), unless there's a security problems, or important bugfixes that cannot be backported individually. At least, that's how Debian works, and why we call our suite "stable" (ie: it doesn't change). However, if a component is declared release-independent, the OpenStack community expects it to be updated for all downward releases. But that's not what's going on in reality. Also, if there's a security fix, we're supposed to get this through an update of a new version, which may include changes not suitable for a distribution update. The danger is that the update may be refused by the Debian Stable release team, for very good reasons (which I agree on). I'm also questioning myself: since we have pinned global requirements, will these new updates be included in the tests of older OpenStack releases? Is this really the case right now? So, while I do understand why it's nicer to set release-independent tags on some components, it is my opinion that this doesn't help stability and poses numerous problems to downstream distributions. Could we limit the number of such release-independent released components to the strict minimum low-profile (ie: with the least amount of code) components, rather than constantly increasing the list? Your thoughts? Cheers, Thomas Goirand (zigo)

4 12

[OSSA-2021-006] Neutron: Routes middleware memory leak for nonexistent controllers (CVE-2021-40797)
by Jeremy Stanley 09 Sep '21

09 Sep '21

======================================================================== OSSA-2021-006: Routes middleware memory leak for nonexistent controllers ======================================================================== :Date: September 09, 2021 :CVE: CVE-2021-40797 Affects ~~~~~~~ - Neutron: <16.4.1, >=17.0.0 <17.2.1, >=18.0.0 <18.1.1 Description ~~~~~~~~~~~ Slawek Kaplonski with Red Hat reported a vulnerability in Neutron's routes middleware. By making API requests involving nonexistent controllers, an authenticated user may cause the API worker to consume increasing amounts of memory, resulting in API performance degradation or denial of service. All Neutron deployments are affected. Patches ~~~~~~~ - https://review.opendev.org/807638 (Queens) - https://review.opendev.org/807637 (Rocky) - https://review.opendev.org/807636 (Stein) - https://review.opendev.org/807635 (Train) - https://review.opendev.org/807634 (Ussuri) - https://review.opendev.org/807633 (Victoria) - https://review.opendev.org/807632 (Wallaby) - https://review.opendev.org/807335 (Xena) Credits ~~~~~~~ - Slawek Kaplonski from Red Hat (CVE-2021-40797) References ~~~~~~~~~~ - https://launchpad.net/bugs/1942179 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-40797 Notes ~~~~~ - The stable/train, stable/stein, stable/rocky, and stable/queens branches are under extended maintenance and will receive no new point releases, but patches for them are provided as a courtesy. -- Jeremy Stanley

1 0

[neutron] Drivers meeting agenda - 10.09.2021
by Slawek Kaplonski 09 Sep '21

09 Sep '21

Hi, Agenda for our tomorrow's drivers meeting is at [1]. We have 3 RFEs to discuss: * https://bugs.launchpad.net/neutron/+bug/1830014 - [RFE] add API for neutron debug tool "probe" - there is also spec proposed for that one [2] * https://bugs.launchpad.net/neutron/+bug/1916428 - dibbler tool for dhcpv6 is concluded * https://bugs.launchpad.net/neutron/+bug/1870319 - [RFE] Network cascade deletion API call All of them are pretty old but we need to get back to them and hopefully decide something :) See You all on the meeting tomorrow. [1] https://wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda [2] https://review.opendev.org/c/openstack/neutron-specs/+/662541 -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[heat] Issue happend during server_group creation with policies
by ueha.ayumu＠fujitsu.com 09 Sep '21

09 Sep '21

Hi heat team. I reported the following bug report [1] on Sep 2. [1] https://storyboard.openstack.org/#!/story/2009164 Have you confirmed it? This error seems to appear to have occurred after patch [2] was merged. [2] https://review.opendev.org/c/openstack/heat/+/783971 Please kindly check it, thanks. Regards, Ayumu Ueha

1 0

Horizon connection errors from object store
by Michel Niyoyita 09 Sep '21

09 Sep '21

Hello team , I am facing an issue when I am trying to connect to the object store containers on the horizon dashboad . Once click on containers it automatically disconnect. please find below logs I am getting and help for further analysis. [Thu Sep 09 06:35:22.185771 2021] [wsgi:error] [pid 167:tid 139887608641280] [remote 10.10.29.150:55130] Attempted scope to domain Default failed, will attempt to scope to another domain. [Thu Sep 09 06:35:22.572522 2021] [wsgi:error] [pid 167:tid 139887608641280] [remote 10.10.29.150:55130] Login successful for user "admin" using domain "Default", remote address 10.10.29.150. [Thu Sep 09 06:35:51.494815 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] REQ: curl -i http://ceph-mon2:8080/swift/v1?format=json&limit=1001 -X GET -H "X-Auth-Token: gAAAAABhOasqHFyB..." -H "Accept-Encoding: gzip" [Thu Sep 09 06:35:51.495140 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP STATUS: 401 Unauthorized [Thu Sep 09 06:35:51.495541 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP HEADERS: {'Content-Length': '119', 'X-Trans-Id': 'tx00000000000000000000f-006139ab44-9fc1a-default', 'X-Openstack-Request-Id': 'tx00000000000000000000f-006139ab44-9fc1a-default', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Thu, 09 Sep 2021 06:35:51 GMT', 'Connection': 'Keep-Alive'} [Thu Sep 09 06:35:51.495792 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP BODY: b'{"Code":"AccessDenied","RequestId":"tx00000000000000000000f-006139ab44-9fc1a-default","HostId":"9fc1a-default-default"}' [Thu Sep 09 06:35:51.498743 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] Unauthorized: /api/swift/containers/ [Thu Sep 09 06:35:52.924169 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] REQ: curl -i http://ceph-mon2:8080/swift/v1?format=json&limit=1001 -X GET -H "X-Auth-Token: gAAAAABhOasqHFyB..." -H "Accept-Encoding: gzip" [Thu Sep 09 06:35:52.924520 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP STATUS: 401 Unauthorized [Thu Sep 09 06:35:52.924789 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP HEADERS: {'Content-Length': '119', 'X-Trans-Id': 'tx000000000000000000010-006139ab48-9fc1a-default', 'X-Openstack-Request-Id': 'tx000000000000000000010-006139ab48-9fc1a-default', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Thu, 09 Sep 2021 06:35:52 GMT', 'Connection': 'Keep-Alive'} [Thu Sep 09 06:35:52.925034 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] RESP BODY: b'{"Code":"AccessDenied","RequestId":"tx000000000000000000010-006139ab48-9fc1a-default","HostId":"9fc1a-default-default"}' [Thu Sep 09 06:35:52.929398 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] Unauthorized: /api/swift/containers/ [Thu Sep 09 06:35:52.935799 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:56016] Logging out user "admin". [Thu Sep 09 06:35:53.061489 2021] [wsgi:error] [pid 166:tid 139887608641280] [remote 10.10.29.150:55806] Logging out user "". [Thu Sep 09 06:35:54.541593 2021] [wsgi:error] [pid 165:tid 139887608641280] [remote 10.10.29.150:55852] The request's session was deleted before the request completed. The user may have logged out in a concurrent request, for example. [Thu Sep 09 06:35:54.542896 2021] [wsgi:error] [pid 165:tid 139887608641280] [remote 10.10.29.150:55852] Bad Request: /api/swift/policies/ [Thu Sep 09 06:35:54.566055 2021] [wsgi:error] [pid 167:tid 139887608641280] [remote 10.10.29.150:55860] The request's session was deleted before the request completed. The user may have logged out in a concurrent request, for example. [Thu Sep 09 06:35:54.567130 2021] [wsgi:error] [pid 167:tid 139887608641280] [remote 10.10.29.150:55860] Bad Request: /api/swift/info/ (kolla-open1) stack@kolla-open1 :/var/lib/docker/volumes/kolla_logs/_data/horizon$ [image: image.png] [image: image.png] Michel

1 0

[neutron][neutron-vpnaas] Propose to EOL stable/queens, stable/rocky and stable/stein branches or neutron-vpnaas
by Slawek Kaplonski 09 Sep '21

09 Sep '21

Hi, On our last meeting of the CI team we discussed the problem with broken stable branches (stein and older) in the neutron-vpnaas project [1]. I checked that last patches which were merged in those branches are more than year ago: * Queens - last merged patch in April 2019 * Rocky - last merged patch in February 2020 * Stein - last merged patch in September 2019 Giving the above and the current status of the CI in those branches, I propose to make them End Of Life. I will wait until end of next week for anyone who would like to maybe step up as maintainer of those branches and who would at least try to fix CI of them. If there will be no volunteers for that, I will EOL those branches in neutron- vpnaas. [1] https://meetings.opendev.org/meetings/neutron_ci/2021/neutron_ci. 2021-09-07-15.00.log.html#l-64 -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

Question about vDPA support for live migration
by Jeffrey Zhang 09 Sep '21

09 Sep '21

Hey everyone, I am testing vDPA on OpenStack Wallaby. I have set it up successfully. But I found there are several features that are not implemented, including live/code migration[0], which I wanted. Anyone know what's the latest progress about this? I found no talk about this. [0] https://review.opendev.org/c/openstack/nova/+/780866/3/releasenotes/notes/v… <https://signaldomn.online/click?redirect=https%3A%2F%2Freview.opendev.org%2…> --- Regards, Jeffrey Zhang Blog: http://xcodest.me <https://signaldomn.online/click?redirect=http%3A%2F%2Fxcodest.me%2F&dID=163…>

1 0

[neutron] Slow router provisioning during full resync of L3 Agent
by Patryk Jakuszew 09 Sep '21

09 Sep '21

Hello, I have a long-standing issue with L3 Agent which I would like to finally solve - *very* slow router provisioning in L3 Agent. We are operating a Rocky-based OpenStack deployment with three bare-metal L3 Agent nodes running in legacy mode. After restarting the L3 node, it takes a really long time for the L3 agent to become fully operational. There are two parts of resync which take much time: getting a list of routers from neutron-server and actually recreate them in the L3 node. While the long running time of router list retrieval is somewhat understandable, the router provisioning process itself proves to be very troublesome in our operations tasks. In our production deployment with around 250 routers, it takes around 2 hours (!) to recreate the router namespaces and have the L3 node fully functional again. Two hours of router re-provisioning is actually an optimistic scenario, this proved to be much longer during the outages we encountered (sometimes the sync took nearly 6-8 hours). This effectively prolongs any maintenance upgrades, configuration changes and OpenStack release upgrades. Another thing is, on that same production environment the first 100 routers usually get provisioned fast (around 30 minutes), after that it slows down with each router - this kind of non deterministic behavior makes it hard to communicate the maintenance finish ETA for our users. We also have a test environment with Stein already installed, where this problem is also present - full resync of 150 routers, with only one external gateway ports, takes around an hour to complete. Are there any operators here who also encountered that issue? Does anyone have any experience with similar situation and are willing to share their observations and optimizations? -- Regards, Patryk Jakuszew

3 3

[all][tc] Technical Committee next weekly meeting on Sept 9th at 1500 UTC
by Ghanshyam Mann 08 Sep '21

08 Sep '21

Hello Everyone, Technical Committee's next weekly meeting is scheduled for Sept 9th at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, Sept 8th, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[devstack][openstack-qa] DevStack on VirtualBox with Host-only Adapter
by os-user157 08 Sep '21

08 Sep '21

Hi all, I'm trying to deploy DevStack inside an Ubuntu Server 20.04.3 LTS virtual machine inside VirtualBox, but I'm struggling a lot, I got a lot of different errors. My use case: I want to test Zuul and DevStack, so I'll create a virtual machine for DevStack and one for Zuul, both on VirtualBox. I'm using a Host-only adapter with CIDR 192.168.200.0/24, and I need DevStack to be accessible on this network, and to use this network as the floating range for giving floating IP addresses to instances inside DevStack, so that Zuul can also access them. My local.conf: https://paste.openstack.org/show/809182/ Screenshots of the network configuration of my VirtualBox VM: First adapter, NAT (used for internet connectivity): https://i.imgur.com/C7XTiLJ.png Second adapter, host-only, used for communication with my Windows host and other VMs in the same network: https://i.imgur.com/Qeciuqx.png DevStack is erroring with the arping command on lib/neutron-legacy, arping gets 0 responses and exits the script with exit code 1 Does anyone have experience in dpeloying DevStack inside VirtualBox in such manner? I really need some help with this, if anyone could give me some ideas of waht could be wrong, it'd very much appreciated! Thanks.

1 0

Tenant vlan Network Issue
by Karera Tony 08 Sep '21

08 Sep '21

Hello Team, Whenever I create an internal network with type vlan. The Instances don't get IPs but for external Network it's fine. Below is the etc/kolla/config/neutron/ml2_conf.ini file [ml2] type_drivers = flat,vlan,vxlan tenant_network_types = vlan,flat mechanism_drivers = openvswitch,l2population [ml2_type_vlan] network_vlan_ranges = physnet1 [ml2_type_flat] flat_networks = physnet1 Is there something I need to change or I need to configure the interface differently Regards Tony Karera

4 12

[TripleO] Yoga PTG topics etherpad
by James Slagle 08 Sep '21

08 Sep '21

I've started an etherpad where we can collect topics for discussion at the upcoming PTG: https://etherpad.opendev.org/p/tripleo-yoga-topics Please add topics over the next couple of weeks, and then we'll work on a schedule. Thanks. -- -- James Slagle --

1 0