June 2022 - openstack-discuss - lists.openstack.org

[magnum] [xena] [IPv6] Review Patch for dual-stack IPv4/6 work in Magnum-managed Kubernetes 1.21
by Namrata Sitlani 27 Jun '22

27 Jun '22

Hello All, We run release Xena, and we have deployed Kubernetes version 1.21.10 on Magnum with Fedora CoreOS version 34 and we are trying to have IPv6 support for that Kubernetes cluster, as Kubernetes 1.21 and later versions have dual-stack(IPv4/IPv6) support is enabled by default. To achieve that, we also need IPv6 support in magnum, which is not currently supported but there has been a patch for "Support IPv6 dual subnet" in a pending state for almost a year. Following is the patch link: https://review.opendev.org/c/openstack/magnum/+/802235 We are interested in the use case for dual-stack support in magnum. We would highly appreciate it if someone could help us land that patch. Kindly let us know if we can provide any additional information or help for the same. Thanks, Namrata Sitlani

1 0

[vmware-nsx] Missing releases and project status
by Takashi Kajinami 25 Jun '22

25 Jun '22

Hello, I recently noticed that the vmware-nsx repo doesn't have the stable/wallaby branch and the stable/yoga branch. Also, no release has been created since victoria release. There is a bug fix recently merged directly to stable/xena, and this was backported to ussuri and train with victoria skipped. https://review.opendev.org/c/x/vmware-nsx/+/836034 I'm quite confused with this because it looks like master and victoria are no longer maintained. May I ask for some clarification about the current status of the project, especially regarding the following points. - Is there any plan to create stable/wallaby and stable/yoga ? - Is there any plan to create a release for stable/wallaby and later ? - May I know the maintenance status of each branch ? - What is the future plan of that repo? Will you have stable/zed release ? I'm asking this because the plugin is supported by puppet-neutron but we are thinking of deprecating/removing the implementation if the plugin will be unmaintained. I'd appreciate your input on this. Thank you, Takashi

1 1

[all][tc] What's happening in Technical Committee: summary 24 June 2022: Reading: 5 min
by Ghanshyam Mann 25 Jun '22

25 Jun '22

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We had this week's meeting on 23 June. Most of the meeting discussions are summarized in this email. Meeting full logs are available @https://meetings.opendev.org/meetings/tc/2022/tc.2022-06-23-15.00.log.html * Next TC weekly meeting will be on 30 July Thursday at 15:00 UTC, feel free to add the topic on the agenda[1] by 29 July. 2. What we completed this week: ========================= * Defined "Emerging technology and inactive projects" framework[2]. Details can be found here[4]. Thanks to Slawek for working on this framework. A brief about this framework: 1. When a new project applies to be an official OpenStack project and they are not fully ready/doing things as OpenStack way. Still, they can be an official OpenStack with the agreement of filling the gap for OpenStack way. This framework will help such projects to have them early in the OpenStack community but also with the timeline of filling the gaps. (Skyline is a good example of such a project). 2. Inactive projects: If any project become inactive and no maintainer merges the things then this framework will help to detect such projects before the final release of work and give time to recover before we decide to retire them. This will avoid last-minute issues during the final release for example: what we had in the Yoga release[3] * Clarified "ATC" and "AC" terms in TC charter as per the bylaws perspective[5] * Added cinder ibm storwize charm repo[6] 3. Activities In progress: ================== TC Tracker for Zed cycle ------------------------------ * Zed tracker etherpad includes the TC working items[7], Two are completed and others items are in-progress. Open Reviews ----------------- * Four open reviews for ongoing activities[8]. Create the Environmental Sustainability SIG --------------------------------------------------- There is a proposal for 'Environmental Sustainability ' SIG[9]. Because of its wider scope at the OpenInfra projects level (not just OpenStack) we are discussing if it is a good idea to be an OpenStack SIG or something at the OpenInfra level like a working group under Board or so. New ELK service dashboard: e-r service ----------------------------------------------- Updates from dpawlik: "Changes in ci log processing: created account on docker hub for ci log processing project (credentials are in AWS secret service); enabled pushing logscraper and log gearman container images into the docker hub." Ananya working on integrating the Elastic recheck with Upstream opensearch[10] Dasm working on merging 'origin/rdo' into the merge branch[11] Consistent and Secure Default RBAC ------------------------------------------- We discussed the operator feedback in the policy popup meeting on Tuesday 21 June[12] where operators from KDDI joined and wrote their feedback in etherpad[13]. After considering the feedback and all the challenges we have with 'scope', we decided to drop the 'scope' implementation and work on implementing the 'project persona'. I have proposed the goal document updates[14] 2021 User Survey TC Question Analysis ----------------------------------------------- No update on this. The survey summary is up for review[15]. Feel free to check and provide feedback. Zed cycle Leaderless projects ---------------------------------- No updates on this. Only Adjutant project is leaderless/maintainer-less. We will check Adjutant's the situation again on ML and hope Braden will be ready with their company side permission[16]. Fixing Zuul config error ---------------------------- Requesting projects with zuul config error to look into those and fix them which should not take much time[17][18]. Project updates ------------------- * Add Cinder Dell EMC PowerStore charm[19] * Retire openstack-helm-deployments[20] 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[21]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [22] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/839880 [3] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027864.ht… [4] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [5] https://review.opendev.org/c/openstack/governance/+/844882 [6] https://review.opendev.org/c/openstack/governance/+/846900 [7] https://etherpad.opendev.org/p/tc-zed-tracker [8] https://review.opendev.org/q/projects:openstack/governance+status:open [9] https://review.opendev.org/c/openstack/governance-sigs/+/845336 [10] https://review.opendev.org/c/opendev/elastic-recheck/+/845777 [11] https://review.opendev.org/c/opendev/elastic-recheck/+/847405/ [12] https://etherpad.opendev.org/p/rbac-zed-ptg#L171 [13] https://etherpad.opendev.org/p/rbac-operator-feedback#L45 [14] https://review.opendev.org/c/openstack/governance/+/847418 [15] https://review.opendev.org/c/openstack/governance/+/836888 [16] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027626.ht… [17] https://etherpad.opendev.org/p/zuul-config-error-openstack [18] http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028603.html [19] https://review.opendev.org/c/openstack/governance/+/846890 [20] https://review.opendev.org/c/openstack/governance/+/847413 [21] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [22] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann

1 0

[oslo] Propose to EOL stable/queens, stable/rocky on all the oslo scope
by Herve Beraud 24 Jun '22

24 Jun '22

Hi, On our last meeting of the oslo team we discussed the problem with broken stable branches (rocky and older) in oslo's projects [1]. Indeed, almost all these branches are broken. Előd Illés kindly generated a list of periodic-stable errors on Oslo's stable branches [2]. Given the lack of active maintainers on Oslo and given the current status of the CI in those branches, I propose to make them End Of Life. I will wait until the end of month for anyone who would like to maybe step up as maintainer of those branches and who would at least try to fix CI of them. If no one will volunteer for that, I'll EOLing those branches for all the projects under the oslo umbrella. Let us know your thoughts. Thank you for your attention. [1] https://meetings.opendev.org/meetings/oslo/2021/oslo.2021-10-04-15.00.log.t… [2] http://lists.openstack.org/pipermail/openstack-discuss/2021-July/023939.html -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud

5 5

ovn-controller/OVS stranger behaviour
by Tiago Pires 24 Jun '22

24 Jun '22

Hi all, I'm trying to understand a stranger's behaviour regarding to ovn-controller/OVS. In my setup I have OVN 21.09/ OVS 2.16 and Ubuntu Xena and sometimes when a new VM is created, this VM can reach other VMs in east-west traffic (even in differents Chassis) but it can't reach an external network (e.g. Internet) through Chassi Gateway. I ran the following trace: # ovs-appctl ofproto/trace br-int in_port="93",icmp,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.140,nw_dst=8.8.8.8,nw_ttl=64 And I got this output: Final flow: recirc_id=0xc157b1,eth,icmp,reg0=0x300,reg11=0xd,reg12=0x10,reg13=0xf,reg14=0x3,reg15=0x2,metadata=0x29,in_port=93,vlan_tci=0x0000,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.140,nw_dst=8.8.8.8,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=0,icmp_code=0 Megaflow: recirc_id=0xc157b1,ct_state=+new-est-rel-rpl-inv+trk,ct_label=0/0x1,eth,icmp,in_port=93,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src= 192.168.40.128/26,nw_dst=8.0.0.0/7,nw_ttl=64,nw_frag=no Datapath actions: ct(commit,zone=15,label=0/0x1,nat(src)),set(eth(src=fa:16:3e:ec:7f:dd,dst=00:00:00:00:00:00)),set(ipv4(ttl=63)),userspace(pid=3451843211,controller(reason=1,dont_send=1,continuation=0,recirc_id=12670898,rule_cookie=0x3e26215e,controller_id=0,max_len=65535)) It seems the Datapath is querying the controller and I did not understand the reason. So, I did an ovn-controller recompute (ovn-appctl -t ovn-controller recompute) on the Chassi where the VM is placed to check if it could change the behaviour and I could trace the packet with success and the VM started to communicate with the Internet normally: Final flow: recirc_id=0x2,eth,icmp,reg0=0x300,reg11=0xd,reg12=0x10,reg13=0xf,reg14=0x3,reg15=0x2,metadata=0x29,in_port=93,vlan_tci=0x0000,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.140,nw_dst=8.8.8.8,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=0,icmp_code=0 Megaflow: recirc_id=0x2,ct_state=+new-est-rel-rpl-inv+trk,ct_label=0/0x1,eth,icmp,tun_id=0/0xffffff,tun_metadata0=NP,in_port=93,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src= 192.168.40.128/26,nw_dst=8.0.0.0/7,nw_ecn=0,nw_ttl=64,nw_frag=no Datapath actions: ct(commit,zone=15,label=0/0x1,nat(src)),set(tunnel(tun_id=0x2a,dst=10.X6.X3.133,ttl=64,tp_dst=6081,geneve({class=0x102,type=0x80,len=4,0x30002}),flags(df|csum|key))),set(eth(src=fa:16:3e:ec:7f:dd,dst=00:00:5e:00:04:00)),set(ipv4(ttl=63)),2 The Datapath action is using the tunnel with the Chassi Gateway. It happens always with new VMs but sometimes. After running the recompute on the Chassi, I created additional VMs and this issue did not happen. In my Chassi I have enable these parameters also: ovn-monitor-all="true" ovn-openflow-probe-interval="0" ovn-remote-probe-interval="180000" I did some troubleshooting and I'm seeing this error (ovs-vswitchd) always when a VM is created in a Chassi: 2022-06-23T11:47:08.385Z|07907|bridge|WARN|could not open network device tap8a43df0c-fd (No such device) 2022-06-23T11:47:09.282Z|07908|bridge|INFO|bridge br-int: added interface tap8a43df0c-fd on port 51 2022-06-23T11:47:09.645Z|07909|bridge|INFO|bridge br-int: added interface tap3200bf1c-20 on port 52 2022-06-23T11:47:19.329Z|07911|connmgr|INFO|br-int<->unix#1468: 430 flow_mods in the 7 s starting 10 s ago (410 adds, 20 deletes) On this commit http://patchwork.ozlabs.org/project/ovn/patch/1608197000-637-1-git-send-ema… it solved something similar to my issue. It seems the ovs-vswitchd is missing some flows and when I run the recompute it fixes it. So, in order to avoid this issue I'm testing at this moment to run the recompute through libvirt hook when a VM gets "started" status. Do you know this behaviour could be bug related? Regards, Tiago Pires Do you know this behaviour could be bug related?

2 2

[heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team
by Yoshito Ito 24 Jun '22

24 Jun '22

Hi heat-translator team! I would like to propose Hiromu Asahina (h-asahina) as a member of the heat-translator core team. I'm now not able to actively contribute to heat-translator, and he can lead the team instead of me. He is mainly contributing to Tacker by fixing issues, refactoring existing codes, and a lot of helpful reviews, with his experience as a research engineer in NTT. Tacker highly depends on heat-translator in its main functionality, so I believe he can manage it with his knowledge of Tacker. Please vote for his nomination by answering this mail till June 1st. Best Regards, Yoshito Ito

7 10

[ER] Revive elastic recheck
by Dariusz Smigiel 24 Jun '22

24 Jun '22

Hey Team! I started cherry picking changes from the rdo branch back to the master branch for Elastic Recheck. [1] I'm planning to continue doing so, to revive the master branch and continue development in the main branch. Some of the changes are incompatible between releases. We're gonna clear them case by case to enable it in the future. Thanks, Darius [1]: https://review.opendev.org/c/opendev/elastic-recheck/+/847259

1 1

[all][tc] Technical Committee next weekly meeting on 23 June 2022 at 1500 UTC
by Ghanshyam Mann 23 Jun '22

23 Jun '22

Hello Everyone, The technical Committee's next weekly meeting is scheduled for 23 June 2022, at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, 22 June at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

Swift issues in one cluster
by Albert Braden 23 Jun '22

23 Jun '22

We have multiple Swift clusters, all configured the same. One of them started failing this week. The symptom is that Swift commands take a long time to execute and sometimes they fail: $ openstack container list Unable to establish connection to https://swift.<region>.<domain>:8080/v1/AUTH_<project>: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer') When we look at the logs we see lots of swift-proxy-server errors: (from Splunk): Payload: swift-proxy-server: STDERR: File "/usr/lib64/python3.6/socket.py", line 604, in write#012 return self._sock.send(b) Payload: swift-proxy-server: STDERR: BlockingIOError Payload: swift-proxy-server: STDERR: os.read(self.rfd, 1) Payload: swift-proxy-server: STDERR: File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 818, in process_request#012 proto.__init__(conn_state, self) Payload: swift-proxy-server: STDERR: File "/usr/lib/python3.6/site-packages/eventlet/greenio/base.py", line 397, in send Payload: swift-proxy-server: STDERR: return self._sock.send(b) When we look at network connections, we see haproxy stacking up (many lines of this): # netstat -ntup | sort -b -k2 -n -r | head -n +100 tcp 5976932 0 127.0.0.1:60738 127.0.0.1:8080 ESTABLISHED 13045/haproxy tcp 5976446 0 127.0.0.1:58480 127.0.0.1:8080 ESTABLISHED 13045/haproxy tcp 5973217 0 127.0.0.1:33244 127.0.0.1:8080 ESTABLISHED 13045/haproxy tcp 5973120 0 127.0.0.1:51836 127.0.0.1:8080 ESTABLISHED 13045/haproxy tcp 5971968 0 127.0.0.1:58516 127.0.0.1:8080 ESTABLISHED 13045/haproxy ... If we restart the swift_haproxy and swift_proxy_server containers then the problem goes away, and comes back over a few minutes. Where should we be looking for the root cause of this issue?

2 4

[TripleO] Proposing Brendan Shephard and Takashi Kajinami
by James Slagle 23 Jun '22

23 Jun '22

I'm formally proposing that we add Brendan Shephard and Takashi Kajinami to the core reviewers team for TripleO. Takashi is already considered core on puppet-tripleo, and I'm proposing we expand that to all of TripleO, and likewise for Brendan. They both have been contributing and reviewing for a little while now, and I feel it's time to add them to the team. Please +1 or let me know any concerns before Friday June 15th. Thanks! -- -- James Slagle --

15 15