- openstack-discuss - lists.openstack.org

IBM z/VM CI is planning to migrate new environment
by Dong JV Ma 07 Jan '20

07 Jan '20

Hello, Due to the hardware issues, the IBM z/VM CI system [1] is planning to migrate to the new hardware in the next few months, the planning time will be in 3 months. We will do following changes with IBM z/VM CI environment: 1. Migrate the IBM z/VM CI to new hardware 2. Upgrade IBM z/VM CI from python2 to python3 During this time, IBM z/VM CI will not stable, please ignore the IBM z/VM CI failures. For any questions feel free to reach out to me(Dong Ma). [1] https://wiki.openstack.org/wiki/ThirdPartySystems/IBM_z/VM_CI Thanks & Best Regards! Dong Ma (马冬) LinuxONE IaaS Infra & CI/CD, China Systems Lab E-mail: madongma(a)cn.ibm.com IRC: larainema Address: 2/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC

3 2

[tc] Expediting patch to unblock governance's CI
by Jean-Philippe Evrard 07 Jan '20

07 Jan '20

Hello everyone, Our governance repo's testing is currently failing, and I have patches to fix it (short term [1] and long term). Because it's been a while it's in there (and got recently updated), I will now merge the short term fix, even if we don't have enough votes (nor time). It will unblock many patches and prevent useless rechecks, at the expense of being out of policy. I will gladly take the blame, should there be any. Regards, JP [1] https://review.opendev.org/#/c/700422/

1 0

[kolla] neutron-l3-agent namespace NAT table not working?
by Jon Masters 06 Jan '20

06 Jan '20

Hi there, I've got a weird problem with the neutron-l3-agent container on my deployment. It comes up, sets up the iptables rules in the qrouter namespace (and I can see these using "ip netns...") but traffic isn't having DNAT or SNAT applied. What's most strange is that manually adding a LOG jump target to the iptables nat PRE/POSTROUTING chains (after enabling nf logging sent to the host kernel, confirmed that works) doesn't result in any log entries. It's as if the nat table isn't being applied at all for any packets traversing the qrouter namespace. This is driving me crazy :) Anyone got some quick suggestions? (assume I tried the obvious stuff). Jon. -- Computer Architect

8 19

[Scientific] Scientific SIG meeting July 7th 2100 UTC
by Martial Michel 06 Jan '20

06 Jan '20

The first meeting of the new year at 2100 UTC on Tuesday, July 7th. Mostly an Any Other Business meeting to get its participants back from the holidays as reflected by the agenda :) https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_January_7th_2020

1 0

[all] Is there something I can do to get a simple fix done?
by Neil Jerram 06 Jan '20

06 Jan '20

I'm struggling to say this positively, but... it feels like OpenStack promotes refactoring work that will likely break something, but is very slow when a corresponding fix is needed, even when the fix is trivial. Is there something we could do to get fixes done more quickly when needed? My case in point: my team's networking plugin (networking-calico) does not do "extraroutes", and so was broken by some python-openstackclient change (possibly [1]) that wrongly assumed that. I posted a fix [2] that passed CI on 21st October, and asked for it to be reviewed on IRC a couple of days later. Édouard Thuleau posted a similar fix [3] on 4th December, and we agreed that his was better, so I abandoned mine. His fix attracted a +2 on 11th December, but has been sitting like that ever since. It's a fix that I would expect to be simple to review, so I wonder if there's something else we could have done here to get this moving? Or if there is a systemic problem here that deserves discussion? [1] https://opendev.org/openstack/python-openstackclient/commit/c44f26eb7e41c28… [2] https://review.opendev.org/#/c/685312/ [3] https://review.opendev.org/#/c/697240/ Many thanks, Neil

2 1

[all][neutron][neutron-vpnaas] Maintainers needed
by Slawek Kaplonski 06 Jan '20

06 Jan '20

Hi, Over the past couple of cycles we have noticed that new contributions and maintenance efforts for neutron-vpnaas were almost non existent. This impacts patches for bug fixes, new features and reviews. The Neutron core team is trying to at least keep the CI of this project healthy, but we don’t have enough knowledge about the details of the neutron-vpnaas code base to review more complex patches. During the PTG in Shanghai we discussed that with operators and TC members during the forum session [1] and later within the Neutron team during the PTG session [2]. During these discussions, with the help of operators and TC members, we reached the conclusion that we need to have someone responsible for maintaining project. This doesn’t mean that the maintainer needs to spend full time working on this project. Rather, we need someone to be the contact person for the project, who takes care of the project’s CI and review patches. Of course that’s only a minimal requirement. If the new maintainer works on new features for the project, it’s even better :) If we don’t have any new maintainer(s) before milestone Ussuri-2, which is Feb 10 - Feb 14 according to [3], we will need to mark neutron-vpnaas as deprecated and in “V” cycle we will propose to move the project from the Neutron stadium, hosted in the “openstack/“ namespace, to the unofficial projects hosted in the “x/“ namespace. So if You are using this project now, or if You have customers who are using it, please consider the possibility of maintaining it. Otherwise, please be aware that it is highly possible that the project will be deprecated and moved out from the official OpenStack projects. [1] https://etherpad.openstack.org/p/PVG-Neutron-stadium-projects-the-path-forw… [2] https://etherpad.openstack.org/p/Shanghai-Neutron-Planning-restored - Lines 379-421 [3] https://releases.openstack.org/ussuri/schedule.html -- Slawek Kaplonski Senior software engineer Red Hat

1 1

[neutron] Bug deputy report - week of 30th December
by Slawek Kaplonski 06 Jan '20

06 Jan '20

Hi, I was on bug deputy last week. It was pretty quiet week with only few bugs reported. Below is my summary of it. Critical: https://bugs.launchpad.net/neutron/+bug/1858260 - Upstream CI neutron-tempest-plugin-* fails - I marked it as critical as it cause gate failures, I will have to take a look at it closer next week, Medium: https://bugs.launchpad.net/neutron/+bug/1858086 - qrouter's local link route cannot be restored - confirmed by me on local env, it would be good if someone from L3 subteam can take a look at it, Undecided and others: https://bugs.launchpad.net/neutron/+bug/1858377 - probably bug for openstackclient rather than neutron, but I would like to wait for confirmation from bug reporter first, https://bugs.launchpad.net/neutron/+bug/1858262 - duplicate of other bug, https://bugs.launchpad.net/neutron/+bug/1858419 - docs bug to update config options for large scale deployments, see http://lists.openstack.org/pipermail/openstack-discuss/2020-January/011820.… for details — Slawek Kaplonski Senior software engineer Red Hat

1 0

[oslo][kolla][requirements][release][infra] Hit by an old, fixed bug
by Radosław Piliszek 06 Jan '20

06 Jan '20

Hi Folks, as the subject goes, my installation has been hit by an old bug: https://bugs.launchpad.net/oslo.messaging/+bug/1828841 (bug details not important, linked here for background) I am using Stein, deployed with recent Kolla-built source-based images (with only slight modifications compared to vanilla ones). Kolla's procedure for building source-based images considers upper constraints, which, unfortunately, turned out to be lagging behind a few releases w.r.t. oslo.messaging at least. The fix was in 9.7.0 released on May 21, u-c still point to 9.5.0 from Feb 26 and the latest of Stein is 9.8.0 from Jul 18. It seems oslo.messaging is missing from the automatic updates that bot proposes: https://review.opendev.org/#/q/owner:%22OpenStack+Proposal+Bot%22+project:o… Per: https://opendev.org/openstack/releases/src/branch/master/doc/source/referen… this upper-constraint proposal should be happening for all releases. I would be glad if someone investigated why it happens(/ed) and audited whether other OpenStack projects don't need updating as well to avoid running on old deps when new are awaiting for months. :-) Please note this might apply to other branches as well. PS: for some reason oslo.messaging Stein release notes ( https://docs.openstack.org/releasenotes/oslo.messaging/stein.html ) are stuck at 9.5.0 as well, this could be right (I did not inspect the sources) but I am adding this in PS so you have more things to correlate if they need be. -yoctozepto

5 7

[kolla] Adding Dincer Celik to kolla-core and kolla-ansible-core
by Mark Goddard 06 Jan '20

06 Jan '20

Hi, I recently proposed to the existing cores that we add Dincer Celik (osmanlicilegi) to the kolla-core and kolla-ansible-core groups and we agreed to go ahead. Thanks for your contribution to the project so far Dincer, I'm glad to have you on the team. Cheers, Mark

1 0

[neutron][nova][cinder][glance][largescale-sig] Documentation update for large-scale
by Arnaud Morin 06 Jan '20

06 Jan '20

Hey all, With the new "Large scale SIG", we were thinking about updating documentation to help operators setting up large deployments. To do so, we would like to propose, at least, some documentation changes to identify options that affect large scale. The plan is to have a small note on some options and eventually a link to a specific page for large scale (see attachments). I know that nova started working on collecting those parameters here: https://bugs.launchpad.net/nova/+bug/1838819 Do you know if something similar exists on other projects? Moreover, we would like to collect more parameters that could be tuned on a large scale deployment, for every project. So, if you have any, feel free to answer to this mail or add some info on the following etherpad: https://etherpad.openstack.org/p/large-scale-sig-documentation Thanks for your help! Regards. The large-scale team. -- Arnaud Morin

2 1

[cloudkitty] Stepping down from PTL
by Luka Peschke 06 Jan '20

06 Jan '20

Hi all, I'm moving to a new position that doesn't involve OpenStack, and won't leave me the required time to be Cloudkitty's PTL. This is why I have to step down from the PTL position. jferrieu will take my position for the end of the U cycle (he's been a major contributor recently), with the help of huats, who's been the Cloudkitty PTL before me, and has been around in the community for a long time. I've been the PTL for two and a half cycles, and I think that it is a good thing for the project to take a new lead, with a new vision. I'm grateful for my experience within the OpenStack community. Thanks, -- Luka Peschke (peschk_l)

3 2

[auto-scaling][self-healing] Discussion to merge two SIG to one
by Rico Lin 05 Jan '20

05 Jan '20

Dear all As we discussed in PTG about merge two SIG to one. I would like to continue the discussion on ML. In PTG, Eric proposes the idea to merge two SIG due to the high overlapping of domains and tasks. I think this is a great idea since, over the last 6 months, most of the discussions in both SIG are overlapped. So I'm onboard with this idea. Here's how I think we can continue this idea: 1. Create new SIG (maybe 'Automation SIG'? feel free to propose name which can cover both interest.) 2. Redirect docs and wiki to new SIG. And rework on index so there will be no confusion 3. Move repos from both SIGs to new SIG 4. Mark auto-scaling SIG and self-healing SIG as inactive. 5. remove auto-scaling SIG and self-healing SIG after a reasonable waiting time Let us know what you think about this. Otherwise, we definitely expect this to happen soon. -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

4 7

[meta-sig][multi-arch] propose forming a Multi-arch SIG
by Rico Lin 05 Jan '20

05 Jan '20

Dear all In summit, there's a forum for ARM support [1] in Summit which many people show they're interested in ARM support for OpenStack. And since also we have Linaro shows interest in donating servers to OpenStack infra. It's time for community to think about what we should deal with those ARM servers once we have them in community infrastructure. One thing we should do as a community is to gather people for this topic. So I propose we create a Multi-arch SIG and aim to support ARM architecture as very first step. I had the idea to call it ARM SIG before, but since there might be high overlap knowledge between support ARM 64 and other architectures. I propose we go for Multi-arch instead. This SIG will be a nice place to collect all the documents, gate jobs, and to trace tasks. If you're also interested in that group, please reply to this email, introduce yourself and tell us what you would like the group scope and objectives to be, and what you can contribute to the group. [1] https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/24355… -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

11 15

About the use of security groups with neutron ports
by ahmed.zaky.abdallah＠gmail.com 04 Jan '20

04 Jan '20

Hi All, I am trying to wrap my head around something I came across in one of the OpenStack deployments. I am running Telco VNFs one of them is having different VMs using SR-IOV interfaces. On one of my VNFs on Openstack, I defined a wrong IPv6 Gm bearer interface to be exactly the same as the IPv6 Gateway. As I hate re-onboarding, I decided to embark on a journey of changing the IPv6 of the Gm bearer interface manually on the application side, everything went on fine. After two weeks, my customer started complaining about one way RTP flow. The customer was reluctant to blame the operation I carried out because everything worked smooth after my modification. After days of investigation, I remembered that I have port-security enabled and this means AAP “Allowed-Address-Pairs” are defined per vPort (AAP contain the floating IP address of the VM so that the security to allow traffic to and from this VIP). I gave it a try and edited AAP “Allowed-Address-Pairs” to include the correct new IPv6 address. Doing that everything started working fine. The only logical explanation at that time is security group rules are really invoked. Now, I am trying to understand how the iptables are really invoked. I did some digging and it seems like we can control the firewall drivers on two levels: * Nova compute * ML2 plugin I was curious to check nova.conf and it has already the following line: firewall_driver=nova.virt.firewall.NoopFirewallDriver However, checking the ml2 plugin configuration, the following is found: 230 [securitygroup] 231 232 # 233 # From neutron.ml2 234 # 235 236 # Driver for security groups firewall in the L2 agent (string value) 237 #firewall_driver = <None> 238 firewall_driver = openvswitch So, I am jumping to a conclusion that ml2 plugin is the one responsible for enforcing the firewall rules in my case. Have you had a similar experience? Is my assumption correct: If I comment out the ml2 plugin firewall driver then the port security carries no sense at all and security groups won’t be invoked? Cheers, Ahmed

4 4

[stein][cinder][backup][swift] issue
by Ignazio Cassano 03 Jan '20

03 Jan '20

Hello All, I configured openstack stein on centos 7 witch ceph. Cinder works fine and object storage on ceph seems to work fine: I can clreate containers, volume etc ..... I configured cinder backup on swift (but swift is using ceph rados gateway) : backup_driver = cinder.backup.drivers.swift.SwiftBackupDriver swift_catalog_info = object-store:swift:publicURL backup_swift_enable_progress_timer = True #backup_swift_url = http://10.102.184.190:8080/v1/AUTH_ backup_swift_auth_url = http://10.102.184.190:5000/v3 backup_swift_auth = per_user backup_swift_auth_version = 1 backup_swift_user = admin backup_swift_user_domain = default #backup_swift_key = <None> #backup_swift_container = volumebackups backup_swift_object_size = 52428800 #backup_swift_project = <None> #backup_swift_project_domain = <None> backup_swift_retry_attempts = 3 backup_swift_retry_backoff = 2 backup_compression_algorithm = zlib If I run a backup as user admin, it creates a container named "volumebackups". If I run a backup as user demo and I do not specify a container name, it tires to write on volumebackups and gives some errors: ClientException: Container PUT failed: http://10.102.184.190:8080/swift/v1/AUTH_964f343cf5164028a803db91488bdb01/v… 409 Conflict BucketAlreadyExists Does it mean I cannot use the same containers name on differents projects ? My ceph.conf is configured for using keystone: [client.rgw.tst2-osctrl01] rgw_frontends = "civetweb port=10.102.184.190:8080" # Keystone information rgw keystone api version = 3 rgw keystone url = http://10.102.184.190:5000 rgw keystone admin user = admin rgw keystone admin password = password rgw keystone admin domain = default rgw keystone admin project = admin rgw swift account in url = true rgw keystone implicit tenants = true Any help, please ? Best Regards Ignazio

3 4

[openstack-ansible] strange execution delays
by Joe Topjian 02 Jan '20

02 Jan '20

Hello, I don't think this is an OSA issue per-se, but it's happening on infrastructure deployed with OSA. Using both the Rocky and Stein releases of OSA on Ubuntu 18.04, we're running into a strange a strange issue where the run times of Ansible will eventually take longer and longer. Digging into the issue more, we find that it's when Ansible executes a command within a container and the call to "su" triggers the delay. Over time, the delay increases more and more. The same behavior is seen if we SSH into containers. The only workaround we've been able to identify that works is to stop and start the containers. We have not been able to identify a root cause of this, nor any consistency to what containers this will happen on. Has anyone run into this? Thanks, Joe

2 5

Re: openstack-discuss Digest, Vol 15, Issue 1
by Soniya Vyas 02 Jan '20

02 Jan '20

> Message: 4 > Date: Tue, 31 Dec 2019 19:03:22 -0600 > From: Ghanshyam Mann <gmann(a)ghanshyammann.com> > To: "openstack-discuss" <openstack-discuss(a)lists.openstack.org> > Subject: [qa] QA Office hour new timing for 2020 > Message-ID: > <16f5ea0ef6f.1010065b175496.3744470588375359656(a)ghanshyammann.com> > Content-Type: text/plain; charset="UTF-8" > > Hello Everyone, > > We have few members who actively started contribution to QA from India as well as from Europe TZ. > The current office hour time is more convenient for CT and JST only and very difficult for members from India > Europe to join. > > I would like to adjust office hour timing to include all four TZs. To do that someone has to wake up early or stay > late at night :). I gave preference to new members and selected Tuesday 13.30 UTC [1]. which will be an early morning > for me and late-night in Tokyo. > > let me know if any objection or better suggestion. Accordingly, I will make the new time effective from 7th Jan. > > Also cancelling the 1st Jan office hour and wish you all a very happy new year. > > [1] https://www.timeanddate.com/worldclock/meetingdetails.html?year=2020&month=… Thanks a lot to whole QA team and Ghanshyam Mann for considering our timing issues. Looking forward to join QA Office hours meeting. Regards, Soniya Vyas

1 0

[manila] No IRC/Community meeting on 2nd January 2020
by Goutham Pacha Ravi 01 Jan '20

01 Jan '20

Hello Zorillas, Due to the holidays we're expecting multiple community members (myself included) to be unavailable for the weekly IRC meeting tomorrow (2nd January 2020, 15:00 UTC), so we'll skip it. Please add any agenda items to the next meeting (9th January 2020, 15:00 UTC) [1]. Happy New Year, and the rest of your holidays! Thanks, Goutham [1] https://wiki.openstack.org/wiki/Manila/Meetings

1 0

[qa] QA Office hour new timing for 2020
by Ghanshyam Mann 31 Dec '19

31 Dec '19

Hello Everyone, We have few members who actively started contribution to QA from India as well as from Europe TZ. The current office hour time is more convenient for CT and JST only and very difficult for members from India Europe to join. I would like to adjust office hour timing to include all four TZs. To do that someone has to wake up early or stay late at night :). I gave preference to new members and selected Tuesday 13.30 UTC [1]. which will be an early morning for me and late-night in Tokyo. let me know if any objection or better suggestion. Accordingly, I will make the new time effective from 7th Jan. Also cancelling the 1st Jan office hour and wish you all a very happy new year. [1] https://www.timeanddate.com/worldclock/meetingdetails.html?year=2020&month=… -gmann

1 0

[neutron] Slow floating IP activation when FWaaS v2 enabled
by Jason Anderson 31 Dec '19

31 Dec '19

Hi all, We recently started using the FWaaS feature (v2) in Neutron for our deployment. However, we noticed that when this feature is enabled, floating IP activation (the setting up of iptables rules that perform DNAT to the destination instance) takes a long time to happen. The effect is that the user has to wait for a while before the floating IP becomes active on an instance. I was wondering if anybody else has experienced this, and if there are any workarounds. I suspect there may be some queuing happening behind an iptables lock somewhere but I haven't been able to verify anything for sure yet. Thank you, -- Jason Anderson Chameleon DevOps Lead Consortium for Advanced Science and Engineering, The University of Chicago Mathematics & Computer Science Division, Argonne National Laboratory

1 0

[horizon][requirements] django-3 blocked by usage of django-babel
by Matthew Thode 31 Dec '19

31 Dec '19

it looks like django-babel is unmaintained since 2017, so perhaps it's time to look for a replacement. Looking through github issues, it looks like enmerkar is a possible replacement. If that's fine with horizon / django consumers we can start by adding enmerkar to global-requirements and upper-constraints. Thanks, -- Matthew Thode

4 9

[neutron] Bug deputy report week of December 23rd
by Miguel Lavalle 30 Dec '19

30 Dec '19

Hi, I was the bug deputy for the week of December 23rd. As expected for an end of year Holidays week, We had very little activity: Requested more information: - https://bugs.launchpad.net/neutron/+bug/1857422 neutron-keepalived-state-change and keeplived cannot be cleanup for those routers which is deleted during l3-agent died

2 1

Openstack Rocky to Train upgrade
by Md. Farhad Hasan Khan 30 Dec '19

30 Dec '19

Hi, Could you please help me how can I upgrade from Openstack Rocky version to Train. Thanks & B'Rgds, Rony

3 5

[devstack][ceilometer][tacker]ceilometer devstack installation fails
by Hiroyuki JO 25 Dec '19

25 Dec '19

Hi ceilometer team and all, After a patch[1] was merged, Tacker zuul check fails because ceilometer devstack installation error[2]. Do you have any work aroud of this problem? Addition, why devstack uses pip version 9.0.3 when installing python packages? The cause of our problem is installation error of conluent-kafka. But with latest pip (version 19), I was able to install the package. [1] https://opendev.org/openstack/requirements/commit/c99ab09bc44818b92fe86842f… [2] https://zuul.opendev.org/t/openstack/build/e6d04b35d16e4447901aba441721115a… Regards, 徐広幸 (Hiroyuki JO) Email: hiroyuki.jo.mt(a)hco.ntt.co.jp / Tel(direct): +81-422-59-7394 NTT Network Systems Laboratories

3 5

[cyborg] No weekly IRC meeting for next 2 weeks
by Nadathur, Sundar 23 Dec '19

23 Dec '19

There will be no weekly Cyborg IRC meeting on Dec 26 or Jan 2. Happy Holidays to all. Regards, Sundar

1 0