- openstack-discuss - lists.openstack.org

[Octavia] Help with bridge interfaces with openstack queens in a existing neutron openvswitch dvr snat environment
by Bujack, Stefan 05 May '20

05 May '20

Hello, I am trying to install octavia in our existing Openstack Queens Cloud and have problems with the lb-mgmt network ports. Is there anybody who could please help me with this issue? Thanks in advance, Greets Stefan Bujack

2 1

[puppet] Configuring Debian's uwsgi for each service
by Thomas Goirand 05 May '20

05 May '20

Hi, In Debian all service APIs are by default using uwsgi. This means that there's no reason to setup Apache for the API, as uwsgi comes by default. However, there are a number of things that operators may want to configure, like for example, the number of workers. This is why I started this PR: https://review.opendev.org/725065 Since Tobias asked for more details about what I was doing, I thought it was a good idea to start a thread in the list, rather than just discussing in the patch review. So, what I did for Neutron is only the beginning. If it works as I expect, I intend to generalize this on all OpenStack services supported by Debian. This means at least: - nova - keystone - glance - cinder - barbican - neutron - aodh - panko - gnocchi - etc. The first step is having this neutron_uwsgi_config provider up and running, then I'll see how to integrate with the rest of puppet-openstack and have things like number of workers integrated with <SERVICE>::api stuff... Your thoughts? Cheers, Thomas Goirand (zigo)

4 6

[openstack-ansible] dropping suse support
by Mohammed Naser 05 May '20

05 May '20

Hi everyone, I've been trying to avoid writing this email for the longest time ever. It's really painful to have to reach this state, but I think we've hit a point where we can no longer maintain support for SUSE within OpenStack Ansible. Unfortunately, with the recent layoffs, the OSA team has taken a huge hit in terms of contributors which means that there is far less contributors within the project. This means that we have less resources to go around and it forces us to focus on a more functional, reliable and well tested set of scenarios. Over the past few cycles, there has been effort to add SUSE support to OpenStack Ansible, during the time that we had maintainers, it was great and SUSE issues were being fixed promptly. In addition, due to the larger team at the time, we found ourselves having some extra time where we can help unbreak other gates. Jesse used to call this a "labour of love", which I admired at the time and hoped we continue to do as much as we can of. However, the lack of a committed maintainer for OpenSUSE has resulted in constantly failing jobs[1][2] (which were moved to non-voting, wasting CI resources as no one fixed them). In addition, it's causing several gate blocks for a few times with no one really finding the time to clean them up. We are resource constrained at this point and we need the resource to go towards making the small subset of supported features functional (i.e. CentOS/Ubuntu). We struggle with that enough, and there seems to be no deployers that are running SUSE in real life at the moment based on bugs submitted. With that, I propose that we drop SUSE support this cycle. If anyone would like to volunteer to maintain it, we can review that option, but that would require a serious commitment as we've had maintainers step off and it hurts the velocity of the project as no one can merge code anymore. ..Really wish I didn't have to write this email Mohammed [1]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct… [2]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct…

6 9

[all][qa] grenade zuulv3 native job available now and its migration plan
by Ghanshyam Mann 05 May '20

05 May '20

Hello Everyone, Finally, after so many cycles, grenade base job has been migrated to zuulv3 native[1]. This is merged in the Ussuri branch. Thanks to tosky for keep working on this and finish it!! New jobs 'grenade', 'grenade-multinode', 'grenade-postgresql' and 'grenade-forward' are zuulv3 native now and we kept old job name 'grenade-py3' as an alias to 'grenade' so that it would not break the gate using 'grenade-py3'. * Start using this new base job for your projects specific grenade jobs. * Migration to new job name: The integrated template and projects are using old job name 'grenade-py3' need to move to a new job name. Job is defined in the integrated template and also in the project's zuul.yaml for irrelevant-files. So we need to switch both places at the same time otherwise you will be seeing the two jobs running on your gate (master as well as on Ussuri). Integrated service specific template has switched to new job[2] which means you might see two jobs running 'grenade' & 'grenade-py3' running on your gate until you change .zuul.yaml. Example: Nova did for master as well as Ussuri gate - https://review.opendev.org/#/q/I212692905a1d645cd911c2a161c13c794c0e0f4d It needs to be done in Ussuri also as Tempest where integrated templates are present is branchless and apply the change for Ussuri job also. [1] https://review.opendev.org/#/c/548936/ [2] https://review.opendev.org/#/c/722551/ -gmann

7 14

[tripleo] Container image tooling roadmap
by Kevin Carter 05 May '20

05 May '20

Hello Stackers, As you may have seen, the TripleO project has been testing the idea of building container images using a simplified toolchain [0]. The idea is to build smaller, more easily maintained images to simplify the lives of TripleO consumers. Since TripleO's move to containers, the project has been leveraging Kolla to provide Dockerfiles, and while this has worked, TripleO has created considerable tooling to bend Kolla images to its needs. Sadly this has resulted in an image size explosion and the proliferation of difficult to maintain tools, with low bus factors, which are essential to the success of the project. To address the risk centered around the TripleO containerization story, we've drafted a spec [0], which we believe outlines a more sustainable future. In this specification, we've designed a new, much more straightforward, approach to building container images for the TripleO project. The "simple container generation," specification does not intend to be a general-purpose tool used to create images for the greater OpenStack community, both Loci and Kolla do that already. This effort is to build containers only for TripleO using distro provided repositories, with distro maintained tooling. By focusing only on what we need, we're able to remove all general-purpose assumptions and create a vertically integrated stack resulting in a much smaller surface area. To highlight how all this works, we've put together several POC changes: * Role and playbook to implement the Containerfile specification [1]. * Tripleoclient review to interface with the new role and playbook [2]. * Directory structure for variable file layout [3]. * To see how this works using the POC code, building images we've tested in real deployments, please watch the ASCII-cast [4]. * Example configuration file examples are here [5][6][7]. A few examples of size comparisons between our proposed tooling versus current Kolla based images [8]: - base: + Kolla: 588 MB - new: 211 MB # based on ubi8, smaller than centos8 - nova-base: + Kolla: 1.09 GB - new: 720 MB - nova-libvirt: + Kolla: 2.14 GB - new: 1.9 GB - keystone: + Kolla: 973 MB - new: 532 MB - memcached: + Kolla: 633 MB - new: 379 MB While the links shown are many, the actual volume of the proposed change is small, although the impact is massive: * With more straightforward to understand tools, we'll be able to get broader participation from the TripleO community to maintain our containerization efforts and extend our service capabilities. * With smaller images, the TripleO community will serve OpenStack deployers and operators better; less bandwidth consumption and faster install times. We're looking to chart a more reliable path forward, and making the TripleO user experience a great one. While the POC changes are feature-complete and functional, more work is needed to create the required variable files; however, should the proposed specification be ratified, we expect to make quick work of what's left. As such, if you would like to be involved or have any feedback on anything presented here, please don't hesitate to reach out. We aim to provide regular updates regarding our progress on the "Simple container generation" initiative, so stay tuned. Thanks, Kevin and Emilien [0] https://review.opendev.org/#/c/723665/ [1] https://review.opendev.org/#/c/722557/ [2] https://review.opendev.org/#/c/724147/ [3] https://review.opendev.org/#/c/722486/ [4] https://files.macchi.pro:8443/demo-container-images/ [5] http://paste.openstack.org/show/792995/ [6] http://paste.openstack.org/show/792994/ [7] http://paste.openstack.org/show/792993/ [8] https://4ce3fa2efa42bb6b3a69-771991cd07d409aaec3e4ca5eafdd7e0.ssl.cf2.rackc… Kevin Carter IRC: kecarter

11 27

[blazar] Updating blazar-core and blazar-release groups
by Pierre Riteau 05 May '20

05 May '20

Hello, I noticed in a recent blazar-specs patch [1] that lots of historical contributors were requested as reviewers. I assume this is because the blazar-core group was used (this is not a feature that I've used before on Gerrit). To avoid spamming people who haven't worked on Blazar for years, I propose to remove the following members of the blazar-core group: Dina Belova Hiroaki Kobayashi Nikolay Starodubtsev Pablo Andres Fuente Ryota MIBU Swann Croiset Sylvain Bauza I would also remove Hiroaki Kobayashi from the blazar-release group. Please let me know if you have any objections. [1] https://review.opendev.org/#/c/723115/

2 2

[neutron] bug deputy report (4/27-5/3)
by Akihiro Motoki 05 May '20

05 May '20

Hi, I was a bug deputy last week. The last week was relatively a quiet week. Higher priority bugs have assignees. They are not release blockers, while it is nice to have them backported. A bug on DVR with flat network needs to be covered by L3 team. ## Untriaged * https://bugs.launchpad.net/neutron/+bug/1876092 DVR with Flat network result in ICMP reply DUP New, Undecided It needs to be checked by DVR folks. I am not sure DVR with flat network is not supported by design or it is just a bug. * https://bugs.launchpad.net/neutron/+bug/1876021 create multiple reserved_dhcp_port doesn't work as expected I will try to reproduce it after the holiday week in my side is over. ## Assigned * https://bugs.launchpad.net/neutron/+bug/1875849 not ensure default security group exists when filter by project_id Medium, In Progress, the reporter is working on the fix ## In Progress * https://bugs.launchpad.net/nova/+bug/1863021 eventlet monkey patch results in assert `len(_active) == 1` AssertionError High, In Progress, a fixed is in the gate * https://bugs.launchpad.net/neutron/+bug/1876148 OVNL3RouterPlugin should add_router_interface as in RouterPluginBase Medium, In Progress * https://bugs.launchpad.net/neutron/+bug/1876094 Dnsmasq 2.81 broke neutron's DHCP service High, In Progress ## New RFEs * https://bugs.launchpad.net/neutron/+bug/1875516 [RFE] Allow sharing security groups as read-only will be discussed in coming drivers meetings * https://bugs.launchpad.net/neutron/+bug/1875852 [RFE] [OVN] SRIOV routing on VLAN Tenant networks ## Fix Released * https://bugs.launchpad.net/neutron/+bug/1875865 SRIOV OVN metadata namespaces not cleaned up after ports are unbounded * https://bugs.launchpad.net/bugs/1875344 Cleanup in neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON may fail in dvr deployments

1 0

[all][docs] docs.openstack.org: Check Ussuri Index pages
by Andreas Jaeger 05 May '20

05 May '20

The Ussuri index pages [1] are now published, the default on docs.o.o to Ussuri will change with release. The index pages only list entries that existed yesterday, so if a project was branched but did not merge any change, then no Ussuri content might exist and thus the project is not listed in the index pages. In such a case: Merge the usual bot updates, wait for the content to be published, and then propose a change to [3]. For details see [2]. Andreas [1] https://docs.openstack.org/ussuri/ [2] https://docs.openstack.org/doc-contrib-guide/doc-index.html [3] https://opendev.org/openstack/openstack-manuals/src/branch/master/www/proje… -- Andreas Jaeger aj(a)suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB

1 0

[Watcher] No IRC meeting tomorrow
by licanwei 05 May '20

05 May '20

1 0

[ironic][ptg] Virtual PTG - Victoria
by Iury Gregory 05 May '20

05 May '20

Hello Ironicers! We need to choose the slots for the Virtual PTG[1] on the official schedule [2]. So, if you are interested in participating you need to do two things: 1- Add your name and topics to the etherpad [3]. 2- Fill out the doodle in [4]. Thank you! [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014126.ht… [2] https://ethercalc.openstack.org/126u8ek25noy [3] https://etherpad.opendev.org/p/Ironic-VictoriaPTG-Planning [4] https://doodle.com/poll/mbpr2x7z3t5hqec6 -- *Att[]'sIury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Software Engineer at Red Hat Czech* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory(a)gmail.com <iurygregory(a)gmail.com>*

2 2

[neutron][QoS] IRC meeting cancelled on May 5
by Rodolfo Alonso 05 May '20

05 May '20

Hello: Due to the lack of agenda, I'm going to cancel the Neutron QoS meeting today. Regards. Rodolfo Alonso (ralonsoh)

1 0

[all][PTL][TC] PTG Signup Last Call
by Kendall Nelson 04 May '20

04 May '20

Hello! In an effort to get a schedule set and posted on the website and so that you can all block your calendars for the event, we ask that *teams finish signing up for time by Sunday, May 10th at 7:00 UTC*. We have done some consolidation to save room resources (as we will have to set up/pay for rooms ahead of time) so if you are in the list below, please note that your room may have changed, but the timing should be the same. Also note, some teams may be set to switch rooms throughout the week. Currently Signed Up Teams: Airship Automation SIG Cinder Edge Computing Group First Contact SIG Interop WG Ironic Glance Heat Horizon Kata Containers Kolla Manila Monasca Multi-Arch SIG Neutron Nova Octavia OpenDev OpenStackAnsible OpenStackAnsibleModules OpenStack-Helm Oslo QA Scientific SIG Security SIG Tacker TripleO Thanks! -The Kendalls(diablo_rojo & wendallkaters)

1 0

[all] Virtual PTG Registration Live
by Kendall Waters 04 May '20

04 May '20

Hey everyone, Registration for the June 2020 Virtual PTG is now live! It is free but its still extremely important that you register as this is primarily so we have a way to contact attendees with information about the event as we figure the rest of the details out. So please take a minute to register: https://virtualptgjune2020.eventbrite.com <https://virtualptgjune2020.eventbrite.com/> Let us know if you have any questions! -the Kendalls (diablo_rojo & wendallkaters)

1 0

cross distro Virtual PTG planning
by Thomas Goirand 04 May '20

04 May '20

Dear package maintainers, We used to have a cross-distro discussion during each PTG (well, summits, since I didn't attend any PTG since the split). I was wondering if people from other distros would be interested in having such a session again. Please tell if you hare as a follow-up here, then maybe we can add ourselves in the ethercalc. I used to know everyone, like Haikel, people from SuSE, etc. Nowadays, I only know people from Canonical. I'd love to meet the new guys around. Cheers, Thomas Goirand (zigo)

3 2

[cinder] critical bugs/patches for RC-2
by Brian Rosmaita 03 May '20

03 May '20

The proposed list is here: https://etherpad.opendev.org/p/cinder-ussuri-rc-backport-potential Please prioritize your reviewing accordingly. There are several proposed but not approved bugs on the list; I could use some feedback on whether they are release-critical or not. Please reply on the etherpad. thanks, brian

1 2

[blazar] IRC meeting cancelled on May 5
by Pierre Riteau 03 May '20

03 May '20

Hello, Due to a scheduling conflict, I will not be able to chair the Blazar IRC meeting on Tuesday May 5. I propose to cancel and meet as usual on May 12. Best wishes, Pierre Riteau (priteau)

1 0

[InteropWG] Draft Guidelines for 2020.06 Ussuri core and addons for orchestration and dns
by prakash RAMCHANDRAN 03 May '20

03 May '20

Hi all, We have been trying to draft Ussuri guidelines API for Market logo mark as part of InteropWGIn case anyone is interested in reviewing the draft patch please feel free to comment with +1 only.The +2's are reserved for Mark-Voelker & Egle.sigler c Vice Chair and past Vice-Chairs for Interop WGCall for Review: https://review.opendev.org/722137 We are investing add-ons for OpenStack Edge , Containers and Swift backends.If you have any suggestions please pitch in. Please add your ideas to PTG etherpad link in the follwoing (Jine 1st - 1st 2 hour slot)https://etherpad.opendev.org/p/interop ThnaksPrakash https://etherpad.opendev.org/p/ecg_virtual_ptg_planning_june_2020

2 1

[goals][Drop Python 2.7 Support] Status Report: COMPLETED \o/
by Ghanshyam Mann 02 May '20

02 May '20

Hello Everyone, Please find the final status for the py2.7 drop goal. Finally, it is COMPLETED!! With this (after three continuous cycles), I am taking a break from community goals (at least for V cycle :))!!. Wiki page is updated: https://wiki.openstack.org/wiki/Python3 I have added the 'Python3 Only' column here https://wiki.openstack.org/wiki/Python3#Python_3_Status_of_OpenStack_projec… Completion Report: =============== * Swift & Storlets are the two projects keeping py2.7 support. * All projects have completed the goal and now are python3-only. * It has been another roller coaster :) and I appreciate everyone helped in this work and all the projects reviewing the patches on time with few exceptions :). * A Lot of gate issues occurred and fixed during this transition especially making branchless tooling testing the mixed python version ( stable branch on py2 and other on py3). * I finished the audit for all the projects. requirement repo will be cleaning up the py2 caps in Victoria cycle[1]. Some ongoing work on deployement projects repo or unmaintained repo: ------------------------------------------------------------------------------------------- These are dpendendant on other work for testing framework etc. This can be continued and need not to be tracked under this goal. * python-barbicanclient: https://review.opendev.org/#/c/699096/ ** This repo seems unmaintained for last 6 months and the gate is already broken. * Openstack Charms - Most of them merged, few are waiting for the team to migrate to Python3 Zaza functional test framework. * Openstackansible - This will finish once centos jobs will be migrated to Centos8. * Openstack-Helm - No conclusion yet from help team on what else to do for py2 drop. -gmann

6 6

[all][qa] Gate Status: week 20-18
by Ghanshyam Mann 01 May '20

01 May '20

Hello Everyone, I am highlighting the few of the gate issues we fixed or in-progress recently. 1. Ubuntu xenial node jobs on stable stein onwards gate or 3rd party CI. Tempest 23.0.0 is the last version supported for py2.7 or py3.5. And we used the that version to tests stable/rocky jobs(py3.5 env) gate and any py2.7 job on stein or train gate. It was fixed on many projects when gate started failing with master Tempest. Last week when Tempest new version 24.0.0 is released and the same is updated in master upper-constraint, we found xenial node job running on stein gate (for Octavia and Ironic at least). Fixing those is not easy which needs overriding the constraint for that particular job. Octavia and Ironic removed the xenial job which is the right step. But few 3rd party CI not migrated to Bionic stein onwards will stop working now. One workaround can be to override the constraint to stable/rocky via env var UPPER_CONSTRAINTS_FILE but I have not tried if that works or not. If not, moving your CI to Bionic is the way forward. I am trying a new approach to cap the constraint in tox.ini itself whenever a new Tempest version is released. That can avoid hacking the same via UPPER_CONSTRAINTS_FILE. For example, Tempest 24.0.0 tag's tox.ini use stable/ussuri constraint. 2. Bug#1876330. devstack-plugin-ceph-tempest-py3 failing for rescue BFV instance test New tests added for rescue BFV instance feature on Nova failed on ceph job. lyarwood is working on the bug but for now, we have taken the step forward to skip the tests for ceph. Any new tests which do not work on ceph, we need to add the tests in ceph blacklist file until the bug is fixed. Job is green now. if it was failing on your gate where it is running as voting, please recheck. - https://review.opendev.org/#/c/724866/ I have also proposed to make ceph job as voting on the Tempest side (it was n-v till now) so that we can take these steps well before merging the new tests on the Tempest gate. - https://review.opendev.org/#/c/724948/1 -gmann

1 0

[all] A call for consolidation and simplification
by Thierry Carrez 01 May '20

01 May '20

Hi all, I'd like to issue a call for consolidation and simplification for OpenStack development. In the early years of the project, we faced a lot of challenges. We had to spread the development load across manageable-size groups, so we encouraged the creation of a lot of project teams. We wanted to capture all the energy that was sent towards the project, so we passed project structure reforms (like the big tent) that would aggressively include new community groups in the "official" OpenStack community. We needed to remove bottlenecks, so we encouraged decentralized decision making. And we had to answer unique challenges, so we created software to match them (Zuul). In summary, we had a lot of people, and not enough systems to organize them, so we created those. Fast-forward to 2020, and our challenges are different. The many systems that we created in the early days have created silos, with very small groups of people working in isolation, making cross-project work more difficult than it should be. The many systems that we created generate a lot of fragmentation. Like we have too many meetings (76, in case you were wondering), too much energy spent running them, too much frustration when nobody joins. Finally, the many systems that we created represent a lot of complexity for newcomers to handle. We have 180 IRC channels, most of them ghost towns where by the time someone answers, the person asking the question is long gone. So I think it's time to generally think about simplifying and consolidating things. It's not as easy as it sounds. Our successful decentralization efforts make it difficult to make the centralized decision to regroup. It's hard to justify time and energy spent to /remove/ things, especially those that we spent time creating in the first place. But we now have too many systems and not enough people, so we need to consolidate and simplify. Back around Havana, when we had around the same number of active contributors as today, we used to have 36 meetings and 20 teams. Do we really need 180 IRC channels, 76 meetings, 63 project teams (not even counting SIGs)? Yes, we all specialized over time, so it's hard to merge for example Oslo + Requirements, or QA + Infrastructure, or Stable + Release Management, or Monasca + Telemetry. We are all overextended so it's hard to learn new tricks or codebases. And yet, while I'm not really sure what the best approach is, I think it's necessary. Comments, thoughts? -- Thierry Carrez (ttx)

9 16

[Reno][Neutron] Release notes for stable/ussuri
by Slawek Kaplonski 01 May '20

01 May '20

Hi, I wanted to add some prelude section to Neutron's release notes for stable/ussuri but I screw it up and forgot to do it before stable/ussuri branch was created. So now I have patch [1] but notes aren't going to stable/ussuri branch at all. I found the way how to exclude it from current (V) release notes but is there any way to somehow force to add it to stable/ussuri's release notes now? [1] https://review.opendev.org/#/c/724809 -- Slawek Kaplonski Senior software engineer Red Hat

2 2

[tripleo] CI
by Wesley Hayutin 01 May '20

01 May '20

Greetings, Status update... Master: GREEN Stable Branches impacted by: https://bugs.launchpad.net/tripleo/+bug/1875890 fixed Now we are trying to promote each branch to level out pacemaker on the node and containers. Queens is promoting now. Train: GREEN Queens: RED ( current priority ) In addition to the pacemaker issue which has resolved in our periodic testing jobs, we're hitting issues w/ instances timing out in tempest https://bugs.launchpad.net/tripleo/+bug/1876087 Stein: RED Also seems to have the same issue as Queens https://bugs.launchpad.net/tripleo/+bug/1876087 ( under investigation for Stein ) Rocky: RED Also seems to have the same issue as Queens https://bugs.launchpad.net/tripleo/+bug/1876087 ( under investigation for Rocky ) I will be promoting Rocky to level out pacemaker next. In order to get the voting jobs in queens back to green, I'm disabling tempest on containers-multinode. https://review.opendev.org/#/c/724703/ Additional notes may be found: https://hackmd.io/1pY-KQB_QwOe-a-5oEXTRg

2 2

[release] Release countdown for week R-1, May 4 - 8
by Sean McGinnis 01 May '20

01 May '20

Development Focus ----------------- We are on the final mile of the Ussuri development cycle! Remember that the Ussuri final release will include the latest release candidate (for cycle-with-rc deliverables) or the latest intermediary release (for cycle-with-intermediary deliverables) available. May 7 is the deadline for final Ussuri release candidates as well as any last cycle-with-intermediary deliverables. We will then enter a quiet period until we tag the final release on May 13. Teams should be prioritizing fixing release-critical bugs, before that deadline. Otherwise it's time to start planning the Victoria development cycle, including discussing PTG sessions content, in preparation of the virtual Victoria PTG event taking place June 1-5. Actions ------- Watch for any translation patches coming through on the stable/ussuri branch and merge them quickly. If you discover a release-critical issue, please make sure to fix it on the master branch first, then backport the bugfix to the stable/ussuri branch before triggering a new release. Please drop by #openstack-release with any questions or concerns about the upcoming release! Upcoming Deadlines & Dates -------------------------- Final RC deadline: May 7 (R-1 week) Final Ussuri release: May 13 Virtual Victoria PTG: June 1-5

1 0

[manila][stable] Additions and Removals from stable-core
by Goutham Pacha Ravi 01 May '20

01 May '20

Hello Stackers, I'd like to add a couple of maintainers to manila-stable-maint: https://review.opendev.org/#/admin/groups/1099,members - dviroel (Douglas Viroel) - silvacarlos (Carlos Eduardo) Both Douglas and Carlos understand the stable branch policy [1] and adhere to it. Thanks! Goutham [1] https://docs.openstack.org/project-team-guide/stable-branches.html

4 5

[qa][ptg] Virtual PTG planning
by Masayuki Igawa 01 May '20

01 May '20

Hi team, Short summary: 1. Please take a look at the QA PTG etherpad[2] and put your name and topics there 2. Please vote on the doodle poll[3] Details: As we discussed in the office hours, we're now collecting topics[3] for the PTG[1]. And, we need to fix our slots for the PTG. We have now 5 topics[2], and we'll have some more, possibly. So, we'll need 2 or 3 hours slots for 2 days or more if we have more topics. Anyway, please fill out the doodle poll[3] if you'd like to participate in the PTG. And, please add your name and topics to the etherpad so that we can discuss them during the following office hours. [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014126.ht… [2] https://etherpad.opendev.org/p/qa-victoria-ptg [3] https://doodle.com/poll/5awstyshtquuwcq3 -- Masayuki

1 2

[manila][ptg] Victoria Cycle PTG in Vancouver - Planning
by Goutham Pacha Ravi 30 Apr '20

30 Apr '20

Hello Zorillas and other interested Stackers, We've requested space to gather at the Open Infrastructure PTG event in Vancouver, BC between June 8th-11th 2020. This event gives us the opportunity to catch up in person (and via telepresence) and discuss the next set of goals for manila and its ecosystem of projects. As has been the norm, there's now a planning etherpad for us to collect topics and vote upon [1]. Depending on how much space/time we are budgeted, we will select and schedule these topics. Please take a look and add your topics to the etherpad. Thanks, Goutham [1] https://etherpad.openstack.org/p/vancouver-ptg-manila-planning

1 1

[tacker][ptg] Virtual PTG Planning
by Yasufumi Ogawa 30 Apr '20

30 Apr '20

Hi Tacker team, As talked in the last IRC meeting, we have decided to have vTPG on 2nd and 3rd June [1]. We have etherpad for the discussion [2], so please add your name and topics if you have any suggestions for the next release. - 02 June: 6am-8am UTC - 03 June: 6am-8am UTC [1] http://eavesdrop.openstack.org/meetings/tacker/2020/tacker.2020-04-28-08.07… [2] https://etherpad.opendev.org/p/Tacker-PTG-Victoria Thanks, Yasufumi

1 0

[openstack-helm][ptg] Virtual PTG Planning
by Gage Hugo 30 Apr '20

30 Apr '20

As we discussed in Tuesday's meeting, we have schedules some tentative time-slots for the virtual PTG across multiple days: Current Time-Slots: - 1300 - 1700 UTC Tue June 2nd - 1300 - 1700 UTC Wed June 3rd We also have an etherpad for topics if anyone has anything they want to discuss. Hope to see you there! https://etherpad.opendev.org/p/openstack-helm-ptg-victoria

1 0

[security][ptg] Virtual PTG Planning
by Gage Hugo 30 Apr '20

30 Apr '20

Hello, As discussed in today's meeting, the Security SIG has scheduled some tentative timeslots for the virtual PTG to discuss any topics. We also have an etherpad to fill out if anyone has anything they want to discuss. Hope to see you there! Current Timeslots: - 1500 - 1700 UTC Mon June 1st 2020 - 2100 - 2300 UTC Mon June 1st 2020 https://etherpad.opendev.org/p/security-sig-ptg-victoria

1 0

[ironic][redfish] Breaking (for us) underlying schema/behavior change and what do we do?
by Julia Kreger 30 Apr '20

30 Apr '20

Greetings fellow awesome sentient beings, and the occasional text indexers! We have a growing conundrum with Redfish. A long time ago, in a red shifted galaxy inhabited by Redfish, a feature was defined early on in the DMTF specification defining a "BootSourceOverrideEnabled" field as part of the ComputerSystem Boot object[0]. This setting, in concert with the "BootSourceOverrideTarget" and "BootSourceOverrideMode" setting allowed Ironic to signal an override preference in regards to what device the machine should boot to. As time as moved forward, schema field descriptions were added, and notes further detailed (observable by looking at [1], see ComputerSystem 1.0.0, 1.5.0, 1.6.0, 1.8.0). One such case, at least depending on what field you looked at also noted that "BootSourceOverrideEnabled" was not possible with the "Continuous" setting, however that was only noted under the one time override option "UefiTargetBootSourceOverride" which is leveraged by the explicit "UEFITarget" target option, at least until ComputerSystem 1.8.0. In ComputerSystem 1.8.0, an explicit note was added into the description text for "BootSourceOverrideEnabled" detailing that one time override use was the only option for UEFI. The published documents also have not been very clear on this, but there are numerous documents on the subject so it seems very easy for the schema details note being added and enhanced in 1.8.0 drove vendors to begin to change the logic around their "BootSourceOverrideEnabled" option to represent what the current documentation detailed, regardless of their schema version. Also, the schema/api versions are disjointed and BMCs out there exist without some of the defined fields. It just kind of is our life with hardware. So why now? The ironic team has had reports recently, across multiple vendors, where with newer firmware versions beginning to implement the changes in schema 1.8.0, where it seems we will no longer be able to use the option we were using, which seemed logical at least when looking at the BMC conformance to the standard. And while some may view the fact it worked on multiple vendors as a regression, I'm sure they see it as a bug as it was seemingly never intended to work that way when we look at the "UEFITargetBootSourceOverride" field notes from the early schemas. tl;dr: As BMC's vendors add the newer schema fields and resulting behavior, the existing redfish driver's ability to assert persistent boot devices breaks. With all of that having been said, I think we should remove our persistent setting capability in our redfish driver, and back port the fix where possible. That being said, this is also a behavior change, so it is time to get everyone's thoughts, since we will need to stress that machines by default should boot from persistent storage instead of network by default. We also need to launch a long term effort to try and duplicate the setting capabilities through the other BMC interfaces/methods/settings, but the variation by vendor is what is going to make it very difficult until the latest redfish schema version is implemented by vendors. Of course, that too may be a while. Thoughts? -Julia [0]: DMTF Redfish Schema Supplement 2019.1a, Page 71: https://www.dmtf.org/sites/default/files/standards/documents/DSP0268_2019.1… or the 2016.2 version of the document, Page 16: https://www.dmtf.org/sites/default/files/standards/documents/DSP2046_2016.2… And the same information can be found in the and 2019.2, 2019.3 schema document versions. [1]: Latest schema bundle pack: https://www.dmtf.org/sites/default/files/standards/documents/DSP8010_2019.4…

2 3

[MURANO] DevStack Murano "yaql_function" Error
by İzzettin Erdem 30 Apr '20

30 Apr '20

Hello folk, Is there anyone who used murano before? I am trying to install Murano on devstack stable/train release. When i try to deploy an environment, it throws an error. At murona-engine logs error is: "func.__yaql_function__ = fd: murano.dsl.dsl_exception.MuranoPlException: [AttributeError]: 'method' object has no attribute '__yaql_function__'" Any suggestioıns? Thanks. Regards.

3 4

[puppet][packaging] Switching to policy.yaml (over policy.json)
by Thomas Goirand 30 Apr '20

30 Apr '20

Hi team! In the light of the discussion that recently happened in #openstack-nova, it looks like switching from a .json file to a .yaml file is what we should do. Indeed, the file generated in .json format, if used pristine with the nova.conf and without enforcing scope, makes a nova-api service that simply doesn't work. Please read https://bugs.launchpad.net/nova/+bug/1875418 for more insights. The issue is that both packages are generating a .json (at least in Debian and Ubuntu) and puppet expect a policy.json, not a policy.yaml. With the policy.yaml, we don't have the same problem as by default, all policies are commented out. Operators just need to uncomment to activate. So, my proposal is the following: before the final release of OpenStack, I will modify all Debian OpenStack packages to generate (and package) both policy.json and policy.yaml. Then puppet-openstack can switch over to the .yaml file, and uncomment only the parts that the operator sees as relevant. I also would like to add a policy.d folder by default in each package, where operators can override stuff. Just having the folder will be a sign to operators that they are invited to write stuff in there, and that it will not be overwritten by a package upgrade. Then what I would like to do, is get puppet-openstack to only write there, for example in /etc/nova/policy.d/my-custom-policy.yaml. The file in /etc/nova/policy.yaml will be marked as "CONFFILE" in Debian, meaning that dpkg will prompt for changes on upgrades, while what's in the policy.d will remain. Last, I do believe that the yaml files are a way more easy to handle with puppet than the .json counterpart. Indeed, we could use something like the .ini management thing, with the : (Semicolumn) sign replacing the = (equal) sign. Moreover, the .yaml files contain comments which the .json files are lacking, making it auto-documented for operators. The only reason why I didn't use .yaml files earlier in the Debian packages was that, somehow, loading them with the API didn't work. I confirm that now it looks like working (though I'd have to test if changing a value is taken into account, I didn't do that yet). Your thoughts everyone? Cheers, Thomas Goirand (zigo)

2 1

[all] Curating the openstack org on GitHub
by Thierry Carrez 30 Apr '20

30 Apr '20

Hi everyone, As you know, for practical and "code marketing" reasons, we maintain mirrors of the opendev.org/openstack repositories on GitHub: https://github.com/openstack/ A few months ago jroll and I took on the task of maintaining the "openstack" organization on GitHub... In particular pushing "moved" final commits to repositories that are now maintained elsewhere on Opendev, and marking them retired. It's all a bit of a challenge. GitHub does not let you set the "mirror" flag by API (you have to ask GitHub support to manually do it, and they don't like it when you ask for it to be set on 1,831 repositories). The "pinned repositories" UI fails miserably and does not let us select "Nova" (again, probably too many repositories, GitHub support does not have a solution for us). There are limited opportunities to describe the fact that we actually only use GitHub as a mirror, leading to confusion. For inactive repositories (no longer under OpenStack governance), the situation is even more confusing. Some are noted "MOVED" or "RETIRED" in their description, some have a closing commit (some others don't), some have the "archived" flag. Some don't have anything and appear active while they are not (see attached CSV for those who like data). There is also the openstack-dev and openstack-attic organizations, which are leftovers from ancient times. In summary, for a "code marketing" opportunity, it does not paint a great picture. I'd like to make the following suggestion: - aggressively delete all non-openstack things from the openstack org, keeping only official, active repositories - delete the openstack-dev and openstack-attic organizations We shied away from doing that in the past, mostly to not break people who may have cloned those repositories... But I think the benefits of cleaning up now outweigh the drawbacks. The reference code always exists at opendev.org anyway. Also maybe once we are back to a more reasonable number of visible repositories, the pins UI will work again. And GitHub will actually be OK with setting the mirror flag. Thoughts, comments ? -- Thierry Carrez (ttx)

13 27

[neutron] network converting from flat to vlan
by Ionut Biru 30 Apr '20

30 Apr '20

Hi guys, I made a mistake a long time ago creating a flat network instead of vlan. Is there a way to convert this network to vlan? Did anyone try to do that and have a procedure in place? I'm running rocky. -- Ionut Biru - https://fleio.com

1 0

[neutron] CI meeting cancelled
by Slawek Kaplonski 30 Apr '20

30 Apr '20

Hi, As 1st of May is Labor Day and public holiday in Poland, I will not be available to chair drivers meeting. Lets cancel it this week. See You all on the meeting next week. Have a great (long) weekend :) -- Slawek Kaplonski Senior software engineer Red Hat

1 0

[TC] Ussuri Virtual Celebration!
by Kendall Nelson 29 Apr '20

29 Apr '20

Hello :) So! Since we won't be able to see each other in June and celebrate the recent release together, is there interest in doing a virtual cheers as a community? We could do something fun like.. trivia or a baking contest or something? I am reaching out to the TC specifically because we would probably want to do more than one of these and spread across all timezones so we could have a TC member or two 'host' the cheers. Can't wait to hear what you all think! -Kendall (diablo_rojo)

9 12

[openstack-helm-infra][elasticsearch]Error deploying chart in k8s cluster with local storage
by Sanjay K 29 Apr '20

29 Apr '20

Hello everyone, I have been trying to deploy elasticsearch helm chart in my k8s cluster consisting of 4 nodes (1 k8s master, 1 controller and 2 computes) after deploying fluentbit and fluentd helm charts which are deployed with success. I have plan to make the EFK (Elasticsearch-Fluentd-Kibana) stack running in my k8s cluster setup for logs management. However I face issues while deploying elasticsearch helm chart using the below command : $ helm install elasticsearch local/elasticsearch --values values_overrides/local-storage.yaml -n openstack Error: unable to build kubernetes objects from release manifest: error validating "": error validating data: ValidationError(StatefulSet.spec): unknown field "strategy" in io.k8s.api.apps.v1.StatefulSetSpec Can anyone point me if I missing anything? What are the prerequisites before deploying elasticsearch chart which intended to use local storage/pv in my case avoiding the default S3 storage as per the chart? Thanks for your pointers and suggestions. ' best regards, SanjayK

2 7

[requirements][all] - requirements is now frozen for non-critical updates
by Matthew Thode 29 Apr '20

29 Apr '20

After the Ussuri-3 milestone, only critical requirements and constraints changes will be allowed. Freezing our requirements list gives packagers downstream an opportunity to catch up and prepare packages for everything necessary for distributions of the upcoming release. The requirements remain frozen until the stable branches are created, with the release candidates. All future changes not previously approved will need a FFE request set to the list, please prefix it with [FFE][requirements][PROJECTNAME]. -- Matthew Thode

1 1

Re: [TripleO][train][rdo] installation of undercloud fails during Run container-puppet tasks step1
by Ruslanas Gžibovskis 29 Apr '20

29 Apr '20

During installation, I can see them running fro 25sec max. not sure if it is as expected, Can I check somehow, how to get them up? if I do podman run -it Image_ID will it be enough to test if images are good? On Wed, 29 Apr 2020 at 12:01, Wu, Heng <wuh.fnst(a)cn.fujitsu.com> wrote: > > I am new to containers, can I somehow transfer these images between, as > I understand, it might help? > > > > The image will be downloaded automatically with the installation, you > don't need to do it manually. > > The problem seems to be that the container failed to start, not the errors > in the images themselves. > > You may need to check the configuration of undercloud.conf and proxy > settings (if they exist). > > > > *From:* Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > *Sent:* 2020年4月29日 15:48 > *To:* Alex Schultz <aschultz(a)redhat.com> > *Cc:* Openstack-discuss <openstack-discuss(a)lists.openstack.org>; > kent.martin(a)ericsson.com > *Subject:* Re: [TripleO][train][rdo] installation of undercloud fails > during Run container-puppet tasks step1 > > > > On the other installation I have, I see a bit more images: > > > > > I am new to containers, can I somehow transfer these images between, as I > understand, it might help? > > > > On Wed, 29 Apr 2020 at 09:45, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > > I just now realized, that I have seen in some log, messages, missing some > puppet-dependencies... Cannot find log again... > > > > On Wed, 29 Apr 2020 at 09:39, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > > podman ps -a = clean, no containers at all. > > I have a watch -d "sudo podman ps -a ; sudo podman images -a ; sudo df -h" > > > > paunch.log is empty. (I did several reinstallations). > > > > > > > Later I saw connectionpool retrying, but I have not seen " > tripleo_common.image.image_uploader" with same. > > > > > > > On Tue, 28 Apr 2020 at 20:10, Alex Schultz <aschultz(a)redhat.com> wrote: > > On Tue, Apr 28, 2020 at 11:57 AM Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > > > > Hi all, > > > > I am running a fresh install of rdo train, on centos7 > > I almost a week I am facing error at this step: > > TASK [Run container-puppet tasks (generate config) during step 1] > > > > So I have ansible.log attached, I cannot find anything, where it is > failing. > > According to some understanding in ansible, it fails if it finds stderr > output. > > I cannot find error/fail or smth, I see Notices and Warnings, but I > believe it is not stderr? > > > > I see containers running and removed after some time. > > (as it should be I think)... > > > > Could you help me, where to dig? > > > > 2020-04-27 22:27:46,147 p=132230 u=root | TASK [Start containers for > step 1 using paunch] > > ***************************************************************************************************************************** > 2020-04-27 22:27:46,148 p=132230 u=root | Monday 27 April 2020 > 22:27:46 +0200 (0:00:00.137) 0:04:44.326 **********? > 2020-04-27 22:27:46,816 p=132230 u=root | ok: [remote-u] > 2020-04-27 22:27:46,914 p=132230 u=root | TASK [Debug output for > task: Start containers for step 1] > > ******************************************************************************************************************* > 2020-04-27 22:27:46,915 p=132230 u=root | Monday 27 April 2020 > 22:27:46 +0200 (0:00:00.767) 0:04:45.093 **********? > 2020-04-27 22:27:46,977 p=132230 u=root | fatal: [remote-u]: FAILED! => { > "failed_when_result": true,? > "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | > default([]))": [] > > Check /var/log/paunch.log. It probably has additional information as > to why the containers didn't start. You might also check the output > of 'sudo podman ps -a' to see if any containers exited with errors. > > > -- > > Ruslanas Gžibovskis > > +370 6030 7030 > > > > > -- > > Ruslanas Gžibovskis > +370 6030 7030 > > > > > -- > > Ruslanas Gžibovskis > +370 6030 7030 > > > > > -- > > Ruslanas Gžibovskis > +370 6030 7030 > -- Ruslanas Gžibovskis +370 6030 7030

1 1

[neutron] CI meeting
by Slawek Kaplonski 29 Apr '20

29 Apr '20

Hi, Sorry for late notice but I totally forgot about it. I need to cancel today's Neutron CI meeting. There is RH Summit going on and I have some sessions which I want to attend during our CI meeting. I checked grafana, and I don't see any urgent issues related to our CI so lets cancel this meeting and see You all next week :) -- Slawek Kaplonski Senior software engineer Red Hat

1 0

[Neutron] Cancel neutron L3 meeting
by LIU Yulong 29 Apr '20

29 Apr '20

Hi there, Because I have to some local workload, I will not be able to run the L3 meeting tonight. And I'm going to take a vacation next week. So the next L3 meeting will be set on 13th May 2020. See you guys online then. Regards, LIU Yulong

1 0

Re: [TripleO][train][rdo] installation of undercloud fails during Run container-puppet tasks step1
by Ruslanas Gžibovskis 29 Apr '20

29 Apr '20

Moving output to pastebin I have found it very interesting, that I was able to download images by running some pull commands (below), podman images -a show it, but the installation still gets 401 errors. http://paste.debian.net/1143698/ > > On Wed, 29 Apr 2020 at 09:47, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > >> On the other installation I have, I see a bit more images: >> >> I am new to containers, can I somehow transfer these images between, as I >> understand, it might help? >> >> On Wed, 29 Apr 2020 at 09:45, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> >> wrote: >> >>> I just now realized, that I have seen in some log, messages, missing >>> some puppet-dependencies... Cannot find log again... >>> >>> On Wed, 29 Apr 2020 at 09:39, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> >>> wrote: >>> >>>> podman ps -a = clean, no containers at all. >>>> I have a watch -d "sudo podman ps -a ; sudo podman images -a ; sudo df >>>> -h" >>>> >>>> paunch.log is empty. (I did several reinstallations). >>>> >>>> I found in image logs: >>>> 2020-04-29 08:52:49,854 140572 DEBUG urllib3.connectionpool [ ] >>>> https://registry-1.docker.io:443 "GET /v2/ HTTP/1.1" 401 87 >>>> 2020-04-29 08:52:49,855 140572 DEBUG >>>> tripleo_common.image.image_uploader [ ] >>>> https://registry-1.docker.io/v2/ status code 401 >>>> 2020-04-29 08:52:49,855 140572 DEBUG >>>> tripleo_common.image.image_uploader [ ] Token parameters: params {'scope': >>>> 'repository:tripleotrain/centos-binary-zaqar-wsgi:pull', 'service': ' >>>> registry.docker.io'} >>>> >>>> Later I saw connectionpool retrying, but I have not seen " >>>> tripleo_common.image.image_uploader" with same. >>>> >>>> On Tue, 28 Apr 2020 at 20:10, Alex Schultz <aschultz(a)redhat.com> wrote: >>>> >>>>> On Tue, Apr 28, 2020 at 11:57 AM Ruslanas Gžibovskis <ruslanas(a)lpic.lt> >>>>> wrote: >>>>> > >>>>> > Hi all, >>>>> > >>>>> > I am running a fresh install of rdo train, on centos7 >>>>> > I almost a week I am facing error at this step: >>>>> > TASK [Run container-puppet tasks (generate config) during step 1] >>>>> > >>>>> > So I have ansible.log attached, I cannot find anything, where it is >>>>> failing. >>>>> > According to some understanding in ansible, it fails if it finds >>>>> stderr output. >>>>> > I cannot find error/fail or smth, I see Notices and Warnings, but I >>>>> believe it is not stderr? >>>>> > >>>>> > I see containers running and removed after some time. >>>>> > (as it should be I think)... >>>>> > >>>>> > Could you help me, where to dig? >>>>> > >>>>> >>>>> 2020-04-27 22:27:46,147 p=132230 u=root | TASK [Start containers for >>>>> step 1 using paunch] >>>>> >>>>> ***************************************************************************************************************************** >>>>> 2020-04-27 22:27:46,148 p=132230 u=root | Monday 27 April 2020 >>>>> 22:27:46 +0200 (0:00:00.137) 0:04:44.326 **********? >>>>> 2020-04-27 22:27:46,816 p=132230 u=root | ok: [remote-u] >>>>> 2020-04-27 22:27:46,914 p=132230 u=root | TASK [Debug output for >>>>> task: Start containers for step 1] >>>>> >>>>> ******************************************************************************************************************* >>>>> 2020-04-27 22:27:46,915 p=132230 u=root | Monday 27 April 2020 >>>>> 22:27:46 +0200 (0:00:00.767) 0:04:45.093 **********? >>>>> 2020-04-27 22:27:46,977 p=132230 u=root | fatal: [remote-u]: FAILED! >>>>> => { >>>>> "failed_when_result": true,? >>>>> "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | >>>>> default([]))": [] >>>>> >>>>> Check /var/log/paunch.log. It probably has additional information as >>>>> to why the containers didn't start. You might also check the output >>>>> of 'sudo podman ps -a' to see if any containers exited with errors. >>>>> >>>>> -- Ruslanas Gžibovskis +370 6030 7030

1 0

[all][TC][PTL][release] Ussuri Community Meeting Proposal
by Allison Price 28 Apr '20

28 Apr '20

Hi everyone, As we prepare for the upcoming Ussuri release on May 13, I wanted to see if there was interest among the community to do a community meeting with folks presenting release highlights / answering questions that people may have. We were thinking that atleast one member from the TC or release management team could moderate the discussion and call on various PTLs to present what their teams accomplished during this cycle. If possible, we could do one that is around 1400 UTC to hit Europe and the United States and potentially another one around 0200 UTC for APAC, so we may need two volunteers. We would also like to promote Ussuri demos in the release comms, including the landing page and promotional materials. If you are planning to do a demo or would like to, please let me know and we can include it on the Ussuri landing page. Let me know what y’all think about doing a community meeting and if particular folks would like to volunteer to moderate and / or present. Looking forward to hearing feedback, and, in particular, if anyone is interested in volunteering to help moderate or participate in the Ussuri focused community meeting. Thanks! Allison Allison Price OpenStack Foundation allison(a)openstack.org <mailto:allison@openstack.org>

7 12

[magnum] issues with cloud-init and user-data on fedora-atomic image
by Jean-Philippe Méthot 28 Apr '20

28 Apr '20

Hi, I’ve been testing out magnum (rocky) in my staging environment and I’ve been running into an issue I can’t really explain. Essentially, Magnum is able to provision all the VMs it needs according to the template. However, when it comes time to install kubernetes through cloud-init, cloud-init is unable to fetch the user-data. I must specify that ONLY the user-data is affected. Everything else that’s read from the metadata gets through without any issue. I can also do 'curl http://169.254.169.254/2009-04-04/' <http://169.254.169.254/2009-04-04/'> from the VM without any issue. When I try to curl http://169.254.169.254/2009-04-04/user-data <http://169.254.169.254/2009-04-04/user-data> , curl just hangs without any data showing up : sudo curl -v http://169.254.169.254/2009-04-04/user-data * Trying 169.254.169.254... * TCP_NODELAY set * Connected to 169.254.169.254 (169.254.169.254) port 80 (#0) > GET /2009-04-04/user-data HTTP/1.1 > Host: 169.254.169.254 > User-Agent: curl/7.59.0 > Accept: */* Tcpdump -vvv looks like this : sudo tcpdump -vvv -i eth0 host 169.254.169.254 tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes 20:36:22.284195 IP (tos 0x0, ttl 64, id 11248, offset 0, flags [DF], proto TCP (6), length 60) kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896 > 169.254.169.254.http: Flags [S], cksum 0x5e34 (incorrect -> 0x51a5), seq 2359174217, win 26730, options [mss 8910,sackOK,TS val 583688334 ecr 0,nop,wscale 7], length 0 20:36:22.284793 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 169.254.169.254.http > kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896: Flags [S.], cksum 0x9ed1 (correct), seq 2099498427, ack 2359174218, win 26694, options [mss 8910,sackOK,TS val 4068504964 ecr 583688334,nop,wscale 9], length 0 20:36:22.284875 IP (tos 0x0, ttl 64, id 11249, offset 0, flags [DF], proto TCP (6), length 52) kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896 > 169.254.169.254.http: Flags [.], cksum 0x5e2c (incorrect -> 0x522e), seq 1, ack 1, win 209, options [nop,nop,TS val 583688335 ecr 4068504964], length 0 20:36:22.284962 IP (tos 0x0, ttl 64, id 11250, offset 0, flags [DF], proto TCP (6), length 151) kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896 > 169.254.169.254.http: Flags [P.], cksum 0x5e8f (incorrect -> 0xb692), seq 1:100, ack 1, win 209, options [nop,nop,TS val 583688335 ecr 4068504964], length 99: HTTP, length: 99 GET /2009-04-04/user-data HTTP/1.1 Host: 169.254.169.254 User-Agent: curl/7.59.0 Accept: */* 20:36:22.325212 IP (tos 0x0, ttl 64, id 47160, offset 0, flags [DF], proto TCP (6), length 52) 169.254.169.254.http > kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896: Flags [.], cksum 0x523e (correct), seq 1, ack 100, win 53, options [nop,nop,TS val 4068505005 ecr 583688335], length 0 20:36:33.546523 IP (tos 0x0, ttl 64, id 11251, offset 0, flags [DF], proto TCP (6), length 52) kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896 > 169.254.169.254.http: Flags [F.], cksum 0x5e2c (incorrect -> 0x25a4), seq 100, ack 1, win 209, options [nop,nop,TS val 583699596 ecr 4068505005], length 0 20:36:33.586154 IP (tos 0x0, ttl 64, id 47169, offset 0, flags [DF], proto TCP (6), length 52) 169.254.169.254.http > kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896: Flags [.], cksum 0x91fc (correct), seq 26695, ack 101, win 53, options [nop,nop,TS val 4068516266 ecr 583699596], length 0 So the instance does receive packets as a reply to the curl, but they’re empty. I have tested on both Fedora-Atomic-27-20180212.2.x86_64.qcow2 and Fedora-AtomicHost-28-20180425.0.x86_64.qcow2 images and reproduced the issue both times. Nova-api and neutron-metadata logs show http 200 for each of my requests to nova. What could be blocking the user-data from reaching the instance in this case? Is there a way to set magnum-created intances to use configdrive instead? I must also specify that user-data works on all regular instances. It’s only affecting instances created by magnum. Best regards, Jean-Philippe Méthot Senior Openstack system administrator Administrateur système Openstack sénior PlanetHoster inc.

1 0

[all][PTL] U Community Goal: Project PTL & Contrib Docs Update #8
by Kendall Nelson 28 Apr '20

28 Apr '20

Hello Everyone! More excellent progress[1] has been made since last week! Again, thank you for all the hard work! Projects missing progress/merged status according to the story[4] are as follows: - Adjutant - Barbican - Cloudkitty - Congress - Cyborg - Ec2-API - Freezer - Heat - Horizon - Karbor - Kuryr - LOCI - Masakari - Mistral - Monasca - Murano - OpenStack Charms - openstack-chef - OpenStackAnsible - OpenStackClient - Oslo - Packaging-RPM - Placement - Puppet OpenStack - Rally - Release Management - Requirements - Senlin - Solum - Storlets - Swift - Tacker - Telemetry - Tricircle - TripleO - Winstackers - Zaqar If you are on this list, please get started IMMEDIATELY. All told, if you are doing the bare minimum of only filling out the template[2] it shouldn't take more than 5-10 minutes per repo. Please add me if you need help with reviews and let me know if there is any other help I can provide. If you have questions about the goal itself, here is the link to that[3]. And as you push patches, please be sure to update the StoryBoard story[4]. Thanks! -Kendall (diablo_rojo) [1] Progress: https://review.opendev.org/#/q/topic:project-ptl-and-contrib-docs+(status:o… [2] Cookiecutter Template: https://opendev.org/openstack/cookiecutter/src/branch/master/%7b%7bcookiecu… [3] Accepted Goal: https://governance.openstack.org/tc/goals/selected/ussuri/project-ptl-and-c… [4] Task Tracking: https://storyboard.openstack.org/#!/story/2007236

1 0

[scientific-sig] IRC Meeting today 2100 UTC - Open Infra labs, Train upgrade experiences
by Stig Telfer 28 Apr '20

28 Apr '20

Hi All, We have a Scientific SIG IRC meeting in #openstack-meeting at 2100 UTC (about 45 minutes). Everyone is welcome. Today’s agenda is here: https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_April_28th_2020 <https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_April_28th_2020> This week we have a discussion on the Open Infra Labs project, plus we are looking to share operational experiences on OpenStack Train. Cheers, Stig

1 0

[tripleo] ptg
by Wesley Hayutin 28 Apr '20

28 Apr '20

Greetings, Just a reminder to review the TripleO time slots for PTG [1] Looking for feedback on the times. We'll start a schedule etherpad for folks to align their topics to a particular time slot soon. Note, the time slot for Q&A or really anything for folks not able to join the 13-17UTC time slot. Thanks [1] https://ethercalc.openstack.org/126u8ek25noy [2] https://etherpad.opendev.org/p/tripleo-victoria-topics

1 0

Announcing the general availability of Debian packages for Ussuri
by Thomas Goirand 28 Apr '20

28 Apr '20

Hi, Even after 9 years packaging OpenStack, I'm still excited to announce the availability of the last release of OpenStack in Debian. Indeed, today, I was able to launch my first VM using Ussuri, running on top of Debian Buster. Even if we haven't run Tempest on this release yet, it's already usable, and I wanted everyone to know about it. Packages are available either from Debian Experimental (they will be uploaded to Unstable the day of the official release, then hopefully migrate automatically to Debian Testing) or from the debian.net backports to Buster, from both of these repos: deb http://buster-ussuri.debian.net/debian buster-ussuri-backports main deb-src http://buster-ussuri.debian.net/debian buster-ussuri-backports main deb http://buster-ussuri.debian.net/debian buster-ussuri-backports-nochange main deb-src http://buster-ussuri.debian.net/debian buster-ussuri-backports-nochange main *OR* from this server (which contains all release of OpenStack since the Kilo release): deb http://osbpo.debian.net/debian buster-ussuri-backports main deb-src http://osbpo.debian.net/debian buster-ussuri-backports main deb http://osbpo.debian.net/debian buster-ussuri-backports-nochange main deb-src http://osbpo.debian.net/debian buster-ussuri-backports-nochange main Please make sure to configure your servers to use *both* buster-ussuri-backports and buster-ussuri-backports-nochange suites. The first one contains packages maintained by the Debian OpenStack team, while the later are backports of packages maintained by other people in Debian (like those maintained in the Debian Python Module Team, but not only). Still missing from this release... - vitrage-dashboard: There's some new XStatic packages to make, work is ongoing. - some neutron plugins: Many aren't released yet... - heat: It just got released a few hours ago, and I didn't have time yet to figure out what's wrong with these 3 unit tests: http://paste.openstack.org/show/792781/ Hopefully, this will be addressed tomorrow. Note that OCI, the Debian cluster installer for OpenStack, already supports that version (that's how I launched my first instance), and is running the latest version of all puppet modules of puppet-openstack: https://salsa.debian.org/openstack-team/debian/openstack-cluster-installer Some fun statistics: 187 packages were updated in the OpenStack team's Git repository to make this release (plus some updates of packages not within the team). Note that the last piece of software (cinder) was converted to uwsgi, so this release comes with absolutely all API servers running under uwsgi. Please try these packages and report bugs to the Debian BTS as usual, so we can fix as many bugs as possible before the final release. Some packages are still waiting for the FTP master's approval before they can reach Debian proper. Noticeably, many Tempest plugins (as I decided to package them all in Debian). 6 of these are pending, and probably more will be packaged. Stay tuned. Cheers, Thomas Goirand (zigo)

5 5

[neutron] Low-throughput over high-latency connections
by Eric K. Miller 28 Apr '20

28 Apr '20

Hi, I'm at a loss after quite a bit of testing, so thought I'd see if someone has any ideas... We have a Stein cluster that was/is deployed with Kolla Ansible with DVR without OVS+DPDK - rather just the standard kernel OVS. Everything appears to work well with low-latency local and Internet destinations, but as the latency increases, throughput drops significantly. I would have blamed the standard bandwidth-delay product and TCP window size not being large enough, etc., but that definitely isn't the issue, since we have shown that VMs on our VMware clusters (VLAN-based, not NSX-based) do not have the issue and use the same many-10Gbps-connected ISPs to our routers as the OpenStack configuration. I am using a Vultr machine in London as a test destination since the latency is around 86ms - a good high-latency test. The good environment we are testing against: Using iperf3 with 1 thread on a VMware VM results in an average of 155Mbits/sec sent to Vultr London with a few retries recorded that lower the window size to around 1.78MiBytes. This is roughly what we expect, if not more, with faster machines. The problem: Performing the same test on a "much" faster server (E-2288G based, without hyperthreading enabled, so 4.5GHz on all 8 cores), with no other VMs on it, a VM with iperf3 with 1 thread results in an average of only around 32Mbits/sec with a window size that drops significantly at the start and increases to only about 386KiBytes before retries start to happen. This average does not change when using other machines in our cluster, including some Xeon Gold and EPYC processor machines. The only thing I can point to is OVS and/or something in Linux. However, both tests (VMware and KVM) were performed with the same fresh installation and configuration (CentOS 7 with all updates and the standard kernel). So, my doubts are that it has something to do with Linux. On the physical host, the vhost process barely registers any CPU usage, and neither does the VM - as expected on such a fast machine. I even tried to enable multiqueue, since we had it disabled. With a 2 vCPU VM, ethtool -l eth0 displays that 2 queues are in use. Same results - 32Mbits/sec throughput to Vultr London. All machines are connected via 100Gbps or 25Gbps connections with Mellanox Connect-X 4 and 5 cards with Mellanox switches, and iperf can saturate these links between the physical hosts when run on the physical hosts' CentOS. No errors (tx errors, rx errors, packet loss, etc.) are being reported on any of the logical or physical links. I thought a queue got full somewhere, so I looked for where queues were used and found that virtio has a tx_queue_size that can be adjusted, but unfortunately, it appears this can only be done with vhost-user, which I believe is specific to OVS+DPDK. The default tx_queue_size in virtio is 256 elements. I'm not sure what this represents, though, much less how to measure the queue usage statistics. Running iperf3 in "Reverse mode", so the Vultr London machine sends data back to the KVM VM, bandwidth averages around 254Mbits/sec! So the issue appears to be only in one direction - sending network traffic outbound. Running the same iperf3 test from the KVM VM to a Vultr machine that is very close to us (latency of 2ms or so), results in an average bandwidth of 373Mbits/sec, but with 39 retries per second on average. The window size never adjusts low enough to stop the retries - it averages 146KiBytes in size. Using "Reverse mode" results in an average of 1.21Gbits/sec! So, the OVS software isn't slow, but something is causing issues when transmitting data when latency is large - something that doesn't happen with the VMware VMs with the same network and Internet connections. Also, pings from the VMware and KVM VMs are nearly identical - so there isn't some crazy amount of latency caused by OVS - at least when it is essentially idle. I'm not sure where to go from here in terms of diagnostics. Any ideas? Thanks for any help, in advance! Eric

1 0

[tc][puppet][Glare] Propose retirement of puppet-glare
by Takashi Kajinami 28 Apr '20

28 Apr '20

Hi, Currently Glare[1] project is located under x instead of openstack, and IIUC this means that Glare is no longer maintained and has been retired already. [1] https://opendev.org/x/glare Regarding its status, I'll propose to retire puppet-glare[2] because it doesn't make sense to maintain a puppet module for the retired project. [2] https://github.com/openstack/puppet-glare May I ask TC to confirm my understanding about Glare project status ? Also, please let me know if anybody has concern with retiring puppet-glare. Thank you, Takashi Kajinami

4 4