November 2019 - openstack-discuss

Re: [openstack-dev] Are we ready to put stable/ocata into extended maintenance mode?
by Alex Schultz 03 Mar '21

03 Mar '21

On Tue, Sep 18, 2018 at 1:30 PM Alex Schultz <aschultz(a)redhat.com> wrote: > > On Tue, Sep 18, 2018 at 1:27 PM, Matt Riedemann <mriedemos(a)gmail.com> wrote: > > The release page says Ocata is planned to go into extended maintenance mode > > on Aug 27 [1]. There really isn't much to this except it means we don't do > > releases for Ocata anymore [2]. There is a caveat that project teams that do > > not wish to maintain stable/ocata after this point can immediately end of > > life the branch for their project [3]. We can still run CI using tags, e.g. > > if keystone goes ocata-eol, devstack on stable/ocata can still continue to > > install from stable/ocata for nova and the ocata-eol tag for keystone. > > Having said that, if there is no undue burden on the project team keeping > > the lights on for stable/ocata, I would recommend not tagging the > > stable/ocata branch end of life at this point. > > > > So, questions that need answering are: > > > > 1. Should we cut a final release for projects with stable/ocata branches > > before going into extended maintenance mode? I tend to think "yes" to flush > > the queue of backports. In fact, [3] doesn't mention it, but the resolution > > said we'd tag the branch [4] to indicate it has entered the EM phase. > > > > 2. Are there any projects that would want to skip EM and go directly to EOL > > (yes this feels like a Monopoly question)? > > > > I believe TripleO would like to EOL instead of EM for Ocata as > indicated by the thead > http://lists.openstack.org/pipermail/openstack-dev/2018-September/134671.ht… > Bringing this backup to see what we need to do to get the stable/ocata branches ended for the TripleO projects. I'm bringing this up because we have https://review.openstack.org/#/c/647009/ which is for the upcoming rename but CI is broken and we have no interest in continue to keep the stable/ocata branches alive (or fix ci for them). Thanks, -Alex > Thanks, > -Alex > > > [1] https://releases.openstack.org/ > > [2] > > https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… > > [3] > > https://docs.openstack.org/project-team-guide/stable-branches.html#extended… > > [4] > > https://governance.openstack.org/tc/resolutions/20180301-stable-branch-eol.… > > > > -- > > > > Thanks, > > > > Matt > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

5 11

[Syntribos] Does anyone still use this?
by Gage Hugo 07 May '20

07 May '20

The Security SIG has recently been looking into updating several sites/documentation and one part we discussed the last couple meetings was the security tools listings[0]. One of the projects listed there, Syntribos, doesn't appear to have much activity[1] outside of zuul/python/docs maintenance and the idea of retiring the project was mentioned. However, there does seem to be an occasional bug-fix submitted, so we are sending this email out to see if anyone is still utilizing Syntribos. If so, please either respond here, reach out to us in the #openstack-security irc channel, or fill out the section in the Security SIG Agenda etherpad[2]. Thanks! [0] https://security.openstack.org/#security-tool-development [1] https://review.opendev.org/#/q/project:openstack/syntribos [2] https://etherpad.openstack.org/p/security-agenda

2 2

[openstack-ansible] dropping suse support
by Mohammed Naser 05 May '20

05 May '20

Hi everyone, I've been trying to avoid writing this email for the longest time ever. It's really painful to have to reach this state, but I think we've hit a point where we can no longer maintain support for SUSE within OpenStack Ansible. Unfortunately, with the recent layoffs, the OSA team has taken a huge hit in terms of contributors which means that there is far less contributors within the project. This means that we have less resources to go around and it forces us to focus on a more functional, reliable and well tested set of scenarios. Over the past few cycles, there has been effort to add SUSE support to OpenStack Ansible, during the time that we had maintainers, it was great and SUSE issues were being fixed promptly. In addition, due to the larger team at the time, we found ourselves having some extra time where we can help unbreak other gates. Jesse used to call this a "labour of love", which I admired at the time and hoped we continue to do as much as we can of. However, the lack of a committed maintainer for OpenSUSE has resulted in constantly failing jobs[1][2] (which were moved to non-voting, wasting CI resources as no one fixed them). In addition, it's causing several gate blocks for a few times with no one really finding the time to clean them up. We are resource constrained at this point and we need the resource to go towards making the small subset of supported features functional (i.e. CentOS/Ubuntu). We struggle with that enough, and there seems to be no deployers that are running SUSE in real life at the moment based on bugs submitted. With that, I propose that we drop SUSE support this cycle. If anyone would like to volunteer to maintain it, we can review that option, but that would require a serious commitment as we've had maintainers step off and it hurts the velocity of the project as no one can merge code anymore. ..Really wish I didn't have to write this email Mohammed [1]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct… [2]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct…

6 9

[cinder][ops][extended-maintenance-sig][public-cloud-sig][enterprise-wg] Cinder to EOL some branches
by Brian Rosmaita 13 Apr '20

13 Apr '20

This is a courtesy notice that having received no responses to my email of 28 October [0] proposing to EOL some currently open Cinder branches, and following the policy articulated in [1], at today's Virtual PTG meeting the Cinder project team has decided to put the following stable branches into the End of Life state: driverfixes/mitaka driverfixes/newton stable/ocata stable/pike I will submit the paperwork to get this process moving one week from today (2 December 2019). cheers, brian [0] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010385.… [1] https://docs.openstack.org/project-team-guide/stable-branches.html#end-of-l…

4 8

[neutron][OVN] Multiple mechanism drivers
by Sam Morrison 07 Apr '20

07 Apr '20

We are looking at using OVN and are having some issues with it in our ML2 environment. We currently have 2 mechanism drivers in use: linuxbridge and midonet and these work well (midonet is the default tenant network driver for when users create a network) Adding OVN as a third mechanism driver causes the linuxbridge and midonet networks to stop working in terms of CRUD operations etc. It looks as if the OVN driver thinks it’s the only player and is trying to do things on ports that are in linuxbridge or midonet networks. Am I missing something here? (We’re using Stein version) Thanks, Sam

5 6

[ops] nova wsgi config
by Arnaud Morin 27 Feb '20

27 Feb '20

Hey all, I am trying to configure apache as a WSGI. Is there any other documentation than this: https://docs.openstack.org/nova/stein/user/wsgi.html Is there any recommendations? Thanks in advance! -- Arnaud Morin

4 7

[all][neutron][neutron-fwaas] Maintainers needed
by Slawek Kaplonski 20 Feb '20

20 Feb '20

Hi, Over the past couple of cycles we have noticed that new contributions and maintenance efforts for neutron-fwaas project were almost non existent. This impacts patches for bug fixes, new features and reviews. The Neutron core team is trying to at least keep the CI of this project healthy, but we don’t have enough knowledge about the details of the neutron-fwaas code base to review more complex patches. During the PTG in Shanghai we discussed that with operators and TC members during the forum session [1] and later within the Neutron team during the PTG session [2]. During these discussions, with the help of operators and TC members, we reached the conclusion that we need to have someone responsible for maintaining project. This doesn’t mean that the maintainer needs to spend full time working on this project. Rather, we need someone to be the contact person for the project, who takes care of the project’s CI and review patches. Of course that’s only a minimal requirement. If the new maintainer works on new features for the project, it’s even better :) If we don’t have any new maintainer(s) before milestone Ussuri-2, which is Feb 10 - Feb 14 according to [3], we will need to mark neutron-fwaas as deprecated and in “V” cycle we will propose to move the project from the Neutron stadium, hosted in the “openstack/“ namespace, to the unofficial projects hosted in the “x/“ namespace. So if You are using this project now, or if You have customers who are using it, please consider the possibility of maintaining it. Otherwise, please be aware that it is highly possible that the project will be deprecated and moved out from the official OpenStack projects. [1] https://etherpad.openstack.org/p/PVG-Neutron-stadium-projects-the-path-forw… [2] https://etherpad.openstack.org/p/Shanghai-Neutron-Planning-restored - Lines 379-421 [3] https://releases.openstack.org/ussuri/schedule.html -- Slawek Kaplonski Senior software engineer Red Hat

8 14

[tripleo][ironic][ansible][openstack-ansible] Ironic/Baremetal Ansible modules
by Sagi Shnaidman 21 Jan '20

21 Jan '20

Hi, all in the light of finding the new home place for openstack related ansible modules [1] I'd like to discuss the best strategy to create Ironic ansible modules. Existing Ironic modules in Ansible repo don't cover even half of Ironic functionality, don't fit current needs and definitely require an additional work. There are a few topics that require attention and better be solved before modules are written to save additional work. We prepared an etherpad [2] with all these questions and if you have ideas or suggestions on how it should look you're welcome to update it. We'd like to decide the final place for them, name conventions (the most complex one!), what they should look like and how better to implement. Anybody interested in Ansible and baremetal management in Openstack, you're more than welcome to contribute. Thanks [1] https://review.opendev.org/#/c/684740/ [2] https://etherpad.openstack.org/p/ironic-ansible-modules -- Best regards Sagi Shnaidman

9 18

[cinder] consistency group not working
by Francois Scheurer 16 Jan '20

16 Jan '20

Dear Cinder Experts We are running the rocky release. |We can create a consistency group: openstack consistency group create --volume-type b9f67298-cf68-4cb2-bed2-c806c5f83487 fsc-consgroup Bug 1: but adding volumes is not working: openstack consistency group add volume c3f49ef0-601e-4558-a75a-9b758304ce3b b48752e3-641f-4a49-a892-6cb54ab6b74d c0022411-59a4-4c7c-9474-c7ea8ccc7691 0f4c6493-dbe2-4f75-8e37-5541a267e3f2 => Invalid volume: Volume is not local to this node. (HTTP 400) (Request-ID: req-7f67934a-5835-40ef-b25c-12591fd79f85) Bug 2: deleting consistency group is also not working (silently failing): openstack consistency group delete c3f49ef0-601e-4558-a75a-9b758304ce3b |||=> AttributeError: 'RBDDriver' object has no attribute 'delete_consistencygroup'| See details below. Using the --force option makes no difference and the consistency group is not deleted. Do you think this is a bug or a configuration issue? Thank you in advance. | Cheers Francois |Details: ==> /var/lib/docker/volumes/kolla_logs/_data/cinder/cinder-api-access.log <== 10.0.129.17 - - [27/Sep/2019:12:16:24 +0200] "POST /v3/f099965b37ac41489e9cac8c9d208711/consistencygroups/3706bbab-e2df-4507-9168-08ef811e452c/delete HTTP/1.1" 202 - 109720 "-" "python-cinderclient" ==> /var/lib/docker/volumes/kolla_logs/_data/cinder/cinder-volume.log <== 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server [req-9010336e-d569-47ad-84e2-8dd8b729939c b141574ee71f49a0b53a05ae968576c5 f099965b37ac41489e9cac8c9d208711 - default default] Exception during message handling: AttributeError: 'RBDDriver' object has no attribute 'delete_consistencygroup' 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/osprofiler/profiler.py", line 159, in wrapper 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/volume/manager.py", line 3397, in delete_group 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server vol_obj.save() 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server self.force_reraise() 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/volume/manager.py", line 3362, in delete_group 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server self.driver.delete_consistencygroup(context, cg, 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server AttributeError: 'RBDDriver' object has no attribute 'delete_consistencygroup' 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server| -- EveryWare AG François Scheurer Senior Systems Engineer Zurlindenstrasse 52a CH-8003 Zürich tel: +41 44 466 60 00 fax: +41 44 466 60 10 mail: francois.scheurer(a)everyware.ch web: http://www.everyware.ch

2 2

[tc][all] Updates on Ussuri cycle community-wide goals
by Ghanshyam Mann 15 Jan '20

15 Jan '20

Hello Everyone, We have two goals with their champions ready for review. Please review and provide your feedback on Gerrit. 1. Add goal for project specific PTL and contributor guides - Kendall Nelson - https://review.opendev.org/#/c/691737/ 2. Propose a new goal to migrate all legacy zuul jobs - Luigi Toscano - https://review.opendev.org/#/c/691278/ We are still looking for the Champion volunteer for RBAC goal[1]. If you have any new ideas for goal, do not hesitate to add in etherpad[2] [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010291.… [2] https://etherpad.openstack.org/p/PVG-u-series-goals -gmann & diablo_rojo

1 2

[all][neutron][neutron-vpnaas] Maintainers needed
by Slawek Kaplonski 06 Jan '20

06 Jan '20

Hi, Over the past couple of cycles we have noticed that new contributions and maintenance efforts for neutron-vpnaas were almost non existent. This impacts patches for bug fixes, new features and reviews. The Neutron core team is trying to at least keep the CI of this project healthy, but we don’t have enough knowledge about the details of the neutron-vpnaas code base to review more complex patches. During the PTG in Shanghai we discussed that with operators and TC members during the forum session [1] and later within the Neutron team during the PTG session [2]. During these discussions, with the help of operators and TC members, we reached the conclusion that we need to have someone responsible for maintaining project. This doesn’t mean that the maintainer needs to spend full time working on this project. Rather, we need someone to be the contact person for the project, who takes care of the project’s CI and review patches. Of course that’s only a minimal requirement. If the new maintainer works on new features for the project, it’s even better :) If we don’t have any new maintainer(s) before milestone Ussuri-2, which is Feb 10 - Feb 14 according to [3], we will need to mark neutron-vpnaas as deprecated and in “V” cycle we will propose to move the project from the Neutron stadium, hosted in the “openstack/“ namespace, to the unofficial projects hosted in the “x/“ namespace. So if You are using this project now, or if You have customers who are using it, please consider the possibility of maintaining it. Otherwise, please be aware that it is highly possible that the project will be deprecated and moved out from the official OpenStack projects. [1] https://etherpad.openstack.org/p/PVG-Neutron-stadium-projects-the-path-forw… [2] https://etherpad.openstack.org/p/Shanghai-Neutron-Planning-restored - Lines 379-421 [3] https://releases.openstack.org/ussuri/schedule.html -- Slawek Kaplonski Senior software engineer Red Hat

1 1

[auto-scaling][self-healing] Discussion to merge two SIG to one
by Rico Lin 05 Jan '20

05 Jan '20

Dear all As we discussed in PTG about merge two SIG to one. I would like to continue the discussion on ML. In PTG, Eric proposes the idea to merge two SIG due to the high overlapping of domains and tasks. I think this is a great idea since, over the last 6 months, most of the discussions in both SIG are overlapped. So I'm onboard with this idea. Here's how I think we can continue this idea: 1. Create new SIG (maybe 'Automation SIG'? feel free to propose name which can cover both interest.) 2. Redirect docs and wiki to new SIG. And rework on index so there will be no confusion 3. Move repos from both SIGs to new SIG 4. Mark auto-scaling SIG and self-healing SIG as inactive. 5. remove auto-scaling SIG and self-healing SIG after a reasonable waiting time Let us know what you think about this. Otherwise, we definitely expect this to happen soon. -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

4 7

[meta-sig][multi-arch] propose forming a Multi-arch SIG
by Rico Lin 05 Jan '20

05 Jan '20

Dear all In summit, there's a forum for ARM support [1] in Summit which many people show they're interested in ARM support for OpenStack. And since also we have Linaro shows interest in donating servers to OpenStack infra. It's time for community to think about what we should deal with those ARM servers once we have them in community infrastructure. One thing we should do as a community is to gather people for this topic. So I propose we create a Multi-arch SIG and aim to support ARM architecture as very first step. I had the idea to call it ARM SIG before, but since there might be high overlap knowledge between support ARM 64 and other architectures. I propose we go for Multi-arch instead. This SIG will be a nice place to collect all the documents, gate jobs, and to trace tasks. If you're also interested in that group, please reply to this email, introduce yourself and tell us what you would like the group scope and objectives to be, and what you can contribute to the group. [1] https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/24355… -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

11 15

[heat] Addressing the large patch backlog.
by David Peacock 23 Dec '19

23 Dec '19

Hi all, For those interested in the workings of the Heat project, I'd like to kick off a call to action. At the time of writing there are approximately 200 open patches against the core Heat project repo alone, not counting the other Heat repos. Recently I started going through and triaging the patches I'd consider "historical" with an arbitrary cut off for this definition of August 1st of this year. There are 148 patches which meet this definition, dating all the way back to 2015. I have gone through them all and placed them into a spreadsheet [0] which I'd invite you all to check. Provided is a link to the patch in question, initial upload date, last meaningful update date, primary author, and a high level summary of the patch. Additionally I've broken the patches down into three recommended states based on a high level first pass. *Abandon* 34 patches are candidates to be abandoned; they usually are of debatable utility, have significant outstanding concerns, or have no followup from the original developer in a very long time. In many cases, all of these conditions. *Without good reason or explanation from the original developer, these patches may ultimately be cleared out.* *Rebase + Merge* 38 patches are with a high level look in reasonably good shape, perform a stated goal, and may be trivial to core review and ultimately rebase and merge. *If you're the original developer or otherwise interested in these patches and wish to see them through the merge process, please rebase the patch.* *Research* 76 patches are sufficiently complex that they'll need a much closer look. Some of these patches are in a seemingly "finished" state, some are a way off. Some have unanswered concerns from core review and have been left dangling. *If you're the original developer or otherwise interested in working these patches through to completion, please do get involved.* When I started this little mission I wasn't quite sure what to expect. What I have found is that as much as there was anticipated cruft to clear out, there is a great deal of very good work lurking here, waiting to see the light of day, and it would be so good to see this work realised. :-) If you have anything to say, feel free to write back on list, and if you'd like to coordinate with me any efforts with these patches I can be found by email or on Freenode in the #heat channel; I'm dpeacock. Based on feedback of this idea, and indeed on each individual patch, I hope we can get this backlog under control, and harvest some of this excellent code! Thank you, David Peacock [0] https://ethercalc.openstack.org/b3qtqyhkg9g1 Please be mindful of accidental edits.

2 2

State of the Gate
by Matt Riedemann 20 Dec '19

20 Dec '19

Things are great! Surprise! I just wanted to let everyone know. Later! . . . . . Now that you've been tricked, on Halloween no less, I'm here to tell you that things suck right now. This is your periodic digest of issues. Grab some fun-sized candy bars and read on. I think right now we have three major issues. 1. http://status.openstack.org/elastic-recheck/index.html#1763070 This has resurfaced and I'm not sure why, nor do I think we ever had a great handle on what is causing this or how to work around it so if anyone has new ideas please chip in. 2. http://status.openstack.org/elastic-recheck/index.html#1844929 I've done some digging into this one and my notes are in the bug report. It mostly affects grenade jobs but is not entirely restricted to grenade jobs. It's also mostly on OVH and FortNebula nodes but not totally. From looking at the mysql logs in the grenade jobs, mysqld is (re)started three times. I think (1) for initial package install, (2) for stacking devstack on the old side, and (3) for stacking devstack on the new side. After the last restart, there are a lot of aborted connection messages in the msyql error log. It's around then that grenade is running post-upgrade smoke tests to create a server and the nova-scheduler times out communicating with the nova_cell1 database. I have a few patches up to grenade/devstack [1] to try some things and get more msyql logs but so far they aren't really helpful. We need someone with some more mysql debugging experience to help here, maybe zzzeek or mordred? 3. CirrOS guest SSH issues There are several (some might be duplicates): http://status.openstack.org/elastic-recheck/index.html#1848078 http://status.openstack.org/elastic-recheck/index.html#1808010 (most hits) http://status.openstack.org/elastic-recheck/index.html#1463631 http://status.openstack.org/elastic-recheck/index.html#1849857 http://status.openstack.org/elastic-recheck/index.html#1737039 http://status.openstack.org/elastic-recheck/index.html#1840355 http://status.openstack.org/elastic-recheck/index.html#1843610 A few notes here. a) We're still using CirrOS 0.4.0 since Stein: https://review.opendev.org/#/c/521825/ And that image was published nearly 2 years ago and there are no newer versions on the CirrOS download site so we can't try a newer image to see if that fixes things. b) Some of the issues above are related to running out of disk in the guest. I'm not sure what is causing that, but I have posted a devstack patch that is related: https://review.opendev.org/#/c/690991 tl;dr before Stein the tempest flavors we used had disk=0 so nova would fit the root disk to the size of the image. Since Stein the tempest flavors specify root disk size (1GiB for the CirrOS images). My patch pads an extra 1GiB to the root disk on the tempest flavors. One side effect of that is the volumes tempest creates will go from 1GiB to 2GiB which could be a problem if a lot of tempest volume tests run at the same time, though we do have a volume group size of 24GB in gate runs so I think we're OK for now. I'm not sure my patch would help, but it's an idea. As for the other key generation and dhcp lease failures, I don't know what to do about those. We need more eyes on these issues to generate some ideas or see if we're doing something wrong in our tests, e.g. generating too much data for the config drive? Not using config drive in some cases? Metadata API server is too slow (note we cache the metadata since [2])? Other ideas on injecting logs for debug? [1] https://review.opendev.org/#/q/topic:bug/1844929+status:open [2] https://review.opendev.org/#/q/I9082be077b59acd3a39910fa64e29147cb5c2dd7 -- Thanks, Matt

7 23

[resource-management-sig] Status of the "Resource Management" SIG
by Thierry Carrez 18 Dec '19

18 Dec '19

Hi, I was wondering about the status of the Resource Management SIG... It's been "forming" according to https://wiki.openstack.org/wiki/Res_Mgmt_SIG since January 2018... And I could'nt find a reference or log to any meeting after that. Does anyone have updated status on this one? Should it be removed from the list of active SIGs at https://governance.openstack.org/sigs/ ? -- Thierry Carrez (ttx)

5 6

Re: [OpenStack Foundation] [tc][board][all] - Adding OpenStack community support to the savedotorg campaign
by Graham Hayes 16 Dec '19

16 Dec '19

On 26/11/2019 16:44, Alan Clark wrote: > Who owns savedotorg.org? I see EFF references on the page but when I look it up the domain it appears to all be "REDACTED". >From https://savedotorg.org/index.php/about/ - it is ran by NTEN (https://www.nten.org/) It is supported by the EFF (among others) > >> -----Original Message----- >> From: Graham Hayes [mailto:gr@ham.ie] >> Sent: Tuesday, November 26, 2019 9:35 AM >> To: Jimmy McArthur <jimmy(a)openstack.org> >> Cc: openstack-discuss(a)lists.openstack.org; foundation- >> board(a)lists.openstack.org; foundation(a)lists.openstack.org >> Subject: Re: [OpenStack Foundation] [tc][board][all] - Adding OpenStack >> community support to the savedotorg campaign >> >> On 26/11/2019 16:16, Jimmy McArthur wrote: >>> From my reading of the situation, it’s a done deal. It seems like ICAAN just >> crammed it through. Is there new info out that could reverse it? >>> >>> Thanks, >>> Jimmy >> >> My understanding is it looks bleak, but there is a chance to shame the owners >> into halting the sale. >> >> I think ICAAN can still block the same (until December 13th, 30 days after they >> were notified), and through pure poor marketing, and raising our voices, this >> could help convince the current owners of PIR (who is ISOC) to re-consider the >> idea. >> >> >>>> On Nov 26, 2019, at 8:50 AM, Amy <amy(a)demarco.com> wrote: >>>> >>>> I agree with the OSF submitting on behalf of OpenStack and the other projects. >> I also like the idea of individuals signing as contributors. >>>> >>>> Amy Marrich (spotz) >>>> >>>>>> On Nov 26, 2019, at 8:17 AM, Sean McGinnis <sean.mcginnis(a)gmail.com> >> wrote: >>>>>> >>>>>> On Tue, Nov 26, 2019 at 7:00 AM Graham Hayes <gr(a)ham.ie> wrote: >>>>>> >>>>>> Hey All, >>>>>> >>>>>> I am not sure if this has been seen by everyone or not - but there >>>>>> is a change in how the .org top level domain is being ran in the >>>>>> works, in a way that may not be in the best interests of the non >>>>>> profits that it was created to facilitate. [1] >>>>>> >>>>>> A lot of well known non profits have already joined, and as a >>>>>> community that has an interest in the internet as a whole, and uses >>>>>> a .org domain, I think we should add our voice in support. >>>>>> >>>>>> What do people think? Are we happy to have the TC use its voice on >>>>>> behalf of the OpenStack project, or do we think the board should >>>>>> use its voice on behalf of the entire foundation? >>>>>> >>>>>> - Graham >>>>>> >>>>>> >>>>>> 1 - https://savedotorg.org/ >>>>>> >>>>> >>>>> I agree this is a Foundation level thing (though individuals can add >>>>> their names to the petition as well). I would support adding OSF to >>>>> the list of organizations. >>>>> >>>>> Sean >>>>> >>>> >>>> _______________________________________________ >>>> Foundation mailing list >>>> Foundation(a)lists.openstack.org >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/foundation >>> >> >> >> _______________________________________________ >> Foundation mailing list >> Foundation(a)lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/foundation

5 6

[nova][ptg] Continue QoS port support
by Balázs Gibizer 14 Dec '19

14 Dec '19

Hi Novas! Here is my summarized plans for Ussuri about continuing the work on supporting qos neutron ports (those that has bandwidth resource request) in nova. What is missing: * Tempest coverage is missing for migrate and resize support that is merged in Train. This work is already underway and bugs has been caught [1][2] * Support evacuate, live migrate, unshelve. The work is described in [3][4] and the first set of patches for the evacuation support is up for review [5] * Support for cross cell resize with qos port needs some work. Matt prepared the cross cell resize code already in a way that no new RPC change will be needed [6] and I have a plan what to do [7]. * InstancePCIRequest persists parent_ifname during migration but the such change is not rolled back if the migration fails. This is ugly but I think it does not cause any issues [8]. I will look into this to remove the ugliness. The bandwidth support for the nova-manage heal_allocation tool was merged in Train. Originally I planned to backport that to Stein but that patch grown so big and incorporated may refactors along the way that I'm not sure any more that it is reasonable to backport it. I'm now thinking about keeping it as-is and suggesting operators to install Train nova in a virtualenv to run heal allocations for bandwidth aware servers if needed in Stein. I do have to run some manual tests to see if it actually works. Any feedback is welcome! cheers, gibi [1] https://bugs.launchpad.net/nova/+bug/1849695 [2] https://bugs.launchpad.net/nova/+bug/1849657 [3] https://blueprints.launchpad.net/nova/+spec/support-move-ops-with-qos-ports… [4] https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/supp… [5] https://review.opendev.org/#/q/status:open+project:openstack/nova+branch:ma… [6] https://review.opendev.org/#/c/635080/43/nova/compute/manager.py@5375 [7] https://review.opendev.org/#/c/633293/49/nova/compute/manager.py@4742 [8] https://review.opendev.org/#/c/688387/6/nova/compute/manager.py@3404

2 3

[nova][cinder][ops] question/confirmation of legacy vol attachment migration
by Matt Riedemann 06 Dec '19

06 Dec '19

Hello Cinderinos, I've now got a working patch that migrates legacy volume attachments to new style v3 attachments [1]. The fun stuff showing it working is in this paste [2]. We want to do this data migration in nova because we still have a lot of compatibility code since Queens for pre-v3 style attachments and we can't remove that compatibility code (ever) if we don't first make sure we provide a data migration routine for operators to roll through. So for example if this lands in Ussuri we can can enforce a nova-status upgrade check in V and rip out code in X. Without digging into the patch, this is the flow: 1. On nova-compute restart, query the nova DB for instances on the compute host with legacy volume attachments. 2. For each of those, create a new style attachment with the host connector and update the BlockDeviceMapping information in the nova DB (attachment_id and connection_info). 3. Delete the existing legacy attachment so when the server is deleted the volume status goes back to 'available' due to proper attachment reference counting in the Cinder DB. My main question is on #3. Right now I'm calling the v3 attachment delete API rather than the v2 os-terminate_connection API. Is that sufficient to cleanup the legacy attachment on the storage backend even though the connection was created via os-initialize_connection originally? Looking at the cinder code, attachment_delete hits the connection terminate code under the covers [3]. So that looks OK. The only thing I can really think of is if a host connector is not provided or tracked with the legacy attachment, is that going to cause problems? Note that I think volume drivers are already required to deal with that today anyway because of the "local delete" scenario in the compute API where the compute host that the server is on is down and thus we don't have a host connector to provide to Cinder to terminate the connection. So Cinder people, are you OK with this flow? Hello Novaheads, Do you have any issues with the above? Note the migration routine is threaded out on compute start so it doesn't block, similar to the ironic flavor data migration introduced in Pike. One question I have is if we should add a config option for this so operators can enable/disable it as needed. Note that this requires nova to be configured with a service user that has the admin role to do this stuff in cinder since we don't have a user token, similar to nova doing things with neutron ports without a user token. Testing this with devstack requires [4]. By default [cinder]/auth_type is None and not required so by default this migration routine is not going to run so maybe that is sufficient? Hello Operatorites, Do you have any issues with what's proposed above? [1] https://review.opendev.org/#/c/549130/ [2] http://paste.openstack.org/show/781063/ [3] https://github.com/openstack/cinder/blob/410791580ef60ddb03104bf20766859ed9… [4] https://review.opendev.org/#/c/685488/ -- Thanks, Matt

5 10

[neutron] Proposing Jakub Libosvar as Neutron core reviewer
by Slawek Kaplonski 05 Dec '19

05 Dec '19

Hi neutrinos, We already started process of migrating networking-ovn driver to be one of in-tree neutron drivers. Blueprint for that is [1]. As part of this process I today proposed to include networking-ovn-core group into neutron-core group. Mail about it can be found at [2]. One of persons in networking-ovn-group is Jakub Libosvar who was Neutron core for very long time in the past. He knows very well not only ovn related code but also have great knowledge about all Neutron code base. So I would like to propose to Jakub as Neutron core reviewer again as he will be back working on neutron again now, after ovn will be in-tree driver. What do You think about it? I will wait for Your opinions for 1 week from now. Thx for all Your comments about it. [1] https://blueprints.launchpad.net/neutron/+spec/neutron-ovn-merge [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011240… — Slawek Kaplonski Senior software engineer Red Hat

7 7

[manila] Remote-only PTG poll
by Goutham Pacha Ravi 04 Dec '19

04 Dec '19

Hey Zorillas* and interested Stackers, (* manila contributors in the wild) A part of our community will be attending the Summit+Forum+PTG in Shanghai in a couple of weeks. However, we haven't figured out a way to enable remote participation during our PTG (Wednesday, 6th Nov 2019 between 9:00 AM and 4:30 PM) there. So, as discussed during our weekly meeting, I'd like to take a poll of when you would like to gather for a remote PTG that would supplement the gathering in Shanghai. Please take this poll and indicate your availability: https://xoyondo.com/dp/qQbsoHtNhi4DCki The prospective date for this meeting is between 2nd Dec and 13th Dec (Ussuri-1 Milestone is on 12th Dec). The meeting time will be 15:00 UTC to 22:00 UTC. This time may not work for our contributors in the Far East (APAC), but I'm hoping we'll address them primarily at the Shanghai get together, and relay their thoughts here. We will also be recording the entire meeting and will share notes on these lists. Thanks, Goutham

1 1

[neutron][nova] Rootwrap daemon and privsep
by Arnaud Morin 03 Dec '19

03 Dec '19

Hey, If I believe both the nova and neutron documentation: https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.us… https://docs.openstack.org/neutron/latest/configuration/neutron.html#agent.… At scale, we are supposed to enable the rootwrap-daemon option. I have issues enabling that, but before going further on my issue, I'd like to understand the difference with privsep daemon. Is privsep a new daemon which is supposed to replace the rootwrap one? Is privsep being launch after rootwrap? IS privsep enabled by default, so I should not care about rootwrap at all? I'd like to understand more about that. Everything I try to find on popular search engine seems outdated, so if someone could give me a hand on finding the good page to understand that, I'l love it :p Cheers, -- Arnaud Morin

3 7

[gnocchi][telemetry][ceilometer][cloudkitty] Gnocchi unmaintained
by Radosław Piliszek 03 Dec '19

03 Dec '19

Hello Folks, It looks like gnocchi is "officially" marked as unmaintained: https://github.com/gnocchixyz/gnocchi/issues/1049 Has there been any discussion regarding how it affects OpenStack projects? And/or are there any plans to amend this situation? -yoctozepto

15 28

[stein][manila-ui] error
by Ignazio Cassano 02 Dec '19

02 Dec '19

Hello, I just installed openstack stein on centos. Manila works fine my command line but when I click "share" in the dashboard the following error appears: Environment: Request Method: GET Request URL: http://10.102.184.190/dashboard/project/shares/ Django Version: 1.11.20 Python Version: 2.7.5 Installed Applications: ['openstack_dashboard.dashboards.project', 'neutron_lbaas_dashboard', 'heat_dashboard', 'openstack_dashboard.dashboards.admin', 'openstack_dashboard.dashboards.identity', 'openstack_dashboard.dashboards.settings', 'dashboards', 'openstack_dashboard', 'django.contrib.contenttypes', 'django.contrib.auth', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.humanize', 'django_pyscss', 'debreach', 'openstack_dashboard.django_pyscss_fix', 'compressor', 'horizon', 'openstack_auth'] Installed Middleware: ('openstack_auth.middleware.OpenstackAuthMonkeyPatchMiddleware', 'debreach.middleware.RandomCommentMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'horizon.middleware.OperationLogMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'horizon.middleware.HorizonMiddleware', 'horizon.themes.ThemeMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerClientMiddleware', 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerMiddleware') Traceback: File "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py" in inner 41. response = get_response(request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" in _get_response 187. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" in _get_response 185. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 36. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 52. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 36. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 113. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 84. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" in view 68. return self.dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" in dispatch 88. return handler(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in get 223. handled = self.construct_tables() File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in construct_tables 214. handled = self.handle_table(table) File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in handle_table 123. data = self._get_data_dict() File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in _get_data_dict 43. data.extend(func()) File "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py" in wrapped 109. value = cache[key] = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py" in get_shares_data 57. share_nets = manila.share_network_list(self.request) File "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py" in share_network_list 280. return manilaclient(request).share_networks.list(detailed=detailed, Exception Type: AttributeError at /project/shares/ Exception Value: 'NoneType' object has no attribute 'share_networks' Anyone can help, please ?

3 7

[cinder] Anastasiya accepted for Outreachy
by Sofia Enriquez 02 Dec '19

02 Dec '19

Hi Cinder team, I'd like to announce that Anastasiya will be working with us improving the Tempest coverage this round. The internship schedule starts on Dec. 3, 2019, to March 3, 2020. Feel free to reach her on IRC *as anastzhyr* if something comes up. On the other hand, If you have any suggestions for tempest test scenarios or possible ideas, please let me know. Regards, Sofi -- L. Sofía Enriquez she/her Associate Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

6 7

[mitaka][keystone] Authentication over keycloak server possible?
by Michael Stang 02 Dec '19

02 Dec '19

Hi, we have an OpenStack Mitaka installation running (yes I know it's pretty old ;-) ) at our lab and would like to use the keycloak-server from the central IT for authentication. So I would like to know if it is already possible in mitaka to use this external keycloak server or if this only possible in a later OpenStack version? Maybe anyone know and if yes is there any documentation how to do it? Was searching for it but found not much about it by now... Thanks :) Kind regards Michael Michael Stang Laboringenieur, Dipl. Inf. (FH) Duale Hochschule Baden-Württemberg Mannheim Baden-Wuerttemberg Cooperative State University Mannheim ZeMath Zentrum für mathematisch-naturwissenschaftliches Basiswissen Fachbereich Informatik, Fakultät Technik Coblitzallee 1-9 68163 Mannheim michael.stang(a)dhbw-mannheim.de mailto:michael.stang@dhbw-mannheim.de http://www.mannheim.dhbw.de http://www.dhbw-mannheim.de/

2 2

[all] Nominations for the "V" release name
by Sean McGinnis 02 Dec '19

02 Dec '19

Hey everyone, There is ongoing discussion about changing our release naming process, but for the time being we are going to stick with what we have been doing. That means it's time to start thinking about the "V" release name! The next developer event will take place in Vancouver, BC. The geographic location for this release will be things starting with "V" in the British Columbia province. The nomination period is now open. Please add suitable names to https://wiki.openstack.org/wiki/Release_Naming/V_Proposals. We will accept nominations until December 6, 2019 23:59:59 UTC. A recap of our current naming rules: * Each release name must start with the letter of the ISO basic Latin alphabet following the initial letter of the previous release, starting with the initial release of "Austin". After "Z", the next name should start with "A" again. * The name must be composed only of the 26 characters of the ISO basic Latin alphabet. Names which can be transliterated into this character set are also acceptable. * The name must refer to the physical or human geography of the region encompassing the location of the OpenStack design summit for the corresponding release. The exact boundaries of the geographic region under consideration must be declared before the opening of nominations, as part of the initiation of the selection process. * The name must be a single word with a maximum of 10 characters. Words that describe the feature should not be included, so "Foo City" or "Foo Peak" would both be eligible as "Foo". Names which do not meet these criteria but otherwise sound really cool should be added to a separate section of the wiki page and the TC may make an exception for one or more of them to be considered in the Condorcet poll. The naming official is responsible for presenting the list of exceptional names for consideration to the TC before the poll opens. Additional information about the release naming process can be found here: https://governance.openstack.org/tc/reference/release-naming.html Looking forward to having a name for our next release! Sean

4 7

[neutron] Including networking-ovn-core team into neutron-core
by Slawek Kaplonski 02 Dec '19

02 Dec '19

Hi neutrinos, We recently merged spec [1] and now we are starting to moving networking-ovn code to be in neutron tree. Blueprint to track progress of this is on [2]. As a consequence of this merge of code, I think that we need to also include networking-ovn-core team into neutron-core to give people who are cores on networking-ovn today ability to merge ovn related patches in neutron after this migration will be done. Of course current networking-ovn cores should only approve patches related to ovn driver, and not approve patches related to different areas of neutron code. So if there will be no objections for that until the end of this week, I will include networking-ovn-core group in neutron-core. [1] https://review.opendev.org/#/c/658414/ [2] https://blueprints.launchpad.net/neutron/+spec/neutron-ovn-merge — Slawek Kaplonski Senior software engineer Red Hat

2 2

[Usage][Designate] Salve Bind servers
by Steven Relf 01 Dec '19

01 Dec '19

Hi list.... I'm looking at making use of Designate, and have so farmanaged to get designate configured and working against a single bind9 server. The issue I'm having, and this could be due to notunderstanding bind config well, is how do I configure other bind servers to beslaves of the server used by designate. I keep running into an issue where theslave doesn’t want to do a transfer and errors with "received notify forzone 'domain': not authoritative" I'm assuming this has something to do with the slave notknowing about the zone. So the question, do I have to have all my bind servershooked upto designate, and managed with a mdns service, or am I missingsomething obvious. Rgds Steve. The future has already arrived. It's just not evenlydistributed yet - William Gibson

2 1

[Magnum] Virtual PTG planning
by Feilong Wang 01 Dec '19

01 Dec '19

Hi team, As we discussed on last weekly team meeting, we'd like to have a virtual PTG before the Xmas holiday to plan our work for the U release. The general idea is extending our current weekly meeting time from 1 hour to 2 hours and having 2 sessions with total 4 hours. My current proposal is as below, please reply if you have question or comments. Thanks. Pre discussion/Ideas collection: 20th Nov 9:00AM-10:00AM UTC 1st Session: 27th Nov 9:00AM-11:00AM UTC 2nd Session: 4th Dec 9:00AM-11:00AM UTC -- Cheers & Best regards, Feilong Wang (王飞龙) Head of R&D Catalyst Cloud - Cloud Native New Zealand -------------------------------------------------------------------------- Tel: +64-48032246 Email: flwang(a)catalyst.net.nz Level 6, Catalyst House, 150 Willis Street, Wellington --------------------------------------------------------------------------

3 4

Openstack - Rocky Install
by doug schmidt 30 Nov '19

30 Nov '19

Hi, I'm having an issue with httpd not starting on my install. I have installed rocky on a windows 10 laptop with virtualbox and centos 7 guests. So far I have 2 controllers and 3 compute nodes. I have followed the install docs for rocky. https://docs.openstack.org/install-guide/openstack-services.html#minimal-de… Minimal services have been installed and configured.keystone, glance, nova, neutron are configured and running. I followed the install for horizon dashboard, and that is where I'm getting httpd.service failing. I have gone over the configuration a few times, but I can not find what the issue is. Any ideas of where to look or what to fix? Thanks for any help with this ------------------------ [root@openstack-cntr1 ~]# systemctl | egrep -i 'openstack|neutron|httpd' ● httpd.service loaded failed failed The Apache HTTP Server neutron-dhcp-agent.service loaded active running OpenStack Neutron DHCP Agent neutron-linuxbridge-agent.service loaded active running OpenStack Neutron Linux Bridge Agent neutron-metadata-agent.service loaded active running OpenStack Neutron Metadata Agent neutron-server.service loaded active running OpenStack Neutron Server openstack-glance-api.service loaded active running OpenStack Image Service (code-named Glance) API server openstack-glance-registry.service loaded active running OpenStack Image Service (code-named Glance) Registry server openstack-nova-api.service loaded active running OpenStack Nova API Server openstack-nova-conductor.service loaded active running OpenStack Nova Conductor Server openstack-nova-consoleauth.service loaded active running OpenStack Nova VNC console auth Server openstack-nova-novncproxy.service loaded active running OpenStack Nova NoVNC Proxy Server openstack-nova-scheduler.service loaded active running OpenStack Nova Scheduler Server [root@openstack-cntr1 ~]# systemctl start httpd.service Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details. [root@openstack-cntr1 ~]# systemctl status httpd.service ● httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled) Drop-In: /usr/lib/systemd/system/httpd.service.d └─openstack-dashboard.conf Active: failed (Result: exit-code) since Sat 2019-11-30 09:45:46 EST; 15s ago Docs: man:httpd(8) man:apachectl(8) Process: 12855 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear -v0 (code=exited, status=1/FAILURE) Nov 30 09:45:46 openstack-cntr1 python[12855]: File "/usr/share/openstack-dashboard/openstack_dashboard/settings.py", line 376, in <module> Nov 30 09:45:46 openstack-cntr1 python[12855]: from local.local_settings import * # noqa: F403,H303 Nov 30 09:45:46 openstack-cntr1 python[12855]: File "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.py", line 399 Nov 30 09:45:46 openstack-cntr1 python[12855]: 'supported_vnic_types': ['*'], Nov 30 09:45:46 openstack-cntr1 python[12855]: ^ Nov 30 09:45:46 openstack-cntr1 python[12855]: IndentationError: unexpected indent Nov 30 09:45:46 openstack-cntr1 systemd[1]: httpd.service: control process exited, code=exited status=1 Nov 30 09:45:46 openstack-cntr1 systemd[1]: Failed to start The Apache HTTP Server. Nov 30 09:45:46 openstack-cntr1 systemd[1]: Unit httpd.service entered failed state. Nov 30 09:45:46 openstack-cntr1 systemd[1]: httpd.service failed. [root@openstack-cntr1 ~]#

3 2

[requirements][stable] Capping requirements in stable branches
by Akihiro Motoki 30 Nov '19

30 Nov '19

Hi, I have a question on version capping in requirements files in stable branches. When some newer version of dependent library does not work with my project, do we accept a patch to add version capping of the library in stable branches? For example, the horizon team received a patch to novaclient to <16 [1]. novaclient 16.0.0 was released after Train release, so there is no surprise that we don't have the version cap novaclient<16 in Train release. My understanding is that we don't usually cap versions of libraries after the release, but I am sending this mail to check our general guideline. Thanks, Akihiro Motoki (irc: amotoki) [1] https://review.opendev.org/#/c/693000/

4 6

[neutron] security group api slow at scale
by Erik Olof Gunnar Andersson 29 Nov '19

29 Nov '19

I was wondering if there has been any work since Rocky to improve the performance of CRUD operations for security groups. When you have many thousands of rules the performance of simple operations gets exponentially slower. It got a little better after we applied the patches from this bug. https://bugs.launchpad.net/neutron/+bug/1810563 But in one of our environments getting security group rule lists, even when there are only 6-8 rules in that group takes almost 60 seconds. Best Regards, Erik Olof Gunnar Andersson

1 0

[horizon][stein] issue
by Ignazio Cassano 29 Nov '19

29 Nov '19

Hello, I just installed stein on centos 7 and when I have a log of instances clicking "next" on horizon the following messages appears in httpd error log (please help me): [Fri Nov 29 10:52:47.306231 2019] [:error] [pid 27395] INFO openstack_auth.forms Login successful for user "admin" using domain "default", remote address 10.102.184.193. [Fri Nov 29 10:53:04.594482 2019] [:error] [pid 27395] ERROR django.request Internal Server Error: /dashboard/project/instances/ [Fri Nov 29 10:53:04.594503 2019] [:error] [pid 27395] Traceback (most recent call last): [Fri Nov 29 10:53:04.594507 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner [Fri Nov 29 10:53:04.594510 2019] [:error] [pid 27395] response = get_response(request) [Fri Nov 29 10:53:04.594513 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response [Fri Nov 29 10:53:04.594515 2019] [:error] [pid 27395] response = self.process_exception_by_middleware(e, request) [Fri Nov 29 10:53:04.594518 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response [Fri Nov 29 10:53:04.594521 2019] [:error] [pid 27395] response = wrapped_callback(request, *callback_args, **callback_kwargs) [Fri Nov 29 10:53:04.594523 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec [Fri Nov 29 10:53:04.594526 2019] [:error] [pid 27395] return view_func(request, *args, **kwargs) [Fri Nov 29 10:53:04.594538 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 52, in dec [Fri Nov 29 10:53:04.594561 2019] [:error] [pid 27395] return view_func(request, *args, **kwargs) [Fri Nov 29 10:53:04.594563 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec [Fri Nov 29 10:53:04.594566 2019] [:error] [pid 27395] return view_func(request, *args, **kwargs) [Fri Nov 29 10:53:04.594568 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 113, in dec [Fri Nov 29 10:53:04.594586 2019] [:error] [pid 27395] return view_func(request, *args, **kwargs) [Fri Nov 29 10:53:04.594589 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 84, in dec [Fri Nov 29 10:53:04.594591 2019] [:error] [pid 27395] return view_func(request, *args, **kwargs) [Fri Nov 29 10:53:04.594594 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view [Fri Nov 29 10:53:04.594597 2019] [:error] [pid 27395] return self.dispatch(request, *args, **kwargs) [Fri Nov 29 10:53:04.594600 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, in dispatch [Fri Nov 29 10:53:04.594603 2019] [:error] [pid 27395] return handler(request, *args, **kwargs) [Fri Nov 29 10:53:04.594605 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 223, in get [Fri Nov 29 10:53:04.594608 2019] [:error] [pid 27395] handled = self.construct_tables() [Fri Nov 29 10:53:04.594611 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 214, in construct_tables [Fri Nov 29 10:53:04.594613 2019] [:error] [pid 27395] handled = self.handle_table(table) [Fri Nov 29 10:53:04.594616 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 123, in handle_table [Fri Nov 29 10:53:04.594638 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 252, in _get_data_dict [Fri Nov 29 10:53:04.594641 2019] [:error] [pid 27395] self._data = {self.table_class._meta.name: self.get_data()} [Fri Nov 29 10:53:04.594644 2019] [:error] [pid 27395] File "/usr/share/openstack-dashboard/openstack_dashboard/dashboards/project/instances/views.py", line 186, in get_data [Fri Nov 29 10:53:04.594661 2019] [:error] [pid 27395] boot_volume.volume_image_metadata['image_id'] in [Fri Nov 29 10:53:04.594664 2019] [:error] [pid 27395] KeyError: 'image_id

2 3

[horizon][stein]
by Ignazio Cassano 29 Nov '19

29 Nov '19

Hello, I just installed stein on centos 7 and when I have a log of instances clicking "next" on horizon the following images appears. Please help me [image: image.png]

1 0

[zun][zun-ui][horizon] Request code review for a breaking fix
by Hongbin Lu 29 Nov '19

29 Nov '19

Hi Horizon folks, We have an issue that needs to be fixed at horizon side. Please check this bug: https://bugs.launchpad.net/zun-ui/+bug/1847889 We propose a fix on Horizon but the patch hasn't been moved forward for a while. Would I ask for a code review for the patch https://review.opendev.org/#/c/688290/ ? Without the fix, our horizon plugin couldn't work correctly. Best regards, Hongbin

2 2

[stein] [manila-ui] error
by Ignazio Cassano 28 Nov '19

28 Nov '19

Hello, I just installed openstack stein on centos. Manila works fine my command line but when I click "share" in the dashboard the following error appears: Environment: Request Method: GET Request URL: http://10.102.184.190/dashboard/project/shares/ Django Version: 1.11.20 Python Version: 2.7.5 Installed Applications: ['openstack_dashboard.dashboards.project', 'neutron_lbaas_dashboard', 'heat_dashboard', 'openstack_dashboard.dashboards.admin', 'openstack_dashboard.dashboards.identity', 'openstack_dashboard.dashboards.settings', 'dashboards', 'openstack_dashboard', 'django.contrib.contenttypes', 'django.contrib.auth', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.humanize', 'django_pyscss', 'debreach', 'openstack_dashboard.django_pyscss_fix', 'compressor', 'horizon', 'openstack_auth'] Installed Middleware: ('openstack_auth.middleware.OpenstackAuthMonkeyPatchMiddleware', 'debreach.middleware.RandomCommentMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'horizon.middleware.OperationLogMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'horizon.middleware.HorizonMiddleware', 'horizon.themes.ThemeMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerClientMiddleware', 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerMiddleware') Traceback: File "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py" in inner 41. response = get_response(request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" in _get_response 187. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" in _get_response 185. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 36. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 52. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 36. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 113. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 84. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" in view 68. return self.dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" in dispatch 88. return handler(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in get 223. handled = self.construct_tables() File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in construct_tables 214. handled = self.handle_table(table) File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in handle_table 123. data = self._get_data_dict() File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in _get_data_dict 43. data.extend(func()) File "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py" in wrapped 109. value = cache[key] = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py" in get_shares_data 57. share_nets = manila.share_network_list(self.request) File "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py" in share_network_list 280. return manilaclient(request).share_networks.list(detailed=detailed, Exception Type: AttributeError at /project/shares/ Exception Value: 'NoneType' object has no attribute 'share_networks' Anyone can help, please ? Ignazio

1 0

Re: [nova] There is no screen for UEFI instance from console
by YuChengDe 28 Nov '19

28 Nov '19

Hi Marcin Thanks~ I cloud not find uefi mode cloud image in ubuntu bionic. Instead, use the image bionic-server-cloudimg-amd64.img with metadata "hw_firmware_type: uefi" https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64… However, console shows no screen. Besides, neither creating a CentOS image with console available isn't work. GRUB_CMDLINE_LINUX = "... console=tty0 console=ttyS0,115200n8" Cloud you please provide some suggestions how to implement console input/output with uefi mode os. My appreciation~ Hi. After launch an UEFI mode instance in OpenStack, the console screen show the message “Guest has no initialized the displayed (yet) " Engage your help, many thanks Base on ubuntu cloud image, listed as below https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64… // // Please check with bionic image. // // On arm64 xenial image was unable to initialize graphics for uefi hosts. -- ————————————————————————————— 九州云信息科技有限公司 99CLOUD Inc. 于成德产品开发部邮箱(Email): yu.chengde(a)99cloud.net 手机(Mobile): 13816965096 地址(Addr): 上海市局门路427号1号楼206 Room 206, Bldg 1, No.427 JuMen Road, ShangHai, China 网址(Site): http://www.99cloud.net

1 0

Neutron sfc not working in openstack stein
by Brijesh 28 Nov '19

28 Nov '19

Hi I have installed openstack stein using Kolla Ansible along with neutron sfc. Everything is working fine apart from SFC. I have compared all the configuration related to sfc and it seems fine with me. I was trying to create a SFC with following flow: VM1 --> VM2 --> VM3 When i am pinging from vm1 to vm3 without SFC creation everything is working fine. But when i try to ping after SFC i am not able to ping. I cann't see packet on vm2 (using tcpdump) on any interface after sfc. I tried to see packet drop on br-int and br-tun using openflow rule but i cannot see any packet drop on that also. I have one controller and one compute, so all instances are on same host. All vm have 2 port of same network one is ingress and one egress. vm1 has port as p11 and p12 vm2 has port as p13 and p14 vm3 has port as p15 and p16 10.0.0.174 (vm3 ip) openstack server list +--------------------------------------+------+--------+---------------------------------+--------+----------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+------+--------+---------------------------------+--------+----------+ | ea663fee-c0c4-479d-8d33-42014bd633af | x3 | ACTIVE | demo-net=10.0.0.174, 10.0.0.76 | xenial | m1.small | | 11203289-e008-4a71-adee-51c69546a18c | x2 | ACTIVE | demo-net=10.0.0.187, 10.0.0.126 | xenial | m1.small | | 1699e74c-706b-42b8-b340-7f7077b062d6 | x1 | ACTIVE | demo-net=10.0.0.122, 10.0.0.42 | xenial | m1.small | +--------------------------------------+------+--------+---------------------------------+--------+----------+ openstack port list |grep p1 | 1391dea1-32bc-42d8-aafa-d66f9003a816 | p13 | fa:16:3e:0b:08:7d | ip_address='10.0.0.187', subnet_id='949b8df3-9832-42fb-86c1-0483f57d31d8' | ACTIVE | | 3f6e9791-940a-4466-b085-521a65511bbb | p16 | fa:16:3e:85:54:ef | ip_address='10.0.0.76', subnet_id='949b8df3-9832-42fb-86c1-0483f57d31d8' | ACTIVE | | 6955e1b1-4fcf-4998-8791-4a49ed298dc5 | p14 | fa:16:3e:4d:4e:31 | ip_address='10.0.0.126', subnet_id='949b8df3-9832-42fb-86c1-0483f57d31d8' | ACTIVE | | a51175db-6272-4748-a6e6-3af719a33b86 | p12 | fa:16:3e:6a:da:60 | ip_address='10.0.0.42', subnet_id='949b8df3-9832-42fb-86c1-0483f57d31d8' | ACTIVE | | dad20742-67e4-47c7-a759-8025f042df45 | p15 | fa:16:3e:96:eb:f8 | ip_address='10.0.0.174', subnet_id='949b8df3-9832-42fb-86c1-0483f57d31d8' | ACTIVE | | db8b775f-363a-4a61-88f4-5db1b42a7681 | p11 | fa:16:3e:b6:a9:0e | ip_address='10.0.0.122', subnet_id='949b8df3-9832-42fb-86c1-0483f57d31d8' | ACTIVE | openstack sfc flow classifier list +--------------------------------------+------+-----------------------------------------------------------------+ | ID | Name | Summary | +--------------------------------------+------+-----------------------------------------------------------------+ | 20a58a64-8339-493f-8418-922c756c2b91 | FC11 | protocol: any, | | | | source[port]: any[any:any], | | | | destination[port]: 10.0.0.174/32[any:any], | | | | neutron_source_port: db8b775f-363a-4a61-88f4-5db1b42a7681, | | | | neutron_destination_port: dad20742-67e4-47c7-a759-8025f042df45, | | | | l7_parameters: {} | +--------------------------------------+------+-----------------------------------------------------------------+ openstack sfc port chain list +--------------------------------------+------+-----------------------------------------------------------------------------------------------------------------------------+-------------------------------------------+------------------------------------------------+----------+ | ID | Name | Port Pair Groups | Flow Classifiers | Chain Parameters | Chain ID | +--------------------------------------+------+-----------------------------------------------------------------------------------------------------------------------------+-------------------------------------------+------------------------------------------------+----------+ | f342d2cc-dbfe-4832-9e7c-b22cc9629891 | PC11 | [u'080395fa-21fe-4d5b-b9c2-9f196e93ac98', u'89110fed-59cd-4ac8-ac45-8ce954fa99c0', u'd16b65a4-4a23-4b38-a937-a57a7058eee0'] | [u'20a58a64-8339-493f-8418-922c756c2b91'] | {u'symmetric': False, u'correlation': u'mpls'} | 1 | +--------------------------------------+------+-----------------------------------------------------------------------------------------------------------------------------+-------------------------------------------+------------------------------------- openstack sfc port pair group list +--------------------------------------+-------+-------------------------------------------+---------------------------------------------------------------------------------------------+-------------+ | ID | Name | Port Pair | Port Pair Group Parameters | Tap Enabled | +--------------------------------------+-------+-------------------------------------------+---------------------------------------------------------------------------------------------+-------------+ | 080395fa-21fe-4d5b-b9c2-9f196e93ac98 | PPG11 | [u'10f9e355-3680-4dca-809f-7628bbbebd7f'] | {u'lb_fields': [], u'ppg_n_tuple_mapping': {u'ingress_n_tuple': {}, u'egress_n_tuple': {}}} | False | | 89110fed-59cd-4ac8-ac45-8ce954fa99c0 | PPG12 | [u'22af2ffa-72d1-477c-9b68-2582cca22f79'] | {u'lb_fields': [], u'ppg_n_tuple_mapping': {u'ingress_n_tuple': {}, u'egress_n_tuple': {}}} | False | | d16b65a4-4a23-4b38-a937-a57a7058eee0 | PPG13 | [u'cf8d263e-abab-499a-b82b-b290e85b2831'] | {u'lb_fields': [], u'ppg_n_tuple_mapping': {u'ingress_n_tuple': {}, u'egress_n_tuple': {}}} | False | +--------------------------------------+-------+-------------------------------------------+----------------------------------- Please help. If anyone can. Launchpad Bug #1854327. Thanks and Regards Brijesh DISCLAIMER: This electronic message and all of its contents, contains information which is privileged, confidential or otherwise protected from disclosure. The information contained in this electronic mail transmission is intended for use only by the individual or entity to which it is addressed. If you are not the intended recipient or may have received this electronic mail transmission in error, please notify the sender immediately and delete / destroy all copies of this electronic mail transmission without disclosing, copying, distributing, forwarding, printing or retaining any part of it. Hughes Systique accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus.

1 0

Re: [keystone][nova][barbican][neutron][cinder][tc][policy] Proposal for policy popup team⁩
by yumeng bao 28 Nov '19

28 Nov '19

>> Please nominate Yumeng Bao (yumeng_bao(a)yahoo.com) as the liaison with >> Cyborg team. She will contribute the spec. We have informed Howard of >> this. > Thanks Sundar, I have replaced Howard with Yumeng as the liaison for cyborg. > (Side note - the wiki can be edited by anyone, and I am tracking changes in it, so anyone may feel free to change their project liaison or add or remove team members and I will be notified of the change.) > Colleen Thanks Colleen for the update and the note reminder ! I will update the cyborg team progress on wiki paget later. Regards, Yumeng

1 0

[nova][api]
by Surya Seetharaman 28 Nov '19

28 Nov '19

Hello everyone, We came across this bug [1] in nova recently and wanted to know what people think is the best (relatively) way to fix this. In the past, the project id validation was added as a best effort to prevent users from being able to enter random values into the database. When this validation is used from the os flavor set/unset admin apis [2], there are chances that keystone returns a 403 which gets silently ignored by nova [3] allowing the user to enter the provided project_id/name without validation or warning or remove an existing flavor-project mapping. There were a couple of options discussed on IRC [4] to fix this behaviour out of which the practically reasonable ones are: 1) close the bug as invalid - tweak your config (we could add docs, idk if that would be found or help) to do what you need to avoid the 403 from keystone 2) change the 403 case as an error and raise it back to the compute api caller - maybe enough time has passed to not worry about backward compat with the old non-validating behavior Option 2 seems better than option 1 for most of us, however what we cannot agree upon is if this change should be accompanied by a microversion bump or not. [1] https://bugs.launchpad.net/nova/+bug/1854053 [2] https://github.com/openstack/nova/blob/fd67f69cfdaf04620f2e8a5f1fbf57370969… [3] https://github.com/openstack/nova/blob/d621914442855ce67ce0b99003f7e69e8ee5… [4] http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.… Cheers, Surya.

5 9

[Tacker] [Mistral] [Network Service] Regarding parameters to Network Service
by Nehal Bansal 28 Nov '19

28 Nov '19

Hi, Is there a way to pass parameters from Network Service Descriptor to the VNF Descriptor using a parameter file? Kindly advise as it would be of great help. Thank you. Regards, Nehal

1 0

[nova] There is no screen for UEFI instance from console display
by yu.chengde＠99cloud.net 28 Nov '19

28 Nov '19

Hi. After launch an UEFI mode instance in OpenStack, the console screen show the message “Guest has no initialized the displayed (yet) " Engage your help, many thanks Base on ubuntu cloud image, listed as below https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64… <https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64…>

2 1

[nova][neutron][cyborg] Bandwidth (and accel) providers are broken if CONF.host is set
by Eric Fried 28 Nov '19

28 Nov '19

Hi all. Sean mentioned a "downstream bug" in IRC today [1], which we discussed a little bit, but without gibi; and then Matt and I discussed it more later [2], but without gibi *or* Sean. Since I don't know if there's a bug report I can comment on, I wanted to summarize here for now so I don't forget. The Problem =========== Neutron needs to know the compute node resource provider on which to hang the child providers for QoS bandwidth. Today it assumes CONF.host is the name of that provider. That's wrong. The name of the provider is `hypervisor_hostname`, which for libvirt [3] happens to match the *default* value of CONF.host [4]. Per the bug Sean describes, if you override CONF.host, neutron won't find the compute node provider, and things break. The problem will be the same for any non-nova wishing to discover the compute node RP -- e.g. cyborg for purposes of creating child providers for accelerators. The Right Solution ================== Neutron (and any $service) should look up the compute node provider by its UUID. That's returned by the /os-hypervisors APIs after microversion 2.53, e.g. [5], but, Catch-22, you can currently only filter those results on hypervisor_hostname. So you would have to e.g. GET /os-hypervisors/detail and then walk the list looking for service.host matching CONF.host. That's way heavy for your CERNs. So the proposal moving forward is to add (in a new microversion) a ?service_host=XXX qparam to those APIs to let you filter down to just the one entry for your CONF.host. The UUID of that entry will also be the UUID of the compute node resource provider. (At that point you don't even need to ask Placement for that provider; you can just use that UUID directly in the APIs that create the child providers. Yay, you got your extra API call back.) Now, that's not backportable, and this problem exists in stable releases (at least those that support QoS bandwidth). So we should totally do it, but we also need... The Backportable Solution ========================= Neutron should use `gethostname()` rather than CONF.host to discover the compute node resource provider. I don't consider this a viable permanent solution because it is tightly coupled to knowing that hypervisor_hostname == `gethostname()`, which happens to be true for libvirt, but not necessarily for other drivers. We can get away with it for stable because we happen to know that we're only supporting QoS bandwidth via Placement for libvirt. Upgrade Concerns ================ Matt and I didn't nail down whether neutron and compute are allowed to be at different versions on a given host, or what those are allowed to be. But things should be sane if neutron (or any $service) logics like this in >=ussuri: if new_nova_microversion_available: do_the_os_hypervisors_thing() elif using_new_non_libvirt_feature: raise YouCantDoThisWithOldNova() else: do_the_gethostname_thing() Action Summary ============== If the above sounds reasonable, it would entail the following actions: - Neutron(/Cyborg?): backportable patch to s/CONF.host/socket.gethostname()/ - Nova: GET /os-hypervisors*?service_host=X in a new microversion. - Neutron/Cyborg: master-only patch to do the logic described in `Upgrade Concerns`_ (though for now without the `elif` branch). Thanks, efried [1] http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.… [2] http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.… [3] https://opendev.org/openstack/nova/src/commit/1cd5563f2dd2b218db2422397c8aa… [4] https://opendev.org/openstack/nova/src/commit/1cd5563f2dd2b218db2422397c8aa… [5] https://docs.openstack.org/api-ref/compute/?expanded=list-hypervisors-detai…

6 19

Re: [lists.openstack.org代发][nova][ptg] Support re-configure deleted_on_termination in server
by Brin Zhang(张百林) 28 Nov '19

28 Nov '19

Hi Sylvain, Matthew: https://review.opendev.org/#/c/693828/ Here is the PoC code, please review this change, thanks. Sylvain Bauza [mailto:sbauza@redhat.com] Items: [lists.openstack.org代发][nova][ptg] Support re-configure deleted_on_termination in server Spec is https://review.opendev.org/#/c/580336/ Most people seem to think this makes sense but realize there are already other ways to do this (snapshot) and therefore it's not totally necessary. The agreement in the room was to post the code up for the change, as this will help sell people on it if it's trivial enough and document the use case (i.e. are there scenarios where this would make life 10x easier?)

1 0

[kolla] kolla-ceph future
by Michał Nasiadka 28 Nov '19

28 Nov '19

Hello, Ceph deployment functionality in Kolla-Ansible has been deprecated in Train (https://review.opendev.org/#/c/669214/1) Given the fact that Ceph deployment was (and is) important functionality in Kolla-Ansible - I’d like to propose following steps: 1) Improve external Ceph functionality in Kolla-Ansible (configurable keyrings, users, improved documentation) - in progress now 2) Replace current kolla-ceph CI jobs with an equivalent using ceph-ansible (and possibly ceph orchestrator after Octopus is out around March 2020) 3) Remove kolla-ceph functionality in Ussuri release from Kolla-Ansible code base - or (if there are volunteers) extract it as a separate repository with a separate set of maintainers/core team. 4) (OPTIONAL) Find a volunteer to work on ceph-ansible/ceph-orchestrator existing deployment takeover functionality for ceph-kolla (current core team has no spare cycles to work on that) Essentially this mail is a call for volunteers for 3) - if there is somebody that is interested in picking up the code and maintaining it - please speak up. If there are none - we will start preparing for removing the code in 2020. Thank you for your understanding in this not easy decision. Kind regards, Michal Nasiadka

1 0

[olso][pbr][i18n] Using setup.cfg [files] data_files to install localization files
by info＠dantalion.nl 27 Nov '19

27 Nov '19

Hello everyone :), I was wondering what the preferred method to install localization files is. I can think of some probably solutions such as: 1: Including the locale files as part of a package for the target systems package manager (pacman, yum, apt, etc). 2: adding the locale files to the [files] directive in setup.cfg: I hope someone can answer my question. Kind regards, Corne Lukken

3 5

Re: [keystone][nova][barbican][neutron][cinder][tc][policy] Proposal for policy popup team⁩
by yumeng bao 27 Nov '19

27 Nov '19

>> Please nominate Yumeng Bao (yumeng_bao(a)yahoo.com) as the liaison with >> Cyborg team. She will contribute the spec. We have informed Howard of >> this. > Thanks Sundar, I have replaced Howard with Yumeng as the liaison for cyborg. > (Side note - the wiki can be edited by anyone, and I am tracking changes in it, so anyone may feel free to change their project liaison or add or remove team members and I will be notified of the change.) > Colleen Thanks Colleen for the update and the note reminder ! I will update the cyborg team progress on wiki paget later. Regards, Yumeng

1 0

[meta][K8s][API][Extended Maintenance][Operation Docs] Change SIG to Advisory status
by Rico Lin 27 Nov '19

27 Nov '19

Dear SIG members and chairs To follow the discussion in Meta SIG PTG room [1]. I would like to propose to change the following SIGs to Advisory status [2] to represent the SIG stays around for provide help, make sure everything stays working and provide advice when needed. - K8s SIG - API SIG - Extended Maintenance SIG - Operation Docs SIG If you think your SIG should not belong to `advisory` statue. Please advise from the following statuses: - active: SIG reaches out for discussion and event, have plans for the current cycle, host meetings or send ML out regularly. - forming: SIG still setting up. - advisory: SIG stays around for help, make sure everything stays working and provide advice when needed. - complete: SIG completes its mission. If that sounds correct, I need at least one chair from each SIG +1 on [2], so we can make sure it's what SIGs agreed on. [1] https://etherpad.openstack.org/p/PVG-meta-sig [2] https://review.opendev.org/#/c/695625/ -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

3 4

[meta-sig] Add Thierry Carrez to core reviewer
by Rico Lin 27 Nov '19

27 Nov '19

I would like to propose adding Thierry to core reviewer for governance-sigs I don't think anyone will doubt his knowledge of maintaining governance-sigs repo. He will be able to do core review on all patches. To create SIG patch, it will require both UC and TC to give their blessing. That means besides Amy's +2, we only need Thierry or mine +2 to approve the workflow. Let me know what you think if you have other thoughts, otherwise, I will add him in core think week. -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

2 2

[keystone][nova][barbican][neutron][cinder][tc][policy] Proposal for policy popup team
by Colleen Murphy 27 Nov '19

27 Nov '19

Hi, At the Shanghai forum, we discussed forming a popup team around reforming default policies for several projects. I've formally proposed this team here: https://review.opendev.org/695993 I've also created a wiki page to document the effort and coordinate the work: https://wiki.openstack.org/wiki/Consistent_and_Secure_Default_Policies_Popu… This is the best way I can think of to organize a cross-project initiative like this, since I think the openstack specs repo is kind of defunct? I'm open to other ideas for coordinating this. I've gone ahead and named project-specific liaisons based on discussions at the forum, and also created a member list based on the list of volunteers in the forum etherpad. If you weren't at the forum but would like to be involved, please go ahead and add your name to the list on the wiki page. To form the popup team, we need a second co-lead: please let me know if you're interested. As hinted in the wiki page, I'm not seeking to lead this long-term, so there are actually two lead positions open. Let me know your thoughts in this thread. Colleen

5 8

[oslo] Adoption of microversion-parse
by Thierry Carrez 27 Nov '19

27 Nov '19

Hi Osloites, A recent review of the OpenStack namespace yielded a library that no project team owns, and yet is specific to openstack and present in global-requirements: https://opendev.org/openstack/microversion-parse Ideally it should be adopted by one of our project teams. After discussing it with cdent, and since it is openstack-specific and depended on by most openstack projects, it feels like Oslo would be a good home for it. It would follow an "independent" release model and I don't think it would trigger much updates or maintenance. I can post patches to make it happen if you confirm you're OK to host it. Just keep me posted :) Cheers, -- Thierry Carrez (ttx)

7 13

[security] Security SIG Meeting Tomorrow Canceled
by Gage Hugo 27 Nov '19

27 Nov '19

Just a reminder that the security sig meeting tomorrow is cancelled, hope everyone has a great holiday weekend, and we will be back next week!

1 0

[ops] registration now open for OpenStack Ops Metup, Bloomberg London jan 7,8 2020
by Chris Morgan 27 Nov '19

27 Nov '19

Registration is now open for the OpenStack Ops Meetup at Bloomberg London, January 7,8 2020: https://go.bloomberg.com/attend/invite/openstack-operators-meetup-london202… Please sign up if attending. Agenda still under formation via normal community process (link on sign-up page). See you there! Chris -- Chris Morgan <mihalis68(a)gmail.com>

1 0

[neutron][L3][OVN] add a time slot for OVN L3 functionality in the L3 meeting
by LIU Yulong 27 Nov '19

27 Nov '19

Hi all, Neutron team recently merged the convergence ML2+OVS and OVN [1], it was also discussed during the PTG in Shanghai [2]. And more about that is the train is now getting started [3][4]. So during the L3 meeting, we may want to add a new section in L3 meeting to talk about some OVN L3 related topics, such as feature gaps, bugs, new ideas or anything needs to draw the attention of the neutron team. OVN is also a long-developed project with various functions. So, an independent L3 meeting time looks appropriate, we cannot cover all the OVN stuff in one meeting. So if you are interested in OVN and some L3 stuff, do not be hesitant, please come to the L3 meeting with your thoughts, ideas or questions. The Neutron L3 Sub-team Meeting [5] will be held weekly on Wednesday at 14:00 UTC in irc channel #openstack-meeting. [1] http://specs.openstack.org/openstack/neutron-specs/specs/ussuri/ml2ovs-ovn-… [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010702… [3] https://blueprints.launchpad.net/neutron/+spec/neutron-ovn-merge [4] https://review.opendev.org/#/q/topic:bp/neutron-ovn-merge [5] http://eavesdrop.openstack.org/#Neutron_L3_Sub-team_Meeting Regards, LIU Yulong

1 0

[largescale-sig] First meeting summary and next actions
by Thierry Carrez 27 Nov '19

27 Nov '19

Hi everyone, As announced in [1], the Large Scale SIG had its first IRC meeting today. You can access summary and logs at: http://eavesdrop.openstack.org/meetings/large_scale_sig/2019/large_scale_si… This meeting was mostly focused on agreeing on team logistics, and an early discussion on short-term objectives for the group. As a result of this meeting I created: - Reference information about the SIG at https://wiki.openstack.org/wiki/Large_Scale_SIG - A change formally proposing the SIG at https://review.opendev.org/696302 - Two etherpads to further refine our two initial ideas for SIG short-term goals: Scaling within one cluster, and instrumentation of the bottlenecks: https://etherpad.openstack.org/p/large-scale-sig-cluster-scaling Document large scale configuration and tips &tricks: https://etherpad.openstack.org/p/large-scale-sig-documentation Between now and next meeting, the idea is to validate those initial documents, get the SIG formally set up, and refine those goals on those two etherpads, ideally splitting them into early objectives and initial steps. Current proposal is to have our next meeting on Dec 18. Belmiro is unfortunately not available on that week, but the week before amorin and myself are not available, and the week(s) after are the end-of-year holidays for some of us, so that is still probably the best date before next year. Cheers, [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011062… -- Thierry Carrez (ttx)

1 0

[Kolla] FAILED - RETRYING: wait for MariaDB to be available via HAProxy (10 retries left).
by Bhupathi, Ramakrishna 27 Nov '19

27 Nov '19

Mark, Thanks for the tip. Now I get a failure (as below) during deploy (I have all-in-one) config. I am on stein. It appears to me that this has been presumably fixed . But I still running into this error. I do not see anything explicit in the HAProxy/MariaDB logs. TASK [mariadb : wait for MariaDB to be available via HAProxy] ************************************************************************************************** FAILED - RETRYING: wait for MariaDB to be available via HAProxy (10 retries left). FAILED - RETRYING: wait for MariaDB to be available via HAProxy (9 retries left). FAILED - RETRYING: wait for MariaDB to be available via HAProxy (8 retries left). FAILED - RETRYING: wait for MariaDB to be available via HAProxy (7 retries left). FAILED - RETRYING: wait for MariaDB to be available via HAProxy (6 retries left). FAILED - RETRYING: wait for MariaDB to be available via HAProxy (5 retries left). FAILED - RETRYING: wait for MariaDB to be available via HAProxy (4 retries left). FAILED - RETRYING: wait for MariaDB to be available via HAProxy (3 retries left). FAILED - RETRYING: wait for MariaDB to be available via HAProxy (2 retries left). FAILED - RETRYING: wait for MariaDB to be available via HAProxy (1 retries left). fatal: [localhost]: FAILED! => {"attempts": 10, "changed": false, "elapsed": 60, "msg": "Timeout when waiting for search string MariaDB in 10.1.0.250:3306"} NO MORE HOSTS LEFT ********************************************************************************************************************************************* PLAY RECAP ***************************************************************************************************************************************************** localhost : ok=76 changed=0 unreachable=0 failed=1 skipped=74 rescued=0 ignored=0 Command failed ansible-playbook -i ../../all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e kolla_action=deploy /home/cloud-user/kolla-ansible/ansible/site.yml --RamaK -----Original Message----- From: Mark Goddard [mailto:mark@stackhpc.com] Sent: Monday, November 25, 2019 11:08 AM To: Bhupathi, Ramakrishna <C-Ramakrishna.Bhupathi(a)charter.com> Cc: openstack-discuss(a)lists.openstack.org Subject: Re: [Kolla] RabbitMQ failure during deploy (Openstack with Kolla) On Mon, 25 Nov 2019 at 13:54, Bhupathi, Ramakrishna <C-Ramakrishna.Bhupathi(a)charter.com> wrote: > > Hey , I am evaluating Openstack with Kolla (have the latest ) and following the steps. I see that rabbitmq fails to start up and (keeps restarting). Essentially the script fails in deploy. I have a single node all in one config. > > > > Can someone tell me what the cause for this failure? > > > > > > RUNNING HANDLER [rabbitmq : Waiting for rabbitmq to start on first > node] > ********************************************************************** > ****************************************** > > fatal: [localhost]: FAILED! => {"changed": true, "cmd": "docker exec > rabbitmq rabbitmqctl wait /var/lib/rabbitmq/mnesia/rabbitmq.pid", > "delta": "0:00:07.732346", "end": "2019-11-25 13:33:12.102342", "msg": > "non-zero return code", "rc": 137, "start": "2019-11-25 > 13:33:04.369996", "stderr": "", "stderr_lines": [], "stdout": "Waiting > for 'rabbit@kolla-ubuntu'\npid is 6", "stdout_lines": ["Waiting for > 'rabbit@kolla-ubuntu'", "pid is 6"]} > > > > RUNNING HANDLER [rabbitmq : Restart rabbitmq container (rest of > nodes)] > ********************************************************************** > ******************************************* > > > > NO MORE HOSTS LEFT > ********************************************************************** > ********************************************************************** > ************************** > > > > PLAY RECAP > ********************************************************************** > ********************************************************************** > ********************************** > > localhost : ok=95 changed=2 unreachable=0 failed=1 skipped=78 rescued=0 ignored=0 > > > > Command failed ansible-playbook -i ./all-in-one -e > @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e > CONFIG_DIR=/etc/kolla -e kolla_action=deploy > /home/ubuntu/venv/share/kolla-ansible/ansible/site.yml > > > > > > Accessing the container logs for rabbitmq .. this is what I see > > > > BOOT FAILED > > =========== > > > > Error description: > > {could_not_start,rabbitmq_management, > > {rabbitmq_management, > > {bad_return, > > {{rabbit_mgmt_app,start,[normal,[]]}, > > {'EXIT', > > {{could_not_start_listener, > > [{port,15672}], > > {shutdown, > > {failed_to_start_child,ranch_acceptors_sup, > > > {listen_error,rabbit_web_dispatch_sup_15672,eaddrinuse}}}}, Hi RamaK, here is your issue. RabbitMQ management fails to start up due to the port already being in use (eaddrinuse). Perhaps RabbitMQ is running on the host already? > > {gen_server,call, > > [rabbit_web_dispatch_registry, > > {add,rabbit_mgmt, > > [{port,15672}], > > #Fun<rabbit_web_dispatch.0.49789095>, > > > > --RamaK > > The contents of this e-mail message and any attachments are intended > solely for the > addressee(s) and may contain confidential and/or legally privileged > information. If you are not the intended recipient of this message or > if this message has been addressed to you in error, please immediately > alert the sender by reply e-mail and then delete this message and any > attachments. If you are not the intended recipient, you are notified > that any use, dissemination, distribution, copying, or storage of this > message or any attachment is strictly prohibited. E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

3 3

Communication problem between ironic-python-agent and CI server.
by Guannan GN2 Sun 27 Nov '19

27 Nov '19

Hi team, I'm now trying to use ironic deployed with devstack to manage baremetal machine. However when it run into deploying stage, I open the BM server terminal and see it successfully load ramdisk and boot into it. It get the ip I assigned and I can ping it from CI server side. But it then deploy failed just about 2 minutes later. When I check ironic-conductor log with command "sudo journalctl -a --unit devstack@ir-cond" and found error like this: ERROR ironic.drivers.modules.agent_client [None req-de37bc21-8d62-41db-8983-c06789939818 None None] Failed to connect to the agent running on node ea88ba26-756d-4d32-89f4-7ff086fa8868 for invoking command iscsi.start_iscsi_target. Error: HTTPConnectionPool(host='10.0.0.25', port=9999): Max retries exceeded with url: /v1/commands?wait=true (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f5461027f10>, 'Connection to 10.0.0.25 timed out. (connect timeout=60)')): ConnectTimeout: HTTPConnectionPool(host='10.0.0.25', port=9999): Max retries exceeded with url: /v1/commands?wait=true (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f5461027f10>, 'Connection to 10.0.0.25 timed out. (connect timeout=60)')) I can ping it from CI server side, so it is strange why the connection time out between ironic-python-agent and CI server. Does anyone meet similar problem or have idea about it? Thank you! Best Regards, Guannan

3 3

[tc][board][all] - Adding OpenStack community support to the savedotorg campaign
by Graham Hayes 26 Nov '19

26 Nov '19

Hey All, I am not sure if this has been seen by everyone or not - but there is a change in how the .org top level domain is being ran in the works, in a way that may not be in the best interests of the non profits that it was created to facilitate. [1] A lot of well known non profits have already joined, and as a community that has an interest in the internet as a whole, and uses a .org domain, I think we should add our voice in support. What do people think? Are we happy to have the TC use its voice on behalf of the OpenStack project, or do we think the board should use its voice on behalf of the entire foundation? - Graham 1 - https://savedotorg.org/

7 8

Freezer Project Update
by Deepa 26 Nov '19

26 Nov '19

Hello Team Good Day I am Deepa from Fingent Global Solutions and we are a big fan of Openstack and we do have 4 + openstack setup (including production) We have deployed Openstack using juju and Maas .So when we check for backup feasibility other than cinder-backup we were able to see Freezer Project. But couldn't find any charms for it in juju charms. Also there isn't a clear documentation on how to install freezer . https://docs.openstack.org/releasenotes/freezer/train.html. No proper release notes in the latest version as well. Can you please tell me whether this project is in developing state? Whether charms will be added to juju in future. Can you also share a proper documentation on how to install Freezer in cluster setup. Thanks for your help. Regards, Deepa K R

3 6

manila share group replication support
by Dong.Ding＠dell.com 26 Nov '19

26 Nov '19

Hi, experts, I'm trying to implement the share replication feature for EMC Unity Manila driver. Unity isn't capable to promote a single share. The share must be promoted together with its share server. The problem is that we have several shares exported from one single server. So, promoting a share will cause all shares being promoted. My question is that is there any solution to promote several shares as they are in a group? I was trying to find something useful about 'manila group replication' ,only find Ocata Doc mentioned it, but no detail information: https://specs.openstack.org/openstack/manila-specs/specs/ocata/manila-share… And there is no code or commit history matches 'group replication' in Manila. Do you have any suggestions for our situation? Thanks, Ding Dong

2 2

Re: [nova] [cyborg] Impact of moving bind to compute
by Nadathur, Sundar 26 Nov '19

26 Nov '19

Hi, We had a thread [1] on this subject from May of this year. The preference was that "ARQ creation happens in conductor and binding happens in compute" [2]. The ARQ binding involves device preparation and FPGA programming, which may take a while. So, it is done asynchronously. It is desirable to kickstart the binding ASAP, to maximize the overlap with other tasks needed for VM creation. We wound up doing all of binding in the compute for the following reason. If we call Cyborg to initiate ARQ binding and then wait for the notification event, we may miss the event if it comes in the window in between. So we had to call wait_for_instance_event() and, within its scope, call Cyborg for binding. This logic moved everything to compute. But now we are close to having an improved wait_for_instance_event() [3]. So I propose to: A. Start the binding in the conductor. This gets maximum concurrency between binding and other tasks. B. Wait for the binding notification in the compute manager (without losing the event). In fact, we can wait inside _build_resources, which is where Neutron/Cinder resources are gathered as well. That will allow for doing the cleanup in a consistent manner as today. C. Call Cyborg to get the ARQs in the virt driver, like today. Please LMK if you have any objections. [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006541.html [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006979.html [3] https://review.opendev.org/#/c/695985/ Regards, Sundar

3 4

[scientific-sig] Meeting today, 2100UTC - Supercomputing, networking, monitoring, and more
by Stig Telfer 26 Nov '19

26 Nov '19

Hi All - We have an IRC meeting coming up at 2100 UTC (about 2.5 hours time) in channel #openstack-meeting. Everyone is welcome. Today’s agenda is here: https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_November_29th_20… <https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_November_29th_20…> It’s pretty packed today, we have a wash-up from SC to discuss, plus an update on Linuxbridge/ML2, plus an update on mixed virt and baremetal, plus a discussion on telemetry after Gnocchi. Loads to cover! Cheers, Stig

1 0

[sahara] Cancelling Sahara meeting November 28, and looking ahead to December 12
by Jeremy Freudberg 26 Nov '19

26 Nov '19

Hi all, Due to the Thanksgiving holiday in the US there will be no Sahara meeting 2019-11-28. Additionally I will be on PTO the date of the following meeting 2019-12-12. I may establish a special meeting for 2019-12-05 depending on interest. Thanks, Jeremy

1 0

Re: [ironic]: Timeout reached while waiting for callback for node
by fsbiz＠yahoo.com 26 Nov '19

26 Nov '19

Thanks Julia. In addition to what you mentioned this particular issue seems to have cropped up when we added 100 more baremetal nodes. I've also narrowed down the issue (TFTP timeouts) when 3-4 baremetal nodes are in "deploy" state and downloading the OS via iSCSI. Each iSCSI transfer takes about 6 Gbps and thus with four transfers we are over our 20Gbps capacity of the leaf-spine links. We are slowly migrating to iPXE so it should help. That being said is there a document on large scale ironic design architectures?We are looking into a DC design (primarily for baremetals) for upto 2500 nodes. thanks,Fred, On Wednesday, October 23, 2019, 03:19:41 PM PDT, Julia Kreger <juliaashleykreger(a)gmail.com> wrote: Greetings Fred! Reply in-line. On Tue, Oct 22, 2019 at 12:47 PM fsbiz(a)yahoo.com <fsbiz(a)yahoo.com> wrote: [trim] TFTP logs: shows TFTP client timed out (weird). Any pointers here? Sadly this is one of those things that comes with using TFTP. Issues like this is why the community tends to recommend using ipxe.efi to chainload as you can perform transport over TCP as opposed to UDP where in something might happen mid-transport. tftpd shows ramdisk_deployed completed. Then, it reports that the client timed out. Grub does tend to be very abrupt and not wrap up very final actions. I suspect it may just never be sending the ack back and the transfer may be completing. I'm afraid this is one of those things you really need to see on the console what is going on. My guess would be that your deploy_ramdisk lost a packet in transfer or that it was corrupted in transport. It would be interesting to know if the network card stack is performing checksum validation, but for IPv4 it is optional. [trim] This has me stumped here. This exact failure seems to be happening 3 to 4 times a week on different nodes.Any pointers appreciated. thanks,Fred.

3 3

[tc][stable] Changing stable branch policy
by Mohammed Naser 26 Nov '19

26 Nov '19

Hi everyone, At the PTG, the TC discussed what we can do about our stable branch policy and there was a few different ideas put on the table, however, something that I felt made a lot of sense was revisiting the way that we currently apply it. We all know that we're definitely a lot more resource limited as a community and it's important for us to start breaking down some of those ideas which made sense when the velocity of the project was very high. One of the things that used to make sense is maintaining a dedicated stable core team across all projects. At the current time: 1. Some projects seem to have some sort of power of their stable branches through historical reasons 2. Some projects don't have access to merging things into stable branches and need to rely on the stable maintenance team to do that 3. We are *really* thankful for our current stable team, but it seems that there is a lot of work that does bottleneck other teams (and really, stable reviews is a difficult task). The proposal that I had was that in mind would be for us to let teams self manage their own stable branches. I think we've reached a point where we can trust most of our community to be familiar with the stable branch policy (and let teams decide for themselves what they believe is best for the success of their own projects). I'd like to invite the community to comment on this change, the approach that we can take to do this (or other ideas) -- being mindful with the limited set of resources that we have inside the community. Thanks, Mohammed

12 26

[magnum] Kubernetes, multiple versions: kube-system pods in pending status on newly spun-up clusters (since last Thursday)
by Namrata Sitlani 26 Nov '19

26 Nov '19

Hello folks, As of last week (14 Nov), our Magnum (Rocky) environment has stopped spinning up working Kubernetes clusters. To be precise, Magnum does report the cluster status as CREATE_COMPLETE, but once it is up all its Kubernetes pods are stuck in the Pending state. We use the following commands to create the Kubernetes clusters: http://paste.openstack.org/show/786348/. All pods show pending status which shows cluster could not select a minion node for them. The deployment fails with the following output " 0/2 nodes are available: 2 node(s) had taints that the pod didn't tolerate" For reference: http://paste.openstack.org/show/786717/. For reference output of kubectl get all -A http://paste.openstack.org/show/786729/ If we manually remove NoSchedule taint from the minion nodes we get all the pods running For reference: http://paste.openstack.org/show/786718/ But after the manual fix too openstack-cloud-controller-manager pods are missing so any interaction from the Kubernetes control plane to OpenStack services is non-functional. We are assuming that the missing openstack cloud controller manager pod is also the reason for the taint issue which we are encountering. For the node taint issue, https://ask.openstack.org/en/question/120442/magnum-kubernetes-noschedule-t… suggests to add [trust] cluster_user_trust = true to magnum.conf. But there is OSA variable named magnum_cluster_user_trust that can be set to true for this purpose. However, the default for this variable has been True and we have confirmed we have the parameter in our environment cluster_user_trust=True as well. Note: The kubernetes cluster deployed here uses kube_tag 1.14.8 but we are getting same result with other kube_tag versions v1.13.12, v1.14.8 also. Ideally, we should not remove taints manually so can you please confirm our findings and help us find a way forward We can provide more logs if needed Thanks Namrata Sitlani

1 0

[kolla][openstack-ansible][tripleo] CentOS 8 host migration
by Mark Goddard 26 Nov '19

26 Nov '19

Hi, During the recent kolla PTG discussions [1], we covered CentOS 8 in some detail. For kolla there are several aspects to CentOS 8 support: * kolla-build execution host * container images * kolla-ansible execution host * kolla-ansible remote hosts In this context I am discussing the last of these - kolla-ansible remote hosts, i.e. the OS on the controllers and computes etc. Given that this could be quite a thorny problem, I'd like to propose that we collaborate between deployment projects. It is my understanding that upgrades from CentOS 7 to 8 will not be supported, and a reinstall is required. We set out some goals for the migration: * Migrate hosts from CentOS 7 to CentOS 8 * Upgrade from Train CentOS 7 containers to Ussuri CentOS 8 containers * Note: this means Ussuri containers (CentOS 8) don't land on CentOS 7 * Decouple the CentOS 7 to 8 and OpenStack upgrades, to avoid operator pain * Avoid excessive downtime There is a Red Hat policy [2] that suggests that if you mix host and container OS versions, it is safer for the host to be ahead of the container. This makes sense if you think about userland accessing kernel features. This leads us to the following migration path: * Start with Train release, CentOS 7 containers * Redeploy (rolling) hosts with CentOS 8 * Provision * Configure host OS * Redeploy Train containers with CentOS 7 * Upgrade to Ussuri release, CentOS 8 containers The logic we used to come up with this procedure is as follows: * Host must be upgraded to CentOS 7 before containers (going on policy) * Train containers don't support CentOS 8 base due to RDO * Ussuri containers don't support CentOS 7 base due to RDO * Want to separate CentOS 7 -> 8 migration from OpenStack upgrade This leads us to the conclusion that at least one release of kolla-ansible must bridge these worlds, since we will be deploying Train containers on CentOS 8. We could do either of: 1. Train k-a supports CentOS 8 hosts 2. Ussuri k-a supports deploying Train containers While 1. may require us to backport more than we would like, supporting deployment of multiple OpenStack releases could be challenging, so we expect to go with 1 here. There are some risks involved, including: * some Train services may not work on CentOS 8 * ceph? * OVS & neutron agents? * libvirt? * Ansible minimum version 2.8 for CentOS 8 (ref?) Hopefully we can save ourselves some effort and solve this together. [1] https://etherpad.openstack.org/p/kolla-ussuri-ptg [2] https://access.redhat.com/support/policy/rhel-container-compatibility Thanks, Mark

3 4

Re: [OpenStack Foundation] [tc][board][all] - Adding OpenStack community support to the savedotorg campaign
by Graham Hayes 26 Nov '19

26 Nov '19

On 26/11/2019 14:25, Jean-Philippe Evrard wrote: > Hello, > > I agree with others: IMO, it is the Foundation who should act on this. > I would love to see the OpenStack Foundation logo out there. > > Thank Graham for showing us this petition! > > Regards, > JP > I am not sure of the procedure, I am more than willing to put this on the board agenda for Dec 10th, if that is allowed. It should be a simple board resolution along the lines of: The Board supports the aims of the savedotorg campaign, and resolves to add its name to the list of supporting organizations. Unless someone shouts I will propose it on the agenda [1] by EOW. 1 - https://wiki.openstack.org/wiki/Governance/Foundation/10December2019BoardMe… > > > _______________________________________________ > Foundation mailing list > Foundation(a)lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/foundation >

1 0

[neutron][scientific-sig] SIG help with Linuxbridge ML2 maintenance?
by Stig Telfer 26 Nov '19

26 Nov '19

Hi all - Following this discussion [1] around the Linuxbridge ML2 driver, I’m aware that a number of members of the Scientific SIG use this driver and appreciate its performance and simplicity. Would anyone from the Neutron project involved in this issue be interested in joining a Scientific SIG meeting to discuss how SIG members can help with keeping this driver maintained? Our next meeting is Tuesday 26th November at 2100 UTC. If that’s possible, please let me know and we’ll put it on the agenda. Many thanks, Stig [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010761… <http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010761…>

3 2

[kolla][tripleo] Infra style images
by Marcin Juszkiewicz 26 Nov '19

26 Nov '19

One of things we have on a list of things to do during Ussuri cycle is implementation of 'infra' images. BP: https://blueprints.launchpad.net/kolla/+spec/infra-images # What are infra images? Images that are always built from binary packages (or are Java monsters). We have about 70 such ones from quick check. All those Ceph, Prometheus, MariaDB, cron, chrony, storm, sensu etc ones. ## libvirt image There is 'nova-libvirt' image. Contains libvirt daemon (with qemu and all required packages) so it would get renamed to 'libvirt'. # Building The idea is that 'infra' will be a new build type (like we have 'binary', 'source' etc). With all source base images marked as unbuildable so there will be no 'debian-infra-nova-compute' one. On the other hand building of 'binary'/'source' type images would lock out all 'infra' ones to not get images with names like 'debian-source-ceph-mon'. # Pros - Clean split between OpenStack components (binary/source) and infrastructure needed to get them running (infra). - Less images to publish on CI. Infra ones can be built weekly as they do not change much. - No more questions how did we built ceph-mon from source ;D # Cons - We need to change kolla-ansible, tripleo and maybe some other projects' code to use new type of images. - Migration from previous releases would be more complicated due to image renames.

2 1

openrc issue with keystone LDAP integration
by Satish Patel 25 Nov '19

25 Nov '19

I am running openstack stein and very strange issue going on, life was good until today when i finish my keystone + LDAP integration with multi-domain setup and all role assignment in SQL. when today one of user complained that his openrc isn't working correctly, look like something openrc doesn't like about LDAP integration, but same user can access everything from Horizon. my ldap domain is "eng" and here is my openrc file. # COMMON OPENSTACK ENVS export OS_ENDPOINT_TYPE=internalURL export OS_INTERFACE=internalURL export OS_USERNAME=spatel export OS_PASSWORD='MyLDAPPassword123' export OS_PROJECT_NAME=eng export OS_TENANT_NAME=eng export OS_AUTH_TYPE=password export OS_AUTH_URL=http://172.28.16.9:5000/v3 export OS_NO_CACHE=1 export OS_USER_DOMAIN_NAME=eng export OS_PROJECT_DOMAIN_NAME=eng export OS_REGION_NAME=RegionOne # For openstackclient export OS_IDENTITY_API_VERSION=3 export OS_AUTH_VERSION=3 [root@openstack ~]# source spatel.rc [root@openstack ~]# nova list ERROR (Unauthorized): The request you have made requires authentication. (HTTP 401) (Request-ID: req-5877deee-b8be-4b21-9ff6-855ae43e268e) but if i take same openrc file and add "admin" account it and "default" domain then it works so don't know why it doesn't like LDAP creds?

1 0

[keystone] Upcoming team meetings and PTGs
by Colleen Murphy 25 Nov '19

25 Nov '19

Hi team, Tomorrow, October 29, we are having our virtual pre-PTG meeting from 14:00-17:30 UTC in lieu of our regular 16:00 IRC meeting. Tuesday, November 5 some of us will be in Shanghai for the Forum and I will not be available to chair the meeting, so I propose we skip it. Tuesday, November 12, we will have our virtual post-PTG meeting from 15:00-17:30 UTC in lieu of our regular 16:00 IRC meeting. Details about the PTG sessions can be found on the planning etherpad: https://etherpad.openstack.org/p/keystone-shanghai-ptg Colleen

1 2

[ops] Running VMware atop of OpenStack (eg: esxi)
by Paul Belanger 25 Nov '19

25 Nov '19

Greetings, I wanted to ask if anybody in the community is running VMware a top of OpenStack in any capacity? In Ansible we are doing a POC as part of our testing platform and running into some common ops issue. For example, what do people usually do for things like using config-drive? Is there any specific tooling you us to customize esxi images to run atop OpenStack. Bascially, looking for humans to bounce questions off and see who else is doing it. Thanks! Paul

3 5

[keystone] PTG recap
by Colleen Murphy 25 Nov '19

25 Nov '19

The keystone team didn't formally meet at the IRL PTG in Shanghai, but we had virtual meetings before and after the event to discuss the upcoming cycle. Below is a summary of what we discussed, more detailed notes were kept in the main etherpad[1] and sub-pads linked from there. [1] https://etherpad.openstack.org/p/keystone-shanghai-ptg Cycle Retrospective ------------------- https://etherpad.openstack.org/p/keystone-train-retrospective We started the pre-PTG with our usual end-of-cycle retrospective. One outcome of this was reevaluating the effectiveness of structured weekly office hours: we weren't having them frequently because often no one proposed a topic for the meeting, so we decided to replace the weekly-but-only-sometimes face-to-face meetings with once per month bug squash meetings. We're also going to experiment with having a weekly rotation of team members who will act as the point person for helping triage incoming bugs and responding to users with support questions, in an effort to create some structure to spread the load of ensuring users have a positive encounter with the team. Although we tried to have milestone-ly checkpoints on feature work, that did not seem to be enough to keep up momentum throughout the cycle on important features, so we'll try to have more frequent checkpoints, more precise deadlines, and better decomposition of tasks. Policy ------ https://etherpad.openstack.org/p/PVG-keystone-forum-policy The purpose of the policy discussion was mainly to prepare for the forum session happening the following week, so most of the result is captured by the forum recap[2]. This was the discussion during which the idea of forming a pop-up team to work through an initial set of projects, rather than creating a community goal and attempting to get all projects done in one cycle, was proposed. We had also discussed creating some kind of "policy-ready" tool to help operators figure out which projects had completed their migration, but this seemed only marginally more useful than simply keeping a manually updated list. (We discussed other ideas for operator tooling at the forum.) [2] http://www.gazlene.net/shanghai-forum-ptg.html Limits ------ https://etherpad.openstack.org/p/keystone-ussuri-ptg-unified-limits We didn't have any design discussion on Unified Limits, since nothing much has changed since it was discussed at the last PTG. The only issue is that no one has been working on it in the last cycle, so the next steps were simply to followup with stakeholders and PoC authors, reassess the priority of the work, and reinstigate the work (this is now in progress). SCIM Extension -------------- We discussed a proposal to add the SCIM protocol[3 as an API extension. The desire is to be able to sync user state between the keystone SQL database (including shadow users) and an external identity provider. The main rationale is to be able to sync the state of the users in an external IdP with the state of the cloud resources they own, so that, for example, decommissioned users would not have orphaned cloud resources. However, resources are actually owned by keystone projects, not users, and there is a many-to-many relationship between users and projects, so there is no direct link between the state of a user in the keystone database and the state of a resource they touched. Moreover, although keystone has a v3-ext section in its API reference[4], we don't effectively support API extensions because we don't have a discovery mechanism for them, so for the sake of interoperability we discourage adding config-enabled API extensions. We suggested that this feature could instead potentially be implemented as a proxy service in front of keystone, and if such an implementation turned out to be infeasible, we could revisit adding this to keystone at a later time. [3] https://tools.ietf.org/html/rfc7644 [4] https://docs.openstack.org/api-ref/identity/v3-ext/ Cycle Planning -------------- https://tree.taiga.io/project/keystone-ussuri-roadmap/kanban This cycle will be lighter in keystone-specific feature work, as much of our focus will be on cross-project goals. The only new spec to be proposed for Ussuri will be the Alembic migration[5] which has been in the backlog for a while. We'll also continue work on unfinished specs from Train[6][7]. We're also accounting for time needed to work on the community goals[8], as well as improving our documentation and improving the CLI. [5] http://specs.openstack.org/openstack/keystone-specs/specs/keystone/backlog/… [6] http://specs.openstack.org/openstack/keystone-specs/specs/keystone/backlog/… [7] http://specs.openstack.org/openstack/keystone-specs/specs/keystone/backlog/… [8] https://governance.openstack.org/tc/goals/selected/ussuri/index.html

1 0

[requirements][taskflow] networkx bump is blocked on what looks to be a taskflow issue.
by Matthew Thode 25 Nov '19

25 Nov '19

I've made a test review so people can see and test against, but it looks like the bump from 2.3 to 2.4 is causing some issues. hits nova,neutron,octavia,glance,cinder -- Matthew Thode

4 3

[oslo] Virtual PTG Planning
by Ben Nemec 25 Nov '19

25 Nov '19

Hi Osloers, Given that a lot of the team was not in Shanghai and we had a few topics proposed that didn't make sense to discuss as a result, I would like to try doing a virtual PTG the way a number of the other teams are. I've added a section to the PTG etherpad[0] with some proposed details, but in general I'm thinking we meet on Jitsi (it's open source) around the time of the Oslo meeting. It's possible we might be able to get through everything in the regularly scheduled hour, but if possible I'd like to keep the following hour (1600-1700 UTC) open as well. If everyone's available we could do it next week (the 18th) or possibly the following week (the 25th), although that runs into Thanksgiving week in the US so people might be out. I've created a Doodle poll[1] with selections for the next three weeks so please respond there if you can make it any of those days. If none of them work well we can discuss alternative options. Thanks. -Ben 0: https://etherpad.openstack.org/p/oslo-shanghai-topics 1: https://doodle.com/poll/8bqiv865ucyt8499

3 5

[keystone] Keystone Team Update - Week of 18 November 2019
by Colleen Murphy 25 Nov '19

25 Nov '19

# Keystone Team Update - Week of 18 November 2019 ## Meta I haven't done one of these updates in a while because of travel and $life. This week there were a few things worth sending a newsletter about, so here you go. However, as many of you are aware, my job focus is changing and I need to be more strategic and selective about the activities I put my time into. These weekly updates consume a not-insignificant amount of my time, and so from now on I'll not plan on continuing it. Many people have given me the feedback that they find this newsletter useful, and so I encourage anyone who has some pulse on the keystone team's activities to take up this weekly summary (need not be a core). The template is linked at the bottom of this email, and I've been using Lance's launchpad scripts[1] to generate bug statistics and some oneliners[2] to scrape Gerrit data. [1] https://github.com/lbragstad/launchpad-toolkit [2] https://gist.github.com/cmurphy/ee802fc0dc4bf57dffbea02265cc9e92 ## News ### PTG Recap We concluded our second virtual PTG session this week. Notes were kept in the etherpad[3] and I will send out a summary some time next week. [3] https://etherpad.openstack.org/p/keystone-shanghai-ptg ### Documentation for Upgrade Issues As operators start upgrading to Train[4] it's becoming clear that some of the changes we made last cycle aren't well documented and we need better tooling and documentation to help deal with them, especially immutable roles and deprecated policies. [4] http://eavesdrop.openstack.org/irclogs/%23openstack-keystone/%23openstack-k… ### User Support and Bug Duty To help share the responsibility of ensuring users and operators have a positive experience when coming to the keystone team for help, we started a weekly rotation of support duty, organized by etherpad[5]. Being on-duty means being responsible for triaging new bugs, and responding to support requests on the mailing list and on IRC. It does not mean having to solve people's problems right away, only ensuring that they know someone is listening and cares about their issues. If you would like to help, feel free to add your name to the list - you need not be a keystone developer, just willing to help triage user issues, confirm bug reports, ask for more information, and help give pointers to resources that may help. We'll proceed roughly down the bulletted list but will take time at each IRC meeting to confirm the next person on-duty. [5] https://etherpad.openstack.org/p/keystone-l1-duty ### Unified Limits Update Work on the Unified Limits initiative[6] had stalled out last cycle but work on the oslo.limit implementation, which was the main thing blocking further progress in Nova or other projects, has been reignited this week[7]. Thanks John for picking this back up! [6] http://specs.openstack.org/openstack/keystone-specs/specs/keystone/ongoing/… [7] https://review.opendev.org/#/q/project:openstack/oslo.limit+is:open ### No Meeting Next Week Next week is the Thanksgiving holiday in the US so we'll skip the team meeting (Tuesday November 26). ## Open Specs Ussuri specs: https://bit.ly/2XDdpkU Ongoing specs: https://bit.ly/2OyDLTh ## Recently Merged Changes Search query: https://bit.ly/2pquOwT We merged 3 changes this week. ## Changes that need Attention Search query: https://bit.ly/2tymTje There are 32 changes that are passing CI, not in merge conflict, have no negative reviews and aren't proposed by bots. ## Bugs This week we opened 2 new bugs and closed 1. Bugs opened (2) Bug #1853170 (oslo.policy:High) opened by Ben Nemec https://bugs.launchpad.net/oslo.policy/+bug/1853170 Bug #1853038 (oslo.policy:Wishlist) opened by Ben Nemec https://bugs.launchpad.net/oslo.policy/+bug/1853038 Bugs closed (1) Bug #1852547 (keystone:Undecided) https://bugs.launchpad.net/keystone/+bug/1852547 ## Milestone Outlook https://releases.openstack.org/ussuri/schedule.html Specs for Ussuri must be proposed and ready to review by Milestone 1 which is in about three weeks. ## Shout-outs Thanks for Adam Harwell and Zachary Buhman for joining the keystone PTG and participating in the design discussion, John Garbutt for picking up the unified limits work, and everyone who signed up to help with bug duty! ## Help with this newsletter Help contribute to this newsletter by editing the etherpad: https://etherpad.openstack.org/p/keystone-team-newsletter

2 1

[Kolla] RabbitMQ failure during deploy (Openstack with Kolla)
by Bhupathi, Ramakrishna 25 Nov '19

25 Nov '19

Hey , I am evaluating Openstack with Kolla (have the latest ) and following the steps. I see that rabbitmq fails to start up and (keeps restarting). Essentially the script fails in deploy. I have a single node all in one config. Can someone tell me what the cause for this failure? RUNNING HANDLER [rabbitmq : Waiting for rabbitmq to start on first node] **************************************************************************************************************** fatal: [localhost]: FAILED! => {"changed": true, "cmd": "docker exec rabbitmq rabbitmqctl wait /var/lib/rabbitmq/mnesia/rabbitmq.pid", "delta": "0:00:07.732346", "end": "2019-11-25 13:33:12.102342", "msg": "non-zero return code", "rc": 137, "start": "2019-11-25 13:33:04.369996", "stderr": "", "stderr_lines": [], "stdout": "Waiting for 'rabbit@kolla-ubuntu'\npid is 6", "stdout_lines": ["Waiting for 'rabbit@kolla-ubuntu'", "pid is 6"]} RUNNING HANDLER [rabbitmq : Restart rabbitmq container (rest of nodes)] ***************************************************************************************************************** NO MORE HOSTS LEFT ********************************************************************************************************************************************************************** PLAY RECAP ****************************************************************************************************************************************************************************** localhost : ok=95 changed=2 unreachable=0 failed=1 skipped=78 rescued=0 ignored=0 Command failed ansible-playbook -i ./all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e kolla_action=deploy /home/ubuntu/venv/share/kolla-ansible/ansible/site.yml Accessing the container logs for rabbitmq .. this is what I see BOOT FAILED =========== Error description: {could_not_start,rabbitmq_management, {rabbitmq_management, {bad_return, {{rabbit_mgmt_app,start,[normal,[]]}, {'EXIT', {{could_not_start_listener, [{port,15672}], {shutdown, {failed_to_start_child,ranch_acceptors_sup, {listen_error,rabbit_web_dispatch_sup_15672,eaddrinuse}}}}, {gen_server,call, [rabbit_web_dispatch_registry, {add,rabbit_mgmt, [{port,15672}], #Fun<rabbit_web_dispatch.0.49789095>, --RamaK E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

2 1

[kolla] Forum project update slides
by Mark Goddard 25 Nov '19

25 Nov '19

Hi, During the PTG there were some requests for the slides from the Kolla project update forum session given by Surya and Jeffrey. https://docs.google.com/presentation/d/19Lj9WcSjjcs8f5ljL1r4HMIQFl4Nokepsij… Please get in touch if you can't reach the above and would like to see the slides. Thanks, Mark

1 0

[nova] splitting "ERROR" into 2 states
by Mohammed Naser 25 Nov '19

25 Nov '19

Hi everyone, This is just a very open ended discussion that might bring up some interesting ideas, either that I'm going the wrong way about this, or perhaps this is something we need to think more about. As we go about increasing the instrumentation of the clouds we run, one of the interesting ideas was to measure the "ERROR" instance rate to see if more VMs than usual are hitting ERROR state. The problem with this right now is that there are a few factors at the moment where an instance can hit an ERROR state which are *not* cause of concern for the operator. For example, if you're booting an instance from a volume and you're at your quota, the instance will fail to boot, end up in "ERROR" state. If we're instrumenting that, we are likely going to be alerted on a high # of ERROR state instances but there's not much that we can do about it realistically. However, if we're getting a lot of ERROR instances because of "NoValidHost" or because some other valid failures such as in libvirt or RBD, then we'd probably want to be alerted on those. Does anyone have any ideas on how we can either better instrument this, or perhaps seeing how inside Nova, we have a "system error" and a "user error" Thanks :) Mohammed

2 1

[neutron] Bug deputy report week of November 18th
by Miguel Lavalle 25 Nov '19

25 Nov '19

Hi, I was the bugs deputy for the week of November 18th. Here's the list of reported bugs: High: - https://bugs.launchpad.net/neutron/+bug/1853223 [VPNaaS]: Python3 RuntimeError: dictionary changed size during iteration Fix in progress: https://review.opendev.org/#/c/695130 - https://bugs.launchpad.net/neutron/+bug/1853603 SG rules not enforced This happens with Kuryr tests. Per https://bugzilla.redhat.com/show_bug.cgi?id=1688323, Nate Johnson is reported working on it in the Red Hat bugzilla Medium: - https://bugs.launchpad.net/neutron/+bug/1853171 Deprecate and remove any "ofctl" code in Neutron and related projects ralonsoh reported it and working on it - https://bugs.launchpad.net/neutron/+bug/1853637 Assign floating IP to port owned by another tenant is not override-able with RBAC policy Reported by Octavia team. Needs owner Low: - https://bugs.launchpad.net/neutron/+bug/1853582 ovs_all_ports option used by ovs cleanup has a misleading help string Requires further investigation: - https://bugs.launchpad.net/neutron/+bug/1853613 VMs don't get ip from dhcp after compute restart Submitter was asked for more information Incomplete: - https://bugs.launchpad.net/neutron/+bug/1853632 designate dns driver does not use domain settings for auth More information requested from submitter. Potential fix proposed: https://review.opendev.org/#/c/695726 Invalid: - https://bugs.launchpad.net/neutron/+bug/1853089 openstack env in Stein, CLI or Dashboard response slowly Report is too vague. More data requested

2 1

[Kolla] Configure docker service fails during bootstrap_servers
by Bhupathi, Ramakrishna 25 Nov '19

25 Nov '19

Folks, Looking for help on this. When I am attempting to install OpenStack Kolla (development mode) I am running into this error when running bootstrap_servers ./kolla-ansible -i ../../multinode bootstrap-servers TASK [baremetal : Configure docker service] **************************************************************************************************** fatal: [localhost]: FAILED! => {"changed": false, "msg": "AnsibleUndefinedVariable: 'ansible.vars.hostvars.HostVarsVars object' has no attribute u'ansible_eth0'"} PLAY RECAP ************************************************************************************************************************************* localhost : ok=29 changed=3 unreachable=0 failed=1 skipped=16 rescued=0 ignored=0 Command failed ansible-playbook -i all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e kolla_action= bootstrap-servers /home/ubuntu/kolla-env/share/kolla-ansible/ansible/kolla-host.yml Can someone tell me what is going wrong here? --RamaK E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

3 2

[kolla] PTG summary
by Mark Goddard 25 Nov '19

25 Nov '19

Hi, A couple of weeks ago we had our partially virtual PTG. The full notes are in Etherpad [1], but I will try to summarise some of the discussions here. We had a good turnout, and good engagement in the discussions. # Priorities I'll start at the end, as this is arguably the most tangible outcome of the discussions. At the end of the sessions, we gathered all potential work items for Ussuri, and voted on them in another Etherpad [2]. Afterwards these were cleaned up, ordered, and assigned blueprints. The priorities were transferred to the whiteboard [3] for tracking. Of course other things will come along, and some things won't get done, but these priorities should be used as a guide for anyone reviewing or developing patches during the Ussuri cycle # General topics Sustainability is a key theme in kolla recently. There are two main aspects to this - keeping the community healthy, and ensuring the scope of the project is maintainable. We discussed having some additional community meetings on a regular basis, as a way to keep in touch with operators and to build up more of a community outside of IRC. I proposed having a virtual onboarding session which a number of people expressed an interest in attending. Pulling this session out of the summit should allow for a more level playing field. ## Ceph Ansible We are continuing to investigate Ceph Ansible as an alternative to our native Ceph deployment. The work to migrate from an existing kolla deployment is still ongoing. There are some potential blockers in the form of no Ubuntu container image support in ceph-ansible, and no ARM container images published by the ceph-container project. ## Kolla-cli This project has been restored in the Train cycle, and we continue to keep an eye on its progress. ## Extensibility We discussed making kolla and kolla-ansible more extensible as a way to avoid the need to support every service under the sun. This should be simple in kolla, but kolla-ansible will require more effort to define how custom playbooks or roles are hooked in. ## Inverting kolla images There was a proposal to use more upstream docker images, and potentially add our tooling on top where necessary. This is an interesting idea, but could add complexity with more image distros to track. ## Cloud native logging The cloud-native folk seem to have agreed on capturing container logs from stdout, which doesn't align with our file-based model. We also miss some logs during startup that don't get logged to files which could be interesting. We agreed to try ingesting these into fluentd as a starting point. ## Tracking bug fixes in release notes We agreed to start experimenting with tracking bug fixes in release notes from the Ussuri release. Previously we have just tracked features, deprecations and upgrade notes. This will raise the bar for contribution slightly, so we will keep an eye on it. # Kolla ## CentOS 8 Supporting CentOS 8 base container images is key for us this cycle, as it allows us to move to python 3 based CentOS images. These images may start to break at any moment as services drop python 2 support. The Ussuri release will not support CentOS 7 base container images. We are currently blocked by a number of missing yum repositories for CentOS 8. Supporting running kolla-build on a CentOS 8 host is a simpler task, although getting Docker installed requires a few contortions at this point. ## Drop python 2 This depends on CentOS 8 host and container support. We are heavily at risk of being broken as other projects drop python 2 support. We plan to keep CentOS CI happy for as long as possible but expect it may break at some point during the cycle. ## Zuul proposal bot We talked about adding a zuul proposal bot to update source package versions on stable branches (running tools/version-check.py). I looked into it and put forward a PoC, but we are going to try switching to a YAML format definition before proceeding with this. ## Remove EPEL Everyone likes to hate EPEL, so we will try to remove it. We also discussed an off by default approach for our custom package repositories to provide some damage limitation if they go AWOL. ## Support matrix We made a good start with the support matrix [4] this cycle. We'd like to continue this effort, and continue to evaluate which images we support. The next step seems to be to better define our categories of images, and use these to define voting vs. non-voting build failures. We'd like to find community owners for some of our 'community maintained' images. ## RabbitMQ upgrade RabbitMQ 3.8 brings a prometheus exporter, which a number of people have expressed an interest in. This will require an erlang upgrade. ## Prometheus 2 There is no migration path between prometheus 1 and 2. We discussed a few options for a smooth transition, and I think we landed on this: * keep old prometheus container around, configure it as a remote read source * configure haproxy to flip to the new prometheus when ready # Kolla Ansible ## CentOS 8 This is where it gets interesting. How do we migrate a running system from CentOS 7 to 8? Ideally we would not couple this to an OpenStack upgrade, so at least one release needs to support both CentOS7 and 8 hosts. I will follow up on this topic separately as it's a big one, and I'd like to try a cross-project approach. ## Drop python 2 The main interesting decision here is: don't drop py2 for remote hosts in Ussuri, until we are sure that Ussuri will only need to run against CentOS 8 hosts (see above). ## OVN support Neutron seems to be making moves to deprecate OVS and LinuxBridge ML2 drivers, replacing them with OVN in tree. We have OVN images, but no deployment support in kolla ansible. We'd like to add it this cycle. Interesting questions around migration from OVS to OVN came up. Tripleo has some tooling which might help here. ## More host-level commands (day 2 ops) We have the bootstrap-servers command for bootstrapping hosts, but lack some commands for ongoing operations. Common examples include: * reconfiguring or upgrading docker in a safe manner (without live-restore, a docker restart takes down your containers). * adding new hosts. This requires updating /etc/hosts everywhere, but running bootstrap-servers again is heavy handed and risks a docker restart. Containers don't automatically pick up changes to /etc/hosts, so we need to address that. * pruning docker images ## Restarting services There was a request for a command to restart services. It could probably be cobbled together from existing code quite easily. ## More destruction It should be possible to run the destroy command against a subset of services. We could also do more to thoroughly clean up. ## More security friendliness (especially transport security) * Could we integrate with letsencrypt? Possibly. * Should we default to use TLS with self-signed certs? Probably, but expiry could cause some surprises without explicit buy-in from the operator. * Can we use per-host RabbitMQ usernames and passwords? Potentially... ## SELinux Comes up often, but never gets voted for in our priorities. It would be nice to get this one sorted though. ## Fluentd reconfiguration Currently it's not possible to deploy the common services (cron, kolla-toolbox, fluentd) without also deploying another service. There are a few fiddly details, but it should be possible to resolve. ## Ansible lint We agreed to try running ansible-lint on our codebase. The group has had mixed results with it before, but was open to trying again. ## Ansible maximum version pinning We agreed to define a maximum version of Ansible that we support. This will help to prevent breakage out of our control. ## Nova cells v2 Work continues this cycle on cells with support for shared cell controllers, and deployment of multiple RabbitMQ and MariaDB clusters. ## Config file audit We should use the oslo config validator [5] to ensure our config is valid. ## Podman This one keeps coming up, but we never agree to implement it. Possible issues include a lack of a full-featured Python library, and lack of a supported package for Debian/Ubuntu. We agreed to start thinking about how we might perform a migration from Docker one day, given the direction of Red Hat. [1] https://etherpad.openstack.org/p/kolla-ussuri-ptg [2] https://etherpad.openstack.org/p/kolla-ussuri-priorities [3] https://etherpad.openstack.org/p/KollaWhiteBoard [4] https://docs.openstack.org/kolla/latest/support_matrix.html [5] https://docs.openstack.org/oslo.config/latest/cli/validator.html Well that turned into more of an exhaustive list than I'd expected. Well done for reading (or scrolling) to the end. Cheers, Mark

3 5

[nova] unshelve image_ref bugs
by Alexandre Arents 25 Nov '19

25 Nov '19

Hey all, We are using more and more shelve feature and we recently hit this referenced shelve/unshelve bug: https://bugs.launchpad.net/nova/+bug/1732428 http://lists.openstack.org/pipermail/openstack-dev/2017-December/125124.html To summup the issue: -Once we shelve/unshelve a qcow2 instance, we cannot anymore live-migrate/cold-migrate/resize without breaking it, involving data loss.. ->This is because the unshelved instance have a backing file corresponding to the deleted snapshot(shelve-snapshot), not the original image_ref in glance. So when we migrate the instance, destination host fetch original image from glance, not the one currently used by instance. I've tried to summup possible solution: A) Use patch proposed by Matt (abandonned): https://review.opendev.org/#/c/524726/ This change assume the existence of the snapshot image created by a shelve, that is to say don't delete image during unshelve and update instane.image_ref with it. PROS: -there is no more "hidden image" CONS: -each shelved/unshelved create images in glance(it uses space, can be a problem or not) -It breaks the assumption: "I want my instance remains unchanged after unshelve and keep original image_ref, so when I rebuild my instance, I use the original image (at spawn time), not the one used during last shelve" Note: Matt mention a possible workaround of the rebuild issue by using image_id in in request.spec instead of image_ref[1]) B) Someone propose to rebase backing file [2]: I did not check feasibility/complexity of this ? CONS: -What we do when original image is deleted? C) Change create_image()/imagebackend driver behavior, to create a flatten qcow2 file in case of unshelving. flattening disk may be a solution because there will be no more "orphan backing file". (Basicly doing like "flat" backend driver except we need to stay in qcow2 instead of RAW) PROS: -we keep orignal unshelve behavior/assumption CONS: -It means that in your infra configured in COW some instances will be in "qcow2 flat", Flat qcow2 instance works great (livemigration/resize..). Would all installation ok with that ? Ok it seems a little odd to ask COW driver to not do COW in some case. Alternatilevy we can force using flat driver if unshelving, but we need to change flat driver to support also qcow2. D) During spawn() if unshelving we convert "qcow2 disk with backing file" to a "flatten qcow2 disk", just after self._create_image(). It looks more like a workaround than a long term solution as it need to convert something created before, that do not meet the need(better to do C). E) Any other idea ? Currently to make short term OPS and User happy, we are about to do (D) as it works great in our environment, but we are looking for the project solution. Any recommendation? [1] http://lists.openstack.org/pipermail/openstack-dev/2018-September/134855.ht… [2] https://bugs.launchpad.net/nova/+bug/1732428/comments/6 -- Alexandre Arents aarents

2 2

[cinder] weekly meeting time and location change
by Brian Rosmaita 24 Nov '19

24 Nov '19

Beginning with the 4 December 2019 meeting, the Cinder weekly meeting will be held as follows: Day: Wednesday Time: 1400 UTC Location: #openstack-meeting-4 Agenda: https://wiki.openstack.org/wiki/CinderMeetings Updated ICS file: http://eavesdrop.openstack.org/calendars/cinder-team-meeting.ics Note that the *time* and *IRC chat room* have changed. The day, agenda, meeting log locations, etc., remain the same. Thanks to Liang Fang for initiating this change, which will bring our meeting into a more reasonable time frame for contributors in Asia time zones. cheers, brian

2 1

[qa][form][ptg] QA Summary for Forum & PTG
by Ghanshyam Mann 24 Nov '19

24 Nov '19

Hello Everyone, I am summarizing the QA PTG and Forum sessions held in Shanghai. * OpenStack QA – Project Update: We gave the updates on what we finished on Train and draft plan for the Ussuri cycle. due to fewer contributors in QA, Train cycle activities are decreased as compare to Stein. We tried to maintain the daily QA activity and finished a few important things. Slides are uploaded on the schedule site. * Users / Operators adoption of QA tools / plugins : Etherpad: https://etherpad.openstack.org/p/PVG-forum-qa-ops-user-feedback. This is another useful session for QA to get feedback as well as information about downstream tooling. Few tools we talked about: - Fault injection tests - https://opendev.org/x/tobiko/ - whitebox testing(Whitebox Tempest Plugin) - high availability testing: https://review.opendev.org/#/c/443504/ Run patrole jobs in the neutron upstream(Keystone was interested before) One big concern shared from a few people about a long time to get merged tempest patches. One idea to solve this is to bring critical reviews in Office hours. QA PTG: It was a small gathering this time for one day for PTG on Wednesday. Even with the small number of developers, we had good discussions on many topics. Etherpad: https://etherpad.openstack.org/p/shanghai-ptg-qa * Train Retrospective: Retrospective bought up the few key issues where we need improvement. We collected the below action items including bug triage. Untriage QA bugs are increasing day by day. Action: - need to discuss blacklist plugins and how to notify and remove them if dead - gmann - start the process of community-goal work in QA - masayuki - sprint for bug triage with number of volunteers - - (chandankumar)Include one bug in each sprint in TripleO CI tempest member - Traige the new bug and then pick the bug based on priority - For tripleo Ci team we will track here: https://tree.taiga.io/project/tripleo-ci-board/ - chandankumar * How to deal with an aging testing stack: With testtools being not so active, we need to think on the alternate or best suitable options to solve this issue. We discussed the few options which need to be discussed further on ML. - Can we fork the dependecies of testtools in Temepst or stestr ? - As we are removing the py2.7 support in tempest, we can completly ignore/remove the unittest2 things but that is not case for testtools ? - Similar example done in different project: https://github.com/plone/Products.CMFPlone/issues/1882 - Remove the support of unittest2 from testtools ? py2.7 is going away from everywhere and testools can create tag or something for py2.7 usage ? - Since Python2 is going EOL on 01st Jan, 2020, so let's create a tag and remove the unitest2 with unitest for python3 release only Action: - Document the official supported test runner by Tempest. - Soniya Vyas/Chandan Kumar - ML to discuss the above options - gmann * Remove/migrate the .testr.conf to .stestr: 60 openstack/* repositories have .stestr.conf AND .testr.conf. We don't need to have both files at least. Let's take a look some of them and make a plan to remove if we can. - http://paste.openstack.org/show/785832/ - If both exist then remove the .testr.conf and Then verify that .stestr conf has the correct test path. If only .testr.conf then migrate to stestr.conf - We need to figure out the purpose of pbr .testr.conf code before removing. Is this just old codes or necessary? - https://opendev.org/openstack/pbr/src/branch/master/pbr/hooks/commands.py#L… - https://opendev.org/openstack/pbr/src/branch/master/pbr/testr_command.py * Moving subunit2html script from os-testr: Since os-testr runner piece in os-testr project is deprecated but subunit2html project still exists there, it is widely used across the OpenStack ecosystem, Can we move to somewhere else? I do not find any benefits to move those scripts to other places. We asked chandan to open an issue on stestr to discuss moving to stestr repo. mtreinish replied on this: os-testr meant to be the place in openstack that we could host the ostestr runner wrapper/script subunit2html, generate_subunit, etc. Just because ostestr is deprecated and being removed doesn't mean it's not the proper home for those other tools. * Separate integrated services tests can be used in TriplO CI: TriplO CI maintains a separate file to run dependent tests per service. Tempest has dependent services tox and integrated jobs and the same can be used in TriplO CI. For example: - blacklist file for networking testing [1]. - tox for networking[2]. * RBAC testing strategy: This was a cross-project strategy for positive/negative testing for system scope and new defaults in keystone. Keystone has implemented the new defaults and system scope in its policy and added a unit test to cover the new policies. Nova is implementing the same in Ussuri cycle. As discussed in Denver PTG also, Tempest will implement the new credential for all 9 personas available in keystone. Slowly migrate the tests start using the new policies. That will be done via a flag switching Tempest to use system scope or new defaults and that flag will be false to keep using the old policies for stable branch testing. We can use patrole tests or implement new tests in the Tempest plugin and verify the response. Both have the issue of performing the complete operation which is not required always for policy verification. Running full functional tests is expensive and duplicates existing tests. One solution for that (we talked about it in Denver PTG also) is via some flag like os-profiler by just do the policy check and return the API response with specific return code. AGREE: - Tempest to provide the all 9 personas available from keystone. Slowly migrate Tempest existing tests to run with new policies. - We agreed to have two ways to test the policy: - Tempest like tests in tempest plugins with the complete operation and verify the things on response, not just policy return code. It depends on the project if they want to implement such tests. - Unit/Functional tests on the project's side. - Document the both way so that project can adopt the best suitable one. * How to remove tempest plugin sanity BLACKLIST: We have Tempest plugin blacklist. It should be removed in the future if possible. Some of them shouldn't be as a tempest-plugin because they're just neutron studium things which already moved to neutron-tempest-plugin but still exiting in repo also. Some of them are less active. Remove below plugins from BLACKLIST: - x/group-based-policy - https://opendev.org/x/group-based-policy - openstack/networking-generic-switch needs to be checked (setup.py/cfg?) Action: - Add the start date in blacklist doc so that we can know how long a plugin is blacklisted. - After 60 days: we send the email notification to openstack-discuss, PTL, maitainer and TC to either fix it or remove it from the governance. * Python 2.7 drop plan: We discussed the next steps to drop the py2 from Tempest and other QA tools. AGREE: - Will be doing before milestone 2 - Create a new tag for python-2.7 saying it is the last tag and document that the Tempest tag needs Train u-c. - Test the Tempest tag with Train u-c, if fail then we will disucss. - TripleO and OSA is going to use CentOS 8 for train and master * Adding New glance tests to Tempest: We discussed on testing the new glance v2 api and feature. Below are the glance features and agreed points on how to test them. - Hide old images: Test can be added in Tempest. Hide the image and try to boot the server from the image in scenario tests. - Delete barbican secrets from glance images: This test belongs to barbican-tempest-plugin which can be run as part of the barbican gate using an existing job. Running barbican job on glance gate is not required, we can add a new job (multi stores) on glance gate which can run this + other new features tests. - Multiple stores: DevStack patch is already up, add a new zuul job to set up multiple stores and run on the glance gate with api and scenario tests. gmann to setup the zuulv3 job for that. * Tempest volunteers for reviewing patches: We've noticed that the amount of merged patches in October is less than in September and much less than it was during the summer. This has been brought in feedback sessions also. There is no perfect solution for this. Nowadays QA has less active core developers. We encourage people to bring up the critical or stuck patches in office hours. * Improving Tempest cleanup: Tempest cleanup is not so stable and not a perfect design. We have spec up to redesign that but could not get a consensus on that. I am ok to move with resource prefix with UUID. We should extend the cleanup tool for plugins also. * Ussuri Priority & Planning: This was the last session for the PTG which could not happen on Wed due to strict time-up policy of the conference place which I really liked. Time-based working is much needed for IT people :). We met on Thursday morning in coffee area and discussed about priority for Ussuri cycle. QA Ussuri Priority Etherpad[3] has the priority items with the assignee. [1] https://github.com/openstack/tempest/blob/9cdd5250615bb6ab26a1a9a80743a03cc… [2] https://github.com/openstack/tempest/blob/9cdd5250615bb6ab26a1a9a80743a03cc… [3] https://etherpad.openstack.org/p/qa-ussuri-priority -gmann

1 0

[cinder] Virtual PTG reminder--Monday 25 Nov at 1500 UTC
by Brian Rosmaita 24 Nov '19

24 Nov '19

This is a quick reminder that we're having the first session of the Cinder Virtual PTG for Ussuri from 1500-1700 UTC on Monday 25 November. We'll meet in this BlueJeans room: https://bluejeans.com/3228528973 The session will be recorded. The rough schedule is here: https://etherpad.openstack.org/p/cinder-ussuri-virtual-ptg-planning It's not too late to add a topic if there's something you'd like to have discussed. The second session is Wednesday 27 November from 1500-1700 UTC. See you there! brian

1 0

[cinder] Ussuri Shanghai PTG summary
by Brian Rosmaita 24 Nov '19

24 Nov '19

Sorry for the delay. A summary of what happened at the Cinder part of the Shanghai PTG is available: https://wiki.openstack.org/wiki/CinderUssuriPTGSummary Some topics are flagged for follow-up at this week's Cinder Virtual PTG, but any topics that you think could use more discussion are fair game to add to the Virtual PTG planning etherpad: https://etherpad.openstack.org/p/cinder-ussuri-virtual-ptg-planning The list of topics for the Virtual PTG is unordered right now. I'll send an email out tonight (New York time) when I've got them ordered so you'll know what to expect (roughly). cheers, brian

1 0

[ptg][neutron] Ussuri PTG summary
by Slawek Kaplonski 24 Nov '19

24 Nov '19

Hi Neutron team, First if all thank to all of You for great and very productive week during the PTG in Shanghai. Below is summary of our discussions from whole 3 days. If I forgot about something, please respond to the email and update missing informations. But if You want to have follow up discussion about one of the topics from this summary, please start a new thread to keep this one only as high level summary of the PTG. On boarding =========== Slides from onboarding session can be found at [1] If You have any follow up questions to us about onboarding, or You need help with starting any work in Neutron team, please contact me or Miguel Lavalle by email or on IRC. My IRC nick is slaweq and Miguel's nick is mlavalle. We are available on #openstack-neutron channel @freenode. Train retrospective =================== Good things in Train cycle: * working with this team is still good experience * core team is stable, and we didn't lost any core reviewers during the cycle, * networking is still one of key reasons why people use OpenStack Not good things: * dimished vitality in stadium projects - we had also forum session and follow discussion about this later during the PTG, * gate instability - we have seen many issues which were out of our control, like infra problems, grenade jobs failures, other projects failures, but also many bugs on our side, * we have really a lot of jobs in our check/gate queue. If each of them is failing 5% of times, it's hard to merge any patch as almost every time, one of jobs will fail. Later during the PTG we also discussed that topic and we were looking for some jobs which we maybe can potentially drop from our queues. See below for summary about that, Action items/improvements: * many team meetings each week. We decided to limit number of meetings by: ** consolidate performance subteam meeting into weekly team meeting - this topic will be added to the team meeting's agenda for team meetings on Monday, ** consolidate ovn convergence meeting into weekly team meeting - this topic will be added to the team meeting's agenda for team meetings on Tuesday, ** we need to check if QoS subteam meeting is still needed, * Review process: list of actual review priorities would be useful for the team, we will add "Review-Priority" label to the Neutron reviews board and try to use it during the Ussuri cycle. Openvswitch agent enhancements ============================== We had bunch of topics related to potential improvements for neutron-openvswitch-agent proposed mostly by Liu Yulong. Slides with his proposals are available at [2]. * retire DHCP agent - resyncs of DHCP agent are problematic, especially when agent hosts many networks. Proposal was to add new L2 agent's extension which could be used instead of "regular" DHCP agent and to provide only basic DHCP functionalities. Such solutions would work in the way quite similar to how networking-ovn works today but we would need to implement and maintain own dhcp server application. Problems of this solution are: ** problems with compatibility e.g. with Ironic, ** how it would work with mixed deployments, e.g. with ovs and sriov agents, ** support for dhcp options, Advantages of this solution: ** fully distributed DHCP service, ** no DHCP agents, so less RPC messages on the bus and easier maintanance of the agents, Team's feedback for that is that this is potentially nice solution which may helps in some specific, large scale deploymnets. We can continue discussion about this during Ussuri cycle for sure. * add accepted egress fdb flows We agreed that this is a bug and we should continue work on this to propose some way to fix it. Solution proposed by LIU during this discussion wasn't good as it could potentially break some corner cases. * new API and agent for L2 traffic health check The team asked to add to the spec some more detailed and concrete use cases with explanation how this new API may help operator of the cloud to investigate where the problem actually is. * Local flows cache and batch updating The team agreed that as long as this will be optional solution which operator can opt-in we can give it a try. But spec and discuss details there will be necessary. * stop processing ports twice in ovs-agent We all agreed that this is a bug and should be fixed. But we have to be careful as fixing this bug may cause some other problems e.g. with live-migration - see nova-neutron cross project session. * ovs-agent: batch flow updates with --bundle We all agreed that this can be done as an improvement of existing code. Similar option is already used in openvswitch firewall driver. Neutron - Cyborg cross project session ====================================== Etherpad for the session is at [3]. Cyborg team wants to include Neutron in workflow of spawning VMs with Smart NICs or accelerator cards. From Neutron's side, required change is to allow including "accel" data in port binding profile. As long as this will be well documented what can be placed there, there should be no problem with doing that. Technically we can place almost anything there. Neutron - Kuryr cross project session ===================================== Etherpad for the session is at [4]. Kuryr team proposed 4 improvements for Neutron which would help a lot Kuryr. Ideas are: * Network cascade deletion, * Force subport deletion, * Tag resources at creation time, * Security group creation with rules & bulk security group rule creation All of those ideas makes sense for Neutron team. Tag resources at creation time is even accepted rfe already - see [5] but there was no volunteer to implement it. We will add it to list of our BPs tracked weekly on team meeting. Miguel Lavalle is going to take a look at it during this cycle. For other proposals we need to have RFEs reported first. Starting the process of removing ML2/Linuxbridge ================================================ Currently in Neutron tree we have 4 drivers: * Linuxbridge, * Openvswitch, * macvtap, * sriov. SR-IOV driver is out of discussion here as this driver is addressing slightly different use case than other out drivers. We started discussion about above topic because we don't want to end up with too many drivers in-tree and we also had some discussions (and we have spec for that already) about include networking-ovn as in-tree driver. So with networking-ovn in-tree we would have already 4 drivers which can be used on any hardware: linuxbridge, ovs, macvtap and ovn. Conclusions from the discussion are: * each driver requires proper testing in the gate, so we need to add many new jobs to our check/gate queue, * currently linuxbridge driver don't have a lot of development and feature parity gaps between linuxbridge and ovs drivers is getting bigger and bigger (e.g. dvr, trunk ports), * also macvtap driver don't have a lot of activity in last few cycles. Maybe this one could be also considered as candidate to deprecation, * we need to have process of deprecating some drivers and time horizon for such actions should be at least 2 cycles. * we will not remove any driver completly but rather we will move it to be in stadium process first so it still can be maintained by people who are interested in it. Actions to do after this discussion: * Miguel Lavalle will contact RAX and Godaddy (we know that those are Linuxbridge users currently) to ask about their feedback about this, * if there are any other companies using LB driver, Nate Johnston is willing to help conctating them, please reach to him in such case. * we may ratify marking linuxbridge as deprecated in the team meeting during Ussuri cycle if nothing surprising pops in. Encrypted(IPSec) tenant networks ================================ Interesting topic proposed but we need to have RFE and spec with more detailed informations about it to continue discussions. Medatada service over IPv6 ========================== This is continuation of old RFE [6]. The only real problem is to choose proper IPv6 address which will be well known address used e.g. by cloud-init. Original spec proposed fe80::a9fe:a9fe as IPv6 address to access metadata service. We decided to be bold and define the standard. Bence Romsics and Miguel Lavalle volunteered to reach out to cloud-init maintainers to discuss that. walkthrough of OVN ================== Since some time we have in review spec about ml2/ovs and ovn convergence. See [7] for details. List of parity gaps between those backends is available at [8]. During the discussion we talked about things like: * migration from ml2/ovs to ml2/ovn - some scripts are already done in [9], * migration from ml2/lb to ml2/ovn - there was no any work done in this topic so far but it should be doable also if someone would need it and want to invest own time for that, * include networking-ovn as in-tree neutron driver and reasons why it could be good idea. Main reasons of that are: ** that would help growing networking-ovn community, ** would help to maintain a healthy project team, ** the default drivers have always been in-tree, However such inclusion may also hurt modularity/logical separation/dependency management/packaging/etc so we need to consider it really carefully and consider all points of view and opinions. Next action item on this topic is to write more detailed summary of this topic and send it to ML and ask wider audience for feedback. IPv6 devstack tempest test configuration vs OVN =============================================== Generally team supports idea which was described during this session and we should change sligtly IPv6 config on e.g. devstack deployments. Neutron - Edge SIG session ========================== We discussed about RFE [10]. This will require also changes on placement side. See [11] for details. Also some cyborg and ovn related changes may be relevant to topics related to Edge. Currently specs which we have are only related to ML2/OVS solution. Neutron - Nova cross project session ==================================== Etherpad for this session is on [12]. Summary written already by gibi can be found at [13]. On [14] You can find image which shows in visual way problem with live-migration of instances with SR-IOV ports. Policy handling in Neutron ========================== The goal of the session was to plan on Neutron's side similar effort to what services like nova are doing now to use new roles like reader and scopes, like project, domain, system provided by Keystone. Miguel Lavalle volunteered to work on this for Neutron and to be part of popup team for cross project collaboration on this topic. Neutron performance improvements ================================ Miguel Lavalle shown us his new profiling decorator [15] and how we all can use it to profile some of API calls in Neutron. Reevaluate Stadium projects =========================== This was follow up discussion after forum session. Notes from forum session can be found at [16]. Nate also prepared some good data about stadium projects activity in last cycles. See at [17] and [18] for details. We all agreed that projects which are in (relatively) good condition now are: * networking-ovn, * networking-odl, * ovsdbapp Projects in bad condition are other projects, like: * neutron-interconnection, * networking-sfc, * networking-bagpipe/bgpvpn, * networking-midonet, * neutron-fwaas and neutron-fwaas-dashboard, * neutron-dynamic-routing, * neutron-vpnaas and neutron-vpnaas-dashboard, We decided to immediately remove neutron-interconnection project as it was never really implemented. For other of those projects, we will send emails to ML to ask for potential maintainers of those projects. If there will be no any volunteers to maintain some of those projects, we will deprecated them and move to "x/" namespace in 2 cycles. Floating IP's On Routed Networks ================================ There is still interest of doing this. Lajos Katona started adding some scenario tests for routed networks already as we need improved test coverage for this feature. Miguel Lavalle said that he will possibly try to work on implementing this in Ussuri cycle. L3 agent enhancement ==================== We talked about couple potential improvements of existing L3 agent, all proposed by LIU Yulong. * retire metering-agent It seems that there is some interest in metering agent recently so we shouldn't probably consider of retiring it for now. We also talked about adding new "tc based" driver to the metering agent and this discussion can be continue on rfe bug [19]. * Centralized DNAT (non-DVR) traffic (floating IP) Scale-out This is proposal of new DVR solution. Some details of this new solution are available at [20]. We agreed that this proposal is trying to solve some very specific use case, and it seems to be very complicated solution with many potential corner cases to address. As a community we don't want to introduce and maintain such complicated new L3 design. * Lazy-load agent side router resources when no related service port Team wants to see RFE with detailed description of the exact problem which this is trying to solve and than continue discussion on such RFE. Zuul jobs ========= In this session we talked about jobs which we can potentially promote to be voting (and we didn't found any of such) and about jobs which we maybe can potentially remove from our queues. Here is what we agreed: * we have 2 iptables_hybrid jobs - one on Fedora and one on Ubuntu - we will drop one of those jobs and left only one of them, * drop neutron-grenade job as it is running still on py27 - we have grenade-py3 which is the same job but run on py36 already, * as it is begin of the cycle, we will switch in devstack neutron uwsgi to be default choice and we will remove "-uwsgi" jobs from queue, * we should compare our single node and multinode variants of same jobs and maybe promote multinode jobs to be voting and then remove single node job - I volunteered to do that, * remove our existing experimental jobs as those jobs are mostly broken and nobody is run those jobs in experimental queue actually, * Yamamoto will check failing networking-midonet job and propose patch to make it passing again, * we will change neutron-tempest-plugin jobs for branch in EM phase to always use certain tempest-plugin and tempest tag, than we will remove those jobs from check and gate queue in master branch, Stateless security groups ========================= Old RFE [21] was approved for neutron-fwaas project but we all agreed that this should be now implemented for security groups in core Neutron. People from Nuage are interested in work on this in upstream. We should probably also explore how easy/hard it will be to implement it in networking-ovn backend. Old, stagnant specs =================== During this session we decided to abandon many of old specs which were proposed long time ago and there is currently no any activity and interest in continue working on them. If anyone would be interested in continue work on some of them, feel free to contact neutron core team on irc or through email and we can always reopen such patch. Community Goal things ===================== We discussed about currently proposed community goals and who can take care of which goal on Neutron's side. Currently there are proposals of community goals as below: * python3 readiness - Nate will take care of this, * move jobs definitions to zuul v3 - I will take care of it. In core neutron and neutron-tempest-plugin we are (mostly) done. On stadium projects' side this will require some work to do, * Project specific PTL and contributor guides - Miguel Lavalle will take care of this goal as former PTL, We will track progress of community goals weekly in our team meetings. Neutron-lib =========== As some time ago our main neutron-lib maintainer (Boden) leaved from the project, we need some new volunteers to continue work on it. Todo list is available on [22]. This should be mostly important for people who are maintaining stadium projects or some 3rd party drivers/plugins so if You are doing things like that, please check list from [22] and reach out to us on ML or #openstack-neutron IRC channel. [1] https://www.slideshare.net/SawomirKaposki/neutron-on-boarding-room [2] https://github.com/gotostack/shanghai_ptg/blob/master/shanghai_neutron_ptg_… [3] https://etherpad.openstack.org/p/Shanghai-Neutron-Cyborg-xproj [4] https://etherpad.openstack.org/p/kuryr-neutron-nice-to-have [5] https://bugs.launchpad.net/neutron/+bug/1815933 [6] https://bugs.launchpad.net/neutron/+bug/1460177 [7] https://review.opendev.org/#/c/658414/ [8] https://etherpad.openstack.org/p/ML2-OVS-OVN-Convergence [9] https://github.com/openstack/networking-ovn/tree/master/migration [10] https://bugs.launchpad.net/neutron/+bug/1832526 [11] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/009991.… [12] https://etherpad.openstack.org/p/ptg-ussuri-xproj-nova-neutron [13] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010654… [14] https://imgur.com/a/12PrQ9W [15] https://review.opendev.org/678438 [16] https://etherpad.openstack.org/p/PVG-Neutron-stadium-projects-the-path-forw… [17] https://ethercalc.openstack.org/neutron-stadium-train-metrics [18] https://ibb.co/SBzDGdD [19] https://bugs.launchpad.net/neutron/+bug/1817881 [20] https://imgur.com/a/6MeNUNb [21] https://bugs.launchpad.net/neutron/+bug/1753466 [22] https://etherpad.openstack.org/p/neutron-lib-volunteers-and-punch-list -- Slawek Kaplonski Senior software engineer Red Hat

7 8

[neutron] [all] Networking-ovn and neutron convergence
by Brian Haley 24 Nov '19

24 Nov '19

Hi, For some time we have been discussing in the Neutron community the possibility of including the networking-ovn driver [1] as one of the in-tree Neutron drivers. There is already a spec [2] describing in detail why we want to do this and why we think it is good idea. We also discussed this during the PTG in Shanghai within our team [3], and had a discussion at the ops-meetup as well [4]. The OVN backend is free of many well-known issues which are impacting the existing ML2/OVS reference implementation today with the OVS agent. For example, OVN provides: * Control plane performance optimizations by not using rabbitmq; * DVR (and HA) by default, based on OpenFlow so it can be easy offloaded e.g. by SmartNICs; * Distributed DHCP; There are some feature parity gaps when comparing it to ML2/OVS that we plan to address. See [2] for details. We think that merging this code into the neutron repository will help to grow the networking-ovn community and will help us to keep a healthy Neutron team as well by increasing the number or contributors. Our current plan is to start merging code from networking-ovn into the neutron repository as soon as possible in the Ussuri cycle. But we also wanted to get any additional opinions from the wider community about this plan. What do users and operators of Neutron think about this? [1] https://opendev.org/openstack/networking-ovn [2] https://review.opendev.org/#/c/658414/ [3] https://etherpad.openstack.org/p/Shanghai-Neutron-Planning-restored Lines 252 - 295 [4] https://etherpad.openstack.org/p/shanghai-ptg-ops-meetup Lines 62 - 69 Thanks for any feedback, -Brian (and Slawek)

5 4

[all][neutron][networking-bagpipe][networking-bgpvpn] Maintainers needed
by Slawek Kaplonski 24 Nov '19

24 Nov '19

Hi, Over the past couple of cycles we have noticed that new contributions and maintenance efforts for networking-bagpipe and networking-bgpvpn were almost non existent. This impacts patches for bug fixes, new features and reviews. The Neutron core team is trying to at least keep the CI of this project healthy, but we don’t have enough knowledge about the details of the <stadium project name> code base to review more complex patches. During the PTG in Shanghai we discussed that with operators and TC members during the forum session [1] and later within the Neutron team during the PTG session [2]. During these discussions, with the help of operators and TC members, we reached the conclusion that we need to have someone responsible for maintaining those projects. This doesn’t mean that the maintainer needs to spend full time working on those projects. Rather, we need someone to be the contact person for the project, who takes care of the project’s CI and review patches. Of course that’s only a minimal requirement. If the new maintainer works on new features for the project, it’s even better :) If we don’t have any new maintainer(s) before milestone Ussuri-2, which is Feb 10 - Feb 14 according to [3], we will need to mark networking-bgpvpn and networking-bagpipe as deprecated and in “V” cycle we will propose to move the projects from the Neutron stadium, hosted in the “openstack/“ namespace, to the unofficial projects hosted in the “x/“ namespace. So if You are using this project now, or if You have customers who are using it, please consider the possibility of maintaining it. Otherwise, please be aware that it is highly possible that the project will be deprecated and moved out from the official OpenStack projects. [1] https://etherpad.openstack.org/p/PVG-Neutron-stadium-projects-the-path-forw… [2] https://etherpad.openstack.org/p/Shanghai-Neutron-Planning-restored - Lines 379-421 [3] https://releases.openstack.org/ussuri/schedule.html -- Slawek Kaplonski Senior software engineer Red Hat

2 2

[neutron][ovn] networking-ovn-metadata-agent and neutron agent liveness
by Chris Apsey 23 Nov '19

23 Nov '19

All, Currently experimenting with networking-ovn (rdo/train packages on centos7) and I've managed to cobble together a functional deployment with two exceptions: metadata agents and agent liveness. Ref: the metadata issues, it appears that the local compute node ovsdb server listens on a unix socket at /var/run/openvswitch/db.sock as openvswitch:hugetlbfs 0750. Since networking-ovn-metadata-agent runs as neutron, it's not able to interact with the local ovs database and gets stuck in a restart loop and complains about the inaccessible database socket. If I edit the systemd unit file and let the agent run as root, it functions as expected. This obviously isn't a real solution, but indicates to me a possible packaging bug? Not sure what the correct mix of permissions is, or if the local database should be listening on tcp:localhost:6640 as well and that's how the metadata agent should connect. The docs are sparse in this area, but I would imagine that something like the metadata-agent should 'just work' out of the box without having to change systemd unit files or mess with unix socket permissions. Thoughts? Secondly, ```openstack network agent list``` shows that all agents (ovn-controller) are all dead, all the time. However, if I display a single agent ```openstack network agent show $foo```, it shows as live. I looked around and saw some discussions about getting networking-ovn to deal with this better, but as of now the agents are reported as dead consistently unless they are explicitly polled, at least on centos 7. I haven't noticed any real impact, but the testing I'm doing is small scale. Other than those two issues, networking-ovn is great, and based on the discussions around possibly deprecating linuxbridge as an in-tree driver, it would make a great 'default' networking configuration option upstream, given the docs get cleaned up. Thanks in advance, r Chris Apsey

2 2

[kolla] all-in-one install with swift failing
by Ryan Phillips 23 Nov '19

23 Nov '19

Hi All, I've been trying to install Kolla (train) as an all-in-one install. The base install works great: I can get to the dashboard and create a nova instance. I am attempting to add swift support and followed the directions to configure swift. The problem is all the swift containers are coming up and exiting with: + sudo -E kolla_set_configs sudo: no tty present and no askpass program specified + sudo -E kolla_set_configs sudo: no tty present and no askpass program specified + sudo -E kolla_set_configs sudo: no tty present and no askpass program specified + sudo -E kolla_set_configs sudo: no tty present and no askpass program specified Does anyone know what is wrong? Regards, Ryan CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES cea6edf16348 kolla/ubuntu-binary-horizon:train "dumb-init --single-…" 7 minutes ago Up 7 minutes horizon 6c88d426a07d kolla/ubuntu-binary-heat-engine:train "dumb-init --single-…" 7 minutes ago Up 7 minutes heat_engine c28e5ce612dc kolla/ubuntu-binary-heat-api-cfn:train "dumb-init --single-…" 7 minutes ago Up 7 minutes heat_api_cfn bfc6cdd6b969 kolla/ubuntu-binary-heat-api:train "dumb-init --single-…" 7 minutes ago Up 7 minutes heat_api aa5c37b09733 kolla/ubuntu-binary-neutron-metadata-agent:train "dumb-init --single-…" 8 minutes ago Up 8 minutes neutron_metadata_agent 07154a5af33b kolla/ubuntu-binary-neutron-l3-agent:train "dumb-init --single-…" 8 minutes ago Up 8 minutes neutron_l3_agent 17e2265f8d34 kolla/ubuntu-binary-neutron-dhcp-agent:train "dumb-init --single-…" 8 minutes ago Up 8 minutes neutron_dhcp_agent 1fdb7bea5434 kolla/ubuntu-binary-neutron-openvswitch-agent:train "dumb-init --single-…" 8 minutes ago Up 8 minutes neutron_openvswitch_agent e1fe4fffe34d kolla/ubuntu-binary-neutron-server:train "dumb-init --single-…" 8 minutes ago Up 8 minutes neutron_server cc20097e0787 kolla/ubuntu-binary-openvswitch-vswitchd:train "dumb-init --single-…" 9 minutes ago Up 9 minutes openvswitch_vswitchd 3cd5b3ac32a9 kolla/ubuntu-binary-openvswitch-db-server:train "dumb-init --single-…" 9 minutes ago Up 9 minutes openvswitch_db 22684295751d kolla/ubuntu-binary-nova-compute:train "dumb-init --single-…" 10 minutes ago Up 10 minutes nova_compute 535e6c81e8f5 kolla/ubuntu-binary-nova-libvirt:train "dumb-init --single-…" 10 minutes ago Up 10 minutes nova_libvirt 59e06d362509 kolla/ubuntu-binary-nova-ssh:train "dumb-init --single-…" 10 minutes ago Up 10 minutes nova_ssh 575ea9e95e33 kolla/ubuntu-binary-nova-novncproxy:train "dumb-init --single-…" 10 minutes ago Up 10 minutes nova_novncproxy 150d2d5723e6 kolla/ubuntu-binary-nova-conductor:train "dumb-init --single-…" 10 minutes ago Up 10 minutes nova_conductor e6cd7b383972 kolla/ubuntu-binary-nova-api:train "dumb-init --single-…" 10 minutes ago Up 10 minutes nova_api af9df9d41b0f kolla/ubuntu-binary-nova-scheduler:train "dumb-init --single-…" 10 minutes ago Up 10 minutes nova_scheduler 1892ab9d3d19 kolla/ubuntu-binary-placement-api:train "dumb-init --single-…" 12 minutes ago Up 12 minutes placement_api eed26f6ed1b2 kolla/ubuntu-binary-glance-api:train "dumb-init --single-…" 13 minutes ago Up 13 minutes glance_api 983b379c9d52 kolla/ubuntu-binary-swift-proxy-server:train "dumb-init --single-…" 13 minutes ago Restarting (1) 55 seconds ago swift_proxy_server 0a7e6a59b493 kolla/ubuntu-binary-swift-object-expirer:train "dumb-init --single-…" 13 minutes ago Restarting (1) Less than a second ago swift_object_expirer d426ebe62e71 kolla/ubuntu-binary-swift-object:train "dumb-init --single-…" 13 minutes ago Restarting (1) 3 seconds ago swift_object_updater 8aa837f99b40 kolla/ubuntu-binary-swift-object:train "dumb-init --single-…" 13 minutes ago Restarting (1) 4 seconds ago swift_object_replicator 06e5000f82ce kolla/ubuntu-binary-swift-object:train "dumb-init --single-…" 13 minutes ago Restarting (1) 4 seconds ago swift_object_auditor ac17c224738b kolla/ubuntu-binary-swift-object:train "dumb-init --single-…" 14 minutes ago Restarting (1) 4 seconds ago swift_object_server fca502a360fb kolla/ubuntu-binary-swift-container:train "dumb-init --single-…" 14 minutes ago Restarting (1) 8 seconds ago swift_container_updater 8def0c6efde9 kolla/ubuntu-binary-swift-container:train "dumb-init --single-…" 14 minutes ago Restarting (1) 8 seconds ago swift_container_replicator e2aa84ebed7d kolla/ubuntu-binary-swift-container:train "dumb-init --single-…" 14 minutes ago Restarting (1) 8 seconds ago swift_container_auditor e7aabf20c11a kolla/ubuntu-binary-swift-container:train "dumb-init --single-…" 14 minutes ago Restarting (1) 11 seconds ago swift_container_server cc754157a930 kolla/ubuntu-binary-swift-account:train "dumb-init --single-…" 14 minutes ago Restarting (1) 14 seconds ago swift_account_reaper b3ef7952d2d3 kolla/ubuntu-binary-swift-account:train "dumb-init --single-…" 14 minutes ago Restarting (1) 14 seconds ago swift_account_replicator 7b984325b708 kolla/ubuntu-binary-swift-account:train "dumb-init --single-…" 14 minutes ago Restarting (1) 14 seconds ago swift_account_auditor 06454711e2f9 kolla/ubuntu-binary-swift-account:train "dumb-init --single-…" 14 minutes ago Restarting (1) 14 seconds ago swift_account_server 226002e50961 kolla/ubuntu-binary-swift-rsyncd:train "dumb-init --single-…" 14 minutes ago Restarting (1) 19 seconds ago swift_rsyncd 06ede7f32329 kolla/ubuntu-binary-keystone-fernet:train "dumb-init --single-…" 18 minutes ago Up 18 minutes keystone_fernet b6f258bfee93 kolla/ubuntu-binary-keystone-ssh:train "dumb-init --single-…" 18 minutes ago Up 18 minutes keystone_ssh c237390ab2d3 kolla/ubuntu-binary-keystone:train "dumb-init --single-…" 18 minutes ago Up 18 minutes keystone 9d55b6233854 kolla/ubuntu-binary-rabbitmq:train "dumb-init --single-…" 18 minutes ago Up 18 minutes rabbitmq 037db9b550d9 kolla/ubuntu-binary-memcached:train "dumb-init --single-…" 19 minutes ago Up 19 minutes memcached 747026ed2053 kolla/ubuntu-binary-mariadb:train "dumb-init -- kolla_…" 19 minutes ago Up 19 minutes mariadb 07148ecdfbd9 kolla/ubuntu-binary-chrony:train "dumb-init --single-…" 19 minutes ago Up 19 minutes chrony cc6883333991 kolla/ubuntu-binary-cron:train "dumb-init --single-…" 19 minutes ago Up 19 minutes cron 2eb7cb285418 kolla/ubuntu-binary-kolla-toolbox:train "dumb-init --single-…" 20 minutes ago Up 19 minutes kolla_toolbox b21c793b31d3 kolla/ubuntu-binary-fluentd:train "dumb-init --single-…" 20 minutes ago Up 20 minutes fluentd

2 2

[ops] topics for London ops meetup jan 202
by Chris Morgan 22 Nov '19

22 Nov '19

https://twitter.com/osopsmeetup/status/1197920312555429889?s=20 -- Chris Morgan <mihalis68(a)gmail.com>

1 0

[oslo] Reminder: Virtual PTG on Monday the 25th
by Ben Nemec 22 Nov '19

22 Nov '19

Time: 1500-1700 UTC (starts at the regular meeting time) Etherpad: https://etherpad.openstack.org/p/oslo-shanghai-topics Location: https://meet.jit.si/oslo-ptg (note that I've had trouble using Firefox with Jitsi, Chrome seems to work fine) I look forward to seeing everyone! -Ben

1 0

[dev] Upgrading flake8 to support f-strings
by Thierry Carrez 22 Nov '19

22 Nov '19

Hi everyone, TL;DR: is it time for us to update flake8 and break the world? Long version: Ussuri-supported Python runtimes will be python 3.6 and 3.7, which opens the marvelous world of f-strings to us. For those not familiar with f-strings, I recommend reading: https://realpython.com/python-f-strings/ In a recent project[1] I tried to use them, only to get pep8 job failures[2]. The old flake8 version we are using in hacking is exploding trying to parse f-strings, with a cryptic "AttributeError: 'FlakesChecker' object has no attribute 'JOINEDSTR'" error. Of course that was long-fixed in pyflakes (>=1.4.0) but that version is only used starting in flake8>=3.3.0, and hacking is capping flake8<2.7.0. We do cap those aggressively for a reason: bumping that cap triggers pep8 job failures everywhere as flake8 decides to pay attention to slightly different things, and we lose a lot of time collectively chasing down those syntax glitches everywhere. Which is why we haven't really bumped those since 2016... But proper support of f-strings might be a good reason to update? Thoughts? [1] https://review.opendev.org/#/c/695457/1 [2] https://zuul.opendev.org/t/openstack/build/bd97e5397b184176aac94251cc0a9220 -- Thierry Carrez (ttx)

10 21

[nova] Why don't we have a SNAPSHOTTING status?
by Matt Riedemann 22 Nov '19

22 Nov '19

This came up in IRC today [1]. There are two issues: 1. As far as I can see, when creating a snapshot of a volume-backed server the API does not change the instance task_state so technically the user could perform some other action on the server while we're creating the volume snapshot(s), like try to resize it. My guess is that would fail since we likely can't be doing things like creating and deleting volume attachments on a volume that is actively being snapshot. 2. For image-backed servers, we set the progress the task_states through a few states [2] but those aren't conveyed in the overall server "status" field in the API [3]. Does anyone have any historical context on either of those issues? Issue #1 reminds me that we don't have a task_state transition while confirming a resized server either [4] but I guess that's less of an issue because there are not a lot of things you can do to a server in VERIFY_RESIZE status (delete and revert the resize I think), but that does mean I could fire off separate confirmResize and then revertResize actions on the same server at the same time and the confirm will probably fail down in compute rather than in the API with a 409 error. [1] http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.… [2] https://github.com/openstack/nova/blob/991d675675c1c6bb87a2b9d19327e2b4473f… [3] https://github.com/openstack/nova/blob/991d675675c1c6bb87a2b9d19327e2b4473f… [4] https://github.com/openstack/nova/blob/991d675675c1c6bb87a2b9d19327e2b4473f… -- Thanks, Matt

1 0

[sig] Forming a Large scale SIG
by Thierry Carrez 22 Nov '19

22 Nov '19

Hi everyone, In Shanghai we held a forum session to gauge interest in a new SIG to specifically address cluster scaling issues. In the past we had several groups ("Large deployments", "Performance", LCOO...) but those efforts were arguably a bit too wide and those groups are now abandoned. My main goal here is to get large users directly involved in a domain where their expertise can best translate into improvements in the software. It's easy for such a group to go nowhere while trying to boil the ocean. To maximize its chances of success and make it sustainable, the group should have a narrow focus, and reasonable objectives. My personal idea for the group focus was to specifically address scaling issues within a single cluster: basically identify and address issues that prevent scaling a single cluster (or cell) past a number of nodes. By sharing analysis and experience, the group could identify common pain points that, once solved, would help raising that number. There was a lot of interest in that session[1], and it predictably exploded in lots of different directions, including some that are definitely past a single cluster (like making Neutron better support cells). I think it's fine: my initial proposal was more of a strawman. Active members of the group should really define what they collectively want to work on. And the SIG name should be picked to match that. I'd like to help getting that group off the ground and to a place where it can fly by itself, without needing external coordination. The first step would be to identify interested members and discuss group scope and objectives. Given the nature of the group (with interested members in Japan, Europe, Australia and the US) it will be hard to come up with a synchronous meeting time that will work for everyone, so let's try to hold that discussion over email. So to kick this off: if you are interested in that group, please reply to this email, introduce yourself and tell us what you would like the group scope and objectives to be, and what you can contribute to the group. Thanks! [1] https://etherpad.openstack.org/p/PVG-large-scale-SIG -- Thierry Carrez (ttx)

9 9

[tripleo] TripleO CI Summary: Sprint 39
by Rafael Folco 22 Nov '19

22 Nov '19

Greetings, The TripleO CI team has just completed Sprint 39 / Unified Sprint 18 (Oct 31 thru Nov 20). The following is a summary of completed work during this sprint cycle: - Evaluated and implemented CI jobs in Zuul that deal with RPM build artifacts for ceph-ansible and podman 3rd party testing. - Explored design options and created a PoC for individual component testing in the promotion pipeline. This effort will add an additional verification layer to check OpenStack components (compute, networking, storage, etc) with stable builds, and ease root cause determination when it breaks the code. - Fixed the docker login module in the new promotion code. - Improve tests for verifying a full promotion workflow running on the staging environment. The planned work for the next sprint [1] are: - Close-out work on 3rd-party testing jobs for podman and ceph-ansible. - Close-out work on the design enhancements to the promotion server. - Implement component pipeline running daily and promoting separated from the integration jobs. - Address issues and technical debt tasks in TripleO CI realm. The Ruck and Rover for this sprint are Marios Andreou (marios) and Chandan Kumar (chkumar). Please direct questions or queries to them regarding CI status or issues in #tripleo, ideally to whomever has the ‘|ruck’ suffix on their nick. Ruck/rover notes are being tracked in etherpad [2]. Thanks, rfolco [1] https://tree.taiga.io/project/tripleo-ci-board/taskboard/unified-sprint-19 [2] https://etherpad.openstack.org/p/ruckroversprint19

1 0