September 2020 - openstack-discuss

Re: [neutron][networking-midonet] Maintainers needed
by Sebastian Saemann 10 Sep '20

10 Sep '20

Hi Slawek, we at NETWAYS are running most of our neutron networking on top of midonet and wouldn't be too happy if it gets deprecated and removed. So we would like to take over the maintainer role for this part. Please let me know how to proceed and how we can be onboarded easily. Best regards, Sebastian -- Sebastian Saemann Head of Managed Services NETWAYS Managed Services GmbH | Deutschherrnstr. 15-19 | D-90429 Nuernberg Tel: +49 911 92885-0 | Fax: +49 911 92885-77 CEO: Julian Hein, Bernd Erk | AG Nuernberg HRB25207 https://netways.de | sebastian.saemann(a)netways.de ** NETWAYS Web Services - https://nws.netways.de **

6 17

[neutron] Drivers meeting 11.09.2020 cancelled
by Slawek Kaplonski 10 Sep '20

10 Sep '20

Hi, There is no any RFE for tomorrow drivers meeting so lets cancel it and focus on review of the opened patches during that time. See You all next week on the meetings. Have a great weekend :) -- Slawek Kaplonski Principal software engineer Red Hat

1 0

[Keystone] TypeError: list indices must be integers or slices, not str
by Tony Liu 10 Sep '20

10 Sep '20

Any clues to this error? 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context Traceback (most recent call last): 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/middleware/auth_context.py", line 103, in _inner 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context return method(self, request) 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/middleware/auth_context.py", line 353, in process_request 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context resp = super(AuthContextMiddleware, self).process_request(request) 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystonemiddleware/auth_token/__init__.py", line 411, in process_request 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context allow_expired=allow_expired) 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystonemiddleware/auth_token/__init__.py", line 445, in _do_fetch_token 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context data = self.fetch_token(token, **kwargs) 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/middleware/auth_context.py", line 248, in fetch_token 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context token, access_rules_support=ACCESS_RULES_MIN_VERSION) 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/common/manager.py", line 115, in wrapped 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context __ret_val = __f(*args, **kwargs) 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/token/provider.py", line 146, in validate_token 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context self._is_valid_token(token, window_seconds=window_seconds) 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/token/provider.py", line 199, in _is_valid_token 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context self.check_revocation(token) 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/common/manager.py", line 115, in wrapped 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context __ret_val = __f(*args, **kwargs) 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/token/provider.py", line 137, in check_revocation 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context return self.check_revocation_v3(token) 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/common/manager.py", line 115, in wrapped 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context __ret_val = __f(*args, **kwargs) 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/token/provider.py", line 133, in check_revocation_v3 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context token_values = self.revoke_api.model.build_token_values(token) 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/models/revoke_model.py", line 245, in build_token_values 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context if token.roles is not None: 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/models/token_model.py", line 458, in roles 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context roles = self._get_project_roles() 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/models/token_model.py", line 423, in _get_project_roles 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context roles.append({'id': r['id'], 'name': r['name']}) 2020-09-10 11:03:44.913 30 ERROR keystone.server.flask.request_processing.middleware.auth_context TypeError: list indices must be integers or slices, not str Thanks! Tony

1 0

[cinder][stable] branch freeze for ocata, pike
by Brian Rosmaita 10 Sep '20

10 Sep '20

tl;dr - do not approve any backports to stable/ocata or stable/pike in any Cinder project deliverable stable/ocata has been tagged with ocata-eol in cinder, os-brick, python-cinderclient, and python-brick-cinderclient-ext. Nothing should be merged into stable/ocata in any of these repositories during the interim period before the branches are deleted. stable/pike: the changes discussed in [0] have merged, and I've proposed the pike-eol tags [1]. Nothing should be merged into stable/pike in any of our code repositories from now until the branches are deleted. [0] http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016076.html [1] https://review.opendev.org/#/c/742523/

4 6

[Keystone] KeyError: 'domain_id'
by Tony Liu 10 Sep '20

10 Sep '20

Here is another exception. Any clues? 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context [req-534d9855-8113-450d-8f9f-d93c0d961d24 113ee63a9ed0466794e24d069efc302c 4c142a681d884010ab36a7ac687d910c - default default] 'domain_id': KeyError: 'domain_id' 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context Traceback (most recent call last): 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/middleware/auth_context.py", line 103, in _inner 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context return method(self, request) 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/middleware/auth_context.py", line 353, in process_request 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context resp = super(AuthContextMiddleware, self).process_request(request) 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystonemiddleware/auth_token/__init__.py", line 411, in process_request 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context allow_expired=allow_expired) 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystonemiddleware/auth_token/__init__.py", line 445, in _do_fetch_token 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context data = self.fetch_token(token, **kwargs) 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/middleware/auth_context.py", line 248, in fetch_token 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context token, access_rules_support=ACCESS_RULES_MIN_VERSION) 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/common/manager.py", line 115, in wrapped 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context __ret_val = __f(*args, **kwargs) 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/token/provider.py", line 145, in validate_token 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context token = self._validate_token(token_id) 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "</usr/lib/python3.6/site-packages/decorator.py:decorator-gen-26>", line 2, in _validate_token 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/dogpile/cache/region.py", line 1360, in get_or_create_for_user_func 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context key, user_func, timeout, should_cache_fn, (arg, kw) 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/dogpile/cache/region.py", line 962, in get_or_create 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context async_creator, 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/dogpile/lock.py", line 187, in __enter__ 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context return self._enter() 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/dogpile/lock.py", line 94, in _enter 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context generated = self._enter_create(value, createdtime) 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/dogpile/lock.py", line 180, in _enter_create 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context return self.creator() 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/dogpile/cache/region.py", line 916, in gen_value 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context *creator_args[0], **creator_args[1] 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/token/provider.py", line 179, in _validate_token 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context token.mint(token_id, issued_at) 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/models/token_model.py", line 580, in mint 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context self._validate_token_user() 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/models/token_model.py", line 503, in _validate_token_user 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context if not self.user_domain.get('enabled'): 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context File "/usr/lib/python3.6/site-packages/keystone/models/token_model.py", line 141, in user_domain 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context self.user['domain_id'] 2020-09-10 10:16:45.050 28 ERROR keystone.server.flask.request_processing.middleware.auth_context KeyError: 'domain_id' Thanks! Tony

1 0

[gance_store][FFE] Cinder multiple stores support
by Rajat Dhasmana 10 Sep '20

10 Sep '20

Hi Team, Last week we released glance_store 2.3.0 which adds support for configuring cinder multiple stores as glance backend. While adding functional tests in glance for the same [1], we have noticed that it is failing with some hard requirements from oslo side to use project_id instead of tenant and user_id instead of user. It is really strange behavior as this failure occurs only in functional tests but works properly in the actual environment without any issue. The fix is proposed in glance_store [2] to resolve this issue. I would like to apply for FFE with this glance_store patch [2] to be approved and release a new version of glance_store 2.3.1. Kindly provide approval for the same. [1] https://review.opendev.org/#/c/750144/ [2] https://review.opendev.org/#/c/750131/ Thanks and Regards, Rajat Dhasmana

2 2

Farewell Party for Monty
by corvus＠inaugust.com 10 Sep '20

10 Sep '20

Hi, Monty is starting a new gig and won't be spending as much time with us. Since we all haven't seen each other in a while, let's have one more beer[1] together and say farewell. Join us for a virtual going-away party on meetpad tomorrow (Friday) at 21:00 UTC at this URL: https://meetpad.opendev.org/farewell-mordred Stop by and chat for old time's sake. -Jim [1] Bring your own beer.

1 0

[requirements][FFE] Cinder multiple stores support
by Rajat Dhasmana 10 Sep '20

10 Sep '20

Hi Team, Last week we released glance_store 2.3.0 which adds support for configuring cinder multiple stores as glance backend. While adding functional tests in glance for the same [1], we have noticed that it is failing with some hard requirements from oslo side to use project_id instead of tenant and user_id instead of user. It is really strange behavior as this failure occurs only in functional tests but works properly in the actual environment without any issue. The fix is proposed in glance_store [2] to resolve this issue. I would like to apply for FFE with this glance_store patch [2] to be approved and release a new version of glance_store 2.3.1. Kindly provide approval for the same. [1] https://review.opendev.org/#/c/750144/ [2] https://review.opendev.org/#/c/750131/ Thanks and Regards, Rajat Dhasmana

2 2

Moving on
by Monty Taylor 10 Sep '20

10 Sep '20

Hi everybody, After 10 years of OpenStack, the time has come for me to move on to the next challenge. Actually, the time came a few weeks ago, but writing farewells has always been something I’m particularly bad at. My last day at Red Hat was actually July 31 and I’m now working in a non-OpenStack job. I’m at a loss for words as to what more to say. I’ve never done anything for 10 years before, and I’ll be very surprised if I do anything else for 10 years again. While I’m excited about the new things on my plate, I’ll obviously miss everyone. As I am no longer being paid by an OpenStack employer, I will not be doing any OpenStack things as part of my day job. I’m not sure how much spare time I’ll have to be able to contribute. I’m going to hold off on resigning core memberships pending a better understanding of that. I think it’s safe to assume I won’t be able to continue on as SDK PTL though. I wish everyone all the best, and I hope life conspires to keep us all connected. Thank you to everyone for an amazing 10 years. Monty

16 15

[tc] meeting summary
by Mohammed Naser 10 Sep '20

10 Sep '20

Hi everyone, Here's a summary of what happened in our TC monthly meeting last Thursday, September 3. # ATTENDEES (LINES SAID) - mnaser (72) - gmann (35) - ttx (22) - diablo_rojo (19) - belmoreira (18) - knikolla (16) - njohnston (11) - fungi (7) - smcginnis (2) - ricolin (2) # MEETING SUMMARY 1. Rollcall (mnaser, 14:01:13) 2. Follow up on past action items (mnaser, 14:04:27) - http://eavesdrop.openstack.org/meetings/tc/2020/tc.2020-08-06-14.00.html (mnaser, 14:04:59) 3. OpenStack User-facing APIs and CLIs (belmoreira, 14:11:33) 4. W cycle goal selection start (mnaser, 14:33:43) 5. Completion of retirement cleanup (gmann, 14:34:45) - https://etherpad.opendev.org/p/tc-retirement-cleanup (mnaser, 14:34:52) - https://review.opendev.org/#/c/745403/ (mnaser, 14:35:06) 6. Audit and clean-up tags (gmann, 14:37:25) - https://review.opendev.org/#/c/749363/ (mnaser, 14:37:35) 7. open discussion (mnaser, 14:43:54) # ACTION ITEMS - tc-members to follow up and review "Resolution to define distributed leadership for projects". - mnaser schedule session with sig-arch and k8s steering committee. - njohnston to find someone to work with on getting goals groomed/proposed for W cycle. - belmoreira/knikolla figure out logistics of a document with gaps within osc. - diablo_rojo help schedule forum session for OSC gaps. To read the full logs of the meeting, please refer to http://eavesdrop.openstack.org/meetings/tc/2020/tc.2020-09-03-14.01.log.html Thanks, Mohammed -- Mohammed Naser VEXXHOST, Inc.

1 0

[largescale-sig] Next meeting: September 9, 16utc
by Thierry Carrez 10 Sep '20

10 Sep '20

Hi everyone, Our next meeting will be a EU-US-friendly meeting, today Wednesday, September 9 at 16 UTC[1] in the #openstack-meeting-3 channel on IRC: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200909T16 Feel free to add topics to our agenda at: https://etherpad.openstack.org/p/large-scale-sig-meeting A reminder of the TODOs we had from last meeting, in case you have time to make progress on them: - all to contact US large deployment friends to invite them to next EU-US meeting - ttx to request Forum/PTG sessions - belmoreira, ttx to push for OSops resurrection - all to describe briefly how you solved metrics/billing in your deployment in https://etherpad.openstack.org/p/large-scale-sig-documentation - masahito to push latest patches to oslo.metrics - ttx to look into a basic test framework for oslo,metrics - amorin to see if oslo.metrics could be tested at OVH Talk to you all later, -- Thierry Carrez

1 1

Cloudinit resets network scripts to default configuration DHCP once Config Drive is removed after sometime on Openstack platform
by Vijayendra R Radhakrishna 10 Sep '20

10 Sep '20

1 0

[all][PTL][release] Victoria Cycle Highlights
by Kendall Nelson 09 Sep '20

09 Sep '20

Hello Everyone! It's time to start thinking about calling out 'cycle-highlights' in your deliverables! As PTLs, you probably get many pings and emails from various parties (marketing, management, journalists, etc) asking for highlights of what is new and what significant changes are coming in the new release. By putting them all in the same place it makes them easy to reference because they get compiled into a pretty website like this from the last few releases: Rocky[1], Stein[2], Train[3], Ussuri[4]. As usual, we don't need a fully fledged marketing message, just a few highlights (3-4 ideally), from each project team. Looking through your release notes is a good place to start. *The deadline for cycle highlights is the end of the R-5 week [5] on Sept 11th.* How To Reminder: ------------------------- Simply add them to the deliverables/train/$PROJECT.yaml in the openstack/releases repo like this: cycle-highlights: - Introduced new service to use unused host to mine bitcoin. The formatting options for this tag are the same as what you are probably used to with Reno release notes. Also, you can check on the formatting of the output by either running locally: tox -e docs And then checking the resulting doc/build/html/train/highlights.html file or the output of the build-openstack-sphinx-docs job under html/train/ highlights.html. Can't wait to see what you've accomplished! -Kendall Nelson (diablo_rojo) [1] https://releases.openstack.org/rocky/highlights.html [2] https://releases.openstack.org/stein/highlights.html [3] https://releases.openstack.org/train/highlights.html [4] https://releases.openstack.org/ussuri/highlights.html [5] htt <https://releases.openstack.org/train/schedule.html#t-cycle-highlights> https://releases.openstack.org/victoria/schedule.html

1 1

[cinder][requirements] FFE request for os-brick
by Brian Rosmaita 09 Sep '20

09 Sep '20

Hello Requirements Team, The victoria release of os-brick 4.0.0 that we did last week contained a bugfix that requires a heavyweight binary dependency. Unfortunately, we did not realize this until yesterday. The cinder team reverted that patch [0] and replaced it with a lightweight fix [1] that does not require any new dependencies and has proposed the release of os-brick 4.0.1 [2]. We are asking for a requirements freeze exception so that os-brick 4.0.1 can be included in the victoria openstack release. [0] https://review.opendev.org/750649 [1] https://review.opendev.org/750655 [2] https://review.opendev.org/#/c/750808/ thank you, brian

2 1

[tc] [all] Topics for Cross Community Discussion with Kubernetes ...
by Jay Bryant 09 Sep '20

09 Sep '20

All, Recently, the OpenStack TC has reached out to the Kubernetes Steering Committee for input as we have proposed adding a starter-kit:kubernetes-in-virt tag for projects in OpenStack. This request was received positively and as a result the TC has started brainstorming other topics that we could approach with the k8s community in this [1] etherpad. If you have topics that may be appropriate for this discussion please see the etherpad and add your ideas. Thanks! Jay IRC: jungleboyj [1] https://etherpad.opendev.org/p/kubernetes-cross-community-topics

3 3

[infra][tact-sig] October 2020 vPTG space for TaCT SIG work
by Jeremy Stanley 09 Sep '20

09 Sep '20

Just one last quick check, does anyone think we need dedicated TaCT SIG space in the vPTG schedule? I can snag some for us if so. There seemed to be some consensus on IRC that the OpenStack Testing and Collaboration Tools SIG can just glom onto OpenDev and OpenStack QA team vPTG sessions where relevant, which seems entirely prudent to me, and means we don't need our own separate times to talk about things which are neither relevant to OpenDev nor QA (which I estimate to be roughly nil). -- Jeremy Stanley

1 0

[tc] monthly meeting
by Mohammed Naser 09 Sep '20

09 Sep '20

Hi everyone, Here’s the agenda for our monthly TC meeting. It will happen tomorrow (Thursday the 3rd) at 1400 UTC in #openstack-tc and I will be your chair. If you can’t attend, please put your name in the “Apologies for Absence” section. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting ## ACTIVE INITIATIVES * Follow up on past action items * OpenStack User-facing APIs and CLIs (belmoreira) * W cycle goal selection start * Completion of retirement cleanup (gmann): https://etherpad.opendev.org/p/tc-retirement-cleanup * Audit and clean-up tags (gmann) + Remove tc:approved-release tag https://review.opendev.org/#/c/749363 Thank you, Mohammed -- Mohammed Naser VEXXHOST, Inc.

2 1

[nova] Changes for out-of-tree drivers
by Stephen Finucane 09 Sep '20

09 Sep '20

We're aiming to remove the long-deprecated XenAPI driver before Victoria ends. With that removed, there are a number of XenAPI-specific virt driver APIs that we plan to remove in follow-ups. These are noted at [1]. If you maintain an out-of-tree driver, you will need to account for these changes. Cheers, Stephen [1] https://review.opendev.org/#/c/749300/1/nova/virt/driver.py

1 0

[summit] Schedule is Live for the 2020 Virtual Open Infrastructure Summit
by Thierry Carrez 09 Sep '20

09 Sep '20

Hi everyone, We are excited to announce that the schedule[1] for the virtual 2020 Open Infrastructure Summit is now live featuring keynotes and sessions from users like Volvo, Workday, Société Générale and Ant Group: [1] https://www.openstack.org/summit/2020/summit-schedule The virtual event takes place October 19-23 and includes more than 100 sessions and thousands of attendees are expected to participate, representing 30+ open source communities from more than 100 countries. Sessions for the virtual summit are led by users from global enterprises and research institutions building and operating open infrastructure at scale. The Summit includes: - Sessions spanning 30+ open source projects from technical community leaders and organizations including Alibaba Cloud, AT&T, China Mobile, CERN, European Weather Cloud, GE Digital and Volvo Cars and many more. - Collaborative sessions with project leaders and open source communities, including Airship, Ansible, Ceph, Docker, Kata Containers, Kubernetes, ONAP, OpenStack, Open vSwitch, OPNFV, StarlingX, and Zuul. - Get hands on workshops around open source technologies directly from the developers and operators building the software. - Familiarize with the updated Certified OpenStack Administrator (COA) exam and test your OpenStack knowledge at the COA sessions, sponsored by the OpenStack Foundation, in collaboration with Mirantis. Now what? Register[2] for your free virtual Summit pass and meet the users, developers, and vendors who are building and operating open infrastructure on October 19-23! [2] https://www.eventbrite.com/e/open-infrastructure-summit-2020-tickets-969672… Thank you to our Summit Headline, Premier and Exhibitor sponsors: Huawei, Cisco, InMotion Hosting, Trilio and ZTE. Event sponsors gain visibility with a wide array of open source infrastructure developers, operators and decision makers. Download the Open Infrastructure Summit sponsor prospectus[3] for more information. [3] https://www.openstack.org/summit/2020/sponsors/ Questions? Reach out to summit(a)openstack.org “See you” in October! Cheers, -- Thierry Carrez (ttx)

1 0

[tc] weekly update
by Mohammed Naser 09 Sep '20

09 Sep '20

Hi everyone, Here's an update for what happened in the OpenStack TC this week. You can get more information by checking for changes in openstack/governance repository. We've also included a few references to some important mailing list threads that you should check out. # Patches ## Open Reviews - Add openstack/osops to Ops Docs and Tooling SIG https://review.opendev.org/749835 - Reinstate weekly meetings https://review.opendev.org/749279 - Create starter-kit:kubernetes-in-virt tag https://review.opendev.org/736369 - Resolution to define distributed leadership for projects https://review.opendev.org/744995 - Add assert:supports-standalone https://review.opendev.org/722399 - Retire devstack-plugin-pika project https://review.opendev.org/748730 - Remove tc:approved-release tag https://review.opendev.org/749363 - Retire the devstack-plugin-zmq project https://review.opendev.org/748731 ## Project Updates - Add openstack-helm-deployments to openstack-helm https://review.opendev.org/748302 - Add openstack-ansible/os_senlin role https://review.opendev.org/748677 - kolla-cli: deprecation - Mark as deprecated https://review.opendev.org/749694 - Move ansible-role-XXX-hsm projects to Barbican team https://review.opendev.org/748027 ## General Changes - Drop all exceptions for legacy validation https://review.opendev.org/745403 ## Abandoned Changes - kolla-cli: deprecation - Mark kolla-cli as Deprecated https://review.opendev.org/749746 # Email Threads - Focal Goal Update #4: http://lists.openstack.org/pipermail/openstack-discuss/2020-September/01707… - Forum Submissions Open: http://lists.openstack.org/pipermail/openstack-discuss/2020-September/01693… # Other Reminders - Forum CFP closes Sept 14th! - PTG Signup closes Sept 11th! Thanks for reading! Mohammed & Kendall -- Mohammed Naser VEXXHOST, Inc.

1 0

[tripleo][ansible] ensure python dependencies on hosts for modules/plugins
by Bogdan Dobrelya 09 Sep '20

09 Sep '20

Since most of tripleo-ansible modules do 'import foobar', we should ensure that we have the corresponding python packages installed on target hosts. Some of them, like python3-dmidecode may be in base Centos8 images. But some may not, especially for custom deployed-servers provided by users for deployments. Those packages must be tracked and ensured to be installed by tripleo (preferred), or validated deploy-time (nah...), or at least documented as the modules get created or changed by devs. That also applies to adding action plugins' deps for python-tripleoclient or tripleo-ansible perhaps. Shall we write a spec for that or just address that as a bug [0]? [0] https://bugs.launchpad.net/tripleo/+bug/1894957 -- Best regards, Bogdan Dobrelya, Irc #bogdando

3 2

Application Credentials with federated users
by Alexander Dibbo - UKRI STFC 09 Sep '20

09 Sep '20

Hi All, Is it possible for a user logging in via an oidc provider to generate application credentials? When I try it I get an error about there being no role for the user in the project. We map the users to groups based on assertions in their tokens. It looks like it would work if we mapped users individually to local users in keystone and then gave those roles. I would prefer to avoid using per user mappings for this if possible as it would be a lot of extra work for my team. Regards Alexander Dibbo - Cloud Architect / Cloud Operations Group Leader For STFC Cloud Documentation visit https://stfc-cloud-docs.readthedocs.io<https://stfc-cloud-docs.readthedocs.io/> To raise a support ticket with the cloud team please email cloud-support(a)gridpp.rl.ac.uk<mailto:cloud-support@gridpp.rl.ac.uk> To receive notifications about the service please subscribe to our mailing list at: https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=STFC-CLOUD To receive fast notifications or to discuss usage of the cloud please join our Slack: https://stfc-cloud.slack.com/ This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses. Opinions, conclusions or other information in this message and attachments that are not related directly to UKRI business are solely those of the author and do not represent the views of UKRI.

2 1

Floating IP problem in HA OVN DVR with TripleO
by Reza Bakhshayeshi 09 Sep '20

09 Sep '20

Hi all, I deployed an environment with TripleO Ussuri with 3 HA Controllers and some Compute nodes with neutron-ovn-dvr-ha.yaml Instances have Internet access through routers with SNAT traffic (in this case traffic is routed via a controller node), and by assigning IP address directly from provider network (not having a router). But in case of assigning FIP from provider to an instance, VM Internet connection is lost. Here is the output of router nat lists, which seems OK: # ovn-nbctl lr-nat-list 587182a4-4d6b-41b0-9fd8-4c1be58811b0 TYPE EXTERNAL_IP EXTERNAL_PORT LOGICAL_IP EXTERNAL_MAC LOGICAL_PORT dnat_and_snat X.X.X.X 192.168.0.153 fa:16:3e:0a:86:4d e65bd8e9-5f95-4eb2-a316-97e86fbdb9b6 snat Y.Y.Y.Y 192.168.0.0/24 I replaced FIP with X.X.X.X and router IP with Y.Y.Y.Y When I remove * EXTERNAL_MAC* and *LOGICAL_PORT*, FIP works fine and as it has to be, but traffic routes from a Controller node and it won't be distributed anymore. Any idea or suggestion would be grateful. Regards, Reza

4 7

[nova] Feature Freeze is coming
by Balázs Gibizer 09 Sep '20

09 Sep '20

Hi, Nova will hit Feature Freeze for Victoria release on Thursday. Feature patches that are approved before end of Thursday can be rechecked and rebased to get them merged. Feature authors having patches that are not approved until the freeze but are close can request Feature Freeze Exception on the ML. Granting FFE requires ready patches and a core sponsor who will review those patches. Please observer that we only have 2 weeks before RC1 so time are short for FFEs. Cheers, gibi

1 0

[Image Encryption] No meeting next week
by Josephine Seifert 09 Sep '20

09 Sep '20

Hi, I will be on vacation next week, so there will be no meeting. Josephine(Luzi)

1 0

[neutron] CI meeting 09/09/2020 cancelled
by Slawek Kaplonski 08 Sep '20

08 Sep '20

Hi, I'm off today and I will not be able to chair our CI meeting this week. So lets cancel it and see You on the meeting next week. -- Slawek Kaplonski Principal software engineer Red Hat

1 0

[blazar] Wallaby PTG
by Pierre Riteau 08 Sep '20

08 Sep '20

Hello, As mentioned in a recent IRC meeting [1], I likely won't be able to attend the PTG this year. Tetsuro Nakamura who is the only other active core reviewer at the moment said he won't have time either. Given that we still have lots of pending tasks on Blazar from the last PTG [2], I propose that we don't plan to meet during the PTG in October 2020. Best wishes, Pierre Riteau (priteau) [1] http://eavesdrop.openstack.org/meetings/blazar/2020/blazar.2020-08-18-09.00… [2] https://etherpad.opendev.org/p/blazar-ptg-victoria

1 0

Tripleo Standalone Deployment Issues - Reproduceable
by yuuta.takanashi＠aol.com 08 Sep '20

08 Sep '20

Hi Experts, I am facing on below issues while following guide of Tripleo Standalone Deployment: https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deploym… Issue 1) Using custom domain will cause vm instantiation error after machine reboot Description =========== After Openstack installed, it's adding centos82.localdomain on /etc/hosts and causing OVN Southbound (ovn-sbctl show) using this centos82.localdomain. But after the machine rebooted, "ovn-sbctl show" is showing the correct fqdn: centos82.domain.tld, and it is causing VM instantiate error (Refusing to bind port <UUID> due to no OVN chassis for host: centos82.localdomain). Steps to reproduce ================== Base installation: https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deploym… Tested on Centos 8.2 latest, tested several times on Train and Ussuri. 100% reproduceable. Set the custom FQDN hostname hostnamectl set-hostname centos82.domain.tld hostnamectl set-hostname centos82.domain.tld --transient (also tested with or without this line) cat /etc/hosts 127.0.0.1 centos82.domain.tld localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 centos82.domain.tld localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.156.82 centos82.domain.tld also tested with hostnamectl set-hostname centos82.domain.tld hostnamectl set-hostname centos82.domain.tld --transient (also tested with or without this line) cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.156.82 centos82.domain.tld centos82 also tested with hostnamectl set-hostname centos82.domain.tld hostnamectl set-hostname centos82.domain.tld --transient (also tested with or without this line) cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.156.82 centos82.domain.tld Part of standalone_parameters.yaml for domain: # domain name used by the host NeutronDnsDomain: domain.tld Follow the guide to install (without ceph) (Issue 1B) with ceph was having another error, just tested once on Train: fatal: [undercloud]: FAILED! => { "ceph_ansible_std_out_err": [ "Using /home/stack/standalone-ansible-huva1ccw/ansible.cfg as config file", "ERROR! the playbook: /usr/share/ceph-ansible/site-container.yml.sample could not be found" ], "failed_when_result": true } All will be installed successfully, and able to instantiate VMs and these VMs are able to ping to each other. Then do "reboot" on this standalone host/server. Do instantiate a new VM, and then this new VM status will be ERROR. On Neutron logs: Refusing to bind port <UUID> due to no OVN chassis for host: centos82.localdomain bind_port Found the cause on OVN: Previous after installed (before reboot) [root@centos82 /]# ovn-sbctl show Chassis "<UUID>" hostname: centos82.localdomain Encap geneve ip: "192.168.156.82" options: {csum="true"} [root@centos82 /]# After reboot [root@centos82 /]# ovn-sbctl show Chassis "<UUID>" hostname: centos82.domain.tld Encap geneve ip: "192.168.156.82" options: {csum="true"} [root@centos82 /]# So that the OVN seems could not bind the port to different hostname (that already changed). Environment =========== 1. Exact version of OpenStack you are running: Any, tested Train and Ussuri 2. Centos 8.2 latest update Others (without ceph): https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deploym… Workaround ========== Don't use custom domain, instead use "localdomain" as on the document hostnamectl set-hostname centos82.localdomain [root@centos82 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.156.82 centos82.localdomain centos82 Deploy, test instantiate, reboot, test instantiate, and working perfectly fine. Question: For custom domain, where did I do wrongly? Is it expected or kind of bugs? Issue 2) This is not Openstack issue, perhaps bugs or perhaps my issue, but it is still related with the topic on Tripleo Standalone Deployment. On the same Standalone Deployment Guide (https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deploym…) with Centos 7.6 or 7.7 with "python-tripleoclient" and this has dependency of Docker 1.13.1, encounters kind of Dns querying issue which I posted here: https://serverfault.com/questions/1032816/centos-7-and-docker-1-13-1-error-… Perhaps if anybody knows how to resolve this. Also, just wondering is there any way for "python-tripleoclient" to use newer docker-ce 1.19.x instead of docker 1.13.1? ThanksBest regards, TY

2 1

Windows 10 instance hostname not updating
by its-openstack＠zohocorp.com 08 Sep '20

08 Sep '20

Dear openstack, I have installed openstack train branch, I am facing issue with windows image. all windows 10 instance dosen't get its hostname updated from the metadata, but able to get the metadata(hostname) from inside the instance using powershell. ``` $ Invoke-WebRequest http://169.254.169.254/latest/meta-data/hostname -UseBasicParsing ``` windows2016 instance no issue. using the stable cloudbase-init package for preparation of windows 10. if you would so kindly help us with this issue Regards sysadmin

1 0

[cloudkitty] Virtual PTG planning
by Pierre Riteau 08 Sep '20

08 Sep '20

Hi, You may know that the next PTG will be held virtually during the week of October 26-30, 2020. I will very likely *not* be available during that time, so I would like to hear from the CloudKitty community: - if you would like to meet and for how long (a half day may be enough depending on the agenda) - what day and time is preferred (see list in https://ethercalc.openstack.org/7xp2pcbh1ncb) - if anyone is willing to chair the discussions (I can help you prepare an agenda before the event) Thanks in advance, Pierre Riteau (priteau)

2 2

Create Application Credential not appearing in Horizon
by Alexander Dibbo - UKRI STFC 08 Sep '20

08 Sep '20

Hi All, I'm having an issue where users with the "user" role in my deployment are not seeing the "Create Application Credential" button in horizon although users with the "admin" role can. I have tried modifying the keystone policy a couple of different ways to fix this but none have worked: "identity:create_application_credential": "user_id:%(user_id)s", "identity:create_application_credential": "role: user", "identity:create_application_credential": "rule:owner", "identity:create_application_credential": "", Could anyone point me in the right direction? My deployment is running the Train release of OpenStack Thanks, Alex Regards Alexander Dibbo - Cloud Architect / Cloud Operations Group Leader For STFC Cloud Documentation visit https://stfc-cloud-docs.readthedocs.io<https://stfc-cloud-docs.readthedocs.io/> To raise a support ticket with the cloud team please email cloud-support(a)gridpp.rl.ac.uk<mailto:cloud-support@gridpp.rl.ac.uk> To receive notifications about the service please subscribe to our mailing list at: https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=STFC-CLOUD To receive fast notifications or to discuss usage of the cloud please join our Slack: https://stfc-cloud.slack.com/ This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses. Opinions, conclusions or other information in this message and attachments that are not related directly to UKRI business are solely those of the author and do not represent the views of UKRI.

2 2

[release] Release countdown for week R-5 Sept 7 - 11
by Sean McGinnis 08 Sep '20

08 Sep '20

We are getting close to the end of the Victoria cycle! Next week on September 10 is the victoria-3 milestone, also known as feature freeze. It's time to wrap up feature work in the services and their client libraries, and defer features that won't make it to Wallaby. General Information ------------------- This coming week is the deadline for client libraries: their last feature release needs to happen before "Client library freeze" on September 3. Only bugfix releases will be allowed beyond this point. When requesting those library releases, you can also include the stable/victoria branching request with the review. As an example, see the "branches" section here: https://opendev.org/openstack/releases/src/branch/master/deliverables/pike/… September 10 is also the deadline for feature work in all OpenStack deliverables following the cycle-with-rc model. To help those projects produce a first release candidate in time, only bugfixes should be allowed in the master branch beyond this point. Any feature work past that deadline has to be raised as a Feature Freeze Exception (FFE) and approved by the team PTL. Finally, feature freeze is also the deadline for submitting a first version of your cycle-highlights. Cycle highlights are the raw data hat helps shape what is communicated in press releases and other release activity at the end of the cycle, avoiding direct contacts from marketing folks. See https://docs.openstack.org/project-team-guide/release-management.html#cycle… for more details. Upcoming Deadlines & Dates -------------------------- Client library freeze: September 10 (R-5 week) Victoria-3 milestone: September 10 (R-5 week) Cycle Highlights Due: September 10 (R-5 week) Final Victoria release: October 14 Open Infra Summit: October 19-23 Wallaby PTG: October 26-30

1 0

[neutron] Wallaby PTG planning
by Slawek Kaplonski 08 Sep '20

08 Sep '20

Hi, It's again that time of the cycle (time flies) when we need to start thinking about next cycle already. As You probably know, next virtual PTG will be in October 26-30. I need to book some space for the Neuton team before 11th of September so I prepared doodle [1] with possible time slots. Please reply there what are the best days and hours for You so we can try to schedule our sessions in the time slots which fits best most of us :) Please fill this doodle before 4.09 so I will have time to summarize it and book some slots for us. I also prepared etherpad [2]. Please add Your name if You are going to attend the PTG sessions. Please also add proposals of the topics which You want to discuss during the PTG. [1] https://doodle.com/poll/2ppmnua2nuva5nyp [2] https://etherpad.opendev.org/p/neutron-wallaby-ptg -- Slawek Kaplonski Principal software engineer Red Hat

1 1

[tripleo] docker.io rate limiting
by Wesley Hayutin 08 Sep '20

08 Sep '20

Greetings, Some of you have contacted me regarding the recent news regarding docker.io's new policy with regards to container pull rate limiting [1]. I wanted to take the opportunity to further socialize our plan that will completely remove docker.io from our upstream workflows and avoid any rate limiting issues. We will continue to upload containers to docker.io for some time so that individuals and the community can access the containers. We will also start exploring other registries like quay and newly announced github container registry. These other public registries will NOT be used in our upstream jobs and will only serve the communities individual contributors. Our test jobs have been successful and patches are starting to merge to convert our upstream jobs and remove docker.io from our upstream workflow. [2]. Standalone and multinode jobs are working quite well. We are doing some design work around branchful, update/upgrade jobs at this time. Thanks 0/ [1] https://hackmd.io/ermQSlQ-Q-mDtZkNN2oihQ [2] https://review.opendev.org/#/q/topic:new-ci-job+(status:open+OR+status:merg…

7 13

[tripleo] Proposing Takashi Kajinami to be core on puppet-tripleo
by Emilien Macchi 07 Sep '20

07 Sep '20

Hi people, If you don't know Takashi yet, he has been involved in the Puppet OpenStack project and helped *a lot* in its maintenance (and by maintenance I mean not-funny-work). When our community was getting smaller and smaller, he joined us and our review velicity went back to eleven. He became a core maintainer very quickly and we're glad to have him onboard. He's also been involved in taking care of puppet-tripleo for a few months and I believe he has more than enough knowledge on the module to provide core reviews and be part of the core maintainer group. I also noticed his amount of contribution (bug fixes, improvements, reviews, etc) in other TripleO repos and I'm confident he'll make his road to be core in TripleO at some point. For now I would like him to propose him to be core in puppet-tripleo. As usual, any feedback is welcome but in the meantime I want to thank Takashi for his work in TripleO and we're super happy to have new contributors! Thanks, -- Emilien Macchi

12 12

who is running RabbitMQ HiPE mode?
by Satish Patel 07 Sep '20

07 Sep '20

Folks, I was reading about HiPE mode and thinking to give it a shot but i would like to know feedback from the community? Is it safe to enable HiPE for rabbitMQ to boost performance?

2 1

[Kolla Ansible] Unable to connect to external network after Initial deployment
by Lukasluedke＠web.de 07 Sep '20

07 Sep '20

1 0

[sdk/cli] virtual PTG planning
by Artem Goncharov 07 Sep '20

07 Sep '20

Hi all, I’ve booked a slot for SDK/CLI PTG on 28th of Oct 14:00-17:00 UTC time. Please fill in the etherpad [1] if you want to come in. Regards, Artem (gtema) [1] https://meetpad.opendev.org/etherpad/p/wallaby_sdk_cli <https://meetpad.opendev.org/etherpad/p/wallaby_sdk_cli>

1 0

[nova] virtual PTG and Forum planning
by Balázs Gibizer 07 Sep '20

07 Sep '20

Hi, As you probably know the next virtual PTG will be held between October 26-30. I need to book time slots for Nova [1] so please add your availability to the doodle [2] before 7th of September. I have created an etherpad [3] to collect the PTG topics for the Nova sessions. Feel free to add your topics. Also there will be a Forum between October 19-23 [4]. You can use the PTG etherpad [3] to brainstorm forum topics before the official CFP opens. Cheers, gibi [1] https://ethercalc.openstack.org/7xp2pcbh1ncb [2] https://doodle.com/poll/a5pgqh7bypq8piew [3] https://etherpad.opendev.org/p/nova-wallaby-ptg [4] https://wiki.openstack.org/wiki/Forum/Virtual202

1 1

[ironic] [stable] Bifrost stable/stein is broken (by eventlet?): help needed
by Dmitry Tantsur 07 Sep '20

07 Sep '20

Hi folks, I'm trying to revive the Bifrost stable/stein CI, and after fixing a bunch of issues in https://review.opendev.org/749014 I've hit a wall with what seems an eventlet problem: ironic-inspector fails to start with: Exception AttributeError: "'_SocketDuckForFd' object has no attribute '_closed'" in <bound method _SocketDuckForFd.__del__ of _SocketDuckForFd:16> ignored I've managed to find similar issues, but they should have been resolved in the eventlet version in stein (0.24.1). Any ideas? If we cannot fix it, we'll have to EOL stein and earlier on bifrost. Dmitry -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

3 4

watchdog fed successfully event of 6300esb
by ligang (P) 07 Sep '20

07 Sep '20

Hi folks, I have an question to discuss about the 6300esb watchdog. I think is it possible that qemu can send an event while the watchdog successfully fed by the vm at the first time. Here is the situation: Qemu will send an VIR_DOMAIN_EVENT_ID_WATCHDOG event while watch dog timeout, and if the action of the watchdog in xml of the vm was set to "reset", the vm will be rebooted while timeout. I have an monitor process that register callback function of the VIR_DOMAIN_EVENT_ID_WATCHDOG event, the callback function will send an alarm to my upper layer monitor platform indicate that the vm is fault, and the cluster deployed business on the vm will isolate the vm by the alarm. And after the vm rebooted , the monitor process will receive an reboot event and send it to the platform, the upper layer monitor platform will clear the alarm, and business continue to run on the vm. In most cases ,the watch dog process in vm will feed the watchdog after vm rebooted and all things go back on track. In some other cases,the guestos may failed to start (in my environment vm start failed by io error), but the reboot event will still be received and the alarm will be cleared and the vm is still fault. So the this may not a good idea to clear the alarm by the reboot event. So, I think it will be helpful that the qemu can send an event while the watchdog successfully fed by the vm at the first time. So I can exactly know that the guest os go back on running and the watch dog initialized successfully. Or any other opintion about this situation. Thanks.

1 0

New deployed packstack openstack, Failed create instance due to volume failed to created
by dangerzone ar 07 Sep '20

07 Sep '20

Hi, my clean install openstack queens are not able to create simple instance. Got error *Error: Failed to perform requested operation on instance "test1", the instance has an error status: Please try again later [Error: Build of instance 18c607fd-e919-4022-be7d-d178d7ab410e aborted: Volume 8a3b3e80-1593-46e5-8aa8-a5083bfbb46f did not finish being created even after we waited 9 seconds or 4 attempts. And its status is error.].* Filesystem Size Used Avail Use% Mounted on devtmpfs 3.9G 0 3.9G 0% /dev tmpfs 3.9G 4.0K 3.9G 1% /dev/shm tmpfs 3.9G 25M 3.8G 1% /run tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup /dev/mapper/centos-root 48G 34G 15G 71% / /dev/sda1 1014M 182M 833M 18% /boot /dev/mapper/centos-home 24G 33M 24G 1% /home tmpfs 783M 0 783M 0% /run/user/1000 /dev/loop1 1.9G 6.1M 1.7G 1% /srv/node/swiftloopback tmpfs 783M 0 783M 0% /run/user/0 pvs PV VG Fmt Attr PSize PFree /dev/loop0 cinder-volumes lvm2 a-- <20.60g 1012.00m /dev/sda2 centos lvm2 a-- <79.00g 4.00m vgs VG #PV #LV #SN Attr VSize VFree centos 1 3 0 wz--n- <79.00g 4.00m cinder-volumes 1 1 0 wz--n- <20.60g 1012.00m seems my cinder volume no free space. I install VM with 80G storage but cinder volume no space. I tried extend cinder volume as follows *lvm vgextend "cinder-volumes" /dev/loop3 Physical volume "/dev/loop3" successfully created. Volume group "cinder-volumes" successfully extendedpvscan PV /dev/loop0 VG cinder-volumes lvm2 [<20.60 GiB / 1012.00 MiB free] PV /dev/loop3 VG cinder-volumes lvm2 [<30.00 GiB / <30.00 GiB free] PV /dev/sda2 VG centos lvm2 [<79.00 GiB / 4.00 MiB free] Total: 3 [<129.59 GiB] / in use: 3 [<129.59 GiB] / in no VG: 0 [0 ]* *vgs VG #PV #LV #SN Attr VSize VFree centos 1 3 0 wz--n- <79.00g 4.00m cinder-volumes 2 1 0 wz--n- 50.59g 30.98g* When try to create new instance I keep getting the same volume error... Please help and advise what should I do...Please help...why 80G still not enough for cinder volume. Appreciate help. Thank you

3 5

[kolla] Kolla klub break
by Mark Goddard 07 Sep '20

07 Sep '20

Hi, We agreed in Wednesday's IRC meeting to take a short summer break from the klub. Let's meet again on 10th September. Thanks to everyone who has taken part in these meetings so far, we've had some really great discussions. As always, if anyone has ideas for topics, please add them to the Google doc. Looking forward to some more great sessions in September. https://docs.google.com/document/d/1EwQs2GXF-EvJZamEx9vQAOSDB5tCjsDCJyHQN5_… Thanks, Mark

1 1

[kolla] virtual PTG
by Mark Goddard 07 Sep '20

07 Sep '20

Hi, The next PTG will be virtual again, and held from Monday October 26th to Friday October 30th. For those who have not attended before, the PTG is where we discuss various aspects of the project, and eventually decide on priorities for the next development cycle. I have provisionally booked the same slots as last time: * Monday 26th 13:00 - 17:00 UTC (kolla & kolla ansible) * Tuesday 27th 13:00 - 17:00 UTC (kolla & kolla ansible) * Wednesday 28th 13:00 - 15:00 UTC (kayobe) Please get in touch if you would like to attend but these slots are not suitable for you. In particular, if you are in a time zone where these times are unsociable, we could consider running some sessions during a different 4 hour block. Thanks, Mark

1 0

[neutron][policy] Admin user can do anything without the control of policy.json?
by Zhi CZ Chang 07 Sep '20

07 Sep '20

2 1

[neutron] Bug deputy report (week starting on 2020-08-31)
by Bernard Cafarelli 07 Sep '20

07 Sep '20

Hello neutrinos, back to school week in some parts of the world, and new bug deputy rotation for us! Here is the bug list for week 36 A relatively quiet week on the new numbers count, all urgent bugs were handled Critical * openstack-tox-py36-with-ovsdbapp-master periodic job is failing everyday since 28.08.2020 - https://bugs.launchpad.net/neutron/+bug/1893965/ Patch merged in neutron https://review.opendev.org/#/c/749537/ High * [OVN Octavia Provider] OVN provider fails during listener delete - https://bugs.launchpad.net/neutron/+bug/1894136 Assigned to Brian Medium * [ovn] Limit the number of metadata workers - https://bugs.launchpad.net/neutron/+bug/1893656 Aims to use a separate config option than metadata_workers one, unassigned in neutron (working in tripleo and fixed in charm) * neutron-ovn-db-sync-util: Unhandled error: oslo_config.cfg.NoSuchOptError: no such option keystone_authtoken in group [DEFAULT] - https://bugs.launchpad.net/neutron/+bug/1894048 This was fixed previously in bug 1882020, but is observed again, unassigned * ovn: filtering hash ring nodes is not time zone aware - https://bugs.launchpad.net/neutron/+bug/1894117 Timezone fun, unassigned Wishlist * For neutron-l3-agent, after the execution of the linux command fails, it is not displayed which command failed to execute - https://bugs.launchpad.net/neutron/+bug/1893627 Add failed command to error log Patch merged: https://review.opendev.org/#/c/749076/ Incomplete * Upgrading from train(el7) to ussuri(el8): Packet sequence number wrong - got 2 expected 1 - https://bugs.launchpad.net/neutron/+bug/1894077 Migration error on port forwarding description Undecided * Get domain name for dhcp from neutron db - https://bugs.launchpad.net/neutron/+bug/1893802 This one needs confirmation from subject experts, but as described in comment I think this was attempted and reverted before - and this marked as invalid (or to fix in another way) -- Bernard Cafarelli

1 0

[mq][mariadb] What kind of reasons are considered when choosing the mq/mariadb version
by Brin Zhang(张百林) 07 Sep '20

07 Sep '20

Hi all We are using kolla to build the OpenStack, use kolla-ansible to deploy rabbitmq is v3.7.10, mariadb is v10.1.x when building Rocky release rabbitmq is v3.8.5, mariadb is v10.3.x when building Ussuri release what kind of reasons are considered as rabbitmq version is changed from v3.7.10 to v3.8.5 ? what kind of reasons are considered as mariadb version is changed from v10.1.x to v10.3.x ? If you can provide an explanation, or key explanation, we would be very grateful. Thanks. brinzhang

5 8

[kolla-ansible] prometheus.yml is empty
by Tony Liu 06 Sep '20

06 Sep '20

Hi, I deployed Prometheus with Kolla Ansible (10.1.0.dev36 from PIP). Given the following log, the change has been made on nodes. ============================= TASK [prometheus : Copying over prometheus config file] ************************ skipping: [os-control-2] => (item=/usr/local/share/kolla-ansible/ansible/roles/prometheus/templates/prometheus.yml.j2) skipping: [os-control-1] => (item=/usr/local/share/kolla-ansible/ansible/roles/prometheus/templates/prometheus.yml.j2) skipping: [os-control-3] => (item=/usr/local/share/kolla-ansible/ansible/roles/prometheus/templates/prometheus.yml.j2) skipping: [compute-1] => (item=/usr/local/share/kolla-ansible/ansible/roles/prometheus/templates/prometheus.yml.j2) changed: [monitor-2] => (item=/usr/local/share/kolla-ansible/ansible/roles/prometheus/templates/prometheus.yml.j2) changed: [monitor-1] => (item=/usr/local/share/kolla-ansible/ansible/roles/prometheus/templates/prometheus.yml.j2) changed: [monitor-3] => (item=/usr/local/share/kolla-ansible/ansible/roles/prometheus/templates/prometheus.yml.j2) ============================= But the /etc/kolla/prometheus-server/prometheus.yml on monitor-* nodes is empty. ============================= {} ============================= Anything I am missing here? Thanks! Tony

1 1

[sdk][python-openstackclient] Request the code review about the story: openstack CLI - Create an instance using --image-property filtering not working.
by Myeong Chul Chae 06 Sep '20

06 Sep '20

Hi. I researched the story 'openstack CLI - Create an instance using --image-property filtering not working' and modified the code to solve it. This is the issue that I opened. - Link <https://review.opendev.org/#/c/746405/> And the hyperlink of the story is here <https://storyboard.openstack.org/#!/story/2007860>. In addition, there is a review <https://review.opendev.org/#/c/740455/> posted before my review of the same story, so conflict resolution is necessary. Please check the commit message and history of the two reviews and continue the discussion.

1 0

[kolla-ansible] Difference between deploy and reconfigure?
by Tony Liu 06 Sep '20

06 Sep '20

Hi, What's the difference between deploy and reconfigure? I checked some roles, reconfigure.yml just include_tasks deploy.yml. And I didn't find reconfigure as a condition for any tasks or handlers. I checked kolla-ansible script, the difference is ANSIBLE_SERIAL is specified by reconfigure. I also see comment saying "Serial is not recommended and disabled by default.". Here are some cases. I'd like to know which should be used. #1 Add compute. After updating inventory with the new compute node, "deploy --limit new-compute" and "reconfigure --limit new-compute", do they have the same result? If not, what's the difference? #2 Change configuration. In case configuration is changed in global.yml or inventory, which of "deploy" or "reconfigure" should be executed to update the cluster? Same or any difference? #3 Fix the cluster I was told to rerun playbook to fix some problems, like a container was deleted. "deploy --limit host" or "reconfigure --limit host"? Same or any difference? Thanks! Tony

2 1

[Kolla Ansible] Error in "heat" stage during deployment - Followed Quick Start Guide (ubuntu 18.04.5, virtualbox)
by Lukasluedke＠web.de 05 Sep '20

05 Sep '20

Hi everyone, I am new to openstack and just followed the "Quick Start" guide for kolla-ansible [https://docs.openstack.org/kolla-ansible/ussuri/user/quickstart.html] on ussuri release, but now having some problems during deployment. I used virtualbox to create a new vm with ubuntu 18.04.5 [https://releases.ubuntu.com/18.04/ubuntu-18.04.5-live-server-amd64.iso.torr…] and attached one interface (enp0s3, 10.0.2.15) with Nat-Network and the second interface as for the neutron interface (enp0s8, no ip) an internal network and named it "neutron". Then I followed the guide to create a new virtual python environment (logged in over ssh using "user" user). -- sudo apt-get update sudo apt-get install python3-dev libffi-dev gcc libssl-dev sudo apt-get install python3-venv python3 -m venv $(pwd) source $(pwd)/bin/activate pip install -U pip pip install 'ansible<2.10' -- Everything seems to work as mentioned with no problems so far. (Beside the missing "python-venv" package in the docs) -- pip install kolla-ansible sudo mkdir -p /etc/kolla sudo chown $USER:$USER /etc/kolla cp -r $(pwd)/share/kolla-ansible/etc_examples/kolla/* /etc/kolla cp $(pwd)/share/kolla-ansible/ansible/inventory/* . mkdir /etc/ansible sudo nano /etc/ansible/ansible.cfg ### content from /etc/ansible/ansible.cfg [defaults] host_key_checking=False pipelining=True forks=100 ### -- I then modified the "multinode" file to only deploy on localhost (replaced all control01, etc. with localhost) as I first wanted to try openstack on one node. The ping worked, changed the globals.yml file according to the documentation, then bootstrap and the prechecks worked so far with no issue. -- ansible -i multinode all -m ping kolla-genpwd sudo nano /etc/kolla/globals.yml ### additional content of /etc/kolla/globals.yml kolla_base_distro: "ubuntu" kolla_install_type: "source" network_interface: "enp0s3" neutron_external_interface: "enp0s8" kolla_internal_vip_address: "10.0.2.10" ### kolla-ansible -i ./multinode bootstrap-servers kolla-ansible -i ./multinode prechecks -- But on the deploy stage I am stuck right now: -- kolla-ansible -i ./multinode deploy -- The mentioned error is telling me, that the keystone service is not available. I attached the error log file, because the error is longer. As a side node, now when I try to login using tty, the vm is nearly frozen and only prompts very, very slowly (about 10 minutes for prompt). ssh in putty is also frozen and the connection broke after about 30 minutes. Does anyone know what is wrong or has some hints/insight on how to correctly deploy openstack using kolla-ansible? Best Regards, Lukas Lüdke

2 2

how-to enable s3api
by admin 05 Sep '20

05 Sep '20

hi where is there a normal manual on how to enable s3api support in swift? trying to go through https://docs.openstack.org/swift/latest/middleware.html and as a result keystone gives an error in authorization (trying to connect via (s3curl) Authorization failed. The request you have made requires authentication. from x.x.x.x Unauthorized: The request you have made requires authentication.

2 1

[mq][mariadb] what kind of reasons are considered when choosing the mq/mariadb version
by Sheldon Hu(胡玉鹏) 04 Sep '20

04 Sep '20

Hi all I use kolla to build the openstack version, use kolla-ansible to deploy rabbitmq is v3.7.10, mariadb is v10.1.x when building rocky version rabbitmq is v3.8.5, mariadb is v10.3.x when building ussuri version what kind of reasons are considered as rabbitmq version is changed from v3.7.10 to v3.8.5 ? what kind of reasons are considered as mariadb version is changed from v10.1.x to v10.3.x ? many thanks. Sheldon Hu | 胡玉鹏 CBRD | 云计算与大数据研发部 T: 18663761785 E: <mailto:huyp@inspur.com> huyp(a)inspur.com 浪潮电子信息产业股份有限公司 Inspur Electronic Information Industry Co.,Ltd. 山东省济南市历城区东八区企业公馆A7-1 Building A7-1, Dongbaqu Office Block, Licheng District, Jinan,Shandong Province, PRC 浪潮云海

2 1

[scientific]
by Foundjem Armstrong 04 Sep '20

04 Sep '20

Hello Blair, I am wondering of this email is still functional. Regards, Armstrong

2 1

[TC] Looking for a volunteer to help with community goal selection
by Nate Johnston 04 Sep '20

04 Sep '20

TC members, Since I have not been able to get in contact with Graham, I would like to see if there is anyone who would like to work with me for the selection of community goals for the W series. The commitment would be to identify a few good candidates that have well formulated goal proposals and champions, which usually involves a meeting followed by some reaching out and logistical work. I am hoping to increase the pool of those familiar with the goal selection process, so this is a great place to jump in if you have not done this before. Also, if Graham does have time then we can make sure to include him as well. Two is good but three is better! Please let me know if you are interested. Thanks! Nate

1 0

Mentoring Boston University Students
by Kendall Nelson 04 Sep '20

04 Sep '20

Hello! As you may or may not know, the last two years various community members have mentored students from North Dakota State University for a semester to work on projects in OpenStack. Recently, I learned of a similar program at Boston University and they are still looking for mentors interested for the upcoming semester. Essentially you would have 5 to 7 students for 13 weeks to mentor and work on some feature or effort in your project. The time to apply is running out however as the deadline is Sept 3rd. If you are interested, please let me know ASAP! I am happy to help get the students up to speed with the community and getting their workspaces set up, but the actual work they would do is more up to you :) -Kendall (diablo_rojo)

2 3

[tripleo] centos-binary -> openstack-
by Wesley Hayutin 04 Sep '20

04 Sep '20

Greetings, The container names in master have changed from centos-binary* to openstack*. https://opendev.org/openstack/tripleo-common/src/branch/master/container-im… https://opendev.org/openstack/tripleo-common/commit/90f6de7a7fab15e9161c1f0… If your patches are failing to pull https://registry-1.docker.io/v2/tripleomaster/centos-binary* it's not going to be fixed in a recheck. Check that your patches are rebased and your dependent patches are rebased. Thanks 0/

4 3

[oslo][ffe] request for oslo
by Herve Beraud 04 Sep '20

04 Sep '20

Hey Oslofolk, I request an FFE for these oslo.messaging changes [1]. The goal of these changes is to run rabbitmq heartbeat in python thread by default. Also these changes deprecating this option to prepare future removal and force to always run heartbeat in a python thread whatever the context. Land these changes during the victoria cycle can help us to prime the option removal during the next cycle. Thanks for your time, [1] https://review.opendev.org/#/c/747395/ -- Hervé Beraud Senior Software Engineer Red Hat - Openstack Oslo irc: hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

3 3

[all][TC] OpenStack Client (OSC) vs python-*clients
by Belmiro Moreira 04 Sep '20

04 Sep '20

Hi, during the last PTG the TC discussed the problem of supporting different clients (OpenStack Client - OSC vs python-*clients) [1]. Currently, we don't have feature parity between the OSC and the python-*clients. Different OpenStack projects invest in different clients. This can be a huge problem for users/ops. Depending on the projects deployed in their infrastructures, they need to use different clients for different tasks. It's confusing because of the partial implementation in the OSC. There was also the proposal to enforce new functionality only in the SDK (and optionally the OSC) and not the project’s specific clients to stop increasing the disparity between the two. We would like to understand first the problems and missing pieces that projects are facing to move into OSC and help to overcome them. Let us know. Belmiro, on behalf of the TC [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-June/015418.html

13 20

[nova] Session Recording for Feedback
by Apsey, Christopher 04 Sep '20

04 Sep '20

Nova supports using the websockify -record functionality to capture frames sent over arbitrary console sessions. Currently, the record = path/to/file option in nova.conf saves these sessions on nova-(spice|vnc)proxy endpoints at the specified path in the format of /path/to/file.(session number). These session numbers appear to be incrementally created starting from 1 and don't really have an association with their respective instance from what I can tell. It would be useful to have these files saved using the instance UUID instead. Ignoring that minor problem, playing back these files is a challenge. I've tried using noVNCs vnc_playback.html function from both the master branch and stable/v0.6 with no luck - it looks like the libraries and the utilities haven't been maintained at the same cadence and there are missing functions that the player is expecting. My goal here is to be able to capture the activities of students as they progress through various exercises and then have instructors be able to go over their work with them after the fact if they have questions. Has anyone been able to do this (or something like this) successfully, or are we just better off trying to do this in-band per instance? Chris Apsey GEORGIA CYBER CENTER

2 1

Re: Kuryr openstack
by mahdi abbasi 03 Sep '20

03 Sep '20

Thanks a lot This issue has been resolved On Thu, 3 Sep 2020, 23:28 Radosław Piliszek, <radoslaw.piliszek(a)gmail.com> wrote: > Hi Mahdi, > > have you created any networks (as in Neutron resource) in that project? > You need to have at least one network for Zun to use. > > -yoctozepto > > On Thu, Sep 3, 2020 at 6:50 PM mahdi abbasi <mahdi.abbasi.2013(a)gmail.com> > wrote: > > > > Hi development team, > > > > Do i need a special configuration to use kuryr when using openvswitch? > > I get the following error when starting Docker container in zun: > > > > Unable to create the network.No tenant network is availble for > allocation. > > > > Please help me. > > > > Best regards > > Mahdi >

1 0

Kuryr openstack
by mahdi abbasi 03 Sep '20

03 Sep '20

Hi development team, Do i need a special configuration to use kuryr when using openvswitch? I get the following error when starting Docker container in zun: Unable to create the network.No tenant network is availble for allocation. Please help me. Best regards Mahdi

2 2

[InteropWG] Brainstorming and looking for Ideas from cross project teams
by prakash RAMCHANDRAN 03 Sep '20

03 Sep '20

Hi all, Interop WG i seeking cross platform participation to enable Baremetal and Kubernetes ready Modules in OpenStack & Open Infrastructure. Please add what you would like to get help from Interop Group and how in turn you can help Interop WG goals of enabling Multi-Cloud & Hybrid Cloud Logo Programs to be initiated in CY Q4 2019. We like to discuss and get any new ideas whetted and supported by community. Please add and join us in Forum, PTG and Summit. https://etherpad.opendev.org/p/2020-Wallaby-interop-brainstorming Look forward to your Projects valued participation in Interop Branding efforts.Our next Interop Call is shceduled for Sept 11th 10 AM PDT, please add before that all you can bring to table. For Interop WG Prakash RamchandranIndi Member BoD

1 0

[oslo] Feature Freeze
by Ben Nemec 03 Sep '20

03 Sep '20

This is just a heads up that Oslo is now in feature freeze. Any changes that would require a feature release of an Oslo library will need to request an FFE if they want to make the Victoria release. Please send an email tagged "[oslo][ffe]" to the list to make such a request. I expect we'll branch the Oslo repos in the near future, so some feature development can continue then. However, we generally prefer not to merge invasive changes between feature freeze and release just in case a last-minute bugfix backport is needed. Disentangling a bug fix from a major feature could be a chore. Thanks. -Ben

1 0

[neutron] Drivers meeting - 04.09.2020
by Slawek Kaplonski 03 Sep '20

03 Sep '20

Hi, As there is no agenda for tomorrow's meeting, I will cancel it. Lets focus on reviewing patches related to the RFE scheduled for Victoria-3: https://launchpad.net/neutron/+milestone/victoria-3 See You all online and have a great weekend o/ -- Slawek Kaplonski Principal software engineer Red Hat

1 0

Issue with libvirt unable to kill processes
by Grant Morley 03 Sep '20

03 Sep '20

Hi All, I was wondering if anyone has come across an issue with libvirt seemingly having an issue with instances all of a sudden "locking up" with the following error: Failed to terminate process 2263874 with SIGKILL: Device or resource busy In the nova logs I am seeing: 2020-09-03 09:13:43.208 2659995 INFO nova.compute.manager [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Rebooting instance 2020-09-03 09:15:52.429 2659995 WARNING nova.virt.libvirt.driver [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Failed to soft reboot instance. Trying hard reboot. 2020-09-03 09:16:32.450 2659995 WARNING nova.virt.libvirt.driver [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Error from libvirt during destroy. Code=38 Error=Failed to terminate process 2263874 with SIGKILL: Device or resource busy; attempt 1 of 3: libvirtError: Failed to terminate process 2263874 with SIGKILL: Device or resource busy 2020-09-03 09:17:12.484 2659995 WARNING nova.virt.libvirt.driver [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Error from libvirt during destroy. Code=38 Error=Failed to terminate process 2263874 with SIGKILL: Device or resource busy; attempt 2 of 3: libvirtError: Failed to terminate process 2263874 with SIGKILL: Device or resource busy 2020-09-03 09:17:52.516 2659995 WARNING nova.virt.libvirt.driver [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Error from libvirt during destroy. Code=38 Error=Failed to terminate process 2263874 with SIGKILL: Device or resource busy; attempt 3 of 3: libvirtError: Failed to terminate process 2263874 with SIGKILL: Device or resource busy 2020-09-03 09:17:52.526 2659995 ERROR nova.compute.manager [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Cannot reboot instance: Failed to terminate process 2263874 with SIGKILL: Device or resource busy: libvirtError: Failed to terminate process 2263874 with SIGKILL: Device or resource busy 2020-09-03 09:17:53.026 2659995 INFO nova.compute.manager [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Successfully reverted task state from reboot_started on failure for instance. It seems to be caused when a reboot happens to an instance. If you reset the state and try again, the same error occurs. You also seemingly cannot kill off any libvirt process that is attached to that instance. To me it looks like it could be a kernel issue with libvirt but I could be wrong? Does anyone know of a workaround for this other than maybe restarting a compute host? Many thanks,

2 1

Kuryr openstack
by mahdi abbasi 03 Sep '20

03 Sep '20

Hi development team, Do i need a special configuration to use kuryr when using openvswitch? I get the following error when starting Docker container in zun. Please help me. Best regards Mahdi

1 0

[openstack-ansible] OpenStack Ansible deployment fails due to lxc containers not having network connection
by Oliver Wenz 03 Sep '20

03 Sep '20

I'm trying to deploy OpenStack Ansible. When running the first playbook ```openstack-ansible setup-hosts.yml```, there are errors for all containers during the task ```[openstack_hosts : Remove the blacklisted packages]``` (see below) and the playbook fails. ``` fatal: [infra1_repo_container-1f1565cd]: FAILED! => {"changed": false, "cmd": "apt-get update", "msg": "E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic Release' no longer has a Release file. E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-updates Release' no longer has a Release file. E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-backports Release' no longer has a Release file. E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-security Release' no longer has a Release file.", "rc": 100, "stderr": "E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic Release' no longer has a Release file. E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-updates Release' no longer has a Release file. E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-backports Release' no longer has a Release file. E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-security Release' no longer has a Release file. ", "stderr_lines": ["E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic Release' no longer has a Release file.", "E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-updates Release' no longer has a Release file.", "E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-backports Release' no longer has a Release file.", "E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-security Release' no longer has a Release file."], "stdout": "Ign:1 http://ubuntu.mirror.lrz.de/ubuntu bionic InRelease Ign:2 http://ubuntu.mirror.lrz.de/ubuntu bionic-updates InRelease Ign:3 http://ubuntu.mirror.lrz.de/ubuntu bionic-backports InRelease Ign:4 http://ubuntu.mirror.lrz.de/ubuntu bionic-security InRelease Err:5 http://ubuntu.mirror.lrz.de/ubuntu bionic Release Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable) Err:6 http://ubuntu.mirror.lrz.de/ubuntu bionic-updates Release Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable) Err:7 http://ubuntu.mirror.lrz.de/ubuntu bionic-backports Release Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable) Err:8 http://ubuntu.mirror.lrz.de/ubuntu bionic-security Release Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable) Reading package lists... ", "stdout_lines": ["Ign:1 http://ubuntu.mirror.lrz.de/ubuntu bionic InRelease", "Ign:2 http://ubuntu.mirror.lrz.de/ubuntu bionic-updates InRelease", "Ign:3 http://ubuntu.mirror.lrz.de/ubuntu bionic-backports InRelease", "Ign:4 http://ubuntu.mirror.lrz.de/ubuntu bionic-security InRelease", "Err:5 http://ubuntu.mirror.lrz.de/ubuntu bionic Release", " Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable)", "Err:6 http://ubuntu.mirror.lrz.de/ubuntu bionic-updates Release", " Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable)", "Err:7 http://ubuntu.mirror.lrz.de/ubuntu bionic-backports Release", " Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable)", "Err:8 http://ubuntu.mirror.lrz.de/ubuntu bionic-security Release", " Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable)", "Reading package lists..."]} ``` When I attach to any container and run ```ping 192.168.100.6``` (local DNS), I get the same error (```connect: Network is unreachable```). However, when I specify an interface by running ```ping -I eth1 192.168.100.6``` there is a successful connection. Running ```ip r``` on the infra_cinder container yields: ``` 10.0.3.0/24 dev eth2 proto kernel scope link src 10.0.3.5 192.168.110.0/24 dev eth1 proto kernel scope link src 192.168.110.232 ``` so there seems to be no default route which is why the connection fails (similar for the other infra containers). Shouldn't OSA automatically configure this? I didn't find anything regarding a default route on containers in the Docs. Here's my openstack_user_config.yml: ``` cidr_networks: container: 192.168.110.0/24 tunnel: 192.168.32.0/24 storage: 10.0.3.0/24 used_ips: - "192.168.110.1,192.168.110.2" - "192.168.110.111" - "192.168.110.115" - "192.168.110.117,192.168.110.118" - "192.168.110.131,192.168.110.140" - "192.168.110.201,192.168.110.207" - "192.168.32.1,192.168.32.2" - "192.168.32.201,192.168.32.207" - "10.0.3.1" - "10.0.3.11,10.0.3.14" - "10.0.3.21,10.0.3.24" - "10.0.3.31,10.0.3.42" - "10.0.3.201,10.0.3.207" global_overrides: # The internal and external VIP should be different IPs, however they # do not need to be on separate networks. external_lb_vip_address: 192.168.100.168 internal_lb_vip_address: 192.168.110.201 management_bridge: "br-mgmt" provider_networks: - network: container_bridge: "br-mgmt" container_type: "veth" container_interface: "eth1" ip_from_q: "container" type: "raw" group_binds: - all_containers - hosts is_container_address: true - network: container_bridge: "br-vxlan" container_type: "veth" container_interface: "eth10" ip_from_q: "tunnel" type: "vxlan" range: "1:1000" net_name: "vxlan" group_binds: - neutron_linuxbridge_agent - network: container_bridge: "br-ext1" container_type: "veth" container_interface: "eth12" host_bind_override: "eth12" type: "flat" net_name: "ext_net" group_binds: - neutron_linuxbridge_agent - network: container_bridge: "br-storage" container_type: "veth" container_interface: "eth2" ip_from_q: "storage" type: "raw" group_binds: - glance_api - cinder_api - cinder_volume - nova_compute - swift-proxy ### ### Infrastructure ### # galera, memcache, rabbitmq, utility shared-infra_hosts: infra1: ip: 192.168.110.201 # repository (apt cache, python packages, etc) repo-infra_hosts: infra1: ip: 192.168.110.201 # load balancer haproxy_hosts: infra1: ip: 192.168.110.201 ### ### OpenStack ### os-infra_hosts: infra1: ip: 192.168.110.201 identity_hosts: infra1: ip: 192.168.110.201 network_hosts: infra1: ip: 192.168.110.201 compute_hosts: compute1: ip: 192.168.110.204 compute2: ip: 192.168.110.205 compute3: ip: 192.168.110.206 compute4: ip: 192.168.110.207 storage-infra_hosts: infra1: ip: 192.168.110.201 storage_hosts: lvm-storage1: ip: 192.168.110.202 container_vars: cinder_backends: lvm: volume_backend_name: LVM_iSCSI volume_driver: cinder.volume.drivers.lvm.LVMVolumeDriver volume_group: cinder_volumes iscsi_ip_address: "{{ cinder_storage_address }}" limit_container_types: cinder_volume ``` I also asked this question on the server fault stackexchange: https://serverfault.com/questions/1032573/openstack-ansible-deployment-fail… Kind regards, Oliver

2 1

[API SIG] Ending office hours for the API-SIG
by Michael McCune 03 Sep '20

03 Sep '20

hello all, the API-SIG has held meetings and office hours for several years now. since our migration from a weekly meeting to an office hour we have monitored a continual decrease in attendance for these hours, aside from the core membership. after discussion over the course of the last month or two, we have agreed to end the API-SIG office hours. we will hold office hours this week as our final meeting. if you need to contact the SIG in the future, we are still available in the #openstack-sdks channel on freenode as well as here on the mailing list. in the future we would like to see continued migration of the API-SIG closer to the SDKs group. these two groups will essentially become a singular place to discuss all things related to SDKs and APIs in OpenStack. thank you all for the years of participation and discussion that helped us as a community to define a set of best practices and guidelines for API work in OpenStack. peace o/

1 0

senlin auto scaling question
by Satish Patel 03 Sep '20

03 Sep '20

Folks, I have created very simple cluster using following command openstack cluster create --profile myserver --desired-capacity 2 --min-size 2 --max-size 3 --strict my-asg It spun up 2 vm immediately now because the desired capacity is 2 so I am assuming if any node dies in the cluster it should spin up node to make count 2 right? so i killed one of node with "nove delete <instance-foo-1>" but senlin didn't create node automatically to make desired capacity 2 (In AWS when you kill node in ASG it will create new node so is this senlin different then AWS?)

3 5

[kolla] questions when using external mysql
by sosogh 03 Sep '20

03 Sep '20

Hi list: I want to use kolla-ansible to deploy openstack , but using external mysql. I am following these docs: https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html https://docs.openstack.org/kolla-ansible/latest/reference/databases/externa…. I have some questions: ################ ## Question 1 ## ################ According to the offical doc , if setting it in inventory file(multinode), kolla-ansible -i ./multinode deploy will throw out error: I guest when kolla-ansible running the playbook against myexternalmariadbloadbalancer.com , the """register: find_custom_fluentd_inputs""" in """TASK [common : Find custom fluentd input config files]""" maybe null . ################ ## Question 2 ## ################ According to the offical doc , If the MariaDB username is not root, set database_username in /etc/kolla/globals.yml file: But in kolla-ansible/ansible/roles/xxxxxx/tasks/bootstrap.yml , they use ''' login_user: "{{ database_user }}" ''' ， for example : So at last , I took the following steps: 1. """not""" setting [mariadb] in inventory file(multinode) 2. set "database_user: openstack" for "privillegeduser" PS: My idea is that if using an external ready-to-use mysql (cluster), it is enough to tell kolla-ansible only the address/user/password of the external DB. i.e. setting them in the file /etc/kolla/globals.yml and passwords.yml , no need to add it into inventory file(multinode) Finally , it is successful to deploy openstack via kolla-ansible . So far I have not found any problems. Are the steps what I took good ( enough ) ? Thank you !

2 4

[tacker] Wallaby vPTG
by Yasufumi Ogawa 02 Sep '20

02 Sep '20

Hi team, I booked our slots, 27-29 Oct 6am-8am UTC, for the next vPTG[1] as we agreed in previous irc meeting. I also prepared an etherpad [2], so please add your name and suggestions. [1] https://ethercalc.openstack.org/7xp2pcbh1ncb [2] https://etherpad.opendev.org/p/Tacker-PTG-Wallaby Thanks, Yasufumi

2 2

[ovn][neutron][ussuri] How to configure for overlay in DPDK with OVN ?
by 한승진 02 Sep '20

02 Sep '20

Hi, I’m trying to implement dpdk-ovs with ovn for our openstack environment. I create a bond interface(tunbound) with ovs and tun0 and tun1 are slave interfaces of the bond interface named tunbond. Bridge br-int fail_mode: secure datapath_type: netdev Port tunbond Interface tun0 type: dpdk options: {dpdk-devargs="0000:1a:00.1", n_rxq="2"} Interface tun1 type: dpdk options: {dpdk-devargs="0000:d8:00.1", n_rxq="2"} Port br-int Interface br-int type: internal Now, I need to configure remote-ovn options to the ovs. *I am confused how can I define the IP address of the bonded interface with ovs?* Theses are commands for connect remote ovn services. ovs-vsctl set open . external-ids:ovn-remote=tcp:{controller ip}:6642 ovs-vsctl set open . external-ids:ovn-encap-type=geneve *1* This below command make it me confused. ovs-vsctl set open . external-ids:ovn-encap-ip={local ip} How should I resolve this issue? Regards, John Haan

2 2

[tripleo, ironic] Error: Could not retrieve ... pxelinux.0
by Pete Zaitcev 02 Sep '20

02 Sep '20

Hello: I wanted to give the TripleO a try, so started follow our installation guide for Ussuri, and eventually made it to "openstack undercloud install". It fails with something like this: Aug 28 10:10:53 undercloud puppet-user[48657]: Error: /Stage[main]/Ironic::Pxe/File[/var/lib/ironic/tftpboot/ipxe.efi]: Could not evaluate: Could not retrieve information from environment production source(s) file:/usr/share/ipxe/ipxe-x86_64.efi Aug 27 20:05:42 undercloud puppet-user[37048]: Error: /Stage[main]/Ironic::Pxe/Ironic::Pxe::Tftpboot_file[pxelinux.0]/File[/var/lib/ironic/tftpboot/pxelinux.0]: Could not evaluate: Could not retrieve information from environment production source(s) file:/tftpboot/pxelinux.0 Does anyone have an idea what it wants? I added a couple of packages on the host system that provided the files mentioned in the message, but it made no difference. Ussuri is conteinerized anyway. Since I'm very new to this, I have no clue where to look at all. The nearest task is a wrapper of some kind, so the install-undercloud.log looks like this: 2020-08-28 14:11:31.397 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] TASK [Run container-puppet tasks (generate config) during step 1 with paunch] *** 2020-08-28 14:11:31.397 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] Friday 28 August 2020 14:11:31 -0400 (0:00:00.302) 0:06:28.734 ********* 2020-08-28 14:11:32.223 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] changed: [undercloud] 2020-08-28 14:11:32.325 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] 2020-08-28 14:11:32.326 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] TASK [Wait for container-puppet tasks (generate config) to finish] ************* 2020-08-28 14:11:32.326 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] Friday 28 August 2020 14:11:32 -0400 (0:00:00.928) 0:06:29.663 ********* 2020-08-28 14:11:32.948 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] WAITING FOR COMPLETION: Wait for container-puppet tasks (generate config) to finish (1200 retries left). . . . If anyone could tell roughly what is supposed to be going on here, it would be great. I may be able figure out the rest. Greetings, -- Pete

2 3

Re: [neutron][ovn] OVN Performance
by Apsey, Christopher 02 Sep '20

02 Sep '20

All, Just wanted to loop back here and give an update. For reference, [1] (blue means successful action, red means failed action) is the result we got when booting 5000 instances in rally [2] before the Red Hat OVN devs poked around inside our environment, and [3] is the result after. The differences are obviously pretty significant. I think the biggest change was setting metadata_workers = 2 in neutron_ovn_metadata_agent.ini on the compute nodes per https://bugs.launchpad.net/neutron/+bug/1893656. We have 64C/128T on all compute nodes, so the default neutron calculation of scaling metadata workers based on available cores created 900+ connections to the southbound db at idle; after the control plane got loaded up it just quit around 2500 instances (my guess is it hit the open file limit, although I don’t think increasing it would have made it better for much longer since the number of connections were increasing exponentially). Capping the number of metadata workers decreased open southbound connections by 90%. Even more telling was that rally was able to successfully clean up after itself after we made that change, whereas previously it wasn’t even able to successfully tear down any of the instances that were made, indicating that the control plane was completely toast. Note that the choppiness towards the end of [3] had nothing to do with OVN – our compute nodes had a loadavg approaching 1000 at that point, so they were just starved for cpu cycles. This would have scaled even better with additional compute nodes. The other piece was RAFT. Currently, RDO is shipping with ovs 2.12, but 2.13 has a bunch of RAFT fixes in it that improve stability and knock out some bugs. We were having issues with chassis registration on 2.12, but after using the 2.13 package from cbs, all those issues went away. Big thanks to the great people at Red Hat on the cc line for volunteering their valuable time to take a look. I’m now significantly more comfortable with defaulting to OVN as the backend of choice as the performance delta is now gone. That said, should the community consider dropping linuxbridge as the backend in the official upstream docs and jump straight to OVN rather than ml2/OVS? I think that would increase the test base and help shine light on other issues as time goes on. My org can devote some time to doing this work if the community agrees that it’s the right action to take. Hope that’s helpful! [1] https://ibb.co/GTjZP2y [2] https://pastebin.com/5pEDZ7dY [3] https://ibb.co/pfB9KTV Chris Apsey GEORGIA CYBER CENTER From: Apsey, Christopher Sent: Thursday, August 27, 2020 11:33 AM To: Assaf Muller <amuller(a)redhat.com> Cc: openstack-discuss(a)lists.openstack.org; Lucas Alvares Gomes Martins <lmartins(a)redhat.com>; Jakub Libosvar <jlibosva(a)redhat.com>; Daniel Alvarez Sanchez <dalvarez(a)redhat.com> Subject: RE: [EXTERNAL] Re: [neutron][ovn] OVN Performance Assaf, We can absolutely support engineering poking around in our environment (and possibly an even larger one at my previous employer that was experiencing similar issues during testing). We can take this offline so we don’t spam the mailing list. Just let me know how to proceed, Thanks! Chris Apsey GEORGIA CYBER CENTER From: Assaf Muller <amuller(a)redhat.com<mailto:amuller@redhat.com>> Sent: Thursday, August 27, 2020 11:18 AM To: Apsey, Christopher <CAPSEY(a)augusta.edu<mailto:CAPSEY@augusta.edu>> Cc: openstack-discuss(a)lists.openstack.org<mailto:openstack-discuss@lists.openstack.org>; Lucas Alvares Gomes Martins <lmartins(a)redhat.com<mailto:lmartins@redhat.com>>; Jakub Libosvar <jlibosva(a)redhat.com<mailto:jlibosva@redhat.com>>; Daniel Alvarez Sanchez <dalvarez(a)redhat.com<mailto:dalvarez@redhat.com>> Subject: [EXTERNAL] Re: [neutron][ovn] OVN Performance CAUTION: EXTERNAL SENDER This email originated from an external source. Please exercise caution before opening attachments, clicking links, replying, or providing information to the sender. If you believe it to be fraudulent, contact the AU Cybersecurity Hotline at 72-CYBER (2-9237 / 706-722-9237) or 72CYBER(a)augusta.edu<mailto:72CYBER@augusta.edu> The most efficient way about this is to give one or more of the Engineers working on OpenStack OVN upstream (I've added a few to this thread) temporary access to an environment that can reproduce issues you're seeing, we could then document the issues and work towards solutions. If that's not possible, if you could provide reproducer scripts, or alternatively sharpen the reproduction method, we'll take a look. What you've described is not something that's 'acceptable', OVN should definitely not scale worse than Neutron with the Linux Bridge agent. It's possible that the particular issues you ran in to is something that we've already seen internally at Red Hat, or with our customers, and we're already working on fixes in future versions of OVN - I can't tell you until you elaborate on the details of the issues you're seeing. In any case, the upstream community is committed to improving OVN scale and fixing scale issues as they pop up. Coincidentally, Red Hat scale engineers just published an article [1] about work they've done to scale RH-OSP 16.1 (== OpenStack Train on CentOS 8, with OVN 2.13 and TripleO) to 700 compute nodes. [1] https://www.redhat.com/en/blog/scaling-red-hat-openstack-platform-161-more-…<https://www.redhat.com/en/blog/scaling-red-hat-openstack-platform-161-more-…> On Thu, Aug 27, 2020 at 10:44 AM Apsey, Christopher <CAPSEY(a)augusta.edu<mailto:CAPSEY@augusta.edu>> wrote: > > All, > > > > I know that OVN is going to become the default neutron backend at some point and displace linuxbridge as the default configuration option in the docs, but we have noticed a pretty significant performance disparity between OVN and linuxbridge on identical hardware over the past year or so in a few different environments[1]. I know that example is unscientific, but similar results have been borne out in many different scenarios from what we have observed. There are three main problems from what we see: > > > > 1. OVN does not handle large concurrent requests as well as linuxbridge. Additionally, linuxbridge concurrent capacity grows (not linearly, but grows nonetheless) by adding additional neutron API endpoints and RPC agents. OVN does not really horizontally scale by adding additional API endpoints, from what we have observed. > > 2. OVN gets significantly slower as load on the system grows. We have observed a soft cap of about 2000-2500 instances in a given deployment before ovn-backed neutron stops responding altogether to nova requests (even for booting a single instance). We have observed linuxbridge get to 5000+ instances before it starts to struggle on the same hardware (and we think that linuxbridge can go further with improved provider network design in that particular case). > > 3. Once the southbound database process hits 100% CPU usage on the leader in the ovn cluster, it’s game over (probably causes 1+2) > > > > It's entirely possible that we just don’t understand OVN well enough to tune it [2][3][4], but then the question becomes how do we get that tuning knowledge into the docs so people don’t scratch their heads when their cool new OVN deployment scales 40% as well as their ancient linuxbridge-based one? > > > > If it is ‘known’ that OVN has some scaling challenges, is there a plan to fix it, and what is the best way to contribute to doing so? > > > > We have observed similar results on Ubuntu 18.04/20.04 and CentOS 7/8 on Stein, Train, and Ussuri. > > > > [1] https://pastebin.com/kyyURTJm<https://pastebin.com/kyyURTJm> > > [2] https://github.com/GeorgiaCyber/kinetic/tree/master/formulas/ovsdb<https://github.com/GeorgiaCyber/kinetic/tree/master/formulas/ovsdb> > > [3] https://github.com/GeorgiaCyber/kinetic/tree/master/formulas/neutron<https://github.com/GeorgiaCyber/kinetic/tree/master/formulas/neutron> > > [4] https://github.com/GeorgiaCyber/kinetic/tree/master/formulas/compute<https://github.com/GeorgiaCyber/kinetic/tree/master/formulas/compute> > > > > Chris Apsey > > GEORGIA CYBER CENTER > >

2 1

[manila] Victoria Collab Review next Tuesday (Sep 1st)
by Douglas 02 Sep '20

02 Sep '20

Hi everybody We will have a new edition of our collaborative review next Tuesday, September 1st, where we'll go through the code and review the proposed feature Share Server Migration[1][2]. This meeting is scheduled for two hours, starting at 5:00PM UTC. Meeting notes and videoconference links will be available here[3]. Feel free to attend if you are interested and available. Hoping to see you there, - dviroel [1] https://opendev.org/openstack/manila-specs/src/branch/master/specs/victoria… [2] https://review.opendev.org/#/q/topic:bp/share-server-migration+(status:open) [3] https://etherpad.opendev.org/p/manila-victoria-collab-review

1 1

Graceful stopping of RabbitMQ AMQP notification listener
by Anil Jangam 02 Sep '20

02 Sep '20

Hi, I have coded OpenStack AMQP listener following the example and it is working fine. https://github.com/gibizer/nova-notification-demo/blob/master/ws_forwarder.… The related code snippets of the NotificationHandler class are shown as follows. # Initialize the AMQP listener def init(self, cluster_ip, user, password, port): cfg.CONF() cluster_url = "rabbit://" + user + ":" + password + "@" + cluster_ip + ":" + port + "/" transport = oslo_messaging.get_notification_transport(cfg.CONF, url=cluster_url) targets = [ oslo_messaging.Target(topic='versioned_notifications'), ] endpoints = [self.__endpoint] # Initialize the notification listener try: self.__amqp_listener = oslo_messaging.get_notification_listener(transport, targets, endpoints, executor='threading') except NotImplementedError as err: LOGGER.error("Failed to initialize the notification listener {}".format(err)) return False LOGGER.debug("Initialized the notification listener {}".format(cluster_url)) return True # Arm the compute event listeners def start_amqp_event_listener(self): # Start the notification handler LOGGER.debug("Started the OpenStack notification handler") self.__amqp_listener.start() # Disarm the compute event listeners def stop_amqp_event_listener(self): LOGGER.debug("Stopping the OpenStack notification handler") if self.__amqp_listener is not None: self.__amqp_listener.stop() I am using this interface from a new process handler function, however, when I invoke the stop_amqp_eent_listener() method, my process hangs. It does not terminate naturally. I verified that the self.__amqp_listener.stop() function is not returning. Is there anything missing in this code? Is there any specific consideration when calling the listener from a new process? Can someone provide a clue? # Stop the worker def stop(self): # Stop the AMQP notification handler self.__amqp_handler.stop_amqp_event_listener() LOGGER.debug("Stopped the worker for {}".format(self.__ops_conn_info.cluster_ip)) /anil.

1 1

[Magnum][Kayobe] Magnum Kubernetes clusters failing to be created (bugs?)
by Tony Pearce 02 Sep '20

02 Sep '20

Hi guys, I hope you are all keeping safe and well at the moment. I am trying to launch Kubernetes clusters into Openstack Train which has been deployed via Kayobe (Kayobe as I understand is a wrapper for kolla-ansible). There have been a few strange issues here and I've struggled to isolate them. These issues started recently after a fresh Openstack deployment some months ago (around February 2020) to give some context. This Openstack is not "live" as I've been trying to get to the bottom of the issues: Issue 1. When trying to launch a cluster we get error "Resource Create Failed: Forbidden: Resources.Kube Masters.Resources[0].Resources.Kube-Master: Only Volume-Backed Servers Are Allowed For Flavors With Zero Disk. " Issue 2. After successfully creating a cluster of a smaller node size, the "resize cluster" is failing (however update the cluster is working). Some background on this specific environment: Deployed via Kayobe, with these components: Cinder, Designate, iscsid, Magnum, Multipathd, neutron provider networks The Cinder component integrates with iSCSI SAN storage using the Nimble driver. This is the only storage. In order to prevent Openstack from allocating Compute node local HDD as instance storage, I have all flavours configured with root disk / ephemeral disk / swap disk = "0MB". This then results in all instance data being stored on the backend Cinder storage appliance. I was able to get a cluster deployed by first creating the template as needed, then when launching the cluster Horizon prompts you for items already there in the template such as number of nodes, node flavour and labels etc. I re-supplied all of the info (as to duplicate it) and then tried creating the cluster. After many many times trying over the course of a few weeks to a few months it was successful. I was then able to work around the issue #2 above to get it increased in size. When looking at the logs for issue #2, it looks like some content is missing in the API but I am not certain. I will include a link to the pastebin below [1]. When trying to resize the cluster, Horizon gives error: "Error: Unable to resize given cluster id: 99693dbf-160a-40e0-9ed4-93f3370367ee". I then searched the controller node /var/log directory for this ID and found "horizon.log [:error] [pid 25] Not Found: /api/container_infra/clusters/99693dbf-160a-40e0-9ed4-93f3370367ee/resize". Going to the Horizon menu "update cluster" allows you to increase the number of nodes and then save/apply the config which does indeed resize the cluster. Regarding issue #1, we've been unable to deploy a cluster in a new project and the error is hinting it relates to the flavours having 0MB disk specified, though this error is new and we've been successful previously with deploying clusters (albeit with the hit-and-miss experiences) using the flavour with 0MB disk as described above. Again I searched for the (stack) ID after the failure, in the logs on the controller and I obtained not much more than the error already seen with Horizon [2]. I was able to create new flavours with root disk = 15GB and then successfully deploy a cluster on the next immediate try. Update cluster from 3 nodes to 6 nodes was also immediately successful. However I see the compute nodes "used" disk space increasing after increasing the cluster size which is an issue as the compute node has very limited HDD capacity (32GB SD card). At this point I also checked 1) previously installed cluster using the 0MB disk flavour and 2) new instances using the 0MB disk flavour. I notice that the previous cluster is having host storage allocated but while the new instance is not having host storage allocated. So the cluster create success is using flavour with disk = 0MB while the result is compute HDD storage being consumed. So with the above, please may I clarify on the following? 1. It seems that 0MB disk flavours may not be supported with magnum now? Could the experts confirm? :) Is there another way that I should be configuring this so that compute node disk is not being consumed (because it is slow and has limited capacity). 2. The issue #1 looks like a bug to me, is it known? If not, is this mail enough to get it realised? Pastebin links as mentioned [1] http://paste.openstack.org/show/797316/ [2] http://paste.openstack.org/show/797318/ Many thanks, Regards, Tony Pearce

2 4

[ptg][glance] Wallaby PTG planning etherpad
by Abhishek Kekane 02 Sep '20

02 Sep '20

Hi All, The Wallaby PTG will be held next month from Oct 26 to Oct 30. I have booked some time slots in the ethercalc[0], however these are not fixed, if the booked time does not suit your time, please let me know. I have also created a planning etherpad [1] where you can add your topics for discussion during PTG. [0]https://ethercalc.openstack.org/7xp2pcbh1ncb. [1] https://etherpad.opendev.org/p/Glance-Wallaby-PTG-planning Thank you, Abhishek Kekane

1 0

Issue with heat and magnum
by Alexander Dibbo - UKRI STFC 02 Sep '20

02 Sep '20

Hi, I am having an issue with magnum creating clusters when I have multiple active heat-engine daemons running. I get the following error in the heat engine logs: 2020-08-14 10:36:30.237 598383 INFO heat.engine.resource [req-a2c862eb-370c-4e91-a2c6-dca32c7872ce - - - - -] signal SoftwareDeployment "master_config_deployment" [67ba9ce2-aba5-4c15-a7ea -6b774659a0e2] Stack "kubernetes-test-26-3uzjqqob47fh-kube_masters-mhctjio2b4gh-0-pbhumflm5mn5" [dc66e4d9-0c9b-4b18-a2c6-dd9724fa51a9] : Authentication cannot be scoped to multiple target s. Pick one of: project, domain, trust or unscoped 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource Traceback (most recent call last): 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 2462, in _handle_signal 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource signal_result = self.handle_signal(details) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/heat/software_deployment.py", line 514, in handle_signal 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource timeutils.utcnow().isoformat()) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/rpc/client.py", line 788, in signal_software_deployment 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource version='1.6') 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/rpc/client.py", line 89, in call 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource return client.call(ctxt, method, **kwargs) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 165, in call 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource msg_ctxt = self.serializer.serialize_context(ctxt) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/common/messaging.py", line 46, in serialize_context 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource _context = ctxt.to_dict() 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 185, in to_dict 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource 'roles': self.roles, 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 315, in roles 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource self._load_keystone_data() 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 292, in wrapped_f 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource return self.call(f, *args, **kw) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 358, in call 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource do = self.iter(retry_state=retry_state) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 319, in iter 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource return fut.result() 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/concurrent/futures/_base.py", line 422, in result 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource return self.__get_result() 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 361, in call 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource result = fn(*args, **kwargs) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 306, in _load_keystone_data 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource auth_ref = self.auth_plugin.get_access(self.keystone_session) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 134, in get_access 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource self.auth_ref = self.get_auth_ref(session) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource return self._plugin.get_auth_ref(session, **kwargs) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/v3/base.py", line 144, in get_auth_ref 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource message='Authentication cannot be scoped to multiple' 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource AuthorizationFailure: Authentication cannot be scoped to multiple targets. Pick one of: project, domain, trust or unscoped 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service [req-a2c862eb-370c-4e91-a2c6-dca32c7872ce - - - - -] Unhandled error in asynchronous task: ResourceFailure: AuthorizationFailure: resources.master_config_deployment: Authentication cannot be scoped to multiple targets. Pick one of: project, domain, trust or unscoped 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service Traceback (most recent call last): 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 132, in log_exceptions 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service gt.wait() 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 181, in wait 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service return self._exit_event.wait() 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 132, in wait 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service current.throw(*self._exc) 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 221, in main 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service result = function(*args, **kwargs) 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 123, in _start_with_trace 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service return func(*args, **kwargs) 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 1871, in _resource_signal 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service needs_metadata_updates = rsrc.signal(details, need_check) 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 2500, in signal 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service self._handle_signal(details) 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 2480, in _handle_signal 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service raise failure 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service ResourceFailure: AuthorizationFailure: resources.master_config_deployment: Authentication cannot be scoped to multiple targets. Pi ck one of: project, domain, trust or unscoped 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service Each of the individual heat-engine daemons create magnum clusters correctly when they are the only ones online. Attached are the heat and magnum config files. Any ideas where to look would be appreciated? Regards Alexander Dibbo - Cloud Architect / Cloud Operations Group Leader For STFC Cloud Documentation visit https://stfc-cloud-docs.readthedocs.io<https://stfc-cloud-docs.readthedocs.io/> To raise a support ticket with the cloud team please email cloud-support(a)gridpp.rl.ac.uk<mailto:cloud-support@gridpp.rl.ac.uk> To receive notifications about the service please subscribe to our mailing list at: https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=STFC-CLOUD To receive fast notifications or to discuss usage of the cloud please join our Slack: https://stfc-cloud.slack.com/ This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses. Opinions, conclusions or other information in this message and attachments that are not related directly to UKRI business are solely those of the author and do not represent the views of UKRI.

2 1

[ptg][cyborg] Cyborg Wallaby PTG schedule and resources
by yumeng bao 02 Sep '20

02 Sep '20

Hi team, The next virtual Wallaby PTG will be held next month from Oct 26 to Oct 30. I have booked the same time slots in the ethercalc[0] as our last PTG did. However, these are not fixed, if the booked time does not suit your time, Please don't hesitate to add your availability to the doodle[1]. BTW, PTG Topics are open to collect and you can add to the etherpad[2]. [0]https://ethercalc.openstack.org/7xp2pcbh1ncb. [1]https://doodle.com/poll/mrudbrn87vh48ayq [2]https://etherpad.opendev.org/p/cyborg-wallaby-goals Regards， Yumeng

1 0

[scientific-sig] SIG Meeting: Ironic kexec, CANOPIE-HPC and the PTG
by Stig Telfer 01 Sep '20

01 Sep '20

Hi All - We have a Scientific SIG meeting on IRC channel #openstack-meeting at 2100 UTC (about 45 minutes). Everyone is welcome. Agenda is here: https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_September_1st_20… Today we'd like to talk about supporting the development of kexec support for Ironic deployments. Also some up-coming CFPs and PTG participation. Cheers, Stig

1 0

[nova] providing a local disk to a Nova instance
by Thomas Goirand 01 Sep '20

01 Sep '20

Hi Nova team! tl;dr: we would like to contribute giving instances access to physical block devices directly on the compute hosts. Would this be accepted? Longer version: About 3 or 4 years ago, someone wrote a spec, so we'd be able to provide a local disk of a compute, directly to a VM to use. This was then rejected, because at the time, Cinder had the blockdevice drive, which was more or less achieving the same thing. Unfortunately, because nobody was maintaining the blockdevice driver in Cinder, and because there was no CI that could test it, the driver got removed. We've investigated how we could otherwise implement it, and one solution would be to use Cinder, but then we'd be going through an iSCSI export, which would drastically reduce performances. Another solution would be to manage KVM instances by hand, not touching anything to libvirt and/or OpenVSwitch, but then we would loose the ease of using the Nova API, so we would prefer to avoid this direction. So we (ie: employees in my company) need to ask the Nova team: would you consider a spec to do what was rejected before, since there's now no other good enough alternative? Our current goal is to be able to provide a disk directly to a VM, so that we could build Ceph clusters with an hyper-converged model (ie: storage hosted on the compute nodes). In this model, we wouldn't need live-migration of a VM with an attached physical block device (though the feature could be added on a later stage). Before we start investigating how this can be done, I need to know if this has at least some chances to be accepted or not. If there is, then we'll probably start an experimental patch locally, then write a spec to properly start this project. So please let us know. Cheers, Thomas Goirand (zigo)

4 3

[neutron] Bug deputy report August 24 - 30
by Rodolfo Alonso Hernandez 01 Sep '20

01 Sep '20

Hello: This is the list of bugs in the period from August 24 to August 30. Critical: (none) High: - https://bugs.launchpad.net/neutron/+bug/1892861: [neutron-tempest-plugin] If paramiko SSH client connection fails because of authentication, cannot reconnect (unassigned) - https://bugs.launchpad.net/neutron/+bug/1893314: set neutron quota not take effect (ZhouHeng, https://review.opendev.org/#/c/748578/) Medium: - https://bugs.launchpad.net/neutron/+bug/1892496: 500 on SG deletion: Cannot delete or update a parent row (unassigned) - https://bugs.launchpad.net/neutron/+bug/1888878: Data not present in OVN IDL when requested (fix released) - https://bugs.launchpad.net/neutron/+bug/1892846: AttributeError in l3-agent changing HA router to non-HA (Slawek, https://review.opendev.org/#/c/747922/) Low: - https://bugs.launchpad.net/neutron/+bug/1892741: neutron-sriov-nic-agent cannot be disabled (unassigned) - https://bugs.launchpad.net/neutron/+bug/1893188: [tempest] Randomize subnet CIDR to avoid test clashes (Lajos, https://review.opendev.org/#/c/749041/) Wishlist: - https://bugs.launchpad.net/neutron/+bug/1893625: Implement router gateway IP QoS in OVN backend (zhanghao, https://review.opendev.org/#/c/749012/) Incomplete: - https://bugs.launchpad.net/neutron/+bug/1893015: ping with large package size fails Regards.

3 2

2020 Virtual Summit: Forum Submissions Now Accepted
by Jimmy McArthur 01 Sep '20

01 Sep '20

Hello Everyone! We are now accepting Forum [1] submissions for the 2020 Virtual Open Infrastructure Summit [2]. Please submit your ideas through the Summit CFP tool [3] through September 14th. Don't forget to put your brainstorming etherpad up on the Shanghai Forum page [4]. This is not a classic conference track with speakers and presentations. OSF community members (participants in development teams, operators, working groups, SIGs, and other interested individuals) discuss the topics they want to cover and get alignment on and we welcome your participation. The Forum is your opportunity to help shape the development of future project releases. More information about the Forum [1]. The timeline for submissions is as follows: Aug 31st | Formal topic submission tool opens: https://cfp.openstack.org. Sep 14th | Deadline for proposing Forum topics. Scheduling committee meeting to make draft agenda. Sep 21st | Draft Forum schedule published. Crowd sourced session conflict detection. Forum promotion begins. Sept 28th | Forum schedule final Oct 19th | Forum begins! If you have questions or concerns, please reach out to speakersupport(a)openstack.org. Cheers, Jimmy [1] https://wiki.openstack.org/wiki/Forum [2] https://www.openstack.org/summit/2020/ [3] https://cfp.openstack.org [4]https://wiki.openstack.org/wiki/Forum/Virtual2020

1 1

[tripleo][ussuri][undercloud][tripleo_rabbitmq] do not start after reboot
by Ruslanas Gžibovskis 01 Sep '20

01 Sep '20

Hi all, I think this issue might be known to RHOSP16 [1], but it is same in RDO - ussuri with centos8. if anything is needed, I could try to help/test. my workaround is to run openstack undercloud install --force-stack-update [1] https://access.redhat.com/solutions/5269241 -- Ruslanas Gžibovskis +370 6030 7030

1 0

[horizon] horizon-integration-tests job is broken
by Ivan Kolodyazhny 01 Sep '20

01 Sep '20

Hi team, Please, do not trigger recheck if horizon-integration-tests failed like [1]: ______________ TestDashboardHelp.test_dashboard_help_redirection _______________ 'NoneType' object is not iterable While I'm trying to figure out what is happening there [2], any help with troubleshooting is welcome. [1] https://51bb980dc10c72928109-9873e0e5415ff38d9f1a5cc3b1681b19.ssl.cf1.rackc… [2] https://review.opendev.org/#/c/749011 Regards, Ivan Kolodyazhny, http://blog.e0ne.info/

1 1

[ops] Restructuring OSOPS tools
by Sean McGinnis 31 Aug '20

31 Aug '20

Hello everyone, We recently expanded the scope of the Ops Docs SIG to also include any ops tooling. I think it's now time to move on to the next step of actually getting some of the old tooling in place and organized how we want it. We have several semi-abandoned repos from back when there was more work being done on ops tooling. During the great rebranding, those all were moved under the x/ namespace: https://opendev.org/x/?tab=&sort=recentupdate&q=osops Since these are now owned by an official SIG, we can move this content back under the openstack/ namespace. That should help increase visibility somewhat, and make things look a little more official. It will also allow contributors to tooling to get recognition for contributing to an import part of the OpenStack ecosystem. I do think it's can be a little more difficult to find things spread out over several repos though. For simplicity with finding tooling, as well as watching for reviews and helping with overall maintenance, I would like to move all of these under a common openstack/osops. Under that repo, we can then have a folder structure with tools/logging, tools/monitoring, etc. Then with everything in one place, we can have docs published in one place that helps find everything and easily links between tools. We can also capture some metadata about the tools, and use that to reflect their state in those docs. Please let me know if there are any objects to this plan. Otherwise, I will start cleaning things up and getting it staged in a new repo to be imported as an official repo owned by the SIG. Thanks! Sean

3 2

[TripleO][Ussuri] image prepare, cannot download containers image prepare recently
by Ruslanas Gžibovskis 31 Aug '20

31 Aug '20

Hi all, I have noticed, that recently my undercloud is not able to download images [0]. I have provided newly generated containers-prepare-parameter.yaml and outputs from container image prepare providing --verbose and later beginning of --debug (in the end) [0] were there any changes? As "openstack tripleo container image prepare default --output-env-file containers-prepare-parameter.yaml --local-push-destination" have prepared a bit different file, compared what was previously: NEW # namespace: docker.io/tripleou VS namespace: docker.io/tripleomaster # OLD [0] - http://paste.openstack.org/show/rBCNAQJBEe9y7CKyi9aG/ -- Ruslanas Gžibovskis +370 6030 7030

2 1