- openstack-discuss - lists.openstack.org

[ops][nova][neutron] Proper way to migrate instances between nodes with different ML2 agents types
by Antoine Millet 08 Nov '19

08 Nov '19

Hi here, I'm trying to find a solution to migrate instances between hypervisors of an openstack cluster with nodes running different ML2 agents (OVS and bridges, I'm actually migrating the whole cluster to the latter). The cluster is running Rocky. I enabled both mechanisms in the neutron- server configuration and some nodes are running the neutron- openvswitch-agent and some other the neutron-linuxbridge-agent. My network nodes (running the l3 agent) are currently running the neutron- openvswitch-agent. I also noticed that when nova-compute is starting up, VIF plugins for OVS and Bridges are loaded ("INFO os_vif [-] Loaded VIF plugins: ovs, linux_bridge"). When I start a live migration for an instance running on an hypervisor using the OVS agent to an hypervisor using the bridge agent, it fails because the destination hypervisor try to execute 'ovs-*' commands to bind the VM to its network. I also tried cold migration and just restarting an hypervisor with the bridge agent instead of the OVS one, but it fails similarly when the instances startup. After some research, I discovered that the mechanism used to bind an instance port to a network is stored in the port binding configuration in the database and that the code that executes the 'ovs-*' commands is actually located in the os_vif library that is used by the nova-compute agent. So, I tried to remove the OVS plugin from the os_vif library. Ubuntu ship both plugins in the same package so I just deleted the plugin directory in /usr/lib/python2.7/dist-packages directory (don't judge me please, it's for science ;-)). And... it worked as expected (port bindings are converted to bridge mechanism), at least for the cold migration (hot migration is cancelled without any error message, I need to investigate more). How can I do those migration the proper way? Thank you for any help! Antoine

5 5

[nova][ptg] Virtual PTG
by Eric Fried 08 Nov '19

08 Nov '19

Hello nova contributors and other stakeholders. As you are aware, nova maintainers will be sparser than usual at the ussuri PTG. For that reason, and also because it promotes better inclusion anyway, I'd like us to do the majority of decision making via the mailing list. The PTG is still a useful place to talk through design ideas, but this will give those not attending a voice in the final direction. To that end, I call your attention to the etherpad [1]. As usual, list your topics there. And if your topic is something for which you only need (or wish to start with) in-person discussions (e.g. "I'd like to do $thing but could use some help figuring out $how"), you're done. But if what you're shooting for is discussion leading to some kind of decision, like... - My spec has been stalled because we can't decide among N different approaches; we need to reach a consensus. - My feature is really important; can we please prioritize it for ussuri? ...then in addition to putting your topic on the etherpad, please initiate a (separate) thread on this mailing list, including [nova][ptg] in your subject line. Some of these topics may be resolved before the PTG itself. Others may be discussed in Shanghai. However, even if a consensus is reached in person, expect that decision to be tentative pending closure of the ML thread. Thanks, efried [1] https://etherpad.openstack.org/p/nova-shanghai-ptg

3 3

[nova][ptg]Support volume local cache
by Balázs Gibizer 08 Nov '19

08 Nov '19

spec: https://review.opendev.org/#/c/689070/ TODOs: LiangFang to update the spec based on the discussion in the room[1]: * use traits to driver scheduling. The cache is not sliced per instance so it cannot be a resource class * document the alternative between doing a hard scheduling decision or only implement caching as a best effort optimization for the guest. * document the alternative to do the whole cache management on libvirt (or QEMU) level [1] https://etherpad.openstack.org/p/nova-shanghai-ptg

1 0

[nova][ptg] Community Goals
by Balázs Gibizer 08 Nov '19

08 Nov '19

python3 readiness: Stick with only considering py3 as default when writing code, not caring about refactoring existing code for the purpose of being Py3 pedantic. Write Py3-only code on master, and eventually make the backport py2-compatible on stable branches if needed. dropping paste: Need an operator ML post: is anybody using this? aarents (OVH?) said they are using paste.ini to insert middlewares. So we cannot simply drop paste. improved contributor documentation: we have PTL doc in tree [2]. Work on the CONTRIBUTING.rst [3]. [1] https://etherpad.openstack.org/p/nova-shanghai-ptg [2] https://docs.openstack.org/nova/latest/contributor/ptl-guide.html [3] https://review.opendev.org/#/c/640970/

1 0

[kuryr] Meeting at PTG - follow up
by Michał Dulko 08 Nov '19

08 Nov '19

Hi, Thank you all for a constructive discussion at the Shanghai PTG. First of all I want to clarify that the Neutron improvements in bulk port creation code I talked about were only merged in Stein and are not available in Queens. Sorry about that, my mistake. Below I want to list some highlights from the discussion. * There seemed no pushback on switching to independent release model, so I'll follow up on that soon. * We discussed using RPC instead of API polling to detect when port becomes ACTIVE in Neutron (https://review.opendev.org/#/c/669642/) I commented on the review with a follow up that came from discussion with Neutron team. Apparently there are 2 better ways of doing this. Let's continue that discussion on the review. * We know that we don't support running as a second CNI plugin on Multus because of eth0 being hardcoded as interface name and kuryr- controller handling pods that do not had CNI requests. This is something to solve. * Apparently we all suffer the same problems with Neutron performance when starting a bigger workload on Kuryr-configured K8s cluster (think big Helm chart, multiple operators or simultaneous test suite). This all comes to reducing the number of calls to Neutron. We still don't see a feasible solution to solve this problem, but it seems to be a priority for both Samsung and Red Hat at the moment. Let's see if we'll meet again in Vancouver! Thanks, Michał

1 1

[kolla] Repository setup in non-internet environment.
by Eddie Yen 07 Nov '19

07 Nov '19

Hi, I'm thinking about the deployment in non-internet environment. As we know Kolla has already prepared docker registry and kolla-build to let user can create local registry for deployment. But there's still have two problem about non-internet deployment. 1. Docker-ce repository. 2. Pip repository. (Also having others perhaps.) Does Kolla planning to support non-internet deployment? I would like to do this if possible. Looking forward to hearing from you, Eddie.

4 5

ephemeral dics storage
by Immo Wetzel 07 Nov '19

07 Nov '19

Gday mates, We do run a pike installation and do run into a problem with the ephemeral discs. As written these are on one side the common way for normal VMs to create a disk as long as the vm exists. On the other side these are stored on local discs. But these are usually not the fastest way and therefore the documentation said that in production environments, which we are going to be, these should be stored on shared spaces too. Like a SAN. I found some descriptions how to use ceph for locale storage via rbd backend but we don't use ceph. Each compute node has an FC connection which is used via cinder for the volumes. So what would be the recommendation to use ephemeral disc with FC SAN ? THX a lot Immo

2 1

CPU pinning blues
by Albert Braden 06 Nov '19

06 Nov '19

I'm following this document to setup CPU pinning on Rocky: https://www.redhat.com/en/blog/driving-fast-lane-cpu-pinning-and-numa-topol… I followed all of the steps except for modifying non-pinned flavors and I have one aggregate containing a single NUMA-capable host: root@us01odc-dev1-ctrl1:/var/log/nova# os aggregate list +----+-------+-------------------+ | ID | Name | Availability Zone | +----+-------+-------------------+ | 4 | perf3 | None | +----+-------+-------------------+ root@us01odc-dev1-ctrl1:/var/log/nova# os aggregate show 4 +-------------------+----------------------------+ | Field | Value | +-------------------+----------------------------+ | availability_zone | None | | created_at | 2019-10-30T23:05:41.000000 | | deleted | False | | deleted_at | None | | hosts | [u'us01odc-dev1-hv003'] | | id | 4 | | name | perf3 | | properties | pinned='true' | | updated_at | None | +-------------------+----------------------------+ I have a flavor with the NUMA properties: root@us01odc-dev1-ctrl1:/var/log/nova# os flavor show s1.perf3 +----------------------------+-------------------------------------------------------------------------+ | Field | Value | +----------------------------+-------------------------------------------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | None | | disk | 35 | | id | be3d21c4-7e91-42a2-b832-47f42fdd3907 | | name | s1.perf3 | | os-flavor-access:is_public | True | | properties | aggregate_instance_extra_specs:pinned='true', hw:cpu_policy='dedicated' | | ram | 30720 | | rxtx_factor | 1.0 | | swap | 7168 | | vcpus | 4 | +----------------------------+-------------------------------------------------------------------------+ I create a VM with that flavor: openstack server create --flavor s1.perf3 --image NOT-QSC-CentOS6.10-19P1-v4 --network it-network alberttest4 but it goes to error status, and I see this in the logs: 2019-10-30 16:17:55.590 3248800 INFO nova.virt.hardware [req-d0c2de13-db23-41bd-8da3-34c68ff1d998 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Computed NUMA topology CPU pinning: usable pCPUs: [[4], [5], [6], [7]], vCPUs mapping: [(0, 4), (1, 5), (2, 6), (3, 7)] 2019-10-30 16:17:55.595 3248800 INFO nova.virt.hardware [req-d0c2de13-db23-41bd-8da3-34c68ff1d998 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Computed NUMA topology CPU pinning: usable pCPUs: [[0], [1], [2], [3], [4], [5], [6], [7]], vCPUs mapping: [(0, 0), (1, 1), (2, 2), (3, 3)] 2019-10-30 16:17:55.595 3248800 INFO nova.filters [req-d0c2de13-db23-41bd-8da3-34c68ff1d998 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Filter AggregateInstanceExtraSpecsFilter returned 0 hosts 2019-10-30 16:17:55.596 3248800 INFO nova.filters [req-d0c2de13-db23-41bd-8da3-34c68ff1d998 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Filtering removed all hosts for the request with instance ID '73b1e584-0ce4-478c-a706-c5892609dc3f'. Filter results: ['RetryFilter: (start: 3, end: 3)', 'AvailabilityZoneFilter: (start: 3, end: 3)', 'CoreFilter: (start: 3, end: 2)', 'RamFilter: (start: 2, end: 2)', 'ComputeFilter: (start: 2, end: 2)', 'ComputeCapabilitiesFilter: (start: 2, end: 2)', 'ImagePropertiesFilter: (start: 2, end: 2)', 'ServerGroupAntiAffinityFilter: (start: 2, end: 2)', 'ServerGroupAffinityFilter: (start: 2, end: 2)', 'DifferentHostFilter: (start: 2, end: 2)', 'SameHostFilter: (start: 2, end: 2)', 'NUMATopologyFilter: (start: 2, end: 2)', 'AggregateInstanceExtraSpecsFilter: (start: 2, end: 0)'] It looks like my hypervisor is passing the hw:cpu_policy='dedicated' requirement but it is failing on "pinned=true" The interesting part of the problem is that if I add a second apparently identical hypervisor to the aggregate it starts working. I create s1.perf3 VMs and they land on us01odc-dev1-hv002 and the XML shows that they are correctly pinned. When us01odc-dev1-hv002 is full then they start failing again. What should I be looking for here? What could cause one apparently identical hypervisor to fail AggregateInstanceExtraSpecsFilter while another one passes? In the nova-compute log of the failing hypervisor I see this: 2019-10-31 10:43:01.147 1103 INFO nova.compute.resource_tracker [req-dda65a9c-9d0a-4888-b4cb-0bf4423dd2f3 - - - - -] Instance 4856d505-c220-4873-b881-836b5b75f7bb has allocations against this compute host but is not found in the database. 2019-10-31 10:43:01.148 1103 INFO nova.compute.resource_tracker [req-dda65a9c-9d0a-4888-b4cb-0bf4423dd2f3 - - - - -] Final resource view: name=us01odc-dev1-hv003.internal.synopsys.com phys_ram=128888MB used_ram=38912MB phys_disk=1208GB used_disk=297GB total_vcpus=8 used_vcpus=6 pci_stats=[] Openstack can't find a VM with UUID 4856d505-c220-4873-b881-836b5b75f7bb. There are no VMs on hv003 but I can create a non-pinned VM there and it works. Do I have a "phantom" VM that is consuming resources on hv003? How can I fix that?

2 5

[nova] Operator input on automatic heal behaviors
by Dan Smith 06 Nov '19

06 Nov '19

Hi all, If you're a nova operator, you probably know (and love) our increasing number of "heal $thing" commands in nova-manage. Despite appearances, we do not try to make these inconsistent, duplicative, and confusing. However, in reality, they pretty much are. Further, they require something being broken, an operator noticing, and then manual execution to (hopefully) fix things. While reviewing another such proposed nova-manage command today, I decided to propose a potential solution to make this better in the future. I've got a spec proposed to create new standalone command/service for nova that will consolidate all of these into a very consistent interface, with a daemon mode that can be run in the background to constantly (and slowly) periodically audit these things and heal them when issues are found. If you're an operator and have strong feelings on this topic, please review and opine here: https://review.opendev.org/#/c/693226 Thanks! --Dan

1 0

[neutron][tap-as-a-service] ERSPAN support
by Jean Bernard Beuque (jbeuque) 06 Nov '19

06 Nov '19

Hello, I'd like to add ERSPAN support to the Tap-as-a-Service project. I've currently implemented a prototype that can be used with networking-vpp: https://opendev.org/x/networking-vpp The modified version of tap as a service is available here (The API has been extended to support ERSPAN): https://github.com/jbeuque/tap-as-a-service I don't know who maintains the Taas project. But if you think adding this functionality could be useful, please contact me. (Please take the modified version of Taas as a proposal to be discussed). Regards, Jean-Bernard Beuque

2 2

[kuryr] Kuryr team at the PTG
by Michal Dulko 06 Nov '19

06 Nov '19

Hi, I had not reserved Kuryr space on the PTG as we weren't expecting many Kuryr team members here, but turns out there's some representation. We'll meet on Thursday at 2 PM Shanghai time to discuss anything related to Kuryr. We want meet in the Blue Room (where the tables are) and will try to find some space to run the discussion. Today you can find me at the K8s SIG table. Feel free to join! Thanks, Michał

3 4

[cinder] ussuri PTG schedule
by Brian Rosmaita 06 Nov '19

06 Nov '19

The Ussuri PTG schedule is live: https://etherpad.openstack.org/p/shanghai-ptg-cinder Please check the schedule and let me know right away if your session causes a conflict for you. Except for the few fixed-time topics, we will follow the cinder tradition of dynamic scheduling, giving each topic exactly as much time as it needs and adjusting as we go. cheers, brian

2 1

[nova][ptg] Team dinner
by Stephen Finucane 06 Nov '19

06 Nov '19

Hi all, Thanks to Alex Xu, we have organized a table for dinner this evening, Wed 6th November at 7pm. Xibo restaurant (Xinjiang food) 3/F 83 Changshu Rd, Jingan Qu, Shanghai, China (near "Changshu Road" or "Jing An Temple" subway stations) 中國上海市静安区常熟路83号 The address is on Google Maps - hopefully it's accurate :) Anyone working on nova is welcome, though I'd ask that you'd note your attendance on the PTG etherpad [1]. Looking forward to seeing everyone, Stephen (stephenfin) [1] https://etherpad.openstack.org/p/nova-shanghai-ptg

2 1

[barbican] PTG Schedule
by Douglas Mendizábal 06 Nov '19

06 Nov '19

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hello Barbicaneers! I hope everyone made it to Shanghai safely. Our PTG session will be on Wednesday Nov 6 from 10:30am - 4:30p at the Kilo table. We've set up an etherpad to collect topics to talk about. Please feel free to add any topics you're interested in: https://etherpad.openstack.org/p/barbican-ussuri-ptg Additionally we've reserved a spot for a Team Photo on Thursday at 11am. Hope to see y'all soon! Cheers, - - Douglas Mendizabal -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEwcapj5oGTj2zd3XogB6WFOq/OrcFAl2/iBEACgkQgB6WFOq/ OrfFew/+IjhZe+qRCi/4EmaVEDf7QxJyZDIVUlLsPWHmF98wCdj+GsbzoWUuFfHM sJCpfpVAUjxrIFOEo5uF9WiZhU36G9pgoLd1Y8Kb0/QRIQEQQcKGnlYhCn+jQjbW J2tlDrkU0GBwEBzDt91gM5JCncviY8yT6nhlr/SSLqvZRQnPewerJNyJbYsVh6N2 moXQzfeRjg1SGqR0KVUcDVPe/pE+at8A5ARFCxDiJaOIUTP0qcfKtDXh714bevyi Sw2qgDZHbLHa1nEv3umuYGcrGpKz8Uuj5ju+7oGpPh4hX4pfPxbVDSzu8srfzTui ggvcxFrpZQvdff3Lec1eclxnB+c9Z1tBKYF7pPUVtN3NPfCATkVCSQACYORPZLdh GAnyxiiUXRwzIfOo0b6koa2pRi7ZWoz0DjVzpnl+D7qztUzyiguaj3KDnuTvlfQl iMQev1QHD6fAVvByHgDRj4dyUqUi2+V/DtNZ9w29AX7C+U/afSbNGvygc8yNCtHF vbkw68aPpj5zeB0OTjPQ6N5vsUc6bSXYGnECuGw24untnutvPKR+W9g9VQEUyN1h vhvn0IPHZ9QyBJ0ctpdfA6O9PNsjY/DQNyDeiNGljTIpBjepUmqMTXvycsn8VN/E yY0OL2QFGPhcsK7Q/yeUCzMm1sken2zMg8Bdxt10qbj4GsCMtyQ= =fdMR -----END PGP SIGNATURE-----

2 1

[Neutron] cancel the L3 meeting today
by LIU Yulong 06 Nov '19

06 Nov '19

Hi all, Due to the Summit and PTG next week, I will cancel the Neutron L3 meeting today. So see you there, and travel safe. Thanks. LIU Yulong

1 1

[cinder][osc][docs] openstackclient docs for cinder v2 vs v3
by Eric Fried 05 Nov '19

05 Nov '19

Howdy cinderinos. I've been on a mission to get all of the python-openstackclient command [1] and plugin [2] docs autogenerated rather than hardcoded [3] so you don't have to remember to update two places when you add/change a subcommand option. I'm almost done -- cinder is the last one -- but I want to confirm some odd observations before I dig in. - All of the v3 subcommands are implemented by code in the openstackclient.volume.v2 package. Where there's overlap, the command classes are identical from v2 to v3. However, it appears as though the v2 commands are a *superset* of the v3 commands. Specifically, the following appear in v2 but not v3 [4]: volume_backup_record_export volume_backup_record_import volume_backend_capability_show volume_backend_pool_list volume_host_failover Observations: * v3 has no other 'volume backup record' subcommands, but otherwise has the same 'volume backup' subcommands as v2. * v3 has no 'volume backend' subcommands. * v2 has both 'volume host failover' and 'volume host set', but v3 has only the latter. * It seems suspicious that the "missing" v3 commands comprise a contiguous block under the v2 entry point. So before I go creating a mess of v2-only and v2+v3 documents, I wanted to confirm that the above was actually intentional. - The existing hardcoded documents mention v1 and/or v2, but don't mention v3 at all (e.g. [5]). I want to confirm that it's okay for me to add mention of v3 where appropriate. Thanks, efried [1] https://docs.openstack.org/python-openstackclient/latest/cli/command-list.h… [2] https://docs.openstack.org/python-openstackclient/latest/cli/plugin-command… [3] https://review.opendev.org/#/q/topic:generate-docs+(status:open+OR+status:m… [4] https://opendev.org/openstack/python-openstackclient/src/tag/4.0.0/setup.cf… [5] https://docs.openstack.org/python-openstackclient/train/cli/command-objects…

2 1

Installation DOC for Three node
by Umesh Mishra 05 Nov '19

05 Nov '19

Dear Sir, This is inform you that, We want to build the openstack in our data center so request you please help to provide me doc so that we can build asap . Please help. Best Regards, Umesh Mishra Manager-IT&Devops | Click2Cloud Inc. | www.click2cloud.net<http://www.click2cloud.net/> Email: umesh.mishra(a)click2cloud.net<mailto:umesh.mishra@click2cloud.net> | Mobile: +91 7738599311

3 3

[neutron] PTG - remote access and reminders
by Slawek Kaplonski 05 Nov '19

05 Nov '19

Hi, Tomorrow (Wednesday) we are starting Neutron PTG session. Agenda is available at [1]. If You are not in Shanghai but would maybe try to participate remotely in the sessions, please reach out to me directly through email or IRC. I will try to provide some access and stream from the session. Also, please remember that on Wednesday morning we have onboarding session for new contributors. So if You are interested in contributing to Neutron, feel free to reach out to us in *room 431* - we are starting at 9:00 am :) [1] https://etherpad.openstack.org/p/Shanghai-Neutron-Planning -- Slawek Kaplonski Senior software engineer Red Hat

1 0

[tacker] Ussuri PTG
by Dharmendra Kushwaha 05 Nov '19

05 Nov '19

Tacker Folks, As our scheduled sessions will be finished by 12:30 pm, so lets meet after lunch. I will be available in PTG area full day. https://etherpad.openstack.org/p/Tacker-PTG-Ussuri Thanks & Regards Dharmendra Kushwaha ________________________________ The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. It shall not attach any liability on the originator or NECTI or its affiliates. Any views or opinions presented in this email are solely those of the author and may not necessarily reflect the opinions of NECTI or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately.

1 0

[neutron][QoS] QoS meeting cancelled November 5
by Rodolfo Alonso 05 Nov '19

05 Nov '19

Hello Neutrinos: Due to the summit this week, the Neutron QoS meeting will be cancelled. Next meeting will be November 19. Regards.

1 0

[qa] QA Office hour canceled for this week
by Ghanshyam Mann 05 Nov '19

05 Nov '19

Hello Everyone, As we are in PTG, I will cancel the QA office hour for this week and will continue the same on 14th Nov week. -gmann

1 0

[dev][ops][ptg][keystone] Join the keystone onboarding session!
by Colleen Murphy 05 Nov '19

05 Nov '19

Hello Stackers, If you're a developer, technical writer, operator, or user and interested in getting involved in the keystone project, stop by the keystone onboarding session in Shanghai next week! We will be at the Kilo table in the Blue Room on Wednesday from 9 to 10:30. The format will be open ended, so come with all your questions about how you can participate on the keystone team. Can't make it to the session? Take a look at our contributing guide[1] and feel free to get in touch with me directly. Colleen Murphy / cmurphy (keystone PTL) [1] https://docs.openstack.org/keystone/latest/contributor/how-can-i-help.html

1 1

[openstack-dev][kuryr] Unable to create pod
by VeeraReddy 05 Nov '19

05 Nov '19

HI,Pod is not creating . Status of pod is always "ContainerCreating" I installed openstack & kubernetes using below linkhttps://docs.openstack.org/kuryr-kubernetes/latest/installation/devstac… devstack(a)kuryr-daemon.service log :http://paste.openstack.org/show/785758/ Regards, Veera.

4 9

[glance] Shanghai Project Update
by Abhishek Kekane 05 Nov '19

05 Nov '19

Hi All, We had a very good project update session in Shanghai OpenInfra summit, where we have covered what we have done in Train cycle and what our priorities are in upcoming Ussuri cycle. Attaching the project update PDF file here for your reference. Thanks & Best Regards, Abhishek Kekane

1 0

[infra][qa] multiline tracebacks not being indexed anymore
by Matt Riedemann 05 Nov '19

05 Nov '19

We used to be able to query for things like this: message:"in reserve_block_device_name" AND message:"MessagingTimeout" AND tags:"screen-n-api.txt" to fingerprint a traceback in logstash like this [1] but that no longer works. The multiline logstash filter is at [2] but doesn't seem to be getting applied anymore. I asked about this in -infra today and fungi said: "(4:44:00 PM) fungi: mriedem: i suspect that coincided with switching away from osla, we may need some means of parsing tracebacks out of logs in the indexer" I don't know what that means (what's osla? is [2] no longer used?) but if someone could point me at some things to look at I could see if I can generate a fix. [1] https://zuul.opendev.org/t/openstack/build/ef0196fe84804b44ac106d011c8c29ea… [2] https://opendev.org/openstack/logstash-filters/src/branch/master/filters/op… -- Thanks, Matt

2 1