- openstack-discuss - lists.openstack.org

[Multi-Arch SIG] summary and actions from last meeting
by Rico Lin 09 Jan '20

09 Jan '20

Hi all Thanks to all who sign up and help to form Multi-Arch SIG We hosted our two initial meetings this week [1] and both successful. And for who cares about multi-arch (for example like ARM support in community), feel free to join our future meetings [2]. Here are some actions from this week's meetings: * Create StoryBoard for Multi-Arch SIG (ricolin) * Build multi-arch SIG repo (ricolin) * Collect update ppc64le actions and resources in community if see any (mrda) * Help with documentations once Multi-Arch SIG repo is ready (jeremyfreudberg) * Update governance-sigs to make a more clear description for Multi-Arch SIG (jeremyfreudberg) There are two docs. ideas `oh, on arm64 you need to do XYZ in other way - here is how and why` and `use cases, who, and issues they had`. Both of them seem like great docs. to start with. So that I assume will be something this SIG will try to work on as one of our first step goals. We need more people to help and more resource in CI as well. There's a lot of WIP resources and CI jobs in our community and all of them are collected in our etherpad [3] now. Feel free to update that etherpad and also sign you or your organization's name on it. Please join us:) Once we have our StoryBoard ready, we should be able to create tasks on it so everyone can create and track tasks like CI job, documentations, etc. last but not least, we are looking for people to nominate or volunteer for chair roles (which can be multiple). There are some really experienced community members been nominated and I will check with them if willing to take the role. On the other hand, I'm volunteering to apply for one of SIG chair seats and help to build this SIG, but happy to give it to others if we can have more people sign up for that role:) So let me know if you're interested. [1] http://eavesdrop.openstack.org/meetings/multi_arch/2020/ [2] http://eavesdrop.openstack.org/#Multi-Arch_SIG_Meeting [3] https://etherpad.openstack.org/p/Multi-arch -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

1 0

[cinder] ussuri virtual mid-cycle poll
by Brian Rosmaita 08 Jan '20

08 Jan '20

As determined at the virtual PTG (which seems like it happened only a few weeks ago), we'll be doing a two-phase virtual mid-cycle, meeting for two hours the week of 20 January (before the spec freeze) and again around the week of 16 March (the Cinder new feature status checkpoint). There's a poll up to determine a suitable time for the first virtual mid-cycle meeting: https://doodle.com/poll/n3tmq8ep43dyi7tv Please fill out the poll as soon as you can. If all the times are horrible for you, please suggest an alternative in a comment on the poll. The poll will close at 23:59 UTC on Saturday 11 January. (I know it's soon, but that way we'll have time to make adjustments if necessary.) cheers, brian

1 0

[all][infra] Compressed job log artifacts not loading in browser
by Clark Boylan 08 Jan '20

08 Jan '20

Over the holidays the infra team noticed that some of our release artifacts were in the wrong file format. On further investigation we discovered the reason for this was some swift implementations were inflating compressed tarballs when retrieved by clients not setting accept-encoding: gzip. We then ended up with tar files when we expected .tar.gz format files. This behavior seems to be controlled by the content-encoding we set on object upload. If we tell swift that the object is a gzip'd file on upload then the webservers helpfully inflate them when a client retrieves them. In order to fix our release artifacts we have updated our swift upload tooling to stop setting content-type on gzip files. This forces the swift implementation to return the files in the same format they are uploaded when retrieved. A side effect of this is that any gzip'd files (like testr_results.html.gz) are no longer automatically decompressed for you when you retrieve them. We have fixes up to handle common occurrences of this at https://review.opendev.org/#/c/701578/ and https://review.opendev.org/701282 (note the second case is already handled on OpenDev's Zuul). If you run into other files you expect to be browesable but get compressed instead, the fix is to stop compressing the files explicitly in the job. We will still upload files in compressed form to swift for efficiency but we should operate on these files as if they were uncompressed. Then for files that should be compressed (like .tar.gz files) we can pass them through as is avoiding any format change problems. Clark

2 1

[cliff][docs][requirements] new cliff versions causes docs to fail to build
by Matthew Thode 08 Jan '20

08 Jan '20

Looks like some things changed in the new version that we depended upon and are now causing failures. Exception occurred: File "/home/zuul/src/opendev.org/openstack/python-openstackclient/.tox/docs/lib/python3.6/site-packages/cliff/sphinxext.py", line 245, in _load_app if not issubclass(cliff_app_class, app.App): TypeError: issubclass() arg 1 must be a class -- Matthew Thode

4 6

[Manila] First meeting of 2020
by Tom Barron 08 Jan '20

08 Jan '20

Hey Zorillas! Just a reminder of our first meeting in 2020, 9 January at 1500 UTC on Freenode #openstack-meeting-alt. Feel free to update the agenda [1]. -- Tom [1] https://wiki.openstack.org/wiki/Manila/Meetings#Next_meeting

1 0

Compute node NIC bonding for increased instance throughput
by shubjero 08 Jan '20

08 Jan '20

Good day, I have a question for the OpenStack community, hopefully someone can help me out here. Goal ------------------ Provision an NFS instance capable of providing 20Gbps of network throughput to be used by multiple other instances within the same project/network. Background ------------------ We run an OpenStack Stein cluster on Ubuntu 18.04. Our Neutron architecture is using openvswitch and GRE. Our compute nodes have two 10G NIC's and are configured in a layer3+4 LACP to the Top of Rack switch. Observations ------------------ Successfully see 20Gbps of traffic balanced across both slaves in the bond when performing iperf3 tests at the *baremetal/os/ubuntu* layer with two other compute nodes as iperf3 clients. Problem ------------------ We are unable to achieve 20Gbps at the instance level. We have tried multiple iperf3 connections from multiple other instances on different compute nodes and we are only able to reach 10Gbps and notice that traffic is not utilizing both slaves in the bond. One slave gets all of the traffic while the other slave sits basically idle. I have some configuration output here: http://paste.openstack.org/show/QdQq76q6VI1XN5tLW0xH/ Any help would be appreciated! Jared Baker Cloud Architect, Ontario Institute for Cancer Research

2 1

Re: OIDC/OAuth2 token introspection in Keystone
by Nikolla, Kristi 08 Jan '20

08 Jan '20

There is an patch to improve the documentation for using the CLI with OIDC, but it hasn't merged yet. See here https://review.opendev.org/#/c/693838 Keystoneauth has plugins in place for authenticating with the OIDC IdP in multiple ways, including using an access token, see here https://github.com/openstack/keystoneauth/blob/master/keystoneauth1/identit… Best, Kristi On Jan 8, 2020, at 10:31 AM, mcarpene <m.carpen(a)cineca.it<mailto:m.carpen@cineca.it>> wrote: Many thanks Nikolla, I was able to federate using OIDC IdP via the dashboard. I meant the problem is authenticating via CLI providing a OIDC token via command line, but maybe you already answered to my request. BR, Michele On 08/01/20 16:28, wrote: Hi Michele, We just approved a feature request for that [0], however it was merged to backlog, meaning no specific timeline for it being implemented yet. With the current implementation, you can use OAuth 2.0 Access Tokens with Keystone, however the token introspection endpoint will be used, therefore only the claims contained in the access token will be returned. I am assuming your question is with regards to the userinfo endpoint and OIDC claims, which we do not currently support. [0]. https://review.opendev.org/#/c/373983/ On Jan 8, 2020, at 8:01 AM, mcarpene <m.carpen(a)cineca.it<mailto:m.carpen@cineca.it>> wrote: Hi all, my question is: could OS Keystone support OIDC/OAuth2 token introspection/validation. I mean for example executing a swift command via CLI adding a OIDC token bearer as a parameter to the swift command. In this case Keystone should validate the OIDC token towards and external IdP (using introspection endpoint/protocol for oidc). Is this currently supported, or eventually would be done in the near future? thanks Michele -- Michele Carpené SuperComputing Applications and Innovation Department CINECA - via Magnanelli, 6/3, 40033 Casalecchio di Reno (Bologna) - ITALY Tel: +39 051 6171730 Fax: +39 051 6132198 Skype: mcarpene http://www.hpc.cineca.it/ -- Michele Carpené SuperComputing Applications and Innovation Department CINECA - via Magnanelli, 6/3, 40033 Casalecchio di Reno (Bologna) - ITALY Tel: +39 051 6171730 Fax: +39 051 6132198 Skype: mcarpene http://www.hpc.cineca.it/

1 0

OIDC/OAuth2 token introspection in Keystone
by mcarpene 08 Jan '20

08 Jan '20

Hi all, my question is: could OS Keystone support OIDC/OAuth2 token introspection/validation. I mean for example executing a swift command via CLI adding a OIDC token bearer as a parameter to the swift command. In this case Keystone should validate the OIDC token towards and external IdP (using introspection endpoint/protocol for oidc). Is this currently supported, or eventually would be done in the near future? thanks Michele -- Michele Carpené SuperComputing Applications and Innovation Department CINECA - via Magnanelli, 6/3, 40033 Casalecchio di Reno (Bologna) - ITALY Tel: +39 051 6171730 Fax: +39 051 6132198 Skype: mcarpene http://www.hpc.cineca.it/

2 1

[charms][watcher] OpenStack Watcher Charm
by Stamatis Katsaounis 08 Jan '20

08 Jan '20

Hi all, Purpose of this email is to let you know that we released an unofficial charm of OpenStack Watcher [1]. This charm gave us the opportunity to deploy OpenStack Watcher to our charmed OpenStack deployment. After seeing value in it, we decided to publish it through GRNET GitHub Organization account for several reasons. First of all, we would love to get feedback on it as it is our first try on creating an OpenStack reactive charm. Secondly, we would be glad to see other OpenStack operators deploy Watcher and share with us knowledge on the project and possible use cases. Finally, it would be ideal to come up with an official OpenStack Watcher charm repository under charmers umbrella. By doing this, another OpenStack project is going to be available not only for Train version but for any future version of OpenStack. Most important, the CI tests are going to ensure that the code is not broken and persuade other operators to use it. Before closing my email, I would like to give some insight on the architecture of the code base and the deployment process. To begin with, charm-watcher is based on other reactive OpenStack charms. During its deployment Barbican, Designate, Octavia and other charms' code bases were counseled. Furthermore, the structure is the same as any official OpenStack charm, of course without functional tests, which is something we cannot provide. Speaking about the deployment process, apart from having a basic charmed OpenStack deployment, operator has to change two tiny configuration options on Nova cloud controller and Cinder. As explained in the Watcher configuration guide, special care has to be done with Oslo notifications for Nova and Cinder [2]. In order to achieve that in charmed OpenStack some issues were met and solved with the following patches [3], [4], [5], [6]. With these patches, operator can set the extra Oslo configuration and this is the only extra configuration needs to take place. Finally, with [7] Keystone charm can accept a relation with Watcher charm instead of ignoring it. To be able to deploy GRNET Watcher charm on Train, patches [3], [4], [5] and [7] have to be back-ported to stable/19.10 branch but that will require the approval of charmers team. Please let me know if such an option is available and in that case I am going to open the relevant patches. Furthermore, if you think that it could be a good option to create a spec and then introduce an official Watcher charm, I would love to help on that. I wish all a happy new year and I am looking forward to your response and possible feedback. PS. If we could have an Ubuntu package for watcher-dashboard [8] like octavia-dashboard [9] we would release a charm for it as well. Best regards, Stamatis Katsaounis [1] https://github.com/grnet/charm-watcher [2] https://docs.openstack.org/watcher/latest/configuration/configuring.html#co… [3] https://review.opendev.org/#/c/699079/ [4] https://review.opendev.org/#/c/699081/ [5] https://review.opendev.org/#/c/699657/ [6] https://github.com/juju/charm-helpers/pull/405 [7] https://review.opendev.org/#/c/699082/ [8] https://github.com/openstack/watcher-dashboard [9] https://launchpad.net/ubuntu/+source/octavia-dashboard -- [cid:part10.101270FA.EDBB3A4A@admin.grnet.gr] Stamatis Katsaounis DevOps Engineer t :(+30) 210 7471130 (ext. 483) f : + 30 210 7474490 GRNET | Networking Research and Education www.grnet.gr<http://www.grnet.gr> | 7, Kifisias Av., 115 23, Athens [Facebook icon]<https://www.facebook.com/grnet.gr/> [Twitter icon] <https://twitter.com/grnet_gr> [Youtube icon] <https://www.youtube.com/user/EDETvideos> [LinkedIn icon] <https://www.linkedin.com/company/55267/> [Instagram icon] <https://www.instagram.com/grnet.gr/>

1 0

[neutron][ironic] dynamic routing protocol status for routed network
by 양유석 08 Jan '20

08 Jan '20

Hi, For ironic flat provider network, I found it's hard to scale manually (for many racks). So I'm trying to use routed network for the purpose. One blur thing for routed network is how we can handle segment connectivity. >From reference (https://docs.openstack.org/neutron/latest/admin/config-routed-networks.html) there is future work about dynamic routing protocol though, I could not find any hints for the functionality. How do you guys using routed network handle the segments' connectivity? Is there any project to speak subnet info via BGP? Thanks in advance.

1 0

VM boot volume disappears from compute's multipath Daemon when NetApp controller is placed offline
by Ahmed ZAKY 07 Jan '20

07 Jan '20

I have a setup where each VM gets assigned two vDisks, one encrypted boot volume and another storage volume. Storage used is NetApp (tripleo-netapp). With two controllers on NetApp side working in active/ active mode. My test case goes as follows: - I stop one of the active controllers. - I stop one of my VMs using OpenStack server stop - I then start my VM one more time using OpenStack server start. - VM fails to start. Here're my findings, hope someone would help if they can explain me the behaviour seen below: My VM: vel1bgw01-MCM2, it is running on compute overcloud-sriovperformancecompute-3.localdomain [root@overcloud-controller-0 (vel1asbc01) cbis-admin]# openstack server show vel1bgw01-MCM2 +--------------------------------------+------------------------------------------------------------------------------------------------------------+ | Field | Value | +--------------------------------------+------------------------------------------------------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | zone1 | | OS-EXT-SRV-ATTR:host | overcloud-sriovperformancecompute-3.localdomain | | OS-EXT-SRV-ATTR:hypervisor_hostname | overcloud-sriovperformancecompute-3.localdomain | | OS-EXT-SRV-ATTR:instance_name | instance-00000 <+4400000>e93 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-12-18T15:49:37.000000 <+4437000000> | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | SBC01_MGW01_TIPC=192.168.48.22 <+441921684822>; SBC01_MGW01_DATAPATH_MATE=192.168.16.11 <+441921681611>; SBC01_MGW01_DATAPATH=192.168.32.8 <+44192168328> | | config_drive | True | | created | 2019-12-18T15:49:16Z | | flavor | SBC_MCM (asbc_mcm) | | hostId | 7886 <+447886>df0f7a3d4e131304 <+44131304>a8eb860e6a704c5fda2a7ed751b544ff2bf5 | | id | 5c70a984-89 <+4498489>a9-44ce-876d-9e2e568eb819 | | image | | | key_name | CBAM-b5fd59a066e8450 <+448450>ca9f104a69da5a043-Keypair | | name | vel1bgw01-MCM2 | | os-extended-volumes:volumes_attached | [{u'id': u'717e5744-4786-42 <+445744478642>dc-9e3e-3c5e6994 <+446994>c482'}, {u'id': u'd6cf0cf9-36d1-4b 62-86 <+446286>b4-faa4a6642166 <+446642166>'}] | | progress | 0 | | project_id | 41777 <+4441777>c6f1e7b4f8d8fd76b5e0f67e5e8 | | properties | | | security_groups | [{u'name': u'vel1bgw01-TIPC-Security-Group'}] | | status | ACTIVE | | updated | 2020-01-07T17:18:32Z | | user_id | be13deba85794016 <+4485794016>a00fec9d18c5d7cf | +--------------------------------------+------------------------------------------------------------------------------------------------------------+ *It is mapped to the following vDisks (seen using virsh list on compute-3)*: - dm-uuid-mpath-3600 <+443600>a098000 <+44098000>d9818 <+449818>b0000185 <+440000185>c5dfa0714 <+440714> è Boot Volume - dm-uuid-mpath-3600 <+443600>a098000 <+44098000>d9818 <+449818>b 000018565 <+44000018565>dfa069e è Storage volume <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <source dev='/dev/disk/by-id/dm-uuid-mpath-3600 <+443600>a098000 <+44098000> d9818 <+449818>b0000185 <+440000185>c5dfa0714 <+440714>'/> <backingStore/> <target dev='vda' bus='virtio'/> <serial>717e5744-4786-42 <+445744478642>dc-9e3e-3c5e6994 <+446994> c482</serial> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000 <+440000>' bus='0x00' slot='0x04' function='0x0'/> </disk> <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <source dev='/dev/disk/by-id/dm-uuid-mpath-3600 <+443600>a098000 <+44098000> d9818 <+449818>b000018565 <+44000018565>dfa069e'/> <backingStore/> <target dev='vdb' bus='virtio'/> <serial>d6cf0cf9-36d1-4b62-86 <+446286>b4-faa4a6642166 <+446642166></serial> <alias name='virtio-disk1'/> <address type='pci' domain='0x0000 <+440000>' bus='0x00' slot='0x08' function='0x0'/> </disk> Name: crypt-dm-uuid-mpath-3600 <+443600>a098000 <+44098000>d9818 <+449818>b 0000185 <+440000185>c5dfa0714 <+440714> State: ACTIVE Read Ahead: 256 Tables present: LIVE Open count: 1 Event number: 0 Major, minor: 253, 5 Number of targets: 1 UUID: CRYPT-LUKS1-769 <+441769>cc20bc5af469c8c9075 <+449075> a2a6fc4aa0-crypt-dm-uuid-mpath-*3600 <+443600>a098000 <+44098000>d9818 <+449818>b0000185 <+440000185>c5dfa0714 <+440714>* Name: *mpathpy* State: ACTIVE Read Ahead: 256 Tables present: LIVE Open count: 1 Event number: 32 Major, minor: 253, *4* Number of targets: 1 UUID: mpath-*3600 <+443600>a098000 <+44098000>d9818 <+449818>b0000185 <+440000185>c5dfa0714 <+440714>* Name: crypt-dm-uuid-mpath-3600 <+443600>a098000 <+44098000>d9818 <+449818>b 000018565 <+44000018565>dfa069e State: ACTIVE Read Ahead: 256 Tables present: LIVE Open count: 1 Event number: 0 Major, minor: 253, 7 Number of targets: 1 UUID: CRYPT-LUKS1-4015 <+4414015>c585a0df4074821 <+444074821> ca312c4caacca-crypt-dm-uuid-mpath-*3600 <+443600>a098000 <+44098000>d9818 <+449818>b000018565 <+44000018565>dfa069e* Name: *mpathpz* State: ACTIVE Read Ahead: 256 Tables present: LIVE Open count: 1 Event number: 28 Major, minor: 253, *6* Number of targets: 1 UUID: mpath-*3600 <+443600>a098000 <+44098000>d9818 <+449818>b000018565 <+44000018565>dfa069e* This means boot volume is represented by dm-4 while storage volume is represented by dm-6 Dumping the multipath daemon on the controller shows that at a steady running state both DMs are accounted for (see below). multipathd> show maps name sysfs uuid *mpathpy dm-4 3600 <+4443600>a098000 <+44098000>d9818 <+449818>b0000185 <+440000185>c5dfa0714 <+440714>* *mpathpz dm-6 3600 <+4463600>a098000 <+44098000>d9818 <+449818>b000018565 <+44000018565>dfa069e* mpathqi dm-12 3600 <+44123600>a098000 <+44098000>d9818 <+449818>b000018 <+44000018>df5dfafd40 mpathqj dm-13 3600 <+44133600>a098000 <+44098000>d9818 <+449818>b000018 <+44000018>de5dfafd10 mpathpw dm-0 3600 <+4403600>a098000 <+44098000>d9818 <+449818>b000018425 <+44000018425>dfa059f mpathpx dm-1 3600 <+4413600>a098000 <+44098000>d9818 <+449818>b0000184 <+440000184>c5dfa05fc mpathqk dm-16 3600 <+44163600>a098000 <+44098000>d9818 <+449818>b000018 <+44000018>eb5dfafe80 mpathql dm-17 3600 <+44173600>a098000 <+44098000>d9818 <+449818>b000018 <+44000018>e95dfafe26 mpathqh dm-9 3600 <+4493600>a098000 <+44098000>d9818 <+449818>b000018 <+44000018>c65dfafa91 These vDisks are mapped to the following multipaths: multipathd> show topology mpathpy (3600 <+443600>a098000 <+44098000>d9818 <+449818>b0000185 <+440000185>c5dfa0714) <+440714> dm-4 NETAPP ,INF-01-00 <+440100> size=21G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 rdac' wp=rw |-+- policy='service-time 0' prio=14 status=active | |- 30:0:0:82 sdm 8:192 active ready running | `- 32:0:0:82 sdk 8:160 active ready running `-+- policy='service-time 0' prio=0 status=enabled |- 33:0:0:82 sdn 8:208 failed faulty running `- 31:0:0:82 sdl 8:176 failed faulty running mpathpz (3600 <+443600>a098000 <+44098000>d9818 <+449818>b000018565 <+44000018565>dfa069e) dm-6 NETAPP ,INF-01-00 <+440100> size=10G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 rdac' wp=rw |-+- policy='service-time 0' prio=14 status=active | |- 30:0:0:229 sdr 65:16 active ready running | `- 32:0:0:229 sdp 8:240 active ready running `-+- policy='service-time 0' prio=0 status=enabled |- 31:0:0:229 sdo 8:224 failed faulty running `- 33:0:0:229 sdq 65:0 failed faulty running Now, it starts getting very interesting, if I shutdown controller-A from NetApp side, dm-4 disappears but dm-6 is still running while detecting the active path is controller B while standby path is controller-A which now is displayed as failed multipathd> show topology mpathpz (3600 <+443600>a098000 <+44098000>d9818 <+449818>b000018565 <+44000018565>dfa069e) dm-6 NETAPP ,INF-01-00 <+440100> size=10G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 rdac' wp=rw |-+- policy='service-time 0' prio=0 status=enabled | |- 30:0:0:229 sdr 65:16 failed faulty running | `- 32:0:0:229 sdp 8:240 failed faulty running `-+- policy='service-time 0' prio=11 status=active |- 31:0:0:229 sdo 8:224 active ready running `- 33:0:0:229 sdq 65:0 active ready running multipathd> show maps name sysfs uuid *mpathpz dm-6 3600 <+4463600>a098000 <+44098000>d9818 <+449818>b000018565 <+44000018565>dfa069e* mpathqi dm-12 3600 <+44123600>a098000 <+44098000>d9818 <+449818>b000018 <+44000018>df5dfafd40 mpathqj dm-13 3600 <+44133600>a098000 <+44098000>d9818 <+449818>b000018 <+44000018>de5dfafd10 mpathpw dm-0 3600 <+4403600>a098000 <+44098000>d9818 <+449818>b000018425 <+44000018425>dfa059f mpathpx dm-1 3600 <+4413600>a098000 <+44098000>d9818 <+449818>b0000184 <+440000184>c5dfa05fc mpathqk dm-16 3600 <+44163600>a098000 <+44098000>d9818 <+449818>b000018 <+44000018>eb5dfafe80 mpathql dm-17 3600 <+44173600>a098000 <+44098000>d9818 <+449818>b000018 <+44000018>e95dfafe26 mpathqg dm-8 3600 <+4483600>a098000 <+44098000>d9818 <+449818>b000018 <+44000018>c75dfafac0 mpathqh dm-9 3600 <+4493600>a098000 <+44098000>d9818 <+449818>b000018 <+44000018>c65dfafa91 If I restore Controller-A into service from NetApp side while fail only the path to controller A from multipathd everything works fine, dm-4 is still present and the VM can be put into service. multipathd> fail path sdk ok multipathd> multipathd> fail path sdm ok mpathpy (3600 <+443600>a098000 <+44098000>d9818 <+449818>b0000185 <+440000185>c5dfa0714) <+440714> dm-4 NETAPP ,INF-01-00 <+440100> size=21G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 rdac' wp=rw |-+- policy='service-time 0' prio=0 status=enabled | |- 32:0:0:82 sdk 8:160 failed faulty running | `- 30:0:0:82 sdm 8:192 failed faulty running `-+- policy='service-time 0' prio=9 status=active |- 31:0:0:82 sdl 8:176 active ready running `- 33:0:0:82 sdn 8:208 active ready running multipathd> reinstate path sdk ok multipathd> multipathd> reinstate path sdm ok mpathpy (3600 <+443600>a098000 <+44098000>d9818 <+449818>b0000185 <+440000185>c5dfa0714) <+440714> dm-4 NETAPP ,INF-01-00 <+440100> size=21G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 rdac' wp=rw |-+- policy='service-time 0' prio=14 status=active | |- 32:0:0:82 sdk 8:160 active ready running | `- 30:0:0:82 sdm 8:192 active ready running `-+- policy='service-time 0' prio=9 status=enabled |- 31:0:0:82 sdl 8:176 active ready running `- 33:0:0:82 sdn 8:208 active ready running It is observed in the working case, the storage volume disappears (which seems normal), also the instance totally vanishes from the virsh list and no trace can be found at the KVM level if we run ps -def | grep fd | grep <instance_ID>. However, the boot volume is always present in the multipathd records when we stop the VM at normal conditions without stopping NetApp controller. Any ideas? Kind regards, Ahmed

1 0

[infra] Retire x/dox
by Andreas Jaeger 07 Jan '20

07 Jan '20

The x/dox repo is unused, let's retire it. I'll put up changes with topic "retire-dox", Andreas -- Andreas Jaeger aj(a)suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB

1 0

[cinder] first meeting of 2020 tomorrow (8 January)
by Brian Rosmaita 07 Jan '20

07 Jan '20

Just wanted to send a quick reminder that the first Cinder team meeting of 2020 well be held tomorrow (8 January) on the usual day (Wednesday) at the usual time (1400 UTC) and in the usual place (#openstack-meeting-4). https://etherpad.openstack.org/p/cinder-ussuri-meetings See you there! brian

1 0

IBM z/VM CI is planning to migrate new environment
by Dong JV Ma 07 Jan '20

07 Jan '20

Hello, Due to the hardware issues, the IBM z/VM CI system [1] is planning to migrate to the new hardware in the next few months, the planning time will be in 3 months. We will do following changes with IBM z/VM CI environment: 1. Migrate the IBM z/VM CI to new hardware 2. Upgrade IBM z/VM CI from python2 to python3 During this time, IBM z/VM CI will not stable, please ignore the IBM z/VM CI failures. For any questions feel free to reach out to me(Dong Ma). [1] https://wiki.openstack.org/wiki/ThirdPartySystems/IBM_z/VM_CI Thanks & Best Regards! Dong Ma (马冬) LinuxONE IaaS Infra & CI/CD, China Systems Lab E-mail: madongma(a)cn.ibm.com IRC: larainema Address: 2/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC

3 2

[tc] Expediting patch to unblock governance's CI
by Jean-Philippe Evrard 07 Jan '20

07 Jan '20

Hello everyone, Our governance repo's testing is currently failing, and I have patches to fix it (short term [1] and long term). Because it's been a while it's in there (and got recently updated), I will now merge the short term fix, even if we don't have enough votes (nor time). It will unblock many patches and prevent useless rechecks, at the expense of being out of policy. I will gladly take the blame, should there be any. Regards, JP [1] https://review.opendev.org/#/c/700422/

1 0

[kolla] neutron-l3-agent namespace NAT table not working?
by Jon Masters 07 Jan '20

07 Jan '20

Hi there, I've got a weird problem with the neutron-l3-agent container on my deployment. It comes up, sets up the iptables rules in the qrouter namespace (and I can see these using "ip netns...") but traffic isn't having DNAT or SNAT applied. What's most strange is that manually adding a LOG jump target to the iptables nat PRE/POSTROUTING chains (after enabling nf logging sent to the host kernel, confirmed that works) doesn't result in any log entries. It's as if the nat table isn't being applied at all for any packets traversing the qrouter namespace. This is driving me crazy :) Anyone got some quick suggestions? (assume I tried the obvious stuff). Jon. -- Computer Architect

8 19

[Scientific] Scientific SIG meeting July 7th 2100 UTC
by Martial Michel 07 Jan '20

07 Jan '20

The first meeting of the new year at 2100 UTC on Tuesday, July 7th. Mostly an Any Other Business meeting to get its participants back from the holidays as reflected by the agenda :) https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_January_7th_2020

1 0

[all] Is there something I can do to get a simple fix done?
by Neil Jerram 07 Jan '20

07 Jan '20

I'm struggling to say this positively, but... it feels like OpenStack promotes refactoring work that will likely break something, but is very slow when a corresponding fix is needed, even when the fix is trivial. Is there something we could do to get fixes done more quickly when needed? My case in point: my team's networking plugin (networking-calico) does not do "extraroutes", and so was broken by some python-openstackclient change (possibly [1]) that wrongly assumed that. I posted a fix [2] that passed CI on 21st October, and asked for it to be reviewed on IRC a couple of days later. Édouard Thuleau posted a similar fix [3] on 4th December, and we agreed that his was better, so I abandoned mine. His fix attracted a +2 on 11th December, but has been sitting like that ever since. It's a fix that I would expect to be simple to review, so I wonder if there's something else we could have done here to get this moving? Or if there is a systemic problem here that deserves discussion? [1] https://opendev.org/openstack/python-openstackclient/commit/c44f26eb7e41c28… [2] https://review.opendev.org/#/c/685312/ [3] https://review.opendev.org/#/c/697240/ Many thanks, Neil

2 1

[all][neutron][neutron-vpnaas] Maintainers needed
by Slawek Kaplonski 06 Jan '20

06 Jan '20

Hi, Over the past couple of cycles we have noticed that new contributions and maintenance efforts for neutron-vpnaas were almost non existent. This impacts patches for bug fixes, new features and reviews. The Neutron core team is trying to at least keep the CI of this project healthy, but we don’t have enough knowledge about the details of the neutron-vpnaas code base to review more complex patches. During the PTG in Shanghai we discussed that with operators and TC members during the forum session [1] and later within the Neutron team during the PTG session [2]. During these discussions, with the help of operators and TC members, we reached the conclusion that we need to have someone responsible for maintaining project. This doesn’t mean that the maintainer needs to spend full time working on this project. Rather, we need someone to be the contact person for the project, who takes care of the project’s CI and review patches. Of course that’s only a minimal requirement. If the new maintainer works on new features for the project, it’s even better :) If we don’t have any new maintainer(s) before milestone Ussuri-2, which is Feb 10 - Feb 14 according to [3], we will need to mark neutron-vpnaas as deprecated and in “V” cycle we will propose to move the project from the Neutron stadium, hosted in the “openstack/“ namespace, to the unofficial projects hosted in the “x/“ namespace. So if You are using this project now, or if You have customers who are using it, please consider the possibility of maintaining it. Otherwise, please be aware that it is highly possible that the project will be deprecated and moved out from the official OpenStack projects. [1] https://etherpad.openstack.org/p/PVG-Neutron-stadium-projects-the-path-forw… [2] https://etherpad.openstack.org/p/Shanghai-Neutron-Planning-restored - Lines 379-421 [3] https://releases.openstack.org/ussuri/schedule.html -- Slawek Kaplonski Senior software engineer Red Hat

1 1

[neutron] Bug deputy report - week of 30th December
by Slawek Kaplonski 06 Jan '20

06 Jan '20

Hi, I was on bug deputy last week. It was pretty quiet week with only few bugs reported. Below is my summary of it. Critical: https://bugs.launchpad.net/neutron/+bug/1858260 - Upstream CI neutron-tempest-plugin-* fails - I marked it as critical as it cause gate failures, I will have to take a look at it closer next week, Medium: https://bugs.launchpad.net/neutron/+bug/1858086 - qrouter's local link route cannot be restored - confirmed by me on local env, it would be good if someone from L3 subteam can take a look at it, Undecided and others: https://bugs.launchpad.net/neutron/+bug/1858377 - probably bug for openstackclient rather than neutron, but I would like to wait for confirmation from bug reporter first, https://bugs.launchpad.net/neutron/+bug/1858262 - duplicate of other bug, https://bugs.launchpad.net/neutron/+bug/1858419 - docs bug to update config options for large scale deployments, see http://lists.openstack.org/pipermail/openstack-discuss/2020-January/011820.… for details — Slawek Kaplonski Senior software engineer Red Hat

1 0

[oslo][kolla][requirements][release][infra] Hit by an old, fixed bug
by Radosław Piliszek 06 Jan '20

06 Jan '20

Hi Folks, as the subject goes, my installation has been hit by an old bug: https://bugs.launchpad.net/oslo.messaging/+bug/1828841 (bug details not important, linked here for background) I am using Stein, deployed with recent Kolla-built source-based images (with only slight modifications compared to vanilla ones). Kolla's procedure for building source-based images considers upper constraints, which, unfortunately, turned out to be lagging behind a few releases w.r.t. oslo.messaging at least. The fix was in 9.7.0 released on May 21, u-c still point to 9.5.0 from Feb 26 and the latest of Stein is 9.8.0 from Jul 18. It seems oslo.messaging is missing from the automatic updates that bot proposes: https://review.opendev.org/#/q/owner:%22OpenStack+Proposal+Bot%22+project:o… Per: https://opendev.org/openstack/releases/src/branch/master/doc/source/referen… this upper-constraint proposal should be happening for all releases. I would be glad if someone investigated why it happens(/ed) and audited whether other OpenStack projects don't need updating as well to avoid running on old deps when new are awaiting for months. :-) Please note this might apply to other branches as well. PS: for some reason oslo.messaging Stein release notes ( https://docs.openstack.org/releasenotes/oslo.messaging/stein.html ) are stuck at 9.5.0 as well, this could be right (I did not inspect the sources) but I am adding this in PS so you have more things to correlate if they need be. -yoctozepto

5 7

[kolla] Adding Dincer Celik to kolla-core and kolla-ansible-core
by Mark Goddard 06 Jan '20

06 Jan '20

Hi, I recently proposed to the existing cores that we add Dincer Celik (osmanlicilegi) to the kolla-core and kolla-ansible-core groups and we agreed to go ahead. Thanks for your contribution to the project so far Dincer, I'm glad to have you on the team. Cheers, Mark

1 0

[neutron][nova][cinder][glance][largescale-sig] Documentation update for large-scale
by Arnaud Morin 06 Jan '20

06 Jan '20

Hey all, With the new "Large scale SIG", we were thinking about updating documentation to help operators setting up large deployments. To do so, we would like to propose, at least, some documentation changes to identify options that affect large scale. The plan is to have a small note on some options and eventually a link to a specific page for large scale (see attachments). I know that nova started working on collecting those parameters here: https://bugs.launchpad.net/nova/+bug/1838819 Do you know if something similar exists on other projects? Moreover, we would like to collect more parameters that could be tuned on a large scale deployment, for every project. So, if you have any, feel free to answer to this mail or add some info on the following etherpad: https://etherpad.openstack.org/p/large-scale-sig-documentation Thanks for your help! Regards. The large-scale team. -- Arnaud Morin

2 1

[cloudkitty] Stepping down from PTL
by Luka Peschke 06 Jan '20

06 Jan '20

Hi all, I'm moving to a new position that doesn't involve OpenStack, and won't leave me the required time to be Cloudkitty's PTL. This is why I have to step down from the PTL position. jferrieu will take my position for the end of the U cycle (he's been a major contributor recently), with the help of huats, who's been the Cloudkitty PTL before me, and has been around in the community for a long time. I've been the PTL for two and a half cycles, and I think that it is a good thing for the project to take a new lead, with a new vision. I'm grateful for my experience within the OpenStack community. Thanks, -- Luka Peschke (peschk_l)

3 2

[auto-scaling][self-healing] Discussion to merge two SIG to one
by Rico Lin 06 Jan '20

06 Jan '20

Dear all As we discussed in PTG about merge two SIG to one. I would like to continue the discussion on ML. In PTG, Eric proposes the idea to merge two SIG due to the high overlapping of domains and tasks. I think this is a great idea since, over the last 6 months, most of the discussions in both SIG are overlapped. So I'm onboard with this idea. Here's how I think we can continue this idea: 1. Create new SIG (maybe 'Automation SIG'? feel free to propose name which can cover both interest.) 2. Redirect docs and wiki to new SIG. And rework on index so there will be no confusion 3. Move repos from both SIGs to new SIG 4. Mark auto-scaling SIG and self-healing SIG as inactive. 5. remove auto-scaling SIG and self-healing SIG after a reasonable waiting time Let us know what you think about this. Otherwise, we definitely expect this to happen soon. -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

4 7