- openstack-discuss - lists.openstack.org

[kuryr] Meeting at PTG - follow up
by Michał Dulko 07 Nov '19

07 Nov '19

Hi, Thank you all for a constructive discussion at the Shanghai PTG. First of all I want to clarify that the Neutron improvements in bulk port creation code I talked about were only merged in Stein and are not available in Queens. Sorry about that, my mistake. Below I want to list some highlights from the discussion. * There seemed no pushback on switching to independent release model, so I'll follow up on that soon. * We discussed using RPC instead of API polling to detect when port becomes ACTIVE in Neutron (https://review.opendev.org/#/c/669642/) I commented on the review with a follow up that came from discussion with Neutron team. Apparently there are 2 better ways of doing this. Let's continue that discussion on the review. * We know that we don't support running as a second CNI plugin on Multus because of eth0 being hardcoded as interface name and kuryr- controller handling pods that do not had CNI requests. This is something to solve. * Apparently we all suffer the same problems with Neutron performance when starting a bigger workload on Kuryr-configured K8s cluster (think big Helm chart, multiple operators or simultaneous test suite). This all comes to reducing the number of calls to Neutron. We still don't see a feasible solution to solve this problem, but it seems to be a priority for both Samsung and Red Hat at the moment. Let's see if we'll meet again in Vancouver! Thanks, Michał

1 1

[kolla] Repository setup in non-internet environment.
by Eddie Yen 07 Nov '19

07 Nov '19

Hi, I'm thinking about the deployment in non-internet environment. As we know Kolla has already prepared docker registry and kolla-build to let user can create local registry for deployment. But there's still have two problem about non-internet deployment. 1. Docker-ce repository. 2. Pip repository. (Also having others perhaps.) Does Kolla planning to support non-internet deployment? I would like to do this if possible. Looking forward to hearing from you, Eddie.

4 5

ephemeral dics storage
by Immo Wetzel 06 Nov '19

06 Nov '19

Gday mates, We do run a pike installation and do run into a problem with the ephemeral discs. As written these are on one side the common way for normal VMs to create a disk as long as the vm exists. On the other side these are stored on local discs. But these are usually not the fastest way and therefore the documentation said that in production environments, which we are going to be, these should be stored on shared spaces too. Like a SAN. I found some descriptions how to use ceph for locale storage via rbd backend but we don't use ceph. Each compute node has an FC connection which is used via cinder for the volumes. So what would be the recommendation to use ephemeral disc with FC SAN ? THX a lot Immo

2 1

CPU pinning blues
by Albert Braden 06 Nov '19

06 Nov '19

I'm following this document to setup CPU pinning on Rocky: https://www.redhat.com/en/blog/driving-fast-lane-cpu-pinning-and-numa-topol… I followed all of the steps except for modifying non-pinned flavors and I have one aggregate containing a single NUMA-capable host: root@us01odc-dev1-ctrl1:/var/log/nova# os aggregate list +----+-------+-------------------+ | ID | Name | Availability Zone | +----+-------+-------------------+ | 4 | perf3 | None | +----+-------+-------------------+ root@us01odc-dev1-ctrl1:/var/log/nova# os aggregate show 4 +-------------------+----------------------------+ | Field | Value | +-------------------+----------------------------+ | availability_zone | None | | created_at | 2019-10-30T23:05:41.000000 | | deleted | False | | deleted_at | None | | hosts | [u'us01odc-dev1-hv003'] | | id | 4 | | name | perf3 | | properties | pinned='true' | | updated_at | None | +-------------------+----------------------------+ I have a flavor with the NUMA properties: root@us01odc-dev1-ctrl1:/var/log/nova# os flavor show s1.perf3 +----------------------------+-------------------------------------------------------------------------+ | Field | Value | +----------------------------+-------------------------------------------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | None | | disk | 35 | | id | be3d21c4-7e91-42a2-b832-47f42fdd3907 | | name | s1.perf3 | | os-flavor-access:is_public | True | | properties | aggregate_instance_extra_specs:pinned='true', hw:cpu_policy='dedicated' | | ram | 30720 | | rxtx_factor | 1.0 | | swap | 7168 | | vcpus | 4 | +----------------------------+-------------------------------------------------------------------------+ I create a VM with that flavor: openstack server create --flavor s1.perf3 --image NOT-QSC-CentOS6.10-19P1-v4 --network it-network alberttest4 but it goes to error status, and I see this in the logs: 2019-10-30 16:17:55.590 3248800 INFO nova.virt.hardware [req-d0c2de13-db23-41bd-8da3-34c68ff1d998 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Computed NUMA topology CPU pinning: usable pCPUs: [[4], [5], [6], [7]], vCPUs mapping: [(0, 4), (1, 5), (2, 6), (3, 7)] 2019-10-30 16:17:55.595 3248800 INFO nova.virt.hardware [req-d0c2de13-db23-41bd-8da3-34c68ff1d998 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Computed NUMA topology CPU pinning: usable pCPUs: [[0], [1], [2], [3], [4], [5], [6], [7]], vCPUs mapping: [(0, 0), (1, 1), (2, 2), (3, 3)] 2019-10-30 16:17:55.595 3248800 INFO nova.filters [req-d0c2de13-db23-41bd-8da3-34c68ff1d998 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Filter AggregateInstanceExtraSpecsFilter returned 0 hosts 2019-10-30 16:17:55.596 3248800 INFO nova.filters [req-d0c2de13-db23-41bd-8da3-34c68ff1d998 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Filtering removed all hosts for the request with instance ID '73b1e584-0ce4-478c-a706-c5892609dc3f'. Filter results: ['RetryFilter: (start: 3, end: 3)', 'AvailabilityZoneFilter: (start: 3, end: 3)', 'CoreFilter: (start: 3, end: 2)', 'RamFilter: (start: 2, end: 2)', 'ComputeFilter: (start: 2, end: 2)', 'ComputeCapabilitiesFilter: (start: 2, end: 2)', 'ImagePropertiesFilter: (start: 2, end: 2)', 'ServerGroupAntiAffinityFilter: (start: 2, end: 2)', 'ServerGroupAffinityFilter: (start: 2, end: 2)', 'DifferentHostFilter: (start: 2, end: 2)', 'SameHostFilter: (start: 2, end: 2)', 'NUMATopologyFilter: (start: 2, end: 2)', 'AggregateInstanceExtraSpecsFilter: (start: 2, end: 0)'] It looks like my hypervisor is passing the hw:cpu_policy='dedicated' requirement but it is failing on "pinned=true" The interesting part of the problem is that if I add a second apparently identical hypervisor to the aggregate it starts working. I create s1.perf3 VMs and they land on us01odc-dev1-hv002 and the XML shows that they are correctly pinned. When us01odc-dev1-hv002 is full then they start failing again. What should I be looking for here? What could cause one apparently identical hypervisor to fail AggregateInstanceExtraSpecsFilter while another one passes? In the nova-compute log of the failing hypervisor I see this: 2019-10-31 10:43:01.147 1103 INFO nova.compute.resource_tracker [req-dda65a9c-9d0a-4888-b4cb-0bf4423dd2f3 - - - - -] Instance 4856d505-c220-4873-b881-836b5b75f7bb has allocations against this compute host but is not found in the database. 2019-10-31 10:43:01.148 1103 INFO nova.compute.resource_tracker [req-dda65a9c-9d0a-4888-b4cb-0bf4423dd2f3 - - - - -] Final resource view: name=us01odc-dev1-hv003.internal.synopsys.com phys_ram=128888MB used_ram=38912MB phys_disk=1208GB used_disk=297GB total_vcpus=8 used_vcpus=6 pci_stats=[] Openstack can't find a VM with UUID 4856d505-c220-4873-b881-836b5b75f7bb. There are no VMs on hv003 but I can create a non-pinned VM there and it works. Do I have a "phantom" VM that is consuming resources on hv003? How can I fix that?

2 5

[nova] Operator input on automatic heal behaviors
by Dan Smith 06 Nov '19

06 Nov '19

Hi all, If you're a nova operator, you probably know (and love) our increasing number of "heal $thing" commands in nova-manage. Despite appearances, we do not try to make these inconsistent, duplicative, and confusing. However, in reality, they pretty much are. Further, they require something being broken, an operator noticing, and then manual execution to (hopefully) fix things. While reviewing another such proposed nova-manage command today, I decided to propose a potential solution to make this better in the future. I've got a spec proposed to create new standalone command/service for nova that will consolidate all of these into a very consistent interface, with a daemon mode that can be run in the background to constantly (and slowly) periodically audit these things and heal them when issues are found. If you're an operator and have strong feelings on this topic, please review and opine here: https://review.opendev.org/#/c/693226 Thanks! --Dan

1 0

[neutron][tap-as-a-service] ERSPAN support
by Jean Bernard Beuque (jbeuque) 06 Nov '19

06 Nov '19

Hello, I'd like to add ERSPAN support to the Tap-as-a-Service project. I've currently implemented a prototype that can be used with networking-vpp: https://opendev.org/x/networking-vpp The modified version of tap as a service is available here (The API has been extended to support ERSPAN): https://github.com/jbeuque/tap-as-a-service I don't know who maintains the Taas project. But if you think adding this functionality could be useful, please contact me. (Please take the modified version of Taas as a proposal to be discussed). Regards, Jean-Bernard Beuque

2 2

[kuryr] Kuryr team at the PTG
by Michal Dulko 06 Nov '19

06 Nov '19

Hi, I had not reserved Kuryr space on the PTG as we weren't expecting many Kuryr team members here, but turns out there's some representation. We'll meet on Thursday at 2 PM Shanghai time to discuss anything related to Kuryr. We want meet in the Blue Room (where the tables are) and will try to find some space to run the discussion. Today you can find me at the K8s SIG table. Feel free to join! Thanks, Michał

3 4

[cinder] ussuri PTG schedule
by Brian Rosmaita 05 Nov '19

05 Nov '19

The Ussuri PTG schedule is live: https://etherpad.openstack.org/p/shanghai-ptg-cinder Please check the schedule and let me know right away if your session causes a conflict for you. Except for the few fixed-time topics, we will follow the cinder tradition of dynamic scheduling, giving each topic exactly as much time as it needs and adjusting as we go. cheers, brian

2 1

[nova][ptg] Team dinner
by Stephen Finucane 05 Nov '19

05 Nov '19

Hi all, Thanks to Alex Xu, we have organized a table for dinner this evening, Wed 6th November at 7pm. Xibo restaurant (Xinjiang food) 3/F 83 Changshu Rd, Jingan Qu, Shanghai, China (near "Changshu Road" or "Jing An Temple" subway stations) 中國上海市静安区常熟路83号 The address is on Google Maps - hopefully it's accurate :) Anyone working on nova is welcome, though I'd ask that you'd note your attendance on the PTG etherpad [1]. Looking forward to seeing everyone, Stephen (stephenfin) [1] https://etherpad.openstack.org/p/nova-shanghai-ptg

2 1

[barbican] PTG Schedule
by Douglas Mendizábal 05 Nov '19

05 Nov '19

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hello Barbicaneers! I hope everyone made it to Shanghai safely. Our PTG session will be on Wednesday Nov 6 from 10:30am - 4:30p at the Kilo table. We've set up an etherpad to collect topics to talk about. Please feel free to add any topics you're interested in: https://etherpad.openstack.org/p/barbican-ussuri-ptg Additionally we've reserved a spot for a Team Photo on Thursday at 11am. Hope to see y'all soon! Cheers, - - Douglas Mendizabal -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEwcapj5oGTj2zd3XogB6WFOq/OrcFAl2/iBEACgkQgB6WFOq/ OrfFew/+IjhZe+qRCi/4EmaVEDf7QxJyZDIVUlLsPWHmF98wCdj+GsbzoWUuFfHM sJCpfpVAUjxrIFOEo5uF9WiZhU36G9pgoLd1Y8Kb0/QRIQEQQcKGnlYhCn+jQjbW J2tlDrkU0GBwEBzDt91gM5JCncviY8yT6nhlr/SSLqvZRQnPewerJNyJbYsVh6N2 moXQzfeRjg1SGqR0KVUcDVPe/pE+at8A5ARFCxDiJaOIUTP0qcfKtDXh714bevyi Sw2qgDZHbLHa1nEv3umuYGcrGpKz8Uuj5ju+7oGpPh4hX4pfPxbVDSzu8srfzTui ggvcxFrpZQvdff3Lec1eclxnB+c9Z1tBKYF7pPUVtN3NPfCATkVCSQACYORPZLdh GAnyxiiUXRwzIfOo0b6koa2pRi7ZWoz0DjVzpnl+D7qztUzyiguaj3KDnuTvlfQl iMQev1QHD6fAVvByHgDRj4dyUqUi2+V/DtNZ9w29AX7C+U/afSbNGvygc8yNCtHF vbkw68aPpj5zeB0OTjPQ6N5vsUc6bSXYGnECuGw24untnutvPKR+W9g9VQEUyN1h vhvn0IPHZ9QyBJ0ctpdfA6O9PNsjY/DQNyDeiNGljTIpBjepUmqMTXvycsn8VN/E yY0OL2QFGPhcsK7Q/yeUCzMm1sken2zMg8Bdxt10qbj4GsCMtyQ= =fdMR -----END PGP SIGNATURE-----

2 1

[Neutron] cancel the L3 meeting today
by LIU Yulong 05 Nov '19

05 Nov '19

Hi all, Due to the Summit and PTG next week, I will cancel the Neutron L3 meeting today. So see you there, and travel safe. Thanks. LIU Yulong

1 1

[cinder][osc][docs] openstackclient docs for cinder v2 vs v3
by Eric Fried 05 Nov '19

05 Nov '19

Howdy cinderinos. I've been on a mission to get all of the python-openstackclient command [1] and plugin [2] docs autogenerated rather than hardcoded [3] so you don't have to remember to update two places when you add/change a subcommand option. I'm almost done -- cinder is the last one -- but I want to confirm some odd observations before I dig in. - All of the v3 subcommands are implemented by code in the openstackclient.volume.v2 package. Where there's overlap, the command classes are identical from v2 to v3. However, it appears as though the v2 commands are a *superset* of the v3 commands. Specifically, the following appear in v2 but not v3 [4]: volume_backup_record_export volume_backup_record_import volume_backend_capability_show volume_backend_pool_list volume_host_failover Observations: * v3 has no other 'volume backup record' subcommands, but otherwise has the same 'volume backup' subcommands as v2. * v3 has no 'volume backend' subcommands. * v2 has both 'volume host failover' and 'volume host set', but v3 has only the latter. * It seems suspicious that the "missing" v3 commands comprise a contiguous block under the v2 entry point. So before I go creating a mess of v2-only and v2+v3 documents, I wanted to confirm that the above was actually intentional. - The existing hardcoded documents mention v1 and/or v2, but don't mention v3 at all (e.g. [5]). I want to confirm that it's okay for me to add mention of v3 where appropriate. Thanks, efried [1] https://docs.openstack.org/python-openstackclient/latest/cli/command-list.h… [2] https://docs.openstack.org/python-openstackclient/latest/cli/plugin-command… [3] https://review.opendev.org/#/q/topic:generate-docs+(status:open+OR+status:m… [4] https://opendev.org/openstack/python-openstackclient/src/tag/4.0.0/setup.cf… [5] https://docs.openstack.org/python-openstackclient/train/cli/command-objects…

2 1

Installation DOC for Three node
by Umesh Mishra 05 Nov '19

05 Nov '19

Dear Sir, This is inform you that, We want to build the openstack in our data center so request you please help to provide me doc so that we can build asap . Please help. Best Regards, Umesh Mishra Manager-IT&Devops | Click2Cloud Inc. | www.click2cloud.net<http://www.click2cloud.net/> Email: umesh.mishra(a)click2cloud.net<mailto:umesh.mishra@click2cloud.net> | Mobile: +91 7738599311

3 3

[neutron] PTG - remote access and reminders
by Slawek Kaplonski 05 Nov '19

05 Nov '19

Hi, Tomorrow (Wednesday) we are starting Neutron PTG session. Agenda is available at [1]. If You are not in Shanghai but would maybe try to participate remotely in the sessions, please reach out to me directly through email or IRC. I will try to provide some access and stream from the session. Also, please remember that on Wednesday morning we have onboarding session for new contributors. So if You are interested in contributing to Neutron, feel free to reach out to us in *room 431* - we are starting at 9:00 am :) [1] https://etherpad.openstack.org/p/Shanghai-Neutron-Planning -- Slawek Kaplonski Senior software engineer Red Hat

1 0

[tacker] Ussuri PTG
by Dharmendra Kushwaha 05 Nov '19

05 Nov '19

Tacker Folks, As our scheduled sessions will be finished by 12:30 pm, so lets meet after lunch. I will be available in PTG area full day. https://etherpad.openstack.org/p/Tacker-PTG-Ussuri Thanks & Regards Dharmendra Kushwaha ________________________________ The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. It shall not attach any liability on the originator or NECTI or its affiliates. Any views or opinions presented in this email are solely those of the author and may not necessarily reflect the opinions of NECTI or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately.

1 0

[neutron][QoS] QoS meeting cancelled November 5
by Rodolfo Alonso 05 Nov '19

05 Nov '19

Hello Neutrinos: Due to the summit this week, the Neutron QoS meeting will be cancelled. Next meeting will be November 19. Regards.

1 0

[qa] QA Office hour canceled for this week
by Ghanshyam Mann 05 Nov '19

05 Nov '19

Hello Everyone, As we are in PTG, I will cancel the QA office hour for this week and will continue the same on 14th Nov week. -gmann

1 0

[dev][ops][ptg][keystone] Join the keystone onboarding session!
by Colleen Murphy 05 Nov '19

05 Nov '19

Hello Stackers, If you're a developer, technical writer, operator, or user and interested in getting involved in the keystone project, stop by the keystone onboarding session in Shanghai next week! We will be at the Kilo table in the Blue Room on Wednesday from 9 to 10:30. The format will be open ended, so come with all your questions about how you can participate on the keystone team. Can't make it to the session? Take a look at our contributing guide[1] and feel free to get in touch with me directly. Colleen Murphy / cmurphy (keystone PTL) [1] https://docs.openstack.org/keystone/latest/contributor/how-can-i-help.html

1 1

[openstack-dev][kuryr] Unable to create pod
by VeeraReddy 04 Nov '19

04 Nov '19

HI,Pod is not creating . Status of pod is always "ContainerCreating" I installed openstack & kubernetes using below linkhttps://docs.openstack.org/kuryr-kubernetes/latest/installation/devstac… devstack(a)kuryr-daemon.service log :http://paste.openstack.org/show/785758/ Regards, Veera.

4 9

[glance] Shanghai Project Update
by Abhishek Kekane 04 Nov '19

04 Nov '19

Hi All, We had a very good project update session in Shanghai OpenInfra summit, where we have covered what we have done in Train cycle and what our priorities are in upcoming Ussuri cycle. Attaching the project update PDF file here for your reference. Thanks & Best Regards, Abhishek Kekane

1 0

[infra][qa] multiline tracebacks not being indexed anymore
by Matt Riedemann 04 Nov '19

04 Nov '19

We used to be able to query for things like this: message:"in reserve_block_device_name" AND message:"MessagingTimeout" AND tags:"screen-n-api.txt" to fingerprint a traceback in logstash like this [1] but that no longer works. The multiline logstash filter is at [2] but doesn't seem to be getting applied anymore. I asked about this in -infra today and fungi said: "(4:44:00 PM) fungi: mriedem: i suspect that coincided with switching away from osla, we may need some means of parsing tracebacks out of logs in the indexer" I don't know what that means (what's osla? is [2] no longer used?) but if someone could point me at some things to look at I could see if I can generate a fix. [1] https://zuul.opendev.org/t/openstack/build/ef0196fe84804b44ac106d011c8c29ea… [2] https://opendev.org/openstack/logstash-filters/src/branch/master/filters/op… -- Thanks, Matt

2 1

[blazar] Shanghai Summit and PTG activities for Blazar; no IRC meetings
by Pierre Riteau 04 Nov '19

04 Nov '19

Hello, Several of the Blazar core reviewers and contributors will be in Shanghai this week. Don't hesitate to talk to them if you are interested in resource reservation as a service. On Tuesday there will be a project update: https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/24373… And on Friday at the PTG, there will be a project onboarding session, as well as technical discussions. Since most of the team is in Shanghai, IRC meetings are cancelled this week. Best wishes, Pierre

1 0

Cinder multi backend quota update
by M Ranga Swami Reddy 04 Nov '19

04 Nov '19

Hello, We use 2 types of volume, like volumes and volumes_ceph. I can update the quota for volumes quota using "cinder quota-update --volumes=20 project-id" But for volumes_ceph, the above CLI failed with volumes_ceph un recognised option.. Any suggestions here?

2 2

Openstack third pary CI Implementation(Zuul V3 , Nodepool, Queens)
by Sikka, Rishabh 04 Nov '19

04 Nov '19

Dear Team , I am installing Zuul V3 ,Nodepool for our openstack third party CI ,Please let me know if any documentation is available for the same as I am struggling with the steps shared on zuul official page. Also please do let me know if #openstack-third-party-ci is correct IRC channel related to third party ci implementation, if it is correct please refer some of the name who had already implemented it. I had already tried posting my questions on same IRC channel but did not get the desired reply. Note -: If above PDL is not correct , please refer it to the correct PDL. Regards Rishabh Sikka

2 1

[self-healing][autohealing][ptg] self-healing session 11/5@1:40PM
by Eric K 04 Nov '19

04 Nov '19

Looking forward to seeing you tomorrow at the PTG! The self-healing SIG will meet to continue the work on making self-healing easy and available. We’re especially eager to hear/discuss user feedback, feature requests, and ideas for making self-healing easier and better. The session will take place on Tuesday 11/5 at 1:40 - 3:10 PM in room 431 [1]. Planned topics include predictive analytics, cross-project testing, user feedback, and unified bug reporting. Additional topics and comments welcome in the etherpad: https://etherpad.openstack.org/p/SHA-self-healing-SIG [1] Map and schedule https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/6e4619c416ff4bd19e1c0…

1 0