September 2020 - openstack-discuss

[Octavia] Please help with deployment of octavia unbound lb port when creating LB
by Bujack, Stefan 16 Sep '20

16 Sep '20

2 2

[masakari] Wallaby PTG planning
by Radosław Piliszek 16 Sep '20

16 Sep '20

Hello, Masakarians*! *Anyone interested in Masakari. It's again this time of the century when a new OpenStack release is up on the horizon. As always, it means there will be a PTG (Project Team Gathering) to discuss the new cycle (and summarise the previous). This time it's going to be fully virtual so we call it a vPTG. I've booked us some time for the Wallaby PTG. [1] There are 4 (four) 2-hour sessions to accommodate both APAC and EMEA/Americas: Thursday October 29, 2020 06:00 - 08:00 (UTC) Thursday October 29, 2020 13:00 - 15:00 (UTC) Friday October 30, 2020 06:00 - 08:00 (UTC) Friday October 30, 2020 13:00 - 15:00 (UTC) I have also created an initial PTG page on Etherpad. [2] Please fill in your names if you are going to participate - with session times that suit you (please consider being flexible, it seems we are spanning the globe now). Add your proposals even if you can't participate directly (add your names to proposals). Please register for the event itself as well, to let the organisers know how many people are going to be there. [3] See YOU there! [1] https://ethercalc.openstack.org/7xp2pcbh1ncb [2] https://etherpad.opendev.org/p/masakari-wallaby-vptg [3] https://www.eventbrite.com/e/project-teams-gathering-october-2020-tickets-1… -yoctozepto

1 0

vPTG October 2020 Team Signup Reminder
by Kendall Waters 16 Sep '20

16 Sep '20

Hello Everyone! Wanted to give you all a reminder that the deadline for signing up teams for the PTG is approaching! The virtual PTG will be held from Monday October 26th to Friday October 30th, 2020. To signup your team, you must complete BOTH the survey[1] AND reserve time in the ethercalc[2] by September 11th at 7:00 UTC. We ask that the PTL/SIG Chair/Team lead sign up for time to have their discussions in with 4 rules/guidelines. 1. Cross project discussions (like SIGs or support project teams) should be scheduled towards the start of the week so that any discussions that might shape those of other teams happen first. 2. No team should sign up for more than 4 hours per UTC day to help keep participants actively engaged. 3. No team should sign up for more than 16 hours across all time slots to avoid burning out our contributors and to enable participation in multiple teams discussions. Once your team is signed up, please register[3]! And remind your team to register! Registration is free, but since it will be how we contact you with passwords, event details, etc. it is still important! If you have any questions, please let us know. -The Kendalls (diablo_rojo & wendallkaters) [1] Team Survey: https://openstackfoundation.formstack.com/forms/oct2020_vptg_survey <https://openstackfoundation.formstack.com/forms/oct2020_vptg_survey> [2] Ethercalc Signup: https://ethercalc.openstack.org/7xp2pcbh1ncb <https://ethercalc.openstack.org/7xp2pcbh1ncb> [3] PTG Registration: https://october2020ptg.eventbrite.com <https://october2020ptg.eventbrite.com/>

3 4

[Requirements] [FFE] python-troveclient 5.1.1
by Lingxian Kong 16 Sep '20

16 Sep '20

Hi, I'd like to ask FFE for bumping python-troveclient to 5.1.1[1] in Victoria. The patch version 5.1.1 includes a fix for multi-region support for python-troveclient[2] OSC plugin. [1]: https://review.opendev.org/#/c/752122/ [2]: https://review.opendev.org/#/q/Ia0580a599fc2385d54def4e18e0780209b82eff7 --- Lingxian Kong Senior Software Engineer Catalyst Cloud www.catalystcloud.nz

2 1

[openstack][cinder] volume revert to snapshot on stein
by Ignazio Cassano 16 Sep '20

16 Sep '20

Hello Stackers, does cinder revert to snapshot works on stein ? The command cinder revert-to-snapshot returns: error: argument <subcommand>: invalid choice: u'revert-to-snapshot' *I also tried using cinder v3 api* post /v3/{project_id}/volumes/{volume_id} /action but it returns : {"itemNotFound": {"message": "The resource could not be found.", "code": 404}} Please, anyone could help me ? Ignazio

2 3

[tripleo] distributed-project-leadership RFC
by Wesley Hayutin 16 Sep '20

16 Sep '20

Greetings, The TripleO project would like to hear your thoughts with regards to breaking down the typical PTL role into smaller roles. Please read through this document [1] and share your thoughts in this thread. If you are interested in growing your leadership skills and being responsible for any of the roles listed there please also indicate that in your response. Thanks all! [1] https://governance.openstack.org/tc/resolutions/20200803-distributed-projec…

3 4

[cinder] propose Lucio Seki for cinder core
by Brian Rosmaita 16 Sep '20

16 Sep '20

Lucio Seki (lseki on IRC) has been very active this cycle doing reviews, answering questions in IRC, and participating in the Cinder weekly meetings and at the midcycles. He's been particularly thorough and helpful in his reviews of backend drivers, and has also been helpful in giving pointers to new driver maintainers who are setting up third party CI for their drivers. Having Lucio as a core reviewer will help improve the team's review bandwidth without sacrificing review quality. In the absence of objections, I'll add Lucio to the core team just before the next Cinder team meeting (Wednesday, 16 September at 1400 UTC in #openstack-meeting-alt). Please communicate any concerns to me before that time. cheers, brian

10 11

[tripleo] Call for Topics
by Wesley Hayutin 16 Sep '20

16 Sep '20

Greetings, You folks know the drill. Please take a moment to add topics for discussion for the Wallaby PTG. https://etherpad.opendev.org/p/tripleo-wallaby-topics Thanks all!

1 0

[ansible-sig][openstack-ansible-modules][PTG] Wallaby PTG for Openstack Ansible collections
by Sagi Shnaidman 16 Sep '20

16 Sep '20

Hi, all We have a scheduled session for Openstack Ansible collection project in PTG: Wednesday 16.00 - 17.00 UTC The Etherpad for it is in https://etherpad.opendev.org/p/os-ansible-wallaby-ptg Please add your name if you plan to attend, questions and topics that you want to address in PTG. If it will be too much for one session, I'll schedule more. Thanks -- Best regards Sagi Shnaidman

1 0

[osc] load command in the CLI is no longer?
by Eric K. Miller 16 Sep '20

16 Sep '20

Hi, We use the "load" command in various OpenStack client CLIs to load a list of commands to run in a single session (one example is when loading a large number of Gnocchi metrics manually). However, when testing with a newer version of the OpenStack Client, it looks like it doesn't exist anymore. OSC version 4.0.0 has the load command, version 5.2.1 does not. Did something happen that justified removing this? Thanks! Eric

1 0

[Neutron] vxlan to vlan bridge
by Ignazio Cassano 16 Sep '20

16 Sep '20

Hello Stackers, is it possibile to create a vxlan to vlan bridge in openstack like vmware nsx does? Thanks Ignazio

3 2

neutron-l3-agent:l3-agent has been creating the "keepalived -P -f..." process
by 宋贺 15 Sep '20

15 Sep '20

we create a '--distributed=True,--ha=True" router in OpenStack Newton. We found that l3-agent cannot read the pid file of keepalived, thus creating the keepalived process again. As a result, a large number of keepalived processes appear on the host node.As picture 1 in the attachment. This problem can be solved when we execute the following content and modify the permissions of the pid file [root@fusion0 ~]# chown neutron:neutron /var/lib/neutron/ha_confs/2a20ab37-ffe8-4955-8148-cefdcb5b6be3.pid [root@fusion0 ~]# chown neutron:neutron /var/lib/neutron/ha_confs/2a20ab37-ffe8-4955-8148-cefdcb5b6be3.pid-vrrp I want to know how to solve this problem completely from the perspective of neutron. neutron-l3-agent.log file is in the attachment.

1 0

[all][elections][ptl][tc] Combined PTL/TC Election Season
by Jeremy Stanley 15 Sep '20

15 Sep '20

Election details: https://governance.openstack.org/election/ The nomination period officially begins Sep 22, 2020 23:45 UTC. Please read the stipulations and timelines for candidates and electorate contained in this governance documentation. Be aware that the Wallaby cycle elections are a few weeks behind schedule[1]. We're still making sure the new TC is confirmed and any PTL runoff polls are closed and resolved before Victoria release day. Due to circumstances of timing, PTL and TC elections for the coming cycle will run concurrently; deadlines for their nomination and voting activities are synchronized but will still use separate ballots. Please note, if only one candidate is nominated as PTL for a project team during the PTL nomination period, that candidate will win by acclaim, and there will be no poll. There will only be a poll if there is more than one candidate stepping forward for a project team's PTL position. If teams do not produce any PTL candidate during the nomination period, or are interested in considering alternatives prior to nominations, the TC may consider requests to switch to the new distributed leadership model they recently documented[2]. In keeping with the established plan[3] to gradually reduce the size of the Technical Committee, we will only fill four (4) seats in this coming election. There will be further announcements posted to the mailing list as action is required from the electorate or candidates. This email is for information purposes only. If you have any questions which you feel affect others please reply to this email thread. If you have any questions that you which to discuss in private please email any of the election officials[4] so that we may address your concerns. [1] https://governance.openstack.org/tc/reference/election-exceptions.html [2] https://governance.openstack.org/tc/resolutions/20200803-distributed-projec… [3] https://governance.openstack.org/tc/reference/charter.html#number-of-seats-… [4] https://governance.openstack.org/election/#election-officials -- Jeremy Stanley on behalf of the OpenStack Technical Election Officials

1 0

[tripleo][ci] Red Tuesday
by Wesley Hayutin 15 Sep '20

15 Sep '20

Greetings, Well if you were working today you know there were several issues upstream. pip issues: https://bugs.launchpad.net/tripleo/+bug/1449136 containers-multinode: https://bugs.launchpad.net/tripleo/+bug/1895290 https://bugs.launchpad.net/tripleo/+bug/1895288 validations: https://launchpad.net/bugs/1895507 Everything I know of atm is either merged on or in the gate. https://review.opendev.org/#/c/751828/ https://review.opendev.org/#/c/751653/ There is a lot going on, please report bugs to launchpad and ping nick-ruck or nick-rover in #tripleo if you see something else. Thanks to everyone helping to push things along. 0/

1 0

Re: [Openstack-mentoring] Neutron subnet with DHCP relay - continued
by Thomas King 15 Sep '20

15 Sep '20

I got the metadata network accessible over a routed subnet! The breakthrough was making sure I pointed 169.254.169.254 to the Neutron DHCP address rather than the physical or bridge IP address, this route added to the physical router (L3 switch) that serves the subnet hosting the Ironic controller. Also, I made sure to add a host route for 169.254.169.254 to the baremetal remote subnet in Neutron so the baremetal nodes will have a proper route to the metadata subnet. There is still some network tweaking left to do. We have a separate subnet for ESXi mgmt vmkernels since they use virtual MAC addresses and neutron doesn't recognize them. However, enabling DHCP relay for that separate subnet breaks routing which also breaks cleaning and deploying, and I need to find out where. This is huge for our metal-as-a-service offering! Thanks, Tom King On Sat, Aug 22, 2020 at 12:53 AM Thomas King <thomas.king(a)gmail.com> wrote: > Ok, thanks. > > On Fri, Aug 21, 2020, 11:39 PM Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > >> No, I didn't. In the beginning I wanted to say "yes" but that is for >> overcloud. >> >> Now I am thinking, maybe even I do no have metadata on undercloud.... >> >> Will need to check on Monday. >> >> 2020-08-22, št 00:51, Thomas King <thomas.king(a)gmail.com> rašė: >> >>> Finally got it worked out except for reaching the metadata service IP >>> (169.254.169.254) from a remote network. Did you put specific routes in >>> place for that IP address on your physical network? >>> >>> Tom >>> >>> On Wed, Jul 22, 2020 at 3:19 AM Ruslanas Gžibovskis <ruslanas(a)lpic.lt> >>> wrote: >>> >>>> Ok >>>> >>>> here is a small copy paste: >>>> http://paste.openstack.org/show/o4Uay0DYbAdkcfUJGOLV/ >>>> >>>> also have relaunched previous commands, as I have redeployed overcloud >>>> :) >>>> >>>> and maybe you will be interested, in undercloud containers running: >>>> >>>> (undercloud) [stack@remote-u overcloud]$ sudo podman ps >>>> CONTAINER ID IMAGE >>>> COMMAND CREATED STATUS >>>> PORTS NAMES >>>> 39c684172ce7 >>>> docker.io/tripleomaster/centos-binary-neutron-dhcp-agent:current-tripleo >>>> /usr/sbin/dnsmasq... 7 days ago Up 7 days ago >>>> neutron-dnsmasq-qdhcp-e6d6c50b-fb69-4375-bb7d-0f6e0cfed5cb >>>> 30ccb169e666 >>>> docker.io/tripleomaster/centos-binary-ironic-pxe:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> ironic_pxe_http >>>> 6a62e08e8e10 >>>> docker.io/tripleomaster/centos-binary-ironic-pxe:current-tripleo >>>> /bin/bash -c BIND... 7 days ago Up 7 days ago >>>> ironic_pxe_tftp >>>> e4cbad7f7488 >>>> docker.io/tripleomaster/centos-binary-neutron-l3-agent:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> neutron_l3_agent >>>> d78d3828c420 >>>> docker.io/tripleomaster/centos-binary-neutron-openvswitch-agent:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago neutron_ovs_agent >>>> 3b67b6a69a87 >>>> docker.io/tripleomaster/centos-binary-neutron-dhcp-agent:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> neutron_dhcp >>>> 46d403293b54 >>>> docker.io/tripleomaster/centos-binary-swift-proxy-server:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> swift_proxy >>>> 15a9792d6f62 >>>> docker.io/tripleomaster/centos-binary-swift-object:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> swift_rsync >>>> 0e6cd094d6dc >>>> docker.io/tripleomaster/centos-binary-swift-object:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> swift_object_updater >>>> 80b7f26e742d >>>> docker.io/tripleomaster/centos-binary-swift-object:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> swift_object_server >>>> 765f79633499 >>>> docker.io/tripleomaster/centos-binary-swift-proxy-server:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> swift_object_expirer >>>> 0e53da61af88 >>>> docker.io/tripleomaster/centos-binary-swift-container:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> swift_container_updater >>>> 8c14cef58eb3 >>>> docker.io/tripleomaster/centos-binary-swift-account:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> swift_account_server >>>> 12930a63dc12 >>>> docker.io/tripleomaster/centos-binary-swift-container:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> swift_container_server >>>> 8a5cd45208c0 >>>> docker.io/tripleomaster/centos-binary-swift-account:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> swift_account_reaper >>>> b871400bfdd9 >>>> docker.io/tripleomaster/centos-binary-mistral-executor:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> mistral_executor >>>> 3e98e5f83f09 >>>> docker.io/tripleomaster/centos-binary-mistral-event-engine:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> mistral_event_engine >>>> 52d711dffca3 >>>> docker.io/tripleomaster/centos-binary-mistral-engine:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> mistral_engine >>>> 23dec12e650f >>>> docker.io/tripleomaster/centos-binary-iscsid:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago iscsid >>>> bafff227d9d7 >>>> docker.io/tripleomaster/centos-binary-haproxy:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago haproxy >>>> 4d17cbd60698 >>>> docker.io/tripleomaster/centos-binary-ironic-inspector:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> ironic_inspector_dnsmasq >>>> f77e49246c35 >>>> docker.io/tripleomaster/centos-binary-ironic-inspector:current-tripleo >>>> kolla_start 7 days ago Up 7 days ago >>>> ironic_inspector >>>> ec70f40bba04 >>>> docker.io/tripleomaster/centos-binary-rabbitmq:current-tripleo >>>> kolla_start 5 weeks ago Up 4 weeks ago rabbitmq >>>> f09c9a129d85 >>>> docker.io/tripleomaster/centos-binary-memcached:current-tripleo >>>> kolla_start 5 weeks ago Up 4 weeks ago >>>> memcached >>>> 0eb1e953dcaa >>>> docker.io/tripleomaster/centos-binary-nova-compute-ironic:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> nova_compute >>>> ac78778d2cb0 >>>> docker.io/tripleomaster/centos-binary-ironic-neutron-agent:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> ironic_neutron_agent >>>> 8660bf80fd9c >>>> docker.io/tripleomaster/centos-binary-ironic-conductor:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> ironic_conductor >>>> 9dead5068168 >>>> docker.io/tripleomaster/centos-binary-mistral-api:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> mistral_api >>>> f5c1c9d6166c >>>> docker.io/tripleomaster/centos-binary-ironic-api:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> ironic_api >>>> 48899d782dd5 >>>> docker.io/tripleomaster/centos-binary-nova-api:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> nova_api_cron >>>> 1b19c94834e8 >>>> docker.io/tripleomaster/centos-binary-nova-api:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago nova_api >>>> 5b66d128c930 >>>> docker.io/tripleomaster/centos-binary-glance-api:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> glance_api >>>> 7e63cdb8d6b4 >>>> docker.io/tripleomaster/centos-binary-placement-api:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> placement_api >>>> 2db105676f8b >>>> docker.io/tripleomaster/centos-binary-zaqar-wsgi:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> zaqar_websocket >>>> d3e5ae7368e6 >>>> docker.io/tripleomaster/centos-binary-zaqar-wsgi:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago zaqar >>>> 6332e26b40ec >>>> docker.io/tripleomaster/centos-binary-nova-scheduler:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> nova_scheduler >>>> a5b0e0904f0c >>>> docker.io/tripleomaster/centos-binary-nova-conductor:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> nova_conductor >>>> 4460302d97c8 >>>> docker.io/tripleomaster/centos-binary-neutron-server:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> neutron_api >>>> d418deb9ea13 >>>> docker.io/tripleomaster/centos-binary-cron:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> logrotate_crond >>>> 7225ff80b26d >>>> docker.io/tripleomaster/centos-binary-heat-engine:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> heat_engine >>>> 84d22b3b1663 >>>> docker.io/tripleomaster/centos-binary-heat-api:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago >>>> heat_api_cron >>>> 226e0d839772 >>>> docker.io/tripleomaster/centos-binary-heat-api:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago heat_api >>>> 6f7871a64325 >>>> docker.io/tripleomaster/centos-binary-keystone:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago keystone >>>> 8402ff823012 >>>> docker.io/tripleomaster/centos-binary-mariadb:current-tripleo >>>> kolla_start 7 weeks ago Up 4 weeks ago mysql >>>> (undercloud) [stack@remote-u overcloud]$ >>>> >>>

1 0

[Neutron] memchached connections
by Tony Liu 15 Sep '20

15 Sep '20

I restarted neutron-server on all 3 nodes. Those memcached connections from neutron-server are gone. Everything is back to normal. It seems like that memcached connections are not closed properly in neutron-server. The connections pile up over time. Is there any know issue related? Could any Neutron experts comment here? Thanks! Tony > -----Original Message----- > From: Tony Liu <tonyliu0592(a)hotmail.com> > Sent: Friday, September 11, 2020 2:26 PM > To: openstack-discuss <openstack-discuss(a)lists.openstack.org> > Subject: memchached connections > > Hi, > > Is there any guidance or experiences to estimate the number of memcached > connections? > > Here is memcached connection on one of the 3 controllers. > Connection number is the total established connections to all 3 > memcached nodes. > > Node 1: > 10 Keystone workers have 62 connections. > 11 Nova API workers have 37 connections. > 6 Neutron server works have 4304 connections. > 1 memcached has 4973 connections. > > Node 2: > 10 Keystone workers have 62 connections. > 11 Nova API workers have 30 connections. > 6 Neutron server works have 3703 connections. > 1 memcached has 4973 connections. > > Node 3: > 10 Keystone workers have 54 connections. > 11 Nova API workers have 15 connections. > 6 Neutron server works have 6541 connections. > 1 memcached has 4973 connections. > > Before I increase the connection limit for memcached, I'd like to > understand if all the above is expected? > > How Neutron server and memcached take so many connections? > > Any elaboration is appreciated. > > BTW, the problem leading me here is memcached connection timeout, which > results all services depending on memcached stop working properly. > > > Thanks! > Tony

2 3

[tc] weekly update
by Mohammed Naser 15 Sep '20

15 Sep '20

Hi everyone, Here's an update for what happened in the OpenStack TC this week. You can get more information by checking for changes in openstack/governance repository. We've also included a few references to some important mailing list threads that you should check out. # Patches ## Open Reviews - Remove tc:approved-release tag https://review.opendev.org/749363 - Retire the devstack-plugin-zmq project https://review.opendev.org/748731 - Add openstack/osops to Ops Docs and Tooling SIG https://review.opendev.org/749835 - Retire devstack-plugin-pika project https://review.opendev.org/748730 - Reinstate weekly meetings https://review.opendev.org/#/c/749279/ - Resolution to define distributed leadership for projects https://review.opendev.org/744995 - Add assert:supports-standalone https://review.opendev.org/722399 ## Project Updates - Add openstack-helm-deployments to openstack-helm https://review.opendev.org/748302 - Add openstack-ansible/os_senlin role https://review.opendev.org/748677 - kolla-cli: deprecation - Mark as deprecated https://review.opendev.org/749694 ## General Changes - Create starter-kit:kubernetes-in-virt tag https://review.opendev.org/736369 # Other Reminders - PTG Brainstorming: https://etherpad.opendev.org/p/tc-wallaby-ptg Thanks for reading! Mohammed & Kendall -- Mohammed Naser VEXXHOST, Inc.

1 0

[nova][barbican][qa] barbican-tempest-plugin change breaking bfv [ceph]
by Mohammed Naser 15 Sep '20

15 Sep '20

Hi everyone, We just had our gating break due to a change merging inside barbican-tempest-plugin which is the following: https://review.opendev.org/#/c/515210/ It is resulting in an exception in our CI: 2020-08-26 18:04:32.663188 | controller | Response - Headers: {'content-length': '257', 'content-type': 'application/json', 'x-openstack-request-id': 'req-7f55e463-c3de-445e-a814-ef79c5f21235', 'connection': 'close', 'status': '409', 'content-location': 'http://glance.openstack.svc.cluster.local/v2/images/dec14e17-0870-415f-82a6…'} 2020-08-26 18:04:32.663200 | controller | Body: b'{"message": "Image dec14e17-0870-415f-82a6-140c1b7e4a39 could not be deleted because it is in use: The image cannot be deleted because it is in use through the backend store outside of Glance.<br /><br />\\n\\n\\n", "code": "409 Conflict", "title": "Conflict"}' This is usually because it's trying to delete the Glance image before deleting an instance that is using it, the specific test that is failing is: barbican_tempest_plugin.tests.scenario.test_certificate_validation.CertificateValidationTest.test_signed_image_invalid_cert_boot_failure[compute,id-6d354881-35a6-4568-94b8-2204bbf67b29,image] This test landed yesterday, we're blacklisting the scenario right now. I do find it quite interesting that in the logs here: http://paste.openstack.org/show/797186/ That the instance reports that it _does_ indeed delete it, so maybe we are trying to delete the image afterwards _too quickly_ and need to wait for Nova to clean up? I'd love to enable that test again and continue full coverage, happy to hear discussion. Thanks Mohammed -- Mohammed Naser VEXXHOST, Inc.

2 1

[ops] picking up some activity
by Chris Morgan 15 Sep '20

15 Sep '20

Greetings! The OpenStack Operators ("ops") meetups team will attempt to have an IRC meeting at the normal time and place (#openstack-operators on freenode at 10am EST/DST) on *Sept 15th*( following a period of complete inactivity for obvious reasons. If you're an official member of the team or even just interested in what we do, please feel free to join us. Whilst we can't yet contemplate resuming in-person meetups during this global pandemic, we can resume attempting to build the openstack operators community, share knowledge and perhaps even do some more virtual get-togethers. See you then Chris on behalf of the openstack ops meetups team -- Chris Morgan <mihalis68(a)gmail.com>

3 2

Need help for stack update feature for scaling out use case or add new resources
by Rahul Kumar 15 Sep '20

15 Sep '20

Hi Team, I have a stack template with 3 nodes , each node having a volume attached ! Now i updated my template with additional node and additional volume for that node ! This results in user_data update of a node in template ! And when i perform stack update feature , it gives me error: Invalid volume: Volume 01e40c6e-4467-42fe-ba9d-ce7012db8978 status must be available or downloading to reserve, but the current status is in-use. This shows for the node (where user_data is changed) and yes it is currently in use ! Then how one can update the stack using stack update feature of openstack with volumes ? Rahul

1 0

device compatibility interface for live migration with assigned devices
by Yan Zhao 15 Sep '20

15 Sep '20

hi folks, we are defining a device migration compatibility interface that helps upper layer stack like openstack/ovirt/libvirt to check if two devices are live migration compatible. The "devices" here could be MDEVs, physical devices, or hybrid of the two. e.g. we could use it to check whether - a src MDEV can migrate to a target MDEV, - a src VF in SRIOV can migrate to a target VF in SRIOV, - a src MDEV can migration to a target VF in SRIOV. (e.g. SIOV/SRIOV backward compatibility case) The upper layer stack could use this interface as the last step to check if one device is able to migrate to another device before triggering a real live migration procedure. we are not sure if this interface is of value or help to you. please don't hesitate to drop your valuable comments. (1) interface definition The interface is defined in below way: __ userspace /\ \ / \write / read \ ________/__________ ___\|/_____________ | migration_version | | migration_version |-->check migration --------------------- --------------------- compatibility device A device B a device attribute named migration_version is defined under each device's sysfs node. e.g. (/sys/bus/pci/devices/0000\:00\:02.0/$mdev_UUID/migration_version). userspace tools read the migration_version as a string from the source device, and write it to the migration_version sysfs attribute in the target device. The userspace should treat ANY of below conditions as two devices not compatible: - any one of the two devices does not have a migration_version attribute - error when reading from migration_version attribute of one device - error when writing migration_version string of one device to migration_version attribute of the other device The string read from migration_version attribute is defined by device vendor driver and is completely opaque to the userspace. for a Intel vGPU, string format can be defined like "parent device PCI ID" + "version of gvt driver" + "mdev type" + "aggregator count". for an NVMe VF connecting to a remote storage. it could be "PCI ID" + "driver version" + "configured remote storage URL" for a QAT VF, it may be "PCI ID" + "driver version" + "supported encryption set". (to avoid namespace confliction from each vendor, we may prefix a driver name to each migration_version string. e.g. i915-v1-8086-591d-i915-GVTg_V5_8-1) (2) backgrounds The reason we hope the migration_version string is opaque to the userspace is that it is hard to generalize standard comparing fields and comparing methods for different devices from different vendors. Though userspace now could still do a simple string compare to check if two devices are compatible, and result should also be right, it's still too limited as it excludes the possible candidate whose migration_version string fails to be equal. e.g. an MDEV with mdev_type_1, aggregator count 3 is probably compatible with another MDEV with mdev_type_3, aggregator count 1, even their migration_version strings are not equal. (assumed mdev_type_3 is of 3 times equal resources of mdev_type_1). besides that, driver version + configured resources are all elements demanding to take into account. So, we hope leaving the freedom to vendor driver and let it make the final decision in a simple reading from source side and writing for test in the target side way. we then think the device compatibility issues for live migration with assigned devices can be divided into two steps: a. management tools filter out possible migration target devices. Tags could be created according to info from product specification. we think openstack/ovirt may have vendor proprietary components to create those customized tags for each product from each vendor. e.g. for Intel vGPU, with a vGPU(a MDEV device) in source side, the tags to search target vGPU are like: a tag for compatible parent PCI IDs, a tag for a range of gvt driver versions, a tag for a range of mdev type + aggregator count for NVMe VF, the tags to search target VF may be like: a tag for compatible PCI IDs, a tag for a range of driver versions, a tag for URL of configured remote storage. b. with the output from step a, openstack/ovirt/libvirt could use our proposed device migration compatibility interface to make sure the two devices are indeed live migration compatible before launching the real live migration process to start stream copying, src device stopping and target device resuming. It is supposed that this step would not bring any performance penalty as -in kernel it's just a simple string decoding and comparing -in openstack/ovirt, it could be done by extending current function check_can_live_migrate_destination, along side claiming target resources.[1] [1] https://specs.openstack.org/openstack/nova-specs/specs/stein/approved/libvi… Thanks Yan

14 118

[manila][ptg] Wallaby PTG Planning
by Goutham Pacha Ravi 14 Sep '20

14 Sep '20

Hello Zorillas and Interested Stackers, As you're aware, the virtual PTG for the Wallaby release cycle is between October 26-30, 2020. If you haven't registered yet, you must do so as soon as possible! [1]. We've signed up for some slots on the PTG timeslots ethercalc [2]. The PTG Planning etherpad [3] is now live. Please go ahead and add your name/irc nick and propose any topics. You may propose topics even if you wouldn't like to moderate the discussion. Thanks, and hope to see you all there! Goutham [1] https://www.eventbrite.com/e/project-teams-gathering-october-2020-tickets-1… [2] https://ethercalc.openstack.org/7xp2pcbh1ncb [3] https://etherpad.opendev.org/p/wallaby-ptg-manila-planning

1 0

[Outreachy] Call for mentors, projects by Sep 29
by Samuel de Medeiros Queiroz 14 Sep '20

14 Sep '20

Hey Stackers, *TL;DR OpenStack is participating in the Outreachy internship program once again!* *Please submit projects as soon as possible, with final deadline being Sept. 29, 2020 at 4 pm UTC: https://www.outreachy.org/communities/cfp/openstack/ <https://www.outreachy.org/communities/cfp/openstack/>* Outreachy's goal is to support people from groups underrepresented in the technology industry. Interns will work remotely with mentors from our community. We are seeking mentors to propose projects that Outreachy interns can work on during their internship. If you want help crafting your project proposal, please contact me < samueldmq(a)gmail.com> or Mahati Chamarthy <mahati.chamarthy(a)gmail.com>. Mentors should read the mentor FAQ: https://www.outreachy.org/mentor/mentor-faq Full details about the Outreachy program and the internship timeline can be found on the Call for Participation page on the Outreachy website: https://www.outreachy.org/communities/cfp/ Thank you, Samuel Queiroz

1 0

memchached connections
by Tony Liu 14 Sep '20

14 Sep '20

Hi, Is there any guidance or experiences to estimate the number of memcached connections? Here is memcached connection on one of the 3 controllers. Connection number is the total established connections to all 3 memcached nodes. Node 1: 10 Keystone workers have 62 connections. 11 Nova API workers have 37 connections. 6 Neutron server works have 4304 connections. 1 memcached has 4973 connections. Node 2: 10 Keystone workers have 62 connections. 11 Nova API workers have 30 connections. 6 Neutron server works have 3703 connections. 1 memcached has 4973 connections. Node 3: 10 Keystone workers have 54 connections. 11 Nova API workers have 15 connections. 6 Neutron server works have 6541 connections. 1 memcached has 4973 connections. Before I increase the connection limit for memcached, I'd like to understand if all the above is expected? How Neutron server and memcached take so many connections? Any elaboration is appreciated. BTW, the problem leading me here is memcached connection timeout, which results all services depending on memcached stop working properly. Thanks! Tony

3 6

WIndows 10 instance hostname not updating
by its-openstack＠zohocorp.com 14 Sep '20

14 Sep '20

Dear openstack, I have installed openstack train branch, I am facing issue with windows image. all windows 10 instance dosen't get its hostname updated from the metadata, but able to get the metadata(hostname) from inside the instance using powershell. ``` $ Invoke-WebRequest http://169.254.169.254/latest/meta-data/hostname -UseBasicParsing ``` windows2016 instance no issue. using the stable cloudbase-init package for preparation of windows 10 (tried bot in v1909 and v2004). windows2016 server dosen't have this issue. if you would so kindly help us with this issue Regards sysadmin

2 1