September 2021 - openstack-discuss

how dpdk improve guest performance
by Satish Patel 30 Sep '21

30 Sep '21

Folks, If I am running dpdk in my compute nodes then sure it will reduce kernel overhead but how does it help to improve guest network performance? Inside guest vm it's going to use kernel to copy packet to userspace. in my example we run high performance haproxy which handles 300k to 500k connections but I have noticed performance is poor and lots of packet drops. If I run the same workload on SRIOV vms it works fine and I notice less kernel context switching inside the guest VM. I think when you run DPDK in compute then it only help compute but not your guest network performance. How do I improve guest performance also with dpdk based compute?

1 0

[nova] WSGI Apache vs Eventlet HTTP Servers
by Ammad Syed 30 Sep '21

30 Sep '21

Hi, I was reviewing below document. https://docs.openstack.org/nova/wallaby/user/wsgi.html Is apache2 wsgi provide better performance then eventlet http servers for nova api ? If yes, can you guys please share the sample or reference apache-nova-api file for wsgi ? - Ammad

1 0

[GPU] Request for suggestions for OpenStack deployment
by Jay See 30 Sep '21

30 Sep '21

Hi, We are planning to buy some servers with GPU cards from DELL(R740/R750)/HPE/Supermicro with NVIDIA A100 40GB (GPU). I have the following questions: 1. What is the best ratio for Memory to GPU? or It doesn´t matter. 2. Anyone using A100? A100 has any known issues with OpenStack deployment? 3. If you have set up with similar server with GPU, could you share the server configuration? It will help us in deciding / identifying the components we have missed. Thanks in advance. Regards, ~ Jayachander.

1 0

DC DR high availability - Snapshot taking feature and user experience
by KK CHN 30 Sep '21

30 Sep '21

List, I am in the process of setting up a DC and DR sites with openstack(ussuri, qemu kvm with debian as base OS). I would like to know what all are the options in DC and DR sites for High availability . What the architecture methods to follow at DC and DR so that, If VM crashes at one physical machine at DC ( for example 3 controllers in DC), or Host machine crashes( RAM, Hardware failure etc. ) / machine power cable detached accidentally so that no services down / VMs and applications not down. How can achieve this ? share the best practices here. Also If I take snapshots of the running VMs ( what will the user experience ? will they in freeze / logged out from applications right now they are logged in ? ) . Can we avoid the service unavailable for users while taking snapshots ? To DR we are rsyncing these snapshots after converting each VM image to qcow2 then rsyncing . ( All these we are performing on Controller node . Generally 100 GB VM will output 16 GB qcow2 image for example. ) its taking 2 minutes for snapshot creation and 20 to 30 Minutes for qcow2 conversion . ) How many snapshots and conversion to qcow2( qemu-img convert) can be performed on this controller machine where we performing this operation . ( Can we apply parallel processing for this and how ? ) . Seet he controller specs where we perform this. The controller spec ( 20 core CPUs with 1.5 TB RAM total 160 processors ) Then copying to DR ( which is 200 KM away from our DC takes 4 minutes with rsync). Then we populate each VM with these copied qcow2 images and attaching the Network ips and nat there. Is this the practice or any other good way to perform this for a robust DC and DR setup . Kindly share your thoughts. Regards, Kris

1 0

[all][tc] Technical Committee next weekly meeting on Sept 30th at 1500 UTC
by Ghanshyam Mann 29 Sep '21

29 Sep '21

Hello Everyone, Technical Committee's next weekly meeting is scheduled for Sept 30th at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, Sept 29th, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

DPDK Poor performance
by Satish Patel 29 Sep '21

29 Sep '21

Folk, I have deployed DPDK on one of my compute nodes to reply to my sriov deployment which doesn't support bonding. This is what I am doing to do a performance benchmark. Spun up VM with 8 vCPU and 8GB memory with single virtual nic. Then install nuttcp tool and start nuttcp -S (server mode) and then use Physical server outside my lab and use following command to blast some traffic nuttcp -T 30 -i -u -w4m -R 2G 10.69.7.130 I found 120kpps rate working fine but after that I noticed a packet drop on nuttcp. Here is the full details of my deployment and load-test result - https://paste.opendev.org/show/809675/ Small confusion related hugepage - I am seeing following in /dev/hugepages/ I believe rtemap_0 created by DPDK root@ovn-lab-comp-dpdk-1:~# ls -l /dev/hugepages/rtemap_0 -rw------- 1 root root 1073741824 Sep 29 02:28 /dev/hugepages/rtemap_0 Why is the qemu instance file empty? Does that mean it's not using a huge page for dpdk performance? root@ovn-lab-comp-dpdk-1:~# ls -l /dev/hugepages/libvirt/qemu/1-instance-00000045/ total 0 I have hw:mem_page_size='large' setting in flavor so it should use hugepage. am i missing something?

1 0

[largescale-sig] Next meeting: Sept 29th, 15utc
by Thierry Carrez 29 Sep '21

29 Sep '21

Hi everyone, The Large Scale SIG meeting is back this Wednesday in #openstack-operators on OFTC IRC, at 15UTC. You can doublecheck how that time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20210929T15 A number of topics have already been added to the agenda, including discussing the final details of our next OpenInfra.Live show! Feel free to add other topics to our agenda at: https://etherpad.openstack.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

1 1

[kolla] Yoga PTG
by Michał Nasiadka 29 Sep '21

29 Sep '21

Hola Koalas! We are approaching the time of the next PTG. Kolla Yoga etherpad is available and initially populated[1], feel free to add any topics that shall be discussed and register your planned attendance. Don't forget to register! [2]. [1] https://etherpad.opendev.org/p/kolla-yoga-ptg [3] https://openinfra-ptg.eventbrite.com Best regards, Michal Nasiadka

1 0

[cinder] Bug deputy report for week of 2021-29-09
by Sofia Enriquez 29 Sep '21

29 Sep '21

This is a bug report from 2021-15-09 to 2021-29-09. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Medium - https://bugs.launchpad.net/os-brick/+bug/1943977 "os_bricks fails to rerty "iscsiadm -m session" when iser_use_multipath and iscsi_use_multipath are set in nova.conf". Assigned to Sophie Huang. - https://bugs.launchpad.net/os-brick/+bug/1945323 "Can't attach NVMe volume" Low - https://bugs.launchpad.net/cinder/+bug/1945169 " [swift] volume backup's status keep in 'restoring' and Lost connection to MySQL server during query". Assigned to silphon zhang. Wishlist - https://bugs.launchpad.net/cinder/+bug/1945249 '[charm] no upgrade path to cinder-lvm subordinate'. Unassigned. Cheers, -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

[Cinder] [Nova] [Interop] "compute-volume-list" API guideline
by Vida Haririan 29 Sep '21

29 Sep '21

Hi Cinder and Nova teams, The Interop team is reviewing all related open stories to determine next steps. We researched the current status of 1638112:“os-volumes should be flagged for removal in the compute API guidelines“ [1] and noticed that "compute-volume-list" was removed in 2017.01 guidelines [2][3]. However, two related tests still remain in tempest [4][5] and we are reaching out to determine if the API is still in use or has been moved/renamed. Please let us know if you have any questions. [1] https://storyboard.openstack.org/#!/story/1638112 [2] https://opendev.org/osf/interop/src/commit/dd7ae2660de5daa82ab4fe080e875d63… [3] https://opendev.org/osf/interop/src/branch/master/guidelines/next.json [4] https://opendev.org/openstack/tempest/src/commit/ae41052a51f5dbb748eb6bf4f2… [5] https://opendev.org/openstack/tempest/src/commit/ae41052a51f5dbb748eb6bf4f2… Thank you, -- *Vida Haririan* Senior Software Quality Engineer Red Hat EMEA <https://www.redhat.com/>

2 1

[announce][debian][xena] general availability of OpenStack Xena in Debian
by Thomas Goirand 29 Sep '21

29 Sep '21

Hi! It's my pleasure to announce the general availability of OpenStack Xena in Debian. Indeed, I've just finished uploading everything to Debian Experimental at the beginning of this week, and was able yesterday, to pop my first VM over on top of Xena. This marks the first readiness of the release, from the Debian point of view (even though there probably will be things to fix, like after addressing https://bugs.launchpad.net/oslo.policy/+bug/1945336 for example, and uploading the final release to replace the RCs when that becomes available). The Bullseye backports are available the usual way, for example using extrepo: apt-get install extrepo extrepo enable openstack_xena apt-get update ... or directly setting-up the http://bullseye-xena.debian.net repository the usual way. I'm currently in the process of uploading all of Xena to Debian Unstable (but please bare with me: there's 225+ packages in this update). Note that Xena can already be installed by OCI [1]. Please report any issue you may find, on OCI or on the Debian packages, through the Debian BTS as usual. Cheers, Thomas Goirand (zigo) [1] https://salsa.debian.org/openstack-team/debian/openstack-cluster-installer

1 0

[kolla][neutron][victoria] metering agent on compute nodes
by Tomasz Rutkowski 29 Sep '21

29 Sep '21

Hi All, I have issue with neutron-metering-agent - on network node it works ok, however on compute nodes the list of routers is constantly empty (and l3-agent works normal)... how to debug why outside network node the metering-agent does not recieve list of routers? with best regards -- Tomasz Rutkowski

2 2

[all] Gate status: All devstack based jobs are failing for placement url issue in devstack
by Ghanshyam Mann 29 Sep '21

29 Sep '21

Hello Everyone, All the devstack based jobs are failing with placement URL issue in devstack. The root cause is not known yet. You will see the below error in devstack/summary log- + ./stack.sh:main:1455 : /usr/local/bin/nova-status --config-file /etc/nova/nova.conf upgrade check Placement logs: INFO placement.requestlog [None req-3ad7ab8b-7c85-477a-b156-f5ca652a38cc service placement] 10.208.226.48 "GET /placemen//" status: 404 len: 162 microversion: 1.0 Please do not recheck until the issue is fixed. -gmann

5 8

OpenInfra Live - September 30th at 9am CT
by Erin Disney 28 Sep '21

28 Sep '21

Hi everyone, This week’s OpenInfra Live episode is brought to you by the OpenStack and Ceph communities. Ceph is a highly scalable distributed-storage open source solution offering object, block, and file storage. Join us as various Community members discuss the basics, ongoing development and integration of OpenStack with Ceph. Date and time: September 30th at 9am CT (1400 UTC) You can watch us live on: YouTube: https://www.youtube.com/watch?v=zJVoleSpSOk LinkedIn: https://www.linkedin.com/video/event/urn:li:ugcPost:6846862917225320448/ Facebook: https://www.facebook.com/104139126308032/posts/4364096940312208/ WeChat: Recording will be posted on OpenStack WeChat after the live stream Speakers: Kendall Nelson (OpenInfra Foundation) John Fulton (Red Hat) Francesco Pantano (Red Hat) Tom Barron (Red Hat) Loan Harrouin (Societe Generale) Pawel Sadowski (OVH) Mohammed Naser (VEXXHOST) Have an idea for a future episode? Share it now at ideas.openinfra.live <https://openinfrafoundation.formstack.com/forms/openinfralive>. Register now for OpenInfra Live: Keynotes, a special edition of OpenInfra Live on November 17-18th starting at 1500 UTC: https://openinfralivekeynotes.eventbrite.com/ Thanks, Erin Erin Disney Event Marketing Open Infrastructure Foundation

1 0

Issue on accessing object storage containers from outside
by Michel Niyoyita 28 Sep '21

28 Sep '21

Dear all I have created containers and upload objects into containers . I can access these containers and objects trough horizon dashboard but once make them public I can not reach them through the provided link . If any one have an idea on how to access containers and their object using the link from outside he can help and guide. I deployed openstack wallaby using kolla-ansible and ceph pacific using ansible , both are using ubuntu 20.04 as OS Michel

3 4

[openstack-qa] opendev.org mirrors
by Eduardo Santos 28 Sep '21

28 Sep '21

Hi folks, On US-based servers, a DevStack deploy takes 15 minutes. When deploying it from Brazil (where I live) however, it takes more than an hour. Examining the stack.sh logs, git_timed appears to be the culprit. I'm using a 250 Mbps connection. Do opendev.org mirrors exist for other regions other than the US? If not, has this been discussed yet? I believe this is more of a financial discussion othet than technical, but I'm curious if other users have raised this question before. I'm not sure if openstack-qa is the most appropriate flag. If not, please feel free to correct. Thanks!

3 3

Creating multiple servers with a shared volume
by open infra 28 Sep '21

28 Sep '21

Hi, I am planning to spin hundreds of servers with a shared storage. I have created one instance and attached a 1 TB volume (multiattach) and created a snapshot of the instance. But when I spin another server using the snapshot, I noticed that instead of expected shared volume, individual 1 TB size additional (dedicated) volume has been attached to each VM. Is there a mechanism where I can create multiple instances along with a shared volume attached to the instance instead of attaching later? Regards, Danishka

3 5

[Tripleo] Support of Alma/Rocky Linux
by Anirudh Gupta 28 Sep '21

28 Sep '21

Hi Team As per the document, the supporting OS mentioned in the documentation is only Centos Stream and No Stream version (as of now) Since Centos has officially stopped releasing new versions and looking for EOL. I Just wanted to check if there is a possibility that we can install Tripleo on a specific version of Alma Linux or Rocky Linux Regards Anirudh Gupta

2 3

RE: Problems with Fluentd buffer [fluentd][kolla-ansible]
by Adam Tomas 28 Sep '21

28 Sep '21

Hi, I’ve turned off „central_logging” and now I have only several errors like below on all nodes (3-7 per day) 0 failed to flush the buffer. retry_time=0 next_retry_seconds=2021-09-27 04:49:32.728326628 +0200 chunk="5ccf11fdc0d6876abdef813211371285" error_class=RestClient::RequestTimeout error="408 Request Timeout” On compute node, which had errors like this : /var/log/kolla/fluentd/fluentd.log:2021-09-27 06:41:38 +0200 [warn]: #0 emit transaction failed: error_class=Fluent::Plugin::Buffer::BufferOverflowError error="buffer space has too many data" location="/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/buffer.rb:293:in `write'" tag=„openstack_python" I’ve restarted fluentd container on that node and now it’s not showing errors like this anymore. So what’s the problem? Performance of Monasca/Kibana/Elasticsearch? But the node (telemetry) on which this stack is running is not overloaded - 20-30% CPU (24 cores on that node) , 40% RAM (there is 64GB), a lot of space on SSD. Best regards Adam Tomas > Hi, > after some time of using newly deployed kolla-ansible I have fluentd errors: > > on all controllers, storage and network nodes: > > /var/log/kolla/fluentd/fluentd.log:2021-09-27 01:40:46 +0200 [warn]: #0 failed to flush the buffer. retry_time=0 next_retry_seconds=2021-09-27 01:40:47.847564807 +0200 chunk="5ccee7b1762157f0ae10dcbeb4e14fd8" error_class=RestClient::GatewayTimeout error="504 Gateway Timeout" > /var/log/kolla/fluentd/fluentd.log:2021-09-27 04:49:31 +0200 [warn]: #0 failed to flush the buffer. retry_time=0 next_retry_seconds=2021-09-27 04:49:32.728326628 +0200 chunk="5ccf11fdc0d6876abdef813211371285" error_class=RestClient::RequestTimeout error="408 Request Timeout” > > > on compute nodes: > > /var/log/kolla/fluentd/fluentd.log:2021-09-27 06:41:38 +0200 [warn]: #0 emit transaction failed: error_class=Fluent::Plugin::Buffer::BufferOverflowError error="buffer space has too many data" location="/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/buffer.rb:293:in `write'" tag="openstack_python" > /var/log/kolla/fluentd/fluentd.log:2021-09-27 06:41:38 +0200 [warn]: #0 emit transaction failed: error_class=Fluent::Plugin::Buffer::BufferOverflowError error="buffer space has too many data" location="/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/buffer.rb:293:in `write'" tag="kolla.var.log.kolla.monasca.agent-statsd.log" > /var/log/kolla/fluentd/fluentd.log:2021-09-27 06:41:38 +0200 [warn]: #0 emit transaction failed: error_class=Fluent::Plugin::Buffer::BufferOverflowError error="buffer space has too many data" location="/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/buffer.rb:293:in `write'" tag="fluent.warn" > /var/log/kolla/fluentd/fluentd.log:2021-09-27 06:41:38 +0200 [error]: #0 failed to emit fluentd's log event tag="fluent.warn" event={"action"=>:throw_exception, "message"=>"failed to write data into buffer by buffer overflow action=:throw_exception"} error_class=Fluent::Plugin::Buffer::BufferOverflowError error="buffer space has too many data" > /var/log/kolla/fluentd/fluentd.log:2021-09-27 06:41:38 +0200 [warn]: #0 emit transaction failed: error_class=Fluent::Plugin::Buffer::BufferOverflowError error="buffer space has too many data" location="/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/buffer.rb:293:in `write'" tag="fluent.warn" > /var/log/kolla/fluentd/fluentd.log:2021-09-27 06:41:38 +0200 [error]: #0 failed to emit fluentd's log event tag="fluent.warn" event={"error"=>"#<Fluent::Plugin::Buffer::BufferOverflowError: buffer space has too many data>", "location"=>"/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/buffer.rb:293:in `write'", "tag"=>"openstack_python", "message"=>"emit transaction failed: error_class=Fluent::Plugin::Buffer::BufferOverflowError error=\"buffer space has too many data\" location=\"/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/buffer.rb:293:in `write'\" tag=\"openstack_python\""} error_class=Fluent::Plugin::Buffer::BufferOverflowError error="buffer space has too many data" > /var/log/kolla/fluentd/fluentd.log:2021-09-27 06:41:38 +0200 [warn]: #0 emit transaction failed: error_class=Fluent::Plugin::Buffer::BufferOverflowError error="buffer space has too many data" location="/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/buffer.rb:293:in `write'" tag="fluent.warn" > /var/log/kolla/fluentd/fluentd.log:2021-09-27 06:41:38 +0200 [error]: #0 failed to emit fluentd's log event tag="fluent.warn" event={"error"=>"#<Fluent::Plugin::Buffer::BufferOverflowError: buffer space has too many data>", "location"=>"/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/buffer.rb:293:in `write'", "tag"=>"kolla.var.log.kolla.monasca.agent-statsd.log", "message"=>"emit transaction failed: error_class=Fluent::Plugin::Buffer::BufferOverflowError error=\"buffer space has too many data\" location=\"/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/buffer.rb:293:in `write'\" tag=\"kolla.var.log.kolla.monasca.agent-statsd.log\""} error_class=Fluent::Plugin::Buffer::BufferOverflowError error="buffer space has too many data" > /var/log/kolla/fluentd/fluentd.log:2021-09-27 06:41:38 +0200 [warn]: #0 emit transaction failed: error_class=Fluent::Plugin::Buffer::BufferOverflowError error="buffer space has too many data" location="/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/buffer.rb:293:in `write'" tag="openstack_python" > /var/log/kolla/fluentd/fluentd.log:2021-09-27 06:41:38 +0200 [warn]: #0 emit transaction failed: error_class=Fluent::Plugin::Buffer::BufferOverflowError error="buffer space has too many data" location="/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/buffer.rb:293:in `write'" tag="fluent.warn" > /var/log/kolla/fluentd/fluentd.log:2021-09-27 06:41:38 +0200 [warn]: #0 emit transaction failed: error_class=Fluent::Plugin::Buffer::BufferOverflowError error="buffer space has too many data" location="/opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/buffer.rb:293:in `write'" tag="kolla.var.log.kolla.monasca.agent-forwarder.log" > /var/log/kolla/fluentd/fluentd.log:2021-09-27 06:41:38 +0200 [error]: #0 failed to emit fluentd's log event tag="fluent.warn" event={"action"=>:throw_exception, "message"=>"failed to write data into buffer by buffer overflow action=:throw_exception"} error_class=Fluent::Plugin::Buffer::BufferOverflowError error="buffer space has too many data” > > and on monitoring node: > > 2021-09-27 14:40:50 +0200 [warn]: #0 buffer flush took longer time than slow_flush_log_threshold: elapsed_time=84.05643947119825 slow_flush_log_threshold=20.0 plugin_id="object:bff4" > 2021-09-27 14:42:53 +0200 [warn]: #0 buffer flush took longer time than slow_flush_log_threshold: elapsed_time=122.65328024700284 slow_flush_log_threshold=20.0 plugin_id="object:bff4" > 2021-09-27 14:44:15 +0200 [warn]: #0 buffer flush took longer time than slow_flush_log_threshold: elapsed_time=82.32426812895574 slow_flush_log_threshold=20.0 plugin_id="object:bff4" > 2021-09-27 14:46:05 +0200 [warn]: #0 Bulk sending messages to monasca-api threw exception exceptionew=#<RestClient::Exceptions::ReadTimeout: Timed out reading data from server> > 2021-09-27 14:46:05 +0200 [warn]: #0 failed to flush the buffer. retry_time=0 next_retry_seconds=2021-09-27 14:46:06.042150289 +0200 chunk="5ccf89c9ec54b09e616f7167d5f93cc1" error_class=RestClient::Exceptions::ReadTimeout error="Timed out reading data from server" > 2021-09-27 14:46:05 +0200 [warn]: #0 suppressed same stacktrace > 2021-09-27 14:47:58 +0200 [warn]: #0 buffer flush took longer time than slow_flush_log_threshold: elapsed_time=112.90601075813174 slow_flush_log_threshold=20.0 plugin_id="object:bff4" > 2021-09-27 14:47:58 +0200 [warn]: #0 retry succeeded. chunk_id="5ccf89c9ec54b09e616f7167d5f93cc1" > 2021-09-27 14:50:16 +0200 [warn]: #0 buffer flush took longer time than slow_flush_log_threshold: elapsed_time=136.24769522389397 slow_flush_log_threshold=20.0 plugin_id="object:bff4" > 2021-09-27 14:52:27 +0200 [warn]: #0 buffer flush took longer time than slow_flush_log_threshold: elapsed_time=129.86474119895138 slow_flush_log_threshold=20.0 plugin_id="object:bff4" > 2021-09-27 14:54:08 +0200 [warn]: #0 buffer flush took longer time than slow_flush_log_threshold: elapsed_time=100.71324555086903 slow_flush_log_threshold=20.0 plugin_id="object:bff4" > 2021-09-27 14:56:31 +0200 [warn]: #0 Bulk sending messages to monasca-api threw exception exceptionew=#<RestClient::Exceptions::ReadTimeout: Timed out reading data from server> > >

2 1

key_repository does not exist or Keystone does not have sufficient permission to access it [kolla-ansible][keystone]
by Adam Tomas 28 Sep '21

28 Sep '21

Hi, some next problem with my kolla-ansible deployment. On controller nodes I have (several times per hour) error messages like below: ERROR keystone.common.fernet_utils [req-386e223e-5f08-4959-bd0b-3dbbfd3b534f 73c7a9a7175e46e9971181963267225b efa5d1ce80324476b4ce189686dda2d1 - default default] Either [None] key_repository does not exist or Keystone does not have sufficient permission to access it: /etc/keystone/credential-keys/ I’ve checked - such path doesn’t exist inside keystone containers. I found bug #1863643 (https://bugs.launchpad.net/kolla/+bug/1863643 <https://bugs.launchpad.net/kolla/+bug/1863643>) but I don’t see any progress on it nor information that victoria is not affected (so I assume it’s the same bug and Victoria is affected?). What should be done to get rid of this errors? Best regards Adam Tomas

1 0

[cinder] reminder: this week's meeting in video+IRC
by Brian Rosmaita 28 Sep '21

28 Sep '21

Quick reminder that this week's Cinder team meeting on Wednesday 29 September, being the final meeting of the month, will be held in both videoconference and IRC at the regularly scheduled time of 1400 UTC. These are the video meeting rules we've agreed to: * Everyone will keep IRC open during the meeting. * We'll take notes in IRC to leave a record similar to what we have for our regular IRC meetings. * Some people are more comfortable communicating in written English. So at any point, any attendee may request that the discussion of the current topic be conducted entirely in IRC. * The meeting will be recorded. connection info: https://bluejeans.com/3228528973 meeting agenda: https://etherpad.opendev.org/p/cinder-xena-meetings cheers, brian

1 0

[qa] Office Hour Sep 28th is cancelled
by Martin Kopec 28 Sep '21

28 Sep '21

Hi everyone, the next QA Office Hour on September 28th at 2pm UTC is cancelled due to a public holiday in my country and I won't be able to moderate it. Thank you for your understanding, -- Martin Kopec Senior Software Quality Engineer Red Hat EMEA

1 0

[OSSA-2021-002] Nova: Open Redirect in noVNC proxy (CVE-2021-3654)
by Jeremy Stanley 27 Sep '21

27 Sep '21

=========================================== OSSA-2021-002: Open Redirect in noVNC proxy =========================================== :Date: July 29, 2021 :CVE: CVE-2021-3654 Affects ~~~~~~~ - Nova: <21.2.3, >=22.0.0 <22.2.3, >=23.0.0 <23.0.3 Description ~~~~~~~~~~~ Swe Aung, Shahaan Ayyub, and Salman Khan with the Monash University Cyber Security team reported a vulnerability affecting Nova's noVNC proxying implementation which exposed access to a well-known redirect behavior in the Python standard library's http.server.SimpleHTTPRequestHandler and thus noVNC's WebSockifyRequestHandler which uses it. By convincing a user to follow a specially-crafted novncproxy URL, the user could be redirected to an unrelated site under control of the attacker in an attempt to convince them to divulge credentials or other sensitive data. All Nova deployments with novncproxy enabled are affected. Errata ~~~~~~ The initial fix did not take into account the possibility of bypass using exactly three slashes. This update provides a more thorough revised fix for the issue. The affected versions list has been updated to indicate versions expected to include the newer solution. Patches ~~~~~~~ - https://review.opendev.org/791807 (Train) - https://review.opendev.org/806629 (errata 1) (Train) - https://review.opendev.org/791806 (Ussuri) - https://review.opendev.org/806628 (errata 1) (Ussuri) - https://review.opendev.org/791805 (Victoria) - https://review.opendev.org/806626 (errata 1) (Victoria) - https://review.opendev.org/791577 (Wallaby) - https://review.opendev.org/805818 (errata 1) (Wallaby) - https://review.opendev.org/791297 (Xena) - https://review.opendev.org/805654 (errata 1) (Xena) Credits ~~~~~~~ - Swe Aung from Monash University Cyber Security team (CVE-2021-3654) - Shahaan Ayyub from Monash University Cyber Security team (CVE-2021-3654) - Salman Khan from Monash University Cyber Security team (CVE-2021-3654) References ~~~~~~~~~~ - https://launchpad.net/bugs/1927677 - https://bugs.python.org/issue32084 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-3654 Notes ~~~~~ - The stable/train branch is under extended maintenance and will receive no new point releases, but a patch for it is provided as a courtesy. OSSA History ~~~~~~~~~~~~ - 2021-09-27 - Errata 1 - 2021-07-29 - Original Version -- Jeremy Stanley

1 0

Not registering Hypervisors
by Karera Tony 27 Sep '21

27 Sep '21

Hello Team, I have deployed Openstack Victoria using Kolla-ansible on Ubuntu 20.04 and ceph as the backend storage for Nova, Cinder and Glance. It finished with no error but it has failed to register any on the Compute Nodes under Hypervisors. Any idea on how to resolve this ? Regards Tony Karera

5 22

[neutron] upgrading port binding:profile.allocation value for port-resource-request-groups API extension
by Balazs Gibizer 27 Sep '21

27 Sep '21

Hi Neutrinos! We found a technical challenge during implementing the port-resource-request-groups API extension[1]. That extension changes the format of the port.resoruce_request as well as the format of the port.binding:profile.allocation. The former is a calculated field on the port so that is easy. However the bindig:profile is persisted in the database so data migration is needed. What is the canonical way to do such DB data translation in Neutron? Can we translate the data in place during alembic migration? Or should we do some kind of online data migration when the data is translated by neutron when it is read from the db? cheers, gibi [1] https://review.opendev.org/c/openstack/neutron/+/805637/5

2 6