April 2021 - openstack-discuss - lists.openstack.org

[OSSN-0089] Missing configuration option in Secure Live Migration guide leads to unencrypted traffic
by Josephine Seifert 13 Apr '21

13 Apr '21

Missing configuration option in Secure Live Migration guide leads to unencrypted traffic -------------------------------------------------------------------------------------------------------------------- ### Summary ### The guide to enable secure live migration with QEMU-native tls on nova compute nodes missed an important config option. Without this option a hard-coded part in nova is triggerd which sets the default route to TCP instead of TLS. This leads to an unecrypted migration of the ram without throwing any kind of Error. ### Affected Services / Software ### Nova / Victoria, Ussuri, Train, Stein (might also be affected: Rocky, Queens, Pike, Ocata) ### Discussion ### In the OpenStack guide to setup secure live migration with QEMU-native tls there are a few configuration options given, which have to be applied to nova compute nodes. After following the instructions and setting up everything it seems to work as expected. But after checking that libvirt is able to use tls using tcpdump to listen on the port for tls while manually executing libvirt commands, the same check for live migration of an instance through openstack fails. Listening on the port for unencrypted tcp-traffic shows that OpenStack still uses the unencrypted TCP path instead of the TLS one for the migration. The reason for this is a patch from Ocata which adds the calculation of the live-migration-uri in code: https://review.opendev.org/c/openstack/nova/+/410817/ The config parameter ``live_migration_uri`` was deprecated in favor of ``live_migration_scheme`` and the default set to tcp. This leads to the problem that if none of these two config options are set, libvirt will always use the default tcp connection. To enable QEMU-native TLS to be used in nova one of them has to be set so that a TLS connection can be established. Currently the guide does not show that this is necessary and there was no other documentation indicating that these config options are important for the usage of QEMU-native TLS. As there is no documentation which recognizes this and it is hard to find this problem as the migration happens even without those config option set - not stating that it is still unencrypted, it might have been unrecognized in various deployments, which followed the guide. ### Recommended Actions ### For deployments using secure live migration with QEMU-native TLS: 1. Check the config of all nova compute nodes. The ``libvirt`` section needs to have either ``live_migration_uri`` (deprecated) or ``live_migration_scheme`` configured. 2. If neither of those config options are present, add ``live_migration_scheme = tls`` to enable the use of the tls connection. #### Patches #### The guide for secure live migration was updated to reflect the necessary configuration options and now has a note, which warns users that not setting all config options may lead into a seemingly working deployment, which still uses unencrypted traffic for the ram-migration. Master(Wallaby): https://review.opendev.org/c/openstack/nova/+/781030 Victoria: https://review.opendev.org/c/openstack/nova/+/781211 Ussuri: https://review.opendev.org/c/openstack/nova/+/782126 Train: https://review.opendev.org/c/openstack/nova/+/782430 Stein: https://review.opendev.org/c/openstack/nova/+/783199 ### Contacts / References ### Author: Josephine Seifert, secustack GmbH This OSSN : https://wiki.openstack.org/wiki/OSSN/OSSN-0089 Original LaunchPad Bug : https://bugs.launchpad.net/nova/+bug/1919357 Mailing List : [Security] tag on openstack-discuss(a)lists.openstack.org OpenStack Security Project : https://launchpad.net/~openstack-ossg

1 0

[ironic] Meetings next week (19-23 April)
by Iury Gregory 13 Apr '21

13 Apr '21

Hello ironicers, Since next week is the PTG, we decided during our upstream meeting today to skip the following meetings: - Upstream Meeting (Monday) - Review Jams (Monday/Tuesday) - SPUC in the APAC time (Friday) because it overlaps with the PTG. Thank you =) -- *Att[]'sIury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Part of the ironic-core and puppet-manager-core team in OpenStack* *Software Engineer at Red Hat Czech* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory(a)gmail.com <iurygregory(a)gmail.com>*

2 1

[ceilometer][octavia][Victoria] No metrics from octavia loadbalancer
by Adam Tomas 13 Apr '21

13 Apr '21

Hi, Im trying to get metrics from octavia’s load balancer, but can’t get any (gnocchi metric list | grep loadbalancer not returning anything). How should I configure ceilometer to get metrics from octavia? Ceilometer asks neutron for load balancer metrics, and neutron responses „resource cannot be found” (and that is obvious, because Neutron LBaaS service is deprecated). How to force neutron to get these resources from Octavia? I’ve tried to use [service_providers] service_provider = LOADBALANCERV2:Octavia:neutron_lbaas.drivers.octavia.driver.OctaviaDriver:default in neutron.conf, but it doesn’t work either… Best regards Adam

2 4

[tacker] irc meeting
by Yasufumi Ogawa 13 Apr '21

13 Apr '21

Hi team, I'll be off today's irc meeting, so please skip, or someone host the meeting if any topic. Thanks, Yasufumi

1 0

[Manila][Interop] request for 15-30 min on Xena PTG for Interop
by Kanevsky, Arkady 13 Apr '21

13 Apr '21

Goutham, As chair of OpenStack Interop WG, Interop/Refstack would like 15-30 min on PTG meeting to go over Interop testing and any changes for Manila tempest or tempest configuration in Wallaby cycle or changes planned for Xena. Once on agenda one of the Interop WG person will attend and lead the discussion. Thanks, Arkady Arkady Kanevsky, Ph.D. SP Chief Technologist & DE Dell Technologies office of CTO Dell Inc. One Dell Way, MS PS2-91 Round Rock, TX 78682, USA Phone: 512 7204955

2 1

[ironic] No Review Jam Tomorrow
by Iury Gregory 13 Apr '21

13 Apr '21

Hello ironicers! During the upstream meeting today we decided to skip the Review Jam that will happen tomorrow, since we don't have any topics that would require attention. We also skipped the review jam from today (we totally forgot about it). -- *Att[]'sIury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Part of the ironic-core and puppet-manager-core team in OpenStack* *Software Engineer at Red Hat Czech* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory(a)gmail.com <iurygregory(a)gmail.com>*

1 0

[oslo][security-sig] Please revisit your open vulnerability report
by Jeremy Stanley 12 Apr '21

12 Apr '21

Please help the OpenStack Vulnerability Management Team by taking a look at the following report: keystonemiddleware connections to memcached from neutron-server grow beyond configured values https://launchpad.net/bugs/1883659 Can it be exploited by a nefarious actor, and if so, how? Is it likely to be fixable in all our supported stable branches, respecting stable backport policy? What deployment configurations and options might determine whether a particular installation is susceptible? This is the sort of feedback we depend on to make determinations regarding whether and how to keep the public notified, so they can make informed decisions. Thanks for doing your part to keep our users safe! -- Jeremy Stanley

2 6

[ops][cinder][nova] os-brick upcoming releases
by Brian Rosmaita 12 Apr '21

12 Apr '21

Hello operators, You may have heard about a potential data-loss bug [0] that was recently discovered. It has been fixed in the upcoming wallaby release and we are planning to backport to all stable branches and do new os-brick releases from the releasable stable branches. In the meantime, the bug occurs if the multipath configuration option on a compute is changed while volumes are attached to instances on that compute. The possible data loss may occur when the volumes are detached (migration, volume-detach, etc.). Thus, before the new os-brick releases are available, the issue can nonetheless be averted by not making such a configuration change under those circumstances. The new os-brick releases will be: - victoria: 4.0.3 - ussuri: 3.0.6 - train: 2.10.6 The stein, rocky, and queens branches are in Extended Maintenance mode and are no longer released from, but critical fixes are backported to them when possible, though it may take a while before these are merged. [0] https://launchpad.net/bugs/1921381

1 1

[sdk]: compute service create_server method, how to create multiple servers
by dmeng 12 Apr '21

12 Apr '21

Hello there, Hope this email finds you well. We are currently using the openstacksdk for developing our product, and have a question about the openstacksdk compute service create_server() method. We are wondering if the "max_count" attribute is supported by the create_server() method? We tried to create multiple servers by setting the max_count value, but only one server has been created. While when we use the python-novaclient package, novaclient.servers.create() method has the attribute max_count which allows creating multiple servers at once if set the value. So we would like to know is there a similar attribute like "max_count" that could allow us to create multiple servers at once in openstacksdk? Thanks and have a great day! Catherine

3 2

Re: [glance][openstack-ansible] Snapshots disappear during saving
by Oliver Wenz 12 Apr '21

12 Apr '21

Hi Dmitriy, I checked nginx logs on the keystone container and there was no obvious error: ontainer-6f64e2e1 nginx: 192.168.110.211 - - [12/Apr/2021:13:25:56 +0000] "HEAD / HTTP/1.0" 300 0 "-" "osa-haproxy-healthcheck" Apr 12 13:26:07 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:07 +0000] "POST /v3/auth/tokens HTTP/1.1" 201 316 "-" "openstack_auth keystoneauth1/4.2.1 python-requests/2.23.0 CPython/3.8.5" Apr 12 13:26:07 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:07 +0000] "POST /v3/auth/tokens HTTP/1.1" 401 109 "-" "openstack_auth keystoneauth1/4.2.1 python-requests/2.23.0 CPython/3.8.5" Apr 12 13:26:07 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:07 +0000] "GET /v3/users/e4e88b0e800e4a79905a586738c32bf1/projects HTTP/1.1" 200 768 "-" "python-keystoneclient" Apr 12 13:26:08 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:08 +0000] "POST /v3/auth/tokens HTTP/1.1" 201 6779 "-" "openstack_auth keystoneauth1/4.2.1 python-requests/2.23.0 CPython/3.8.5" Apr 12 13:26:08 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:08 +0000] "GET / HTTP/1.1" 300 270 "-" "python-novaclient keystoneauth1/4.2.1 python-requests/2.23.0 CPython/3.8.5" Apr 12 13:26:08 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:08 +0000] "POST /v3/auth/tokens HTTP/1.1" 201 6779 "-" "python-novaclient keystoneauth1/4.2.1 python-requests/2.23.0 CPython/3.8.5" Apr 12 13:26:08 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:08 +0000] "GET / HTTP/1.1" 300 270 "-" "python-novaclient keystoneauth1/4.2.1 python-requests/2.23.0 CPython/3.8.5" Apr 12 13:26:08 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:08 +0000] "POST /v3/auth/tokens HTTP/1.1" 201 6779 "-" "python-novaclient keystoneauth1/4.2.1 python-requests/2.23.0 CPython/3.8.5" Apr 12 13:26:08 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.211 - - [12/Apr/2021:13:26:08 +0000] "HEAD / HTTP/1.0" 300 0 "-" "osa-haproxy-healthcheck" Apr 12 13:26:08 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.100 - - [12/Apr/2021:13:26:08 +0000] "GET /v3/auth/tokens HTTP/1.1" 200 6779 "-" "python-keystoneclient" Apr 12 13:26:18 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.28 - - [12/Apr/2021:13:26:18 +0000] "GET /v3/auth/tokens HTTP/1.1" 200 6779 "-" "python-keystoneclient" Apr 12 13:26:18 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.100 - - [12/Apr/2021:13:26:18 +0000] "GET /v3/auth/tokens HTTP/1.1" 200 6779 "-" "python-keystoneclient" Apr 12 13:26:19 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:19 +0000] "GET /v3/users/e4e88b0e800e4a79905a586738c32bf1/projects HTTP/1.1" 200 768 "-" "python-keystoneclient" Apr 12 13:26:20 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:20 +0000] "GET /v3 HTTP/1.1" 200 255 "-" "openstack_dashboard keystoneauth1/4.2.1 python-requests/2.23.0 CPython/3.8.5" Apr 12 13:26:20 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.211 - - [12/Apr/2021:13:26:20 +0000] "HEAD / HTTP/1.0" 300 0 "-" "osa-haproxy-healthcheck" Apr 12 13:26:21 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:21 +0000] "GET /v3/users/e4e88b0e800e4a79905a586738c32bf1/projects HTTP/1.1" 200 768 "-" "python-keystoneclient" Apr 12 13:26:24 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:24 +0000] "GET / HTTP/1.1" 300 270 "-" "python-novaclient keystoneauth1/4.2.1 python-requests/2.23.0 CPython/3.8.5" Apr 12 13:26:24 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:24 +0000] "POST /v3/auth/tokens HTTP/1.1" 201 6779 "-" "python-novaclient keystoneauth1/4.2.1 python-requests/2.23.0 CPython/3.8.5" Apr 12 13:26:25 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.100 - - [12/Apr/2021:13:26:25 +0000] "GET /v3/auth/tokens HTTP/1.1" 200 6779 "-" "python-keystoneclient" Apr 12 13:26:27 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:27 +0000] "GET / HTTP/1.1" 300 270 "-" "python-novaclient keystoneauth1/4.2.1 python-requests/2.23.0 CPython/3.8.5" Apr 12 13:26:28 infra1-keystone-container-6f64e2e1 nginx[13641]: infra1-keystone-container-6f64e2e1 nginx: 192.168.110.250 - - [12/Apr/2021:13:26:28 +0000] "POST /v3/auth/tokens HTTP/1.1" 201 6779 "-" "python-novaclient keystoneauth1/4.2.1 python-requests/2.23.0 CPython/3.8.5" I also noticed, that glance logs show the keystone_authtoken message even when I successfully create a snapshot (e.g. for a cirros instance). So could this be a nova problem afterall? I'm confused why there's a NotImplementedError in the nova logs: http://paste.openstack.org/show/804398/ Kind regards, Oliver > ------------------------------ > > Message: 2 > Date: Fri, 09 Apr 2021 08:24:53 +0300 > From: Dmitriy Rabotyagov <noonedeadpunk(a)ya.ru> > To: "openstack-discuss(a)lists.openstack.org" > <openstack-discuss(a)lists.openstack.org> > Subject: Re: [glance][openstack-ansible] Snapshots disappear during > saving > Message-ID: <500221617945664(a)mail.yandex.ru> > Content-Type: text/plain; charset="utf-8" > > An HTML attachment was scrubbed... > URL: > <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210409…> >

1 0