February 2021 - openstack-discuss

[heat] [kolla] Stack creation fails
by Braden, Albert 24 Feb '21

24 Feb '21

I'm using a heat template to try and build an Openstack cluster on VMs and it is failing in 2 different ways. It starts out fine for both: (openstack) [abraden@chrnc-dev-os-build-01 kolla_multinode]$ os stack create -t kolla_multinode.yaml upgrade-stack +---------------------+----------------------------------------------------------------------------------------+ | Field | Value | +---------------------+----------------------------------------------------------------------------------------+ | id | 7b4a49d2-0418-453a-abde-5e510b588b71 | | stack_name | upgrade-stack | | description | Deploy virtualized multi-node OpenStack and Ceph using kolla-ansible and ceph-ansible. | | | | | creation_time | 2021-02-23T21:17:02Z | | updated_time | None | | stack_status | CREATE_IN_PROGRESS | | stack_status_reason | Stack CREATE started | +---------------------+----------------------------------------------------------------------------------------+ Sometimes it stays in CREATE_IN_PROGRESS status for a long time, and when I look at volumes I see one or more volumes stuck in "creating" status [1]. If I set the stuck volumes to "error" status then heat retries and the cluster builds successfully. Other times the keypair fails. First it goes to DELETE_COMPLETE status and then to CREATE_FAILED [2], and the the stack goes to CREATE_FAILED status. When this happens no volumes are created, and I haven't found a workaround for this failure. Where should I be looking for the issue? Nothing in the log is jumping out at me. RMQ and keystone seem fine. Searching for the id of the failed stack or the stuck volume in /var/log/kolla/* doesn't find any errors or warnings. [1] volumes<https://paste.ubuntu.com/p/ZBhn7TQ2Hv/> [2] failed key<https://paste.ubuntu.com/p/gGwDRyT2Nd/> I apologize for the nonsense below. I have not been able to stop it from being attached to my external emails. E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

1 1

[security][security sig] No meeting tomorrow
by Gage Hugo 24 Feb '21

24 Feb '21

Hello everyone, The security SIG meeting for tomorrow is cancelled, we plan on meeting next week at the usual time. Thanks

1 0

[victoria][neutron] qos-fip extension cannot be enabled with OVN
by Malik Obaid 24 Feb '21

24 Feb '21

Hi, I am using Openstack Victoria release on Ubuntu 20.04. While configuring QOS policy I am facing 2 issues. 1. The command 'openstack extension list' does not show qos-fip, despite qos-fip having been added to the list of extension_drivers. 2. An error is raised when trying to assign a qos-policy to a FIP: BadRequestException: 400: Client Error for url: http://172.16.30.46:9696/v2.0/floatingips/774dd653-3a9b-40d7-ade6-7cf5a3a4b…, Unrecognized attribute(s) 'qos_policy_id' I would really appreciate any help in this regard. Thank you. Regards, Malik Obaid

2 1

[magnum][ingress-nginx][cert-manager] issuing letsencrypt certs
by Ionut Biru 24 Feb '21

24 Feb '21

Hi guys, Not sure if we need some new policy on deploying kubernetes 1.18+ but I'm kinda stuck and I don't know what else to do. I'm trying in the past days to generate letsencrypt ssl for a simple nginx deployment I'm deploying Kubernetes 1.19.8 or 1.20.4 cluster on Openstack using Magnum without any ingress controller. I'm deploying ingress-nginx 0.44 using: wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.44… Cert-manager 1.2.0 wget https://github.com/jetstack/cert-manager/releases/download/v1.2.0/cert-mana… I'm creating a nginx deployment and service with 2 replicas https://paste.xinu.at/PWu/ Configure ingress for my host: https://paste.xinu.at/7c7FH/ Configure the issuer: https://paste.xinu.at/Bf6/ Reconfigure ingress: https://paste.xinu.at/o1j5wD/ kubectl apply -f deploy.yaml kubectl apply -f cert-manager.yaml kubectl apply -f nginx-deployment.yaml kubectl apply -f ioni_ingress.yaml kubectl apply -f prod_issuer.yaml Error in events: Error presenting challenge: pods "cm-acme-http-solver-" is forbidden: PodSecurityPolicy: unable to admit pod: [] Error in kubernetes logs(ssh on node) https://paste.xinu.at/9aMJ/ -- Ionut Biru - https://fleio.com

1 0

[ops] Where is the instance Action Log stored and how to externally access it and send to ELK?
by Linebarger, John 24 Feb '21

24 Feb '21

Where is the instance Action Log stored (the one you can see on the Horizon dashboard if you select an instance) and how can it be externally accessed, either by CLI or HTTPS or in any other way? It doesn't seem to map to anything stored in the Nova logs in /var/log/nova, at least that I can see. We want to send instance Action Logs to an ELK stack for indexing and display, but are not yet able to figure out how to do so. John M. Linebarger, PhD, MBA Principal Member of Technical Staff Sandia National Laboratories (Work) 505-845-8282

2 1

[all][qa] Gate status of stable stein's legacy jobs with tempest plugins
by Ghanshyam Mann 24 Feb '21

24 Feb '21

Hello Everyone, We recently pinned the Tempest version 26.0.0 in devstack for stable/stein branch testing.[1]. In CI/CD workflow tempest venv can be re-created many times in devstack, devstack-gate (for legacy jobs) or run-tempest roles etc. With the Tempest 26.0.0 version, we need to make tempest venv to use the stable constraints instead of master constraints. And used constraints should be consistent otherwise tempest venv gets recreated which end up removing the previously installed tempest plugins. We have taken care of constraints consistency for zuulv3 native jobs in run-tempest ansible role but for the legacy job which uses the devstack-gate path still use the master constraints and recreate the tempest venv. So your legacy job which uses the tempest plugins will be failing. Example: designate-devstack-pdns4 - https://zuul.opendev.org/t/openstack/build/35fd0a4c76e0482e88ed32ef0840291d I have pushed the fixes which are on devstack, devstack-gate, and tempest(tempest patch is merged). If you are seeing a similar issue please do not recheck and wait for the below patches to merge: - https://review.opendev.org/c/openstack/devstack-gate/+/776722 - https://review.opendev.org/c/openstack/devstack/+/776811 [1] https://review.opendev.org/c/openstack/devstack/+/774103 -gmann

1 1

[glance] Replaced parameters "default_store" and "glance_store" breaks cooperation between cinder and glance in rocky release
by Jan Wasilewski 24 Feb '21

24 Feb '21

Hello, I am writing to you, because we faced an issue with broken cooperation between cinder and glance. In our logs(glance-api) we observed such warning message: 2021-02-22 15:30:02.894 756 WARNING oslo_config.cfg [-] Deprecated: Option "stores" from group "glance_store" is deprecated for removal ( This option is deprecated against new config option ``enabled_backends`` which helps to configure multiple backend stores of different schemes. This option is scheduled for removal in the Stein development cycle. ). Its value may be silently ignored in the future. 2021-02-22 15:30:02.896 756 WARNING oslo_config.cfg [-] Deprecated: Option "default_store" from group "glance_store" is deprecated for removal ( This option is deprecated against new config option ``default_backend`` which acts similar to ``default_store`` config option. This option is scheduled for removal in the Stein development cycle. ). Its value may be silently ignored in the future. And we simply wanted to solve this problem, by changing option glance_store to enabled_backends and default_store to default_backend: @@ -2079,7 +2079,7 @@ pool_timeout = 60 # cycle. #stores = file,http -stores = glance.store.rbd.Store +enabled_backends = glance.store.rbd.Store # DEPRECATED: # The default scheme to use for storing images. @@ -2134,7 +2134,7 @@ stores = glance.store.rbd.Store # cycle. #default_store = file -default_store = rbd +default_backend = rbd But then we realized that we're not able to create volumes from new images, because we hit an issue: 2021-02-22 22:45:08.805 5910 ERROR cinder.scheduler.filter_scheduler [req-fe8e8aa2-dd8a-470b-9d70-dbdc757b48d9 54df39e1b74149bcb8cf9ad3eb957d48 afdfb965515f4724a21b6823764ec36d - default default] Error scheduling 9de37fe8-4f9e-4c29-8cfa-c1a71e8c68b7 from last vol-service: cinder-05@huawei_backend#StoragePool001 : [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task\n result = task.execute(**arguments)\n', u' File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/manager/create_volume.py", line 1059, in execute\n **volume_spec)\n', u' File "/usr/lib/python2.7/dist-packages/cinder/utils.py", line 818, in _wrapper\n return r.call(f, *args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/retrying.py", line 206, in call\n return attempt.get(self._wrap_exception)\n', u' File "/usr/lib/python2.7/dist-packages/retrying.py", line 247, in get\n six.reraise(self.value[0], self.value[1], self.value[2])\n', u' File "/usr/lib/python2.7/dist-packages/retrying.py", line 200, in call\n attempt = Attempt(fn(*args, **kwargs), attempt_number, False)\n', u' File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/manager/create_volume.py", line 959, in _create_from_image\n image_service)\n', u' File "<decorator-gen-238>", line 2, in _prepare_image_cache_entry\n', u' File "/usr/lib/python2.7/dist-packages/cinder/coordination.py", line 151, in _synchronized\n return f(*a, **k)\n', u' File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/manager/create_volume.py", line 757, in _prepare_image_cache_entry\n update_cache=True)\n', u' File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/manager/create_volume.py", line 831, in _create_from_image_cache_or_download\n backend_name) as tmp_image:\n', u' File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__\n return self.gen.next()\n', u' File "/usr/lib/python2.7/dist-packages/cinder/image/image_utils.py", line 832, in fetch\n fetch_verify_image(context, image_service, image_id, tmp)\n', u' File "/usr/lib/python2.7/dist-packages/cinder/image/image_utils.py", line 451, in fetch_verify_image\n None, None)\n', u' File "/usr/lib/python2.7/dist-packages/cinder/image/image_utils.py", line 377, in fetch\n tpool.Proxy(image_file))\n', u' File "/usr/lib/python2.7/dist-packages/cinder/image/glance.py", line 358, in download\n _reraise_translated_image_exception(image_id)\n', u' File "/usr/lib/python2.7/dist-packages/cinder/image/glance.py", line 602, in _reraise_translated_image_exception\n six.reraise(type(new_exc), new_exc, exc_trace)\n', u' File "/usr/lib/python2.7/dist-packages/cinder/image/glance.py", line 356, in download\n image_chunks = self._client.call(context, \'data\', image_id)\n', u' File "/usr/lib/python2.7/dist-packages/cinder/image/glance.py", line 225, in call\n return getattr(controller, method)(*args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/glanceclient/common/utils.py", line 545, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n', u' File "/usr/lib/python2.7/dist-packages/glanceclient/v2/images.py", line 208, in data\n resp, body = self.http_client.get(url)\n', u' File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 328, in get\n return self.request(url, \'GET\', **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 349, in request\n return self._handle_response(resp)\n', u' File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 98, in _handle_response\n raise exc.from_response(resp, resp.content)\n', u'HTTPInternalServerError: HTTPInternalServerError (HTTP 500)\n'] Which is visible in cinder-scheduler log file. When we reverted a change, everything started to operate normally. I'm just wondering how we can skip this warning message and makes cinder and glance to cooperate correctly, especially that we're planning to make an upgrade from Rocky to Stein soon and this option is planned to be removed in the Stein release. Thanks in advance for your help here. Pozdrawiam, Jan Wasilewski Administrator systemów i sieci CI ATM Tel: +48 691 493 327 jan.wasilewski(a)atman.pl<mailto:jan.wasilewski@atman.pl>, www.atman.eu<http://www.atman.eu/> Informacja o zasadach przetwarzania danych osobowych przez ATM S.A. znajduje się pod tym linkiem<http://www.atman.pl/Informacja-o-przetwarzaniu-danych-osobowych-cabout-pol-…>.

2 1

[tripleo] migrating master from CentOS-8 to CentOS-8-Stream
by Wesley Hayutin 23 Feb '21

23 Feb '21

Greetings, Just a heads up over the course of the next few weeks upstream TripleO should see a transparent migration from CentOS-8 to CentOS-8-Stream. We do have a few options with regards to how the transition will take place. First and foremost we're going to migrate the master branch only at this time. *Question 1: Artifacts* Option1: New namespaces for artifacts: Containers: https://hub.docker.com/r/tripleomasterstream/ or some combination Images: http://images.rdoproject.org/centosstream8 or other combination Option2: Master content and namespaces is overwritten with centos-8-stream containers and images and will retain the paths and namespaces. Containers: https://hub.docker.com/u/tripleomaster Images: http://images.rdoproject.org/centos8/master/ *Question 2: job names* We can update the master jobs to include "stream" in the name and be explicit for the distro name OR: We can leave the job names as is and just communicate that "centos-8" is now really centos-8-stream Now's your chance to weigh in prior to the upcoming changes. Thanks!

5 5

Unexpected MAC Address Change
by Lazuardi Nasution 23 Feb '21

23 Feb '21

Hi, Today, I find some of our Windows Server VMs cannot be accessed. After some inspection, I find that there is expected MAC address change on those VMs. There are inconsistencies between "ps aux | grep instance-***" output on such compute nodes where affected VMs reside, with "virsh dumpxml instance-***" on the same compute nodes. The "virsh dumpxml instance-***" output is consistent with TAP interface MAC address on "ip link" output and the MAC address of port list on OpenStack dashboard/CLI. Following are one of those outputs. This weirdness is back to normal after I do a live migration to another compute node. virsh dumpxml instance-000003e5 ... <interface type='bridge'> <mac address='fa:16:3e:ef:4e:3b'/> <source bridge='br-int'/> ... ps aux | grep instance-000003e5 ... netdev=hostnet0,id=net0,mac=fa:16:3e:8e:92:f0 ... What can change the MAC address on "ps aux | grep instance-***" output? This weirdness happens on Ussuri with Ubuntu 20.04 and OVN. Is this a bug? What kind of bug? Is there any other information should I look for to find the root cause? Best regards,

1 0

[largescale-sig] Video meeting Feb 24: "Regions vs. Cells"
by Thierry Carrez 23 Feb '21

23 Feb '21

Hi everyone, In order to increase its reach, the Large Scale SIG will organize video meetings around specific scaling topics, just one hour of discussion bootstrapped by a short presentation from operators of large scale deployments of OpenStack. Our first video meeting will be next Wednesday, February 24, at 1500utc on Zoom (link/password will be provided on this thread the day before the event). The theme will be: "Regions vs. Cells" quickstarted by a presentation from Belmiro Moreira (CERN). If you're interested in sharing your experience on this topic, or just want to learn more about that, please mark your calendar and join us next Wednesday ! -- Thierry Carrez

1 1

[cinder/barbican] LUKS encryption for mounted disk - how to decrypt cinder volume
by Jan Wasilewski 23 Feb '21

23 Feb '21

Hi All, I have a question about the possible decryption of LUKS volume. I'm testing currently barbican+cinder, but I'm just wondering if there is a way, to somehow decrypt my LUKS volume with payload generated by a barbican. Is there any procedure for that? I was doing it by myself, but somehow it doesn't work and I got an error: [TEST]root@barbican-01:/usr/lib/python3/dist-packages# barbican secret get --payload --payload_content_type application/octet-stream http://controller.test:9311/v1/secrets/76631940-9ab6-4b8c-9481-e54c3ffdbbfe +---------+--------------------------------------------------------------------------------------------------------+ | Field | Value | +---------+--------------------------------------------------------------------------------------------------------+ | Payload | b'\xbf!i\x97\xf4\x0c\x12\xa4\xfe4\xf3\x16C\xe8@\xdc\x0f\x9d+:\x0c7\xa9\xab[\x8d\xf2\xf1\xae\r\x89\xdc' | +---------+--------------------------------------------------------------------------------------------------------+ cryptsetup luksOpen /dev/disk/by-id/wwn-0x6e00084100ee7e7e7ab0b13c0000386f my-volume Enter passphrase for /dev/disk/by-id/wwn-0x6e00084100ee7e7e7ab0b13c0000386f: *<passphrase from payload>* No key available with this passphrase. I thought that above issue can be related to encoding, so I took payload value directly from vault and use it as a key-file, but problem is exactly the same(my encrypted volume is the last volume list by domblklist option): vault kv get secret/data/e5baa518207e4f9db4810988d22087ce | grep value | awk -F'value:' '{print $2}' 4d4d35676c336567714850663477336d2b415475746b74774c56376b77324b4e73773879724c46704678513d] [TEST]root@comp-02:~# cat bbb 4d4d35676c336567714850663477336d2b415475746b74774c56376b77324b4e73773879724c46704678513d [TEST]root@comp-02:~# cat bbb | base64 -d > pass2 [TEST]root@comp-02:~# cat pass2 ▒▒߻▒▒▒▒▒^<▒N▒▒▒▒~پ5▒▒▒▒▒▒▒z߾▒▒▒▒~▒▒▒▒▒n▒▒▒▒▒]▒[TEST]root@comp-02:~# [TEST]root@comp-02:~# virsh domblklist instance-00000da8 Target Source ------------------------------------------------ vda /dev/dm-17 vdb /dev/disk/by-id/wwn-0x6e00084100ee7e7e74623bd3000036bc vdc /dev/dm-16 vde /dev/disk/by-id/wwn-0x6e00084100ee7e7e7ab0b13c0000386f vdf /dev/disk/by-id/wwn-0x6e00084100ee7e7e7bd45c1b000038b5 [TEST]root@comp-02:~# udisksctl unlock -b /dev/disk/by-id/wwn-0x6e00084100ee7e7e7bd45c1b000038b5 --key-file pass2 Error unlocking /dev/dm-21: GDBus.Error:org.freedesktop.UDisks2.Error.Failed: Error unlocking /dev/dm-21: Failed to activate device: Operation not permitted [TEST]root@comp-02:~# cryptsetup luksOpen /dev/disk/by-id/wwn-0x6e00084100ee7e7e7bd45c1b000038b5 my-volume --master-key-file=pass2 Volume key does not match the volume. I see that nova/cinder and barbican are doing this stuff somehow so I strongly believe there is a way to decrypt this manually. Maybe I’m doing something wrong in my testing-steps. Thanks in advance for any help here! Unfortunately, I haven’t found any materials on how to do this. Best regards, Jan

2 5

[tc] weekly update
by Mohammed Naser 23 Feb '21

23 Feb '21

Hi everyone, Here's an update on what happened in the OpenStack TC this week. You can get more information by checking for changes in openstack/governance repository. # Patches ## Open Reviews - Cool-down cycle goal https://review.opendev.org/c/openstack/governance/+/770616 - monasca-log-api & monasca-ceilometer does not make releases https://review.opendev.org/c/openstack/governance/+/771785 ## Project Updates - Retire tempest-horizon https://review.opendev.org/c/openstack/governance/+/774379 ## General Changes - Add assert:supports-api-interoperability tag to neutron https://review.opendev.org/c/openstack/governance/+/773090 - Add assert:supports-api-interoperability to cinder https://review.opendev.org/c/openstack/governance/+/773684 # Other Reminders - Our next [TC] Weekly meeting is scheduled for February 25th at 1500 UTC. If you would like to add topics for discussion, please go to https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting and fill out your suggestions by Wednesday, February 24th, 2100 UTC. Thanks for reading! Mohammed & Kendall -- Mohammed Naser VEXXHOST, Inc.

1 0

[edge][all] Features and updates relevant to edge computing use cases
by Ildiko Vancsa 22 Feb '21

22 Feb '21

Hi, I’m reaching out as I need a little guidance. I was asked to give a presentation about the latest features and improvements in OpenStack that are relevant for edge computing. I wanted to ask for the community’s help, as the project teams are the experts, to give any pointers that you think would be great to include in my presentation. I appreciate all your guidance in advance. Thanks, Ildikó

2 2

Non-DHCP VLAN Network ARP Problem with OVN
by Lazuardi Nasution 22 Feb '21

22 Feb '21

Hi, One of our OpenStack clusters has a VLAN based network without DHCP enabled. The cluster uses OVN for SDN.That network has to be like it since it will be used for migrating the VMs from other platforms. I have untick the DHCP enable feature on creating that network at Horizon. After the network and subnet are created, I find that the DHCP port is still created even though it is in a down state. I try to delete that port but after some time, that port is back again. When that DHCP port exists, we find that some VMs inside this OpenStack cluster resolve the MAC address of the IP of that disabled DHCP port to the MAC address of the DHCP port. The problem is coming since that IP is used to an existing VM/server outside this OpenStack cluster at the same VLAN, but it only happens for some VMs. Before this finding, I really disregard the existence of the DHCP port since it is down and I don't enable the DHCP feature. After that, I realize that it is a serious problem. My questions are how to completely disable the DHCP feature for such a network, including its DHCP port? How can DHCP disabled networks with a down DHCP port still respond to the ARP request and why does it happen only for some VMs? Best regards,

3 6

OVN Distributed Floating IP and L3 Gateway HA
by Lazuardi Nasution 22 Feb '21

22 Feb '21

Hi, For having OVN distributed floating IP, should I use the "distributed" flag on router creation of OpenStack and/or use all compute nodes as Gateway Chassis? For having L3 Gateway HA for non floating IP instances, should I use the "ha" flag on router creation of OpenStack? I ask this since I find that pinging from outside network to floating IP always comes to one of Gateway Chassis nodes. Is this normal behavior? Best regards.

1 0

[Murano] error in murano-dashboard when clicking on add component in environment
by Bujack, Stefan 22 Feb '21

22 Feb '21

Hello, we have an openstack victoria deployment on ubuntu 20.04. We use the murano-dashboard in horizon. When clicking on add component in APP Catalog --> Applications --> Environments we get an error page and apache2 has an error: ==> /var/log/apache2/error.log <== [Thu Feb 18 14:17:51.223960 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] /usr/lib/python3/dist-packages/django/contrib/staticfiles/templatetags/staticfiles.py:24: RemovedInDjango30Warning: {% load staticfiles %} is deprecated in favor of {% load static %}. [Thu Feb 18 14:17:51.224014 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] warnings.warn( [Thu Feb 18 14:17:51.226557 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] ERROR django.request Internal Server Error: /app-catalog/catalog/switch_environment/281f558410fb40bf8f5e090f04d90248 [Thu Feb 18 14:17:51.226584 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] Traceback (most recent call last): [Thu Feb 18 14:17:51.226590 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 34, in inner [Thu Feb 18 14:17:51.226594 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] response = get_response(request) [Thu Feb 18 14:17:51.226598 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 115, in _get_response [Thu Feb 18 14:17:51.226602 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] response = self.process_exception_by_middleware(e, request) [Thu Feb 18 14:17:51.226606 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 113, in _get_response [Thu Feb 18 14:17:51.226610 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] response = wrapped_callback(request, *callback_args, **callback_kwargs) [Thu Feb 18 14:17:51.226615 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] File "/usr/lib/python3/dist-packages/horizon/decorators.py", line 52, in dec [Thu Feb 18 14:17:51.226619 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] return view_func(request, *args, **kwargs) [Thu Feb 18 14:17:51.226622 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] File "/usr/lib/python3/dist-packages/horizon/decorators.py", line 36, in dec [Thu Feb 18 14:17:51.226626 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] return view_func(request, *args, **kwargs) [Thu Feb 18 14:17:51.226630 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] File "/usr/lib/python3/dist-packages/horizon/decorators.py", line 36, in dec [Thu Feb 18 14:17:51.226634 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] return view_func(request, *args, **kwargs) [Thu Feb 18 14:17:51.226637 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] File "/usr/lib/python3/dist-packages/horizon/decorators.py", line 113, in dec [Thu Feb 18 14:17:51.226641 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] return view_func(request, *args, **kwargs) [Thu Feb 18 14:17:51.226656 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] File "/usr/lib/python3/dist-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view [Thu Feb 18 14:17:51.226660 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] return view_func(request, *args, **kwargs) [Thu Feb 18 14:17:51.226665 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] File "/usr/lib/python3/dist-packages/muranodashboard/catalog/views.py", line 129, in switch [Thu Feb 18 14:17:51.226671 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] if not http_utils.is_safe_url(url=redirect_to, host=request.get_host()): [Thu Feb 18 14:17:51.226674 2021] [wsgi:error] [pid 3970511:tid 139721480873728] [remote 131.169.5.116:43998] TypeError: is_safe_url() got an unexpected keyword argument 'host' Any help is appreciated Thanks in advance Greets Stefan Bujack

2 2

[neutron] Bug Deputy Report Feb 15 - 21
by Oleg Bondarev 22 Feb '21

22 Feb '21

Hello all interested, Please check Neutron new bugs summary for the week Feb 15-21 I'd like to highlight 2 gate blockers looking for an assignees: - 1916332<https://bugs.launchpad.net/neutron/+bug/1916332> - broken openstack-tox-docs job on neutron-specs - 1916041<https://bugs.launchpad.net/neutron/+bug/1916041> - tempest-slow-py3 job fails on stable/stein There is also a medium one 1916276<https://bugs.launchpad.net/neutron/+bug/1916276> that seeks for volunteers. And a discussoin topic for the team: 1916428<https://bugs.launchpad.net/neutron/+bug/1916428> regarding dibbler tool end of support. Full report: Critical * https://bugs.launchpad.net/neutron/+bug/1915692 - Neutron tempest plugin jobs for stable/stein are broken as they are using tempest from master branch o Fix Released o Assigned to Slawek Kaplonski * https://bugs.launchpad.net/neutron/+bug/1916332 - neutron-specs's zuul job openstack-tox-docs fails o Confirmed o Unassigned * https://bugs.launchpad.net/neutron/+bug/1916041 - tempest-slow-py3 ipv6 gate job fails on stable/stein o Confirmed o Unassigned High * https://bugs.launchpad.net/neutron/+bug/1916024 - HA router master instance in error state because qg-xx interface is down o In Progress o Assigned to Rodolfo Alonso Hernandez Medium * https://bugs.launchpad.net/neutron/+bug/1915723 - [OVN] QoS rules do not apply to distributed floating IP traffic o In Progress o Assigned to Rodolfo Alonso Hernandez * https://bugs.launchpad.net/neutron/+bug/1915849 - setting a bound port qos policy from empty to a policy with min bw rules does not update placement allocation o Confirmed o Assigned to Lajos Katona * https://bugs.launchpad.net/neutron/+bug/1915856 - updating a qos policy on a bound port from None to a policy with min_bw rules does not update placement allocation o Confirmed o Assigned to Lajos Katona * https://bugs.launchpad.net/neutron/+bug/1915898 - changing the qos policy on a network does not change the placement allocation of the bound ports in the network o Confirmed o Assigned to Lajos Katona * https://bugs.launchpad.net/neutron/+bug/1916276 - IP address inventory handling for routed net does not work for ip_allocation=deferred o Confirmed o Unassigned Low * https://bugs.launchpad.net/neutron/+bug/1916022 - L3HA Race condition during startup of the agent may cause inconsistent router's states o In Progress o Assigned to Slawek Kaplonski Opinion (Undecided) * https://bugs.launchpad.net/neutron/+bug/1916428 - dibbler tool for dhcpv6 is concluded o Needs discussion Thanks, Oleg --- Advanced Software Technology Lab Huawei

1 0

[Outreachy] Call for mentors, projects by March 7
by Samuel de Medeiros Queiroz 20 Feb '21

20 Feb '21

Hi Stackers, *TL;DR OpenStack is participating in Outreachy!* *Please submit projects as soon as possible, deadline is March 7th, 4 pm UTC:* *https://www.outreachy.org/communities/cfp/openstack/ <https://www.outreachy.org/communities/cfp/openstack/>* Outreachy's goal is to support people from groups underrepresented in the technology industry. Interns will work remotely with mentors from our community. We are seeking mentors to propose projects that Outreachy interns can work on during their internship. If you or your colleagues are interested in mentoring in the next round, please submit your project proposal by *March 7th 2021 *over here https://www.outreachy.org/communities/cfp/openstack/ Mentors should read the mentor FAQ https://www.outreachy.org/mentor/mentor-faq and find more details about the Outreachy program and timeline in https://www.outreachy.org/communities/cfp/ . If you want help crafting your project proposal, please contact me < samueldmq(a)gmail.com> or Mahati Chamarthy <mahati.chamarthy(a)gmail.com> and we will be glad to assist you. Thank you, Samuel Queiroz

1 0

[cinder] os-brick nvmeof connector changes
by Brian Rosmaita 20 Feb '21

20 Feb '21

The nvmeof connector was refactored by Change-Id: I84145ce0be96c3e39b00720bddd179f91703b978 [0] that is included in os-brick release 4.2.0. Unfortunately, it introduced a regression that has broken current drivers consuming that connector [1]. Ivan has a patch up to revert the change and restore the legacy nvmeof connector [2]. Zohar suggests instead adding some compatibility code to the refactored driver. Please take a look at [2] and leave some helpful responses. The Mellanox CI, which runs the SPDK driver that uses the nvmeof connector hasn't been responding recently, but Ivan said he can trigger it manually to test changes, so we'll be able to validate the "old" nvmeof connector interface. [0] https://review.opendev.org/c/openstack/os-brick/+/768575 [1] https://bugs.launchpad.net/os-brick/+bug/1916264 [2] https://review.opendev.org/c/openstack/os-brick/+/776441

1 0

[ops][cinder] warning about os-brick release 4.2.0
by Brian Rosmaita 20 Feb '21

20 Feb '21

This message applies to operators using cinder drivers that use the os-brick nvmeof connector. os-brick release 4.2.0, released on 16 February, contains a refactored nvmeof connector that unfortunately introduced a regression in the connector interface and breaks older drivers that use the nvmeof connector. So if you use such a driver, do not upgrade to 4.2.0. This issue will be resolved before the "official" wallaby os-brick release. You can follow events as they unfold on Bug #1916264: https://bugs.launchpad.net/os-brick/+bug/1916264

1 0

[docs] Web analytics
by Peter Matulis 20 Feb '21

20 Feb '21

Hi, is it possible to access web analytics for the documentation projects one maintains? Thanks, Peter

3 3

Re: Trove Instance fails in build stage
by Lingxian Kong 20 Feb '21

20 Feb '21

(adding openstack-discuss mail address) Hi Abhinav, Typically, there are 2 ports attached to Trove instance, one is mgmt port meant to communicate with rabbitmq but can also be responsible for other mgmt traffic (e.g. docker image pull) after routing table or dns customization, that depends on your business requirements. The other one is the user port mainly for database access. You can find more description here https://docs.openstack.org/trove/latest/admin/run_trove_in_production.html#… . There is only 1 Nova VM for each Trove instance. Yes, you are using the right image if you deployed Ussuri Trove. --- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes) On Fri, Feb 19, 2021 at 6:25 PM Abhinav Tyagi <abhinav31796(a)gmail.com> wrote: > Hi Lingxian, > > Thanks for the timely help. > > We checked the logs and found out the trove guest agent instance is not > able to connect to rabbimq container. We are using OpenStack-Ansible for > deployment. It's a 3 master setup. But trove we have deployed manually on > one compute physical node. > > Could it be because of the management network of trove and containers is > not the same? Does trove guest instance should have 2 NIC's? When we create > trove database instance, are there 2 instances launched? Are we using the > right image? > > Regards, > Aabhinav Tyagi > > On Fri, Feb 19, 2021 at 3:08 AM Lingxian Kong <anlin.kong(a)gmail.com> > wrote: > >> Hi Abhinav, >> >> Please check the trove guest agent log according to this guide >> https://docs.openstack.org/trove/latest/admin/troubleshooting.html. >> >> --- >> Lingxian Kong >> Senior Cloud Engineer (Catalyst Cloud) >> Trove PTL (OpenStack) >> OpenStack Cloud Provider Co-Lead (Kubernetes) >> >> >> On Fri, Feb 19, 2021 at 9:20 AM Abhinav Tyagi <abhinav31796(a)gmail.com> >> wrote: >> >>> Hi, >>> >>> Please help us in using trove. We are using >>> https://tarballs.opendev.org/openstack/trove/images/trove-ussuri-mysql-ubun… >>> image. >>> We have a multi master openstack setup. >>> >>> These are the logs: >>> >>> Feb 18 19:03:30 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:03:30.535 1274599 INFO trove.taskmanager.models [-] Waiting >>> for instance 5fe896f6-dc7b-424a-87c2-64f207f422a2 up and running with >>> timeout 900s >>> Feb 18 19:18:33 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall [-] Dynamic >>> backoff interval looping call >>> 'trove.common.utils.build_polling_task.<locals>.poll_and_check' failed: >>> oslo_service.loopingcall.LoopingCallTimeOut: Looping call timed out after >>> 893.31 seconds >>> Feb 18 19:18:33 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall Traceback >>> (most recent call last): >>> Feb 18 19:18:33 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall File >>> "/usr/lib/python3.6/site-packages/oslo_service/loopingcall.py", line 154, >>> in _run_loop >>> Feb 18 19:18:33 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall idle = >>> idle_for_func(result, self._elapsed(watch)) >>> Feb 18 19:18:33 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall File >>> "/usr/lib/python3.6/site-packages/oslo_service/loopingcall.py", line 351, >>> in _idle_for >>> Feb 18 19:18:33 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall % >>> self._error_time) >>> Feb 18 19:18:33 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall >>> oslo_service.loopingcall.LoopingCallTimeOut: Looping call timed out after >>> 893.31 seconds >>> Feb 18 19:18:33 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall >>> Feb 18 19:18:33 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:18:33.941 1274599 ERROR trove.taskmanager.models [-] Failed >>> to create instance 5fe896f6-dc7b-424a-87c2-64f207f422a2, error: Polling >>> request timed out..: trove.common.exception.PollTimeOut: Polling request >>> timed out. >>> Feb 18 19:18:33 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:18:33.960 1274599 ERROR trove.taskmanager.models [-] Service >>> status: ERROR, service error description: guestagent error: >>> trove.common.exception.PollTimeOut: Polling request timed out. >>> Feb 18 19:18:33 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:18:33.970 1274599 DEBUG trove.db.models [-] Saving >>> DBInstance: {'_sa_instance_state': <sqlalchemy.orm.state.InstanceState >>> object at 0x7fb5447bd668>, 'tenant_id': '19aa2578cb444b828549b394eed8ebfb', >>> 'hostname': None, 'shard_id': None, 'server_status': None, >>> 'compute_instance_id': 'b928d7e4-6706-48d2-9fb9-c75e19b9765c', 'type': >>> None, 'deleted': 0, 'id': '5fe896f6-dc7b-424a-87c2-64f207f422a2', >>> 'task_id': 91, 'region_id': 'Bangalore', 'deleted_at': None, >>> 'task_description': 'Build error: guestagent timeout.', 'encrypted_key': >>> '***', 'datastore_version_id': 'b4bbc01c-e785-4944-8a76-6c154c19ddff', >>> 'task_start_time': None, 'created': datetime.datetime(2021, 2, 18, 13, 33, >>> 24), 'configuration_id': None, 'volume_id': >>> '68accc42-34ae-4ac3-93fc-1efb9fed2af9', 'slave_of_id': None, 'updated': >>> datetime.datetime(2021, 2, 18, 13, 48, 33, 969535), 'flavor_id': >>> 'bbf4bad4-e258-45b4-8860-e5a51c86ed6e', 'name': 'trove_instance_15', >>> 'cluster_id': None, 'volume_size': 10, 'errors': {}} save >>> /usr/lib/python3.6/site-packages/trove/db/models.py:65 >>> Feb 18 19:18:33 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:18:33.981 1274599 ERROR trove.taskmanager.models [-] Trove >>> instance status: ERROR, Trove instance status description: Build error: >>> guestagent timeout.: trove.common.exception.PollTimeOut: Polling request >>> timed out. >>> Feb 18 19:18:33 bngoscompn01.comviva.com trove-taskmanager[1274599]: >>> 2021-02-18 19:18:33.994 1274599 DEBUG trove.db.models [-] Saving >>> DBInstanceFault: {'_sa_instance_state': <sqlalchemy.orm.state.InstanceState >>> object at 0x7fb5448688d0>, 'id': 'f0983744-4cfe-4863-b9ae-6fb33cc8a0d2', >>> 'created': datetime.datetime(2021, 2, 18, 13, 48, 33, 994216), 'deleted': >>> False, 'instance_id': '5fe896f6-dc7b-424a-87c2-64f207f422a2', 'message': >>> 'Polling request timed out.', 'details': 'Traceback (most recent call >>> last):\n File "/usr/lib/python3.6/site-packages/trove/common/utils.py", >>> line 207, in wait_for_task\n return polling_task.wait()\n File >>> "/usr/local/lib/python3.6/site-packages/eventlet/event.py", line 125, in >>> wait\n result = hub.switch()\n File >>> "/usr/local/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 309, in >>> switch\n return self.greenlet.switch()\n File >>> "/usr/lib/python3.6/site-packages/oslo_service/loopingcall.py", line 154, >>> in _run_loop\n idle = idle_for_func(result, self._elapsed(watch))\n >>> File "/usr/lib/python3.6/site-packages/oslo_service/loopingcall.py", line >>> 351, in _idle_for\n % >>> self._error_time)\noslo_service.loopingcall.LoopingCallTimeOut:\n >>> Looping call timed out after 893.31 seconds\n\nDuring handling of the >>> above exception, another exception occurred:\n\nTraceback (most recent call >>> last):\n File >>> "/usr/lib/python3.6/site-packages/trove/taskmanager/models.py", line 431, >>> in wait_for_instance\n time_out=timeout)\n File >>> "/usr/lib/python3.6/site-packages/trove/common/utils.py", line 222, in >>> poll_until\n return wait_for_task(task)\n File >>> "/usr/lib/python3.6/site-packages/trove/common/utils.py", line 209, in >>> wait_for_task\n raise >>> exception.PollTimeOut\ntrove.common.exception.PollTimeOut: Polling request >>> timed out.\n', 'errors': {}, 'updated': datetime.datetime(2021, 2, 18, 13, >>> 48, 33, 994288)} save /usr/lib/python3.6/site-packages/trove/db/models.py:65 >>> >>> >>>

1 0

[security][security-sig] Change meeting frequency/reschedule
by Gage Hugo 19 Feb '21

19 Feb '21

Hello everyone, The security SIG is still meeting each week at our usual time and over the past year there have often been meetings without any topics. One idea that has been thrown around was to reduce the frequency of meetings to either every other week or even once a month. We are also floating the idea of rescheduling to a new time-slot that is more available for folks. Was wondering if anyone had any opinions/insights on this. We can discuss more during next week's meeting. Thanks!

3 2

[oslo] February 22 meeting is canceled
by Herve Beraud 19 Feb '21

19 Feb '21

Hello Osloers, I will be on PTO all next week so I won't be around to run the meeting. If someone else from the Oslo team wants to run it, please feel free. Thanks for your reading -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

3 2

[victoria][cinder] Extending attached volume
by Ammad Syed 19 Feb '21

19 Feb '21

Hi, I am trying to extend attached volume from openstack client. Exporting the volume API version 3.62, its giving me below error. root@controller ~(keystone_ubuntu20_new)# openstack volume list volume version 3.62 is not in supported versions: 1, 2, 3 Invalid volume client version '3.62'. must be one of: 2, 3 However the attached volume extension from cinder cli was successful. Its shows updated volume size. root@controller ~(keystone_ubuntu20_new)# OS_VOLUME_API_VERSION=3.62 cinder extend test 13 However the volume size was not updated in vm. Checking the cinder volume logs, found below error. 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova [req-93a79710-96df-4e4f-9a0c-d4457d289122 2af528fdf3244e15b4f3f8fcfc0889c5 890eb2b7d1b8488aa88de7c34d08817a - - -] Failed to notify nova on events: [{'name': 'volume-extended', 'server_uuid': '0e9512fb-fb18-4572-8aec-b7508c9f6cc0', 'tag': 'af80330c-f910-46cb-9654-3290e047fbed'}].: keystoneauth1.exceptions.http.BadRequest: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-bb86ee85-f6ba-4135-ac55-5917c8658e5c) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova Traceback (most recent call last): 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/cinder/compute/nova.py", line 149, in _send_events 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova response = nova.server_external_events.create(events) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/novaclient/v2/server_external_events.py", line 38, in create 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova return self._create('/os-server-external-events', body, 'events', 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/novaclient/base.py", line 363, in _create 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova resp, body = self.api.client.post(url, body=body) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 401, in post 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/novaclient/client.py", line 69, in request 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova resp, body = super(SessionClient, self).request(url, 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 554, in request 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova resp = super(LegacyJsonAdapter, self).request(*args, **kwargs) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 257, in request 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova return self.session.request(url, method, **kwargs) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 780, in request 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova auth_headers = self.get_auth_headers(auth) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 1191, in get_auth_headers 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova return auth.get_headers(self, **kwargs) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/keystoneauth1/plugin.py", line 95, in get_headers 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova token = self.get_token(session) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/keystoneauth1/identity/base.py", line 88, in get_token 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova return self.get_access(session).auth_token 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/keystoneauth1/identity/base.py", line 134, in get_access 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova self.auth_ref = self.get_auth_ref(session) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova return self._plugin.get_auth_ref(session, **kwargs) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/keystoneauth1/identity/v3/base.py", line 187, in get_auth_ref 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova resp = session.post(token_url, json=body, headers=headers, 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 1139, in post 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova return self.request(url, 'POST', **kwargs) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 976, in request 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova raise exceptions.from_response(resp, method, url) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova keystoneauth1.exceptions.http.BadRequest: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-bb86ee85-f6ba-4135-ac55-5917c8658e5c) 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova 2021-02-19 10:23:25.791 22742 INFO cinder.message.api [req-93a79710-96df-4e4f-9a0c-d4457d289122 2af528fdf3244e15b4f3f8fcfc0889c5 890eb2b7d1b8488aa88de7c34d08817a - - -] Creating message record for request_id = req-93a79710-96df-4e4f-9a0c-d4457d289122 2021-02-19 10:23:25.807 22742 INFO cinder.volume.manager [req-93a79710-96df-4e4f-9a0c-d4457d289122 2af528fdf3244e15b4f3f8fcfc0889c5 890eb2b7d1b8488aa88de7c34d08817a - - -] Extend volume completed successfully. Any advice? - Ammad

3 3