October 2024 - openstack-discuss - lists.openstack.org

[kolla-ansible][haproxy][2024.1] HAProxy template error
by Satish Patel 23 Oct '24

23 Oct '24

I am deploying kolla-ansible latest release 2024.1 on Ubuntu 22.04 and getting strange error at following task When I am running kolla-ansible -i multinode deploy TASK [include_role : glance] ***************************************************************************************************************************************************************** task path: /usr/local/share/kolla-ansible/ansible/site.yml:168 redirecting (type: modules) ansible.builtin.firewalld to ansible.posix.firewalld TASK [haproxy-config : Copying over glance haproxy config] *********************************************************************************************************************************** task path: /usr/local/share/kolla-ansible/ansible/roles/haproxy-config/tasks/main.yml:2 The full traceback is: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/ansible/plugins/action/template.py", line 152, in run resultant = templar.do_template(template_data, preserve_trailing_newlines=True, escape_backslashes=False, overrides=overrides) File "/usr/local/lib/python3.10/dist-packages/ansible/template/__init__.py", line 1010, in do_template res = myenv.concat(rf) File "/usr/local/lib/python3.10/dist-packages/ansible/template/native_helpers.py", line 83, in ansible_concat return ''.join([to_text(v) for v in nodes]) File "/usr/local/lib/python3.10/dist-packages/ansible/template/native_helpers.py", line 83, in <listcomp> return ''.join([to_text(v) for v in nodes]) File "<template>", line 436, in root File "/usr/local/lib/python3.10/dist-packages/ansible/template/__init__.py", line 381, in call return super().call(obj, *args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/jinja2/runtime.py", line 303, in call return __obj(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/jinja2/runtime.py", line 768, in __call__ return self._invoke(arguments, autoescape) File "/usr/local/lib/python3.10/dist-packages/jinja2/runtime.py", line 782, in _invoke rv = self._func(*arguments) File "<template>", line 193, in macro File "/usr/local/lib/python3.10/dist-packages/ansible/template/__init__.py", line 295, in wrapper ret = func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/jinja2/filters.py", line 1030, in do_format return soft_str(value) % (kwargs or args) File "/usr/local/lib/python3.10/dist-packages/markupsafe/__init__.py", line 165, in __mod__ return self.__class__(super().__mod__(value)) File "/usr/local/lib/python3.10/dist-packages/markupsafe/__init__.py", line 370, in __str__ return str(self.escape(self.obj)) File "/usr/local/lib/python3.10/dist-packages/markupsafe/__init__.py", line 235, in escape rv = escape(s) File "/usr/local/lib/python3.10/dist-packages/markupsafe/__init__.py", line 45, in escape return Markup(_escape_inner(str(s))) SystemError: <built-in function _escape_inner> returned NULL without setting an exception failed: [os2-bos-ctrl01] (item={'key': 'glance-api', 'value': {'container_name': 'glance_api', 'group': 'glance-api', 'host_in_groups': True, 'enabled': True, 'image': ' registry.example.com/kolla/glance-api:2024.1-ubuntu-jammy', 'environment': {'http_proxy': '', 'https_proxy': '', 'no_proxy': 'localhost,127.0.0.1,10.0.52.11,10.0.52.10'}, 'privileged': False, 'volumes': ['/etc/kolla/glance-api/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'glance:/var/lib/glance/', '', 'kolla_logs:/var/log/kolla/', '', ''], 'dimensions': {}, 'healthcheck': {'interval': '30', 'retries': '3', 'start_period': '5', 'test': ['CMD-SHELL', 'healthcheck_curl http://10.0.52.11:9292'], 'timeout': '30'}, 'haproxy': {'glance_api': {'enabled': True, 'mode': 'http', 'external': False, 'port': '9292', 'frontend_http_extra': ['timeout client 6h'], 'backend_http_extra': ['timeout server 6h'], 'custom_member_list': ['server os2-bos-ctrl01 10.0.52.11:9292 check inter 2000 rise 2 fall 5', '']}, 'glance_api_external': {'enabled': True, 'mode': 'http', 'external': True, 'external_fqdn': 'openstack-bos-2.example.com', 'port': '9292', 'frontend_http_extra': ['timeout client 6h'], 'backend_http_extra': ['timeout server 6h'], 'custom_member_list': ['server os2-bos-ctrl01 10.0.52.11:9292 check inter 2000 rise 2 fall 5', '']}}}}) => { "ansible_loop_var": "item", "changed": false, "item": { "key": "glance-api", "value": { "container_name": "glance_api", "dimensions": {}, "enabled": true, "environment": { "http_proxy": "", "https_proxy": "", "no_proxy": "localhost,127.0.0.1,10.0.52.11,10.0.52.10" }, "group": "glance-api", "haproxy": { "glance_api": { "backend_http_extra": [ "timeout server 6h" ], "custom_member_list": [ "server os2-bos-ctrl01 10.0.52.11:9292 check inter 2000 rise 2 fall 5", "" ], "enabled": true, "external": false, "frontend_http_extra": [ "timeout client 6h" ], "mode": "http", "port": "9292" }, "glance_api_external": { "backend_http_extra": [ "timeout server 6h" ], "custom_member_list": [ "server os2-bos-ctrl01 10.0.52.11:9292 check inter 2000 rise 2 fall 5", "" ], "enabled": true, "external": true, "external_fqdn": "openstack-bos-2.example.com", "frontend_http_extra": [ "timeout client 6h" ], "mode": "http", "port": "9292" } }, "healthcheck": { "interval": "30", "retries": "3", "start_period": "5", "test": [ "CMD-SHELL", "healthcheck_curl http://10.0.52.11:9292" ], "timeout": "30" }, "host_in_groups": true, "image": " registry.example.com/kolla/glance-api:2024.1-ubuntu-jammy", "privileged": false, "volumes": [ "/etc/kolla/glance-api/:/var/lib/kolla/config_files/:ro", "/etc/localtime:/etc/localtime:ro", "/etc/timezone:/etc/timezone:ro", "glance:/var/lib/glance/", "", "kolla_logs:/var/log/kolla/", "", "" ] } }, "msg": "SystemError: <built-in function _escape_inner> returned NULL without setting an exception" }

1 1

[zaqar]Proposing Takashi Kajinami as Zaqar core member
by hao wang 23 Oct '24

23 Oct '24

Hello, everyone, I would like to propose Takashi Kajinami for the OpenStack Zaqar core member. Takashi has contributed to the Zaqar project since 2021 and has done a great job of improving this project. It is very important to Zaqar to keep it stable and reliable. I am so excited that we can have such a great contributor and I believe we can do more valuable work to the Zaqar project. Thank you for your contributions. Regards, Hao Wang -- Best Wishes For You!

1 0

[nova][ptg] Nova sessions on Wednesday
by Sylvain Bauza 22 Oct '24

22 Oct '24

Hi, Given most of the nova cores will attend the eventlet-removal session at 1300UTC, we'll defer the nova sessions to start at 1400UTC instead of 1300UTC. Please stay online in #openstack-nova IRC channel or please take a look at https://ptg.opendev.org/ptg.html to see the exact time when we spawn again the nova conversations. Thanks, -Sylvain (who just unbooked the austin room for Wed-B1)

1 0

[Horizon] Cancelling weekly meeting tomorrow
by Tatiana Ovchinnikova 22 Oct '24

22 Oct '24

Hello team, Since we have vPTG sessions, I'm cancelling our weekly meeting tomorrow. With kind regards, Tatiana

1 0

[dev][cinder] Bubbling up maintenance status on a volume from a third-party backend
by David Pineau 22 Oct '24

22 Oct '24

Hello Openstack and Cinder team, I'm currently working on the storage management layer of our company's Openstack platform. As we manage the storage ourselves, we may run background operations, which are akin to a maintenance, on individual volumes, or part of the infrastructure. As a bit of context, this infrastructure will be both exposed to external customers, as well as internal departments, and as such, we are looking for a refined user experience where an error is not a simple "an error occurred in service X", but something potentially actionable by whoever encounters it. At the moment in the Cinder documentation (and what I could find on the discuss archive), there seems to be no way for a Cinder Driver and its backing services to: - Tell Cinder that a specific Volume is undergoing a backend maintenance and should be considered unavailable - raise a relevant error to Cinder about ongoing Operations affecting a Volume within its backend, that cinder could properly react to As a first step, I wanted to check if you, the community, had ever considered this issue (or at least we consider it to be one). We'd be very happy if you had recipes or pieces of advice to share with us on how to handle this. If nothing is available or known, how could we help in bringing such an improvement to Cinder ? Kind regards, -- David Pineau (joa)

3 5

[Openstack-ansible]- Need to delete VM with its OS disk even if it has active snapshot available.
by Danish Khan 22 Oct '24

22 Oct '24

Dear All, When I delete a VM then the OS disk of the VM remains even when I have marked the Volume to get deleted on VM deletion. This is happening only when we have a snapshot of the OS disk or snapshot of the complete VM available. Can we configure it in a way then either that the VM should get deleted completely with OS disk even if we have the snapshot available or the VM deletion task should fail if the snapshot is available. Regards, Danish

1 0

[nova][libvirtd] internal error: unable to execute QEMU command 'query-named-block-nodes'
by Satish Patel 22 Oct '24

22 Oct '24

Folks, I have a ceph cluster and recently I configured rbd-mirror to replicate all data to remove ceph cluster for disaster recovery. Yesterday for POC we did a hard cutover on ceph and point openstack to the new cluster. All other vms came back up fine but 2 vms stuck in this error state in libvirt logs 2024-01-31 22:44:37.591+0000: 474597: error : qemuMonitorJSONCheckErrorFull:399 : internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied If it's a wider issue than it should impact all the vms but why only two VM stuck and not starting and libvirt giving me this error in logs nova-compute logs also showing same error 2024-01-31 22:44:40.925 7 INFO nova.compute.manager [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] [instance: 2de0f880-77c7-4d2c-9e01-898c57ad3693] Successfully reverted task state from powering-on on failure for instance. 2024-01-31 22:44:40.944 7 ERROR oslo_messaging.rpc.server [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] Exception during message handling: libvirt.libvirtError: internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied

3 8

[swift][ptg] Ops Feedback Session - 23 October at 14:00 UTC
by Tim Burke 22 Oct '24

22 Oct '24

As we've done in PTGs past, we're getting devs and ops together to talk about Swift: what's working, what isn't, and what would be most helpful to improve. We're meeting on meetpad (https://meetpad.opendev.org/p/swift-epoxy-ops-feedback) on Wednesday at 14:00UTC -- if you run a Swift cluster, we hope to see you there! Even if you can't make it, we'd appreciate it if you can offer some feedback on the etherpad (https://etherpad.opendev.org/p/swift-epoxy-ops-feedback) This has always been a highlight at every PTG for the swift developers. Have your say and help make the next Swift the best one yet! Tim

1 0

[all][rbac] Canceling policy popup team tomorrow meeting
by Ghanshyam Mann 22 Oct '24

22 Oct '24

Hello Everyone, Due to vPTG weeks, I am cancelling the policy pop-up tomorrow meeting. https://wiki.openstack.org/wiki/Consistent_and_Secure_Default_Policies_Popu… -gmann

1 0

[cinder] 2025.1 PTG reminder
by Jon Bernard 22 Oct '24

22 Oct '24

Hello all, We covered the schedule in last week's meeting but I wanted to post it here for reference. For Cinder, we are meeting Wednesday, Thursday, and Friday of this week from 14:00 UTC until 17:00 UTC. I have consolidated and updated our etherpad [1] to include both a tentative schedule that includes cross-project session commitments and the set of proposed topics. As we did in the previous PTG, we'll be using the same google meet link and the sessions will be recorded. Everything should be noted in the etherpad. If you notice something is missing please add it or reach out with any questions. See you all in a couple of days! [1] https://etherpad.opendev.org/p/epoxy-ptg-cinder -- Jon

1 0