February 2024 - openstack-discuss

[cinder] 2024.1 (Caracal) R-7 virtual mid cycle - 2 (14th Feb, 2024)
by Rajat Dhasmana 14 Feb '24

14 Feb '24

Hello Argonauts, We will be conducting the second midcycle of 2024.1 (Caracal) on 14th February, 2024. Following are the details: Date: 14th February, 2024 Time: 1400-1600 UTC Meeting link: https://meet.google.com/cqw-kbpn-sxx Etherpad: https://etherpad.opendev.org/p/cinder-caracal-midcycles We have 1 week so please add your topics timely. Thanks Rajat Dhasmana

1 1

Kolla-Ansible on Ubuntu 22.04 Fluentd Image peesnt for Label Check Error
by KK CHN 14 Feb '24

14 Feb '24

List, 1. I am trying kolla-ansible all-in-one on an Ubuntu VM on Esxi7.0 ( 22.04 Desktop OS. ) 2. Followed https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html *All went well upto here * * 3. Kolla-ansible -i ./all-in-one deploy * fails with message as follows.. Ensure Fluentd Image peesnt for Label Check *vNICs on this Ubuntu VM as follows* 1. "ens192" IP => 10.x.0.128 A DHCP IP of our organization's internal N/W) 2. "ens224 IP => 10.220.0.51 3. Added another interface ens160 without IP ( UP running) in globals.yml I have used 1. "ens224" for network_interface: "ens224" 2. "ens160" for neutron_external_interface: "ens160" 3. an unused 10.220.0.24 ip as kolla_internal_vip_address: "10.220.0.24" Hope I am right doing the interface configurations in globals.yml ? correct me if I am wrong. All went well up to * pre checks .*. *1. (kolla_ansible_venv) ubuntu@ubuntu-VM:~$ kolla-ansible -i ./all-in-one bootstrap-servers* Bootstrapping servers : ansible-playbook -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e kolla_action=bootstrap-servers /home/ubuntu/ PLAY RECAP ************************************************************ ********* localhost : ok=30 changed=2 unreachable=0 failed=0 skipped=24 rescued=0 ignored=0 2. (kolla_ansible_venv) ubuntu@ubuntu-VM:~$ kolla-ansible -i ./all-in-one prechecks PLAY RECAP ************************************************************ ********* localhost : ok=99 changed=0 unreachable=0 failed=0 skipped=158 rescued=0 ignored=0 *But * * 3. Kolla-ansible -i ./all-in-one deploy * fails with message as follows.. Ensure Fluentd Image peesnt for Label Check (kolla_ansible_venv) ubuntu@ubuntu-VM:~$ kolla-ansible -i ./all-in-one deploy TASK [common : Ensure fluentd image is present for label check] **************** fatal: [localhost]: FAILED! => {"changed": true, "msg": "'Traceback (most recent call last):\\n File \"/usr/local/lib/python3.10/dist-packages/docker/api/cli PLAY RECAP ********************************************************************* localhost : ok=6 changed=0 unreachable=0 failed=1 skipped=4 rescued=0 ignored=0 What went wrong? Any hints are much appreciated !!!! Full Error Report: Fatal: [localhost]: FAILED! => {"changed": true, "msg": "'Traceback (most recent call last):\\n File \"/usr/local/lib/python3.10/dist-packages/docker/api/client.py\", line 268, in _raise_for_status\\n response.raise_for_status()\\n File \"/usr/local/lib/python3.10/dist-packages/requests/models.py\", line 1021, in raise_for_status\\n raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.44/images/create?tag=master-ubuntu-jammy-aarch64&fromImage= quay.io%2Fopenstack.kolla%2Ffluentd\\n\\nThe above exception was the direct cause of the following exception:\\n\\nTraceback (most recent call last):\\n File \"/tmp/ansible_kolla_container_payload_fnpt9ldm/ansible_kolla_container_payload.zip/ansible/modules/kolla_container.py\", line 425, in main\\n File \"/tmp/ansible_kolla_container_payload_fnpt9ldm/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 522, in ensure_image\\n self.pull_image()\\n File \"/tmp/ansible_kolla_container_payload_fnpt9ldm/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 209, in pull_image\\n json.loads(line.strip().decode(\\'utf-8\\')) for line in self.dc.pull(\\n File \"/usr/local/lib/python3.10/dist-packages/docker/api/image.py\", line 429, in pull\\n self._raise_for_status(response)\\n File \"/usr/local/lib/python3.10/dist-packages/docker/api/client.py\", line 270, in _raise_for_status\\n raise create_api_error_from_http_exception(e) from e\\n File \"/usr/local/lib/python3.10/dist-packages/docker/errors.py\", line 39, in create_api_error_from_http_exception\\n raise cls(e, response=response, explanation=explanation) from e\\ndocker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.44/images/create?tag=master-ubuntu-jammy-aarch64&fromImage= quay.io%2Fopenstack.kolla%2Ffluentd: Not Found (\"manifest for quay.io/openstack.kolla/fluentd:master-ubuntu-jammy-aarch64 not found: manifest unknown: manifest unknown\")\\n'"} While installing kolla I have executed this as in the doc. pip install git+https://opendev.org/openstack/kolla-ansible@master Is the error above related to this step ?? If so what the solution ?/ More details as follows.. globals.yml parameters enabled ubuntu@ubuntu-VM:~$ grep ^[^#] /etc/kolla/globals.yml --- workaround_ansible_issue_8743: yes kolla_base_distro: "ubuntu" openstack_tag_suffix: "-aarch64" kolla_internal_vip_address: "10.220.0.24" network_interface: "ens224" neutron_external_interface: "ens160" ubuntu@ubuntu-VM:~$ VM details *ubuntu@ubuntu-VM:~$ cat /etc/lsb-release* DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS" *ubuntu@ubuntu-VM:~$ cat /etc/hosts* 127.0.0.1 localhost 10.220.0.51 localhost # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters # BEGIN ANSIBLE GENERATED HOSTS 10.220.0.51 ubuntu-VM # END ANSIBLE GENERATED HOSTS *Thanks in advance * *Krishane*

4 3

[tc][releases][ptl][all] you don't need your own unmaintained-core group!
by Brian Rosmaita 13 Feb '24

13 Feb '24

Hello Stackers, Thank you for your help in transitioning the Yoga series to Unmaintained status. We want to stress that there is a global openstack-unmaintained-core team that, by default, has merge permissions on *all* branches in the unmaintained/* namespace. Therefore, you do *not* need to create a specific core group for your project's unmaintained branches. The aim of Unmaintained status is to free up project teams from worrying about the maintenance of old branches. Thus, by design, project core teams do not have merge powers on these branches. Instead, the merry group of stalwart volunteers who have an interest in keeping legacy branches open have taken on this responsibility. If you are interested in helping to keep Unmaintained branches alive, reach out to the openstack-unmaintained-core team members [0] and let them know your interest. They will also be following up with an email, sometime this week, outlining how to join them. To summarize: you should only create your own separate group if you want to maintain the unmaintained branches yourself and don't want the larger group to do it for you. Otherwise you have no such obligation. If you are interested in helping with the unmaintained branches, it will help the community more if you join the global team instead creating your own. Finally, anyone can propose patches to an unmaintained branch, and can +1 or -1 reviews, just like for any other branch. So there's plenty you can do even without being part of an unmaintained-core team. Thanks for your attention to this email! [0] https://review.opendev.org/admin/groups/4d728691952c04b8b2ec828eabc96b98dc1…

1 0

[nova][nova-novncproxy] handler exception: Origin header protocol does not match this host.
by Satish Patel 13 Feb '24

13 Feb '24

Folks, Life was good till I restarted nova-novncproxy services on all controller nodes after that my vnc console stopped working. I can directly connect to the vm console using vnc viewer but not from Horizon. This is not a new setup or something. What could be wrong here and how do I debug this protocol issue? This is what I have on controller nodes [vnc] novncproxy_host = 192.168.8.14 novncproxy_port = 6080 server_listen = 192.168.8.14 server_proxyclient_address = 192.168.8.14 This is what I have on all computes nodes [vnc] novncproxy_host = 192.168.8.18 novncproxy_port = 6080 server_listen = 192.168.8.18 server_proxyclient_address = 192.168.8.18 novncproxy_base_url = https://os2.example.com:6080/vnc_lite.html 2024-02-13 20:08:16.315 7 INFO nova.console.websocketproxy [-] In exit 2024-02-13 20:08:30.402 7 INFO nova.console.websocketproxy [-] WebSocket server settings: 2024-02-13 20:08:30.403 7 INFO nova.console.websocketproxy [-] - Listen on 192.168.8.14:6080 2024-02-13 20:08:30.403 7 INFO nova.console.websocketproxy [-] - Web server (no directory listings). Web root: /usr/share/novnc 2024-02-13 20:08:30.404 7 INFO nova.console.websocketproxy [-] - No SSL/TLS support (no cert file) 2024-02-13 20:08:30.404 7 INFO nova.console.websocketproxy [-] - proxying from 192.168.8.14:6080 to None:None 2024-02-13 20:08:42.800 7 DEBUG nova.console.websocketproxy [-] 192.168.8.15: new handler Process vmsg /var/lib/kolla/venv/lib/python3.10/site-packages/websockify/websockifyserver.py:605 2024-02-13 20:08:42.810 71 INFO nova.console.websocketproxy [-] 192.168.8.15 - - [13/Feb/2024 20:08:42] 192.168.8.15: Plain non-SSL (ws://) WebSocket connection 2024-02-13 20:08:42.810 71 INFO nova.console.websocketproxy [-] 192.168.8.15 - - [13/Feb/2024 20:08:42] 192.168.8.15: Path: '/?token=03ad44a0-b876-4725-a71f-d383c15efb63' 2024-02-13 20:08:42.832 71 DEBUG oslo_db.sqlalchemy.engines [None req-dc3e111c-7a79-4d56-8077-688c9e813700 - - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /var/lib/kolla/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:335 2024-02-13 20:08:42.923 71 DEBUG nova.objects.console_auth_token [None req-dc3e111c-7a79-4d56-8077-688c9e813700 - - - - - -] Validated token - console connection is ConsoleAuthToken(access_url_base='https://os2.example.com:6080/vnc_lite.html',console_type='novnc',created_at=2024-02-13T20:08:41Z,host='192.168.8.18',id=3993,instance_uuid=7b6e0c71-ea3c-4314-9a4c-1fc86d5680f2,internal_access_path=None,port=5920,token='***',updated_at=None) validate /var/lib/kolla/venv/lib/python3.10/site-packages/nova/objects/console_auth_token.py:153 2024-02-13 20:08:42.961 71 DEBUG oslo_concurrency.lockutils [None req-dc3e111c-7a79-4d56-8077-688c9e813700 - - - - - -] Acquiring lock "compute-rpcapi-router" lock /var/lib/kolla/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312 2024-02-13 20:08:42.961 71 DEBUG oslo_concurrency.lockutils [None req-dc3e111c-7a79-4d56-8077-688c9e813700 - - - - - -] Acquired lock "compute-rpcapi-router" lock /var/lib/kolla/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315 2024-02-13 20:08:42.967 71 INFO nova.compute.rpcapi [None req-dc3e111c-7a79-4d56-8077-688c9e813700 - - - - - -] Automatically selected compute RPC version 6.2 from minimum service version 66 2024-02-13 20:08:42.968 71 DEBUG oslo_concurrency.lockutils [None req-dc3e111c-7a79-4d56-8077-688c9e813700 - - - - - -] Releasing lock "compute-rpcapi-router" lock /var/lib/kolla/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333 2024-02-13 20:08:43.008 71 INFO nova.console.websocketproxy [None req-dc3e111c-7a79-4d56-8077-688c9e813700 - - - - - -] handler exception: Origin header protocol does not match this host. 2024-02-13 20:08:43.008 71 DEBUG nova.console.websocketproxy [None req-dc3e111c-7a79-4d56-8077-688c9e813700 - - - - - -] exception vmsg /var/lib/kolla/venv/lib/python3.10/site-packages/websockify/websockifyserver.py:605 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy Traceback (most recent call last): 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/var/lib/kolla/venv/lib/python3.10/site-packages/websockify/websockifyserver.py", line 662, in top_new_client 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy client = self.do_handshake(startsock, address) 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/var/lib/kolla/venv/lib/python3.10/site-packages/websockify/websockifyserver.py", line 590, in do_handshake 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy self.RequestHandlerClass(retsock, address, self) 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/console/websocketproxy.py", line 93, in __init__ 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy websockify.ProxyRequestHandler.__init__(self, *args, **kwargs) 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/var/lib/kolla/venv/lib/python3.10/site-packages/websockify/websockifyserver.py", line 87, in __init__ 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy super().__init__(req, addr, server) 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/usr/lib/python3.10/http/server.py", line 668, in __init__ 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy super().__init__(*args, **kwargs) 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/usr/lib/python3.10/socketserver.py", line 747, in __init__ 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy self.handle() 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/var/lib/kolla/venv/lib/python3.10/site-packages/websockify/websockifyserver.py", line 297, in handle 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy super().handle() 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/usr/lib/python3.10/http/server.py", line 433, in handle 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy self.handle_one_request() 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/var/lib/kolla/venv/lib/python3.10/site-packages/websockify/websocketserver.py", line 40, in handle_one_request 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy super().handle_one_request() 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/usr/lib/python3.10/http/server.py", line 421, in handle_one_request 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy method() 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/var/lib/kolla/venv/lib/python3.10/site-packages/websockify/websocketserver.py", line 50, in _websocket_do_GET 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy self.handle_upgrade() 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/var/lib/kolla/venv/lib/python3.10/site-packages/websockify/websockifyserver.py", line 203, in handle_upgrade 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy super().handle_upgrade() 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/var/lib/kolla/venv/lib/python3.10/site-packages/websockify/websocketserver.py", line 77, in handle_upgrade 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy self.handle_websocket() 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/var/lib/kolla/venv/lib/python3.10/site-packages/websockify/websockifyserver.py", line 241, in handle_websocket 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy self.new_websocket_client() 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/console/websocketproxy.py", line 215, in new_websocket_client 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy raise exception.ValidationError(detail=detail) 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy nova.exception.ValidationError: Origin header protocol does not match this host. 2024-02-13 20:08:43.008 71 ERROR nova.console.websocketproxy

1 0

[openstack-helm] Ceph-osd + Jammy + loop device is broken now
by Vladimir Kozhukalov 13 Feb '24

13 Feb '24

Hi team, Due to this [1] some of our test jobs which utilize the ceph-osd chart and which run on Ubuntu Jammy are broken now. Recent changes in ioctl are incompatible with parted when used with loop devices. Ceph-osd chart uses parted in the osd init script. So it does not work now with the error like this ``` Error: Partition(s) 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64 on /dev/loop100 have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use. As a result, the old partition(s) will remain in use. You should reboot now before making further changes. ``` The issue appears only on Ubuntu Jammy. My suggestion is to temporarily disable the 2023.2 TLS job which is broken now. And to use the 2023.1 TLS job on Focal. Please review https://review.opendev.org/c/openstack/openstack-helm/+/908947 [1] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2049689 -- Best regards, Kozhukalov Vladimir

1 0

[openstack-ansible] Bug fighting day
by Dmitriy Rabotyagov 13 Feb '24

13 Feb '24

Hey everyone. During the previous meeting we decided that it might be a good idea to take a couple hours together to work on the project bug tracker [1]. We are going to focus on closing irrelevant bugs that were issued towards EOLed releases and do not apply to any supported code or were already fixed. Along with that we will try to triage and prioritize ones that are still relevant and need some attention. In order to decide on the date/time when this activity will take place I have created a poll [2]. It will be open till Sunday, Feb 11. Everyone who is willing to help us is very welcome to join the poll. If you have an outstanding bug that somehow slipped our attention - this is really a good chance to bring it up to the team as well. [1] https://bugs.launchpad.net/openstack-ansible [2] https://doodle.com/meeting/organize/id/enGV50Wa

1 2

http://apps.openstack.org/#tab=murano-apps unavailable
by Karera Tony 13 Feb '24

13 Feb '24

I have deployed Murano in my environment, However when I tried to get the application budle following the Murano guide from http://apps.openstack.org/#tab=murano-apps. I realized that the site is down. Is there an alternative ? Regards Regards Tony Karera

2 1

[nova]: openstack-ansible Ussuri - rabbitmq connection distribution
by keshav bareja 13 Feb '24

13 Feb '24

2 1

[magnum] Weekly meeting (14/Feb/2024) cancelled
by Michał Nasiadka 13 Feb '24

13 Feb '24

Hello, Due to absence of key team members - the weekly meeting tomorrow (14/Feb/2024) is cancelled. See you next week! Best regards, Michal

1 0

[all][tc] OpenStack 2024.2 testing runtime
by Ghanshyam Mann 12 Feb '24

12 Feb '24

Hi All, Its time to deifne the testing runtime for next cycle 2024.2. I have proposed it in gerrit, feel free to review/feedback there. - https://review.opendev.org/c/openstack/governance/+/908862 Changes from previous cycle testing runtime for quick ref: * Removing python 3.8 support because it is EOLing in oct 2024[1] which means python 3.9 will be the minimum python version in testing runtime. * Remove Debian 11 from Additional testing support because 2024.2 is non-SLURP release and previous SLURP release 2024.1 supported Debian 12 also which is what can be used as smooth upgrade to future SLURP release 2025.1 * Removing Nodejs from runtime: Nodejs is not a runtime requirement for Horizon instead it is one of the test dependency. It does not fit as a runtime and can be handle by Horizon team based. Added Horizon team to review this change. -gmann

1 0