July 2024 - openstack-discuss - lists.openstack.org

Updating Instance Flavor Properties in OpenStack Cluster Without Downtime
by Mia smith 25 Jul '24

25 Jul '24

Hello OpenStack Community, In our OpenStack cluster, we recently updated some flavor properties. As a result, we need to apply these changes to all currently active instances that were created with the old flavor properties. We are looking for a solution that allows us to update these instance properties without causing any downtime or needing to resize the instances. Could anyone provide suggestions on how to achieve this? Is there a recommended method or ad-hoc command that can be used to safely update the properties directly in the database? Thank you for your assistance!

3 8

[Openstack-ansible][train] multi node deployment with CEPH-ansible
by Danish Khan 25 Jul '24

25 Jul '24

Dear Team, I am trying to deploy an openstack-ansible train with Ceph-ansible. I am stuck in a loophole due to python dependencies. As suggested, I applied a fix for the rabbitmq version and I am able to deploy openstack-ansible train successfully without ceph. But when I am deploying it with ceph then it is getting stuck at below: TASK [ceph_client : Install ceph packages] ********************************************************************************************************************************* FAILED - RETRYING: Install ceph packages (5 retries left). FAILED - RETRYING: Install ceph packages (4 retries left). FAILED - RETRYING: Install ceph packages (3 retries left). FAILED - RETRYING: Install ceph packages (2 retries left). FAILED - RETRYING: Install ceph packages (1 retries left). fatal: [sp-uat-infra-c1]: FAILED! => {"attempts": 5, "cache_update_time": 1721053914, "cache_updated": false, "changed": false, "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\" install 'ceph-common' 'python-ceph'' failed: E: Unable to correct problems, you have held broken packages.\n", "rc": 100, "stderr": "E: Unable to correct problems, you have held broken packages.\n", "stderr_lines": ["E: Unable to correct problems, you have held broken packages."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSome packages could not be installed. This may mean that you have\nrequested an impossible situation or if you are using the unstable\ndistribution that some required packages have not yet been created\nor been moved out of Incoming.\nThe following information may help to resolve the situation:\n\nThe following packages have unmet dependencies:\n python-ceph : Depends: python-cephfs (>= 12.2.13-0ubuntu0.18.04.11) but it is not going to be installed\n Depends: python-rados (>= 12.2.13-0ubuntu0.18.04.11) but it is not going to be installed\n Depends: python-rbd (>= 12.2.13-0ubuntu0.18.04.11) but it is not going to be installed\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Some packages could not be installed. This may mean that you have", "requested an impossible situation or if you are using the unstable", "distribution that some required packages have not yet been created", "or been moved out of Incoming.", "The following information may help to resolve the situation:", "", "The following packages have unmet dependencies:", " python-ceph : Depends: python-cephfs (>= 12.2.13-0ubuntu0.18.04.11) but it is not going to be installed", " Depends: python-rados (>= 12.2.13-0ubuntu0.18.04.11) but it is not going to be installed", " Depends: python-rbd (>= 12.2.13-0ubuntu0.18.04.11) but it is not going to be installed"]} python-ceph is not getting install due to dependencies: #vi /etc/ansible/roles/ceph_client/vars/debian.yml. python_ceph_packages: - python-ceph - python3-rados - python3-rbd I tried to update *python-ceph* with *python3-ceph* and the deployment was successful but the cinder-volume service is not coming up due to python error: #journalctl -u cinder-volume.service -f 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service [req-53d87e63-50ab-4976-b531-2b8ffdb82900 - - - - -] Error starting thread.: TypeError: expected bytes, str found 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service Traceback (most recent call last): 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/oslo_service/service.py", line 810, in run_service 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service service.start() 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/service.py", line 229, in start 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service service_id=Service.service_id) 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/volume/manager.py", line 444, in init_host 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service self._init_host(added_to_cluster, **kwargs) 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/volume/manager.py", line 478, in _init_host 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service self.driver.init_capabilities() 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/volume/driver.py", line 760, in init_capabilities 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service stats = self.get_volume_stats(True) 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/volume/drivers/rbd.py", line 643, in get_volume_stats 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service self._update_volume_stats() 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/volume/drivers/rbd.py", line 620, in _update_volume_stats 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service free_capacity, total_capacity = self._get_pool_stats() 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/volume/drivers/rbd.py", line 549, in _get_pool_stats 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service '{"prefix":"df", "format":"json"}', '') 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "rados.pyx", line 1067, in rados.Rados.mon_command 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service TypeError: expected bytes, str found 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service Regards, Danish Khan

2 5

[ops] [ec2-api] DB duplicate entry with ec2 list-images
by Massimo Sgaravatto 25 Jul '24

25 Jul '24

Dear all On a Yoga installation I see a problem with the ec2-api: [sgaravat@lxsgaravat ~]$ euca-describe-images -I ${EC2_ACCESS_KEY} -S ${EC2_SECRET_KEY} euca-describe-images: error (DBDuplicateEntry): Unknown error occurred. [sgaravat@lxsgaravat ~]$ In the log I see a duplicate entry errore [*] It's not fully clear to me what the problem is and why this query should add an entry in that table ... Any help is appreciated Thanks, Massimo PS: yes, I am aware that the ec2-api project is "dead" and not part of recent OpenStack distributions [*] 2024-07-25 14:49:11.052 37118 ERROR ec2api.api oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '4ae7a803-d7cd-4725-8b2e-65797a64ef94' for key 'items.items_os_id_idx'") 2024-07-25 14:49:11.052 37118 ERROR ec2api.api [SQL: INSERT INTO items (id, project_id, vpc_id, os_id, data) VALUES (%(id)s, %(project_id)s, %(vpc_id)s, %(os_id)s, %(data)s)] 2024-07-25 14:49:11.052 37118 ERROR ec2api.api [parameters: {'id': 'ami-e561c280', 'project_id': '56c3f5c047e74a78a71438c4412e6e13', 'vpc_id': None, 'os_id': '4ae7a803-d7cd-4725-8b2e-65797a64ef94', 'data': '{"is_public": true}'}] 2024-07-25 14:49:11.052 37118 ERROR ec2api.api (Background on this error at: https://sqlalche.me/e/14/gkpj) 2024-07-25 14:49:11.052 37118 ERROR ec2api.api 2024-07-25 14:49:11.055 37118 ERROR ec2api.api [req-cda08c1a-009c-4c7d-b019-c30bce32baa4 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a71438c4412e6e13 - - -] Environment: {"wsgi.url_scheme": "http", "REQUEST_METHOD": "POST", "SCRIPT_NAME": "", "RAW_PATH_INFO": "/services/Cloud", "PATH_INFO": "/services/Cloud", "CONTENT_TYPE": "application/x-www-form-urlencoded", "CONTENT_LENGTH": "53", "SERVER_PROTOCOL": "HTTP/1.0", "SERVER_NAME": "192.168.60.42", "SERVER_PORT": "8788", "REMOTE_ADDR": "192.168.60.229", "REMOTE_PORT": "33840", "GATEWAY_INTERFACE": "CGI/1.1", "HTTP_ACCEPT_ENCODING": "gzip, deflate", "HTTP_ACCEPT": "*/*", "HTTP_USER_AGENT": "euca2ools/3.4.1 (CPython 2.7.5; Linux 3.10.0-1160.92.1.el7.x86_64; x86_64) requestbuilder/0.7.1 requests/2.19.1", "HTTP_HOST": " cloud-areapd-test.pd.infn.it:8788", "HTTP_X_AMZ_CONTENT_SHA256": "afe617936fea90d9c62d9f87dd3e9d18e1fc5178d08aeebfb8737f32042a0b6d", "HTTP_X_AMZ_DATE": "20240725T124910Z", "HTTP_AUTHORIZATION": "AWS4-HMAC-SHA256 Credential=220cd4ffe5aa435cb56adfd5a9a3b234/20240725/undefined-15030/ec2/aws4_request, SignedHeaders=accept;accept-encoding;content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=c0cc81d64a4477db4dc8be0d478b8746bb013f7273b6460f411d95a5d6efe946", "HTTP_X_FORWARDED_PROTO": "https", "HTTP_X_FORWARDED_FOR": "193.205.157.174", "HTTP_CONNECTION": "close"}: oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '4ae7a803-d7cd-4725-8b2e-65797a64ef94' for key 'items.items_os_id_idx'") 2024-07-25 14:49:11.055 37118 INFO ec2api.api [req-cda08c1a-009c-4c7d-b019-c30bce32baa4 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a71438c4412e6e13 - - -] 0.811724s 192.168.60.229 POST /services/Cloud DescribeImages 500 [euca2ools/3.4.1 (CPython 2.7.5; Linux 3.10.0-1160.92.1.el7.x86_64; x86_64) requestbuilder/0.7.1 requests/2.19.1] application/x-www-form-urlencoded text/xml 2024-07-25 14:49:11.056 37118 INFO ec2api.wsgi.server [req-cda08c1a-009c-4c7d-b019-c30bce32baa4 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a71438c4412e6e13 - - -] 193.205.157.174,192.168.60.229 "POST /services/Cloud HTTP/1.1" status: 500 len: 348 time: 0.8139334 2024-07-25 14:49:14.391 37119 INFO ec2api.api [req-417ab595-646d-4468-9464-f19c7392c201 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a71438c4412e6e13 - - -] 4.822310s 192.168.60.229 POST /services/Cloud DescribeInstances 200 [euca2ools/3.4.1 (CPython 2.7.5; Linux 3.10.0-1062.9.1.el7.x86_64; x86_64) requestbuilder/0.7.1 requests/2.19.1] application/x-www-form-urlencoded text/xml 2024-07-25 14:49:14.392 37119 INFO ec2api.wsgi.server [req-417ab595-646d-4468-9464-f19c7392c201 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a71438c4412e6e13 - - -] 193.206.210.240,192.168.60.229 "POST /services/Cloud HTTP/1.1" status: 200 len: 11274 time: 4.8234410

1 0

[ironic] Call for maintainers: terraform-provider-ironic
by Dmitry Tantsur 25 Jul '24

25 Jul '24

Hi folks! In OpenShift, we used to use https://github.com/openshift-metal3/terraform-provider-ironic to drive Ironic and Inspector during the initial installation process before the control plane is up. Now that we have moved away from Terraform for this purpose, the project is on minimal support with the intention of completely abandoning it once no stable versions of OpenShift use it to provision the control plane. Are there any parties interested in maintaining terraform-provider-ironic after that point? It supports several features like inspection, BIOS settings, RAID. It does not, to the best of my knowledge, support Keystone authentication and has not been migrated away from Inspector. Dmitry

1 0

[cinder] No meeting today (7/24)
by Jon Bernard 24 Jul '24

24 Jul '24

Hello all, I failed in my effort to secure adequate childcare today, so I need to cancel today's meeting. Hope to see you in the next one. -- Jon

1 0

[horizon][puppet-openstack] Proper Python snippets placement
by Francesco Di Nucci 24 Jul '24

24 Jul '24

Hello, some time ago I found that on EL9 Horizon searches for extra Python snippets in "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d", while puppet-horizon places dashboards snippets in "/etc/openstack-dashboard/local_settings.d", so these are not loaded. At the same time, "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.py" is a symlink to "/etc/openstack-dashboard/local_settings". Of the following, what should be adopted as proper approach on a machine? * Having symlinks to /etc/openstack-dashboard for both local_settings.py and local_settings.d * Editing files in /usr/share/openstack-dashboard without having symlinks * Using a symlink for local_settings.py but placing snippets under /usr/share/openstack-dashboard Best regards -- Francesco Di Nucci System Administrator Compute & Networking Service, INFN Naples Email:francesco.dinucci@na.infn.it

4 4

[kolla-ansible]Material choices for ephemerals
by Franck VEDEL 24 Jul '24

24 Jul '24

Hello. I would like your opinion regarding a hardware configuration and my Openstack cluster project. I have the possibility of purchasing 6 servers to build this cluster for students, my current Openstack being too small. I have a proposition from the manufacturer (DELL): 6 servers with characteristics per server: - BOSS-N1 controller card + with 2 M.2 960GB (RAID 1) - 5 x 3.84TB Data Center NVMe Read Intensive AG Drive U2 Gen4 with Carrier - 12 x 64GB RDIMM, 4800MT/s Dual Rank My problem is: BOSS is used to install the system (Ubuntu 22.04 in my case). But, various tests on a small cluster with SSD disks showed that the solution with ephemerals was really the most interesting for what I want to do. Problem: I don't believe you can have the system on the BOSS and the ephemerals on the NVMe disks. So my questions are: 1) am I wrong and ephemerals are possible on NVMe (I use kolla-ansible to deploy, I haven't seen how to do that)? 2) or the solution is to create a Ceph cluster on the NVME of the 6 servers 3) or a better solution exists Thank you in advance if you can help me make these choices. Franck

2 2

Unrestricted AC Access Rules: "You must upgrade keystonemiddleware on all services"
by openstack＠tr.id.au 24 Jul '24

24 Jul '24

Hey there, I'm interested in creating an unrestricted Application Credential (AC) which is still constrained by Access Rules. A use could be allowing admins to use automation to generate read-only ACs for various purposes. The AC for automatically generating other ACs would need to be unrestricted, but should still only have the minimal privileges required to create its children. However, when attempting this with multiple versions of keystone, including 2024.1, I get the following error in the keystone service log: ``` Attempted to use application credential access rules with a middleware that does not understand them. You must upgrade keystonemiddleware on all services that accept application credentials as an authentication method.: NoneType: None ``` A minimal way to reproduce this: 1. Install keystone according to https://docs.openstack.org/keystone/latest/install/keystone-install-ubuntu.…. (I've also tested with keystone running in a virtualenv and in a kolla container.) 2. Generate a "parent" AC using the bootstrap admin: `openstack application credential create --max-width 120 --role admin --access-rules '[ { "path": "/v3/**", "method": "POST", "service": "identity" }, { "path": "/v3/**", "method": "GET", "service": "identity" } ]' --expiration '2024-07-31T23:59:59' --unrestricted test_unrestricted` (This gives expected output, including an AC id and secret.) 3. In another terminal, load the parent AC and then try to generate the child: `openstack application credential create --max-width 120 --role admin --access-rules '[ { "path": "/v3/**", "method": "GET", "service": "identity" } ]' --expiration '2024-07-31T23:59:59' test_read_only`. (This returns `Invalid application credential: Cannot create an application credential with unassigned role` in the client, as well as the earlier error about upgrading middleware in keystone service log.) I've tested that the parent/unrestricted AC *can* generate the child successfully when the parent does *not* have Access Rules. I didn't see anything about these errors or constraints when going through the application credential documentation (eg https://docs.openstack.org/keystone/latest/user/application_credentials.html) or in a web search, or in a search of the openstack-discuss archives. I may raise a bug report/feature request, but before I do that, I just wondered if I have likely overlooked some possible extra configuration or documentation which could explain what I've done wrong? Alternatively, if this were raised as a feature request, would it likely be supported, or would the functionality I'm imagining be too difficult or impossible to implement? Cheers, Tim

1 0

Scheduler and filters
by Gk Gk 23 Jul '24

23 Jul '24

Hi, Can anyone tell me the difference between the schedulers and filters in nova. As per what I grasped, it seems we have the same type of weighers like schedulers. Then what's the difference ? Please explain.. Thanks Y.G

3 2

[OSSA-2024-002] OpenStack Nova: Incomplete file access fix and regression for QCOW2 backing files and VMDK flat descriptors (CVE-2024-40767)
by Jeremy Stanley 23 Jul '24

23 Jul '24

================================================================== OSSA-2024-002: Incomplete file access fix and regression for QCOW2 backing files and VMDK flat descriptors ================================================================== :Date: July 23, 2024 :CVE: CVE-2024-40767 Affects ~~~~~~~ - Nova: <27.4.1, >=28.0.0 <28.2.1, >=29.0.0 <29.1.1 Description ~~~~~~~~~~~ Arnaud Morin (OVH) reported a vulnerability in Nova. By supplying a raw format image which is actually a specially crafted QCOW2 image with a backing file path or VMDK flat image with a descriptor file path, an authenticated user may convince systems to return a copy of the referenced file’s contents from the server resulting in unauthorized access to potentially sensitive data. All Nova deployments are affected. Patches ~~~~~~~ - https://review.opendev.org/924734 (2023.1/antelope) - https://review.opendev.org/924733 (2023.2/bobcat) - https://review.opendev.org/924732 (2024.1/caracal) - https://review.opendev.org/924731 (2024.2/dalmatian) Credits ~~~~~~~ - Arnaud Morin from OVH (CVE-2024-40767) References ~~~~~~~~~~ - https://launchpad.net/bugs/2071734 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-40767 Notes ~~~~~ - The patches linked above should apply cleanly to the public state of their respective branches at time of disclosure, and depend on some commits which merged after the `OSSA-2024-001 <https://security.openstack.org/ossa/OSSA-2024-001.html>`_ fixes as well as the final states of the Nova changes linked from that advisory (those did see some minor adjustments before they merged). - The QCOW2 issue is due to an incomplete fix in OSSA-2024-001 affecting systems where the ``use_cow_images`` configuration option is disabled, while the VMDK issue is a regression of the earlier `OSSA-2023-002 <https://security.openstack.org/ossa/OSSA-2023-002.html>`_ vulnerability reintroduced by the new implementation in OSSA-2024-001. Both problems were identified in the final hours before OSSA-2024-001 publication but, due to time constraints, were redacted from that bug and moved to a separate report. - Neither the methods introduced in these patches nor the fixes for OSSA-2024-001 are capable of blocking malicious images which are already resident in Nova's cache. At this time we do not have useful operator guidance for identifying and removing such existing images from the cache but strongly caution, if you do attempt to use the qemu-img tool to find them, to make sure you're using a version of it patched for `QEMU CVE-2024-4467 <https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-4467>`_. -- Jeremy Stanley OpenStack Vulnerability Management Team

1 0