August 2024 - openstack-discuss - lists.openstack.org

Re: Migratekit: near-live migrations from VMware to OpenStack
by Oliver Weinmann 14 Sep '24

14 Sep '24

Hi Mohammed, We struggled using virt-v2v directly with OpenStack. Our approach was using ovftool to export the VM as OVA and then create a qcow2 image that we can import in OpenStack. So many manual steps involved. So I tested your tool aganst vsphere 7.x vcenter. But I just can't manage to get it working. It keeps complaining about the path of the source VM. I can use the following just fine with virt-v2v: virt-v2v -v -x --password-file pass.txt -ic vpx:// user123(a)vcenter.somedomain.local /GE%Office%20One/Projects/esx01.somedomain.local?no_verify=1 test-vm-01 Using the same in your tool: docker run -it --rm --privileged \ -v /dev:/dev \ -v /usr/lib64/vmware-vix-disklib/:/usr/lib64/vmware-vix-disklib:ro \ --env-file <(env | grep OS_) registry.atmosphere.dev/library/migratekit:latest migrate \ --vmware-endpoint vcenter.somedomain.local \ --vmware-username user123 \ --vmware-password XXXXXX \ --vmware-path /GE%Office%20One/Projects/esx01.somedomain.local/test-vm-01 Results in: Error: vm '/GE%Office%20One/Projects/ esx01.somedomain.local/test-vm-01 ' not found I tried many different variants of the path none has worked. Is there a way to lookup the path of a VM? I can also use: virsh -c 'vpx:// user123(a)vcenter.somedomain.local /GE%Office%20One/Projects/esx01.somedomain.local?no_verify=1 list --all which shows me all VMs running on that particular ESXi host. Best Regards, Oliver

2 9

[charms] Proposing Rodrigo Barbieri for OpenStack Charms stable core
by Edward Hope-Morley 10 Sep '24

10 Sep '24

Hi everyone, I'd like to propose Rodrigo Barbieri for the OpenStack Charms stable maintenance team. Rodrigo has been a contributor to the charms for many years and has an impressive history of contributions to across all the OpenStack charms. He always demonstrates an attention to detail and provides insightful feedback in reviews and as such I believe he would make a valuable addition to our charms-stable maintenance core team. Thank you for your contributions to the project Rodrigo! Regards, Ed

2 1

[swift] Ideas: Object Locking
by David Zanetti 10 Sep '24

10 Sep '24

Hello everyone! We've added an entry to the Swift ideas page about implementing Object Locks. The full description of what this would be and how we expect it to work is in the ideas page here: https://wiki.openstack.org/wiki/Swift/ideas/object-lock I have two questions of the community. The first is whether work on this would be accepted by the Swift project, as we wouldn't want to expend efforts on something we'd have to carry as our own patches to upstream. The second is if there is any interest from other people to collaborate on design/implementation of the locking feature. Happy to take replies off-list if people prefer, and thanks for reading this far! (Ob-reply: Yes, we know Ceph RGW does this already. We use Swift for Object for $REASONS, which we'd like to keep doing.) -- David Zanetti Chief Technology Officer [1]Aotearoa New Zealand's cloud provider e david.zanetti(a)catalystcloud.nz w catalystcloud.nz Follow us on LinkedIn [2] Level 9, 33 Federal Street, Auckland 1010 [1] https://catalystcloud.nz/ [2] Follow us on LinkedIn https://www.linkedin.com/company/catalyst-cloud-limited/

2 1

[all][tc][ptls] Eventlet: Python 3.13 - Attach your seatbelt
by Herve Beraud 06 Sep '24

06 Sep '24

Hi all, As Python 3.13 is close to become a reality, new concerning incompatibilities between this new CPython version and Eventlet also become a reality: - https://github.com/eventlet/eventlet/issues/964 Fedora Python maintainers proposed a patch, but for now the problem remains unsolved. Also, many people reported bugs related to incompatibility between python3.12/dnspython/eventlet/pip-system-cert. There is an example who summarize well the problem https://github.com/eventlet/eventlet/issues/957 Do you have some visibility concerning the adoption of Python 3.13 into Openstack? Turbulences ahead... -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

4 5

[charms] Proposing Brian Haley and Nicolas Bock for OpenStack Charms core
by Edward Hope-Morley 05 Sep '24

05 Sep '24

Hi everyone, I would like to propose Brian Haley and Nicolas Bock for the OpenStack Charms core development team. Both Brian and Nicolas have been members of the stable charms core team since 2023 and been hugely helpful in that role, providing insightful feedback to reviews as well as contributions of their own. Nicolas has also been a contributor to the OpenStack charms for many years and Brian is well versed in the ways of upstream OpenStack maintenance through his role as Neutron PTL so I have full confidence that they will prove valuable additions to the charms-core team. Thank you both for your contributions to the project! Regards, Ed

3 2

[Kolla] [LetsEncrypt] Internal/External/Backend Certificates
by jjjamesg＠proton.me 05 Sep '24

05 Sep '24

Hi all, another one here that can't seem to get kolla running with letsencrypt certs, just curious as to what settings are required for kolla to pull internal/external/backend certs from letsencrypt? I've looked at the docs and have tried a few settings but keep getting errors, sure i had this working last week before i tore it down ready to deploy for production and now i just cant get it to complete the deployment because of certs, only thing i can think of is i need to run kolla-ansible -i multinode certificates but i thought that was just for self made certs? Also, what are best practices for backend tls certs, self signed or using the internal cert etc?? Is it possible to use letsencrypt at all for backend? Cheers all

3 6

[cinder] Offering local storage using the device mapper clone target
by Jan Horstmann 04 Sep '24

04 Sep '24

Hello cinder team and other interested parties, I have recently started thinking about ways to offer local storage as part of the OpenStack block storage service. Naturally any solution would require to transparently migrate volumes to and with the instances they are attached to, so that the user experience of being able to access their storage from anywhere would not be degraded. Fortunately, with the device mapper clone target ([1]), a mechanism is available in the linux kernel which allows to do exactly that: Create copies of devices (even network attached ones) while allowing to immediately use the target device. I have detailed the motivation and some implications of having local storage in the block storage service in a blog post [2], which I do not want to repeat here in order to not make this mail longer than it needs to be. The blog post also outlines a way to implement a cinder driver leveraging this mechanism, which is loosely based on the way cinder migrates volumes between backends, i.e.: When a volume is going to be attached, a new volume is created on the instance's host, which is backed by a device mapper clone target with the original volume as its source; Then the volume objects are switched and a local connector is returned, which may be used immediately. Since then I went ahead and created a proof of concept cinder driver ([3]), which roughly follows what is outlined in the post (some lessons were already learned during implementation). The driver is based on the in-tree lvm driver for basic volume management and can do basic things like create/delete, attach/detach and also handles live migration and rebuilding of instances. There is nothing currently regarding snapshots and volume migrations will definitly not work as expected yet, but it should be sufficient to get a feeling for the general idea. I am currently looking for feedback on the idea and the implementation and would like to know if this is of interest to others. I am happpy to answer any questions or present this in more detail. In case anyone wants to take the driver for a spin I am going to leave some instructions on how to set up the environment in [4]. Regards, Jan [1] https://docs.kernel.org/admin-guide/device-mapper/dm-clone.html [2] https://scs.community/2024/07/12/local-block-storage-via-dm-clone/ [3] https://github.com/janhorstmann/openstack-cinder/tree/59e1808a24d112d2e8d08… [4] - Set up an OpenStack environment - Create an lvm2 volume group and run a cinder volume service on every hypervisor - Configure a new backend on every volume service with the same `volume_backend_name` Configure it like you would an lvm backend, but with `volume_driver = cinder.volume.drivers.dmclone.DMCloneVolumeDriver`, e.g.: [local-volumes] volume_driver = cinder.volume.drivers.dmclone.DMCloneVolumeDriver volume_group = local volume_backend_name = local-volumes target_helper = tgtadm target_protocol = iscsi lvm_type = default I have tested with the values above, but any target driver should work -- Jan Horstmann Senior Cloud Engineer Mail: horstmann(a)osism.tech Web: https://osism.tech OSISM GmbH Talweg 8 / 75417 Mühlacker / Deutschland Geschäftsführer: Christian Berendt Unternehmenssitz: Mühlacker Amtsgericht Mannheim, HRB 750852

1 1

[largescale-sig] Next meeting: August 21, 9utc
by Thierry Carrez 04 Sep '24

04 Sep '24

Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 9UTC. You can doublecheck how that UTC time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20240821T09 Feel free to add topics to the agenda: https://etherpad.opendev.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

3 2

[all][rbac][tacker][horizon][neutron][octavia] oslo.policy 4.4.0 enable the RBAC new defaults by default
by Ghanshyam Mann 03 Sep '24

03 Sep '24

Hi All, You might have seen some fixes and testing changes on your project to test the oslo.policy 4.4.0. oslo.policy 4.4.0 enables the RBAC new defaults by default, which means those will be enabled for all the OpenStack services unless they have disabled them by overriding the default value. It enables both enforce_scope as well enforce_new_defaults. The requirement change[1] adds the oslo.policy 4.4.0 in upper-constraints which will enable the latest oslo.policy in upstream CI. Along with jobs failing on requirement change, I tested more integration jobs and fixing the failures[2]. Most of the fixes have been merged, and a few of them are up for review. Below is the list of fixes up for review, please merge them before the requirement change is merged. If there is something else I missed to test and failed, let me know on IRC or here. In-progress fixes: - Horizon: - https://review.opendev.org/c/openstack/horizon/+/927571 - Tacker: - https://review.opendev.org/c/openstack/tacker/+/926089/5 - Neutron + Designates job (making it non-voting until we figure out the root cause): - https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/927648 - Octavia (Michael is working on more modifications on this ): - https://review.opendev.org/c/openstack/octavia-tempest-plugin/+/926867 [1] https://review.opendev.org/c/openstack/requirements/+/925464 [2] https://review.opendev.org/q/topic:%22secure-rbac%22+owner:gmann@ghanshyamm… -gmann

2 4

[ops][kolla][kolla-ansible] Lets Encrypt Deployment on 2024.1/v18
by Forrest Fuqua 03 Sep '24

03 Sep '24

The documentation seems a little incomplete when it comes to deploying a stack with let's encrypt The attached config is after trying a few settings, but the lets encrypt container errors out every time during deployment causing haproxy to not get a proper ssl certificate the services.d entries are showing its ignoring kolla_internal_fqdn_cert and still using kolla_internal.pem in most services causing the following deployment error TASK [service-ks-register : keystone | Creating services] **************************************************************************************************************************************************** FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (5 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (4 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (3 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (2 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (1 retries left). failed: [100.70.0.1] (item=keystone (identity)) => {"action": "os_keystone_service", "ansible_loop_var": "item", "attempts": 5, "changed": false, "item": {"description": "Openstack Identity Service", "endpoints": [{"interface": "internal", "url": "https://openstack.cyberrange.rit.edu:5000"}, {"interface": "public", "url": "https://openstack.cyberrange.rit.edu:5000"}], "name": "keystone", "type": "identity"}, "module_stderr": "Failed to discover available identity versions when contacting https://openstack.cyberrange.rit.edu:5000. Attempting to parse version from URL.\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 715, in urlopen\n httplib_response = self._make_request(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 404, in _make_request\n self._validate_conn(conn)\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 1058, in _validate_conn\n conn.connect()\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connection.py\", line 419, in connect\n self.sock = ssl_wrap_socket(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/util/ssl_.py\", line 449, in ssl_wrap_socket\n ssl_sock = _ssl_wrap_socket_impl(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/util/ssl_.py\", line 493, in _ssl_wrap_socket_impl\n return ssl_context.wrap_socket(sock, server_hostname=server_hostname)\n File \"/usr/lib/python3.10/ssl.py\", line 513, in wrap_socket\n return self.sslsocket_class._create(\n File \"/usr/lib/python3.10/ssl.py\", line 1100, in _create\n self.do_handshake()\n File \"/usr/lib/python3.10/ssl.py\", line 1371, in do_handshake\n self._sslobj.do_handshake()\nssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n resp = conn.urlopen(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 799, in urlopen\n retries = retries.increment(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/util/retry.py\", line 592, in increment\n raise MaxRetryError(_pool, url, error or ResponseError(cause))\nurllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1021, in _send_request\n resp = self.session.request(method, url, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n resp = self.send(prep, **send_kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n r = adapter.send(request, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/requests/adapters.py\", line 517, in send\n raise SSLError(e, request=request)\nrequests.exceptions.SSLError: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/generic/base.py\", line 133, in _do_create_plugin\n disc = self.get_discovery(session,\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 605, in get_discovery\n return discover.get_discovery(session=session, url=url,\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/discover.py\", line 1459, in get_discovery\n disc = Discover(session, url, authenticated=authenticated)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/discover.py\", line 539, in __init__\n self._data = get_version_data(session, url,\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/discover.py\", line 106, in get_version_data\n resp = session.get(url, headers=headers, authenticated=authenticated)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1154, in get\n return self.request(url, 'GET', **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 930, in request\n resp = send(**kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1025, in _send_request\n raise exceptions.SSLError(msg)\nkeystoneauth1.exceptions.connection.SSLError: SSL exception connecting to https://openstack.cyberrange.rit.edu:5000: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/tmp/ansible-tmp-1720019606.1431425-10481-181523778594420/AnsiballZ_catalog_service.py\", line 107, in <module>\n _ansiballz_main()\n File \"/tmp/ansible-tmp-1720019606.1431425-10481-181523778594420/AnsiballZ_catalog_service.py\", line 99, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/tmp/ansible-tmp-1720019606.1431425-10481-181523778594420/AnsiballZ_catalog_service.py\", line 47, in invoke_module\n runpy.run_module(mod_name='ansible_collections.openstack.cloud.plugins.modules.catalog_service', init_globals=dict(_module_fqn='ansible_collections.openstack.cloud.plugins.modules.catalog_service', _modlib_path=modlib_path),\n File \"/usr/lib/python3.10/runpy.py\", line 224, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib/python3.10/runpy.py\", line 96, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 211, in <module>\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 207, in main\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/module_utils/openstack.py\", line 415, in __call__\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 113, in run\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 175, in _find\n File \"/opt/ansible/lib/python3.10/site-packages/openstack/service_description.py\", line 89, in __get__\n proxy = self._make_proxy(instance)\n File \"/opt/ansible/lib/python3.10/site-packages/openstack/service_description.py\", line 289, in _make_proxy\n found_version = temp_adapter.get_api_major_version()\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/adapter.py\", line 352, in get_api_major_version\n return self.session.get_api_major_version(auth or self.auth, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1289, in get_api_major_version\n return auth.get_api_major_version(self, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 497, in get_api_major_version\n data = get_endpoint_data(discover_versions=discover_versions)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 268, in get_endpoint_data\n service_catalog = self.get_access(session).service_catalog\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 131, in get_access\n self.auth_ref = self.get_auth_ref(session)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/generic/base.py\", line 203, in get_auth_ref\n self._plugin = self._do_create_plugin(session)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/generic/base.py\", line 155, in _do_create_plugin\n raise exceptions.DiscoveryFailure(\nkeystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. SSL exception connecting to https://openstack.cyberrange.rit.edu:5000: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1} #config excerpts from globals.yml kolla_base_distro: "ubuntu" kolla_internal_vip_address: "100.70.0.20" # /etc/hosts updated on all hosts to have openstack.cyberrange.rit.edu point to the internal IP kolla_internal_fqdn: "openstack.cyberrange.rit.edu" kolla_external_vip_address: "129.21.246.130" kolla_external_fqdn: "openstack.cyberrange.rit.edu" kolla_enable_tls_internal: "yes" kolla_enable_tls_external: "yes" kolla_certificates_dir: "/etc/kolla/certificates" kolla_external_fqdn_cert: "{{ kolla_certificates_dir }}/haproxy.pem" kolla_internal_fqdn_cert: "{{ kolla_certificates_dir }}/haproxy.pem" #kolla_admin_openrc_cacert: "" kolla_copy_ca_into_containers: "yes" haproxy_backend_cacert: "{{ 'ca-certificates.crt' if kolla_base_distro in ['debian', 'ubuntu'] else 'ca-bundle.trust.crt' }}" haproxy_backend_cacert_dir: "/etc/ssl/certs" letsencrypt_email: "fffics(a)rit.edu" enable_letsencrypt: yes letsencrypt_cert_server: "https://acme-v02.api.letsencrypt.org/directory" # attempt to renew Let's Encrypt certificate every 12 hours letsencrypt_cron_renew_schedule: "0 */12 * * *"

3 2