April 2024 - openstack-discuss - lists.openstack.org

Instance stuck in BUILD status
by 西岭 26 Apr '24

26 Apr '24

Dear openstack trove module developers, I was having trouble creating a trove instance and kept showing build status. My create command is: trove create mysql_zh-11 1 --size 5 --nic net-id=5f161d13-0cfb-4707-9bd6-f23850d86fbf --databases dbtest --users user:password --datastore mysql --datastore_version mysql-5.6 After entering the trove list command, the final state is: | bdcb298c-86d3-42c5-b7af-1cc0c7f93904 | mysql_zh-11 | mysql | mysql-5.6 | BUILD | 1 | 5 | RegionOne | The image I used was downloaded from the openstack website: trove-train-mysql-ubuntu-xenial Enter the openstack server list command and it will display: 2bf580eb-3c93-4689-9b53-e76307beb241 | mysql_zh-11 | ACTIVE | external-net=192.168.0.56 | trove-train-mysql-ubuntu-xenial | mysql-mini | I don't know the username and password of the image, so I can't see the specific information inside the cloud host. In addition, when I used the tool to change the login username and password of the above image, and created the trove instance with the image, after waiting 3600 seconds, it showed: trove.taskmanager.models [-] Service status: ERROR, service error description: guestagent error: TroveError: Failed to get server 0908ee4e-9f1d-444a-9c69-c8b8cc9d8938 for instance 03c2a10c-87f1-4169-9cf7-13735ff8756c, error: Failed to validate token (HTTP 404) (Request-ID: req-9604fb4c-4aae-4351-8996-5ee4dfbd9ad3) This time, I was able to log in to the cloud host generated during the creation of trove. After entering, I found that the guest-agent service failed to start. The reasons were: sudo chown -R ubuntu:root /etc/trove/conf.d cannot be executed. I wonder if it is because the password of the image has been changed? Then, I used the same image to create the trove instance again, and during the instance creation process, I went to the cloud host and started the guest-agent service manually. The result was the same. What else did I miss when creating the instance? How to solve this problem? See attachment for details of trove log. Thank you very much for your help. ----------------------------------------------------- Xi Ling, from Qingdao, China April 25, 2024

2 1

[openstack][nova] Resize instance cause instance error
by Nguyễn Hữu Khôi 26 Apr '24

26 Apr '24

Hello everyone. Sometimes, I resize instance and this instance cannot be resized then it will be error state. I use Kolla-Ansbile Yoga with Dell Powestore5000T as backend with iscsi protocol. I report in this: https://bugs.launchpad.net/nova/+bug/2063345 Pls give some ideas on it? Thank you much Nguyen Huu Khoi

1 1

red hat integration with ceph error
by Adivya Singh 25 Apr '24

25 Apr '24

hi, anybody see this error before, while integrating red hat open stack with ubuntu ceph overcloud-computeids-2 | error={"ansible_job_id": "34 6042025410.662459", "attempts": 2, "changed": false, "finished": 1, "msg": "Paunch failed with config_id tripleo_step4", "rc": 1, "stderr": "Error executing ['podman', 'container', 'exists' , 'nova_libvirt_init_secret']: returned 1\nDid not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_libvirt_init_secret', '--filter', 'label=config_id=tri pleo_step4', '--format', '{{.Names}}']\" - retrying without config_id\nDid not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_libvirt_init_secret', '--f ormat', '{{.Names}}']\"\nError executing ['podman', 'run', '--name', 'nova_libvirt_init_secret', '--label', 'config_id=tripleo_step4', '--label', 'container_name=nova_libvirt_init_secret', '--label', 'managed_by=tripleo-Computeids', '--label', 'config_data={\"command\": [\"/bin/bash\", \"-c\", \"/usr/bin/virsh secret-define --file /etc/nova/secret.xml && SECRET_UUID=$(fgrep \ \\\\"fsid = \\\\\" /etc/ceph/ceph.conf | cut -d \\\\\" \\\\\" -f 3) && test -n \\\\\"$SECRET_UUID\\\\\" && SECRET_KEY=$(fgrep \\\\\"key = \\\\\" /etc/ceph/ceph.client.openstack.keyring | cu t -d \\\\\" \\\\\" -f 3) && test -n \\\\\"$SECRET_KEY\\\\\" && /usr/bin/virsh secret-set-value --secret $SECRET_UUID --base64 $SECRET_KEY\\\\n\"], \"detach\": false, \"image\": \"192.168.0. 1:8787/rhosp-rhel8/openstack-nova-libvirt:16.2\", \"net\": \"host\", \"privileged\": false, \"security_opt\": \"label=disable\", \"user\": \"root\", \"volumes\": [\"/etc/hosts:/etc/hosts:ro \", \"/etc/localtime:/etc/localtime:ro\", \"/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro\", \"/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro\", \"/etc/pk i/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro\", \"/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro\", \"/etc/pki/tls/cert.pem:/etc/pki/tls/c ert.pem:ro\", \"/dev/log:/dev/log\", \"/etc/puppet:/etc/puppet:ro\", \"/var/lib/config-data/puppet-generated/nova_libvirt/etc/nova:/etc/nova:ro\", \"/etc/ceph:/etc/ceph:ro\", \"/etc/libvirt :/etc/libvirt\", \"/var/run/libvirt:/var/run/libvirt:shared,z\", \"/var/lib/libvirt:/var/lib/libvirt:shared\"], \"environment\": {\"TRIPLEO_CONFIG_HASH\": \"f780b2585d2cb2edc5954608158cf938 \"}}', '--conmon-pidfile=/var/run/nova_libvirt_init_secret.pid', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/nova_libvirt_init_secret.log', '--env=TRIPLEO_CON regards Adivya Sungh

2 1

[I18n][PTG] 2024.2 Dalmatian PTG - I18n SIG summary
by Ian Y. Choi 25 Apr '24

25 Apr '24

Hi all, As I18n SIG lead, I would like to share with you a summary of the recent discussions held during the 2024.2 Dalmatian PTG regarding the Internationalization (I18n) Special Interest Group (SIG). Firstly, I would like to express my sincere appreciation to the entire community for their commitment to make OpenStack more internationalized. The efforts put forth in translating and making OpenStack accessible globally are truly commendable, thanks to the contributions of numerous individuals, although we are under unprecedented situations and even Zanata will sunset soon [1]. During the PTG, I18n SIG covered several important topics with active and invaluable contributors including Seongsoo (a special thanks for his leadership both I18n team and within the Korea User group): 1. Progress on open reviews for I18n: We discussed pending items in the OpenStack/I18n repository and the status of various reviews. 2. Discussion on the OpenStack/I18n repository structure: This included plans to embrace the use of Weblate - wlc library and add tox environment and corresponding requirements.txt. 3. Pilot work progress on governance translation support: A demonstration was provided on setting up a new project with API and CLI. 4. Feasibility discussion on migrating from translate.openstack.org earlier than [1]. Deliberation on target languages and next steps for user survey translation. 5. User survey translation discussion: Target languages and next steps were deliberated upon. In sum, I18n will continue our effort to finalize the migration from Zanata to Weblate to emphasize the importance of I18n as a fundamental component for global OpenStack accessibility. Thank you for your continuation of the I18n SIG. If you have any questions or would like further information on any of the discussed topics, please don't hesitate to reach out. Best regards, /Ian [1] https://lists.osci.io/hyperkitty/list/zanata-sunset@lists.osci.io/thread/6F… [2] I18n PTG Etherpad: https://etherpad.opendev.org/p/apr2024-ptg-i18n [3] I18n PTG Team Photo 1: https://1drv.ms/i/s!AgSCQctkWshsgb8ZvM-l7Ol1UOxNjA?e=5tdLpb [3] I18n PTG Team Photo 2: https://cdn.discordapp.com/attachments/1192271703057911888/1228373792162840…

1 0

Compute Resources
by Gk Gk 25 Apr '24

25 Apr '24

Hi, Is there a way to find out the available ram, disk and vcpus on a compute node for vm creation using any placement APIs or any other way ?

3 3

[cinder][dev][ptl] NVMe/FC
by yian.zong＠dell.com 25 Apr '24

25 Apr '24

Hi cinder team, I'd like to know if cinder supports NVMe/FC. From nvmeof connector [1], it seems only RoCE(RDMA) and TCP. [1] https://opendev.org/openstack/os-brick/src/branch/master/os_brick/initiator… Thank you! Yian

5 5

[cinder][PTG] 2024.2 Dalmatian PTG summary
by Jon Bernard 25 Apr '24

25 Apr '24

Hello everyone, thanks for a very productive PTG for the Dalmatian cycle. Below is a brief summary of the highlights. The raw etherpad notes are here[1] and both the summary and transcribed etherpad notes are captured in our wiki here[2]. # Improve driver documentation When proposing new or updating existing drivers, our interface can be confusing. We agreed on an effort to migrate class documentation to the interface definitions, make clear the mandatory and optional methods for drivers, and maintain this documentation going forward. # User visible information in volume types Volume type metadata and extra specs are not visible to users, making it difficult to ascertain whether a volume type will lead to encryption or replication. Agreement was reached on metadata fields similar to Glance's metadefs, allowing drivers to report capabilities in a standard way and allowing that metadata to become visible to admins and users. # Optional backup driver dependencies Although volume driver dependencies are optional, those of our backup drivers are listed in requirements and are therefore always installed irrespective of deployment configuration. Agreement was reached to move these dependencies to driver-requirements. This should simplify efforts of both deployers and packagers. # Simplify deployments A few ideas were proposed to improve our deployment: * When cinder-volume is deployed in an active-active configuration, our `backend_host` parameter must be updated on every backend to have the same value, we would like to remove this additional step. * Use of predictable names to define tenants, instead of unique IDs. * Have dedicated quote section in our configuration for quote-related options. # Response schema validation A spec to add response schema validation in addition to our existing request validation was proposed. With adequate coverage, clients in various languages could be autogenerated (in theory). No significant objects were raised, agreement to review related patches this cycle. # Documentation of Ceph auth requirements We do not provide comprehensive and easy-to-find documentation on exactly what authentication/permission expectations there are between different services, leading deployers troubleshoot themselves and not be able to rely on upstream best practices. Agreement to improve this situation in the current cycle. # Cinder backup improvements An accumulation of bug fixes and performance improvements have stalled in the review queue. We went through the major ones as a team to attempt to bring awareness and unblock the remaining review requirements. See WIKI notes for specific details. # Migrating backups between multiple backends There is desire to support multiple backup backends where new backups go to a new backend while backups in an old backend remain readable. We want to avoid needing to create a full backup in the new backend to support incremental snapshots (and the associated charge). A spec will be proposed for review and work towards this goal will proceed during this cycle. # Cross-project with Glance In a cross-project collaboration with the Glance team, an improved method for image migration was proposed. Agreement to introduce a new migration operation in Glance. Cinder context was provided and a consensus on path forward was reached. # Cross-project with Nova In a cross-project collaboration with both Nova and Glance, the topic of image encryption was discussed. Dan Smith from the Nova team provided input from the Nova side. Cinder and Nova expect LUKS formatted images, but Glance currently supports GPG encrypted images - requiring re-encryption prior to use. It was noted that LUKS encrypted images can be created without root permission and the Glance team is now looking to drop GPG support and consolidate around LUKS as our unified format. # Active-Active support for NetApp A NetApp engineer raised questions about adding active-active support to the driver. Questions were answered and that work should proceed in this cycle. # Performance of parallel clone operations For the clone operation in cinder we're using a single distributed lock to prevent the source volume from being deleted mid-operation. This causes multiple concurrent clone operations to block. Under the right conditions, this can cause a significant performance degradation. Multiple possible solutions were discussed (read-write locks) and a consensus to use the DB was reached. Some details are unclear, awaiting a spec before moving forward. # Volume encryption with user defined keys Cinder does not currently support encryption with key provided by the user. Users could both manage their own keys and data could be recovered even if keys are lost at the deployment. There are several technical challenges to support this. Several of these hurdles were raised, more thought and research is needed before we have a spec that could be reviewed. [1] https://etherpad.opendev.org/p/dalmatian-ptg-cinder [2] https://wiki.openstack.org/wiki/CinderDalmatianPTGSummary -- Jon

2 1

[ironic][neutron] Another networking path for Ironic?
by Julia Kreger 25 Apr '24

25 Apr '24

Greetings folks, During the Ironic PTG last week, Ironic had our busiest PTG session on the topic of networking, and what to do in order to move things forward. I think the best way to frame how we reached the discussion is to keep in mind that projects in our community have been built upon boundaries of delineation. For example, Nova for is virtualization. Ironic’s area has been bare metal machine deployment and lifecycle management. Neutron’s area has been networking. Ironic has always fit a little more awkwardly into this model due to the nature of bare metal – as evidenced by the networking related (networking-generic-switch, networking-baremetal) projects living under Ironic’s governance driven by the need to be able to toggle physical switch ports to ensure a secure provisioning process. Now it's getting even more awkward as the lines between a server and a switch start to blur. Ironic is one of the few OpenStack projects that has significant use cases outside of a fully integrated OpenStack cluster. Additionally, even in the fully integrated case, bare metal is often not the primary use case considered -- quite understandable since virtualized networking is not a trivial problem. This combination of two factors: physical networking being a secondary use case for Neutron, and operators wanting to get physical network automation even without a fully integrated OpenStack, leads to difficulties both on the development side and the operational side. This creates a situation where Ironic’s answer to just basic network automation of “use Neutron and an ML2 plugin” is really not tenable for many infrastructure operators. The underlying challenge is that infrastructure operators need something for just basic management actions like attaching or detaching hosts from physical networks which also works in a model of respecting separation of duties. This is further complicated with DPUs and the changing landscape DPUs are creating by forcing a much more complex interaction when it comes to just basic networking needs for supporting the deployment of a bare metal host. To attach a network to a host, it is no longer just toggle a switchport configuration, it is potentially toggle a switchport *and* toggle configuration on a DPU device. And in this weeks’ Ironic PTG session, “Future of networking”[0], discussion of DPUs helped us reach a tipping point for the discussion. The idea of a separate network interface (for Ironic to use) which was not neutron, reached a point of being a chorus of contributors. So to put it in different words, Ironic is considering implementing functionality for Switch and DPU device network configuration management in terms of device attachment to networks which would likely leverage the existing ML2 plugin model such that Ironic can trigger port attachment and detachment directly. We’re not talking about doing anything beyond attachment and detachment. This means aspects like IPAM, Security Groups, Routing, and so on and so forth are entirely off the table for us. Our answer for such will continue to be “use neutron.” The discussion thus far has focused on not replacing existing plugins, but to enable their use. To use the plugins today, Neutron and the involvement of other OpenStack services are required – which is not viable for approximately half of Ironic’s usage base. For example, a Metal3 user is only using Ironic as an embedded tool, and if their environment requires VXLAN networks to be terminated on a DPU then they would need to create/invent tooling to facilitate that, or we can solve that with Ironic and account for the overall physical machine lifecycle at the same time. We also see a path where we can better integrate with Neutron and help operator’s meet more restrictive security requirements by delineating part of the functionality into Ironic’s scope when leveraging upstream solutions. We realize there are many unknowns, and welcome Neutron contributors to collaborate[1] with us, as this could ultimately impact project scope. Thoughts? Thanks, Julia [0]: https://etherpad.opendev.org/p/ironic-ptg-april-2024#L609 [1]: https://review.opendev.org/c/openstack/ironic-specs/+/916126/

2 2

Instance stuck in BUILD status
by 西岭 25 Apr '24

25 Apr '24

Dear openstack trove module developers, I was having trouble creating a trove instance and kept showing build status. My create command is: trove create mysql_zh-11 1 --size 5 --nic net-id=5f161d13-0cfb-4707-9bd6-f23850d86fbf --databases dbtest --users user:password --datastore mysql --datastore_version mysql-5.6 After entering the trove list command, the final state is: | bdcb298c-86d3-42c5-b7af-1cc0c7f93904 | mysql_zh-11 | mysql | mysql-5.6 | BUILD | 1 | 5 | RegionOne | The image I used was downloaded from the openstack website: trove-train-mysql-ubuntu-xenial Enter the openstack server list command and it will display: 2bf580eb-3c93-4689-9b53-e76307beb241 | mysql_zh-11 | ACTIVE | external-net=192.168.0.56 | trove-train-mysql-ubuntu-xenial | mysql-mini | I don't know the username and password of the image, so I can't see the specific information inside the cloud host. In addition, when I used the tool to change the login username and password of the above image, and created the trove instance with the image, after waiting 3600 seconds, it showed: trove.taskmanager.models [-] Service status: ERROR, service error description: guestagent error: TroveError: Failed to get server 0908ee4e-9f1d-444a-9c69-c8b8cc9d8938 for instance 03c2a10c-87f1-4169-9cf7-13735ff8756c, error: Failed to validate token (HTTP 404) (Request-ID: req-9604fb4c-4aae-4351-8996-5ee4dfbd9ad3) This time, I was able to log in to the cloud host generated during the creation of trove. After entering, I found that the guest-agent service failed to start. The reasons were: sudo chown -R ubuntu:root /etc/trove/conf.d cannot be executed. I wonder if it is because the password of the image has been changed? Then, I used the same image to create the trove instance again, and during the instance creation process, I went to the cloud host and started the guest-agent service manually. The result was the same. What else did I miss when creating the instance? How to solve this problem? See attachment for details of trove log. Thank you very much for your help. ----------------------------------------------------- Xi Ling, from Qingdao, China April 25, 2024

1 0

Secure RBAC manager role
by Sławek Kapłoński 25 Apr '24

25 Apr '24

Hi, I started looking at the S-RBAC today and for the phase 3 [1] especially. My question is - do we have agreement how this MANAGER will look like? In the linked document there is only info that keystone's spec [2] will have to be changed but I'm not sure if this is final now and if we can/should start thinking and implementing policies for the MANAGER role or not yet. [1] https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rb… [2] https://review.opendev.org/c/openstack/keystone-specs/+/818603 -- Slawek Kaplonski Principal Software Engineer Red Hat

3 5