April 2022 - openstack-discuss - lists.openstack.org

[tc][all][ Zed Virtual PTG RBAC discussions Summary
by Ghanshyam Mann 27 Apr '22

27 Apr '22

Hello Everyone, I tried to attend the RBAC-related sessions on various projects[i] but I am sure I might have missed a few of them. I am summarizing the RBAC discussion on what open questions were from the project side and what we discussed in TC PTG. Feel free to append the discussion you had in your project or any query you want TC to solve. Current status: ------------------ * I have started this etherpad[ii] to track the status of this goal, please keep it up to date as you progress the work in your project. Open question: ------------------ 1. heat create_stack API calling the mixed scope APIs (for example create flavor and create server). what is best scope for heat API so that we do not have any security leak. We have not concluded the solution yet as we need the heat team also join the discussion and agree on that. But we have a few possible solutions listed below: ** Heat accepts stack API with system scope *** This means a stack with system resources would require a system admin role => Need to check with services relying on Heat ** Heat assigns a project-scope role to the requester during a processing stack operation and uses this project scope credential to manage project resources ** Heat starts accepting the new header accepting the extra token (say SYSTEM_TOKEN) and uses that to create/interact the system-level resource like create flavor. 2. How to isolate the host level attribute in GET APIs? (cinder and manila have the same issue). Cinder GET volume API response has the host information. One possible solution we discussed is to have a separate API to show the host information to the system user and the rest of the volume response to the project users only. This is similar to what we have in nova. Then we have a few questions from the Tacker side, where tacker create_vnf API internally call heat create_stack and they are planning to make create_vnf API for non-admin users. Direction on enabling the enforce scope by default ------------------------------------------------------------ As keystone, nova, and neutron are ready with the new RBAC, we wanted to enable the scope checks by default. But after seeing the lack of integration testing and the above mentioned open question (especially heat and any deployment project breaking) we decided to hold it. As the first step, we will migrate the tempest tests to the new RBAC and will enable the scope for these services in devstack. And based on the testing results we will decide on it. But after seeing the amount of work needed in Tempest and on the open question, I do not think we will be able to do it in the Zed cycle. Instead, we will target to enable the 'new defaults' by default. We ran out of time in TC and will continue the discuss these in policy popup meetings. I will push the schedule to Ml. [i] https://etherpad.opendev.org/p/rbac-zed-ptg [ii] https://etherpad.opendev.org/p/rbac-goal-tracking -gmann

2 2

[Triple0 on Centos Stream] Wallaby Overcloud Deployment Issue
by Lokendra Rathour 27 Apr '22

27 Apr '22

Hi Team, We were trying to install Openstack Wallaby Release on Centos Stream. As we were trying the overcloud deployment, getting this error: 2022-04-26 17:20:51.768 374850 INFO tripleoclient.heat_launcher [-] Skipping container image pull. 2022-04-26 17:20:51.776 374850 INFO tripleoclient.heat_launcher [-] Checking that database is up 2022-04-26 17:20:52.251 374850 INFO tripleoclient.heat_launcher [-] Checking that message bus (rabbitmq) is up Error: Source image rejected: Error reading signature from https://access.redhat.com/webassets/docker/content/sigstore/ubi8/pause@sha2…: status 403 (Forbidden) 2022-04-26 17:20:57.687 374850 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Exception occured while running the command: subprocess.CalledProcessError: Command '['sudo', 'podman', 'play', 'kube', '/home/stack/overcloud-deploy/overcloud/heat-launcher/heat-pod.yaml']' returned non-zero exit status 125. 2022-04-26 17:20:57.687 374850 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent call last): 2022-04-26 17:20:57.687 374850 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run Please suggest. -- ~ Lokendra

2 2

[Kolla-ansible][Xena] Error deploying Cloudkitty
by wodel youchi 26 Apr '22

26 Apr '22

Hi, I am trying to deploy Cloudkitty, but I get this error message : TASK [cloudkitty : Creating Cloudkitty influxdb database] > ****************************************************** > task path: > /home/deployer/kollavenv/xenavenv/share/kolla-ansible/ansible/roles/cloudkitty/tasks/bootstrap.yml:36 fatal: [192.168.1.5 -> 192.168.1.5]: FAILED! => { > "action": "influxdb_database", > "changed": false, > "invocation": { > "module_args": { > "database_name": "cloudkitty", > "hostname": "dashint.cloud.cerist.dz", > "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", > "path": "", > "port": 8086, > "proxies": {}, > "retries": 3, > "ssl": false, > "state": "present", > "timeout": null, > "udp_port": 4444, > "use_udp": false, > "username": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", > "validate_certs": true > } > }, > "msg": "('Connection aborted.', RemoteDisconnected('Remote end closed > connection without response',))" > } On the influxdb container I did this : > [root@controllerb ~]# docker ps | grep inf > 68b3ebfefbec > 192.168.1.16:4000/openstack.kolla/centos-source-influxdb:xena > "dumb-init --single-…" 22 minutes ago Up 22 minutes > influxdb > [root@controllerb ~]# docker exec -it influxdb /bin/bash > (influxdb)[influxdb@controllerb /]$ influx > Failed to connect to http://localhost:8086: Get http://localhost:8086/ping: > dial tcp [::1]:8086: connect: connection refused > Please check your connection settings and ensure 'influxd' is running. > (influxdb)[influxdb@controllerb /]$ ps -ef > UID PID PPID C STIME TTY TIME CMD > influxdb 1 0 0 Apr18 ? 00:00:00 dumb-init > --single-child -- kolla_start > influxdb 7 1 0 Apr18 ? 00:00:01 /usr/bin/influxd > -config /etc/influxdb/influxdb.conf > influxdb 45 0 0 00:12 pts/0 00:00:00 /bin/bash > influxdb 78 45 0 00:12 pts/0 00:00:00 ps -ef > (influxdb)[influxdb@controllerb /]$ I have no log file for influxdb, the directory is empty. Any ideas? Regards.

4 7

Setup infrastructure failing
by Derek O keeffe 26 Apr '22

26 Apr '22

Hi all, We get the following error in the openstack.osa.db_setup : Create database for service task: failed: [infra1_keystone_container-e54c8ba5 -> infra1_utility_container-1a1eb7ce(xx.xx.xx.xxx)] (item={'name': 'keystone', 'users': [{'username': 'keystone', 'password': 'PASSWORD'}]}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "keystone", "users": [{"password": "PASSWORD", "username": "keystone"}]}, "msg": "unable to connect to database, check login_user and login_password are correct or /root/.my.cnf has the credentials. Exception message: (2003, \"Can't connect to MySQL server on 'xx.xx.xx.xxx' ([Errno 111] Connection refused)\")"} Also, ansible galera_container -m shell -a "mysql -h localhost -e 'show status like \"%wsrep_cluster_%\";'" is different to the example output: infra1_galera_container-90dd1571 | CHANGED | rc=0 >>Variable_name Valuewsrep_cluster_weight 1wsrep_cluster_capabilities wsrep_cluster_conf_id 1wsrep_cluster_size 1wsrep_cluster_state_uuid c06b8107-c53a-11ec-a54c-a20664362f6cwsrep_cluster_status Primary We checked our infra node and all the containers are present and mysql is installed on the infra1-galera-container and port 3306 is open. MariaDB [(none)]> show databases;+--------------------+| Database |+--------------------+| information_schema || mysql || performance_schema || sys |+--------------------+4 rows in set (0.001 sec) We are not sure what is causing this and it could possibly be that we have forgotten/skipped/missed a configuration step but we have gone over the process multiple times and cannot see what we may have missed. The previous playbooks did have a lot of "skipped" results compute1 : ok=120 changed=2 unreachable=0 failed=0 skipped=31 rescued=0 ignored=0 compute2 : ok=120 changed=2 unreachable=0 failed=0 skipped=31 rescued=0 ignored=0 infra1 : ok=156 changed=3 unreachable=0 failed=0 skipped=31 rescued=0 ignored=0 infra1_aodh_container-7c703324 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_ceilometer_central_container-7590db72 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_cinder_api_container-91806531 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_galera_container-90dd1571 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_glance_container-fff39ac4 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_gnocchi_container-ffd32a8b : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_heat_api_container-a4079838 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_horizon_container-9c041ec0 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_keystone_container-e54c8ba5 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_memcached_container-dc50bcd8 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_neutron_server_container-bfc4d0d2 : ok=92 changed=37 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_nova_api_container-f231bb13 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_placement_container-108d79df : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_rabbit_mq_container-50ca98ed : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_repo_container-ccade01d : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_utility_container-1a1eb7ce : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 localhost : ok=19 changed=0 unreachable=0 failed=0 skipped=14 rescued=0 ignored=0 compute1 : ok=0 changed=0 unreachable=0 failed=0 skipped=10 rescued=0 ignored=0 compute2 : ok=0 changed=0 unreachable=0 failed=0 skipped=10 rescued=0 ignored=0 infra1 : ok=38 changed=0 unreachable=0 failed=0 skipped=35 rescued=0 ignored=0 infra1_galera_container-90dd1571 : ok=69 changed=3 unreachable=0 failed=0 skipped=12 rescued=0 ignored=0 infra1_memcached_container-dc50bcd8 : ok=16 changed=0 unreachable=0 failed=0 skipped=4 rescued=0 ignored=0 infra1_rabbit_mq_container-50ca98ed : ok=69 changed=33 unreachable=0 failed=0 skipped=14 rescued=0 ignored=0 infra1_utility_container-1a1eb7ce : ok=40 changed=18 unreachable=0 failed=0 skipped=11 rescued=0 ignored=0 A lot were around Galera tasks but we are not sure if they were skipped intentionally as the playbooks ran to completion.Any help or troubleshooting steps would be appreciated. Thanks in advance. Regards,Derek

2 1

[skyline] skyline-console whl package question
by Satish Patel 26 Apr '22

26 Apr '22

Folks, I am compiling skyline-apiserver on my bare metal server and found Makefile pulling skyline-console tarball from the following location but that package doesn't contain the latest merged patches. Does CI job is broken which compiles this tarball ? https://tarballs.opendev.org/openstack/skyline-console/

2 2

[oslo][ops] Change in default logging format
by Stephen Finucane 26 Apr '22

26 Apr '22

I'm not sure how many people read release notes for the various oslo libraries, so I'm posting this here since it might have a minor impact for some tooling operators are using. We've just merged [1] which changes the default logging format used by oslo.log, configured using the '[DEFAULT] logging_context_format_string' config option. Previously, our logging format was: %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s This looks like e.g. 2022-04-26 10:49:14.629 4187413 WARNING nova.conductor.api [req-ffddccd3-8804-4fc3-a2f3-a05af1eb569f - - - - -] This has now been modified to include the global request ID as well as the local request ID: %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(user_identity)s] %(instance)s%(message)s This looks like e.g.: 2022-04-26 10:53:16.756 4187784 DEBUG nova.service [None req-f1698728-a2d6-4e02-a6b4-cf2cd398c02f - - - - -] Join ServiceGroup membership for this service compute This will affect pretty much all services using the out-of-the-box logging configuration. However, I suspect very few if any users are using the standard configuration. DevStack overrides '[DEFAULT] logging_context_format_string' for all services [2] and I suspect other installers probably do the same. If you are using the standard configuration, you might need to modify some log parsing/scraping tooling when this rolls around or manually set '[DEFAULT] logging_context_format_string' in your service configuration files. Cheers, Stephen [1] https://review.opendev.org/c/openstack/oslo.log/+/838190 [2] https://github.com/openstack/devstack/blob/3b0c035b9/functions#L677-L707

1 0

[all][neutron][neutron-vpnaas] Maintainers needed
by Lajos Katona 26 Apr '22

26 Apr '22

Hi, In the last few cycles neutron-vpnaas has no serious maintainers, and most patches merged were from Neutron core team or from Release team. Recently even neutron-vpnaas gate jobs started to fail. During the Zed PTG we discussed this topic (see [1]), For the maintenance we need someone to be the contact person for the project, who takes care of the project’s CI and review patches, answers bugs. Of course that’s only a minimal requirement. If the new maintainer works on new features for the project, it’s even better :) If we don’t have any new maintainer(s) before milestone Zed-2, which is July 11 - July 15 week according to [2], we will start marking neutron-vpnaas as deprecated and in the next cycle (AA, or perhapc 2023.1) we will propose to retire the project. So if You are using this project now, or if You have customers who are using it, please consider the possibility of maintaining it. Otherwise, please be aware that it is highly possible that the project will be deprecated and moved out from the official OpenStack projects. [1]: https://etherpad.opendev.org/p/neutron-zed-ptg#L201 [2]: https://releases.openstack.org/zed/schedule.html Lajos Katona (lajoskatona)

2 2

[cinder] how cinder gets the volume name when restore from a backup?
by Tony Liu 26 Apr '22

26 Apr '22

Hi, Here is what I do. * Create a volume. * Create a backup of this volume. * Delete the volume. * Restore the volume from backup. The restored volume has different UUID, but the same name. I don't see volume name is stored in backup metadata. How does Cinder know the volume name when restore it from backup, given the volume is already deleted? Thanks! Tony

2 2

retention policy for deleted resource in database
by Tony Liu 26 Apr '22

26 Apr '22

Hi, I see the record for deleted resource stay in database. What's the retention policy for those records? Is it configurable? Any manual cleanup is required? Thanks! Tony

2 2

[neutron] Bug Deputy Report Apr 18 - Apr 24
by Oleg Bondarev 25 Apr '22

25 Apr '22

Hello Team, Bug report for the week of Apr 18 is below. Two OVN bugs (including Invalid one) need triage from the OVN team. High ------ - https://bugs.launchpad.net/neutron/+bug/1969615 - OVS: flow loop is created with openvswitch version 2.16 - Opinion - seems more an issue in OVS rather than a bug in Neutron - Unassigned Undecided -------------- - https://bugs.launchpad.net/neutron/+bug/1969592 - [OVN] Frequent DB leader changes causes 'VIF creation failed' on nova side - New - Unassigned Invalid --------- - https://bugs.launchpad.net/neutron/+bug/1969354 - ovn-controller don't update new flows - Invalid - but reporter updated with additional info - Unassigned Thanks, Oleg

1 0