April 2022 - openstack-discuss - lists.openstack.org

[Openstack Triple0 Wallaby] Deployment error
by Lokendra Rathour 28 Apr '22

28 Apr '22

Hi Team, we tried OpenStack deployment using *Triple0 Wallaby.* while we were trying to deploy the setup Undercloud was deployed successfully. For the Overcloud Deployment: we generated the templates: Command: ./usr/share/openstack-tripleo-heat-templates/tools/process-templates.py -o ~/openstack-tripleo-heat-templates-rendered_at_wallaby -n /home/stack/templates/network_data.yaml -r /home/stack/templates/roles_data.yaml and using the below command to deploy overcloud: openstack overcloud deploy --templates \ -n /home/stack/templates/network_data.yaml \ -r /home/stack/templates/roles_data.yaml \ -e /home/stack/templates/environment.yaml \ -e /home/stack/templates/environments/network-isolation.yaml \ -e /home/stack/templates/environments/network-environment.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml \ -e /home/stack/templates/ironic-config.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ -e /home/stack/containers-prepare-parameter.yaml after which it fails with error: 2022-04-27 12:34:00.801 607453 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Exception occured while running the command: ValueError: Failed to deploy: ERROR: HEAT-E99001 Service neutron is not available for* resource type OS::TripleO::Network::Ports::ControlPlaneVipPort, reason: neutron network endpoint is not in service catalog.* Traceback (most recent call last): we were able to deploy Triple0 Victoria/ Train /ussuri using the same approach. Please suggest any further changes needed to include in our setup. -Lokendra

1 1

ELK services moving to OpenSearch
by Daniel Pawlik 28 Apr '22

28 Apr '22

Hello, We would like to announce the new Elasticsearch service based on Opensearch. As it was earlier mentioned [1], the old service logstash.openstack.org is deprecated and will be decommissioned in the near future. Same as earlier, each Zuul CI job results will be sent to the Elasticsearch service. With the new CI log processing system [2], we decided to add more logs to be proceeded [3]. If some important files are missing, please let me know. To use this new service, please check out the documentation [4][5] or use the following credentials: url: https://opensearch.logs.openstack.org/_dashboards/app/discover?security_ten… username: openstack password: openstack tenant: global I would also like to take this opportunity to ask for help with the OpenStack ci-log-processing project [6] which is providing configuration of the Opensearch service as well as tools required to process CI logs and push them into the Opensearch service. Cheers, Dan [1] https://lists.openstack.org/pipermail/openstack-discuss/2021-May/022359.html [2] https://lists.openstack.org/pipermail/openstack-discuss/2022-February/02736… [3] https://opendev.org/openstack/ci-log-processing/src/branch/master/logscrape… [4] https://docs.openstack.org/project-team-guide/testing.html#checking-status-… [5] https://governance.openstack.org/sigs/tact-sig.html#opensearch [6] https://opendev.org/openstack/ci-log-processing

1 0

Nova support for multiple vGPUs?
by Sigurd Kristian Brinch 28 Apr '22

28 Apr '22

Hi, As far as I can tell, libvirt/KVM supports multiple vGPUs per VM (https://docs.nvidia.com/grid/14.0/grid-vgpu-release-notes-generic-linux-kvm…) but in OpenStack/Nova it is limited to one vGPU per VM (https://docs.openstack.org/nova/latest/admin/virtual-gpu.html#configure-a-f…) Is there a reason for this limit? What would be needed to enable multiple vGPUs in Nova? BR Sigurd

3 4

[Browbeat]performance/load testing
by Vishwanath 28 Apr '22

28 Apr '22

Hello, I installed openstack using kolla-ansible, i would like to do load/performance testing, i came across below openstack page. https://docs.openstack.org/self-healing-sig/latest/testing/tools-list.html Browbeat looks like a perfect fit for what i want to do.I read browbeat documentation . Questions i have is: - as per the browbeat documentation, in order to install this we need to have openstack installation via tripleo (undercloud/overcloud) , does browbeat work with non triiplo openstack deployment? - I don’t have undercloud/overcloud setup. Regards Vish

1 0

Openstack Trove - Polling request timed out
by Manish Bharti 28 Apr '22

28 Apr '22

Dear Team, We are trying to deploy OpenStack environment for our application and while deploying trove service we are facing the below error(attached screenshot)- Traceback (most recent call last): File "/usr/lib/python3/dist-packages/trove/common/utils.py", line 207, in wait_for_task return polling_task.wait() File "/usr/lib/python3/dist-packages/eventlet/event.py", line 125, in wait result = hub.switch() File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 313, in switch return self.greenlet.switch() File "/usr/lib/python3/dist-packages/oslo_service/loopingcall.py", line 154, in _run_loop idle = idle_for_func(result, self._elapsed(watch)) File "/usr/lib/python3/dist-packages/oslo_service/loopingcall.py", line 349, in _idle_for raise LoopingCallTimeOut( oslo_service.loopingcall.LoopingCallTimeOut: Looping call timed out after 870.99 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/trove/taskmanager/models.py", line 434, in wait_for_instance utils.poll_until(self._service_is_active, File "/usr/lib/python3/dist-packages/trove/common/utils.py", line 223, in poll_until return wait_for_task(task) File "/usr/lib/python3/dist-packages/trove/common/utils.py", line 209, in wait_for_task raise exception.PollTimeOut trove.common.exception.PollTimeOut: Polling request timed out. Please help us on this issue. -- Thank you , Manish Bharti Jodhpur, Raj - 342001. Contact:8875033000

2 1

[all][tc] Technical Committee next weekly meeting on April 28, 2022 at 1500 UTC
by Ghanshyam Mann 28 Apr '22

28 Apr '22

Hello Everyone, Technical Committee's next weekly meeting is scheduled for April 28, 2022 at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, April 27, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[openstack-helm] gnocchi chart removal
by Gage Hugo 28 Apr '22

28 Apr '22

Hey team, One of the topics from the last few PTGs have been reducing the scope of openstack-helm by evaluating which charts were being used and which ones have not been actively maintained and one of those charts that has been discussed was gnocchi. Besides a change to tolerations that was applied to most of the charts in the repo, the gnocchi chart has not seen much activity and there wasn't any immediate interest in maintaining it, so the current plan is to remove it from the repo. If you are someone who is actively using the gnocchi chart and would be interested in helping to maintain it, please let me know!

1 0

[all][qa] Dropping centos-8-stream support and testing
by Ghanshyam Mann 27 Apr '22

27 Apr '22

Hello Everyone, As you know, in zed cycle we are targeting centos-9-stream in testing runtime[1] and dropping the py3.6 support. By dropping the py3.6 support means stop testing the py3.6 and require "python_requires = >=3.8" , For example nova - https://github.com/openstack/nova/blob/5f5551448dcfcde26095963e223f973b90e6… With that centos-8-stream jobs are failing 100% and in qa meeting we decided to drop the support in devstack[2]. Dropping support in devstack means: 1. we will stop testing it[3] 2. Drop support from stack.sh. By disabling centos-8-stream from the supported_distro by default[4] in stack.sh (you can enable it by running with force flag). 1st is done and patch to remove the centos-8-stream job is merged (along with in Tempest and nova). For 2nd one we will wait for 1 more week and will discuss to do in next QA meeting. Please start removing or replacing the c8s jobs with c9s in your projects. You can keep gerrit topic 'drop-c8s-testing' to know the overall status. [1] https://governance.openstack.org/tc/reference/runtimes/zed.html [2] https://meetings.opendev.org/meetings/qa/2022/qa.2022-04-26-15.00.log.html#… [3] https://review.opendev.org/q/topic:drop-c8s-testing [4] https://github.com/openstack/devstack/blob/48417ca241cacff8f4398910792489a5… -gmann

3 4

[cinder] Bug deputy report for week of 04-26-2022
by Sofia Enriquez 27 Apr '22

27 Apr '22

This is a bug report from 04-20-2022 to 04-26-2022. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Medium - https://bugs.launchpad.net/os-brick/+bug/1969794 "backport of the fix for bug #1947370 make lock_path a required config option when prvisouls it was optional." Unassigned. - https://bugs.launchpad.net/cinder/+bug/1969373 "Simultaneous volume creation with the same image in multi-attach mode returns error." Assigned to Rajat. - https://bugs.launchpad.net/cinder/+bug/1969531 "NetApp NFS Storage Migration between backends is Failing." Unassigned. - https://bugs.launchpad.net/cinder/+bug/1969643 "RBD: Unable to delete a volume which has snapshot/volume children." Assigned to Sofia Enriquez and Eric Harney. No fix proposed to master yet. - https://bugs.launchpad.net/cinder/+bug/1969784 "[Pure Storage] Replicated array communication failure not handled correctly." Assigned to Simon Dodsley. No fix proposed to master yet. Low - https://bugs.launchpad.net/cinder/+bug/1967683 "Wrong property to look up remote address." Fix proposed to master. - https://bugs.launchpad.net/cinder/+bug/1970237 "[RBD] Multiple full backups cannot be created from one snapshot." Assigned to liuhuajie. No patch proposed to master yet. - https://bugs.launchpad.net/cinder/+bug/1969913 "[Documentation] Migration in cinder, lvm-rbd example is not correct." Unassigned. - https://bugs.launchpad.net/cinder/+bug/1966904 "Complex config formula evaluation causes RecursionError." Fix proposed to master. Wishlist - https://bugs.launchpad.net/cinder/+bug/1970208 "Rebranding. Dell EMC must be renamed to Dell." Assigned to Alexander Malashenko. Fix proposed to master. - https://bugs.launchpad.net/cinder/+bug/1970115 " [Enhancement] No way to disable image conversion." Assigned to Rico Lin. No fix proposed to master yet. - https://bugs.launchpad.net/cinder/+bug/1970114 " [Enhancement] Image conversion with RBD is not efficient." Unassigned. No fix proposed to master yet. Cheers, -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

RDO Yoga Released
by Amy Marrich 27 Apr '22

27 Apr '22

The RDO community is pleased to announce the general availability of the RDO build for OpenStack Yoga for RPM-based distributions, CentOS Stream and Red Hat Enterprise Linux. RDO is suitable for building private, public, and hybrid clouds. Yoga is the 25th release from the OpenStack project, which is the work of more than 1,000 contributors from around the world. The release is already available on the CentOS mirror network: - For CentOS Stream 8 http://mirror.centos.org/centos/8-stream/cloud/x86_64/openstack-yoga - For CentOS Stream 9 http://mirror.stream.centos.org/SIGs/9-stream/cloud/x86_64/openstack-yoga/ The RDO community project curates, packages, builds, tests and maintains a complete OpenStack component set for RHEL and CentOS Stream and is a member of the CentOS Cloud Infrastructure SIG. The Cloud Infrastructure SIG focuses on delivering a great user experience for CentOS users looking to build and maintain their own on-premise, public or hybrid clouds. All work on RDO and on the downstream release, Red Hat OpenStack Platform, is 100% open source, with all code changes going upstream first. Interesting things in the Yoga release include: - RDO Yoga is the first RDO version built and tested for CentOS Stream 9. - In order to ease transition from CentOS Stream 8, RDO Yoga is also built and tested for CentOS Stream 8. Note that next release of RDO will be available only for CentOS Stream 9. The highlights of the broader upstream OpenStack project may be read via https://releases.openstack.org/yoga/highlights.html TripleO in the RDO Yoga release: Since the Xena development cycle, TripleO follows the Independent release model <https://specs.openstack.org/openstack/tripleo-specs/specs/xena/tripleo-inde…> and will only maintain branches for selected OpenStack releases. In the case of Yoga, TripleO will not support the Yoga release. For TripleO users in RDO, this means that: - RDO Yoga will include packages for TripleO tested at OpenStack Yoga GA time. - Those packages will not be updated during the entire Yoga maintenance cycle. - RDO will not be able to included patches required to fix bugs in TripleO on RDO Yoga. - The lifecycle for the non-TripleO packages will follow the code merged and tested in upstream stable/yoga branches. - There will not be any TripleO Yoga container images built/pushed, so interested users will have to do their own container builds when deploying Yoga. You can find details about this on the RDO Webpage <https://www.rdoproject.org/documentation/tripleo-in-xena/> *Contributors* During the Yoga cycle, we saw the following new RDO contributors: - Adriano Vieira Petrich - Andrea Bolognani - Dariusz Smigiel - David Vallee Delisle - Douglas Viroel - Jakob Meng - Lucas Alvares Gomes - Luis Tomas Bolivar - T. Nichole Williams - Karolina Kula Welcome to all of you and Thank You So Much for participating! But we wouldn’t want to overlook anyone. A super massive Thank You to all 40 contributors who participated in producing this release. This list includes commits to rdo-packages, rdo-infra, and redhat-website repositories: - Adriano Vieira Petrich - Alan Bishop - Alan Pevec - Alex Schultz - Alfredo Moralejo - Amy Marrich (spotz) - Andrea Bolognani - Chandan Kumar - Daniel Alvarez Sanchez - Dariusz Smigiel - David Vallee Delisle - Douglas Viroel - Emma Foley - Gaël Chamoulaud - Gregory Thiemonge - Harald - Jakob Meng - James Slagle - Jiri Podivin - Joel Capitao - Jon Schlueter - Julia Kreger - Kashyap Chamarthy - Lee Yarwood - Lon Hohberger - Lucas Alvares Gomes - Luigi Toscano - Luis Tomas Bolivar - Martin Kopec - mathieu bultel - Matthias Runge - Riccardo Pittau - Sergey - Stephen Finucane - Steve Baker - Takashi Kajinami - T. Nichole Williams - Tobias Urdin - Karolina Kula - User otherwiseguy - Yatin Karel *The Next Release Cycle* At the end of one release, focus shifts immediately to the next release i.e Zed. *Get Started* To spin up a proof of concept cloud, quickly, and on limited hardware, try an All-In-One Packstack installation. You can run RDO on a single node to get a feel for how it works. Finally, for those that don’t have any hardware or physical resources, there’s the OpenStack Global Passport Program. This is a collaborative effort between OpenStack public cloud providers to let you experience the freedom, performance and interoperability of open source infrastructure. You can quickly and easily gain access to OpenStack infrastructure via trial programs from participating OpenStack public cloud providers around the world. *Get Help* The RDO Project has the users(a)lists.rdoproject.org for RDO-specific users and operators. For more developer-oriented content we recommend joining the dev(a)lists.rdoproject.org mailing list. Remember to post a brief introduction about yourself and your RDO story. The mailing lists archives are all available at https://mail.rdoproject.org. You can also find extensive documentation on RDOproject.org. The #rdo channel on OFTC. IRC is also an excellent place to find and give help. We also welcome comments and requests on the CentOS devel mailing list and the CentOS and TripleO IRC channels (#centos, #centos-devel in Libera Chat network, and #tripleo on OFTC), however we have a more focused audience within the RDO venues. *Get Involved* To get involved in the OpenStack RPM packaging effort, check out the RDO contribute pages, peruse the CentOS Cloud SIG page, and inhale the RDO packaging documentation. Join us in #rdo and #tripleo on the OFTC IRC network and follow us on Twitter @RDOCommunity. You can also find us on Facebook and YouTube.

1 0

[magnum] Proposing Michal Nasiadka for core-reviewer
by Spyros Trigazis 27 Apr '22

27 Apr '22

Dear all, I would like to nominate Michal Nasiadka for core reviewer in the magnum project. Michal has been helping with reviews for the past cycles and proposed features and fixes regularly. He would be a great addition to the team. Cheers, Spyros

1 0

[cinder] This week's meeting (Today) will be in video+IRC
by Rajat Dhasmana 27 Apr '22

27 Apr '22

Hello Argonauts, This is a reminder email about this week's meeting i.e. Today (27 April) will be in video + IRC mode. The timings are same as regular cinder meeting i.e. 1400-1500 UTC. The meeting will be held in video[1] and IRC[2]. The reason to keep this in both ways is some people are more comfortable in written communication than verbal and vice versa so we can hold discussions in video or IRC as per author's wish. Also the roll call will happen on IRC and a summary will be given there as well so make sure you're connected to the IRC channel during the meeting. [1] connection info: https://bluejeans.com/556681290 [2] openstack-meeting-alt channel on IRC Thanks and regards Rajat Dhasmana

1 0

[openstack-ansible] Nominate Damian Dąbrowski for openstack-ansible core team
by Dmitriy Rabotyagov 27 Apr '22

27 Apr '22

Hi OSA Cores! I'm happy to nominate Damian Dąbrowski (damiandabrowski) to the core reviewers team. He has been doing a good job lately in reviewing incoming patches, helping out in IRC and participating in community activities, so I think he will be a good match for the Core Reviewers group. So I call for current Core Reviewers to support this nomination or raise objections to it until 22nd of April 2022. If no objections are raised we will add Damian to the team next week. -- Kind regards, Dmitriy Rabotyagov

5 5

[tc][all][ Zed Virtual PTG RBAC discussions Summary
by Ghanshyam Mann 26 Apr '22

26 Apr '22

Hello Everyone, I tried to attend the RBAC-related sessions on various projects[i] but I am sure I might have missed a few of them. I am summarizing the RBAC discussion on what open questions were from the project side and what we discussed in TC PTG. Feel free to append the discussion you had in your project or any query you want TC to solve. Current status: ------------------ * I have started this etherpad[ii] to track the status of this goal, please keep it up to date as you progress the work in your project. Open question: ------------------ 1. heat create_stack API calling the mixed scope APIs (for example create flavor and create server). what is best scope for heat API so that we do not have any security leak. We have not concluded the solution yet as we need the heat team also join the discussion and agree on that. But we have a few possible solutions listed below: ** Heat accepts stack API with system scope *** This means a stack with system resources would require a system admin role => Need to check with services relying on Heat ** Heat assigns a project-scope role to the requester during a processing stack operation and uses this project scope credential to manage project resources ** Heat starts accepting the new header accepting the extra token (say SYSTEM_TOKEN) and uses that to create/interact the system-level resource like create flavor. 2. How to isolate the host level attribute in GET APIs? (cinder and manila have the same issue). Cinder GET volume API response has the host information. One possible solution we discussed is to have a separate API to show the host information to the system user and the rest of the volume response to the project users only. This is similar to what we have in nova. Then we have a few questions from the Tacker side, where tacker create_vnf API internally call heat create_stack and they are planning to make create_vnf API for non-admin users. Direction on enabling the enforce scope by default ------------------------------------------------------------ As keystone, nova, and neutron are ready with the new RBAC, we wanted to enable the scope checks by default. But after seeing the lack of integration testing and the above mentioned open question (especially heat and any deployment project breaking) we decided to hold it. As the first step, we will migrate the tempest tests to the new RBAC and will enable the scope for these services in devstack. And based on the testing results we will decide on it. But after seeing the amount of work needed in Tempest and on the open question, I do not think we will be able to do it in the Zed cycle. Instead, we will target to enable the 'new defaults' by default. We ran out of time in TC and will continue the discuss these in policy popup meetings. I will push the schedule to Ml. [i] https://etherpad.opendev.org/p/rbac-zed-ptg [ii] https://etherpad.opendev.org/p/rbac-goal-tracking -gmann

2 2

[Triple0 on Centos Stream] Wallaby Overcloud Deployment Issue
by Lokendra Rathour 26 Apr '22

26 Apr '22

Hi Team, We were trying to install Openstack Wallaby Release on Centos Stream. As we were trying the overcloud deployment, getting this error: 2022-04-26 17:20:51.768 374850 INFO tripleoclient.heat_launcher [-] Skipping container image pull. 2022-04-26 17:20:51.776 374850 INFO tripleoclient.heat_launcher [-] Checking that database is up 2022-04-26 17:20:52.251 374850 INFO tripleoclient.heat_launcher [-] Checking that message bus (rabbitmq) is up Error: Source image rejected: Error reading signature from https://access.redhat.com/webassets/docker/content/sigstore/ubi8/pause@sha2…: status 403 (Forbidden) 2022-04-26 17:20:57.687 374850 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Exception occured while running the command: subprocess.CalledProcessError: Command '['sudo', 'podman', 'play', 'kube', '/home/stack/overcloud-deploy/overcloud/heat-launcher/heat-pod.yaml']' returned non-zero exit status 125. 2022-04-26 17:20:57.687 374850 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent call last): 2022-04-26 17:20:57.687 374850 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run Please suggest. -- ~ Lokendra

2 2

[Kolla-ansible][Xena] Error deploying Cloudkitty
by wodel youchi 26 Apr '22

26 Apr '22

Hi, I am trying to deploy Cloudkitty, but I get this error message : TASK [cloudkitty : Creating Cloudkitty influxdb database] > ****************************************************** > task path: > /home/deployer/kollavenv/xenavenv/share/kolla-ansible/ansible/roles/cloudkitty/tasks/bootstrap.yml:36 fatal: [192.168.1.5 -> 192.168.1.5]: FAILED! => { > "action": "influxdb_database", > "changed": false, > "invocation": { > "module_args": { > "database_name": "cloudkitty", > "hostname": "dashint.cloud.cerist.dz", > "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", > "path": "", > "port": 8086, > "proxies": {}, > "retries": 3, > "ssl": false, > "state": "present", > "timeout": null, > "udp_port": 4444, > "use_udp": false, > "username": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", > "validate_certs": true > } > }, > "msg": "('Connection aborted.', RemoteDisconnected('Remote end closed > connection without response',))" > } On the influxdb container I did this : > [root@controllerb ~]# docker ps | grep inf > 68b3ebfefbec > 192.168.1.16:4000/openstack.kolla/centos-source-influxdb:xena > "dumb-init --single-…" 22 minutes ago Up 22 minutes > influxdb > [root@controllerb ~]# docker exec -it influxdb /bin/bash > (influxdb)[influxdb@controllerb /]$ influx > Failed to connect to http://localhost:8086: Get http://localhost:8086/ping: > dial tcp [::1]:8086: connect: connection refused > Please check your connection settings and ensure 'influxd' is running. > (influxdb)[influxdb@controllerb /]$ ps -ef > UID PID PPID C STIME TTY TIME CMD > influxdb 1 0 0 Apr18 ? 00:00:00 dumb-init > --single-child -- kolla_start > influxdb 7 1 0 Apr18 ? 00:00:01 /usr/bin/influxd > -config /etc/influxdb/influxdb.conf > influxdb 45 0 0 00:12 pts/0 00:00:00 /bin/bash > influxdb 78 45 0 00:12 pts/0 00:00:00 ps -ef > (influxdb)[influxdb@controllerb /]$ I have no log file for influxdb, the directory is empty. Any ideas? Regards.

4 7

Setup infrastructure failing
by Derek O keeffe 26 Apr '22

26 Apr '22

Hi all, We get the following error in the openstack.osa.db_setup : Create database for service task: failed: [infra1_keystone_container-e54c8ba5 -> infra1_utility_container-1a1eb7ce(xx.xx.xx.xxx)] (item={'name': 'keystone', 'users': [{'username': 'keystone', 'password': 'PASSWORD'}]}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "keystone", "users": [{"password": "PASSWORD", "username": "keystone"}]}, "msg": "unable to connect to database, check login_user and login_password are correct or /root/.my.cnf has the credentials. Exception message: (2003, \"Can't connect to MySQL server on 'xx.xx.xx.xxx' ([Errno 111] Connection refused)\")"} Also, ansible galera_container -m shell -a "mysql -h localhost -e 'show status like \"%wsrep_cluster_%\";'" is different to the example output: infra1_galera_container-90dd1571 | CHANGED | rc=0 >>Variable_name Valuewsrep_cluster_weight 1wsrep_cluster_capabilities wsrep_cluster_conf_id 1wsrep_cluster_size 1wsrep_cluster_state_uuid c06b8107-c53a-11ec-a54c-a20664362f6cwsrep_cluster_status Primary We checked our infra node and all the containers are present and mysql is installed on the infra1-galera-container and port 3306 is open. MariaDB [(none)]> show databases;+--------------------+| Database |+--------------------+| information_schema || mysql || performance_schema || sys |+--------------------+4 rows in set (0.001 sec) We are not sure what is causing this and it could possibly be that we have forgotten/skipped/missed a configuration step but we have gone over the process multiple times and cannot see what we may have missed. The previous playbooks did have a lot of "skipped" results compute1 : ok=120 changed=2 unreachable=0 failed=0 skipped=31 rescued=0 ignored=0 compute2 : ok=120 changed=2 unreachable=0 failed=0 skipped=31 rescued=0 ignored=0 infra1 : ok=156 changed=3 unreachable=0 failed=0 skipped=31 rescued=0 ignored=0 infra1_aodh_container-7c703324 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_ceilometer_central_container-7590db72 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_cinder_api_container-91806531 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_galera_container-90dd1571 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_glance_container-fff39ac4 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_gnocchi_container-ffd32a8b : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_heat_api_container-a4079838 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_horizon_container-9c041ec0 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_keystone_container-e54c8ba5 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_memcached_container-dc50bcd8 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_neutron_server_container-bfc4d0d2 : ok=92 changed=37 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_nova_api_container-f231bb13 : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_placement_container-108d79df : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_rabbit_mq_container-50ca98ed : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_repo_container-ccade01d : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 infra1_utility_container-1a1eb7ce : ok=89 changed=36 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0 localhost : ok=19 changed=0 unreachable=0 failed=0 skipped=14 rescued=0 ignored=0 compute1 : ok=0 changed=0 unreachable=0 failed=0 skipped=10 rescued=0 ignored=0 compute2 : ok=0 changed=0 unreachable=0 failed=0 skipped=10 rescued=0 ignored=0 infra1 : ok=38 changed=0 unreachable=0 failed=0 skipped=35 rescued=0 ignored=0 infra1_galera_container-90dd1571 : ok=69 changed=3 unreachable=0 failed=0 skipped=12 rescued=0 ignored=0 infra1_memcached_container-dc50bcd8 : ok=16 changed=0 unreachable=0 failed=0 skipped=4 rescued=0 ignored=0 infra1_rabbit_mq_container-50ca98ed : ok=69 changed=33 unreachable=0 failed=0 skipped=14 rescued=0 ignored=0 infra1_utility_container-1a1eb7ce : ok=40 changed=18 unreachable=0 failed=0 skipped=11 rescued=0 ignored=0 A lot were around Galera tasks but we are not sure if they were skipped intentionally as the playbooks ran to completion.Any help or troubleshooting steps would be appreciated. Thanks in advance. Regards,Derek

2 1

[skyline] skyline-console whl package question
by Satish Patel 26 Apr '22

26 Apr '22

Folks, I am compiling skyline-apiserver on my bare metal server and found Makefile pulling skyline-console tarball from the following location but that package doesn't contain the latest merged patches. Does CI job is broken which compiles this tarball ? https://tarballs.opendev.org/openstack/skyline-console/

2 2

[oslo][ops] Change in default logging format
by Stephen Finucane 26 Apr '22

26 Apr '22

I'm not sure how many people read release notes for the various oslo libraries, so I'm posting this here since it might have a minor impact for some tooling operators are using. We've just merged [1] which changes the default logging format used by oslo.log, configured using the '[DEFAULT] logging_context_format_string' config option. Previously, our logging format was: %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s This looks like e.g. 2022-04-26 10:49:14.629 4187413 WARNING nova.conductor.api [req-ffddccd3-8804-4fc3-a2f3-a05af1eb569f - - - - -] This has now been modified to include the global request ID as well as the local request ID: %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(user_identity)s] %(instance)s%(message)s This looks like e.g.: 2022-04-26 10:53:16.756 4187784 DEBUG nova.service [None req-f1698728-a2d6-4e02-a6b4-cf2cd398c02f - - - - -] Join ServiceGroup membership for this service compute This will affect pretty much all services using the out-of-the-box logging configuration. However, I suspect very few if any users are using the standard configuration. DevStack overrides '[DEFAULT] logging_context_format_string' for all services [2] and I suspect other installers probably do the same. If you are using the standard configuration, you might need to modify some log parsing/scraping tooling when this rolls around or manually set '[DEFAULT] logging_context_format_string' in your service configuration files. Cheers, Stephen [1] https://review.opendev.org/c/openstack/oslo.log/+/838190 [2] https://github.com/openstack/devstack/blob/3b0c035b9/functions#L677-L707

1 0

[all][neutron][neutron-vpnaas] Maintainers needed
by Lajos Katona 26 Apr '22

26 Apr '22

Hi, In the last few cycles neutron-vpnaas has no serious maintainers, and most patches merged were from Neutron core team or from Release team. Recently even neutron-vpnaas gate jobs started to fail. During the Zed PTG we discussed this topic (see [1]), For the maintenance we need someone to be the contact person for the project, who takes care of the project’s CI and review patches, answers bugs. Of course that’s only a minimal requirement. If the new maintainer works on new features for the project, it’s even better :) If we don’t have any new maintainer(s) before milestone Zed-2, which is July 11 - July 15 week according to [2], we will start marking neutron-vpnaas as deprecated and in the next cycle (AA, or perhapc 2023.1) we will propose to retire the project. So if You are using this project now, or if You have customers who are using it, please consider the possibility of maintaining it. Otherwise, please be aware that it is highly possible that the project will be deprecated and moved out from the official OpenStack projects. [1]: https://etherpad.opendev.org/p/neutron-zed-ptg#L201 [2]: https://releases.openstack.org/zed/schedule.html Lajos Katona (lajoskatona)

2 2

[cinder] how cinder gets the volume name when restore from a backup?
by Tony Liu 26 Apr '22

26 Apr '22

Hi, Here is what I do. * Create a volume. * Create a backup of this volume. * Delete the volume. * Restore the volume from backup. The restored volume has different UUID, but the same name. I don't see volume name is stored in backup metadata. How does Cinder know the volume name when restore it from backup, given the volume is already deleted? Thanks! Tony

2 2

retention policy for deleted resource in database
by Tony Liu 26 Apr '22

26 Apr '22

Hi, I see the record for deleted resource stay in database. What's the retention policy for those records? Is it configurable? Any manual cleanup is required? Thanks! Tony

2 2

[neutron] Bug Deputy Report Apr 18 - Apr 24
by Oleg Bondarev 25 Apr '22

25 Apr '22

Hello Team, Bug report for the week of Apr 18 is below. Two OVN bugs (including Invalid one) need triage from the OVN team. High ------ - https://bugs.launchpad.net/neutron/+bug/1969615 - OVS: flow loop is created with openvswitch version 2.16 - Opinion - seems more an issue in OVS rather than a bug in Neutron - Unassigned Undecided -------------- - https://bugs.launchpad.net/neutron/+bug/1969592 - [OVN] Frequent DB leader changes causes 'VIF creation failed' on nova side - New - Unassigned Invalid --------- - https://bugs.launchpad.net/neutron/+bug/1969354 - ovn-controller don't update new flows - Invalid - but reporter updated with additional info - Unassigned Thanks, Oleg

1 0

[ironic][ptg] Zed PTG Summary
by Iury Gregory 25 Apr '22

25 Apr '22

Hello Ironicers! Sorry for the delay to provide a summary about our PTG =) First of all, thank you to all contributors that took some time to join our sessions! During the sessions we had a peak of 13 attendees on the last three days. Day1 We only had two topics: Feedback about the Yoga Cycle and What should we do about specs. During the of we did a retrospective about the Yoga Cycle, we discussed the good and bad things that happened during the cycle and how we can make things better. In the discussion about what we should do about specs we come up with some ideas to improve our process when we think a spec should be required. Day2: On this day most of the discussions were around topics that can be managing machines. How we can decrease the cost of Data Centers that needs to keep all their machine on even if they are not active or taking minimal load, we would provide a way to power tuning nodes that are already deployed (in the ironic perspective this is a way to reconfigure a node that is already active), we also talked about turning down power usage if we are able to identify that IPA is idle. The custom deploy timeout topic had no strong objections, since we have a single configuration that handles the timeout for all steps we think we can improve this, we still need to define some details related to the implementation after we have the RFE. We discussed how we can generate the inspector.ipxe configuration if we notice some changes in the configuration of the deployment. Day3: We started this day looking at the survey results, this gave us some ideas on possible areas we should improve. The ironic safeguard topic was focusing on two main things: queuing (limit the max number of concurrent cleaning operations) and data disk protection (only clean the root disk (or give a list of skip-disks/disks-to-clean). The community decided that this seems like a good idea and we defined some of the possible paths forwards related to how it could be implemented. The redfish gateway and related ideas topic brought an interesting idea to have an ironic driver that can execute a "script" that is required for their HW to start working properly, there is a recording from the first meeting that can provide more details =). The per-node clean steps topic had no objections, we think this will improve the operators experience for scenarios where they would need specific steps for running on a node that has different disk configuration. Day4: Most of the topics on this day were focused on networking. We discussed the status of OVN DHCP support and how we will move forward on our side to make things work after Neutron and OVN have all the necessary bits in place. During the next topic we talked about adding device configuration capabilities for networking-baremetal since in multi-tenant BMaaS there is a need to configure the ToR network devices (Access/Edge Switches) and many vendors have abandoned their ML2 mechanism plug-ins to support this, so now we are looking to add support for new mechanisms that have more features that could improve the operators experience. We focused on discussing the pros/cons about Netconf and Yang but also other alternatives solutions. The other topic we discussed were: - netboot deprecation: we discussed how we should move forward with some of our testing for partition images + UEFI. - Bluefield DPU: not much discussion since we didn't have many folks interested on the topic. - Anaconda driver: we talked about how we can get a CI for testing the driver. You can find more information about the topics and the discussions in our etherpad: https://etherpad.opendev.org/p/ironic-zed-ptg -- *Att[]'sIury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Part of the ironic-core and puppet-manager-core team in OpenStack* *Software Engineer at Red Hat Czech* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory(a)gmail.com <iurygregory(a)gmail.com>*

1 0

[largescale-sig] Skipping meeting this week
by Thierry Carrez 25 Apr '22

25 Apr '22

Hi everyone, Due to a shortage of participants and meeting chairs, we will skip the meeting planned this week for the Large Scale SIG. Our next meeting will be May 11th at 15UTC in #openstack-operators. You can add topics to discuss to our agenda etherpad: https://etherpad.opendev.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez (ttx)

1 0