January 2024 - openstack-discuss - lists.openstack.org

[ops] help with multiattach volume type in cinder
by Shubham Kumar Yadav 22 Jan '24

22 Jan '24

Hi, i have a question/issue related to multiattach feature in cinder i have an openstack cluster and i have configured multiattach in the openstack cinder now when i create 1 volume (with volume type multiattach) and attach it to 2 instances test1 and test2 *on test1 * *sudo mkfs -t ext4 /dev/sdb * *sudo mkdir /mnt/mydisk * *sudo mount /dev/sdb /mnt/mydisk * *touch /mnt/mydisk/shared_file echo "This is some shared content" | sudo tee /mnt/mydisk/shared_file* *on test2 * *sudo mkdir /mnt/mydisksudo mount /dev/sdb /mnt/mydiskls /mnt/mydisk/cat /mnt/mydisk/shared_file* i can see the shared_file now when i add new files on test1 im not able to see it on test2 untill i unmount the volume with directory and mount the directory again to the volume on test2. what could be the solution for it? how can i make the volume share datas instantly between all the instances connected to the volume i have tried with ext4 and xfs file system Hope to hear from you soon Thank you!

3 2

[neutron] Bug deputy report for week of January 15th
by Brian Haley 22 Jan '24

22 Jan '24

Hi, I was Neutron bug deputy last week. Below is a short summary about the reported bugs. -Brian Critical bugs ------------- * None High bugs --------- * https://bugs.launchpad.net/neutron/+bug/2049211 - neutron_tempest_plugin.api sorted() Function Issue - https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/905853 Medium bugs ----------- * https://bugs.launchpad.net/neutron/+bug/2049590 - Restore the device namespace if "set_netns" fails - https://review.opendev.org/c/openstack/neutron/+/905836 * https://bugs.launchpad.net/neutron/+bug/2049615 - multisegments: cleaning DHCP process for segment 0 should happen first - https://review.opendev.org/c/openstack/neutron/+/905617 * https://bugs.launchpad.net/neutron/+bug/2049524 - [Neutron/Nova] Need to fix attaching a vip port on VMs - confirmed, needs owner * https://bugs.launchpad.net/neutron/+bug/2049624 - vpnaas problem --- strongswan on rocky 9 - Needs owner * https://bugs.launchpad.net/neutron/+bug/2049909 - MLDv2 packets sent from L3 Agent managed networks cause backup routers to be preferred - Issue was filed regarding Linuxbridge, not sure if it is also happening with ML2/OVS - https://review.opendev.org/c/openstack/neutron/+/906114 - Will need more investigation Low bugs -------- * None Misc bugs --------- * https://bugs.launchpad.net/neutron/+bug/2049546 - neutron-linuxbridge-agent ebtables RULE_DELETE failed (Invalid argument) - Duplicate of https://bugs.launchpad.net/neutron/+bug/2038541 * https://bugs.launchpad.net/neutron/+bug/2049622 - Some Port Properties Cannot Be Set - dns-name tests were failing in their environment, asked for more info on configuration as could be user error Wishlist bugs ------------- * https://bugs.launchpad.net/neutron/+bug/2049623 - [RFE] Refactor OVS Trunk plugin to have one single port

1 0

[puppet][ec2-api] Retiring puppet-ec2api
by Takashi Kajinami 22 Jan '24

22 Jan '24

Hello, As announced earlier, the EC2API project has been marked inactive[1]. We'll carefully watch the discussion between TC and the project, but unless the inactive mark is removed and the project is re-added to 2024.1 releases, I'd propose retiring puppet-ec2api, based on inactivity in the EC2 API project as well as no actual user feedback to our module. In case you have any concern, please let me know. If I get no objections and the project status is not changed, I'll start the retirement process of this module at some point during this cycle (probably m-3 or in worst case no later than GA). Thank you Takashi Kajinami irc: tkajinam [1] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack…

2 1

Security Group
by benyamin sam khanyani 22 Jan '24

22 Jan '24

Hi, I am working on the security group and aim to extend its functionality by applying custom rules based on source and destination ports, arbitrary match criteria, actions, and so on. I did it by using ovn-nbctl commands and everything worked well. Unfortunately, the only issue is that I can not see my custom rules in MariaDB and I didn't find any other solution for defining security group rules with my desired options. So, I want to ask you how I can tackle this issue? Regards, Benjamin Sam

2 1

[publiccloud-sig] Reminder - next meeting January 24th - 0800 UTC
by Felix Kronlage-Dammers 22 Jan '24

22 Jan '24

Hi everyone, This Wednesday (January 24th) the next meeting of the public cloud sig is going to happen. Since the majority of participants in the sessions started to like the videocalls, we’ll be doing this as a videocall again. We’ll be meeting at 0800 UTC here: https://conf.scs.koeln:8443/OIF-public-cloud-sig See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG See you Wednesday! felix -- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[kolla-ansible] [cinder-backup] does support HA?
by garcetto 22 Jan '24

22 Jan '24

good afternoon, i was trying cinder-backup, deployed on 2 containers. the first one seems to do all work, the second is sitting idle. I have a couple of questions: 1. can i parallelize backup jobs so are ran on both containers? 2. in case of first container down, the second seems not to be able to do backups/restores, nay clue? thank you.

3 3

[murano][heat][mistral][tc] Maintenance of yaql library
by Takashi Kajinami 21 Jan '24

21 Jan '24

Hello, The yaql library is currently maintained under murano's governance and is used by not only murano but a few other projects such as heat and mistral. In heat the library is essential since we introduced yaql function support. Looking at gerrit groups, I noticed the yaql library is maintained by a very small core group[1] and release group[2], but I'm afraid most of the people in these groups are no longer active in the community. Because heat is dependent on the library, I'm wondering if some of the heat cores can be added to these two groups so that we (at least I) can help maintaining the library. I've checked the changes merged into murano repo for these two years but iiuc all of these changes are gate fix. Also a dummy change[3] indicates that its CI is currently broken. If the murano project is no longer active then I'm also willing to take over yaql and bring it to heat's governance (of cause only if it's acceptable). I'll add the current PTL as cc, but please let me know if anyone has any opinion or concern. # I'll add mistral tag because the project is also dependent on yaql. # Also tc is added because this discussion is related to project governance. Thank you Takashi Kajinami [1] https://review.opendev.org/admin/groups/cba4f6fd54c3cc0d5706fc50b5ebd758860… [2] https://review.opendev.org/admin/groups/d5e894b34e5ab5a9fc2938a0d4e7bc76fd2… [3] https://review.opendev.org/c/openstack/murano/+/901888

3 7

RDO at CentOS Connect and FOSDEM
by Amy Marrich 20 Jan '24

20 Jan '24

I just wanted to let folks know about some opportunities to learn more about RDO and interact with members of the community at the upcoming CentOS Connect[0] on January 31st and February 1st, then followed by FOSDEM[1] on February 2nd and 3rd in Brussels, BE. During Connect, there will be several RDO(Cloud) focused opportunities during the 2 days of Connect, including a morning CentOS Cloud SiG meetup, a presentation on running RDO on OKD, and a Cloud SiG update. At FOSDEM, there will be a shared stand(CentOS, RDO, and OKD) in bldg AW where you can chat with other members of the community and get some SWAG. Lastly, the OpenInfra Foundation will be holding a Happy Hour meetup[2] on Saturday night where you can meet members of the larger OpenStack and OpenInfra Communities. Hope to see you all there! And if you'd like to help out at the Stand please reach out as the signup is still in the works. Amy (spotz) 0 - https://connect.centos.org/ 1 - https://fosdem.org/2024/ 2 - https://www.meetup.com/brussels-openinfra-meetup-group/events/298420649

1 0

[all][ptl] proposed 2024.2 Dalmatian release cycle schedule
by Elõd Illés 19 Jan '24

19 Jan '24

Hi, As usually after Milestone-2, the release management team prepared the next release cycle schedule plan. Please (especially PTLs) have a look at the proposed 2024.2 Dalmatian release schedule [1] (<- this is the latest generated version of the schedule) and comment on the patch [2] if you have any thoughts. Thanks, Előd irc: elodilles @ #openstack-release [1]https://021d3a4791745da2ced8-a8caf4f4ad4b0a7f74dab21fc6a45bed.ssl.cf5.rac… [2] https://review.opendev.org/c/openstack/releases/+/906050

1 0

[neutron] Neutron drivers meeting
by Brian Haley 18 Jan '24

18 Jan '24

Hello Neutrinos, Please remember tomorrow (Friday) we have a Neutron drivers meeting. This is the agenda: https://wiki.openstack.org/wiki/Meetings/NeutronDrivers There are currently two topics: * (lajoskatona): quicky discuss the eventlet hell and what are the first steps: https://review.opendev.org/q/topic:bump-eventlet * (mtomaska): https://bugs.launchpad.net/neutron/+bug/2047849. [RFE] Start using oslo messaging namespaces https://review.opendev.org/c/openstack/neutron/+/905309 I realize there might be a couple of people out, if we do not have quorum will just postpone until next week. Thanks, -Brian

1 0

Problem with failing container-puppet-glance_api_internal during wallaby overcloud deploy.
by davorin.mikulic＠gmail.com 18 Jan '24

18 Jan '24

Hi, I have successfully deployed networks, vips, overcloud nodes and ceph cluster. Now I am proceeding with openstack overcloud deploy command as follows: openstack overcloud deploy --deployed-server --answers-file /home/stack/3XOenvironment/answers.yaml answers.yaml is as follows: (undercloud) [stack@uc4tb27 3XOenvironment]$ cat answers.yaml templates: /home/stack/generated-openstack-tripleo-heat-templates networks: /home/stack/3XOenvironment/40-network_data.yaml roles: /home/stack/3XOenvironment/30-roles_data.yaml environments: - /home/stack/3XOenvironment/deployed-server-environment.yaml - /home/stack/3XOenvironment/network-environment.yaml - /home/stack/3XOenvironment/overcloud-networks-deployed.yaml - /home/stack/3XOenvironment/overcloud-baremetal-deployed.yaml - /home/stack/3XOenvironment/deployed-vips.yaml - /home/stack/3XOenvironment/00-containers-prepare-parameter.yaml - /home/stack/3XOenvironment/10-docker-ha.yaml - /home/stack/3XOenvironment/cephadm.yaml - /home/stack/3XOenvironment/deployed-ceph.yaml We are not using a TLS setup anywhere in the overcloud or the undercloud. During the deploy, after the stack has been successfully created and deploy proceeds to puppet configuration of containers in step 1, I am getting the following error: 2024-01-17 09:30:39.360378 | 000c29b1-9f06-28d0-4abe-00000000d389 | TIMING | tripleo_container_manage : Create containers managed by Podman for /var/lib/tripleo-config/container-puppet-config/step_1 | overcloud-controller-1 | 0:18:04.150199 | 324.03s 2024-01-17 09:30:43.693815 | | WARNING | ERROR: Can't run container container-puppet-glance_api_internal stderr: + /usr/bin/puppet apply --summarize --detailed-exitcodes --color=false --modulepath=/etc/puppet/modules:/usr/share/openstack-puppet/modules --tags '"file,file_line,concat,augeas,cron,glance_api_config,glance_api_paste_ini,glance_swift_config,glance_cache_config,glance_image_import_config"' /etc/config.pp + logger -s -t puppet-user <13>Jan 17 08:25:20 puppet-user: Warning: /etc/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5 <13>Jan 17 08:25:20 puppet-user: (file: /etc/puppet/hiera.yaml) <13>Jan 17 08:25:20 puppet-user: Warning: Undefined variable '::deploy_config_name'; <13>Jan 17 08:25:20 puppet-user: (file & line not available) <13>Jan 17 08:25:20 puppet-user: Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/puppet/7.6/deprecated_language.html <13>Jan 17 08:25:20 puppet-user: (file & line not available) <13>Jan 17 08:25:20 puppet-user: Error: Evaluation Error: Error while evaluating a Resource Statement, Class[Tripleo::Profile::Base::Glance::Api]: <13>Jan 17 08:25:20 puppet-user: has no parameter named 'bind_port' <13>Jan 17 08:25:20 puppet-user: has no parameter named 'log_file' <13>Jan 17 08:25:20 puppet-user: has no parameter named 'show_image_direct_url' <13>Jan 17 08:25:20 puppet-user: has no parameter named 'show_multiple_locations' (file: /etc/config.pp, line: 2, column: 1) on node overcloud-controller-2.tb27.cld + rc=1 + '[' false = false ']' + set +x 2024-01-17 09:30:43.695467 | 000c29b1-9f06-28d0-4abe-00000000d4d9 | FATAL | Create containers managed by Podman for /var/lib/tripleo-config/container-puppet-config/step_1 | overcloud-controller-2 | error={"changed": false, "msg": "Failed containers: container-puppet-glance_api_internal"} 2024-01-17 09:30:43.696471 | 000c29b1-9f06-28d0-4abe-00000000d4d9 | TIMING | tripleo_container_manage : Create containers managed by Podman for /var/lib/tripleo-config/container-puppet-config/step_1 | overcloud-controller-2 | 0:18:08.486292 | 323.73s PLAY RECAP ********************************************************************* localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 overcloud-cephstorage-0 : ok=238 changed=96 unreachable=0 failed=0 skipped=135 rescued=0 ignored=1 overcloud-cephstorage-1 : ok=240 changed=96 unreachable=0 failed=0 skipped=130 rescued=0 ignored=1 overcloud-cephstorage-2 : ok=240 changed=96 unreachable=0 failed=0 skipped=130 rescued=0 ignored=1 overcloud-controller-0 : ok=295 changed=140 unreachable=0 failed=1 skipped=175 rescued=0 ignored=1 overcloud-controller-1 : ok=306 changed=140 unreachable=0 failed=1 skipped=164 rescued=0 ignored=1 overcloud-controller-2 : ok=306 changed=140 unreachable=0 failed=1 skipped=164 rescued=0 ignored=1 overcloud-novacompute-0 : ok=281 changed=132 unreachable=0 failed=0 skipped=165 rescued=0 ignored=1 overcloud-novacompute-1 : ok=288 changed=132 unreachable=0 failed=0 skipped=158 rescued=0 ignored=1 overcloud-novacompute-2 : ok=288 changed=132 unreachable=0 failed=0 skipped=158 rescued=0 ignored=1 undercloud : ok=25 changed=9 unreachable=0 failed=0 skipped=2 rescued=0 ignored=1 2024-01-17 09:30:43.764701 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-01-17 09:30:43.765549 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 1578 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-01-17 09:30:43.766211 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:18:08.556038 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-01-17 09:30:43.766837 | UUID | Info | Host | Task Name | Run Time 2024-01-17 09:30:43.767508 | 000c29b1-9f06-28d0-4abe-00000000d389 | SUMMARY | overcloud-controller-1 | tripleo_container_manage : Create containers managed by Podman for /var/lib/tripleo-config/container-puppet-config/step_1 | 324.03s 2024-01-17 09:30:43.768148 | 000c29b1-9f06-28d0-4abe-00000000d33c | SUMMARY | overcloud-controller-0 | tripleo_container_manage : Create containers managed by Podman for /var/lib/tripleo-config/container-puppet-config/step_1 | 323.73s 2024-01-17 09:30:43.768828 | 000c29b1-9f06-28d0-4abe-00000000d4d9 | SUMMARY | overcloud-controller-2 | tripleo_container_manage : Create containers managed by Podman for /var/lib/tripleo-config/container-puppet-config/step_1 | 323.73s 2024-01-17 09:30:43.769558 | 000c29b1-9f06-28d0-4abe-00000000b974 | SUMMARY | overcloud-novacompute-0 | Pre-fetch all the containers | 181.76s 2024-01-17 09:30:43.770215 | 000c29b1-9f06-28d0-4abe-00000000b9f7 | SUMMARY | overcloud-novacompute-1 | Pre-fetch all the containers | 175.41s 2024-01-17 09:30:43.770929 | 000c29b1-9f06-28d0-4abe-00000000ba70 | SUMMARY | overcloud-novacompute-2 | Pre-fetch all the containers | 169.84s 2024-01-17 09:30:43.771586 | 000c29b1-9f06-28d0-4abe-00000000c777 | SUMMARY | overcloud-controller-2 | Wait for puppet host configuration to finish | 133.41s 2024-01-17 09:30:43.772286 | 000c29b1-9f06-28d0-4abe-00000000c730 | SUMMARY | overcloud-controller-0 | Wait for puppet host configuration to finish | 133.31s 2024-01-17 09:30:43.773010 | 000c29b1-9f06-28d0-4abe-00000000c74d | SUMMARY | overcloud-controller-1 | Wait for puppet host configuration to finish | 133.30s 2024-01-17 09:30:43.773675 | 000c29b1-9f06-28d0-4abe-000000003b66 | SUMMARY | overcloud-controller-2 | Run puppet on the host to apply IPtables rules | 81.78s 2024-01-17 09:30:43.774343 | 000c29b1-9f06-28d0-4abe-000000003b66 | SUMMARY | overcloud-controller-0 | Run puppet on the host to apply IPtables rules | 80.99s 2024-01-17 09:30:43.774928 | 000c29b1-9f06-28d0-4abe-000000003b66 | SUMMARY | overcloud-controller-1 | Run puppet on the host to apply IPtables rules | 80.71s 2024-01-17 09:30:43.775545 | 000c29b1-9f06-28d0-4abe-00000000b7d8 | SUMMARY | overcloud-controller-0 | Pre-fetch all the containers | 80.65s 2024-01-17 09:30:43.776155 | 000c29b1-9f06-28d0-4abe-00000000b8fe | SUMMARY | overcloud-controller-2 | Pre-fetch all the containers | 80.21s 2024-01-17 09:30:43.776781 | 000c29b1-9f06-28d0-4abe-00000000b85e | SUMMARY | overcloud-controller-1 | Pre-fetch all the containers | 76.84s 2024-01-17 09:30:43.777443 | 000c29b1-9f06-28d0-4abe-00000000b58a | SUMMARY | undercloud | tripleo_container_image_prepare : Container image prepare | 66.95s 2024-01-17 09:30:43.778043 | 000c29b1-9f06-28d0-4abe-00000000b78c | SUMMARY | overcloud-controller-2 | tripleo_container_tag : Pull uc4tb27.ctlplane.tb27.cld:8787/tripleowallaby/openstack-cinder-volume:current-tripleo image | 42.81s 2024-01-17 09:30:43.778660 | 000c29b1-9f06-28d0-4abe-00000000c7ab | SUMMARY | overcloud-novacompute-2 | Wait for puppet host configuration to finish | 42.09s 2024-01-17 09:30:43.779285 | 000c29b1-9f06-28d0-4abe-00000000b78c | SUMMARY | overcloud-controller-1 | tripleo_container_tag : Pull uc4tb27.ctlplane.tb27.cld:8787/tripleowallaby/openstack-cinder-volume:current-tripleo image | 42.00s 2024-01-17 09:30:43.779870 | 000c29b1-9f06-28d0-4abe-00000000b78c | SUMMARY | overcloud-controller-0 | tripleo_container_tag : Pull uc4tb27.ctlplane.tb27.cld:8787/tripleowallaby/openstack-cinder-volume:current-tripleo image | 40.08s 2024-01-17 09:30:43.780470 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-01-17 09:30:43.781183 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ State Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-01-17 09:30:43.781794 | ~~~~~~~~~~~~~~~~~~ Number of nodes which did not deploy successfully: 3 ~~~~~~~~~~~~~~~~~ 2024-01-17 09:30:43.782388 | The following node(s) had failures: overcloud-controller-0, overcloud-controller-1, overcloud-controller-2 2024-01-17 09:30:43.783022 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-01-17 09:30:45.104 477248 INFO tripleoclient.utils.utils [-] Temporary directory [ /tmp/tripleo5qf1b220 ] cleaned up 2024-01-17 09:30:45.104 477248 ERROR tripleoclient.utils.utils [-] Ansible execution failed. playbook: /home/stack/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/stack/overcloud-deploy/overcloud/config-download/ansible-playbook-command.sh 2024-01-17 09:30:45.105 477248 WARNING tripleoclient.utils.safe_write [-] The output file /home/stack/overcloud-deploy/overcloud/overcloud-deployment_status.yaml will be overriden: RuntimeError: Ansible execution failed. playbook: /home/stack/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/stack/overcloud-deploy/overcloud/config-download/ansible-playbook-command.sh Host 10.100.27.127 not found in /home/stack/.ssh/known_hosts Overcloud Endpoint: http://10.100.27.127:5000 Overcloud Horizon Dashboard URL: http://10.100.27.127:80/dashboard Overcloud rc file: /home/stack/overcloud-deploy/overcloud/overcloudrc and /home/stack/overcloudrc Overcloud Deployed with error 2024-01-17 09:30:45.793 477248 INFO tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Stopping ephemeral heat. 2024-01-17 09:30:45.863 477248 INFO tripleoclient.heat_launcher [-] Pod does not exist: ephemeral-heat 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Exception occured while running the command: RuntimeError: Ansible execution failed. playbook: /home/stack/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/stack/overcloud-deploy/overcloud/config-download/ansible-playbook-command.sh 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent call last): 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud super(Command, self).run(parsed_args) 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in run 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return super(Command, self).run(parsed_args) 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return_code = self.take_action(parsed_args) or 0 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 1334, in take_action 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud working_dir=self.working_dir) 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud self.force_reraise() 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud raise self.value 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 1322, in take_action 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud 'BlacklistedHostnames', self.working_dir)) 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/deployment.py", line 405, in config_download 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud forks=forks 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 775, in run_ansible_playbook 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud raise RuntimeError(err_msg) 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud RuntimeError: Ansible execution failed. playbook: /home/stack/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/stack/overcloud-deploy/overcloud/config-download/ansible-playbook-command.sh 2024-01-17 09:30:47.952 477248 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud 2024-01-17 09:30:47.967 477248 ERROR openstack [-] Ansible execution failed. playbook: /home/stack/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/stack/overcloud-deploy/overcloud/config-download/ansible-playbook-command.sh: RuntimeError: Ansible execution failed. playbook: /home/stack/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2, To rerun the failed command manually execute the following script: /home/stack/overcloud-deploy/overcloud/config-download/ansible-playbook-command.sh 2024-01-17 09:30:47.971 477248 INFO osc_lib.shell [-] END return value: 1 If I list logs on the failing container and open the /etc/config.pp file I can see that bind_port, log_file, show_image_direct_url and show_multiple_locations are successfully read but it seems they are not defined for specified class. [root@overcloud-controller-0 ~]# podman logs f319ca15fd60 include ::tripleo::packages class { 'tripleo::profile::base::glance::api': bind_port => 9293, tls_proxy_port => 9293, log_file => '/var/log/glance/api_internal.log', show_image_direct_url => true, show_multiple_locations => true, } include tripleo::profile::base::database::mysql::client Running puppet + /usr/bin/puppet apply --summarize --detailed-exitcodes --color=false --modulepath=/etc/puppet/modules:/usr/share/openstack-puppet/modules --tags '"file,file_line,concat,augeas,cron,glance_api_config,glance_api_paste_ini,glance_swift_config,glance_cache_config,glance_image_import_config"' /etc/config.pp + logger -s -t puppet-user <13>Jan 17 08:25:15 puppet-user: Warning: /etc/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5 <13>Jan 17 08:25:15 puppet-user: (file: /etc/puppet/hiera.yaml) <13>Jan 17 08:25:15 puppet-user: Warning: Undefined variable '::deploy_config_name'; <13>Jan 17 08:25:15 puppet-user: (file & line not available) <13>Jan 17 08:25:15 puppet-user: Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/puppet/7.6/deprecated_language.html <13>Jan 17 08:25:15 puppet-user: (file & line not available) <13>Jan 17 08:25:15 puppet-user: Error: Evaluation Error: Error while evaluating a Resource Statement, Class[Tripleo::Profile::Base::Glance::Api]: <13>Jan 17 08:25:15 puppet-user: has no parameter named 'bind_port' <13>Jan 17 08:25:15 puppet-user: has no parameter named 'log_file' <13>Jan 17 08:25:15 puppet-user: has no parameter named 'show_image_direct_url' <13>Jan 17 08:25:15 puppet-user: has no parameter named 'show_multiple_locations' (file: /etc/config.pp, line: 2, column: 1) on node overcloud-controller-0.tb27.cld + rc=1 + '[' false = false ']' + set +x include ::tripleo::packages class { 'tripleo::profile::base::glance::api': bind_port => 9293, tls_proxy_port => 9293, log_file => '/var/log/glance/api_internal.log', show_image_direct_url => true, show_multiple_locations => true, } include tripleo::profile::base::database::mysql::client Running puppet + /usr/bin/puppet apply --summarize --detailed-exitcodes --color=false --modulepath=/etc/puppet/modules:/usr/share/openstack-puppet/modules --tags '"file,file_line,concat,augeas,cron,glance_api_config,glance_api_paste_ini,glance_swift_config,glance_cache_config,glance_image_import_config"' /etc/config.pp + logger -s -t puppet-user <13>Jan 17 09:19:13 puppet-user: Warning: /etc/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5 <13>Jan 17 09:19:13 puppet-user: (file: /etc/puppet/hiera.yaml) <13>Jan 17 09:19:13 puppet-user: Warning: Undefined variable '::deploy_config_name'; <13>Jan 17 09:19:13 puppet-user: (file & line not available) <13>Jan 17 09:19:13 puppet-user: Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/puppet/7.6/deprecated_language.html <13>Jan 17 09:19:13 puppet-user: (file & line not available) <13>Jan 17 09:19:13 puppet-user: Error: Evaluation Error: Error while evaluating a Resource Statement, Class[Tripleo::Profile::Base::Glance::Api]: <13>Jan 17 09:19:13 puppet-user: has no parameter named 'bind_port' <13>Jan 17 09:19:13 puppet-user: has no parameter named 'log_file' <13>Jan 17 09:19:13 puppet-user: has no parameter named 'show_image_direct_url' <13>Jan 17 09:19:13 puppet-user: has no parameter named 'show_multiple_locations' (file: /etc/config.pp, line: 2, column: 1) on node overcloud-controller-0.tb27.cld + rc=1 + '[' false = false ']' + set +x Also the content of config.pp is as follows: include ::tripleo::packages class { 'tripleo::profile::base::glance::api': bind_port => 9293, tls_proxy_port => 9293, log_file => '/var/log/glance/api_internal.log', show_image_direct_url => true, show_multiple_locations => true, } include tripleo::profile::base::database::mysql::client Does anyone know what is the issue here?

2 2

[puppet][sahara] Retiring puppet-sahara
by Takashi Kajinami 18 Jan '24

18 Jan '24

Hello, As you may know, the sahara project was marked inactive during this cycle. In Puppet OpenStack project we have maintained the puppet-sahara module to deploy and manage sahara deployments. We haven't seen any feedback from actual users for a while and I doubt anyone is actually using it. # AFAIK all of the recent updates are common maintenance works. I've updated # the module to improve parameter support coverage but I've never used sahara # by myself. Based on the situation, I propose retiring this module. In case you have any concern (though in that case the concern should be directed to the sahara project itself), please let me know. Unless I hear any objections or improvements about sahara's status, I'll start the retirement process after milestone 3 (Week of Feb 26). Thank you, Takashi Kajinami irc: tkajinam

2 1

help with multiattach volume type in [cinder]
by Shubham Kumar Yadav 18 Jan '24

18 Jan '24

Hi, i have a question/issue related to multiattach feature in cinder i have an openstack cluster and i have configured multiattach in the openstack cinder now when i create 1 volume (with volume type multiattach) and attach it to 2 instances test1 and test2 on test1 sudo mkfs -t ext4 /dev/sdb sudo mkdir /mnt/mydisk sudo mount /dev/sdb /mnt/mydisk touch /mnt/mydisk/shared_file echo "This is some shared content" | sudo tee /mnt/mydisk/shared_file on test2 sudo mkdir /mnt/mydisk sudo mount /dev/sdb /mnt/mydisk ls /mnt/mydisk/ cat /mnt/mydisk/shared_file i can see the shared_file now when i add new files on test1 im not able to see it on test2 untill i unmount the volume with directory and mount the directory again to the volume on test2. what could be the solution for it? how can i make the volume share datas instantly between all the instances connected to the volume i have tried with ext4 and xfs file system Hope to hear from you soon Thank you!

2 1

Problem with deploying deployed ceph in OS wallaby.
by davorin.mikulic＠gmail.com 17 Jan '24

17 Jan '24

Hello. I have a problem deploying ceph cluster prior to deploying OS with the following command. openstack overcloud ceph deploy -o /home/stack/3XOenvironment/deployed-ceph.yaml --stack overcloud --roles-data /home/stack/3XOenvironment/30-roles_data.yaml --network-data /home/stack/3XOenvironment/40-network_data.yaml --config /home/stack/3XOenvironment/ceph.conf --ntp-server 10.100.250.101 --container-image-prepare /home/stack/3XOenvironment/00-containers-prepare-parameter.yaml /home/stack/3XOenvironment/overcloud-baremetal-deployed.yaml -vvv I have successfully provisioned overcloud networks, vips and nodes (HPE GEN 9 machines). Ceph clusters seems to create but it fails on creating the environment file (deployed-ceph.yaml) for the openstack overcloud deploy command. The error involves tld_option which I cannot find anywhere. 2024-01-16 11:29:06.616835 | 000c29b1-9f06-81af-0614-00000000011b | TASK | Ceph cluster status 2024-01-16 11:29:06.654033 | 000c29b1-9f06-81af-0614-00000000011b | OK | Ceph cluster status | overcloud-controller-0 | result={ "changed": false, "msg": [ " cluster:", " id: 472da67d-c46c-5717-880e-574b9105d723", " health: HEALTH_OK", " ", " services:", " mon: 3 daemons, quorum overcloud-controller-0,overcloud-controller-2,overcloud-controller-1 (age 25m)", " mgr: overcloud-controller-0.ewcaqz(active, since 26m), standbys: overcloud-controller-2.gaxwws, overcloud-controller-1.muacne", " osd: 3 osds: 3 up (since 23m), 3 in (since 24m)", " ", " data:", " pools: 1 pools, 1 pgs", " objects: 0 objects, 0 B", " usage: 14 MiB used, 22 TiB / 22 TiB avail", " pgs: 1 active+clean", " " ] } 2024-01-16 11:29:06.655731 | 000c29b1-9f06-81af-0614-00000000011b | TIMING | tripleo_cephadm : Ceph cluster status | overcloud-controller-0 | 0:00:45.445615 | 0.04s 2024-01-16 11:29:06.668350 | 000c29b1-9f06-81af-0614-000000000131 | TASK | Create Deployed Ceph Heat Environment file to pass to overcloud deploy <localhost> ESTABLISH LOCAL CONNECTION FOR USER: stack <localhost> EXEC /bin/sh -c 'echo ~stack && sleep 0' <localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/stack/.ansible/tmp `"&& mkdir "` echo /home/stack/.ansible/tmp/ansible-tmp-1705400946.7091644-595040-96765848827570 `" && echo ansible-tmp-1705400946.7091644-595040-96765848827570="` echo /home/stack/.ansible/tmp/ansible-tmp-1705400946.7091644-595040-96765848827570 `" ) && sleep 0' <localhost> EXEC /bin/sh -c 'rm -f -r /home/stack/.ansible/tmp/ansible-tmp-1705400946.7091644-595040-96765848827570/ > /dev/null 2>&1 && sleep 0' 2024-01-16 11:29:06.768346 | 000c29b1-9f06-81af-0614-000000000131 | FATAL | Create Deployed Ceph Heat Environment file to pass to overcloud deploy | overcloud-controller-0 -> localhost | error={ "changed": false, "msg": "AnsibleUndefinedVariable: {{ tld_option }}: 'tld_option' is undefined" } [WARNING]: ('overcloud-controller-0 -> localhost', '000c29b1-9f06-81af-0614-000000000131') missing from stats PLAY RECAP ********************************************************************* overcloud-controller-0 : ok=37 changed=8 unreachable=0 failed=1 skipped=43 rescued=0 ignored=1 undercloud : ok=10 changed=3 unreachable=0 failed=0 skipped=10 rescued=0 ignored=0 2024-01-16 11:29:06.777893 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-01-16 11:29:06.778573 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 102 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-01-16 11:29:06.779338 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:00:45.569227 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-01-16 11:29:06.780170 | UUID | Info | Host | Task Name | Run Time 2024-01-16 11:29:06.781078 | 000c29b1-9f06-81af-0614-000000000187 | SUMMARY | undercloud | tripleo_run_cephadm : Run ceph-admin-user-playbook for public and private keys | 7.36s 2024-01-16 11:29:06.781746 | 000c29b1-9f06-81af-0614-000000000188 | SUMMARY | undercloud | tripleo_run_cephadm : Run ceph-admin-user-playbook for public keys | 6.19s 2024-01-16 11:29:06.782621 | 000c29b1-9f06-81af-0614-00000000009b | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Set public/cluster network and v4/v6 ms_bind unless already in ceph.conf | 4.41s 2024-01-16 11:29:06.783807 | 000c29b1-9f06-81af-0614-0000000000b2 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Apply spec | 3.20s 2024-01-16 11:29:06.784945 | 000c29b1-9f06-81af-0614-00000000002d | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Install cephadm package | 3.19s 2024-01-16 11:29:06.785972 | 000c29b1-9f06-81af-0614-0000000002fb | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Wait for expected number of osds to be running | 1.27s 2024-01-16 11:29:06.786740 | 000c29b1-9f06-81af-0614-000000000118 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Show the deployed daemons | 1.21s 2024-01-16 11:29:06.787585 | 000c29b1-9f06-81af-0614-00000000011a | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Print the status of the deployed Ceph cluster | 1.21s 2024-01-16 11:29:06.788369 | 000c29b1-9f06-81af-0614-000000000115 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Get the ceph orchestrator status | 1.16s 2024-01-16 11:29:06.789142 | 000c29b1-9f06-81af-0614-00000000029f | SUMMARY | overcloud-controller-0 | tripleo_cephadm : set container image base in ceph configuration | 1.12s 2024-01-16 11:29:06.789892 | 000c29b1-9f06-81af-0614-0000000000af | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Get the ceph orchestrator status | 1.11s 2024-01-16 11:29:06.790586 | 000c29b1-9f06-81af-0614-00000000003b | SUMMARY | overcloud-controller-0 | push tripleo_cephadm_spec to bootstrap node if spec file exsits | 0.89s 2024-01-16 11:29:06.791367 | 000c29b1-9f06-81af-0614-000000000184 | SUMMARY | undercloud | tripleo_run_cephadm : Populate cephadm_admin_limit_file | 0.81s 2024-01-16 11:29:06.792182 | 000c29b1-9f06-81af-0614-000000000030 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : List Ceph daemon instances on this host | 0.69s 2024-01-16 11:29:06.792987 | 000c29b1-9f06-81af-0614-000000000019 | SUMMARY | undercloud | Create Ceph spec based on baremetal_deployed_path and tripleo_roles | 0.68s 2024-01-16 11:29:06.793962 | 000c29b1-9f06-81af-0614-00000000003e | SUMMARY | overcloud-controller-0 | push it to tripleo_cephadm_assimilate_conf on bootstrap node | 0.66s 2024-01-16 11:29:06.794719 | 000c29b1-9f06-81af-0614-000000000185 | SUMMARY | undercloud | tripleo_run_cephadm : Populate cephadm_non_admin_limit_file | 0.59s 2024-01-16 11:29:06.795418 | 000c29b1-9f06-81af-0614-00000000006e | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Stat necessary files to bootstrap with cephadm | 0.56s 2024-01-16 11:29:06.796092 | 000c29b1-9f06-81af-0614-00000000002e | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Stat cephadm file | 0.37s 2024-01-16 11:29:06.796735 | 000c29b1-9f06-81af-0614-000000000038 | SUMMARY | overcloud-controller-0 | Ensure tripleo_cephadm_config_home (e.g. /etc/ceph) exists | 0.37s 2024-01-16 11:29:06.797373 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-01-16 11:29:06.798022 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ State Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-01-16 11:29:06.798652 | ~~~~~~~~~~~~~~~~~~ Number of nodes which did not deploy successfully: 1 ~~~~~~~~~~~~~~~~~ 2024-01-16 11:29:06.799383 | The following node(s) had failures: overcloud-controller-0 2024-01-16 11:29:06.800015 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Temporary directory [ /tmp/tripleozr0mivzo ] cleaned up Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-deployed-ceph.yaml, Run Status: failed, Return Code: 2 Temporary directory [ /tmp/tripleo9zw51ssj ] cleaned up Exception occured while running the command Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in run return super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v2/overcloud_ceph.py", line 739, in take_action skip_tags=','.join(skip_tags), File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 775, in run_ansible_playbook raise RuntimeError(err_msg) RuntimeError: Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-deployed-ceph.yaml, Run Status: failed, Return Code: 2 Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-deployed-ceph.yaml, Run Status: failed, Return Code: 2 clean_up OvercloudCephDeploy: Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-deployed-ceph.yaml, Run Status: failed, Return Code: 2 END return value: 1

2 2

[heat] Propose retirement of heat-cfnclient
by Takashi Kajinami 16 Jan '24

16 Jan '24

Hello, In heat project, we have had the heat-cfnclient library which contains the client utilities to consume AWS-compatible APIs in heat. However this repository is not properly maintained and I doubt that anyone is using (or even can use) it with recent Heat. - The repository still contains the utility for CloudWatch API which was already removed from heat - The heat-cfn command relies on Keystone v2 instead of v3 - No release has been made since the repository was created Based on the current situation I'd propose retiring the repository. I'd appreciate any feedback and if I hear no objections then I'll start the official process to retire the repository. Thank you, Takashi

2 3

[kolla-ansible][Antelope][Senlin] Senlin Rolling Updates Not Working
by Devin Patterson 16 Jan '24

16 Jan '24

Hello, We have deployed Senlin clusters to manage VM health. When attempting to do rolling updates of VMs by updating the profile, the VMs go into an error state with the following error in the senlin engine logs. I've included the Heat template and OS commands. Any help would be appreciated. openstack cluster update --profile <new_profile> <senlin_cluster> cluster event show <event_d> status_reason | Failed in updating nodes. heat_template_version: 2021-04-16 description: OpenStack Heat template to deploy a Senlin autoscaling group resources: profile1: type: OS::Senlin::Profile properties: type: os.nova.server-1.0 properties: flavor: m1.small image: cirros-0.5.1 key_name: keypair networks: - network: network user_data: | #!/bin/sh echo 'hello, world' > /tmp/test_file batch_policy: type: OS::Senlin::Policy properties: type: senlin.policy.batch-1.0 bindings: - cluster: {get_resource: cluster} properties: min_in_service: 1 max_batch_size: 2 pause_time: 3 cluster: type: OS::Senlin::Cluster properties: desired_capacity: 8 min_size: 2 profile: { get_resource: profile1 } receiver_scale_out: type: OS::Senlin::Receiver properties: cluster: {get_resource: cluster} action: CLUSTER_SCALE_OUT type: webhook receiver_scale_in: type: OS::Senlin::Receiver properties: cluster: {get_resource: cluster} action: CLUSTER_SCALE_IN type: webhook outputs: receiver_scale_out: description: Webhook to scale out cluster. value: str_replace: template: curl -X POST LINK params: LINK: {get_attr: [receiver_scale_out, channel, alarm_url]} receiver_scale_in: description: Webhook to scale in cluster. value: str_replace: template: curl -X POST LINK params: LINK: {get_attr: [receiver_scale_in, channel, alarm_url]} Regards, Devin

1 0

[kolla-ansible][Antelope][Heat] Heat Autoscaling Scaling Policy URL
by Devin Patterson 16 Jan '24

16 Jan '24

Hello, I am attempting to create an auto-scaling group with Heat. The group is deployed properly along with scale up and scale down policies. When attempting to curl with POST, I get the following response. This is using the "alarm_url" from the scaling policy. Below is also the Heat template being utilized. Below the error response is the curl command. Wondering why there is an AWSAccessKeyId in the signature of the URL as well. <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>500 Internal Server Error</title> </head><body> <h1>Internal Server Error</h1> <p>The server encountered an internal error or misconfiguration and was unable to complete your request.</p> <p>Please contact the server administrator at [no address given] to inform them of the time this error occurred, and the actions you performed just before this error.</p> <p>More information about this error may be available in the server error log.</p> </body></html> CURL COMMAND: curl -X POST <external-os-url>/v1/signal/arn%3Aopenstack%3Aheat%3A%omitted3Astacks/stack_1/omitted/resources/scale_down_policy\?SignatureMethod\=HmacSHA256\&SignatureVersion\=2\&AWSAccessKeyId\=omitted\&Signature\=omitted TEMPLATE: heat_template_version: 2021-04-16 description: OpenStack Heat template to deploy a autoscaling group resources: asg: type: OS::Heat::AutoScalingGroup properties: desired_capacity: 6 max_size: 8 min_size: 2 rolling_updates: min_in_service: 1 max_batch_size: 2 pause_time: 3 resource: type: OS::Nova::Server properties: flavor: m1.small image: image key_name: my-keypair networks: - network: network user_data: | #!/bin/sh echo 'hello, world123' > /tmp/test_file scale_up_policy: type: OS::Heat::ScalingPolicy properties: adjustment_type: change_in_capacity auto_scaling_group_id: { get_resource: asg } cooldown: 60 scaling_adjustment: 1 scale_down_policy: type: OS::Heat::ScalingPolicy properties: adjustment_type: change_in_capacity auto_scaling_group_id: { get_resource: asg } cooldown: 60 scaling_adjustment: -1 outputs: scale_up_url: description: > This URL is the webhook to scale up the group. You can invoke the scale-up operation by doing an HTTP POST to this URL; no body nor extra headers are needed. value: {get_attr: [scale_up_policy, alarm_url]} scale_dn_url: description: > This URL is the webhook to scale down the group. You can invoke the scale-down operation by doing an HTTP POST to this URL; no body nor extra headers are needed. value: {get_attr: [scale_down_policy, alarm_url]} asg_size: description: > This is the current size of the auto scaling group. value: {get_attr: [asg, current_size]} server_list: description: > This is a list of server names that are part of the group. value: {get_attr: [asg, outputs_list, name]} networks: description: > This is a map of server resources and their networks. value: {get_attr: [asg, outputs, networks]} server_ips: description: > This is a list of first ip addresses of the servers in the group for a specified network. value: {get_attr: [asg, outputs_list, networks, {get_param: network}, 0]} Regards, Devin

1 0

[magnum] k8s-keystone-auth pod CrashLoopBackOff
by Eugen Block 16 Jan '24

16 Jan '24

Hi there, I've noticed several magnum related threads and thought I'd give it a try while I keep digging. I've only recently started to work with magnum, and this is how far I get. Note that this openstack is still running on version Victoria, I'm working towards migrating it to a newer release. My coe cluster template currently looks like this (I'll omit the defaults): | labels | {'kube_tag': 'v1.18.2', 'hyperkube_prefix': 'k8s.gcr.io/'} | | https_proxy | http://<IP>:<PORT> | | http_proxy | http://<IP>:<PORT> | | image_id | fedora-coreos33 | | dns_nameserver | <MY_DNS> | I've already tried quite a lot of different kube_tags and hyperkube_prefixes, this combination gets me at least to have some running containers: [root@kubernetes-cluster20-buuxqog6cfnn-master-0 ~]# podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 7fbe3f8f308f docker.io/openstackmagnum/heat-container-agent:victoria-stable-1 /usr/bin/start-he... 42 minutes ago Up 42 minutes ago heat-container-agent a8b6ae357034 k8s.gcr.io/hyperkube:v1.18.2 kube-apiserver --... 40 minutes ago Up 40 minutes ago kube-apiserver cb7774fbf849 k8s.gcr.io/hyperkube:v1.18.2 kube-controller-m... 40 minutes ago Up 40 minutes ago kube-controller-manager 448f1c02cbd4 k8s.gcr.io/hyperkube:v1.18.2 kube-scheduler --... 40 minutes ago Up 40 minutes ago kube-scheduler 4be28a0a2271 k8s.gcr.io/hyperkube:v1.18.2 kubelet --logtost... 40 minutes ago Up 40 minutes ago kubelet 542ffdb081e7 k8s.gcr.io/hyperkube:v1.18.2 kube-proxy --logt... 40 minutes ago Up 40 minutes ago kube-proxy 61874a4e4595 quay.io/coreos/etcd:v3.4.6 /usr/local/bin/et... 40 minutes ago Up 40 minutes ago etcd But not all pods are starting: [root@kubernetes-cluster20-buuxqog6cfnn-master-0 ~]# /var/srv/magnum/bin/kubectl get pods -o wide --kubeconfig=/etc/kubernetes/admin.conf -A NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES kube-system coredns-786ffb7797-dpkbx 1/1 Running 0 25m 10.100.0.3 kubernetes-cluster20-buuxqog6cfnn-master-0 <none> <none> kube-system coredns-786ffb7797-mslfk 1/1 Running 0 25m 10.100.0.2 kubernetes-cluster20-buuxqog6cfnn-master-0 <none> <none> kube-system dashboard-metrics-scraper-6b4884c9d5-nrvbg 1/1 Running 0 25m 10.100.0.5 kubernetes-cluster20-buuxqog6cfnn-master-0 <none> <none> kube-system k8s-keystone-auth-qt7n5 0/1 CrashLoopBackOff 4 5m12s 10.0.0.116 kubernetes-cluster20-buuxqog6cfnn-master-0 <none> <none> kube-system kube-dns-autoscaler-75859754fd-69zxx 0/1 Pending 0 25m <none> <none> <none> <none> kube-system kube-flannel-ds-cn5jf 1/1 Running 0 25m 10.0.0.116 kubernetes-cluster20-buuxqog6cfnn-master-0 <none> <none> kube-system kubernetes-dashboard-c98496485-qw4hl 1/1 Running 0 25m 10.100.0.4 kubernetes-cluster20-buuxqog6cfnn-master-0 <none> <none> kube-system magnum-metrics-server-79556d6999-hkqcw 0/1 Pending 0 25m <none> <none> <none> <none> I didn't look too deep into the two other failing pods, I think the keystone pod is more important at this point or maybe even is required for the other pods to start, I don't know yet. I can pull the image manually and the "describe pod" command also shows that the container was pulled successfully, but I can't find other hints what is actually failing. Here is the full output of "describe pod": ---snip--- [root@kubernetes-cluster20-buuxqog6cfnn-master-0 ~]# /var/srv/magnum/bin/kubectl --kubeconfig=/etc/kubernetes/admin.conf -n kube-system describe pod/k8s-keystone-auth-qt7n5 Name: k8s-keystone-auth-qt7n5 Namespace: kube-system Priority: 0 Node: kubernetes-cluster20-buuxqog6cfnn-master-0/10.0.0.116 Start Time: Mon, 15 Jan 2024 09:13:16 +0000 Labels: controller-revision-hash=6875f96b46 k8s-app=k8s-keystone-auth pod-template-generation=1 Annotations: kubernetes.io/psp: magnum.privileged Status: Running IP: 10.0.0.116 IPs: IP: 10.0.0.116 Controlled By: DaemonSet/k8s-keystone-auth Containers: k8s-keystone-auth: Container ID: docker://adca0b9a9eb011f22530886a42bf862b4cfa7e6b7965cd77d6dac4f37222b827 Image: docker.io/k8scloudprovider/k8s-keystone-auth:v1.18.0 Image ID: docker-pullable://k8scloudprovider/k8s-keystone-auth@sha256:ab72a7e5b9eca9af2762796690be800bce2de25dee8bd5ad55d0996b29c85146 Port: 8443/TCP Host Port: 8443/TCP Args: ./bin/k8s-keystone-auth --tls-cert-file /etc/kubernetes/certs/server.crt --tls-private-key-file /etc/kubernetes/certs/server.key --policy-configmap-name k8s-keystone-auth-policy --keystone-url http://<CONTROLLER>:5000/v3 --sync-configmap-name keystone-sync-policy --keystone-ca-file /etc/kubernetes/ca-bundle.crt --listen 127.0.0.1:8443 State: Terminated Reason: Error Exit Code: 1 Started: Mon, 15 Jan 2024 09:23:52 +0000 Finished: Mon, 15 Jan 2024 09:23:52 +0000 Last State: Terminated Reason: Error Exit Code: 1 Started: Mon, 15 Jan 2024 09:18:41 +0000 Finished: Mon, 15 Jan 2024 09:18:41 +0000 Ready: False Restart Count: 6 Requests: cpu: 200m Environment: <none> Mounts: /etc/kubernetes from ca-certs (ro) /etc/kubernetes/certs from k8s-certs (ro) /var/run/secrets/kubernetes.io/serviceaccount from k8s-keystone-auth-token-vhm4r (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: k8s-certs: Type: HostPath (bare host directory volume) Path: /etc/kubernetes/certs HostPathType: DirectoryOrCreate ca-certs: Type: HostPath (bare host directory volume) Path: /etc/kubernetes HostPathType: DirectoryOrCreate k8s-keystone-auth-token-vhm4r: Type: Secret (a volume populated by a Secret) SecretName: k8s-keystone-auth-token-vhm4r Optional: false QoS Class: Burstable Node-Selectors: node-role.kubernetes.io/master= Tolerations: :NoSchedule :NoExecute CriticalAddonsOnly node.kubernetes.io/disk-pressure:NoSchedule node.kubernetes.io/memory-pressure:NoSchedule node.kubernetes.io/network-unavailable:NoSchedule node.kubernetes.io/not-ready:NoExecute node.kubernetes.io/pid-pressure:NoSchedule node.kubernetes.io/unreachable:NoExecute node.kubernetes.io/unschedulable:NoSchedule Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled <unknown> default-scheduler Successfully assigned kube-system/k8s-keystone-auth-qt7n5 to kubernetes-cluster20-buuxqog6cfnn-master-0 Warning Failed 9m52s kubelet, kubernetes-cluster20-buuxqog6cfnn-master-0 Error: ErrImagePull Warning Failed 9m52s kubelet, kubernetes-cluster20-buuxqog6cfnn-master-0 Failed to pull image "docker.io/k8scloudprovider/k8s-keystone-auth:v1.18.0": rpc error: code = Unknown desc = Get "https://registry-1.docker.io/v2/k8scloudprovider/k8s-keystone-auth/manifest…": Internal Server Error Normal Pulling 9m37s (x5 over 10m) kubelet, kubernetes-cluster20-buuxqog6cfnn-master-0 Pulling image "docker.io/k8scloudprovider/k8s-keystone-auth:v1.18.0" Normal Pulled 9m36s (x4 over 10m) kubelet, kubernetes-cluster20-buuxqog6cfnn-master-0 Successfully pulled image "docker.io/k8scloudprovider/k8s-keystone-auth:v1.18.0" Normal Created 9m35s (x4 over 10m) kubelet, kubernetes-cluster20-buuxqog6cfnn-master-0 Created container k8s-keystone-auth Normal Started 9m35s (x4 over 10m) kubelet, kubernetes-cluster20-buuxqog6cfnn-master-0 Started container k8s-keystone-auth Warning BackOff 33s (x48 over 10m) kubelet, kubernetes-cluster20-buuxqog6cfnn-master-0 Back-off restarting failed container ---snip--- I'm obviously not allowed to get logs from the pod: [root@kubernetes-cluster20-buuxqog6cfnn-master-0 ~]# /var/srv/magnum/bin/kubectl --kubeconfig=/etc/kubernetes/admin.conf -n kube-system logs pod/k8s-keystone-auth-qt7n5 Error from server: Get https://10.0.0.116:10250/containerLogs/kube-system/k8s-keystone-auth-qt7n5/…: Forbidden I appreciate any hints! Thanks! Eugen

1 3

[keystone] Keystone as a Service Provider, 401 - Unauthorized Exception for SSO
by joeldag7＠gmail.com 16 Jan '24

16 Jan '24

We're currently working on implementing SSO for our university's OpenStack environment using OpenStack Ansible (Yoga version). Our goal is to allow students to log in with their university credentials via an Identity Provider - currently we're using our own Keycloak instance as IdP. The plan is to automatically create a new project for first time users, while returning students will be redirected to their existing projects to manage their instances. For initial testing, we're directing all users to a single project. Eventually, we would like to script the dynamic creation of projects on the controller node for first-time logins later. However, we've run into some problems: Our Keystone configuration in the user_variables.yml file is as follows: ``` keystone_sp: apache_mod: 'mod_auth_openidc' cadf_notifications: true trusted_dashboard_list: - "https://openstack-example.com/auth/websso/" - "http://example.keycloak.com/realms/openstack*" - "https://openstack-example.com/auth/login/" trusted_idp_list: - name: "oidc-idp" oidc_provider_metadata_url: http://example.keycloak.com/realms/openstack/.well-known/openid-configurati… oidc_client_id: ******** oidc_client_secret: ********* oidc_crypto_passphrase: ****** oidc_redirect_uri: https://openstack-example.com/v3/OS-FEDERATION/identity_providers/oidc-idp/… entity_ids: - 'http://exmaple.keycloak.com/realms/openstack' federated_identities: - domain: default project: fedproject group: fedgroup role: member protocols: - name: openid mapping: name: openid-mapping rules: - remote: - type: email local: - group: name: fedgroup domain: name: Default user: name: '{0}' ``` This setup is based on the openstack documentation (https://docs.openstack.org/openstack-ansible-os_keystone/yoga/configure-fed…) After many attempts with different configurations, this one successfully redirects to our Keycloak IdP for authentication. However, after login, when redirected back to OpenStack, we get a 401 Unauthorised error stating "The request you have made requires authentication". The SAML tracer logs confirm that Keycloak issues a valid token along with the user information. Despite this, we find no additional error details in the Keystone container, Apache2 and controller logs. This leads me to believe that the problem may lie in the mapping configuration, although it appears to work when tested with the keystone-manage mapping-engine tool. But still, the doc (https://docs.openstack.org/keystone/queens/advanced-topics/federation/mappi…) claims that a 401 exception could be caused by the mapping. Any insight or suggestions on how to solve this problem would be greatly appreciated, particularly regarding the mapping process during authentication. I'm also looking for advice on how to extract more detailed error information. At the moment, the logs don't reveal much beyond the 401 error message. Am I missing something or is this all the information OpenStack typically provides in such cases? Thank you for your help and guidance.

1 0

OpenStack D Release Name
by Allison Price 15 Jan '24

15 Jan '24

Hi everyone, I am excited to share that the name for the OpenStack D release, scheduled for 2H 2024 is Dalmatian. (Congrats, Amy!) We look forward to developing the 30th release of OpenStack! Cheers, Allison

4 3

[tc] Next Technical Committee Meeting Jan 16 2024, 1800 UTC
by Jay Faulkner 15 Jan '24

15 Jan '24

Hi all, The next weekly OpenStack Technical Committee meeting is tomorrow, Tuesday Jan 16 at 1800 UTC in #openstack-tc on OFTC. The proposed agenda is: Agenda - Roll call - Follow up on tracked action items - JayF - Email mailing list a summary of the discussion around Monasca, highlighting issues around trust and project activity along with a link to the meeting log. - slaweq - Reach out privately to volunteer to maintain Monasca about official vs unofficial options - gmann - Reach out privately to volunteer to maintain Senlin about next steps - Gate health check - Implementation of Unmaintained branch statuses - Inactive projects for Caracal - It is past C-2, these projects will not be in Caracal release. Would we like to take further action in Caracal? - Monasca - Senlin - Freezer - Solum - EC2 API - Sahara - 2024.1 TC Tracker - https://etherpad.opendev.org/p/tc-2024.1-tracker - Open Discussion and Reviews Thanks, Jay Faulkner

1 0

[neutron] Bug deputy - week of Jan 8
by Sławek Kapłoński 15 Jan '24

15 Jan '24

Hi, It was pretty quiet week. Below is summary of the bugs opened this last week and its current status. ## High * https://bugs.launchpad.net/neutron/+bug/2048745 - [ovn] FIP not working when mixing vlan and geneve tenant networks - not assigned, many things to consider before we will start working on this improvement really * https://bugs.launchpad.net/neutron/+bug/2048785 - Trunk parent port (tpt port) vlan_mode is wrong in ovs - WIP fix already started by Bence: https://review.opendev.org/c/openstack/neutron/+/905125 * https://bugs.launchpad.net/neutron/+bug/2049265 - [OVN] The ``MaintenanceWorker`` shoudl register the "Private_Chassis" table in the SB connection - assigned to Terry Wilson, fixes already proposed https://review.opendev.org/q/I711996b7644e80bc195833e4429e4d745728f9cf ## Medium * https://bugs.launchpad.net/neutron/+bug/2048979 - [ml2][ovs] ports without local vlan tag are processed on openflow security group - fix proposed by LIU: https://review.opendev.org/c/openstack/neutron/+/905289 ## Others * https://bugs.launchpad.net/neutron/+bug/2048773 - neutron-meter-agent do not support ovn driver - unassigned, it isn't really completly new RFE IMO but probably approach to take to close this gap may be discussed e.g. in the drivers meeting, * https://bugs.launchpad.net/neutron/+bug/2049398 - [Tempest] neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_delete_policies_while_tenant_attached_to_net "Created Network is Not an External Network" Error - needs some more info to be triaged, I will take care of it -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[Image Encryptione] Cancelling tomorrows meeting
by Josephine Seifert 14 Jan '24

14 Jan '24

Hi, unfortunately I have to cancel tomorrows meeting. greetings, Luzi (Josephine)

1 0

[kolla]Boot one VM with two GPU(in same numa)by pci passthrough cannot have GPUDirect P2P capability
by dazhaoyu＠126.com 12 Jan '24

12 Jan '24

I have two GPU cards, all of them was connect with one same numa CPU socket as below link info: https://paste.opendev.org/show/b7Qi8qCnbLVxO2W0JdQw/ I can boot one nova instance successfully with the two GPU cards by PCI Passthrough way. but in the booted instances, use deviceQuery method would get the below message: Peer access from NVIDIA RTX 6000(GPU0) -> NVIDIA RTX 6000(GPU1): NO Peer access from NVIDIA RTX 6000(GPU1) -> NVIDIA RTX 6000(GPU0): NO The expected return should be as below： Peer access from NVIDIA RTX 6000(GPU0) -> NVIDIA RTX 6000(GPU1): YES Peer access from NVIDIA RTX 6000(GPU1) -> NVIDIA RTX 6000(GPU0): YES so that the memory can be shared between the two GPUs. I'm running Openstack Xena release in Intel Xeon Gold 5220R CPU

1 0

[release] Release countdown for week R-11, Jan 15-19
by Elõd Illés 12 Jan '24

12 Jan '24

Development Focus ----------------- We are now past the Caracal-2 milestone, and entering the last development phase of the cycle. Teams should be focused on implementing planned work for the cycle. Now is a good time to review those plans and reprioritize anything if needed based on the what progress has been made and what looks realistic to complete in the next few weeks. General Information ------------------- Looking ahead to the end of the release cycle, please be aware of the feature freeze dates. Those vary depending on deliverable type: * General libraries (except client libraries) need to have their last feature release before Non-client library freeze (February 22nd, 2024). Their stable branches are cut early. * Client libraries (think python-*client libraries) need to have their last feature release before Client library freeze (February 29th, 2024) * Deliverables following a cycle-with-rc model (that would be most services) observe a Feature freeze on that same date, February 29th, 2024. Any feature addition beyond that date should be discussed on the mailing-list and get PTL approval. After feature freeze, cycle-with-rc deliverables need to produce a first release candidate (and a stable branch) before RC1 deadline (March 14th, 2024) * Deliverables following cycle-with-intermediary model can release as necessary, but in all cases before Final RC deadline (March 28th, 2024) Upcoming Deadlines & Dates -------------------------- Non-client library freeze: February 22nd, 2024 (R-6 week) Client library freeze: February 29th, 2024 (R-5 week) Caracal-3 milestone: February 29th, 2024 (R-5 week) 2024.1 Caracal final release: April 3rd, 2024 2024.2 'D' virtual PTG ( https://openinfra.dev/ptg/ ): April 8th-12th, 2024 Előd Illés irc: elodilles @ #openstack-release

1 0