- openstack-discuss - lists.openstack.org

Re: [kuryr][tacker] fails to bump the k8s version to v1.26
by Michał Dulko 13 Jul '23

13 Jul '23

On Thu, 2023-07-13 at 10:16 +0000, Ayumu Ueha (Fujitsu) wrote: > Hi Michał, > > Thanks for your reply, > > I trying it in my local machine and check logs, but I am not familiar > with troubleshooting with kubelet/kubeadm, so I do not know why it > errored as for now. > We will continue to investigate, but it would be helpful if you could > share any small findings. > > For your reference, I will attach a log of the kubelet.service when > it failed in my local environment. Looks like kube-apiserver and kube-scheduler are failing to start. Checking logs of them would be my next idea. You might want to use `crictl` to find them. > Thank you. > > Best Regards, > Ueha > > -----Original Message----- > From: Michał Dulko <mdulko(a)redhat.com> > Sent: Thursday, July 13, 2023 3:48 PM > To: openstack-discuss(a)lists.openstack.org > Subject: Re: [kuryr][tacker] fails to bump the k8s version to v1.26 > > On Tue, 2023-07-04 at 07:53 +0000, Ayumu Ueha (Fujitsu) wrote: > > > > > > > > Hi kuryr-kubetenetes team, > > > > Tacker uses the kuryr-kubernetes as the setup for the kubernetes > > Environment. > > In the Bobcat version, we will bump the supported version of > > kubenetes > > to 1.26.6, and when I tried to bump the version in devstack's > > local.conf by the patch [1], the following error occurred. (please > > refer the details to Zuul log > > [2]) > > ============================== > > [wait-control-plane] Waiting for the kubelet to boot up the control > > plane as static Pods from directory "/etc/kubernetes/manifests". > > This > > can take up to 4m0s [kubelet-check] Initial timeout of 40s passed. > > > > Unfortunately, an error has occurred: > > timed out waiting for the condition > > > > This error is likely caused by: > > - The kubelet is not running > > - The kubelet is unhealthy due to a misconfiguration > > of > > the node in some way (required cgroups disabled) > > > > If you are on a systemd-powered system, you can try to troubleshoot > > the error with the following commands: > > - 'systemctl status kubelet' > > - 'journalctl -xeu kubelet' > > > > Additionally, a control plane component may have crashed or exited > > when started by the container runtime. > > To troubleshoot, list all containers using your preferred container > > runtimes CLI. > > Here is one example how you may list all running Kubernetes > > containers > > by using crictl: > > - 'crictl --runtime-endpoint > > unix:///var/run/crio/crio.sock ps -a | grep kube | grep -v pause' > > Once you have found the failing container, you can > > inspect its logs with: > > - 'crictl --runtime-endpoint > > unix:///var/run/crio/crio.sock logs CONTAINERID' > > error execution phase wait-control-plane: couldn't initialize a > > Kubernetes cluster ============================== > > > > I know it is not yet supporting version 1.26 of the K8S at the > > kuryr- > > kubernetes as for now, but do you know any way to avoid the above > > error? > > I suspect this is due to a change from version 1.25, but I'm not > > sure > > which one is affecting... > > > > Also, when will you support the K8S v 1.26 at the kuryr-kubernetes? > > Will the Bobcat release support it? > > Please kindly let me know if you know anything. Thank you. > > Kuryr-kubernetes itself supports K8s 1.26 and 1.27 as we test it with > these versions as part of OpenShift. What doesn't seem to work is the > DevStack plugin, I guess due to some breaking changes in kubelet or > kubeadm. > > I can advise to try to deploy DevStack with these settings on your > local machine and then you should have access to all the logs > required to figure out what is causing the problem with the version > bump. > > > [1] https://review.opendev.org/c/openstack/tacker/+/886935 > > Change the parameters: > > KURYR_KUBERNETES_VERSION: 1.25.6 to 1.26.6 > > CRIO_VERSION: 1.25 to 1.26 > > [2] > > https://zuul.opendev.org/t/openstack/build/1a4061da74e640368da133ba219 > > b54d9/log/controller-k8s/logs/devstacklog.txt#9761-9816 > > > > Best Regards, > > Ueha > >

1 0

[cinder-backup][kolla] mix cinder-volume and cinder-backup types (nfs, ceph)
by garcetto 13 Jul '23

13 Jul '23

good evening, it is possible to mix cinder-volume backend (say ceph) and cinder-backup backend (say nfs)? and also do incrementals backup? thank you

3 3

[kuryr][tacker] fails to bump the k8s version to v1.26
by Ayumu Ueha (Fujitsu) 13 Jul '23

13 Jul '23

Hi kuryr-kubetenetes team, Tacker uses the kuryr-kubernetes as the setup for the kubernetes Environment. In the Bobcat version, we will bump the supported version of kubenetes to 1.26.6, and when I tried to bump the version in devstack's local.conf by the patch [1], the following error occurred. (please refer the details to Zuul log [2]) ============================== [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s [kubelet-check] Initial timeout of 40s passed. Unfortunately, an error has occurred: timed out waiting for the condition This error is likely caused by: - The kubelet is not running - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled) If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands: - 'systemctl status kubelet' - 'journalctl -xeu kubelet' Additionally, a control plane component may have crashed or exited when started by the container runtime. To troubleshoot, list all containers using your preferred container runtimes CLI. Here is one example how you may list all running Kubernetes containers by using crictl: - 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock ps -a | grep kube | grep -v pause' Once you have found the failing container, you can inspect its logs with: - 'crictl --runtime-endpoint unix:///var/run/crio/crio.sock logs CONTAINERID' error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster ============================== I know it is not yet supporting version 1.26 of the K8S at the kuryr-kubernetes as for now, but do you know any way to avoid the above error? I suspect this is due to a change from version 1.25, but I'm not sure which one is affecting... Also, when will you support the K8S v 1.26 at the kuryr-kubernetes? Will the Bobcat release support it? Please kindly let me know if you know anything. Thank you. [1] https://review.opendev.org/c/openstack/tacker/+/886935 Change the parameters: KURYR_KUBERNETES_VERSION: 1.25.6 to 1.26.6 CRIO_VERSION: 1.25 to 1.26 [2] https://zuul.opendev.org/t/openstack/build/1a4061da74e640368da133ba219b54d9… Best Regards, Ueha

2 2

[horizon][keystone] Adding different rules in the same protocol for federated logon
by James Leong 13 Jul '23

13 Jul '23

Hi all, I have yoga version openstack with the deployment tool of kolla-ansible. I am trying to combine different mapping rules such as allowing user to login to different domain. However, I am not able to do that in a single JSON file. When I try to include different rule in the same JSON file, only the first rule is being considered. Is there a way to allow multiple rule to redirect user to their account in a different domain. Best, James

2 4

[Glance] Cancelling Weekly Meeting 13th July
by Pranali Deore 12 Jul '23

12 Jul '23

Hello, I will not be around during the meeting time due to some personal reasons and there is nothing much on agenda, so cancelling the meeting for this week. Let's meet next week on 20th July !! Thanks, Pranali D

1 0

Need help
by Gk Gk 12 Jul '23

12 Jul '23

Hi All, If I use the openstacksdk to connect to an openstack cloud, I have to use clouds.yaml file for specifying the cloud configuration which includes username and password as well. Since its a plain text file, how can I mask the password mentioned in clouds.yaml file for security purposes? Thanks Kumar

5 5

Cinder Bug Report 2023-07-12
by Sofia Enriquez 12 Jul '23

12 Jul '23

Hello Argonauts, Cinder Bug Meeting Etherpad <https://etherpad.opendev.org/p/cinder-bug-squad-meeting> High - Cannot create a volume from a backup <https://bugs.launchpad.net/cinder/+bug/2025277>. - *Status*: Fix proposed to master <https://review.opendev.org/c/openstack/cinder/+/887443>, but it needs a release note. - Cinder backup appears as down <https://bugs.launchpad.net/cinder/+bug/2026877>. - *Status*: Unassigned. Medium - [api] Sort by boot and pass in the marker Exception <https://bugs.launchpad.net/cinder/+bug/2027532>. - *Status*: Unassigned. - [os-brick] _get_host_uuid incompatible with btrfs snapshots <https://bugs.launchpad.net/os-brick/+bug/2026257>. - *Status*: Unassigned. Workaround on the launchpad bug report. Low - HPE 3par: Issue observed during retype/migrate. <https://bugs.launchpad.net/cinder/+bug/2026718> - *Status*: Unassigned. Cheers, -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

instance console something went wrong, connection is closed | Wallaby DCN
by Swogat Pradhan 12 Jul '23

12 Jul '23

Hi, I am creating instances in my DCN site and i am unable to get the console sometimes, error: something went wrong, connection is closed I have 3 instances now running on my hci02 node and there is console access on 1 of the vm's and the rest two i am not getting the console, i have used the same flavor, same image same security group for the VM's. Please suggest what can be done. With regards, Swogat Pradhan

3 13

Need help
by Gk Gk 12 Jul '23

12 Jul '23

Hi Folks, I work on openstack platforms. Can anyone suggest some good books or resources which will help me in becoming a cloud solutions architect ? What would this path look like ? What would be my responsibilities ? Thanks Kumar

2 3

[neutron] Bug deputy 3 July - 9 July
by Rodolfo Alonso Hernandez 11 Jul '23

11 Jul '23

Hello Neutrinos: This is the list of patches opened last week: Critical: * https://bugs.launchpad.net/neutron/+bug/2025753. "Job openstack-tox-py310-with-neutron-lib-master is failing every day 02.07.2023". Patch proposed: https://review.opendev.org/c/openstack/neutron/+/887942 High: * https://bugs.launchpad.net/neutron/+bug/2025618. "[UT] "test_negative_update_floatingip_port_forwarding" does not create the ports in the correct subnet". Patch merged: https://review.opendev.org/c/openstack/neutron/+/887495 * https://bugs.launchpad.net/neutron/+bug/2025946. "Neutron 504 Gateway Timeout Openstack Kolla-Ansible : Ussuri". *Not assigned*. Medium: * https://bugs.launchpad.net/neutron/+bug/2025637. "[ovn-octavia-provider] FIP traffic not distributed for members' FIP after lb cascade deletion". Patch merged: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/886611 * https://bugs.launchpad.net/neutron/+bug/2026122. "neutron-l3-agent permission denied when execute (list_network_namespaces)". Assigned, under investigation right now, there is also a configuration fix tested. * https://bugs.launchpad.net/neutron/+bug/2026182. "Add support for the service role in neutron API policies". Assigned, under development right now. * https://bugs.launchpad.net/neutron/+bug/2026264. " "--target" parameter in "network log object create" is only expected for ML2/OVS". Assigned, under development right now. * https://bugs.launchpad.net/neutron/+bug/2026775. "Metadata agents do not parse X-Forwarded-For headers properly". Patch proposed: https://review.opendev.org/c/openstack/neutron/+/888067. * https://bugs.launchpad.net/neutron/+bug/2026825. " [OVN] Disable LSP_OPTIONS_MCAST_FLOOD_REPORTS for LSPs". Patch proposed: https://review.opendev.org/c/openstack/neutron/+/888127. Low: * https://bugs.launchpad.net/neutron/+bug/2026547. "[RBAC] Remove "AddressScope" shared field". Assigned. Wishlist: * https://bugs.launchpad.net/neutron/+bug/2025969. "bgp os_ken ssh_console support". *Not assigned*. * https://bugs.launchpad.net/neutron/+bug/2026286. "Missing documentation for notification events". *Not assigned.* * https://bugs.launchpad.net/neutron/+bug/2026489. " [RFE][quota] Implement "router_route" quota using the Neutron quota engine, replacing "max_routes" config option". *Not assigned.* Incomplete: * https://bugs.launchpad.net/neutron/+bug/2025587. "neutron request body or parameters cause "Internal Server Error"" Regards.

1 0

[ptl] Need PTL volunteer for OpenStack Vitrage
by Ghanshyam Mann 11 Jul '23

11 Jul '23

Hi Eyal, I am reaching out to you as you were PTL for OpenStack Vitrage project in the last cycle. There is no PTL candidate for the next cycle (2023.2), and it is on the leaderless project list. Please check if you or anyone you know would like to lead this project. - https://etherpad.opendev.org/p/2023.2-leaderless Also, if anyone else would like to help leading this project, this is time to let TC knows. -gmann

3 3

[oslo] HTTP base direct RPC oslo.messaging driver contribution
by 室井雅仁 11 Jul '23

11 Jul '23

Hi oslo team, We'd like to contribute HTTP base direct RPC driver to the oslo.messaging community. We have developed the HTTP base driver internally. We have been using the driver in the production with over 10K hypervisors now. I checked the IRC meeting log of the oslo team[1], but there is no regluar meeting in 2023. Is it okay to submit oslo-spec[2] to propose the driver directly, or is there another good place to discuss the feature before submitting a spec? 1. https://meetings.opendev.org/#Oslo_Team_Meeting 2. https://opendev.org/openstack/oslo-specs best regards, Masahito

8 13

vPTG October 2023 Dates & Registration Kickoff
by Kendall Nelson 11 Jul '23

11 Jul '23

Hello Everyone! The next PTG[1] will take place October 23-27, 2023 virtually! Registration is now open[2] and free for all to attend. Keep an eye out in the coming weeks because I will send along the team sign up to collect groups interested in participating. Thanks! -Kendall Nelson(diablo_rojo) [1] https://openinfra.dev/ptg/ [2] http://ptg2023.openinfra.dev/

1 0

[tc] Technical Committee next weekly meeting on July 11, 2023
by Nikolla, Kristi 11 Jul '23

11 Jul '23

Hi all, This is a reminder that the next weekly Technical Committee meeting is to be held on Tuesday, July 11, 2023 at 1800 UTC on Zoom. Meeting link and agenda can be found at https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting Please find below the current agenda for the meeting. • Roll call • Follow up on past action items • knikolla will write alternative proposals for Extended Maintenance. • Extended Maintenance • https://etherpad.opendev.org/p/openstack-exteneded-maintenance • Gate health check • Open Discussion and Reviews • https://review.opendev.org/q/projects:openstack/governance+is:open Thank you, Kristi Nikolla

1 0

[cinder] Extending driver merge deadline to 14th July
by Rajat Dhasmana 10 Jul '23

10 Jul '23

Hi, As discussed in the last cinder meeting[1], due to less review bandwidth, Cinder drivers proposed for 2023.2 Bobcat didn't get much reviews. As a result, we will be extending the deadline to this week and the new date for driver merge deadline will be 14 July, 2023. Here is a list of all the drivers proposed for 2023.1 Bobcat cycle[2], feel free to review and add your comment even if you are not a core reviewer, all reviews are appreciated. [1] https://meetings.opendev.org/irclogs/%23openstack-meeting-alt/%23openstack-… [2] https://etherpad.opendev.org/p/cinder-2023-2-bobcat-drivers Thanks Rajat Dhasmana

1 0

[keystone] Weekly Meeting Move Proposal
by Dave Wilde 10 Jul '23

10 Jul '23

Hello, I'd like to propose moving the Keystone weekly meeting from Tuesdays at 15:00 UTC to Wednesdays at 15:00 UTC. Please let me know any feedback on this proposal, without objection our first Wednesday meeting will be 19-Jul-2023 at 15:00 UTC. Thank you! /Dave

1 0

[tacker][oslo][kolla-ansible] Issue reports for session management in oslo when using kolla-ansible
by Yuta Kazato 10 Jul '23

10 Jul '23

Hi, OpenStack Community. Our Tacker team found an issue with session management in oslo and sqlalchemy when using kolla-ansible. The following report details the issue. [Issue details] - In the case that we build Tacker and other components by using kolla-ansible, an issue with session management in oslo and sqlalchemy occurs when Tacker update the DB with `with context.session.begin(subtransactions=True):` - code: https://github.com/openstack/tacker/blob/7eaf841525d7ead30d7883abbde6c4fb86… - error details === * cli logs tacker@tacker:~/tacker-nic_internal/samples/takt-docker-sample/deployments/tacker$ curl -s -X PATCH http://"10.244.121.249:9890"/vnffm/v1/alarms/f23f39cc-f9a4-45bf-9a87-2f8148541b0a \ -H "Content-type: application/json" \ -H "Version: 1.3.0" \ -d "{\"ackState\": \"ACKNOWLEDGED\"}" \ -w "\n%{http_code}\n" {"status": 500, "detail": "this TransactionFactory is already started"} * tacker logs 2023-06-12 05:28:56.028 1 INFO tacker.service [-] Tacker service started, listening on 0.0.0.0:9890 2023-06-12 05:28:56.028 1 INFO tacker.wsgi [-] (1) wsgi starting up on http://0.0.0.0:9890 2023-06-12 05:30:07.021 1 DEBUG tacker.wsgi [-] (1) accepted ('10.100.2.3', 52508) server /usr/local/lib/python3.10/dist-packages/eventlet/wsgi.py:1004 2023-06-12 05:30:07.027 1 INFO tacker.sol_refactored.api.wsgi [-] PATCH http://10.244.121.249:9890/vnffm/v1/alarms/f23f39cc-f9a4-45bf-9a87-2f814854… 2023-06-12 05:30:07.052 1 DEBUG tacker.sol_refactored.common.coordinate [None req-a19d9e85-ba9f-4f6b-938a-3fde04bce62b - - - - - -] resources f23f39cc-f9a4-45bf-9a87-2f8148541b0a locked. wrapper /usr/local/lib/python3.10/dist-packages/tacker/sol_refactored/common/coordinate.py:95 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi [None req-a19d9e85-ba9f-4f6b-938a-3fde04bce62b - - - - - -] Unknown error: oslo_db.sqlalchemy.enginefacade.AlreadyStartedError: this TransactionFactory is already started 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi Traceback (most recent call last): 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi File "/usr/local/lib/python3.10/dist-packages/tacker/sol_refactored/api/wsgi.py", line 134, in __call__ 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi result = self._dispatch(request, action, args) 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi File "/usr/local/lib/python3.10/dist-packages/tacker/sol_refactored/api/wsgi.py", line 170, in _dispatch 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi return controller_method(request=request, **action_args) 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi File "/usr/local/lib/python3.10/dist-packages/tacker/sol_refactored/api/validator.py", line 48, in wrapper 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi return func(*args, **kwargs) 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi File "/usr/local/lib/python3.10/dist-packages/tacker/sol_refactored/common/coordinate.py", line 96, in wrapper 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi return func(*args, **kwargs) 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi File "/usr/local/lib/python3.10/dist-packages/tacker/sol_refactored/controller/vnffm_v1.py", line 101, in update 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi with context.session.begin(subtransactions=True): 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi File "/usr/local/lib/python3.10/dist-packages/tacker/context.py", line 183, in session 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi self._session = db_api.get_session() 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi File "/usr/local/lib/python3.10/dist-packages/tacker/db/api.py", line 46, in get_session 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi facade = _create_facade_lazily() 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi File "/usr/local/lib/python3.10/dist-packages/tacker/db/api.py", line 32, in _create_facade_lazily 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi context_manager.configure(sqlite_fk=True, **cfg.CONF.database) 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi File "/usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 812, in configure 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi self._factory.configure(**kw) 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi File "/usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 319, in configure 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi self._configure(False, kw) 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi File "/usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 332, in _configure 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi raise AlreadyStartedError( 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi oslo_db.sqlalchemy.enginefacade.AlreadyStartedError: this TransactionFactory is already started 2023-06-12 05:30:07.157 1 ERROR tacker.sol_refactored.api.wsgi 2023-06-12 05:30:07.159 1 INFO tacker.sol_refactored.api.wsgi [None req-a19d9e85-ba9f-4f6b-938a-3fde04bce62b - - - - - -] http://10.244.121.249:9890/vnffm/v1/alarms/f23f39cc-f9a4-45bf-9a87-2f814854… returned with HTTP 500 2023-06-12 05:30:07.159 1 INFO tacker.wsgi [None req-a19d9e85-ba9f-4f6b-938a-3fde04bce62b - - - - - -] 10.100.2.3 - - [12/Jun/2023 05:30:07] "PATCH /vnffm/v1/alarms/f23f39cc-f9a4-45bf-9a87-2f8148541b0a HTTP/1.1" 500 309 0.135927 === - This issue does not occur with a regular Tacker Deploy (manual deployment or devstack), but kola-ansible or container envs only. [Temporary solution] - Remove the codes, but we investigate to check if there are any other impacts. To solve these problems will require the cooperation of not only Tacker but also oslo and kolla-ansible projects. We invite your feedback on these problems and especially welcome your share of similar situation info and solutions. Best Regards, Yuta -- Yuta Kazato NTT Network Innovation Center. tel: +81-422-59-6754 mail: yuta.kazato(a)ntt.com

1 0

TripleO | OS-net config logs | openstack overcloud node provision
by Lokendra Rathour 10 Jul '23

10 Jul '23

Hi Team, Scenario: We were performing overcloud deployment. During this, we were continuously facing the following errors on two of the nodes out of the four nodes: PLAY [Overcloud Node Grow Volumes] ********************************************* 2023-06-02 19:07:33.483504 | 52540049-4965-0177-e681-00000000000a | TASK | Wait for provisioned nodes to boot [WARNING]: Unhandled error in Python interpreter discovery for host overcloud- controller-0: Failed to connect to the host via ssh: ssh: connect to host 30.30.30.219 port 22: No route to host [WARNING]: Unhandled error in Python interpreter discovery for host overcloud- controller-0: Failed to connect to the host via ssh: ssh: connect to host 30.30.30.219 port 22: Connection refused We faced this error when we were performing overcloud node provisioning. To resolve this issue, we had to update the image and create a user to access the nodes. We were able to access the nodes via this user through the BMC console. This implied that OS was getting correctly installed on the node. Once we accessed the logs on the failed node we came across the following logs: 2023-06-13 09:02:16.120 INFO os_net_config.utils._ordered_nics Finding active nics 2023-06-13 09:02:16.120 INFO os_net_config.utils._ordered_nics eno1 is an embedded active nic 2023-06-13 09:02:16.121 INFO os_net_config.utils._ordered_nics ens2f0 is an active nic 2023-06-13 09:02:16.121 INFO os_net_config.utils._ordered_nics eno4 is an embedded active nic 2023-06-13 09:02:16.121 INFO os_net_config.utils._ordered_nics eno2 is an embedded active nic 2023-06-13 09:02:16.121 INFO os_net_config.utils._ordered_nics ens1f0 is not an active nic 2023-06-13 09:02:16.121 INFO os_net_config.utils._ordered_nics lo is not an active nic 2023-06-13 09:02:16.121 INFO os_net_config.utils._ordered_nics ens2f1 is an active nic 2023-06-13 09:02:16.121 INFO os_net_config.utils._ordered_nics eno3 is an embedded active nic 2023-06-13 09:02:16.121 INFO os_net_config.utils._ordered_nics ens1f1 is an active nic 2023-06-13 09:02:16.121 INFO os_net_config.utils._ordered_nics No DPDK mapping available in path (/var/lib/os-net-config/dpdk_mapping.yaml) 2023-06-13 09:02:16.122 INFO os_net_config.utils._ordered_nics Active nics are ['eno1', 'eno2', 'eno3', 'eno4', 'ens1f1', 'ens2f0', 'ens2f1'] 2023-06-13 09:02:16.122 INFO os_net_config.objects.mapped_nics nic2 mapped to: eno2 2023-06-13 09:02:16.122 INFO os_net_config.objects.mapped_nics nic6 mapped to: ens2f0 2023-06-13 09:02:16.122 INFO os_net_config.objects.mapped_nics nic5 mapped to: ens1f1 2023-06-13 09:02:16.122 INFO os_net_config.objects.mapped_nics nic3 mapped to: eno3 2023-06-13 09:02:16.122 INFO os_net_config.objects.mapped_nics nic7 mapped to: ens2f1 2023-06-13 09:02:16.122 INFO os_net_config.objects.mapped_nics nic1 mapped to: eno1 2023-06-13 09:02:16.122 INFO os_net_config.objects.mapped_nics nic4 mapped to: eno4 As we can see in the above logs, only 7 nics are active. But as per our bond config file, we required 8 interfaces: a. nic-1 for management IP b. nic-3 and nic-4 for the br-tenant-2 network. c. nic-5 and nic-6 for the br-tenant network. d. nic-7 and nic 8 for the br-ex network. The wire was faulty in this case and hence only 7 interfaces were available Query: The logs which are visible while we run "openstack overcloud node provision" are not very informative. Is there any way where we could get these logs on the undercloud node itself? This could save some time for future references -- ~ Lokendra skype: lokendrarathour

2 1

[release] Release countdown for week R-12, Jul 10-14
by Herve Beraud 07 Jul '23

07 Jul '23

Development Focus ----------------- We are now past the Bobcat-2 milestone, and entering the last development phase of the cycle. Teams should be focused on implementing planned work for the cycle. Now is a good time to review those plans and reprioritize anything if needed based on the what progress has been made and what looks realistic to complete in the next few weeks. General Information ------------------- Looking ahead to the end of the release cycle, please be aware of the feature freeze dates. Those vary depending on deliverable type: * General libraries (except client libraries) need to have their last feature release before Non-client library freeze (August 24). Their stable branches are cut early. * Client libraries (think python-*client libraries) need to have their last feature release before Client library freeze (August 31) * Deliverables following a cycle-with-rc model (that would be most services) observe a Feature freeze on that same date, August 31. Any feature addition beyond that date should be discussed on the mailing-list and get PTL approval. After feature freeze, cycle-with-rc deliverables need to produce a first release candidate (and a stable branch) before RC1 deadline (September 14) * Deliverables following cycle-with-intermediary model can release as necessary, but in all cases before Final RC deadline (September 28) Upcoming Deadlines & Dates -------------------------- Non-client library freeze: August 24 (R-6 week) Client library freeze: August 31 (R-5 week) Bobcat-3 milestone: August 31 (R-5 week) Final 2023.2 Bobcat release: October 4th, 2023 -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

1 0

[neutron] Neutron drivers meeting cancelled
by Rodolfo Alonso Hernandez 07 Jul '23

07 Jul '23

Hello Neutrinos: Due to the lack of agenda [1], today's meeting is cancelled. Have a nice weekend! [1]https://wiki.openstack.org/wiki/Meetings/NeutronDrivers

1 0

[horizon] The state of the gate for the last 1 month at least
by Thomas Goirand 06 Jul '23

06 Jul '23

Hi team! It's been more than a month that it's impossible to merge anything in Horizon, due to the gate being broken because of one single functional test being broken: FAILED openstack_dashboard/test/integration_tests/tests/test_floatingips.py::TestFloatingipAssociateDisassociate::test_floatingip_associate_disassociate I do understand that you guys want to fix this, and I do understand how hard this puzzle is, with selenium, the chrome driver, jquery and all. It's far from simple, and I very much welcome the team effort. However, could we just disable that unit test until this is fixed, so that simple patches like this one can be merged? https://review.opendev.org/c/openstack/horizon/+/885577 Or more complex one from my colleague? https://review.opendev.org/c/openstack/horizon/+/885570 More than a month without merging any patch is a long time... Cheers, Thomas Goirand (zigo)

2 1

Node provision | stuck in black screen after kernel selection window | Wallaby
by Swogat Pradhan 06 Jul '23

06 Jul '23

Hi, I am trying to provision a new compute node in my central site using the director node that I have already used to deploy multiple sites compute, ceph along with central compute, ceph and controller nodes. For adding the new node, I introspected the node, setup capabilities then ran the node provision command. But it is failing at growfs step as the node is not reachable after booting. ERROR: [WARNING]: Unhandled error in Python interpreter discovery for host overcloud- novacompute-4: Failed to connect to the host via ssh: ssh: connect to host 172.25.201.93 port 22: No route to host 2023-07-06 01:28:38.370427 | 48d539a1-1679-0502-e4b9-000000000017 | UNREACHABLE | Find the growvols utility | overcloud-novacompute-4 After logging into the console of the bare metal server, I found that the OS is not coming UP. I see the bootloader screen, it selects the centos 1st kernel and then goes into black screen. I don't know why it is happening or how to fix it. I thought maybe there was some hardware issue, but i tried installing centos manually and i am getting the console output, without any issues. But for some reason the os is not booting up after the bootloader kernel window and the console is getting stuck in black screen. Can someone please help? With regards, Swogat Pradhan

1 0

[neutron] CI meeting on 11.07.2023 cancelled
by Slawek Kaplonski 05 Jul '23

05 Jul '23

Hi, I will be on PTO next week as well as some other members of the Neutron team. So we decided yesterday to cancel next week's CI meeting. See You on meeting on Tuesday 18th of July. -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[neutron] os-ken migration to Launchpad
by Rodolfo Alonso Hernandez 05 Jul '23

05 Jul '23

Hello Neutrinos: It was decided today in the Neutron meeting that we are moving from storyboard [1] to the Neutron launchpad site. If a new feature or bug is needed in the "os-ken" project, a new bug in [3] should be reported with the tag *[os-ken]* in the title. Regards. [1]https://storyboard.openstack.org/#!/project/1079 [2]https://launchpad.net/neutron [3]https://bugs.launchpad.net/neutron

3 2

[horizon][keystone][kolla-ansible] Authentication failure
by James Leong 05 Jul '23

05 Jul '23

Hi All, I am using the yoga version of OpenStack with the deployment tool of kolla-ansible. I am currently facing the below error when logging in via federated login using Globus Auth. " Login failed: An error occurred authenticating. Please try again later." When attempting to login, we are able to redirect the page to globus and process the request. However, when it comes back to the horizon login page, I am getting an authentication error. I have set up my keystone identity provider in globals.yml as below. keystone_identity_providers: - name: "globus" openstack_domain: "Default" protocol: "openid" identifier: "https://auth.globus.org" public_name: "Authenticate via Globus Auth" attribute_mapping: "globus" metadata_folder: "/home/user/osmetadata" keystone_federation_oidc_jwks_uri: "https://auth.globus.org/jwk.json" keystone_identity_mappings: - name: "globus" file: "/home/user/globus.json" Apart from specifying the identity provider and mapping, below are the other configurations we have set up when deploying. kolla_enable_tls_internal: "no" kolla_enable_tls_external: "yes" kolla_enable_tls_backend: "no" kolla_verify_tls_backend: "yes" Thanks for the help, James

2 1