April 2021 - openstack-discuss - lists.openstack.org

[openstack-helm] Meeting cancelled
by Gage Hugo 06 Apr '21

06 Apr '21

Hey team, Since there are no agenda items [0] for the IRC meeting today April 6th, the meeting is cancelled. Our next IRC meeting will be April 13th. Thanks [0] https://etherpad.opendev.org/p/openstack-helm-weekly-meeting

1 0

[all][elections][tc] TC Vacancy Special Election Nominations end this week
by Jeremy Stanley 05 Apr '21

05 Apr '21

Just a reminder, nominations for one vacant OpenStack TC (Technical Committee) position are only open for three more days, until Apr 08, 2021 23:45 UTC. All nominations must be submitted as a text file to the openstack/election repository as explained at https://governance.openstack.org/election/#how-to-submit-a-candidacy Please make sure to follow the candidacy file naming convention: candidates/xena/<Project_Name or "TC">/<email_address> (for example, "candidates/xena/TC/stacker(a)example.org"). The name of the file should match an email address for your current OpenStack Foundation Individual Membership. Take this opportunity to ensure that your OSF member profile contains current information: https://www.openstack.org/profile/ Any OpenStack Foundation Individual Member can propose their candidacy for the vacant seat on the Technical Committee. This TC vacancy special election will be held from Apr 8, 2021 23:45 UTC through to Apr 15, 2021 23:45 UTC. The electorate for the TC election are the OpenStack Foundation Individual Members who have a code contribution to one of the official teams over the Victoria to Wallaby timeframe, Apr 24, 2020 00:00 UTC - Mar 08, 2021 00:00 UTC, as well as any Extra ATCs who are acknowledged by the TC. Note that the contribution qualifying period for this special election is being kept the same as what would have been used for the original TC election. The four already elected TC members for this term are listed as candidates in the special election, but will not appear on any resulting poll as they have already been officially elected. Only new candidates in addition to the four elected TC members for this term will appear on a subsequent poll for the TC vacancy special election. Please find below the timeline: nomination starts @ Mar 25, 2021 23:45 UTC nomination ends @ Apr 08, 2021 23:45 UTC elections start @ Apr 08, 2021 23:45 UTC elections end @ Apr 15, 2021 23:45 UTC Shortly after election officials approve candidates, they will be listed on the https://governance.openstack.org/election/ page. If you have any questions please be sure to either ask them on the mailing list or to the elections officials: https://governance.openstack.org/election/#election-officials -- Jeremy Stanley on behalf of the OpenStack Technical Elections Officials

1 0

[tripleo] centos-stream-8 and container-tools 3.0
by Wesley Hayutin 05 Apr '21

05 Apr '21

FYI.. The container-tools 3.0 module has been published by the centos team. We're seeing it land in: http://dashboard-ci.tripleo.org/d/jwDYSidGz/rpm-dependency-pipeline?viewPan… We will be moving ALL the upstream centos-stream-8 jobs to use container-tools 3.0 now. https://review.opendev.org/c/openstack/tripleo-quickstart/+/784770 https://review.opendev.org/c/openstack/tripleo-quickstart/+/784768 0/ happy monday

1 0

[TripleO][ussuri] undercloud install # fails on heat launch
by Ruslanas Gžibovskis 05 Apr '21

05 Apr '21

Hi all, I am trying to understand why undercloud install does not pass the heat step. I feel that it is related to undercloud.conf file, something wrong there? Or some special char which python do not want to understand? How to enable heat-launcher more verbose output? as last line is starting engine and that's it... VERY HELPFULLL!!! AND IT CANNOT CONNECT?! when I launch it manuallly it is running for long time... ?!>!???>?>?>?><>?!>?!!>!?!!>?!!?>!?! AAAAAA PAIN!!!!!! openstack undercloud install last lines: http://paste.openstack.org/show/RCUVMaHa4XwXwRM6IS67/ undercloud.conf: https://raw.githubusercontent.com/qw3r3wq/OSP-ussuri/master/undercloud-v3/u… heat.log: http://paste.openstack.org/show/C5yP3iO8VDk7ugLOHTo6/ -- Ruslanas Gžibovskis +370 6030 7030

2 4

can't build a instance successfully
by 朝阳未烈 04 Apr '21

04 Apr '21

Folks,   I am deploying OpenStack manually and have completed minimal development of the Ussuri. My controller node can find my compute node and confirm there are compute hosts in the database with the instruction:   ” openstack compute service list --service nova-compute”   But when I want to create an instance on the compute node, the status of the compute node just remains “build”. And I try to look for faults from the “/var/log/nova/nova-compute.log” of the compute node:   2021-04-03 00:59:43.379 1432 INFO os_vif [req-2ece8c1c-a96f-4d91-b704-5598c1166016 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Successfully unplugged vif VIFBridge(active=True,address=fa:16:3e:55:a5:65,bridge_name='brq3169e77c-99',has_traffic_filtering=True,id=28248ef5-6ad6-44bf-b2ce-3fa7ac2371ef,network=Network(3169e77c-9945-454f-9562-6e9a55e1adce),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tap28248ef5-6a') 2021-04-03 00:59:43.380 1432 INFO nova.virt.libvirt.driver [req-2ece8c1c-a96f-4d91-b704-5598c1166016 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] [instance: 3de9dac7-6704-4df9-bd81-cfdadf3ba59a] Deletion of /var/lib/nova/instances/3de9dac7-6704-4df9-bd81-cfdadf3ba59a_del complete 2021-04-03 00:59:43.426 1432 INFO nova.compute.manager [req-2ece8c1c-a96f-4d91-b704-5598c1166016 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] [instance: 3de9dac7-6704-4df9-bd81-cfdadf3ba59a] Took 0.06 seconds to destroy the instance on the hypervisor. 2021-04-03 00:59:43.980 1432 INFO nova.compute.manager [-] [instance: 3de9dac7-6704-4df9-bd81-cfdadf3ba59a] Took 0.55 seconds to deallocate network for instance. 2021-04-03 00:59:44.139 1432 INFO nova.scheduler.client.report [req-2ece8c1c-a96f-4d91-b704-5598c1166016 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Deleted allocation for instance 3de9dac7-6704-4df9-bd81-cfdadf3ba59a 2021-04-03 01:00:00.763 1432 INFO nova.compute.claims [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] [instance: 0f53309f-2196-4215-873b-ba05fb99de57] Claim successful on node compute1 2021-04-03 01:00:00.991 1432 INFO nova.virt.libvirt.driver [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] [instance: 0f53309f-2196-4215-873b-ba05fb99de57] Creating image 2021-04-03 01:00:01.421 1432 INFO oslo.privsep.daemon [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/usr/share/nova/nova-dist.conf', '--config-file', '/etc/nova/nova.conf', '--privsep_context', 'nova.privsep.sys_admin_pctxt', '--privsep_sock_path', '/tmp/tmpqsoglfcp/privsep.sock'] 2021-04-03 01:00:02.056 1432 INFO oslo.privsep.daemon [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Spawned new privsep daemon via rootwrap 2021-04-03 01:00:01.986 1861 INFO oslo.privsep.daemon [-] privsep daemon starting 2021-04-03 01:00:01.989 1861 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 2021-04-03 01:00:01.991 1861 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/none 2021-04-03 01:00:01.991 1861 INFO oslo.privsep.daemon [-] privsep daemon running as pid 1861 2021-04-03 01:00:02.716 1432 INFO oslo.privsep.daemon [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/usr/share/nova/nova-dist.conf', '--config-file', '/etc/nova/nova.conf', '--privsep_context', 'vif_plug_linux_bridge.privsep.vif_plug', '--privsep_sock_path', '/tmp/tmp50wp4hs_/privsep.sock'] 2021-04-03 01:00:03.404 1432 INFO oslo.privsep.daemon [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Spawned new privsep daemon via rootwrap 2021-04-03 01:00:03.324 1890 INFO oslo.privsep.daemon [-] privsep daemon starting 2021-04-03 01:00:03.326 1890 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 2021-04-03 01:00:03.328 1890 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_NET_ADMIN/CAP_NET_ADMIN/none 2021-04-03 01:00:03.328 1890 INFO oslo.privsep.daemon [-] privsep daemon running as pid 1890 2021-04-03 01:00:03.620 1432 INFO os_vif [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] Successfully plugged vif VIFBridge(active=False,address=fa:16:3e:e6:b4:89,bridge_name='brq3169e77c-99',has_traffic_filtering=True,id=c18f70ac-31ac-40eb-90f0-72c4e559f712,network=Network(3169e77c-9945-454f-9562-6e9a55e1adce),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tapc18f70ac-31') 2021-04-03 01:00:04.177 1432 INFO nova.compute.manager [-] [instance: 0f53309f-2196-4215-873b-ba05fb99de57] VM 已开始 (Lifecycle Event) 2021-04-03 01:00:04.202 1432 INFO nova.compute.manager [req-f136de80-ad6d-497a-b190-4181d739c8ba - - - - -] [instance: 0f53309f-2196-4215-873b-ba05fb99de57] VM 已暂停 (Lifecycle Event) 2021-04-03 01:00:04.246 1432 INFO nova.compute.manager [req-f136de80-ad6d-497a-b190-4181d739c8ba - - - - -] [instance: 0f53309f-2196-4215-873b-ba05fb99de57] During sync_power_state the instance has a pending task (spawning). Skip. 2021-04-03 01:00:36.581 1432 INFO nova.compute.manager [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Updating bandwidth usage cache 2021-04-03 01:00:51.700 1432 ERROR oslo.messaging._drivers.impl_rabbit [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Connection failed: timed out (retrying in 2.0 seconds): socket.timeout: timed out 2021-04-03 01:01:08.908 1432 ERROR oslo.messaging._drivers.impl_rabbit [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Connection failed: timed out (retrying in 4.0 seconds): socket.timeout: timed out 2021-04-03 01:01:13.358 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [b8ded131-047e-4acb-a226-881596889c26] AMQP server on controller:5672 is unreachable: timed out. Trying again in 1 seconds.: socket.timeout: timed out 2021-04-03 01:01:27.960 1432 ERROR oslo.messaging._drivers.impl_rabbit [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Connection failed: timed out (retrying in 6.0 seconds): socket.timeout: timed out 2021-04-03 01:01:37.983 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [b8ded131-047e-4acb-a226-881596889c26] AMQP server on controller:5672 is unreachable: timed out. Trying again in 2 seconds.: socket.timeout: timed out 2021-04-03 01:01:55.240 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [b8ded131-047e-4acb-a226-881596889c26] AMQP server on controller:5672 is unreachable: timed out. Trying again in 4 seconds.: socket.timeout: timed out 2021-04-03 01:02:05.262 1432 ERROR oslo.messaging._drivers.impl_rabbit [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Connection failed: timed out (retrying in 8.0 seconds): socket.timeout: timed out 2021-04-03 01:02:05.262 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [3f1dbf14-730b-4f61-94b2-8674c243c457] AMQP server on controller:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed 2021-04-03 01:02:05.263 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [03922bfe-3f9c-41dd-8219-edd7bfe33e5f] AMQP server on controller:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed 2021-04-03 01:02:21.319 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [3f1dbf14-730b-4f61-94b2-8674c243c457] AMQP server on controller:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error> 2021-04-03 01:02:31.335 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [03922bfe-3f9c-41dd-8219-edd7bfe33e5f] AMQP server on controller:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error> 2021-04-03 01:02:41.351 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [b8ded131-047e-4acb-a226-881596889c26] AMQP server on controller:5672 is unreachable: timed out. Trying again in 6 seconds.: socket.timeout: timed out 2021-04-03 01:02:56.387 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [3f1dbf14-730b-4f61-94b2-8674c243c457] AMQP server on controller:5672 is unreachable: timed out. Trying again in 2 seconds.: socket.timeout: timed out 2021-04-03 01:03:06.406 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [03922bfe-3f9c-41dd-8219-edd7bfe33e5f] AMQP server on controller:5672 is unreachable: timed out. Trying again in 2 seconds.: socket.timeout: timed out 2021-04-03 01:03:16.417 1432 ERROR oslo.messaging._drivers.impl_rabbit [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Connection failed: timed out (retrying in 10.0 seconds): socket.timeout: timed out 2021-04-03 01:03:31.457 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [b8ded131-047e-4acb-a226-881596889c26] AMQP server on controller:5672 is unreachable: timed out. Trying again in 8 seconds.: socket.timeout: timed out 2021-04-03 01:03:41.475 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [3f1dbf14-730b-4f61-94b2-8674c243c457] AMQP server on controller:5672 is unreachable: timed out. Trying again in 4 seconds.: socket.timeout: timed out 2021-04-03 01:03:51.495 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [03922bfe-3f9c-41dd-8219-edd7bfe33e5f] AMQP server on controller:5672 is unreachable: timed out. Trying again in 4 seconds.: socket.timeout: timed out 2021-04-03 01:04:09.540 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [3f1dbf14-730b-4f61-94b2-8674c243c457] AMQP server on controller:5672 is unreachable: timed out. Trying again in 6 seconds.: socket.timeout: timed out 2021-04-03 01:04:19.558 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [03922bfe-3f9c-41dd-8219-edd7bfe33e5f] AMQP server on controller:5672 is unreachable: timed out. Trying again in 6 seconds.: socket.timeout: timed out 2021-04-03 01:04:29.565 1432 ERROR oslo.messaging._drivers.impl_rabbit [req-eb71729b-0263-4577-8248-d8c9f69387ca - - - - -] Connection failed: timed out (retrying in 12.0 seconds): socket.timeout: timed out 2021-04-03 01:04:39.570 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [b8ded131-047e-4acb-a226-881596889c26] AMQP server on controller:5672 is unreachable: timed out. Trying again in 10 seconds.: socket.timeout: timed out 2021-04-03 01:04:59.594 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [3f1dbf14-730b-4f61-94b2-8674c243c457] AMQP server on controller:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error> 2021-04-03 01:05:09.607 1432 ERROR oslo.messaging._drivers.impl_rabbit [-] [03922bfe-3f9c-41dd-8219-edd7bfe33e5f] AMQP server on controller:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error> 2021-04-03 01:05:09.610 1432 WARNING nova.virt.libvirt.driver [req-b1aed986-ee9a-46ee-8408-d65a9357f430 98049570d7a54e26b8af4eaec9e2eca2 8342df14fa614ad79a08e68f097e4487 - default default] [instance: 0f53309f-2196-4215-873b-ba05fb99de57] Timeout waiting for [('network-vif-plugged', 'c18f70ac-31ac-40eb-90f0-72c4e559f712')] for instance with vm_state building and task_state spawning.: eventlet.timeout.Timeout: 300 seconds   And then controller node couldn’t find the compute node I don’t know why these happen. I have checked the port of 5672 is open on controller node and Rabbitmq is running.  Please help me.

2 1

[Ironic][RFE] Enable support for espresso machines in Ironic
by Jacob Anders 04 Apr '21

04 Apr '21

Hi There, I was discussing this RFE with Julia and we decided it would be great to get some feedback on it from the wider community, ideally by the end of today. Apologies for the short notice. Here's the story: https://storyboard.openstack.org/#!/story/2008791 What are your thoughts on this? Please comment in the Story or just reply to thread. Thank you, Jacob

14 18

ML2/OVN DVR question
by Satish Patel 03 Apr '21

03 Apr '21

Folks, I have deployed openstack using ML2/OVN on 1 controller and 2 compute nodes so far everything is working fine, when i configured router it by default used L3HA and i can see active-backup router on both compute nodes. currently my all SNAT traffic going out using compute-1 I don't want bottleneck in network so i am looking for DVR deployment and after reading i found tenant VLAN doesn't support DVR https://bugzilla.redhat.com/show_bug.cgi?id=1704596 After doing more research i found that if i set manually external_mac using the following command then my vm using local compute node to send traffic in/out just like DVR instead of centralized design. root@os-infra-1-neutron-ovn-northd-container-24eea9c2:~# ovn-nbctl find NAT type=dnat_and_snat _uuid : 99bdd866-01ed-425d-853b-9362ae8572c9 external_ids : {"neutron:fip_external_mac"="fa:16:3e:2d:7e:fa", "neutron:fip_id"="025a912a-c0ee-4f36-98ad-8992bd825cfc", "neutron:fip_network_id"="9cccf39d-edba-4069-91ef-5f30afbb6604", "neutron:fip_port_id"="70ad361a-b42e-403b-a5c1-4ee39ddf5e31", "neutron:revision_number"="6", "neutron:router_name"=neutron-8af10b06-c8de-4166-9ab1-ca2f775b08a8} external_ip : "10.40.255.10" external_mac : [] logical_ip : "172.168.0.164" logical_port : "70ad361a-b42e-403b-a5c1-4ee39ddf5e31" options : {} type : dnat_and_snat _uuid : c438e7be-5ff4-472e-b053-8d6ed74cd4dc external_ids : {"neutron:fip_external_mac"="fa:16:3e:f5:9f:f0", "neutron:fip_id"="31e8cb44-0acd-453b-a4e6-39f6ab3a6da4", "neutron:fip_network_id"="9cccf39d-edba-4069-91ef-5f30afbb6604", "neutron:fip_port_id"="44a677c5-86ff-4b6b-a046-54e79f79c4cd", "neutron:revision_number"="2", "neutron:router_name"=neutron-8af10b06-c8de-4166-9ab1-ca2f775b08a8} external_ip : "10.40.255.5" external_mac : [] logical_ip : "172.168.0.67" logical_port : "44a677c5-86ff-4b6b-a046-54e79f79c4cd" options : {} type : dnat_and_snat This is how i set external mac from fip_external_mac"="fa:16:3e:2d:7e:fa" in above command. ovn-nbctl set NAT 99bdd866-01ed-425d-853b-9362ae8572c9 external_mac="fa\:16\:3e\:2d\:7e\:fa" How do i make this behavior default for every single VM, i don't want to do this manually to set the external mac address of each FIP?

1 1

launch VM on volume vs. image
by Tony Liu 03 Apr '21

03 Apr '21

Hi, With Ceph as the backend storage, launching a VM on volume takes much longer than launching on image. Why is that? Could anyone elaborate the high level workflow for those two cases? Thanks! Tony

3 9

[Requirements][all] Requirements branched, freeze lifted
by Matthew Thode 02 Apr '21

02 Apr '21

The requirements freeze is now lifted. If your project has not branched please be aware that master moves on (to xena). -- Matthew Thode

1 0

RE: [Interop][Refstack] No meeting this Friday meeting
by Kanevsky, Arkady 02 Apr '21

02 Apr '21

Team, No meeting this Friday. Happy Holiday! Will update Etherpad. Cheers, Arkady From: prakash RAMCHANDRAN <pramchan(a)yahoo.com> Sent: Thursday, April 1, 2021 1:12 PM To: Jimmy McArthur; Vida Haririan Cc: Ghanshyam Mann; Kanevsky, Arkady; openstack-discuss; Martin Kopec; Goutham Pacha Ravi Subject: Re: [Interop][Refstack] this Friday meeting [EXTERNAL EMAIL] Looks like we can skip this Friday call and sure Arkady - lets cancel it. If you have something urgent we can talk offline - Thanks Prakash On Thursday, April 1, 2021, 11:06:25 AM PDT, Vida Haririan <vhariria(a)redhat.com<mailto:vhariria@redhat.com>> wrote: Hi Arkady, Friday is a company holiday and I will be ooo. Thanks, Vida On Thu, Apr 1, 2021 at 11:10 AM Jimmy McArthur <jimmy(a)openstack.org<mailto:jimmy@openstack.org>> wrote: I forgot this is a holiday. Same on my side. Thanks, Jimmy > On Apr 1, 2021, at 9:25 AM, Ghanshyam Mann <gmann(a)ghanshyammann.com<mailto:gmann@ghanshyammann.com>> wrote: > > > ---- On Thu, 01 Apr 2021 09:15:21 -0500 Kanevsky, Arkady <Arkady.Kanevsky(a)dell.com<mailto:Arkady.Kanevsky@dell.com>> wrote ---- >> >> Team, >> This Friday is Good Friday and some people have a day off. >> Should we cancel this week meeting? >> Please, respond so we can see if we will have quorum. > > Thanks Arkady, > > I will be off from work and would not be able to join. > > -gmann > >> Thanks, >> Arkady >> >> Arkady Kanevsky, Ph.D. >> SP Chief Technologist & DE >> Dell Technologies office of CTO >> Dell Inc. One Dell Way, MS PS2-91 >> Round Rock, TX 78682, USA >> Phone: 512 7204955 >> >> >

2 1

RE: Re: [kolla] Train Centos7 -> Centos8 upgrade fails on 2nd controller
by Braden, Albert 02 Apr '21

02 Apr '21

I opened a bug for this issue: https://bugs.launchpad.net/kolla-ansible/+bug/1922269 -----Original Message----- From: Braden, Albert Sent: Thursday, April 1, 2021 11:34 AM To: 'openstack-discuss(a)lists.openstack.org' <openstack-discuss(a)lists.openstack.org> Subject: RE: Re: [kolla] Train Centos7 -> Centos8 upgrade fails on 2nd controller Sorry that was a typo. Stopping RMQ during the removal of the *second* controller is what causes the problem. Is there a way to tell Centos 8 Train to use RMQ 3.7.24 instead of 3.7.28? -----Original Message----- From: Braden, Albert Sent: Thursday, April 1, 2021 9:34 AM To: 'openstack-discuss(a)lists.openstack.org' <openstack-discuss(a)lists.openstack.org> Subject: RE: Re: [kolla] Train Centos7 -> Centos8 upgrade fails on 2nd controller I did some experimenting and it looks like stopping RMQ during the removal of the first controller is what causes the problem. After deploying the first controller, stopping the RMQ container on any controller including the new centos8 controller will cause the entire cluster to stop. This crash dump appears on the controllers that stopped in sympathy: https://paste.ubuntu.com/p/ZDgFgKtQTB/ This appears in the RMQ log: https://paste.ubuntu.com/p/5D2Qjv3H8c/ -----Original Message----- From: Braden, Albert Sent: Wednesday, March 31, 2021 8:31 AM To: openstack-discuss(a)lists.openstack.org Subject: RE: Re: [kolla] Train Centos7 -> Centos8 upgrade fails on 2nd controller Centos7: {rabbit,"RabbitMQ","3.7.24"}, "Erlang/OTP 22 [erts-10.7.2.8] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:128] [hipe]\n"}, Centos8: {rabbit,"RabbitMQ","3.7.28"}, "Erlang/OTP 22 [erts-10.7.2.8] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:128] [hipe]\n"}, When I deploy the first Centos8 controller, RMQ comes up with all 3 nodes active and seems to be working fine until I shut down the 2nd controller. The only hint of trouble when I replace the 1st node is this error message the first time I run the deployment: https://paste.ubuntu.com/p/h9HWdfwmrK/ and the crash dump that appears on control2: crash dump log: https://paste.ubuntu.com/p/MpZ8SwTJ2T/ First 1500 lines of the dump: https://paste.ubuntu.com/p/xkCyp2B8j8/ If I wait for a few minutes then RMQ recovers on control2 and the 2nd run of the deployment seems to work, and there is no trouble until I shut down control1. -----Original Message----- From: Mark Goddard <mark(a)stackhpc.com> Sent: Wednesday, March 31, 2021 4:14 AM To: Braden, Albert <C-Albert.Braden(a)charter.com> Cc: openstack-discuss(a)lists.openstack.org Subject: [EXTERNAL] Re: [kolla] Train Centos7 -> Centos8 upgrade fails on 2nd controller CAUTION: The e-mail below is from an external source. Please exercise caution before opening attachments, clicking links, or following guidance. On Tue, 30 Mar 2021 at 13:41, Braden, Albert <C-Albert.Braden(a)charter.com> wrote: > > I’ve created a heat stack and installed Openstack Train to test the Centos7->8 upgrade following the document here: > > > > https://docs.openstack.org/kolla-ansible/train/user/centos8.html#migrating-… > > > > I used the instructions here to successfully remove and replace control0 with a Centos8 box > > > > https://docs.openstack.org/kolla-ansible/train/user/adding-and-removing-hos… > > > > After this my RMQ admin page shows all 3 nodes up, including the new control0. The name of the cluster is rabbit(a)chrnc-void-testupgrade-control-0.dev.chtrse.com > > > > (rabbitmq)[root@chrnc-void-testupgrade-control-2 /]# rabbitmqctl cluster_status > > Cluster status of node rabbit@chrnc-void-testupgrade-control-2 ... > > [{nodes,[{disc,['rabbit@chrnc-void-testupgrade-control-0', > > 'rabbit@chrnc-void-testupgrade-control-0-replace', > > 'rabbit@chrnc-void-testupgrade-control-1', > > 'rabbit@chrnc-void-testupgrade-control-2']}]}, > > {running_nodes,['rabbit@chrnc-void-testupgrade-control-0-replace', > > 'rabbit@chrnc-void-testupgrade-control-1', > > 'rabbit@chrnc-void-testupgrade-control-2']}, > > {cluster_name,<<"rabbit(a)chrnc-void-testupgrade-control-0.dev.chtrse.com">>}, > > {partitions,[]}, > > {alarms,[{'rabbit@chrnc-void-testupgrade-control-0-replace',[]}, > > {'rabbit@chrnc-void-testupgrade-control-1',[]}, > > {'rabbit@chrnc-void-testupgrade-control-2',[]}]}] > > > > After that I create a new VM to verify that the cluster is still working, and then perform the same procedure on control1. When I shut down services on control1, the ansible playbook finishes successfully: > > > > kolla-ansible -i ../multinode stop --yes-i-really-really-mean-it --limit control1 > > … > > control1 : ok=45 changed=22 unreachable=0 failed=0 skipped=105 rescued=0 ignored=0 > > > > After this my RMQ admin page stops responding. When I check RMQ on the new control0 and the existing control2, the container is still up but RMQ is not running: > > > > (rabbitmq)[root@chrnc-void-testupgrade-control-0-replace /]# rabbitmqctl cluster_status > > Error: this command requires the 'rabbit' app to be running on the target node. Start it with 'rabbitmqctl start_app'. > > > > If I start it on control0 and control2, then the cluster seems normal and the admin page starts working again, and cluster status looks normal: > > > > (rabbitmq)[root@chrnc-void-testupgrade-control-0-replace /]# rabbitmqctl cluster_status > > Cluster status of node rabbit@chrnc-void-testupgrade-control-0-replace ... > > [{nodes,[{disc,['rabbit@chrnc-void-testupgrade-control-0', > > 'rabbit@chrnc-void-testupgrade-control-0-replace', > > 'rabbit@chrnc-void-testupgrade-control-1', > > 'rabbit@chrnc-void-testupgrade-control-2']}]}, > > {running_nodes,['rabbit@chrnc-void-testupgrade-control-2', > > 'rabbit@chrnc-void-testupgrade-control-0-replace']}, > > {cluster_name,<<"rabbit(a)chrnc-void-testupgrade-control-0.dev.chtrse.com">>}, > > {partitions,[]}, > > {alarms,[{'rabbit@chrnc-void-testupgrade-control-2',[]}, > > {'rabbit@chrnc-void-testupgrade-control-0-replace',[]}]}] > > > > But my hypervisors are down: > > > > (openstack) [root@chrnc-void-testupgrade-build kolla-ansible]# ohll > > +----+-------------------------------------------------+-----------------+--------------+-------+------------+-------+----------------+-----------+ > > | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | vCPUs Used | vCPUs | Memory MB Used | Memory MB | > > +----+-------------------------------------------------+-----------------+--------------+-------+------------+-------+----------------+-----------+ > > | 3 | chrnc-void-testupgrade-compute-2.dev.chtrse.com | QEMU | 172.16.2.106 | down | 5 | 8 | 2560 | 30719 | > > | 6 | chrnc-void-testupgrade-compute-0.dev.chtrse.com | QEMU | 172.16.2.31 | down | 5 | 8 | 2560 | 30719 | > > | 9 | chrnc-void-testupgrade-compute-1.dev.chtrse.com | QEMU | 172.16.0.30 | down | 5 | 8 | 2560 | 30719 | > > +----+-------------------------------------------------+-----------------+--------------+-------+------------+-------+----------------+-----------+ > > > > When I look at the nova-compute.log on a compute node, I see RMQ failures every 10 seconds: > > > > 172.16.2.31 compute0 > > 2021-03-30 03:07:54.893 7 ERROR oslo.messaging._drivers.impl_rabbit [req-70d69b45-c3a7-4fbc-b709-4d7d757e09e7 - - - - -] [aeb317a8-873f-49be-a2a0-c6d6e0891a3e] AMQP server on 172.16.1.132:5672 is unreachable: timed out. Trying again in 1 seconds.: timeout: timed out > > 2021-03-30 03:07:55.905 7 INFO oslo.messaging._drivers.impl_rabbit [req-70d69b45-c3a7-4fbc-b709-4d7d757e09e7 - - - - -] [aeb317a8-873f-49be-a2a0-c6d6e0891a3e] Reconnected to AMQP server on 172.16.1.132:5672 via [amqp] client with port 56422. > > 2021-03-30 03:08:05.915 7 ERROR oslo.messaging._drivers.impl_rabbit [req-70d69b45-c3a7-4fbc-b709-4d7d757e09e7 - - - - -] [aeb317a8-873f-49be-a2a0-c6d6e0891a3e] AMQP server on 172.16.1.132:5672 is unreachable: timed out. Trying again in 1 seconds.: timeout: timed out > > > > In the RMQ logs I see this every 10 seconds: > > > > 172.16.1.132 control2 > > [root@chrnc-void-testupgrade-control-2 ~]# tail -f /var/log/kolla/rabbitmq/rabbit\(a)chrnc-void-testupgrade-control-2.log |grep 172.16.2.31 > > 2021-03-30 03:07:54.895 [warning] <0.13247.35> closing AMQP connection <0.13247.35> (172.16.2.31:56420 -> 172.16.1.132:5672 - nova-compute:7:aeb317a8-873f-49be-a2a0-c6d6e0891a3e, vhost: '/', user: 'openstack'): > > client unexpectedly closed TCP connection > > 2021-03-30 03:07:55.901 [info] <0.15288.35> accepting AMQP connection <0.15288.35> (172.16.2.31:56422 -> 172.16.1.132:5672) > > 2021-03-30 03:07:55.903 [info] <0.15288.35> Connection <0.15288.35> (172.16.2.31:56422 -> 172.16.1.132:5672) has a client-provided name: nova-compute:7:aeb317a8-873f-49be-a2a0-c6d6e0891a3e > > 2021-03-30 03:07:55.904 [info] <0.15288.35> connection <0.15288.35> (172.16.2.31:56422 -> 172.16.1.132:5672 - nova-compute:7:aeb317a8-873f-49be-a2a0-c6d6e0891a3e): user 'openstack' authenticated and granted access to vhost '/' > > 2021-03-30 03:08:05.916 [warning] <0.15288.35> closing AMQP connection <0.15288.35> (172.16.2.31:56422 -> 172.16.1.132:5672 - nova-compute:7:aeb317a8-873f-49be-a2a0-c6d6e0891a3e, vhost: '/', user: 'openstack'): > > > > Why does RMQ fail when I shut down the 2nd controller after successfully replacing the first one? Hi Albert, Could you share the versions of RabbitMQ and erlang in both versions of the container? When initially testing this setup, I think we had 3.7.24 on both sides. Perhaps the CentOS 8 version has moved on sufficiently to become incompatible? Mark > > > > I apologize for the nonsense below. I have not been able to stop it from being attached to my external emails. > > > > The contents of this e-mail message and > any attachments are intended solely for the > addressee(s) and may contain confidential > and/or legally privileged information. If you > are not the intended recipient of this message > or if this message has been addressed to you > in error, please immediately alert the sender > by reply e-mail and then delete this message > and any attachments. If you are not the > intended recipient, you are notified that > any use, dissemination, distribution, copying, > or storage of this message or any attachment > is strictly prohibited. E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

1 0

PTO Monday April 5
by Herve Beraud 02 Apr '21

02 Apr '21

Hello Monday is a public holiday in France. I'll be back Tuesday. Have a nice weekend! -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

1 1

[infra][puppet] No verified score posted by zuul
by Takashi Kajinami 02 Apr '21

02 Apr '21

Hi I have asked for some help in #openstack-infra but didn't get any solution so far Many people might be now on holidays (enjoy holidays!), so let me send this email so that people involved can find this mail after getting back. A few days ago Wallaby release of openstack puppet modules were created, and the release bot submitted release patches. However for some patches zuul doesn't return any CI result(it doesn't put verified score)[1]. I posted +2+A on [2] but it is not merged, because it is not verified by zuul. I tried "recheck" but it didn't solve the problem. [1] https://review.opendev.org/c/openstack/puppet-aodh/+/784213/1 [2] https://review.opendev.org/c/openstack/puppet-cloudkitty/+/784230 Currently we don't have any job triggered for the change with .gitreview and I guess that is why we don't get verified. Actually I see that the same patch for puppet-oslo got verified +1, because tripleo job was unexpectedly triggered for the change in .gitreview [2] https://review.opendev.org/c/openstack/puppet-oslo/+/784302 The easiest solution would be to manually squash these two patches into one. However I remember that we did get verified when we created the last Victoria release, and I suspect some change in infra side which resulted in this situation. So it would be nice if I can ask some insights from infra team about this situation. Thank you, Takashi Kajinami

1 1

[release] Release countdown for week R-1 Apr 05 - Apr 09
by Herve Beraud 01 Apr '21

01 Apr '21

Development Focus ----------------- We are on the final mile of the Wallaby development cycle! Remember that the Wallaby final release will include the latest release candidate (for cycle-with-rc deliverables) or the latest intermediary release (for cycle-with-intermediary deliverables) available. April 8 is the deadline for final Wallaby release candidates as well as any last cycle-with-intermediary deliverables. We will then enter a quiet period until we tag the final release on 14 April, 2021. Teams should be prioritizing fixing release-critical bugs, before that deadline. Otherwise it's time to start planning the Xena development cycle, including discussing Forum and PTG sessions content, in preparation of PTG on the week of April 19. Actions ------- Watch for any translation patches coming through on the stable/wallaby branch and merge them quickly. If you discover a release-critical issue, please make sure to fix it on the master branch first, then backport the bugfix to the stable/wallaby branch before triggering a new release. Please drop by #openstack-release with any questions or concerns about the upcoming release! Upcoming Deadlines & Dates -------------------------- Final Wallaby release: 14 April, 2021 Xena virtual PTG: 19 - 23 April, 2021 Thanks for your attention -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

1 0

[ironic] Review Jams
by Julia Kreger 01 Apr '21

01 Apr '21

In the Ironic team's recent mid-cycle call, we discussed the need to return to occasionally having review jams in order to help streamline the review process. In other words, get eyes on a change in parallel and be able to discuss the change. The goal is to help get people on the same page in terms of what and why. Be on hand to answer questions or back-fill context. This is to hopefully avoid the more iterative back and forth nature of code review, which can draw out a long chain of patches. As always, the goal is not perfection, but forward movement especially for complex changes. We've established two time windows that will hopefully not to be too hard for some contributors to make it to. It doesn't need to be everyone, but it would help for at least some people whom actively review or want to actively participate in reviewing, or whom are even interested in a feature to join us for our meeting. I've added an entry on to our wiki page to cover this, with the current agenda and anticipated review jam topic schedule. The tl;dr is we will use meetpad[1] and meet on Mondays at 2 PM UTC and Tuesdays at 6 PM UTC. The hope is to to enable some overlap of reviewers. If people are interested in other times, please bring this up in the weekly meeting or on the mailing list. I'm not sending out calendar invites for this. Yet. :) See everyone next week! -Julia [0]: https://wiki.openstack.org/wiki/Meetings/Ironic#Review_Jams [1]: https://meetpad.opendev.org/ironic

3 2

[oslo][release] devstack-plugin-(kafka|amqp1) retirement
by Herve Beraud 01 Apr '21

01 Apr '21

Hello Osloers, Our devstack plugins (kafka and amqp1) didn't show a great amount of activity since ussuri, does it still make sense to maintain them? The latest available SHAs for the both projects comes from Victoria (merged in this period). Can we retire them or simply retire them from the coordinated releases? We (the release team) would appreciate some feedback about this point. Let's open the debat. -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

5 10

[tripleo][ci] jobs in retry_limit or skipped
by Wesley Hayutin 01 Apr '21

01 Apr '21

Greetings, Just FYI.. I believe we hit a bump in the road in upstream infra ( not sure yet ). It appears to be global and not isolated to tripleo or centos based jobs. I have a tripleo bug to track it. https://bugs.launchpad.net/tripleo/+bug/1922148 See #opendev for details, it looks like infra is very busy working and fixing the issues atm. http://eavesdrop.openstack.org/irclogs/%23opendev/%23opendev.2021-03-31.log… http://eavesdrop.openstack.org/irclogs/%23opendev/latest.log.html

3 2

[ironic] IPA image does not want to boot with UEFI
by Vuk Gojnic 01 Apr '21

01 Apr '21

Hello everybody, I am using Ironic standalone to provision the HPE Gen10+ node via iLO driver. Ironic version is 16.0.1. Server is configured with UEFI boot mode. Everything on Ironic side works fine. It creates ISO image, powers the server on and configures it to boot from it. Here is the what /var/log/ironic/ironic-conductor.log says: 2021-03-31 17:46:25.541 2618460 INFO ironic.conductor.task_manager [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] Node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 moved to provision state "cleaning" from state "manageable"; target provision state is "available" 2021-03-31 17:46:32.066 2618460 INFO ironic.drivers.modules.ilo.power [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] The node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 operation of 'power off' is completed in 4 seconds. 2021-03-31 17:46:32.088 2618460 INFO ironic.conductor.utils [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] Successfully set node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 power state to power off by power off. 2021-03-31 17:46:34.510 2618460 INFO ironic.drivers.modules.ilo.common [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] Node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 pending boot mode is uefi. 2021-03-31 17:46:37.248 2618460 INFO ironic.drivers.modules.ilo.common [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] Set the node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 to boot from URL https://10.23.137.234/tmp-images/ilo/boot-ed25569f-c107-4fe0-95cd-74fcad9ab… successfully. 2021-03-31 17:46:48.367 2618460 INFO ironic.drivers.modules.ilo.power [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] The node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 operation of 'power on' is completed in 8 seconds. 2021-03-31 17:46:48.388 2618460 INFO ironic.conductor.utils [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] Successfully set node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 power state to power on by rebooting. 2021-03-31 17:46:48.404 2618460 INFO ironic.conductor.task_manager [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] Node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 moved to provision state "clean wait" from state "cleaning"; target provision state is "available" The Grub2 starts and after I select the option “boot_partition", it starts booting and immediately freezes showing just black screen with static red underscore character. I have tried with pre-built IPA images (see below) as well as with custom IPA images made with Ubuntu 18.04 and 20.04 (built using ironic-python-agent-builder) but it is all the same. Does somebody have idea what is the problem with IPA and UEFI in this particular scenario? Output of “openstack baremetal node show” command: allocation_uuid: null automated_clean: null bios_interface: no-bios boot_interface: ilo-uefi-https chassis_uuid: null clean_step: {} conductor: 10.23.137.234 conductor_group: '' console_enabled: false console_interface: no-console created_at: '2021-03-21T13:54:25+00:00' deploy_interface: direct deploy_step: {} description: null driver: ilo5 driver_info: ilo_address: 10.23.137.137 ilo_bootloader: https://ironic-images/Images/esp.img ilo_deploy_kernel: https://ironic-images/Images/ipa-centos8-stable-victoria.kernel ilo_deploy_ramdisk: https://ironic-images/Images/ipa-centos8-stable-victoria.initramfs ilo_password: '******' ilo_username: Administrator snmp_auth_priv_password: '******' snmp_auth_prot_password: '******' snmp_auth_user: iloinspect driver_internal_info: agent_continue_if_ata_erase_failed: false agent_enable_ata_secure_erase: true agent_erase_devices_iterations: 1 agent_erase_devices_zeroize: true agent_erase_skip_read_only: false agent_secret_token: '******' agent_secret_token_pregenerated: true clean_steps: null disk_erasure_concurrency: 1 last_power_state_change: '2021-03-31T17:46:37.894667' extra: {} fault: clean failure inspect_interface: ilo inspection_finished_at: '2021-03-21T13:57:33+00:00' inspection_started_at: null instance_info: deploy_boot_mode: uefi instance_uuid: null last_error: null lessee: null maintenance: true maintenance_reason: management_interface: ilo5 name: null network_data: {} network_interface: noop owner: null power_interface: ilo power_state: power on properties: cpu_arch: x86 cpus: 64 local_gb: 2979 memory_mb: 262144 protected: false protected_reason: null provision_state: clean wait provision_updated_at: '2021-03-31T17:46:48+00:00' raid_config: {} raid_interface: no-raid rescue_interface: no-rescue reservation: null resource_class: null retired: false retired_reason: null storage_interface: noop target_power_state: null target_provision_state: available target_raid_config: {} traits: [] updated_at: '2021-03-31T17:46:48+00:00' uuid: ed25569f-c107-4fe0-95cd-74fcad9ab3f0 vendor_interface: no-vendor Many thanks! Vuk Gojnic Deutsche Telekom Technik GmbH Services & Plattforms (T-SP) Tribe Data Center Infrastructure (T-DCI) Super Squad Cloud Platforms Lifecycle (SSQ-CP) Vuk Gojnic Kubernetes Engine Squad Lead

2 1

How to customize the xml used in libvirt from GUI/opestack command line?
by Evan Zhao 01 Apr '21

01 Apr '21

Hi there, I googled this question and found two major answers: 1. ssh to the compute node and use `virsh dumpxml` and `virsh undefine/define`, etc. 2. edit nova/virt/libvrit/config.py directly. However, it's trivial to ssh to each node and do the modification, and I prefer not to touch the nova source code, is there any better ways to achieve this? I expect to edit the namespace of a certain element and append an additional element <qemu:command> to the xml file. Any information will be appreciated.

2 1

RE: Re: [kolla] Train Centos7 -> Centos8 upgrade fails on 2nd controller
by Braden, Albert 01 Apr '21

01 Apr '21

I did some experimenting and it looks like stopping RMQ during the removal of the first controller is what causes the problem. After deploying the first controller, stopping the RMQ container on any controller including the new centos8 controller will cause the entire cluster to stop. This crash dump appears on the controllers that stopped in sympathy: https://paste.ubuntu.com/p/ZDgFgKtQTB/ This appears in the RMQ log: https://paste.ubuntu.com/p/5D2Qjv3H8c/ -----Original Message----- From: Braden, Albert Sent: Wednesday, March 31, 2021 8:31 AM To: openstack-discuss(a)lists.openstack.org Subject: RE: Re: [kolla] Train Centos7 -> Centos8 upgrade fails on 2nd controller Centos7: {rabbit,"RabbitMQ","3.7.24"}, "Erlang/OTP 22 [erts-10.7.2.8] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:128] [hipe]\n"}, Centos8: {rabbit,"RabbitMQ","3.7.28"}, "Erlang/OTP 22 [erts-10.7.2.8] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:128] [hipe]\n"}, When I deploy the first Centos8 controller, RMQ comes up with all 3 nodes active and seems to be working fine until I shut down the 2nd controller. The only hint of trouble when I replace the 1st node is this error message the first time I run the deployment: https://paste.ubuntu.com/p/h9HWdfwmrK/ and the crash dump that appears on control2: crash dump log: https://paste.ubuntu.com/p/MpZ8SwTJ2T/ First 1500 lines of the dump: https://paste.ubuntu.com/p/xkCyp2B8j8/ If I wait for a few minutes then RMQ recovers on control2 and the 2nd run of the deployment seems to work, and there is no trouble until I shut down control1. -----Original Message----- From: Mark Goddard <mark(a)stackhpc.com> Sent: Wednesday, March 31, 2021 4:14 AM To: Braden, Albert <C-Albert.Braden(a)charter.com> Cc: openstack-discuss(a)lists.openstack.org Subject: [EXTERNAL] Re: [kolla] Train Centos7 -> Centos8 upgrade fails on 2nd controller CAUTION: The e-mail below is from an external source. Please exercise caution before opening attachments, clicking links, or following guidance. On Tue, 30 Mar 2021 at 13:41, Braden, Albert <C-Albert.Braden(a)charter.com> wrote: > > I’ve created a heat stack and installed Openstack Train to test the Centos7->8 upgrade following the document here: > > > > https://docs.openstack.org/kolla-ansible/train/user/centos8.html#migrating-… > > > > I used the instructions here to successfully remove and replace control0 with a Centos8 box > > > > https://docs.openstack.org/kolla-ansible/train/user/adding-and-removing-hos… > > > > After this my RMQ admin page shows all 3 nodes up, including the new control0. The name of the cluster is rabbit(a)chrnc-void-testupgrade-control-0.dev.chtrse.com > > > > (rabbitmq)[root@chrnc-void-testupgrade-control-2 /]# rabbitmqctl cluster_status > > Cluster status of node rabbit@chrnc-void-testupgrade-control-2 ... > > [{nodes,[{disc,['rabbit@chrnc-void-testupgrade-control-0', > > 'rabbit@chrnc-void-testupgrade-control-0-replace', > > 'rabbit@chrnc-void-testupgrade-control-1', > > 'rabbit@chrnc-void-testupgrade-control-2']}]}, > > {running_nodes,['rabbit@chrnc-void-testupgrade-control-0-replace', > > 'rabbit@chrnc-void-testupgrade-control-1', > > 'rabbit@chrnc-void-testupgrade-control-2']}, > > {cluster_name,<<"rabbit(a)chrnc-void-testupgrade-control-0.dev.chtrse.com">>}, > > {partitions,[]}, > > {alarms,[{'rabbit@chrnc-void-testupgrade-control-0-replace',[]}, > > {'rabbit@chrnc-void-testupgrade-control-1',[]}, > > {'rabbit@chrnc-void-testupgrade-control-2',[]}]}] > > > > After that I create a new VM to verify that the cluster is still working, and then perform the same procedure on control1. When I shut down services on control1, the ansible playbook finishes successfully: > > > > kolla-ansible -i ../multinode stop --yes-i-really-really-mean-it --limit control1 > > … > > control1 : ok=45 changed=22 unreachable=0 failed=0 skipped=105 rescued=0 ignored=0 > > > > After this my RMQ admin page stops responding. When I check RMQ on the new control0 and the existing control2, the container is still up but RMQ is not running: > > > > (rabbitmq)[root@chrnc-void-testupgrade-control-0-replace /]# rabbitmqctl cluster_status > > Error: this command requires the 'rabbit' app to be running on the target node. Start it with 'rabbitmqctl start_app'. > > > > If I start it on control0 and control2, then the cluster seems normal and the admin page starts working again, and cluster status looks normal: > > > > (rabbitmq)[root@chrnc-void-testupgrade-control-0-replace /]# rabbitmqctl cluster_status > > Cluster status of node rabbit@chrnc-void-testupgrade-control-0-replace ... > > [{nodes,[{disc,['rabbit@chrnc-void-testupgrade-control-0', > > 'rabbit@chrnc-void-testupgrade-control-0-replace', > > 'rabbit@chrnc-void-testupgrade-control-1', > > 'rabbit@chrnc-void-testupgrade-control-2']}]}, > > {running_nodes,['rabbit@chrnc-void-testupgrade-control-2', > > 'rabbit@chrnc-void-testupgrade-control-0-replace']}, > > {cluster_name,<<"rabbit(a)chrnc-void-testupgrade-control-0.dev.chtrse.com">>}, > > {partitions,[]}, > > {alarms,[{'rabbit@chrnc-void-testupgrade-control-2',[]}, > > {'rabbit@chrnc-void-testupgrade-control-0-replace',[]}]}] > > > > But my hypervisors are down: > > > > (openstack) [root@chrnc-void-testupgrade-build kolla-ansible]# ohll > > +----+-------------------------------------------------+-----------------+--------------+-------+------------+-------+----------------+-----------+ > > | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | vCPUs Used | vCPUs | Memory MB Used | Memory MB | > > +----+-------------------------------------------------+-----------------+--------------+-------+------------+-------+----------------+-----------+ > > | 3 | chrnc-void-testupgrade-compute-2.dev.chtrse.com | QEMU | 172.16.2.106 | down | 5 | 8 | 2560 | 30719 | > > | 6 | chrnc-void-testupgrade-compute-0.dev.chtrse.com | QEMU | 172.16.2.31 | down | 5 | 8 | 2560 | 30719 | > > | 9 | chrnc-void-testupgrade-compute-1.dev.chtrse.com | QEMU | 172.16.0.30 | down | 5 | 8 | 2560 | 30719 | > > +----+-------------------------------------------------+-----------------+--------------+-------+------------+-------+----------------+-----------+ > > > > When I look at the nova-compute.log on a compute node, I see RMQ failures every 10 seconds: > > > > 172.16.2.31 compute0 > > 2021-03-30 03:07:54.893 7 ERROR oslo.messaging._drivers.impl_rabbit [req-70d69b45-c3a7-4fbc-b709-4d7d757e09e7 - - - - -] [aeb317a8-873f-49be-a2a0-c6d6e0891a3e] AMQP server on 172.16.1.132:5672 is unreachable: timed out. Trying again in 1 seconds.: timeout: timed out > > 2021-03-30 03:07:55.905 7 INFO oslo.messaging._drivers.impl_rabbit [req-70d69b45-c3a7-4fbc-b709-4d7d757e09e7 - - - - -] [aeb317a8-873f-49be-a2a0-c6d6e0891a3e] Reconnected to AMQP server on 172.16.1.132:5672 via [amqp] client with port 56422. > > 2021-03-30 03:08:05.915 7 ERROR oslo.messaging._drivers.impl_rabbit [req-70d69b45-c3a7-4fbc-b709-4d7d757e09e7 - - - - -] [aeb317a8-873f-49be-a2a0-c6d6e0891a3e] AMQP server on 172.16.1.132:5672 is unreachable: timed out. Trying again in 1 seconds.: timeout: timed out > > > > In the RMQ logs I see this every 10 seconds: > > > > 172.16.1.132 control2 > > [root@chrnc-void-testupgrade-control-2 ~]# tail -f /var/log/kolla/rabbitmq/rabbit\(a)chrnc-void-testupgrade-control-2.log |grep 172.16.2.31 > > 2021-03-30 03:07:54.895 [warning] <0.13247.35> closing AMQP connection <0.13247.35> (172.16.2.31:56420 -> 172.16.1.132:5672 - nova-compute:7:aeb317a8-873f-49be-a2a0-c6d6e0891a3e, vhost: '/', user: 'openstack'): > > client unexpectedly closed TCP connection > > 2021-03-30 03:07:55.901 [info] <0.15288.35> accepting AMQP connection <0.15288.35> (172.16.2.31:56422 -> 172.16.1.132:5672) > > 2021-03-30 03:07:55.903 [info] <0.15288.35> Connection <0.15288.35> (172.16.2.31:56422 -> 172.16.1.132:5672) has a client-provided name: nova-compute:7:aeb317a8-873f-49be-a2a0-c6d6e0891a3e > > 2021-03-30 03:07:55.904 [info] <0.15288.35> connection <0.15288.35> (172.16.2.31:56422 -> 172.16.1.132:5672 - nova-compute:7:aeb317a8-873f-49be-a2a0-c6d6e0891a3e): user 'openstack' authenticated and granted access to vhost '/' > > 2021-03-30 03:08:05.916 [warning] <0.15288.35> closing AMQP connection <0.15288.35> (172.16.2.31:56422 -> 172.16.1.132:5672 - nova-compute:7:aeb317a8-873f-49be-a2a0-c6d6e0891a3e, vhost: '/', user: 'openstack'): > > > > Why does RMQ fail when I shut down the 2nd controller after successfully replacing the first one? Hi Albert, Could you share the versions of RabbitMQ and erlang in both versions of the container? When initially testing this setup, I think we had 3.7.24 on both sides. Perhaps the CentOS 8 version has moved on sufficiently to become incompatible? Mark > > > > I apologize for the nonsense below. I have not been able to stop it from being attached to my external emails. > > > > The contents of this e-mail message and > any attachments are intended solely for the > addressee(s) and may contain confidential > and/or legally privileged information. If you > are not the intended recipient of this message > or if this message has been addressed to you > in error, please immediately alert the sender > by reply e-mail and then delete this message > and any attachments. If you are not the > intended recipient, you are notified that > any use, dissemination, distribution, copying, > or storage of this message or any attachment > is strictly prohibited. E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

1 0

[ironic] IPA does not want boot with UEFI
by Vuk.Gojnic＠telekom.de 01 Apr '21

01 Apr '21

Hello everybody, I am using Ironic standalone to provision the HPE Gen10+ node via iLO driver. Ironic version is 16.0.1. Server is configured with UEFI boot mode. Everything on Ironic side works fine. It creates ISO image, powers the server on and configures it to boot from it. Here is the what /var/log/ironic/ironic-conductor.log says: 2021-03-31 17:46:25.541 2618460 INFO ironic.conductor.task_manager [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] Node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 moved to provision state "cleaning" from state "manageable"; target provision state is "available" 2021-03-31 17:46:32.066 2618460 INFO ironic.drivers.modules.ilo.power [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] The node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 operation of 'power off' is completed in 4 seconds. 2021-03-31 17:46:32.088 2618460 INFO ironic.conductor.utils [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] Successfully set node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 power state to power off by power off. 2021-03-31 17:46:34.510 2618460 INFO ironic.drivers.modules.ilo.common [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] Node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 pending boot mode is uefi. 2021-03-31 17:46:37.248 2618460 INFO ironic.drivers.modules.ilo.common [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] Set the node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 to boot from URL https://10.23.137.234/tmp-images/ilo/boot-ed25569f-c107-4fe0-95cd-74fcad9ab… successfully. 2021-03-31 17:46:48.367 2618460 INFO ironic.drivers.modules.ilo.power [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] The node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 operation of 'power on' is completed in 8 seconds. 2021-03-31 17:46:48.388 2618460 INFO ironic.conductor.utils [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] Successfully set node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 power state to power on by rebooting. 2021-03-31 17:46:48.404 2618460 INFO ironic.conductor.task_manager [req-b6a19234-5f4b-4852-ab41-4102b9016bb0 - - - - -] Node ed25569f-c107-4fe0-95cd-74fcad9ab3f0 moved to provision state "clean wait" from state "cleaning"; target provision state is "available" The Grub2 starts and after I select the option "boot_partition", it starts booting and immediately freezes with following screen (just red static underscore): [cid:image002.jpg@01D726EE.A920A5C0] I have tried with pre-built IPA images (see below) as well as with custom IPA images made with Ubuntu 18.04 and 20.04 (built using ironic-python-agent-builder) but it is all the same. Does somebody have idea what is the problem with IPA and UEFI in this particular scenario? Output of "openstack baremetal node show" command: allocation_uuid: null automated_clean: null bios_interface: no-bios boot_interface: ilo-uefi-https chassis_uuid: null clean_step: {} conductor: 10.23.137.234 conductor_group: '' console_enabled: false console_interface: no-console created_at: '2021-03-21T13:54:25+00:00' deploy_interface: direct deploy_step: {} description: null driver: ilo5 driver_info: ilo_address: 10.23.137.137 ilo_bootloader: https://ironic-images/Images/esp.img ilo_deploy_kernel: https://ironic-images/Images/ipa-centos8-stable-victoria.kernel ilo_deploy_ramdisk: https://ironic-images/Images/ipa-centos8-stable-victoria.initramfs ilo_password: '******' ilo_username: Administrator snmp_auth_priv_password: '******' snmp_auth_prot_password: '******' snmp_auth_user: iloinspect driver_internal_info: agent_continue_if_ata_erase_failed: false agent_enable_ata_secure_erase: true agent_erase_devices_iterations: 1 agent_erase_devices_zeroize: true agent_erase_skip_read_only: false agent_secret_token: '******' agent_secret_token_pregenerated: true clean_steps: null disk_erasure_concurrency: 1 last_power_state_change: '2021-03-31T17:46:37.894667' extra: {} fault: clean failure inspect_interface: ilo inspection_finished_at: '2021-03-21T13:57:33+00:00' inspection_started_at: null instance_info: deploy_boot_mode: uefi instance_uuid: null last_error: null lessee: null maintenance: true maintenance_reason: management_interface: ilo5 name: null network_data: {} network_interface: noop owner: null power_interface: ilo power_state: power on properties: cpu_arch: x86 cpus: 64 local_gb: 2979 memory_mb: 262144 protected: false protected_reason: null provision_state: clean wait provision_updated_at: '2021-03-31T17:46:48+00:00' raid_config: {} raid_interface: no-raid rescue_interface: no-rescue reservation: null resource_class: null retired: false retired_reason: null storage_interface: noop target_power_state: null target_provision_state: available target_raid_config: {} traits: [] updated_at: '2021-03-31T17:46:48+00:00' uuid: ed25569f-c107-4fe0-95cd-74fcad9ab3f0 vendor_interface: no-vendor Many thanks! Vuk Gojnic Deutsche Telekom Technik GmbH Services & Plattforms (T-SP) Tribe Data Center Infrastructure (T-DCI) Super Squad Cloud Platforms Lifecycle (SSQ-CP) Vuk Gojnic Kubernetes Engine Squad Lead

1 0

Re: [glance][openstack-ansible] Snapshots disappear during saving
by Oliver Wenz 01 Apr '21

01 Apr '21

> So according to the issue, you get 503 while trying to reach > 10.0.3.212:6002/os-objects, which is swift_account_port. > Are there any logs specificly for swift-account? > > Also I guess some adjustments are required for swift as well for this > mechanism to work. > > Eventually I believe the original issue you saw might be related to this > doc: > https://docs.openstack.org/keystone/latest/admin/manage-services.html#confi… Hi Dmitriy, I tried setting 'service_token_roles_required = false' in glance-api.conf but the error is still there. I also checked the account server and I'm seeing lots of 404's: Apr 01 09:31:33 bc1bl12 account-server[694822]: 10.0.3.212 - - [01/Apr/2021:09:31:33 +0000] "GET /os-objects/22/.misplaced_objects" 404 - "GET http://localhost/v1/.misplaced_objects?format=json&marker=&end_marker=&pref…" "tx3794c20a6ef5476f9fcf6-00606592f5" "proxy-server 13882" 0.0007 "-" 694822 - Apr 01 09:32:03 bc1bl12 account-server[694816]: 10.0.3.212 - - [01/Apr/2021:09:32:03 +0000] "HEAD /os-objects/22/.misplaced_objects" 404 - "HEAD http://localhost/v1/.misplaced_objects?format=json" "tx5491e866bc6447d8917f9-0060659313" "proxy-server 13882" 0.0008 "-" 694816 - Apr 01 09:32:03 bc1bl12 account-server[694814]: 10.0.3.212 - - [01/Apr/2021:09:32:03 +0000] "GET /os-objects/22/.misplaced_objects" 404 - "GET http://localhost/v1/.misplaced_objects?format=json&marker=&end_marker=&pref…" "txc98a949e82c940d19656b-0060659313" "proxy-server 13882" 0.0007 "-" 694814 - Apr 01 09:32:33 bc1bl12 account-server[694817]: 10.0.3.212 - - [01/Apr/2021:09:32:33 +0000] "HEAD /os-objects/22/.misplaced_objects" 404 - "HEAD http://localhost/v1/.misplaced_objects?format=json" "txd9ec46414b8e4a98a2b94-0060659331" "proxy-server 13882" 0.0008 "-" 694817 - Apr 01 09:32:33 bc1bl12 account-server[694817]: 10.0.3.212 - - [01/Apr/2021:09:32:33 +0000] "GET /os-objects/22/.misplaced_objects" 404 - "GET http://localhost/v1/.misplaced_objects?format=json&marker=&end_marker=&pref…" "tx972d968e71e94eb683558-0060659331" "proxy-server 13882" 0.0011 "-" 694817 - Apr 01 09:32:43 bc1bl12 account-server[694823]: 10.0.3.212 - - [01/Apr/2021:09:32:43 +0000] "HEAD /os-objects/220/.expiring_objects" 404 - "HEAD http://localhost/v1/.expiring_objects?format=json" "tx4d02362089c4497982940-006065933b" "proxy-server 14171" 0.0008 "-" 694823 - Apr 01 09:33:03 bc1bl12 account-server[694822]: 10.0.3.212 - - [01/Apr/2021:09:33:03 +0000] "HEAD /os-objects/22/.misplaced_objects" 404 - "HEAD http://localhost/v1/.misplaced_objects?format=json" "tx440963ca3e7948ff872ab-006065934f" "proxy-server 13882" 0.0007 "-" 694822 - Apr 01 09:33:03 bc1bl12 account-server[694823]: 10.0.3.212 - - [01/Apr/2021:09:33:03 +0000] "GET /os-objects/22/.misplaced_objects" 404 - "GET http://localhost/v1/.misplaced_objects?format=json&marker=&end_marker=&pref…" "tx547157ac468e476395a0e-006065934f" "proxy-server 13882" 0.0006 "-" 694823 - Apr 01 09:33:33 bc1bl12 account-server[694814]: 10.0.3.212 - - [01/Apr/2021:09:33:33 +0000] "HEAD /os-objects/22/.misplaced_objects" 404 - "HEAD http://localhost/v1/.misplaced_objects?format=json" "txac5c0207e90a4606a7758-006065936d" "proxy-server 13882" 0.0007 "-" 694814 - Apr 01 09:33:33 bc1bl12 account-server[694820]: 10.0.3.212 - - [01/Apr/2021:09:33:33 +0000] "GET /os-objects/22/.misplaced_objects" 404 - "GET http://localhost/v1/.misplaced_objects?format=json&marker=&end_marker=&pref…" "tx0d3afe2f421647f2a117e-006065936d" "proxy-server 13882" 0.0006 "-" 694820 - Kind regards, Oliver

1 0

[cinder][nova][requirements] RFE requested for os-brick
by Brian Rosmaita 01 Apr '21

01 Apr '21

Hello Requirements Team, The Cinder team recently became aware of a potential data-loss bug [0] that has been fixed in os-brick master [1] and backported to os-brick stable/wallaby [2]. We've proposed a release of os-brick 4.4.0 from stable/wallaby [3] and are petitioning for an RFE to include 4.4.0 in the wallaby release. We have three jobs running tempest with os-brick source in master that have passed with [1]: os-brick-src-devstack-plugin-ceph [4], os-brick-src-tempest-lvm-lio-barbican [5],and os-brick-src-tempest-nfs [6]. The difference between os-brick master (at the time the tests were run) and stable/wallaby since the 4.3.0 tag is as follows: master: d4205bd 3 days ago iSCSI: Fix flushing after multipath cfg change (Gorka Eguileor) 0e63fe8 2 weeks ago Merge "RBD: catch read exceptions prior to modifying offset" (Zuul) 28545c7 4 months ago RBD: catch read exceptions prior to modifying offset (Jon Bernard) 99b2c60 2 weeks ago Merge "Dropping explicit unicode literal" (Zuul) 7cfdb76 6 weeks ago Dropping explicit unicode literal (tushargite96) 9afa1a0 3 weeks ago Add Python3 xena unit tests (OpenStack Release Bot) ab57392 3 weeks ago Update master for stable/wallaby (OpenStack Release Bot) 91a1cca 3 weeks ago (tag: 4.3.0) Merge "NVMeOF connector driver connection information compatibility fix" (Zuul) stable/wallaby: f86944b 3 days ago Add release note prelude for os-brick 4.4.0 (Brian Rosmaita) c70d70b 3 days ago iSCSI: Fix flushing after multipath cfg change (Gorka Eguileor) 6649b8d 3 weeks ago Update TOX_CONSTRAINTS_FILE for stable/wallaby (OpenStack Release Bot) f3f93dc 3 weeks ago Update .gitreview for stable/wallaby (OpenStack Release Bot) 91a1cca 3 weeks ago (tag: 4.3.0) Merge "NVMeOF connector driver connection information compatibility fix" (Zuul) This gives us very high confidence that the results of the tests run against master also apply to stable/wallaby at f86944b. Thank you for considering this request. (I've included Nova here because the bug occurs when the configuration option that enables multipath connections on a compute is changed while volumes are attached, so if this RFE is approved, nova might want to raise the minimum version of os-brick in wallaby to 4.4.0.) [0] https://launchpad.net/bugs/1921381 [1] https://review.opendev.org/c/openstack/os-brick/+/782992 [2] https://review.opendev.org/c/openstack/os-brick/+/783207 [3] https://review.opendev.org/c/openstack/releases/+/783641 [4] https://zuul.opendev.org/t/openstack/build/30a103668e4c4a8cb6f1ef907ef3edcb [5] https://zuul.opendev.org/t/openstack/build/bb11eef737d34c41bb4a52f8433850b0 [6] https://zuul.opendev.org/t/openstack/build/3ad3359ca712432d9ef4261d72c787fa

4 9

[neutron] Drivers team meeting 02.04.2021 cancelled
by Slawek Kaplonski 31 Mar '21

31 Mar '21

Hi, As tomorrow's Good Friday is public holiday in many countries, at least in Europe, and I will also be on PTO, let's cancel the drivers meeting. Have a great holidays and see You on the meeting next week. -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

Retiring the planet.openstack.org service
by Ian Wienand 31 Mar '21

31 Mar '21

Hello, We plan to retire the planet.openstack.org RSS aggregation service soon. The host is running an unsupported distribution, and we have not found any open source alternatives that seem to be currently maintained and deployable. On consideration of our limited infra resources, we feel it will be better to sunset this service at this time. It is likely that effort will be better spent getting information to more relevant channels in 2021, such as social media sites, etc. I have extracted an OPML file of the active blogs in [1]; most every feed reader can import this file. Thanks, -i [1] https://review.opendev.org/c/opendev/system-config/+/784191

1 0