- openstack-discuss - lists.openstack.org

[keystone] Weekly Meeting Cancelled (11-Apr-2023)
by Dave Wilde 11 Apr '23

11 Apr '23

Hi Folks, Unfortunately I have to cancel the weekly meeting today. Please reach out to me directly via email if you need anything. Due to equipment failure my IRC availability is spotty. Thanks! /Dave

1 0

[Kolla-ansible][Yoga] Cannot live migrate VMs
by wodel youchi 11 Apr '23

11 Apr '23

Hi, I cannot live-migrate any VM I have. All my compute nodes are enabled. I have a few VMs, I am in the testing phase. I tried to live-migrate toward an empty node, then toward a node which already had a VM in it. It is the same problem. In the log files I have : Source node : Source > ==> /var/log/kolla/nova/nova-compute.log <== > 2023-04-09 11:32:49.419 7 INFO nova.compute.resource_tracker > [req-4e91d562-d7ad-4585-a827-3db6277fd81e - - - - -] [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Updating resource usage from > migration f60eb077-8e1c-404a-aca5-64e5609ca02c > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [-] [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Pre live migration failed at > computehci02: oslo_messaging.rpc.client.RemoteError: Remote error: > ClientException Gateway Time-out (HTTP 504) > ['Traceback (most recent call last):\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", > line 165, in _process_incoming\n res = > self.dispatcher.dispatch(message)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", > line 309, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, > args)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", > line 229, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/exception_wrapper.py", > line 72, in wrapped\n context, exc, binary)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__\n self.force_reraise()\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise\n raise self.value\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/exception_wrapper.py", > line 63, in wrapped\n return f(self, context, *args, **kw)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/utils.py", > line 1439, in decorated_function\n return function(self, context, *args, > **kwargs)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > line 213, in decorated_function\n kwargs[\'instance\'], e, > sys.exc_info())\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__\n self.force_reraise()\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise\n raise self.value\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > line 201, in decorated_function\n return function(self, context, *args, > **kwargs)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > line 8367, in pre_live_migration\n bdm.save()\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__\n self.force_reraise()\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise\n raise self.value\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > line 8304, in pre_live_migration\n connector=connector, > mountpoint=bdm.device_name)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > line 395, in wrapper\n res = method(self, ctx, *args, **kwargs)\n', ' > File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > line 433, in wrapper\n res = method(self, ctx, volume_id, *args, > **kwargs)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > line 813, in attachment_create\n instance_uuid=instance_id)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__\n self.force_reraise()\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise\n raise self.value\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > line 801, in attachment_create\n volume_id, _connector, instance_id)\n', > ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/api_versions.py", > line 421, in substitution\n return method.func(obj, *args, **kwargs)\n', > ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/v3/attachments.py", > line 39, in create\n retval = self._create(\'/attachments\', body, > \'attachment\')\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/base.py", > line 306, in _create\n resp, body = self.api.client.post(url, > body=body)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > line 227, in post\n return self._cs_request(url, \'POST\', **kwargs)\n', > ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > line 215, in _cs_request\n return self.request(url, method, > **kwargs)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > line 201, in request\n raise exceptions.from_response(resp, body)\n', > 'cinderclient.exceptions.ClientException: Gateway Time-out (HTTP 504)\n']. > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Traceback (most recent call last): > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > line 8501, in _do_pre_live_migration_from_source > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] block_migration, disk, dest, > migrate_data) > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/rpcapi.py", > line 948, in pre_live_migration > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] **msg_args) > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", > line 192, in call > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] retry=self.retry, > transport_options=self.transport_options) > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", > line 128, in _send > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] > transport_options=transport_options) > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", > line 691, in send > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] > transport_options=transport_options) > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", > line 681, in _send > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] raise result > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] > oslo_messaging.rpc.client.RemoteError: Remote error: ClientException > Gateway Time-out (HTTP 504) > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] ['Traceback (most recent call > last):\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", > line 165, in _process_incoming\n res = > self.dispatcher.dispatch(message)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", > line 309, in dispatch\n return self._do_dispatch(endpoint, method, ctxt, > args)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", > line 229, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/exception_wrapper.py", > line 72, in wrapped\n context, exc, binary)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__\n self.force_reraise()\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise\n raise self.value\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/exception_wrapper.py", > line 63, in wrapped\n return f(self, context, *args, **kw)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/utils.py", > line 1439, in decorated_function\n return function(self, context, *args, > **kwargs)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > line 213, in decorated_function\n kwargs[\'instance\'], e, > sys.exc_info())\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__\n self.force_reraise()\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise\n raise self.value\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > line 201, in decorated_function\n return function(self, context, *args, > **kwargs)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > line 8367, in pre_live_migration\n bdm.save()\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__\n self.force_reraise()\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise\n raise self.value\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > line 8304, in pre_live_migration\n connector=connector, > mountpoint=bdm.device_name)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > line 395, in wrapper\n res = method(self, ctx, *args, **kwargs)\n', ' > File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > line 433, in wrapper\n res = method(self, ctx, volume_id, *args, > **kwargs)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > line 813, in attachment_create\n instance_uuid=instance_id)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__\n self.force_reraise()\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise\n raise self.value\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > line 801, in attachment_create\n volume_id, _connector, instance_id)\n', > ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/api_versions.py", > line 421, in substitution\n return method.func(obj, *args, **kwargs)\n', > ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/v3/attachments.py", > line 39, in create\n retval = self._create(\'/attachments\', body, > \'attachment\')\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/base.py", > line 306, in _create\n resp, body = self.api.client.post(url, > body=body)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > line 227, in post\n return self._cs_request(url, \'POST\', **kwargs)\n', > ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > line 215, in _cs_request\n return self.request(url, method, > **kwargs)\n', ' File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > line 201, in request\n raise exceptions.from_response(resp, body)\n', > 'cinderclient.exceptions.ClientException: Gateway Time-out (HTTP 504)\n']. > 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] > 2023-04-09 11:33:30.157 7 INFO nova.compute.manager [-] [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Swapping old allocation on > dict_keys(['2857ae62-3519-4b13-b700-858c827dd78d']) held by migration > f60eb077-8e1c-404a-aca5-64e5609ca02c for instance In the destination node : > nova-compute.log > > > > > *2023-04-09 11:32:30.052 7 WARNING os_brick.initiator.connectors.nvmeof > [req-69822759-d982-4d01-b448-232796077002 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - default default] Process execution error > in _get_host_uuid: Unexpected error while running command.Command: blkid > overlay -s UUID -o valueExit code: 2Stdout: ''Stderr: '': > oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while > running command.* > > 2023-04-09 11:33:05.354 7 INFO nova.compute.resource_tracker > [req-9d480442-87f7-4d4e-a866-c46d645ac7f2 - - - - -] [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Updating resource usage from > migration f60eb077-8e1c-404a-aca5-64e5609ca02c > 2023-04-09 11:33:05.392 7 WARNING nova.compute.resource_tracker > [req-9d480442-87f7-4d4e-a866-c46d645ac7f2 - - - - -] Instance > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7 has been moved to another host > computehci01(computehci01). There are allocations remaining against the > source host that might need to be removed: {'resources': {'VCPU': 1, > 'MEMORY_MB': 2048}}. > 2023-04-09 11:33:30.086 7 ERROR nova.volume.cinder > [req-69822759-d982-4d01-b448-232796077002 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - default default] [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Create attachment failed for volume > 3d7bd0af-5e4c-48fc-a027-f6c9dd6280bb. Error: Gateway Time-out (HTTP 504) > Code: 504: cinderclient.exceptions.ClientException: Gateway Time-out (HTTP > 504) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > [req-69822759-d982-4d01-b448-232796077002 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - default default] Exception during > message handling: cinderclient.exceptions.ClientException: Gateway Time-out > (HTTP 504) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server Traceback (most > recent call last): > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", > line 165, in _process_incoming > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server res = > self.dispatcher.dispatch(message) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", > line 309, in dispatch > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > self._do_dispatch(endpoint, method, ctxt, args) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", > line 229, in _do_dispatch > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server result = > func(ctxt, **new_args) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/exception_wrapper.py", > line 72, in wrapped > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server context, > exc, binary) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__ > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > self.force_reraise() > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server raise > self.value > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/exception_wrapper.py", > line 63, in wrapped > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > f(self, context, *args, **kw) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/utils.py", > line 1439, in decorated_function > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > function(self, context, *args, **kwargs) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > line 213, in decorated_function > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > kwargs['instance'], e, sys.exc_info()) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__ > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > self.force_reraise() > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server raise > self.value > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > line 201, in decorated_function > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > function(self, context, *args, **kwargs) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > line 8367, in pre_live_migration > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server bdm.save() > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__ > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > self.force_reraise() > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server raise > self.value > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > line 8304, in pre_live_migration > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > connector=connector, mountpoint=bdm.device_name) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > line 395, in wrapper > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server res = > method(self, ctx, *args, **kwargs) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > line 433, in wrapper > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server res = > method(self, ctx, volume_id, *args, **kwargs) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > line 813, in attachment_create > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > instance_uuid=instance_id) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__ > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > self.force_reraise() > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server raise > self.value > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > line 801, in attachment_create > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server volume_id, > _connector, instance_id) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/api_versions.py", > line 421, in substitution > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > method.func(obj, *args, **kwargs) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/v3/attachments.py", > line 39, in create > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server retval = > self._create('/attachments', body, 'attachment') > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/base.py", > line 306, in _create > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server resp, body = > self.api.client.post(url, body=body) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > line 227, in post > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > self._cs_request(url, 'POST', **kwargs) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > line 215, in _cs_request > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > self.request(url, method, **kwargs) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > line 201, in request > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server raise > exceptions.from_response(resp, body) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > cinderclient.exceptions.ClientException: Gateway Time-out (HTTP 504) > 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > 2023-04-09 11:33:30.238 7 INFO nova.virt.block_device > [req-69822759-d982-4d01-b448-232796077002 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - default default] [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Attempting to driver detach volume > 3d7bd0af-5e4c-48fc-a027-f6c9dd6280bb from mountpoint /dev/vda > 2023-04-09 11:33:30.239 7 WARNING nova.virt.block_device > [req-69822759-d982-4d01-b448-232796077002 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - default default] [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Detaching volume from unknown instance > 2023-04-09 11:33:30.240 7 WARNING nova.virt.libvirt.driver > [req-69822759-d982-4d01-b448-232796077002 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - default default] [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] During detach_volume, instance > disappeared.: nova.exception.InstanceNotFound: Instance > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7 could not be found. > 2023-04-09 11:33:30.678 7 INFO nova.virt.libvirt.driver [-] [instance: > f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Instance destroyed successfully. > 2023-04-09 11:33:30.682 7 INFO os_vif > [req-69822759-d982-4d01-b448-232796077002 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - default default] Successfully unplugged > vif > VIFBridge(active=True,address=fa:16:3e:9b:92:46,bridge_name='qbr46651303-65',has_traffic_filtering=True,id=46651303-65e1-4809-95cc-7c2d97acf16b,network=Network(532470c1-dd2f-4505-8557-ba24e9d4062b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap46651303-65') > All I can see is *b0f76b5c6dcb457fa716762bbf954837 - default default] Process execution error in _get_host_uuid: Unexpected error while running command.Command: blkid overlay -s UUID -o valueExit code: 2Stdout: ''Stderr: '': oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.* Searching the web I didn't find anything useful about this error, and I don't know if it's the origin of the problem. My default storage is ceph rbd, I have an HCI deployment, and all my VMs are for now created on Ceph only. But I have configured two other cinder backends, NFS and LVM, but they are node used for now. Thanks in advance. Regards.

3 4

[neutron] bug deputy report for week of 2023-04-03
by Bence Romsics 11 Apr '23

11 Apr '23

Hi All, We had the following new bugs reported last week: Critical: * https://bugs.launchpad.net/neutron/+bug/2015065 [tempest] "test_security_group_rules_create" unstable in "neutron-ovs-grenade-dvr-multinode" unassigned High: * https://bugs.launchpad.net/neutron/+bug/2015364 [skip-level] OVN tests constantly failing unassigned * https://bugs.launchpad.net/neutron/+bug/2015376 [OVN] min-BW rule, defined in the LSP.options field, must be accompanied with a max-BW value unassigned * https://bugs.launchpad.net/neutron/+bug/2015728 ovs/ovn source(with master branch) deployments broken workarounds proposed/merged by Yatin: https://review.opendev.org/c/openstack/neutron/+/879938 https://review.opendev.org/c/openstack/ovn-octavia-provider/+/879939 * https://bugs.launchpad.net/neutron/+bug/2015090 neutron.agent.dhcp.agent TypeError: 'bool' object is not subscriptable Technically unassigned, but I believe Sahid may be working on it. Medium: * https://bugs.launchpad.net/neutron/+bug/2015388 Deleting network does not remove network namespace fix proposed by Brian: https://review.opendev.org/c/openstack/neutron/+/880006 * https://bugs.launchpad.net/neutron/+bug/2015377 [ovn] If dhcp port is deleted from neutron, it is never recreated fix proposed by Brian: https://review.opendev.org/c/openstack/neutron/+/879913 RFE: * https://bugs.launchpad.net/neutron/+bug/2015471 [RFE] Add ERSPAN for tap-as-a-service with OVS and OVN Triage still in progress: * https://bugs.launchpad.net/neutron/+bug/2015449 Remote security groups don't allow traffic from floating IPs This issue may be undocumented behavior, the following question could be discussed on the weekly meeting: https://meetings.opendev.org/irclogs/%23openstack-neutron/%23openstack-neut… * https://bugs.launchpad.net/neutron/+bug/2015275 Trunk port update_subport_bindings RPC call not processed by RPC workers Cheers, Bence (rubasov)

1 0

[openstackclient] openstack cli broken after update to Wallaby
by Eugen Block 11 Apr '23

11 Apr '23

Hi *, today I upgraded my virtual test environment from V to W when (Ubuntu 20.04) all of a sudden cli commands didn't work anymore with this stack trace: ---snip--- root@control01:~# openstack network agent list Traceback (most recent call last): File "/usr/bin/openstack", line 6, in <module> from openstackclient.shell import main File "/usr/lib/python3/dist-packages/openstackclient/shell.py", line 23, in <module> from osc_lib import shell File "/usr/lib/python3/dist-packages/osc_lib/shell.py", line 24, in <module> from cliff import app File "/usr/lib/python3/dist-packages/cliff/app.py", line 22, in <module> import cmd2 File "/usr/lib/python3/dist-packages/cmd2.py", line 585, in <module> _ = pyperclip.paste() File "/usr/lib/python3/dist-packages/pyperclip/__init__.py", line 667, in lazy_load_stub_paste copy, paste = determine_clipboard() File "/usr/lib/python3/dist-packages/pyperclip/__init__.py", line 558, in determine_clipboard return init_gi_clipboard() File "/usr/lib/python3/dist-packages/pyperclip/__init__.py", line 167, in init_gi_clipboard gi.require_version('Gtk', '3.0') File "/usr/lib/python3/dist-packages/gi/__init__.py", line 129, in require_version raise ValueError('Namespace %s not available' % namespace) ValueError: Namespace Gtk not available ---snip--- I found this bug [1] describing the same issue but there has been no progress. I posted my comments there as well. I found one way to get the openstack shell to work by installing libgtk-3-dev (I found a hint in a search engine). Apparently, python3-cmd2 requires python3-pyperclip which requires python3-gi and so on. Is this really the desired way? I didn't notice anything in the release notes (maybe I missed it). When comparing to a different environment (Victoria on baremetal) I see that libgtk-3 is installed there (not -dev though), but even with libgtk-3-0 the error message was still present. So the question is, which dependencies are missing where? It's not really obvious to me. Could this already be fixed in Xena? If it is fixed there I could do the double upgrade, of course, especially since Wallaby is already under extended maintenance. Any comments are appreciated. Thanks, Eugen [1] https://bugs.launchpad.net/ubuntu/+source/python-openstackclient/+bug/19456…

4 9

[ironic][nova][neutron][oslo][sdk][swift][charm] netifaces has been archived
by Takashi Kajinami 11 Apr '23

11 Apr '23

Hello, I recently (actually, this morning) noticed the netifaces library[1] was archived a while ago, because the maintenor could no longer continue Open Source contributions, unfortunately[2]. [1] https://github.com/al45tair/netifaces [2] https://github.com/al45tair/netifaces/issues/78 I quickly searched the repositories and found some projects which have netifaces in their requirements.txt . We might need to look for alternatives (or offer to maintain the project, which is always the best) before we face any problems caused by that unmaintained library. Thank you, Takashi Kajinami

4 4

StackOver Flow Error: 2103 ERROR oslo.messaging._drivers.impl_rabbit Connection failed: failed to resolve broker hostname
by Trey Fleishman 10 Apr '23

10 Apr '23

Hello all, Recently I tried installing Openstack via this <https://docs.openstack.org/install-guide/openstack-services.html> guide. Upon completion of all the steps including the dashboard step, I was greeted with this error inside the nova_schedular.log file, after I tried creating an instance. 2103 ERROR oslo.messaging._drivers.impl_rabbit Connection failed: failed to resolve broker hostname: OSError: failed to resolve broker hostname When I try to create an instance it hangs in the "scheduling" state and throws this error. I have not configured anything in the "oslo.messaging" section of the Nova config file. I believe that under the [default] section this is what's causing the issue, but the password and hostname are fine. transport_url = rabbit://openstack:BnE9LLfCtjdPvpEnZUnPytSyrfUxE8th@controller Thanks!

2 1

[tc] Technical Committee next weekly meeting on 2023 Apr 11 at 1800 UTC
by Nikolla, Kristi 10 Apr '23

10 Apr '23

Hi all, This is a reminder that the next weekly Technical Committee meeting is to be held tomorrow (April 11) at 1800 UTC on #openstack-tc on OFTC IRC Note: This is the first meeting with the new meeting time, please update your calendars accordingly moving forward. Items can be proposed by editing the wiki page at https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting At the end of today I will finalize the agenda and send out another email. Thank you, Kristi Nikolla

1 1

[kolla ansible] SRIOV compute deployment question
by Satish Patel 08 Apr '23

08 Apr '23

Folks, I have deployed SRIOV compute but i found it deployed sriov-agent and openvswitch-agent together. If I am not mistaken then we don't need openvswitch agent on sriov computes nodes so how do i tell kolla to not deploy openvswitch-agent on sriov compute nodes? Here is the output on the compute node. root@kolla-comp-sriov-1-1:~# docker ps --filter name=agent CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 92af9870ee5b docker-reg:4000/kolla/neutron-sriov-agent:zed "dumb-init --single-…" 32 hours ago Up 31 hours (healthy) neutron_sriov_agent 14aea45e8b47 docker-reg:4000/kolla/neutron-openvswitch-agent:zed "dumb-init --single-…" 32 hours ago Up 3 seconds (health: starting) neutron_openvswitch_agent Configuration: (venv-kolla) root@kolla-infra-1-1:/var/lib/nova# cat /etc/kolla/config/neutron/kolla-comp-sriov-1-1/sriov_agent.ini [sriov_nic] physical_device_mappings = physnet1:eno50 (venv-kolla) root@kolla-infra-1-1:/var/lib/nova# cat /etc/kolla/config/neutron/kolla-comp-sriov-1-1/ml2_conf.ini [ml2] mechanism_drivers = sriovnicswitch [ml2_type_vlan] network_vlan_ranges = physnet1:30:50

1 0

[all] Hold your rechecks
by Dmitriy Rabotyagov 07 Apr '23

07 Apr '23

Hi everyone, Due to the recent update of gnupg2 package for CentOS Stream 9 to the version 2.3.3-3, a lot of repositories, including BaseOS and NFV, are failing to pass GPG check [1]. This affects all latest images, including container and cloud ones. At the moment, images used in Zuul are not affected, but this might change with the next periodic rebuild unless the related BZ report is fixed. Explicit execution of `rpm --import` can be used as a temporary workaround. [1] https://bugzilla.redhat.com/show_bug.cgi?id=2184640

2 3

[kolla-ansible]ask about multi region deploymet
by Nguyễn Hữu Khôi 07 Apr '23

07 Apr '23

Hello. I have read about kolla ansible multi regions. Can i ask that do we need 2 deployment host? And can we use 3 controller each region? I searched on Internet, I see that controller just sit on region one only? Please advise. Thank you.

1 0

[outreachy][cinder] Unable to generate api-ref samples for snapshot transfer
by Toheeb Oyekola 07 Apr '23

07 Apr '23

Hi, I am trying to generate api-ref doc from template by running "tox -e snaphot_transfer" but I keep getting the same error pasted in the link https://paste.openstack.org/show/bFPbWKQAKRbIds4R0wWn and I have installed the missing dependency as shown below, anyone have an Idea what the issue could have been? Thanks for your time. [image: image.png]

2 1

答复: [ptl] Need PTL volunteer for OpenStack Sahara
by Jerry Zhou (周鑫勇) 06 Apr '23

06 Apr '23

Hi Gmann, I can lead this project. -----邮件原件----- 发件人: Ghanshyam Mann <gmann(a)ghanshyammann.com> 发送时间: 2023年3月23日 2:35 收件人: Qiujunting <qiujunting(a)inspur.com>; openstack-discuss <openstack-discuss(a)lists.openstack.org> 主题: [ptl] Need PTL volunteer for OpenStack Sahara Hi Qiu, I am reaching out to you as you were PTL for OpenStack Sahara project in the last cycle. There is no PTL candidate for the next cycle (2023.2), and it is on the leaderless project list. Please check if you or anyone you know would like to lead this project. - https://etherpad.opendev.org/p/2023.2-leaderless Also, if anyone else would like to help leading this project, this is time to let TC knows. -gmann

1 0

[kolla-ansible] updating procedure
by wodel youchi 06 Apr '23

06 Apr '23

Hi, This is what documentation says about updating kolla-ansible : "Inside a series, it is usually sufficient to just update the kolla-ansible package, rebuild (if needed) and pull the images, and run kolla-ansible deploy again. Please follow release notes to check if there are any issues to be aware of." So to summarize, the procedure is like this : 1 - update kolla-ansible 2 - Download the latest containers of the used version 3 - In my case publish them in the local registry 4 - launch kolla-ansible deploy My question : Can I do the update directly , especially on the compute nodes, or do I have to empty some of them by moving VMs and execute the update on the empty nodes only, then proceed with the same logic with the rest of the nodes? Regards.

3 3

Re: multi-region deployment within datacenter scenario
by Satish Patel 06 Apr '23

06 Apr '23

Hi Piotr, Basically we use a vlan base tenant so the network node has very less usage. All my vms talk directly to the physical network and go out via the edge router. Our VM churn rate is also low because it's a private cloud. I am thinking of creating 3x API nodes and 3x RabbitMQ/DB nodes (total 6 nodes control plane and hope it works fine). One big advantage of having multiple pods (deployments) so during upgrade or any issue we have backup to work. On Thu, Apr 6, 2023 at 3:42 PM Piotr Misiak <piotrmisiak1984(a)gmail.com> wrote: > Hi Satish, > > Nova is well designed to handle hundreds of nodes. I would take a look at > Neutron instead, because it doesn’t scale very well. It depends on how your > networking configuration and use cases look like. > > BR, > Piotr > > On Wed, 5 Apr 2023 at 17:46 Satish Patel <satish.txt(a)gmail.com> wrote: > >> Folks, >> >> This is a very broad question and could have many answers but still >> wanted to ask to see what people are thinking and doing. >> >> We have DC with multiple racks and around 600 nodes so we are planning to >> create 3 private openstack and each with 200 nodes. In other data centers >> we have multiple 200 nodes openstack clusters and life is good. In the new >> datacenter I am thinking of deploying the same way openstack but shared >> keystone between multiple openstack environments to have a single place for >> identification and I am going to call it 3 regions even it's in a single >> DC. Hope shared keystone is helpful/easier for doing terrafrom work and >> selecting region easily. >> >> Other option I was thinking about to deploy cellv2 technology and put all >> 600 nodes in single cloud but always worried for operation point of view. >> Now sure how many people running cellv2 successfully without pain. (Except >> CERN) >> >> If people running multi-region openstack cloud then please share some >> thought and idea to make life easier. Not looking for best solution but >> just clue to get idea to deploy and manage multiple openstack. >> >> Thanks in advance. >> S >> >> >>

1 0

multi-region deployment within datacenter scenario
by Satish Patel 06 Apr '23

06 Apr '23

Folks, This is a very broad question and could have many answers but still wanted to ask to see what people are thinking and doing. We have DC with multiple racks and around 600 nodes so we are planning to create 3 private openstack and each with 200 nodes. In other data centers we have multiple 200 nodes openstack clusters and life is good. In the new datacenter I am thinking of deploying the same way openstack but shared keystone between multiple openstack environments to have a single place for identification and I am going to call it 3 regions even it's in a single DC. Hope shared keystone is helpful/easier for doing terrafrom work and selecting region easily. Other option I was thinking about to deploy cellv2 technology and put all 600 nodes in single cloud but always worried for operation point of view. Now sure how many people running cellv2 successfully without pain. (Except CERN) If people running multi-region openstack cloud then please share some thought and idea to make life easier. Not looking for best solution but just clue to get idea to deploy and manage multiple openstack. Thanks in advance. S

3 7

[interop][ptg] Interop Bobcat vPTG summary
by Martin Kopec 06 Apr '23

06 Apr '23

Hi everyone, here is a summary of the discussions. The full agenda can be found here [1]. == Current pain points and challenges of interop testing == The current guidelines don't contain complex enough tests - apart from api tests we should also have more functional tests which mock users' interactions with the cloud - we should test as a real user would use the cloud. People could fake the refstack results - after a discussion we came to an agreement that those people would always find a way, there is no need for coming up with sophisticated ways to prevent it. The point of Interop is to provide a check for vendors/users to see where they stand with their product based on the agreed standards. Use of Interop's tooling is complicated - there have been reports that users struggle with refstack-client and tempest. There is an ongoing effort to create a refstack container which would contain all necessary tools and could be run with just one command. Passing the Interop's guidelines means interoperability on which level? Passing the guidelines should mean being in compliance with the beforehand community agreed upon standards. Getting vendors running the interop tests more often - we only came up with the strategies which are already being used, such as expirations and guideline versioning. == Future strategy == The future strategy could be summarized in 3 points: * discuss the possibility for using sdk-based and tobiko testing * better describe to users the expectations they should have for clouds which assert interoperability * look into expiration of older standards and require that compliant providers indicate to users which version of the standard they last tested/passed [1] https://etherpad.opendev.org/p/bobcat-ptg-interop Regards, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

2 1

[tacker][ptg] Tacker PTG Summary
by Yasufumi Ogawa 06 Apr '23

06 Apr '23

Hi team, Thank you for joining our three days vPTG sessions for Tacker bobcat. It was three days and we had about 20 attendees each day. You can find all the result of discussions in [1] and cross project session with Keystone, Ironic and Barbican for supporting external authz for OAuth2.0 [2]. Here is a summary of our vPTG. - We shared several issues of functional tests have been suffered us for unexpected failures through the latest cycles and agree to revise the tests. Current target include refactoring messed directory structure and useless logs, reducing too much timeouts or redefining test scenarios. - For docs, there is no instruction for supported versions of k8s, helm or prometheus introduced in Tacker in the latest releases. There is also something missing instructions or too old contents. We've agreed to make a plan to update the docs and complete TODOs in bobcat. - Enable tacker to get error messages of management driver as a requirement from operators. - Share the current status of secure RBAC support in Tacker and remained tasks mainly for implementing unit and functional tests and documentation. - We agreed to introduct a new feature for supporting external key manager for authn credentials with encryption for security and custom resources for the latest k8s cluster APIs. - Discussed a requirement for supporting terraform infra-driver for some usecases deloying telco systems on cloud-based infra, and roughly agreed to include the feature into Tacker although we need to have some more surveys for implementation details. - We've had several proposals for improving our dev scheme such as introducing type annotation, plantUML for docs or so. - Shared the current status of Pypi maintainer cleanup. [1] https://etherpad.opendev.org/p/tacker-bobcat-ptg [2] https://etherpad.opendev.org/p/bobcat-ptg-ext_oauth2_server

1 0

[tc] May 2023 OpenInfra Board Sync
by Jeremy Stanley 06 Apr '23

06 Apr '23

The Open Infrastructure Foundation Board of Directors is endeavoring to engage in regular check-ins with official OpenInfra projects. The goal is for a loosely structured discussion one-hour in length, involving members of the board and the OpenStack TC, along with other interested community members. This is not intended to be a formal presentation, and no materials need to be prepared in advance. I've started an Etherpad where participants can brainstorm potential topics of conversation, time-permitting: https://etherpad.opendev.org/p/2023-05-board-openstack-sync Immediately following the February 8 discussion, I proposed to schedule the next call for 20:00 UTC on Wednesday, May 10 and heard no objections, so I've attached a calendar file which can serve as a convenient schedule hold for this, in case anyone needs it. -- Jeremy Stanley

1 0

[nova][cinder] why volumes can only be attached and dettached for running instances
by kaqiu pi 06 Apr '23

06 Apr '23

hello, I can only attached and dettached volumes for running instances in openstack. I would like to ask the reason for this design. And is there any danger if this restriction is turned on through custom modification in a private scene? Thank you so much. Yours

3 2

(no subject)
by Richard Buregeya 06 Apr '23

06 Apr '23

Hello Team, Hope you are doing well. Any one with an idea if there is a modification in COA exam, I heard it is now a multiple question exams instead of practical exam, Anyone with current information, kindly update me. Thanks a lot! Regards

3 6

[nova][cinder] future of rebuild without reimaging
by Bence Romsics 06 Apr '23

06 Apr '23

Hi All! We have users who use 'rebuild' on volume booted servers before nova microversion 2.93, relying on the behavior that it keeps the volume as is. And they would like to keep doing this even after the openstack distro moves to a(n at least) zed base (sometime in the future). As a naive user, it seems to me both behaviors make sense. I can easily imagine use cases for rebuild with and without reimaging. However since the implementation of https://specs.openstack.org/openstack/nova-specs/specs/zed/implemented/volu… rebuild without reimaging is only possible using an old microversion (<2.93). With that change merged, rebuild without reimaging seems to be a somewhat less than fully supported feature. A few examples of what I mean by that: First, there's this warning: https://opendev.org/openstack/python-openstackclient/src/commit/5eb89e4ca1c… In which it is unclear to me what exactly will become an error in a future release. Rebuild with a different image? Or any rebuild with microversion <2.93? Then old nova microversions may get dropped. Though what I heard from nova folks, this is unlikely to happen. Then there are a few hypothetical situations like: a) Rebuild gets a new api feature (in a new microversion) which can never be combined with the do-not-reimage behavior. b) Rebuild may have a bug, whose fix requires a microversion bump. This again can never be combined with the old behavior. What do you think, are these concerns purely theoretical or real? If we would like to keep having rebuild without reimaging, can we rely on the old microversion indefinitely? Alternatively shall we propose and implement a nova spec to explicitly expose the choice in the rebuild api (just to express the idea: osc server rebuild --reimage|--no-reimage)? If the topic is worth further discussion beyond the ML, I can also bring it to the nova ptg. Thanks in advance, Bence Romsics (rubasov) ps: I'll be afk for a few days, but I'll follow up next Tuesday.

7 16

[qa][ptg] QA Bobcat vPTG summary
by Martin Kopec 06 Apr '23

06 Apr '23

Hi everyone, here is a summary of the discussions. The full agenda can be found here [1]. == Antelope Retrospective == Good things: kudos to all who are still around and help with reviews, patches, bug fixes Bad things: unstable gate due to tox update, timeouts, job flakiness (mainly multinode job) == uWSGI is in the maintenance mode == This doesn't require an immediate action as it's still in the maintenance mode = any bugs should be fixed and until we need new features we don't have to rush into replacing it. We should have it on the watchlist though and potentially test some alternatives - I've put this as one of the priority items for this cycle [2]. == FIPS interop testing == We had a FIPS goal since the last PTG [3]. The patches [4] and [5] were to have FIPS on ubuntu and these [6] and [7] are to add FIPS on a centos job. The next goal is to have a FIPS job on Rocky linux as it's more stable than centos. == Retiring Patrole == We are going to retire Patrole project. 2 main facts that speak in favor of retiring: Fact 1: plugins have enough RBAC tests (and are writing more to test SRBAC) - therefore patrole is not needed, the testing coverage got by the plugins is enough. Fact 2: patrole gates have been broken for a long time and no one was affected by that. == Additional PyPi maintainer cleanup == Only 2 projects from QA left with additional maintainers: * openstack/os-performance-tools - additional maintainer SpamapS (Clint Byrum) * openstack/patrole - additional maintainer DavidPurcell If you know any of them, please, tell them to reach out to me. I've sent emails to them, but maybe the email addresses I found aren't checked regularly anymore. == Tempest/Devstack gate stability == Several precautions have been done such as reduce mysql memory [8] or reduce the number of tests executed per job, increase of a timeout etc [9]. However, those precautions are mainly workarounds. The important note is that the instability was only highlighted by our testing, not caused by it. Many problems are inefficiencies in openstack software, e.g.: * openstack-client issues a new token for every operation * python startup time - entrypoint lookup * privesp processes in aggregate use more memory than nova (every service runs their own copy) * inefficient ansible code, etc Any volunteers who would like to first analyse the situation, find which of those inefficiencies are the biggest culprits and how big effect they have on the system? [1] https://etherpad.opendev.org/p/qa-bobcat-ptg [2] https://etherpad.opendev.org/p/qa-bobcat-priority [3] https://etherpad.opendev.org/p/antelope-ptg-interop [4] https://review.opendev.org/c/zuul/zuul-jobs/+/866881 [5] https://review.opendev.org/c/zuul/zuul-jobs/+/873893 [6] https://review.opendev.org/c/openstack/devstack/+/871606 [7] https://review.opendev.org/c/openstack/tempest/+/873697 [8] https://review.opendev.org/c/openstack/devstack/+/873646 [9] https://review.opendev.org/q/topic:bug%252F2004780 Regards, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 0

[cinder] 2023.2 Bobcat PTG Summary
by Rajat Dhasmana 06 Apr '23

06 Apr '23

Hello, Please find the summary of 2023.2 Bobcat PTG in the wiki page[1]. It includes recordings, summary and action items for each topic. More details can be found in the PTG etherpad[2]. I would recommend everyone to please look for the action items assigned to you to plan it according to the cycle deadlines. [1] https://wiki.openstack.org/wiki/CinderBobcatPTGSummary [2] https://etherpad.opendev.org/p/bobcat-ptg-cinder Thanks Rajat Dhasmana

1 0

[cinder]multi backend SAN and Ceph
by Nguyễn Hữu Khôi 06 Apr '23

06 Apr '23

Hello. I have a question that could we use both SAN and Ceph for multi backend? If yes can we change volume type from SAN to Ceph and otherwise? Thanks.

3 3

[Glance] Weekly Meeting Cancelled
by Pranali Deore 05 Apr '23

05 Apr '23

Hi All, Canceling today's glance weekly meeting, I'm on sick leave today. See you next week ! Thanks, Pranali D

1 0