- openstack-discuss - lists.openstack.org

[dev][glance_storage][s3] improving download speed from s3 backend
by Gökhan Koçak 22 Aug '23

22 Aug '23

Hello everyone, I did not find any option to use something like multipart downloads (only uploads) or concurrency when downloading from s3 storage. So my first question is if this is correct? Second, I see that boto3 is being used: https://opendev.org/openstack/glance_store/src/branch/master/glance_store/_… and I wonder if it is possible to implement using the concurrency settings from boto: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/customiza… ? Best, Gökhan

1 0

[nova] Slow nvme performance for local storage instances
by Jan Wasilewski 22 Aug '23

22 Aug '23

Hi, I am reaching out to inquire about the performance of our local storage setup. Currently, I am conducting tests using NVMe disks; however, the results appear to be underwhelming. In terms of my setup, I have recently incorporated two NVMe disks into my compute node. These disks have been configured as RAID1 under md127 and subsequently mounted at /var/lib/nova/instances [1]. During benchmarking using the fio tool within this directory, I am achieving approximately 160,000 IOPS [2]. This figure serves as a satisfactory baseline and reference point for upcoming VM tests. As the next phase, I have established a flavor that employs a root disk for my virtual machine [3]. Regrettably, the resulting performance yields around 18,000 IOPS, which is nearly ten times poorer than the compute node results [4]. While I expected some degradation, a tenfold decrease seems excessive. Realistically, I anticipated no more than a twofold reduction compared to the compute node's performance. Hence, I am led to ask: what should be configured to enhance performance? I have already experimented with the settings recommended on the Ceph page for image properties [5]; however, these changes did not yield the desired improvements. In addition, I attempted to modify the CPU architecture within the nova.conf file, switching to Cascade Lake architecture, yet this endeavor also proved ineffective. For your convenience, I have included a link to my current dumpxml results [6]. Your insights and guidance would be greatly appreciated. I am confident that there is a solution to this performance disparity that I may have overlooked. Thank you in advance for your help. /Jan Wasilewski *References:* *[1] nvme allocation and raid configuration: https://paste.openstack.org/show/bMMgGqu5I6LWuoQWV7TV/ <https://paste.openstack.org/show/bMMgGqu5I6LWuoQWV7TV/>* *[2] fio performance inside compute node: https://paste.openstack.org/show/bcMi4zG7QZwuJZX8nyct/ <https://paste.openstack.org/show/bcMi4zG7QZwuJZX8nyct/>* *[3] Flavor configuration: https://paste.openstack.org/show/b7o9hCKilmJI3qyXsP5u/ <https://paste.openstack.org/show/b7o9hCKilmJI3qyXsP5u/>* *[4] fio performance inside VM: https://paste.openstack.org/show/bUjqxfU4nEtSFqTlU8oH/ <https://paste.openstack.org/show/bUjqxfU4nEtSFqTlU8oH/>* *[5] image properties: https://docs.ceph.com/en/pacific/rbd/rbd-openstack/#image-properties <https://docs.ceph.com/en/pacific/rbd/rbd-openstack/#image-properties>* *[6] dumpxml of vm: https://paste.openstack.org/show/bRECcaSMqa8TlrPp0xrT/ <https://paste.openstack.org/show/bRECcaSMqa8TlrPp0xrT/>*

5 18

[sunbeam] Juju 3.2 stable release break
by James Page 22 Aug '23

22 Aug '23

Morning The Juju project popped a new stable release of the 3.2 series we currently use for Sunbeam; this has created a problem with the mysql-k8s charm that we rely on for Sunbeam deployment and as such any new deployments will fail. The team is focussed on resolving this ASAP as this is breaking all of our check/gates right now as well. Regards James

1 0

[elections][tc] Self-nominate for Technical Committee
by Dmitriy Rabotyagov 21 Aug '23

21 Aug '23

Hello everyone! I want to nominate myself for the second term on a Technical Committee. During the first term, I actively participated in various TC activities, including a change of the platform for PTI, better defining expectations on projects for removing support for older Python versions, and others. While it took me time to get up to speed and understand better how things are done, I feel being efficient since then as well as bringing value to the table during discussions. Apart from TC activities, I am currently a Core Reviewer and Project Team Leader of OpenStack-Ansible and Vitrage projects. I am a strong advocate for diversity and inclusion, which now is even more important than ever before when corporations are diverting from their commitments to open-source software. I believe that diverse and community-driven projects better represent the real needs of end users and operators. Throughout my career, I have been dealing mostly with Public Clouds, which have a way wider variety of use cases and perspectives than any single corporate solution can have. With that in mind, I want to continue supporting an inclusive environment for all community members as well as connecting operators and developers worlds during the next term of the Technical Committee member. Thanks for your consideration!

3 2

swift_objects are not getting saved in container
by Gaurav Oberai 21 Aug '23

21 Aug '23

Hi Team, We have a setup running with OpenStack Swift setup and a proxy server. We have also used s3api middleware in proxy-server configuration to store objects like AWS s3. Now we are facing issues like objects are not getting stored inside the container instead we are able to store the objects on storage nodes successfully and get a successful response for every request. We have verified the same by listing the objects using the aws-s3 command line utility. However, when tried with OpenStack swift command buckets and objects are not visible inside the container. My concern is how objects are saved if the container is not saving them and how can we fix this issue to store all objects inside the container? Could you please help me with this? Thanks & Regards, Gaurav

2 1

[barbican][cinder][cloudkitty][freezer][heat][ironic][keystone][mistral][monasca][murano][nova][OpenStackSDK][oslo][sahara][Telemetry][zaqar][zun] broken gate on master branch
by Elõd Illés 21 Aug '23

21 Aug '23

Hi $teams_in_subject, I've generated test patches to see gate health (to projects' for cycle-with-intermediary deliverables, that have not merged any patches recently) and the patches show [1] that you have broken CI jobs on your gate. Could the teams please check what the issue / issues are and fix as soon as possible? Thanks in advance! (Note: multiple times I see that the failure in docs job is: Handler <function html_page_context at 0x7f86178f55a0> for event 'html-page-context' threw an exception (exception: object of type 'PosixPath' has no len()) so several gates can be fixed with a fix to this issue) [1] https://review.opendev.org/q/topic:release-health-check-cwi-bobcat+is:open Thanks, Előd Illés irc: elodilles @ #openstack-release

3 3

Re: Kayobe VLAN & bond configuration
by Stefan Pinter 21 Aug '23

21 Aug '23

Hi again! After rebooting the server an cleaning up everything in /etc/systemd/network/ it does work again (kayobe overcloud host configure) but I am afraid to run “kayobe overcloud service deploy”. Can anyone confirm that the configuration might be correct or has an idea about what happened in the network there? Multicast storm? Broadcast storm? Troubles with STP? Thank you and best wishes Stefan ________________________________ BearingPoint GmbH Sitz: Wien Firmenbuchgericht: Handelsgericht Wien Firmenbuchnummer: FN 175524z The information in this email is confidential and may be legally privileged. If you are not the intended recipient of this message, any review, disclosure, copying, distribution, retention, or any action taken or omitted to be taken in reliance on it is prohibited and may be unlawful. If you are not the intended recipient, please reply to or forward a copy of this message to the sender and delete the message, any attachments, and any copies thereof from your system.

3 4

[nova-api] Inject public key without create key pair in nova ?
by Pureewat Kaewpoi 21 Aug '23

21 Aug '23

Hi OpenStack Community We have some requirements that need to create instance via API. However nova-api is accept only key_name of key pair created earlier. Can we create instance by passing only public key data without creating a key pair in nova ? Thank you.

4 6

[glance][kolla] importing images on filesystem without copy?
by Rob Jefferson 21 Aug '23

21 Aug '23

Folks, I'm trying to wrap my brain around whether this is possible. Suppose I've configured Glance to use a filesystem backend. Would I be able to point Glance at a directory of existing images, and have it import the images into OpenStack (thus populating the database) without having to copy them? It seems fairly redundant to have the images on the filesystem, and to make copies as part of the creation process. Looking around in the documentation, it seems that this may not be possible, but I'm not sure if there's an alternate path or if what I'm asking is ill-posed. Any advice would be useful. Thanks, Rob

3 2

[charms] Nominate Martin Kalcok for charms-core
by Frode Nordahl 21 Aug '23

21 Aug '23

Hello all, I would like to nominate Martin Kalcok for charms-core. He has been part of the extended charms teams for some time now, and has provided many contributions to OpenStack, Ceph and OVN charms [0]. He is diligent and thorough in his contributions, and has recently become a full time member of the focused OVN team. 0: https://review.opendev.org/q/owner:martin.kalcok%2540canonical.com -- Frode Nordahl

5 4

Unable to Live/Cold migrate instances that have ceph root and secondary netapp storage devices | Wallaby
by Swogat Pradhan 21 Aug '23

21 Aug '23

Hi, I have an instance that has disk /dev/vda present in ceph and the rest of the 6 volumes (10 TB) each on netapp. I am trying to migrate the VM to another host but it is failing to migrate with error volume not found like below: 2023-08-21 09:22:31.852 7 WARNING os_brick.initiator.connectors.iscsi [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] Couldn't find iSCSI nodes because iscsiadm err: iscsiadm: No records found : os_brick.exception.VolumeDeviceNotFound: Volume device not found at . 2023-08-21 09:22:31.866 7 WARNING os_brick.initiator.connectors.iscsi [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. : os_brick.exception.VolumeDeviceNotFound: Volume device not found at . 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] Exception during message handling: os_brick.exception.VolumeDeviceNotFound: Volume device not found at . 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 72, in wrapped 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server context, exc, binary) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server self.force_reraise() 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise self.value 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 63, in wrapped 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/utils.py", line 1437, in decorated_function 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 212, in decorated_function 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server self.force_reraise() 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise self.value 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 200, in decorated_function 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 8288, in pre_live_migration 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server bdm.save() 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server self.force_reraise() 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise self.value 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 8249, in pre_live_migration 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server migrate_data) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 10343, in pre_live_migration 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server self._connect_volume(context, connection_info, instance) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 1808, in _connect_volume 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server vol_driver.connect_volume(connection_info, instance) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/volume/iscsi.py", line 64, in connect_volume 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server device_info = self.connector.connect_volume(connection_info['data']) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/utils.py", line 154, in trace_logging_wrapper 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 391, in inner 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 524, in connect_volume 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server self._cleanup_connection(connection_properties, force=True) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server self.force_reraise() 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise self.value 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 518, in connect_volume 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return self._connect_single_volume(connection_properties) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/utils.py", line 78, in _wrapper 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return r(f, *args, **kwargs) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 409, in call 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server do = self.iter(retry_state=retry_state) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 368, in iter 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise retry_exc.reraise() 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 186, in reraise 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise self.last_attempt.result() 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server return self.__get_result() 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise self._exception 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 412, in call 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server result = fn(*args, **kwargs) 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 597, in _connect_single_volume 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server raise exception.VolumeDeviceNotFound(device='') 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server os_brick.exception.VolumeDeviceNotFound: Volume device not found at . 2023-08-21 09:22:34.348 7 ERROR oslo_messaging.rpc.server 2023-08-21 09:22:34.678 7 INFO nova.virt.block_device [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] Attempting to driver detach volume a005a015-0bb2-498e-9068-ac14570a5f2e from mountpoint /dev/vda 2023-08-21 09:22:34.680 7 WARNING nova.virt.block_device [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] Detaching volume from unknown instance 2023-08-21 09:22:34.682 7 WARNING nova.virt.libvirt.driver [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] During detach_volume, instance disappeared.: nova.exception.InstanceNotFound: Instance 5520dbd7-8f12-4041-bd8c-65b4b55b0452 could not be found. 2023-08-21 09:22:34.707 7 INFO nova.virt.block_device [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] Attempting to driver detach volume 01b9b430-9cfd-43b1-b674-b6fb830ee511 from mountpoint /dev/vdb 2023-08-21 09:22:34.708 7 WARNING nova.virt.block_device [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] Detaching volume from unknown instance 2023-08-21 09:22:34.710 7 WARNING nova.virt.libvirt.driver [req-6ed9d181-fa42-4a4a-b55b-2d08352d4e22 9f20ac539d60ff27487ce83f160e3c29bbeea1017d28fb5159b17c9e77454345 265571cce0b54251b17ace88630b5e46 - 64989c3036764834a755ff2bd55b30b3 64989c3036764834a755ff2bd55b30b3] [instance: 5520dbd7-8f12-4041-bd8c-65b4b55b0452] During detach_volume, instance disappeared.: nova.exception.InstanceNotFound: Instance 5520dbd7-8f12-4041-bd8c-65b4b55b0452 could not be found. Server show : +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ | OS-DCF:diskConfig | AUTO | | OS-EXT-AZ:availability_zone | Watson | | OS-EXT-SRV-ATTR:host | overcloud-novacompute-0.bdxworld.com | | OS-EXT-SRV-ATTR:hostname | hkg2-cctv-archive-server-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | overcloud-novacompute-0.bdxworld.com | | OS-EXT-SRV-ATTR:instance_name | instance-0000012f | | OS-EXT-SRV-ATTR:kernel_id | | | OS-EXT-SRV-ATTR:launch_index | 0 | | OS-EXT-SRV-ATTR:ramdisk_id | | | OS-EXT-SRV-ATTR:reservation_id | r-mdrfy13i | | OS-EXT-SRV-ATTR:root_device_name | /dev/vda | | OS-EXT-SRV-ATTR:user_data | None | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2023-01-05T09:49:54.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | HKG2-CCTV=192.168.240.14 | | config_drive | True | | created | 2023-01-05T09:49:46Z | | description | HKG2-CCTV-Archive-server-2 | | flavor | disk='150', ephemeral='0', extra_specs.hw:cpu_cores='10', extra_specs.hw:cpu_sockets='2', original_name='CCTV-VM-Flavor', ram='16384', swap='0', | | | vcpus='10' | | hostId | 570b5ef66c44d3f80a1a498c9dbd8b9a2b690a1aee748bc9cb2cf6fb | | host_status | UP | | id | 5520dbd7-8f12-4041-bd8c-65b4b55b0452 | | image | N/A (booted from volume) | | key_name | None | | locked | False | | locked_reason | None | | name | HKG2-CCTV-Archive-server-2 | | progress | 0 | | project_id | 265571cce0b54251b17ace88630b5e46 | | properties | | | security_groups | name='CCTV services Incoming' | | | name='CCTV services Outgoing' | | server_groups | [] | | status | ACTIVE | | tags | | | trusted_image_certificates | None | | updated | 2023-08-21T09:22:35Z | | user_id | ff69985eea81fe6f11e9caa3f242623cbddc7ce21cdd864c9585d793533acb9e | | volumes_attached | delete_on_termination='False', id='a005a015-0bb2-498e-9068-ac14570a5f2e' | | | delete_on_termination='False', id='01b9b430-9cfd-43b1-b674-b6fb830ee511' | | | delete_on_termination='False', id='a385a416-1d37-4111-b7ae-f182a54aeaef' | | | delete_on_termination='False', id='95bafa69-b7cc-4d35-9002-723ca556a570' | | | delete_on_termination='False', id='b5091e83-2afb-457c-abf8-ed855156b59f' | | | delete_on_termination='False', id='0e9762bd-5229-433d-825e-6c7a77c6927c' | | | delete_on_termination='False', id='9a00e144-8755-48d5-8907-7fa499172371' | +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ Please suggest how I can migrate the instance. With regards, Swogat Pradhan

1 1

[election][openstack-ansible]
by Dmitriy Rabotyagov 21 Aug '23

21 Aug '23

Hello everyone! I would like to announce my candidacy for the role of PTL of OpenStack-Ansible during the 2024.1 (Caracal) cycle. For the next cycle I want to pay attention on the following topics: * Updating documentation to correspond new defaults * Keep up with latest supported versions of infrastructure software, like MariaDB, RabbitMQ and Ansible * Promote the project on various events and ensure project compatibility with various vendors * Perform clean-up of old functional testing and test repo and finally make progress on re-implementing functional testing of various scenarios for individual repositories. Thank you for your consideration!

1 0

[neutron] Bug deputy report Aug 14 - Aug 20
by Oleg Bondarev 20 Aug '23

20 Aug '23

Hello Neutron Team, Here's the bug report for the week of Aug 14 - Aug 20: Critical: - https://bugs.launchpad.net/neutron/+bug/2031415 - neutron-tempest-plugin-openvswitch* jobs are failing or timing-out Edit - Fixed by haleyb: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/891550 - https://bugs.launchpad.net/neutron/+bug/2031526 - [ndr] "neutron-tempest-plugin-dynamic-routing" failing - Fixed on os-ken side by j-harbott: https://review.opendev.org/c/openstack/os-ken/+/891658 - waits for os-ken release - https://bugs.launchpad.net/neutron/+bug/2031971 - Rally jobs failing in neutron gate - Invalid - fixed on Rally side by haleyb: https://review.opendev.org/c/openstack/rally-openstack/+/892052 High: - Medium: - https://bugs.launchpad.net/neutron/+bug/2031646 - [sqlalchemy-20] "test_has_alive_neutron_server" failing with SQLAlchemy 2.0 - In progress: https://review.opendev.org/c/openstack/neutron/+/891691 - assigned to ralonsoh - https://bugs.launchpad.net/neutron/+bug/2031656 - Shared networks can be returned as "shared=False" in the API request - Confirmed - assigned to ralonsoh Low: - https://bugs.launchpad.net/neutron/+bug/2031520 - [OVN] Document the router port location (dvr, centralized) depending on network type, load balancing, port forwarding, etc - Triaged - unassigned Thanks, Oleg

1 0

[OSA][NEUTRON]
by Murilo Morais 18 Aug '23

18 Aug '23

Good evening everyone! When using Tenant Network I can't make the instances communicate when they are on different hosts, I believe I'm forgetting to configure something related to network_hosts, but I'm not sure what. It wasn't clear to me exactly the difference between "network_hosts", "network-infra_hosts" and "network-agent_hosts". I'm using the following configuration: openstack_user_config.yml: cidr_networks: container: ... storage: ... used_ips: [...] global_overrides: internal_lb_vip_address: ... external_lb_vip_address: ... management_bridge: br-mgmt no_containers: false provider_networks: - network: container_bridge: br-mgmt container_interface: eth1 container_type: veth ip_from_q: container is_container_address: true type: raw group_binds: - all_containers - hosts - network: container_bridge: br-provider container_type: veth type: vlan range: "100:200" network_interface: "eno2" net_name: vlan group_binds: - neutron_openvswitch_agent - network: container_bridge: "br-flat" container_type: "veth" type: "flat" network_interface: "trunk.103" net_name: "router" group_binds: - neutron_openvswitch_agent - network: container_bridge: br-storage container_interface: eth2 container_type: veth ip_from_q: storage type: raw group_binds: - glance_api - cinder_api - cinder_volume - nova_compute - ceph-mon - ceph-osd shared-infra_hosts: dcn2: ip: ...2 coordination_hosts: dcn2: ip: ...2 repo-infra_hosts: dcn2: ip: ...2 haproxy_hosts: dcn2: ip: ...2 identity_hosts: dcn2: ip: ...2 storage-infra_hosts: dcn2: ip: ...2 storage_hosts: dcn2: ip: ...2 image_hosts: dcn2: ip: ...2 placement-infra_hosts: dcn2: ip: ...2 compute-infra_hosts: dcn2: ip: ...2 dashboard_hosts: dcn2: ip: ...2 network_hosts: dcn2: ip: ...2 compute_hosts: dcn2: ip: ...2 dcn3: ip: ...3 dcn8: ip: ...14 dcn10: ip: ...19 Example to upload the instances: network create network_test subnet create --network network_test --subnet-range 192.168.0.0/24 subnet_test server create --flavor m1.large --image debian-11-genericcloud-amd64-20230601-1398 --network network_test --key-name my_key --use-config-drive debian_test1 server create --flavor m1.large --image debian-11-genericcloud-amd64-20230601-1398 --network network_test --key-name my_key --use-config-drive debian_test2 If for some reason any of them go up on different hosts, even configuring the IP manually, they don't communicate, but the instant they are on the same host the ping starts to work.

2 1

Re: [OpenInfra Foundation Support] Re: Ajuda - Help me! Info.
by Suporte 18 Aug '23

18 Aug '23

Desprezada comunidade!! Poderiam me ajudar? Estou tentando apresentar meu TCC sobre o OpenStak mas estou tendo dificuldades em entender como funciona a ferramenta em relação a múltiplos servidores físicos (cluester). Basicamente queria uma orientação se é possível subir 3 servidores físicos e acessar os recursos unificados em uma aba apenas do Horizon? Amaria um retorno. Atenciosamente | Kind Regards, Mateus Dias Cel.: (11) 96872-8478 E-mail: mateus.dias(a)tektel.com.br Tektel Informática ________________________________ De: Open Infrastructure Foundation <support(a)openstack.zendesk.com> Enviado: segunda-feira, agosto 14, 2023 12:49 PM Para: Suporte <suporte(a)tektel.com.br> Assunto: [OpenInfra Foundation Support] Re: Ajuda - Help me! Info. ##- Please type your reply above this line -## Your request (#46153<https://support.openstack.org/hc/requests/46153>) has been solved. To reopen this request, reply to this email or click the link below: https://support.openstack.org/hc/requests/46153 Allison (OpenInfra Foundation Support) Aug 14, 2023, 10:49 CDT Hi Mateus, Por favor compartilhe esta pergunta na lista de discussão do OpenStack para que a comunidade possa ajudá-lo. https://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss Allison This email is a service from OpenInfra Foundation Support. [K20VPL-4E7XM]

2 2

[release] Release countdown for week R-6, Aug 21 - 25
by Thierry Carrez 18 Aug '23

18 Aug '23

Development Focus ----------------- Work on libraries should be wrapping up, in preparation for the various library-related deadlines coming up. Now is a good time to make decisions on deferring feature work to the next development cycle in order to be able to focus on finishing already-started feature work. General Information ------------------- We are now getting close to the end of the cycle, and will be gradually freezing feature work on the various deliverables that make up the OpenStack release. This coming week is the deadline for general libraries (except client libraries): their last feature release needs to happen before "Non-client library freeze" on August 24th. Only bugfixes releases will be allowed beyond this point. When requesting those library releases, you can also include the stable/2023.2 branching request with the review (as an example, see the "branches" section here: https://opendev.org/openstack/releases/src/branch/master/deliverables/pike/… In the next weeks we will have deadlines for: * Client libraries (think python-*client libraries), which need to have their last feature release before "Client library freeze" (August 31st) * Deliverables following a cycle-with-rc model (that would be most services), which observe a Feature freeze on that same date, August 31st. Any feature addition beyond that date should be discussed on the mailing-list and get PTL approval. As we are getting to the point of creating stable/2023.2 branches, this would be a good point for teams to review membership in their $project-stable-maint groups. Once the stable/2023.2 branches are cut for a repo, the ability to approve any necessary backports into those branches for 2023.2 "Bobcat" final release will be limited to the members of that stable team. If there are any questions about stable policy or stable team membership, please reach out in the #openstack-stable channel. Upcoming Deadlines & Dates -------------------------- Non-client library freeze: August 24th, 2023 (R-6 week) Client library freeze: August 31st, 2023 (R-5 week) Bobcat-3 milestone: August 31st, 2023 (R-5 week) Cycle Highlights Due: August 31st, 2023 (R-5 week) Final 2023.2 Bobcat release: October 4th, 2023 -- Thierry Carrez (ttx)

1 0

[neutron] Neutron drivers meeting cancelled
by Rodolfo Alonso Hernandez 18 Aug '23

18 Aug '23

Hello Neutrinos: Due to the lack of agenda [1], today's meeting will be cancelled. Have a nice weekend. [1]https://wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda

1 0

Need assistance
by Gk Gk 18 Aug '23

18 Aug '23

Hi All, We have an openstack ansible setup. We are trying to collect statistics for mysql galera cluster. I am trying to use the following mysql prometheus exporter at https://github.com/prometheus/mysqld_exporter/releases . But when I install the exporter into each of the three mysql galera nodes, I am not finding any mysql client statistics on the metrics url. Also when I checked the mysql databases 'information_schema' and 'performance_schema' , they were empty. How to get the relevant galera stats for the mysql cluster in openstack ? Thanks Kumar

1 0

vPTG October 2023 Team Signup Deadline Approaching!
by Kendall Nelson 17 Aug '23

17 Aug '23

Hello Everyone, Sign up your team up for the next virtual Project Teams Gathering (PTG), which will be held from Monday, October 23 to Friday, October 27 2023! If you haven't already done so and your team is interested in participating, you need to complete the survey[1] by August 20, 2023 at 7:00 UTC. Then make sure to register[2] - it’s free :) Thanks! -Kendall (diablo_rojo) [1] Team Survey: https://openinfrafoundation.formstack.com/forms/oct2023_ptg_team_signup [2] PTG Registration: http://ptg2023.openinfra.dev/

1 1

Call for Mentors & Projects - Boston University Students
by Kendall Nelson 17 Aug '23

17 Aug '23

Hello Everyone! As some of you might know, we have had a working relationship with Boston University once a year where students actively participate in our OpenInfra project communities. The semester in which the Fundamentals of Cloud Computing (the course where we collaborate with them)( is approaching. September 5 is when the course begins and it runs until December 21. The course is a senior/graduate-level class that includes a semester-long capstone-like project, with the projects being proposed and supervised by an industry mentor. The projects are completed in teams of 5-7 students, who are expected to spend 5-8 hours per week on the project over the 13-week term, with presentations to the class every two weeks to evaluate their progress. *So! If you know of a project you would like to propose, please let me know ASAP! The professor would like to have the finalized list of projects to offer students by the first week in September. * Let me know if you have any other questions! -Kendall Nelson

3 3

[kolla] openvswitch-netcontrold image removal
by Michał Nasiadka 17 Aug '23

17 Aug '23

Hello Koalas, The openvswitch-netcontrold image is being dropped, that means in Bobcat release Kolla will not build and publish this image. This image is not supported within kolla-ansible, so it’s not tested and it installs netcontrold outside of the regular kolla venv. It has been used in the past by TripleO - but TripleO will not support Bobcat and beyond. If there is anybody interested in maintaining this image - please come forward posting your comment in the patch [1], otherwise it will be dropped. [1]: https://review.opendev.org/c/openstack/kolla/+/891655 Michal

1 0

[openstack][nova] how check if exist instances were create with AZ or without AZ
by Nguyễn Hữu Khôi 16 Aug '23

16 Aug '23

Hello guys. I need to verify whether an existing instance was created with AZ or without AZ. How can we check on an existing instance? I did not setup default AZ scheduler before. At this time I plan to expand the system by a new AZ. as document If the server was not created in a specific zone then it is free to be moved to other zones, i.e. the AvailabilityZoneFilter <https://docs.openstack.org/nova/latest/admin/scheduling.html#availabilityzo…> is a no-op. I am worried about when I migrate instances and they will be free to be moved to other zones. Nguyen Huu Khoi

2 3

VM state wont update on power failure
by Shrey Chauhan 16 Aug '23

16 Aug '23

Hi, Currently I am running xena version of openstack. I am confused about one thing. Running a 3 node cluster. In case of a power failure my vm still shows as active (vm_state = active, power_state = running), even though in KVM they are in shutoff state. When I bring back the node and restart nova the state changes to 'shutoff'. What I am not understanding, is nova managing only local vms power states. Its a 3 node cluster, so nova compute running on other node should change the state in the database, isnt it? Am I missing any nova or linvirt configuration here? Thanks in advance for any kind of help here Thanks and regards, Shrey

2 1

[olso.messaging] RabbitMQ and streams
by Arnaud Morin 16 Aug '23

16 Aug '23

Hey all, Is there any chance that someone already worked on integrating RabbitMQ streams (see [1]) to replace fanouts in oslo rabbitmq driver? We are wondering if this would help lowering the rabbit usage, and maybe increase the rabbit stability? Our deployment is relying a lot on fanouts to delivers messages to computes (mostly neutron, e.g. secgroup update / remote cache population), this is leading to a massive number of messages beeing delivered by seconds. Cheers, Arnaud [1] https://www.rabbitmq.com/streams.html

2 2

[tripleo] [deployment] heat stack error
by Kamil Madac 16 Aug '23

16 Aug '23

Hi Brendan, thanks for the reply. Existing deployment has been done on different set of servers and on centos stream 8, wallaby, so on centos 9 we deployed a different tripleo repos (python3-tripleo-repos-0.1.1-0.20220214194341.cbbdde6.el8.noarch.rpm vs python3-tripleo-repos-0.1.1-0.20230111082402.c73cb0f.el9.noarch.rpm ) for undercloud deployment. Environment yamls were taken from that working deployment. We will try to remove environment yaml from deploy arguments one by one and will try to identify yaml which causes the issue. On Tue, Aug 15, 2023, 22:49 Brendan Shephard <bshephar(a)redhat.com> wrote: > Hey Kamil, > > There will be an error in one of the environment files that you’re > including in your overcloud deploy command. > > You said it’s your first deployment that was already prepared for you. > Does this mean that the environment has already been deployed and you are > trying to make a change now? If that’s the case, then I would check any of > the files that have changed between the initial deployment and what you’re > trying to do now. > > Regards, > Brendan Shephard > Senior Software Engineer > > > > On 15 Aug 2023, at 23:48, Kamil Madac <kamil.madac(a)gmail.com> wrote: > > > > > > Hello, > > I'm trying to deploy openstack wallaby on centos9. Undercloud is > running, but when I run "openstack overcloud deploy" it ends up with error: > > > > 2023-08-14 14:45:51.426 5 ERROR heat.engine.worker > > 2023-08-14 14:45:52.346 5 DEBUG heat.engine.scheduler > [req-fa1861f1-2a35-4754-8179-7ca9c8a209a8 admin admin - - -] Task create > from TemplateResource "ControllerServiceChain" Stack "keos9" > [18515e00-1d85-4 > > f16-91f7-a88c7553c1ed] running step > /usr/lib/python3.6/site-packages/heat/engine/scheduler.py:207 > > 2023-08-14 14:45:52.352 5 INFO heat.engine.resource > [req-fa1861f1-2a35-4754-8179-7ca9c8a209a8 admin admin - - -] CREATE: > TemplateResource "ControllerServiceChain" > [44cce696-f0ef-4975-88e5-5b4e2b69149c] St > > ack "keos9" [18515e00-1d85-4f16-91f7-a88c7553c1ed] > > 2023-08-14 14:45:52.352 5 ERROR heat.engine.resource Traceback (most > recent call last): > > 2023-08-14 14:45:52.352 5 ERROR heat.engine.resource File > "/usr/lib/python3.6/site-packages/heat/engine/resource.py", line 916, in > _action_recorder > > 2023-08-14 14:45:52.352 5 ERROR heat.engine.resource yield > > 2023-08-14 14:45:52.352 5 ERROR heat.engine.resource File > "/usr/lib/python3.6/site-packages/heat/engine/resource.py", line 1028, in > _do_action > > 2023-08-14 14:45:52.352 5 ERROR heat.engine.resource yield from > self.action_handler_task(action, args=handler_args) > > 2023-08-14 14:45:52.352 5 ERROR heat.engine.resource File > "/usr/lib/python3.6/site-packages/heat/engine/resource.py", line 978, in > action_handler_task > > 2023-08-14 14:45:52.352 5 ERROR heat.engine.resource done = > check(handler_data) > > 2023-08-14 14:45:52.352 5 ERROR heat.engine.resource File > "/usr/lib/python3.6/site-packages/heat/engine/resources/stack_resource.py", > line 408, in check_create_complete > > 2023-08-14 14:45:52.352 5 ERROR heat.engine.resource return > self._check_status_complete(self.CREATE) > > 2023-08-14 14:45:52.352 5 ERROR heat.engine.resource File > "/usr/lib/python3.6/site-packages/heat/engine/resources/stack_resource.py", > line 451, in _check_status_complete > > 2023-08-14 14:45:52.352 5 ERROR heat.engine.resource action=action) > > 2023-08-14 14:45:52.352 5 ERROR heat.engine.resource > heat.common.exception.ResourceFailure: resources.ControllerServiceChain: > Error in 24 output role_data: The values of the "for_each" argument to > "repeat > > " must be lists or maps > > 2023-08-14 14:45:52.352 5 ERROR heat.engine.resource > > 2023-08-14 14:45:52.366 5 INFO heat.engine.stack > [req-fa1861f1-2a35-4754-8179-7ca9c8a209a8 admin admin - - -] Stack CREATE > FAILED (keos9): Resource CREATE failed: resources.ControllerServiceChain: > Error i > > n 24 output role_data: The values of the "for_each" argument to "repeat" > must be lists or maps > > > > > > It is my first tripleo deployment which was already prepared by the > customer, but is not working currently. It is wallaby deployment on centos > stream 9. > > > > Can somebody give me advice on what is the best way to troubleshoot such > issues in heat stacks in tripleo? > > > > Role data has been successfully generated by command: > > > > $ openstack overcloud roles generate -o > /var/workdir/tripleo-environment/tripleo-heat-templates/custom_roles_data.yaml > Controller Compute > > > > and then used to generate heat templates: > > > > > /var/workdir/tripleo-heat-templates-upstream-wallaby/tools/process-templates.py > \ > > -p /var/workdir/tripleo-heat-templates-upstream-wallaby \ > > -r > /var/workdir/tripleo-environment/tripleo-heat-templates/custom_roles_data.yaml > \ > > -n > /var/workdir/tripleo-environment/tripleo-heat-templates/custom_network_data.yaml > \ > > -o /var/workdir/generated-tripleo-heat-templates \ > > --safe > > > > > > > > Kamil Madac > > > >

1 0