September 2021 - openstack-discuss

[all][elections][ptl][cyborg] PTL voting for Cyborg- Bailin Zhang(brinzhang)
by Brin Zhang(张百林) 02 Sep '21

02 Sep '21

Hi all. I was going to attempt to enumerate some sort of grand vision for Cyborg and our priorities for the next cycle. As I thought about it, though, I realized these remain largely the same as they always have been: provide a reliable, scalable, and efficient hardware-accelerated device management platform. In the past, we have added support for GPU, FPGA, NVMe SSD, and will add support for AEP, SPDK and other devices in the future. In order to increase the ability to interact with the VM, we were completed the feature of nova-cyborg interaction in Ussuri release. The exact details of how we do that may vary somewhat from cycle to cycle, and I was completed the support of nova rebuild/evacuate server and shelve/unshelve in the last two releases, which greatly improved the convenience of user operations. Such as the suspend and resume, the PoC code you can review https://review.opendev.org/c/openstack/nova/+/729945. And we also tried to support the code migration in the future, of course that need a spec and to talk with nova team. The log management project (Venus) team and I have completed the integration of Venus to Cyborg. We can use Venus to quickly retrieve Cyborg logs, shorten the problem location time, etc., so as to improve the efficiency of the Cyborg project. In the future, I will have in-depth exchanges with the Venus development team to continue to enhance this feature. In these two releases, I have promoted many newcomers to join the Cyborg team (such as Wenping Song, Eric Xie, fossnqiu, Zhiguang Wang, Arthur Dayne etc.), and they have made great contributions to the Cyborg project. In the future, I would like to help more people. I hope to get your support, thanks. Bailin Zhang (brinzhang) brinzhang Inspur Electronic Information Industry Co.,Ltd.

1 0

[nova][cyborg][gpu] GPU performance testing
by Ildiko Vancsa 02 Sep '21

02 Sep '21

Hi, I got a question about tools and practices to check GPU performance in an OpenStack environment that I need some help to answer. The question is about recommended GPU performance testing/benchmarking tools if there are a few that people in the community are using and would recommend? The scope of the testing work is to check GPU performance in OpenStack VMs (both virtualized and passthrough). All the help and pointers are very much appreciated! Thanks, Ildikó

6 9

Re: Windows2012 VM image importing and Instance Launch Failure
by KK CHN 01 Sep '21

01 Sep '21

On Fri, Aug 27, 2021 at 11:00 AM <DHilsbos(a)performair.com> wrote: > Kris; > > First, you can add properties during the create command, so steps 1 > through 3 can become: > # openstack image create “M_Windows_VM_NAME_Blaah” --file > Windows_VM_disk.qcow2 \ > --disk-format qcow2 --container-format bare --public --property > hw_firmware_type=uefi \ > --property os_secure_boot=required --property hw_disk_bios=ide > > I am able to import the WIndows VM, which was exported from (Rhvm oVirt) to my openstack(ussuri, qemu-kvm, glance, cinder-ceph ) booted the first bootable disk of this multi disk Windows 2012 RC VM. It has additionally a data disk which is of 9GB of data of 600 GB volume size originally. > Secondly, no. Only the boot image (C: drive) needs the properties. So > any additional disks for the same VM can use a command like this: > I executed this step for the second disk qcow2 image on my Controller node. # openstack image create “M_Windows_VM_NAME_DataDisk2” --file > Windows_VM_disk.qcow2 --disk-format qcow2 --public > > Then the image appears in the Horizon Image section. I tried to create Volume through Horizon GUI with 600 GB disk space allocated , it ran for 10 t0 20 minutes and went to Error state. multiple attempts I have made by deleting each error volume and creating new volume but same result. So I tried to do it through CLI First I listed the images uploaded : by #openstack image list c6695789-8524-446c-98c9-282b2880551c | CMDALocalDBWindowsdisk2 | active // This is the data disk image root@Meghctrl1:/home/cloud# #openstack volume create --image c6695789-8524-446c-98c9-282b2880551c --size 600 --availability-zone nova CMDA_DB_SecDisk_Vol root@Meghctrl1:/home/cloud# openstack volume list +--------------------------------------+----------------------+-----------+------+-------------+ | ID | Name | Status | Size | Attached to | +--------------------------------------+----------------------+-----------+------+-------------+ | 32f9452f-368f-4f26-b70d-c066c4c7e01c | CMDA_DB_SecDisk_Vol | available | 600 | | Now I have gone to the Horizon dashboard. I tried to attach this Volume CMDA_DB_SecDisk_Vol to My running Windows VM which is the first disk which was imported and booted successfully . I got the Dashboard Status alert while doing this : " Info: Attaching volume *CMDA_DB_SecDisk_Vol(32f9452f-368f-4f26-b70d-c066c4c7e01c) to instance 6630c9c8-135e-45b7-958b-c563337eb9c4 on /dev/hdb "* But This is not attached to the Running Windows VM. *You can see it is attached to none * root@Meghctrl1:/home/cloud# openstack volume list +--------------------------------------+----------------------+-----------+------+-------------+ | ID | Name | Status | Size | Attached to | +--------------------------------------+----------------------+-----------+------+-------------+ | 32f9452f-368f-4f26-b70d-c066c4c7e01c | CMDA_DB_SecDisk_Vol | available | 600 | | | 0c404d4c-1685-4a5a-84ae-1ee413f02f68 | cloudtest_DMS1 | error | 102 | | | e9949f4f-b23a-4a30-8620-5a8d234b523b | DMS_VM_APPLICATION-2 | available | 100 | Here you can see it attached to None. *Also in the horizon dashboard the running instance I clicked for more information : It shows no Volumes attached to this instance. * TestDB_CMDA_disk1 Create Snapshot <http://10.236.247.230/project/images/6630c9c8-135e-45b7-958b-c563337eb9c4/c…> <http://10.236.247.230/project/instances/6630c9c8-135e-45b7-958b-c563337eb9c…> - Overview <http://10.236.247.230/project/instances/6630c9c8-135e-45b7-958b-c563337eb9c…> - Interfaces <http://10.236.247.230/project/instances/6630c9c8-135e-45b7-958b-c563337eb9c…> - Log <http://10.236.247.230/project/instances/6630c9c8-135e-45b7-958b-c563337eb9c…> - Console <http://10.236.247.230/project/instances/6630c9c8-135e-45b7-958b-c563337eb9c…> - Action Log <http://10.236.247.230/project/instances/6630c9c8-135e-45b7-958b-c563337eb9c…> NameTestDB_CMDA_disk1ID6630c9c8-135e-45b7-958b-c563337eb9c4Description-Project IDb1bf4df41dc34e3ab5c3b0318158263dStatusActiveLockedFalseAvailability Zone novaCreatedSept. 1, 2021, 12:24 p.m.Age5 hours, 9 minutesHostMeghctrl1Specs ------------------------------ Flavor NameMediumFlavor ID5bb6bc50-b2f7-49b2-8f01-7a631af5ff8cRAM4GBVCPUs2 VCPUDisk150GBIP Addresses ------------------------------ Tenant_NW172.16.100.248Security Groups ------------------------------ default - ALLOW IPv6 from default - ALLOW IPv4 icmp to 0.0.0.0/0 - ALLOW IPv4 to 0.0.0.0/0 - ALLOW IPv4 tcp from 0.0.0.0/0 - ALLOW IPv4 icmp from 0.0.0.0/0 - ALLOW IPv4 from default - ALLOW IPv6 to ::/0 Metadata ------------------------------ Key NameTNSDCImage NameCMDALocalDBWindows1 <http://10.236.247.230/project/images/6437fde7-db0c-4298-9d92-1f70aae3c894/>Image ID6437fde7-db0c-4298-9d92-1f70aae3c894Volumes Attached ------------------------------ Volume*No volumes attached.* *Details of the CLI Created disk Volume details which is unable to attach to the running Windows 2012 RC instance is as follows from Horizon GUI* CMDA_DB_SecDisk_Vol Edit Volume <http://10.236.247.230/project/volumes/32f9452f-368f-4f26-b70d-c066c4c7e01c/…> <http://10.236.247.230/project/volumes/32f9452f-368f-4f26-b70d-c066c4c7e01c/#> - Overview <http://10.236.247.230/project/volumes/32f9452f-368f-4f26-b70d-c066c4c7e01c/…> - Snapshots <http://10.236.247.230/project/volumes/32f9452f-368f-4f26-b70d-c066c4c7e01c/…> NameCMDA_DB_SecDisk_VolID32f9452f-368f-4f26-b70d-c066c4c7e01cProject ID b1bf4df41dc34e3ab5c3b0318158263dStatusAvailableGroup-Specs ------------------------------ Size600 GiBType__DEFAULT__BootableNoEncryptedNoCreatedSept. 1, 2021, 4:07 p.m.Attachments ------------------------------ Attached To*Not attached*Volume Source ------------------------------ Image <http://10.236.247.230/project/volumes/32f9452f-368f-4f26-b70d-c066c4c7e01c/> Metadata ------------------------------ None What am I doing wrong here ? : any hints or suggestions to add this data disk to the running WIndows instance are most welcome. Or do I need to add the volume to the instance through CLI, from the docs I am seeing this. Will this work for Running Windows VM instances also ? openstack server add volume 84c6e57d-a6b1-44b6-81eb-fcb36afd31b5 \ 573e024d-5235-49ce-8332-be1576d323f8 --device /dev/vdb *Will you advice to attaching the Volume through CLI like the following * #openstack server add volume 6630c9c8-135e-45b7-958b-c563337eb9c4 32f9452f-368f-4f26-b70d-c066c4c7e01c --device /dev/hdb * Will it solve this volume attachment issue ? or seriously am I missing any crucial parameters or steps for attaching the additional disk volumes for Windows VMs?* *Any hints most welcome: Any other information or specific log files I need to post, kindly tell me know the log file names I can post . I am doing it for the first time. * Thanks in advance for your answers. Kris You might consider making sure that these additional images aren't marked > as bootable. > > That said... Have you considered importing the disks directly into Cinder > instead? > > Thank you, > > Dominic L. Hilsbos, MBA > Vice President – Information Technology > Perform Air International Inc. > DHilsbos(a)PerformAir.com > www.PerformAir.com > > > From: KK CHN [mailto:kkchn.in@gmail.com] > Sent: Thursday, August 26, 2021 10:04 PM > To: Dominic Hilsbos > Subject: Re: Windows2012 VM image importing and Instance Launch Failure > > Hi Dominic, > For Importing Single disk Windows VM I performed the following steps on > the vhdx to qcow2 converted image > Step 1: > root@Meghctrl1:/home/cloud# openstack image create > “M_Windows_VM_NAME_Blaah” --file Windows_VM_disk.qcow2 > --disk-format qcow2 --container-format bare --public > Step 2: > root@Meghctrl1:/home/cloud# openstack image set --property > hw_firmware_type=uefi --property > os_secure_boot=required My_Windows_VM_NAME_Blaah > Step 3: > root@Meghctrl1:/home/cloud# openstack image set --property > hw_firmware_type=uefi > --property hw_disk_bios=ide MY_Windows_VM_NAME_Blaah > > And I am able to see the image in the Horizon dashboard "image" tab > and launch the instance from there . Everything went fine and I am able to > get the Windows Single Disk VM running. > > Query: > If I have a Multi Disc Windows VM , do I need to perform all the above > three steps for all the qcow2 converted images ? ( For eg: I have a > Database Server , A Windows VM already running in HyperV with two disks ( > 100GB disk and 600 GB another disk image) . So in these two disks > one will be WIndows OS and another will be attached disk without Windows > OS right ? > 1. For the first disk I will perform all the above three steps after > converting it from vhdx to qcow2. So that image will appear in the > horizon and I can launch an instance as I did early for Single Disk Windows > VM. > 2. For the 600 GB second disk (vhdx format) I will convert it to > qcow2 and then do I need to perform all the above three steps exactly ? > Or what changes I need to make in the above steps to import this disk as > this is an additional attached disk in hyperV. > Request your guidance or any suggestions in this regard. > > Thank You, > Kris > > > > On Wed, Aug 25, 2021 at 9:12 PM <DHilsbos(a)performair.com> wrote: > Kris; > > This shouldn’t be all that much more difficult than Linux, nor that much > harder than a single-disk Windows VM. > > I would start by disabling the SQL service, as the OpenStack instance will > boot once before you can add the data disks (at least it does for me when > using Horizon). > > Export the disks, convert then, and import them, all as you normally > would. Create the instance, as you normally would. The instance will > start; shut it back down. Attach the additional volumes, and start the > instance back up. > > It would be nice to have an option to not start the instance after > creation. > > You could also investigate the “openstack server create” command; it has a > --volume argument that you may be able to pass more than once, though I > suspect not. > > Thank you, > > Dominic L. Hilsbos, MBA > Vice President – Information Technology > Perform Air International Inc. > DHilsbos(a)PerformAir.com > www.PerformAir.com > > > From: KK CHN [mailto:kkchn.in@gmail.com] > Sent: Tuesday, August 24, 2021 11:19 PM > To: openstack-discuss(a)lists.openstack.org > Subject: Re: Windows2012 VM image importing and Instance Launch Failure > > Thank you all. > > I am able to get the Single disk Windows VM exported from HyperV and > imported to OpenStack(Ussuri, Glance and Qemu KVM ) Instance launched and > its showing the status running in the PowerState Tab of Horizon dashboard. > > (need to check with the administrator user to login and confirm the VM is > working as expected . ) > > My second phase : I need to perform importing multiple disk > Windows VM to my OpenStack setup . > > ( For single disk Windows vhdx file, I 've converted it to qcow2 file and > imported to OpenStack as the steps I posted in my previous emails. ) > > Now how can I import Multiple disk Windows VM to OpenStack. > > First Step is to convert the vhdx files exported from HyperV to qcow2 > files. > > After this step , what steps need to be performed to bring a multi > disk Windows VM to OpeStack ? > > For Eg: I have a Windows VM > > 1. with Application server 1TB single disk image exported from hyperV. > This I can import to my OpenStack setup with the steps I followed in my > earlier emails as it is a single disk image. > > > 2. A Database server for the above application with 700 GB in multiple > disks. ( 100 and 600 GB disks images exported from HyperV and in vhdx > format for DB server) > > How to bring this server imported to my OpeStack setup (ussuri, glance > and QEMU KVM). As this is from Windows HyperV I am new to multidisk > environment from Windows. > ( for Linux VMs exported from oVirt(RHVM) I am able to import multidisk > VMs to OpenStack as they are in LVM and I have to mount it with VG name and > adding entries in /etc/fstab file ) > > But in Windows multi disks how to perform this ? what steps I have to > perform to import this to OpenStack. Any hints or suggestions most > welcome.. > > Kris > > > On Tue, Aug 24, 2021 at 1:40 PM Eugen Block <eblock(a)nde.ag> wrote: > Of course you need a running nova-conductor, how did you launch VMs before? > > > Zitat von KK CHN <kkchn.in(a)gmail.com>: > > > yes. I too suspect this not the problem with image. I have seen the > logs: > > it throwing Errors which pasted here. > > > > Its waiting for nova-conductor service as in the logs. Is this the > issue? > > Do I need to explictly statrt conductor service or some other service ? > > nova-compute logs pasted below the lines. share your thoughts what may > > the issue. > > > > > > On Fri, Aug 20, 2021 at 8:11 AM Mohammed Naser <mnaser(a)vexxhost.com> > wrote: > > > >> I suggest that you have a look at the compute node logs when it fails to > >> spawn. I suspect the problem is not Your images but something inside > >> openstack > >> > >> > > tail: no files remaining > > cloud@Meghctrl1:~$ sudo tail -f /var/log/nova/nova-compute.log > > [sudo] password for cloud: > > 2021-08-19 11:59:59.477 2847961 ERROR oslo_service.periodic_task > > transport_options=transport_options) > > 2021-08-19 11:59:59.477 2847961 ERROR oslo_service.periodic_task > > File > > > "/usr/local/lib/python3.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", > > line 642, in _send > > 2021-08-19 11:59:59.477 2847961 ERROR oslo_service.periodic_task > > call_monitor_timeout) > > 2021-08-19 11:59:59.477 2847961 ERROR oslo_service.periodic_task > > File > > > "/usr/local/lib/python3.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", > > line 531, in wait > > 2021-08-19 11:59:59.477 2847961 ERROR oslo_service.periodic_task > > message = self.waiters.get(msg_id, timeout=timeout) > > 2021-08-19 11:59:59.477 2847961 ERROR oslo_service.periodic_task > > File > > > "/usr/local/lib/python3.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", > > line 409, in get > > 2021-08-19 11:59:59.477 2847961 ERROR oslo_service.periodic_task > > 'to messageID %s' % msg_id) > > 2021-08-19 11:59:59.477 2847961 ERROR oslo_service.periodic_task > > oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a > > reply to message ID 7a4e3a4acad3403a9966570cc259f7b7 > > 2021-08-19 11:59:59.477 2847961 ERROR oslo_service.periodic_task > > 2021-08-19 12:00:01.467 2847961 WARNING oslo.service.loopingcall [-] > > Function 'nova.servicegroup.drivers.db.DbDriver._report_state' run > > outlasted interval by 50.03 sec > > 2021-08-19 12:00:34.012 2847812 WARNING nova.conductor.api > > [req-afca4d09-bf9c-4bbc-ae88-22008d12b978 - - - - -] Timed out waiting > > for nova-conductor. Is it running? Or did this service start before > > nova-conductor? Reattempting establishment of nova-conductor > > connection...: oslo_messaging.exceptions.MessagingTimeout: Timed out > > waiting for a reply to message ID 9f5b593361b34c8d91585c00e0514047 > > 2021-08-19 12:00:59.480 2847961 DEBUG oslo_service.periodic_task > > [req-58a0e182-6e56-4695-b4db-75624ca69c7b - - - - -] Running periodic > > task ComputeManager._check_instance_build_time run_periodic_tasks > > /usr/local/lib/python3.7/dist-packages/oslo_service/periodic_task.py:211 > > 2021-08-19 12:00:59.481 2847961 DEBUG oslo_service.periodic_task > > [req-58a0e182-6e56-4695-b4db-75624ca69c7b - - - - -] Running periodic > > task ComputeManager._sync_scheduler_instance_info run_periodic_tasks > > /usr/local/lib/python3.7/dist-packages/oslo_service/periodic_task.py:211 > > 2021-08-19 12:01:01.499 2847961 WARNING oslo.service.loopingcall [-] > > Function 'nova.servicegroup.drivers.db.DbDriver._report_state' run > > outlasted interval by 50.03 sec > > 2021-08-19 12:01:34.040 2847812 WARNING nova.conductor.api > > [req-afca4d09-bf9c-4bbc-ae88-22008d12b978 - - - - -] Timed out waiting > > for nova-conductor. Is it running? Or did this service start before > > nova-conductor? Reattempting establishment of nova-conductor > > connection...: oslo_messaging.exceptions.MessagingTimeout: Timed out > > waiting for a reply to message ID e8e478be640e4ecab8a3d27c01960440 > > > > > > essage ID d620b1b3938e4428a0500c00df8d68ee > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > [req-58a0e182-6e56-4695-b4db-75624ca69c7b - - - - -] Error during > > ComputeManager._cleanup_expired_console_auth_tokens: > > oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a > > reply to message ID a493182a404242b79c8f11f8ec350e36 > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > Traceback (most recent call last): > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File > > > "/usr/local/lib/python3.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", > > line 405, in get > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > return self._queues[msg_id].get(block=True, timeout=timeout) > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File "/usr/local/lib/python3.7/dist-packages/eventlet/queue.py", line > > 322, in get > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > return waiter.wait() > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File "/usr/local/lib/python3.7/dist-packages/eventlet/queue.py", line > > 141, in wait > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > return get_hub().switch() > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File "/usr/local/lib/python3.7/dist-packages/eventlet/hubs/hub.py", > > line 298, in switch > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > return self.greenlet.switch() > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > _queue.Empty > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > During handling of the above exception, another exception occurred: > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > Traceback (most recent call last): > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File > "/usr/local/lib/python3.7/dist-packages/oslo_service/periodic_task.py", > > line 216, in run_periodic_tasks > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > task(self, context) > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File "/usr/local/lib/python3.7/dist-packages/nova/compute/manager.py", > > line 10450, in _cleanup_expired_console_auth_tokens > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > objects.ConsoleAuthToken.clean_expired_console_auths(context) > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File > "/usr/local/lib/python3.7/dist-packages/oslo_versionedobjects/base.py", > > line 177, in wrapper > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > args, kwargs) > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File "/usr/local/lib/python3.7/dist-packages/nova/conductor/rpcapi.py", > > line 243, in object_class_action_versions > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > args=args, kwargs=kwargs) > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File > "/usr/local/lib/python3.7/dist-packages/oslo_messaging/rpc/client.py", > > line 179, in call > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > transport_options=self.transport_options) > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File > "/usr/local/lib/python3.7/dist-packages/oslo_messaging/transport.py", > > line 128, in _send > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > transport_options=transport_options) > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File > > > "/usr/local/lib/python3.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", > > line 654, in send > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > transport_options=transport_options) > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File > > > "/usr/local/lib/python3.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", > > line 642, in _send > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > call_monitor_timeout) > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File > > > "/usr/local/lib/python3.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", > > line 531, in wait > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > message = self.waiters.get(msg_id, timeout=timeout) > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > File > > > "/usr/local/lib/python3.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", > > line 409, in get > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > 'to message ID %s' % msg_id) > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a > > reply to message ID a493182a404242b79c8f11f8ec350e36 > > 2021-08-19 12:27:00.387 2847961 ERROR oslo_service.periodic_task > > 2021-08-19 12:27:01.681 2847961 DEBUG oslo_service.periodic_task > > [req-58a0e182-6e56-4695-b4db-75624ca69c7b - - - - -] Running periodic > > task ComputeManager._check_instance_build_time run_periodic_tasks > > /usr/local/lib/python3.7/dist-packages/oslo_service/periodic_task.py:211 > > 2021-08-19 12:27:01.687 2847961 DEBUG oslo_service.periodic_task > > [req-58a0e182-6e56-4695-b4db-75624ca69c7b - - - - -] Running periodic > > task ComputeManager._sync_scheduler_instance_info run_periodic_tasks > > /usr/local/lib/python3.7/dist-packages/oslo_service/periodic_task.py:211 > > 2021-08-19 12:27:02.368 2847961 WARNING oslo.service.loopingcall [-] > > Function 'nova.servicegroup.drivers.db.DbDriver._report_state' run > > outlasted interval by 50.03 sec > > > > > > > > Error-nov-log-file_while_sch ... e_in _horizaon_dashboard.txt > > Open with > > Displaying Error-nov-log-file_while_scheduling_Windows_VM_instance_in > > _horizaon_dashboard.txt. > > > > > > > > > > > >> If I was to guess it’s probably missing UEFI firmware packages :) > >> > >> On Wed, Aug 18, 2021 at 9:17 AM KK CHN <kkchn.in(a)gmail.com> wrote: > >> > >>> Error : failed to perform requested operation on instance "WindowsVM > "the > >>> instance has error status. Please try again later [Error exceeded > maximum > >>> number of retries. Exhausted all hosts available for retrying build > >>> failures for instance e3d5c095-7d26-4b1e-89d1-d1a6e20a45041 > >>> > >>> I am trying to import a WIndows2012 Single disk VM, to OpenStack > Ussuri, > >>> with glance and Qemu KVM. > >>> > >>> In bare machine KVM I am able to import and boot this Windows VM which > >>> exported from rhevm hypervisor as vhdx image. > >>> what I have done is > >>> > >>> 1. converted this windows image from vhdx to qcow2 > >>> 2. root@MeghCtrol1:/home/cloud/CMOBB_APP#cirt-install --name WINDOWS > >>> --ram=1048 --vcups=1 --cpu host --hvm --dick > >>> path=BackUP2_CMAPP_disk_1_Windows_qcow2_imagefile,device=disk, > >>> format=qcow2,bus=virtio --graphics vnc --boot uefi > >>> > >>> This uploaded the qcow2 image of WindowsVM to KVM hypervisor and its > >>> working. > >>> > >>> But when I do importing to openstack unable to launch instance from > >>> the image . > >>> > >>> These are the steps I performed.. > >>> > >>> 1. openstack image create "WindowsVM" --file CMAPP_disk_1.qcow2 > >>> --disk-format qcow2 --container-format bare --public > >>> > >>> 4.openstack image set --property hw_firmware_type=uefi --property > >>> os_secure_boot=required WindowsVM > >>> > >>> 5.openstack image set --property hw_firmware_type=uefi --property > >>> hw_disk_bus=ide WindowsVM > >>> > >>> 6.openstack image show WindowsVM > >>> > >>> 7. root@dmzcloud:/home/cloud# openstack image show WindowsVM|grep > >>> "properties" > >>> | properties | hw_disk_bus='ide', hw_firmware_type='uefi', > >>> os_hash_algo='sha512', > >>> > >>> > os_hash_value='753ee596980409e1e72d6d020c8219c56a6ada8b43f634fb575c594a245725a398e45982c0a1ad72b3fc3451cde62cceb9ff22be044863b31ecdd7893b049349', > >>> os_hidden='False', os_secure_boot='required', > >>> owner_specified.openstack.md5='', > >>> owner_specified.openstack.object='images/WindowsVM', > >>> owner_specified.openstack.sha256='' | > >>> root@dmzcloud:/home/cloud# > >>> > >>> > >>> Then I logged into horizon dashboard, from the images selected the > >>> imported image and try to launch the instance. With a Flavour of 550 > GB > >>> disk, 4 vcpus and 8GB Ram .. > >>> > >>> The instance spawning ran for 30 minutes and throws the error which I > >>> pasted first in the right top corner of horizon dashboard. > >>> > >>> How to solve this error and boot the Windows machine successfully .. > >>> > >>> > >>> """ > >>> Error : failed to perform requested operation on instance "WindowsVM > "the > >>> instance has error status. Please try again later [Error exceeded > maximum > >>> number of retries. Exhausted all hosts available for retrying build > >>> failures for instance e3d5c095-7d26-4b1e-89d1-d1a6e20a45041 > >>> > >>> > >>> """ > >>> > >>> Any help highly appreciated. > >>> > >>> Kris > >>> > >>> -- > >> Mohammed Naser > >> VEXXHOST, Inc. > >> > > >

2 1

[largescale-sig] Next meeting: Sept 1st, 15utc
by Thierry Carrez 01 Sep '21

01 Sep '21

Hi everyone, Our next Large Scale SIG meeting will be this Wednesday in #openstack-operators on OFTC IRC, at 15UTC. You can doublecheck how that time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20210901T15 A number of topics have already been added to the agenda, including discussing the topic of our next OpenInfra.Live show! Feel free to add other topics to our agenda at: https://etherpad.openstack.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

1 1

Re: [all][elections][ptl] PTL Voting - Cyborg - Ke Chen
by yumeng bao 01 Sep '21

01 Sep '21

Dear Ke， Thanks for your contributions in cyborg ever since 2018, it’s been more than five release cycles past. By reviewing the patches you’ve committed and suggestions you’ve given to the team, it’s obvious you know cyborg very well, from the bottom to up level including GPU,FPGA,SmartNIC drivers to api, as well as interactions with nova and placement. Although I’m not contributing in cyborg anymore due to my job change, I do agree the five things you listed are important for cyborg, and I would like to see them done in the future. So it would be a big vote +1 from my side. Good luck. Regards， Yumeng > On Sep 1, 2021, at 8:29 PM, 陈克 <joykechen(a)163.com> wrote: > > Hi, > I would like to announce my candidacy for Cyborg PTL for the Yoga cycle, And hope to get your support. > > As an openstack enthusiast, I have been engaged in openstack for 4 years. > I was lucky to make friends with openstack projects such as Cyborg and Watcher in 2018. > > I met many interesting partners, such as Matt Riedemann, Sundar Nadathur, > Huangzhipeng, Liuli, Baoyumeng, Zhurong, Licanwei, Fengshaohe, Wangxinran, > Shogo, Zhangbailin, Songwenping and others. > > You let me see the charm of the community and rigorous working attitude, thanks. > > In the past two years, except for the watcher project, I spent most of my > time dealing with Cyborg. To be honest, I know cyborg very well. > > At present, Cyborg can manage FPGA, GPU, smartnic and other acceleration > devices. Users can create virtual machines with acceleration resources as > needed, which is a very cool thing. > > However, our Cyborg project still has many things to do. > In the next y version, I think there are mainly the following > five things to do: > 1. Optimize the interaction between Nova and cyborg. > The existing interface is synchronous call. On cyborg side, we can change > synchronization to asynchronous to save the landing time of concurrent > virtual machines. > > 2. Support for more drivers and more universal drivers. > At present, vgpu driver is not supported in cyborg. I hope to implement > it in the Yoga cycle. > > 3. Implementation of disable and enable acceleration devices. > > 4. More bugfix and documentation enhancements. > > 5. More virtual machine operation support, such as cold migration. > > I hope we can bring users a more exciting experience in the next Yoga version. > > Thanks, > Ke Chen(chenke)

1 0

[Kolla][Kolla-Ansible] Ironic Node Cleaning Failed
by Anirudh Gupta 01 Sep '21

01 Sep '21

Hi Team, We have deployed 2 node kolla ansible *12.0.0* in order to deploy openstack *wallaby* release. We have also enabled ironic in order to provision the bare metal nodes. On each server we have 3 nics - *eno1* - OAM for external connectivity and endpoint's publicURL - *eno2* - Mgmt for internal communication between various openstack services. - *ens2f0* - Data Interface Corresponding to this we have defined the following fields in globals.yml - kolla_base_distro: "centos" - kolla_install_type: "source" - openstack_release: "wallaby" - network_interface: "eno2" # MGMT interface - kolla_external_vip_interface: "eno1" # OAM Interface - kolla_internal_vip_address: "192.168.10.3" # MGMT Subnet free ip - kolla_external_vip_address: "10.0.1.136" # OAM subnet free IP - neutron_external_interface: "ens2f0" # Data Interface - enable_neutron_provider_networks: "yes" Note: Only relevant fields are being shown in this query Also, for ironic following fields have been defined in globals.yml - enable_ironic: "yes" - enable_ironic_neutron_agent: "{{ enable_neutron | bool and enable_ironic | bool }}" - enable_horizon_ironic: "{{ enable_ironic | bool }}" - ironic_dnsmasq_interface: "*ens2f0*" # Data interface - ironic_dnsmasq_dhcp_range: "20.20.20.10,20.20.20.100" - ironic_dnsmasq_boot_file: "pxelinux.0" - ironic_cleaning_network: "public1" - ironic_dnsmasq_default_gateway: "20.20.20.1" After successful deployment, a flat provider network with the name public1 is being created in openstack using the below commands: - openstack network create public1 --provider-network-type flat --provider-physical-network physnet1 - openstack subnet create subnet1 --network public1 --subnet-range 20.20.20.0/24 --allocation-pool start=20.20.20.10,end=20.20.20.100 --ip-version=4 --gateway=20.20.20.1 --dhcp Issue/Queries: - Is the configuration done in globals.yml correct or is there anything else that needs to be done in order to separate control and data plane traffic? - Also I have set automated_cleaning as "true" in ironic-conductor conatiner settings.But after creating the baremetal node, we run "node manage" command which runs successfully. Running "*openstack baremetal node provide <node id>"* command powers on the machine, sets the boot mode on Network Boot but no DHCP request for that particular mac is obtained on the controller. Is there anything I am missing that needs to be done in order to make ironic work? Note: I have also verified that the nic is PXE enabled in system configuration setting Regards Anirudh Gupta

3 15

[cinder][nova] fsfreeze hooks issues with cinder snapshot/backup
by Arthur Outhenin-Chalandre 01 Sep '21

01 Sep '21

Hello, We are trying to trigger an fsfreeze via a cinder backup or snapshot. We confirmed that the fsfreeze hooks are actually called with a nova snapshot with `/var/log/qga-fsfreeze-hook.log` in the VM, but we can't achieve the same thing with a cinder backup/snapshot attached to the same instance. We are using Wallaby, libvirt, RBD for cinder volumes and RBD as well for cinder-backup. According to this (old) spec [0], cinder should call the `quiesce()` method in nova during backup/snapshot. We looked in the cinder code and couldn't find any clear evidence that this method is actually called by cinder (but we may have missed something). We added some debug messages on quiesce/can_quiesce/require_quiesce/... in `nova/virt/libvirt/driver.py` and they are never called with a cinder backup/snapshot in our setup while they are (and succeed) if we do a nova snapshot. We are starting to suspect that something is missing in cinder, but it could very well be a problem with our setup as well... Does someone use this feature or know if it should be working/implemented? [0]: https://wiki.openstack.org/wiki/Cinder/QuiescedSnapshotWithQemuGuestAgent#C… Cheers, -- Arthur Outhenin-Chalandre

4 4

[cinder] Bug deputy report for week of 2021-01-09
by Sofia Enriquez 01 Sep '21

01 Sep '21

This is a bug report from 2021-18-08 to 2021-01-09. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Medium - https://bugs.launchpad.net/cinder/+bug/1942077 "Online database schema migrations to Xena always fail". - https://bugs.launchpad.net/cinder/+bug/1942218 "Huawei FC driver cannot co-locate cinder-volume service with computer". Unassigned. - https://bugs.launchpad.net/cinder/+bug/1942095 "powerflex driver: no volume caching after 126 volumes". Assigned to Vladislav Belogrudov. - https://bugs.launchpad.net/cinder/+bug/1942090 "SolidFire retype fails due to volume status as retyping and not available". Assigned to Fábio Oliveira. - https://bugs.launchpad.net/cinder/+bug/1941694 "IBM SVF driver: Detaching from second instance fails for multi-attach volumes". Unassigned. Low - https://bugs.launchpad.net/cinder/+bug/1942301 "[synology] embed driver uses the insecure MD5 algorithm". Unassigned. - https://bugs.launchpad.net/cinder/+bug/1941746 "JovianDSS: incorrect operation of ensure export function". Unassigned. - https://bugs.launchpad.net/cinder/+bug/1930267 "ssh exec timeout". Unassigned. Incomplete - https://bugs.launchpad.net/cinder/+bug/1942154 " Backup availability_zone is None". Unassigned. Cheers, -- L. Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

[all][elections][ptl] PTL Voting - Cyborg - Ke Chen
by 陈克 01 Sep '21

01 Sep '21

Hi, I would like to announce my candidacy for Cyborg PTL for the Yoga cycle, And hope to get your support. As an openstack enthusiast, I have been engaged in openstack for 4 years. I was lucky to make friends with openstack projects such as Cyborg and Watcher in 2018. I met many interesting partners, such as Matt Riedemann, Sundar Nadathur, Huangzhipeng, Liuli, Baoyumeng, Zhurong, Licanwei, Fengshaohe, Wangxinran, Shogo, Zhangbailin, Songwenping and others. You let me see the charm of the community and rigorous working attitude, thanks. In the past two years, except for the watcher project, I spent most of my time dealing with Cyborg. To be honest, I know cyborg very well. At present, Cyborg can manage FPGA, GPU, smartnic and other acceleration devices. Users can create virtual machines with acceleration resources as needed, which is a very cool thing. However, our Cyborg project still has many things to do. In the next y version, I think there are mainly the following five things to do: 1. Optimize the interaction between Nova and cyborg. The existing interface is synchronous call. On cyborg side, we can change synchronization to asynchronous to save the landing time of concurrent virtual machines. 2. Support for more drivers and more universal drivers. At present, vgpu driver is not supported in cyborg. I hope to implement it in the Yoga cycle. 3. Implementation of disable and enable acceleration devices. 4. More bugfix and documentation enhancements. 5. More virtual machine operation support, such as cold migration. I hope we can bring users a more exciting experience in the next Yoga version. Thanks, Ke Chen(chenke)

1 0

[release][heat][ironic][requirements][swift][OpenStackSDK][vitrage][barbican][heat][horizon][ironic][neutron][QA] Cycle With Intermediary Unreleased Deliverables
by Herve Beraud 01 Sep '21

01 Sep '21

Hello teams Quick reminder that we'll need a release very soon for a number of deliverables following a cycle-with-intermediary release model but which have not done *any* release yet in the Xena cycle: ansible-role-atos-hsm ansible-role-lunasa-hsm ansible-role-thales-hsm heat-agents horizon ironic-prometheus-exporter ironic-python-agent-builder ironic-ui ovn-octavia-provider patrole python-openstackclient requirements swift tempest vitrage-dashboard vitrage Those should be released ASAP, and in all cases before the week of 13 September, 2021, so that we have a release to include in the final Xena release. Thanks for your attention. -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud

4 6