June 2022 - openstack-discuss - lists.openstack.org

[interop][refstack][summit] Interoperability testing feedback
by Martin Kopec 14 Jun '22

14 Jun '22

Hi everyone, we got great feedback during the summit, if you wanna check the topics we discussed, see this etherpad [1]. Feel also free to add additional concerns with the current interop processes. I was happy to see that many of you are thinking about interoperability and want to improve the testing in this regard. IWG has been seeking feedback from operators as well as users for some time now to make sure it helps with interoperability as much as possible. If you haven't already, please fill out this short (2-3 min) questionnaire [2]. Public Cloud SIG hosted an interesting session in terms of interoperability testing as well, see their etherpad [3]. I'm sure that you'll see further communication in that regard soon. I look forward to future collaboration between IWG and Public Cloud SIG. [1] https://etherpad.opendev.org/p/Berlin2022-Forum-InteropWG [2] https://forms.gle/YW95Hb2LoppaiLxv8 [3] https://etherpad.opendev.org/p/berlin-summit-future-public-cloud-sig Regards, -- Martin Kopec Senior Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 0

[keystone] Getting a token from existing token
by hai wu 14 Jun '22

14 Jun '22

Is it possible to get a new token from an existing token, and at the same time, to have the new token expiration time extended? (token renewal from existing token) It seems the token expiration time is not extended when trying the same curl command as shown in section "Getting a token from a token" in this doc: https://docs.openstack.org/keystone/train/api_curl_examples.html. Quote: curl -i \ -H "Content-Type: application/json" \ -d ' { "auth": { "identity": { "methods": ["token"], "token": { "id": "'$OS_TOKEN'" } } } }' \ "http://localhost:5000/v3/auth/tokens" ; echo

1 0

Re: snapshot problem
by Eugen Block 14 Jun '22

14 Jun '22

You should respond to the list so other users can try to support you. So nova is trying to live snapshot the instance: > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > 4dbffaa9c14e401c8c210e23ebe0ab7b ef940663426b4c87a751afaf13b758e0 - > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] > instance snapshotting > [...] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning > live snapshot process But I don't see any 'rbd snap create' command. Either the rbd image doesn't support it or there is some setting to keep all rbd images "flat". Can you check any relevant configs you might have in nova? Also can you show the output of 'rbd info <pool>/25c8d676-e20a-4238-a45c-d51daa62b941_disk' ? Then to test if the underlying rbd functions work as expected you could try to create a live snapshot manually: rbd snap create <pool>/25c8d676-e20a-4238-a45c-d51daa62b941_disk@test-snap And paste any relevant output here. Zitat von Parsa Aminian <p.aminian.server(a)gmail.com>: > Its not working for any instances and all of them are paused . I enable > debug logs please check the logs : > > 2022-06-12 16:16:13.478 7 DEBUG nova.compute.manager > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync for > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > 2022-06-12 16:16:13.506 7 DEBUG oslo_concurrency.lockutils [-] Lock > "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" > :: waited 0.000s inner > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > 2022-06-12 16:16:43.562 7 DEBUG nova.compute.resource_tracker > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute host > and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > 2022-06-12 16:25:55.104 7 DEBUG nova.compute.manager > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 4dbffaa9c14e401c8c210e23ebe0ab7b > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 4dbffaa9c14e401c8c210e23ebe0ab7b > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > 63426b4c87a751afaf13b758e0 - default default] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning live snapshot process > default default] Lazy-loading 'pci_devices' on Instance uuid > 25c8d676-e20a-4238-a45c-d51daa62b941 obj_load_attr > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > 2022-06-12 16:25:57.250 7 DEBUG nova.objects.instance > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 4dbffaa9c14e401c8c210e23ebe0ab7b > ef940663426b4c87a751afaf13b758e0 - default default] Lazy-loading > 'pci_devices' on Instance uuid 25c8d676-e20a-4238-a45c-d51daa62b941 > obj_load_attr > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > 2022-06-12 16:25:57.317 7 DEBUG nova.virt.driver [-] Emitting event > <LifecycleEvent: 1655034957.3158934, 25c8d676-e20a-4238-a45c-d51daa62b941 > => Paused> emit_event > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > 2022-06-12 16:25:57.318 7 INFO nova.compute.manager [-] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) > 2022-06-12 16:25:57.389 7 DEBUG nova.compute.manager > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > 2022-06-12 16:25:57.395 7 DEBUG nova.compute.manager > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power state > after lifecycle event "Paused"; current vm_state: active, current > task_state: image_pending_upload, current DB power_state: 1, VM > power_state: 3 handle_lifecycle_event > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > 2022-06-12 16:25:57.487 7 INFO nova.compute.manager > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the instance > has a pending task (image_pending_upload). Skip. > 2022-06-12 16:26:02.039 7 DEBUG oslo_concurrency.processutils > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 4dbffaa9c14e401c8c210e23ebe0ab7b > ef940663426b4c87a751afaf13b758e0 - default default] Running cmd > (subprocess): qemu-img convert -t none -O raw -f raw > rbd:vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad > execute > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:384 > 2022-06-12 16:26:17.075 7 DEBUG nova.virt.driver [-] Emitting event > <LifecycleEvent: 1655034962.0316682, 25c8d676-e20a-4238-a45c-d51daa62b941 > => Stopped> emit_event > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > INFO nova.compute.manager [-] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - - - > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > _get_power_state > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - - - > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing > instance power state after lifecycle event "Stopped"; current vm_state: > active, current task_state: image_pending_upload, current DB power_state: > 1, VM power_state: 4 handle_lifecycle_event > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > INFO nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - - - - > -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state > the instance has a pending task (image_pending_upload). Skip. > 2022-06-12 16:26:18.539 7 DEBUG nova.compute.manager > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync for > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > 2022-06-12 16:26:18.565 7 DEBUG oslo_concurrency.lockutils [-] Lock > "25c8d676-e20a-4238 > -a45c-d51daa62b941" acquired by > "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" > :: waited 0.000s inner > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > 2022-06-12 16:26:18.566 7 INFO nova.compute.manager [-] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the instance > has a pending task (image_pending_upload). Skip. > 2022-06-12 16:26:18.566 7 DEBUG oslo_concurrency.lockutils [-] Lock > "25c8d676-e20a-4238-a45c-d51daa62b941" released by > "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" > :: held 0.001s inner > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:371 > 2022-06-12 16:26:25.769 7 DEBUG oslo_concurrency.processutils > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 4dbffaa9c14e401c8c210e23ebe0ab7b > ef940663426b4c87a751afaf13b758e0 - default default] CMD "qemu-img convert > -t none -O raw -f raw > rbd:vms/25c8d676-e20a-4238-a45c-d51daa6b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad" > returned: 0 in 23.730s execute > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot > extracted, beginning image upload > 2022-06-12 16:26:27.981 7 DEBUG nova.virt.driver > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] Emitting event > <LifecycleEvent: 1655034987.9807608, 25c8d676-e20a-4238-a45c-d51daa62b941 > => Started> emit_event > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > 2022-06-12 16:26:27.983 7 INFO nova.compute.manager > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) > [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > _get_power_state > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power state > after lifecycle event "Started"; current vm_state: active, current > task_state: image_pending_upload, current DB power_state: 1, VM > power_state: 1 handle_lifecycle_event > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > 2022-06-12 16:26:28.173 7 INFO nova.compute.manager > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Resumed (Lifecycle Event > 2022-06-12 16:29:00.859 7 DEBUG oslo_concurrency.lockutils > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Acquired lock > "refresh_cache-25c8d676-e20a-4238-a45c-d51daa62b941" lock > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:266 > 25c8d676-e20a-4238-a45c-d51daa62b941] Forcefully refreshing network info > cache for instance _get_instance_nw_info > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:1833 > 2022-06-12 16:29:03.278 7 DEBUG nova.network.neutron > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] Updating instance_info_cache with > network_info: [{"id": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", "address": > "fa:16:3e:ca:00:d9", "network": {"id": > "b86c8304-a9bd-4b39-b7fc-f70ffe76f2a8", "bridge": "br-int", "label": > "External_Network", "subnets": [{"cidr": "141.11.42.0/24", "dns": > [{"address": "8.8.8.8", "type": "dns", "version": 4, "meta": {}}, > {"address": "217.218.127.127", "type": "dns", "version": 4, "meta": {}}], > "gateway": {"address": "141.11.42.1", "type": "gateway", "version": 4, > "meta": {}}, "ips": [{"address": "141.11.42.37", "type": "fixed", > "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, > "meta": {}}], "meta": {"injected": true, "tenant_id": > "ef940663426b4c87a751afaf13b758e0", "mtu": 1500, "physical_network": > "physnet1", "tunneled": false}}, "type": "ovs", "details": {"connectivity": > "l2", "port_filter": true, "ovs_hybrid_plug": true, "datapath_type": > "system", "bridge_name": "br-int"}, "devname": "tapaa2fdd7d-ad", > "ovs_interfaceid": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", "qbh_params": > null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": > {}, "preserve_on_delete": false, "meta": {}}] > update_instance_cache_with_nw_info > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:117 > nstance 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this > compute host and has allocations in placement: {'resources': {'VCPU': 1, > 'MEMORY_MB': 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > 2022-06-12 16:33:37.595 7 INFO nova.compute.manager > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 4dbffaa9c14e401c8c210e23ebe0ab7b > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] Took 461.98 seconds to snapshot the > instance on the hypervisor. > 2022-06-12 16:36:16.459 7 DEBUG nova.compute.manager > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync for > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > Lock "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" > :: waited 0.000s inner > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > 2022-06-12 16:37:05.365 7 DEBUG nova.compute.resource_tracker > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute host > and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > 2022-06-12 09:42:32.687 7 INFO nova.compute.manager > [req-e79e4177-4712-4795-91da-853bc524fac0 93fb420b3c604d4fae408b81135b58e9 > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > > On Sun, Jun 12, 2022 at 3:36 PM Eugen Block <eblock(a)nde.ag> wrote: > >> Have you tried with debug logs? Has it worked with live snapshots >> before for other instances or did it never work and all snapshots were >> "cold"? >> >> Zitat von Parsa Aminian <p.aminian.server(a)gmail.com>: >> >> > Hi >> > kolla-ansible victoria version with ceph backend without volume >> > >> > On Sun, Jun 12, 2022 at 12:45 PM Eugen Block <eblock(a)nde.ag> wrote: >> > >> >> Hi, >> >> >> >> can you share more details about your environment? Which openstack >> >> version is it? What is the storage backend? In earlier releases there >> >> was an option: >> >> >> >> #disable_libvirt_livesnapshot = false >> >> >> >> but this option has been deprecated. But if you're on an older version >> >> that could explain it. >> >> >> >> Zitat von Parsa Aminian <p.aminian.server(a)gmail.com>: >> >> >> >> > When I snapshot from the instance , server will gone away and its not >> >> > reachable until the snapshot is complete here is the logs : >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> >> > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) >> >> > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the >> >> instance >> >> > has a pending task (image_pending_upload). Skip. >> >> > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) >> >> > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 >> >> 93fb420b3c604d4fae408b81135b58e9 >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, beginning >> image >> >> > upload >> >> > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>

2 8

[VICTORIA][IRONIC] - Can inspect but not deploy
by Gaël THEROND 14 Jun '22

14 Jun '22

Hi everyone! I'm dealing with a strange issue today, we deployed IRONIC on a VICTORIA platform, we activated the dnsmasq pxe filtering option at the inspector level, it works great as only IRONIC listed hosts are then served by the dnsmasq DHCP as those hosts are allowed using the dhcp hosts dir. BUT, I'm having a weird issue now. All my nodes are able to get an IP from the DHCP at the INSPECTION step, however, as soon as the inspection step is finished, the mac related file is once again filled with "<mac>,ignore" which prohibits further operations. This means as soon as I put that node as available and try to deploy an "instance" on it (provision the host), it doesn't work as dnsmasq reject the host boot DHCP requests. So, is there a way to instruct the ironic-conductor to edit/allow the host the way the inspector is able to manipulate this file? Did I missed something? Thanks a lot!

3 6

[Nova] Hardware Architecture Emulation - Summit forum - Emulation beyond just Local QEMU -outcomes
by Race, Jonathan 14 Jun '22

14 Jun '22

Great to see everyone at the summit! For those not in attendance and interested, see the below etherpad that is being used to collect input and requirements as our team moves forward with work. I have also included the outcomes that I have identified. https://etherpad.opendev.org/p/Emulation_going_beyond_local_QEMU Berlin Forum Outcomes I appreciate all the feedback during the forum session, and it was great to have active participation on how to grow and shape this unique feature. We discussed the feature, what it allows, how to actually leverage it, and discussed about further refining the applicable use cases so that operators and users can effectively use it and to help drive interest. A prime example was of a company wanting to shift their infra from x86_64 to arm for the sake of power consumption, and whether this feature could lend its hand to perform some initial fact finding and testing. The answer is yes, you can spin up some of the infra (in your desired arch) on top of your existing deployment and perform your tests and validations. One of the great items that came out of the discussion was that this can not only be to support education, research, and training but it can act as a great 1st step within planning. Meaning when you are going from the drawing board to implementation, you can now model and design your network environments within the cloud for IT and OT systems. This is where refining the use cases really came to light, as our organization leverages OpenStack as a private cloud for hosting extremely customizable network environments (i.e. enterprise iT, IOT, Critical infrastructure, medical devices, etc.) While the feature itself is overall in the experimental phase, it is a great start. This etherpad will remain available and feel free to add to it, as we leverage this year to fact find and identify all further requirements going into the AA release cycle. A snapshot of the discussion points can be found below, and ill be posting the recorded session within the week. * Most operators and providers are running a few releases behind so getting accurate feedback at this time is still ongoing due to this feature only being available in yoga and newer. * Pursuit of additional architectures to be supported (riscv, etc.) * This will require custom firmware support * evaluate current AMI/AKI/ARI functionalities (add ability for leveraging glance if able, and add additional security) * Adding capability for trusted signers of OOVMF file descriptors * evaluating current work related to edk2 oovmf file descriptors * https://github.com/riscv-admin/riscv-uefi-edk2-docs * Expansion and documentation of a valid support matrix * something similar to: (arch x | feat 1 | supported | temptest test 1) * Evaluate requirements for leveraging virtio in relation to scsi/sata devices similar to pci passthrough capabilities Jonathan Race Cyber Range Engineer | Georgia Cyber Range GEORGIA CYBER CENTER 100 Grace Hopper Lane | Augusta, Georgia | 30901 (c) 762-716-7733 www.gacyberrange.org<http://www.gacybercenter.org/> [cid:image001.png@01D87FD1.CF558390]

1 0

[neutron] bug deputy (week of Jun 6)
by Akihiro Motoki 14 Jun '22

14 Jun '22

Hi, I was a bug deputy in neutron. Here is a summary. ## Untriaged * https://bugs.launchpad.net/neutron/+bug/1978088 After ovs-agent restart, table=21 and table=22 on br-tun openflow table is missing This happens when a large number of agents are restarted, but it looks like deeper investigations are needed to understand the situation. (liuyolong and ralonsoh already helped) ## In Progress * https://bugs.launchpad.net/neutron/+bug/1977819 [OVS][QoS] "delete_minimum_bandwidth_queue" uses the incorrect Port UUID High, In Progress * https://bugs.launchpad.net/neutron/+bug/1978519 Create auto allocated topology encounter error when enable ndp_proxy service plugin ## RFE * https://bugs.launchpad.net/neutron/+bug/1978039 [ovn] Floating IP adds distributed attributes ## Fix Released * https://bugs.launchpad.net/neutron/+bug/1977752 [OVS][QoS] Minimum bandwidth backend enforcement does not provide max-rate values * https://bugs.launchpad.net/neutron/+bug/1977969 [OVN] DB sync tool: local variable 'members_to_verify' referenced before assignment" * https://bugs.launchpad.net/neutron/+bug/1978035 remove unused updated_at parameter for AgentCache.update * https://bugs.launchpad.net/neutron/+bug/1978389 Basic networking in Neutron - max VLAN should be 4094, not 4095 * https://bugs.launchpad.net/neutron/+bug/1977831 Response time increasing on new subnets over same network Thanks, Akihiro Motoki (amotoki)

1 0

simple build fails to allocate network interface
by tim+openstack.org＠coote.org 14 Jun '22

14 Jun '22

Hullo I’m trying to spin up a simple OS environment in a single vm and think that I’m making some silly mistake, but cannot spot it. Any hints would be greatly expected. I’ve tried this on a couple of laptops, with the same result, so I suspect that it’s something to do with how I’ve set up my vm, but I cannot identify it. (I think that there may be a few other devstack issues, but I’ve got to get it working to log any bugs). I’m using a vagrant box and VirtualBox on x86 Macs (Monterey), and the devstack install/build process. Everything seems to install, but when I try: `openstack server create --flavor 42 --image cirros-0.5.2-x86_64-disk --nic net-id=2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 --security-group default wibble` The network id is identified from this: “”" [vagrant@localhost devstack]$ openstack network list +--------------------------------------+---------+----------------------------------------------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+----------------------------------------------------------------------------+ | 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f | public | 1bbbfa3c-8e0a-483f-a084-4e38241b3315, eecfc603-c057-4b28-90bd-950a47345410 | | 2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 | private | e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd, f915b88c-9988-4e1f-9060-a6295465699a | | fab3fb15-cbd2-45fe-8451-3f0063d59454 | shared | 97e6d8e6-0f57-48bc-8f1d-e30587086153 | +--------------------------------------+---------+——————————————————————————————————————+ “”” Using a public network fails with an error that this isn’t allowed. With the private network above, the failure throws out the following errors. “”" [vagrant@localhost devstack]$ sudo journalctl -f |grep ERROR Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Instance failed to spawn: nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7510, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] guest = self._create_guest( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] next(self.gen) Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 556, in wait_for_instance_event Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._wait_for_instance_events( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 468, in _wait_for_instance_events Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] actual_event = event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 433, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] instance_event = self.event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] result = hub.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] return self.greenlet.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] eventlet.timeout.Timeout: 300 seconds Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] During handling of the above exception, another exception occurred: Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2728, in _build_resources Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] yield resources Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2487, in _build_and_run_instance Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self.driver.spawn(context, instance, image_meta, Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4344, in spawn Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._create_guest_with_network( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7528, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] raise exception.VirtualInterfaceCreateException() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Failed to allocate network(s): nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7510, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] guest = self._create_guest( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] next(self.gen) Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 556, in wait_for_instance_event Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._wait_for_instance_events( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 468, in _wait_for_instance_events Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] actual_event = event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 433, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] instance_event = self.event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] result = hub.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] return self.greenlet.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] eventlet.timeout.Timeout: 300 seconds Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] During handling of the above exception, another exception occurred: Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2487, in _build_and_run_instance Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self.driver.spawn(context, instance, image_meta, Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4344, in spawn Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._create_guest_with_network( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7528, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] raise exception.VirtualInterfaceCreateException() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Build of instance 20e1b5bf-130a-48f9-af4e-9b4c7a475315 aborted: Failed to allocate the network(s), not rescheduling.: nova.exception.BuildAbortException: Build of instance 20e1b5bf-130a-48f9-af4e-9b4c7a475315 aborted: Failed to allocate the network(s), not rescheduling. “”" The Vagrantfile (commented out lines relate to trying this with Ubunutu, which failed the same way, and attempts to use multiple vms): “”" # -*- mode: ruby -*- # vi: set ft=ruby : $script = <<-SHELL dnf update -y #apt update #apt upgrade echo "updated" systemctl disable firewalld systemctl stop firewalld systemctl disable NetworkManager systemctl stop NetworkManager systemctl enable network systemctl start network #dnf config-manager --enable powertools #dnf install -y centos-release-openstack-yoga dnf update -y dnf install -y git python3-pip #dnf install -y openstack-packstack #packstack --provision-demo=n --os-neutron-ml2-type-drivers=vxlan,flat,vlan --gen-answer-file=packstack-answers.txt #sed -i -e 's:10.0.2.15:10.1.0.10:' packstack-answers.txt # not for centos 9 #dnf install -y network-scripts dnf install -y net-tools #cat /vagrant/answers.addon >> packstack-answers.txt # don't want this, but httpd won't start without it setenforce 0 pip3 install invoke #packstack --allinone cd /vagrant # permissions all wrong: ./go.sh # need su - vagrant cd /vagrant && ./go.sh (?) # cd devstack # not working: wrong user ./stack.sh SHELL Vagrant.configure(2) do |config| config.vm.box = "eurolinux-vagrant/centos-stream-9" #config.vm.box = "hashicorp/bionic64" # machines = { # 'node1.example.dd' #'node1.example.dd' => { :ip => '10.1.0.10'}, # 'node2.example.dd' => { :ip =>'10.1.0.12'}, # } config.hostmanager.enabled = true config.hostmanager.manage_host = true config.hostmanager.manage_guest = true config.hostmanager.ignore_private_ip = false config.hostmanager.include_offline = true config.ssh.pty = true config.vm.provision "shell", inline: $script config.vm.network "forwarded_port", guest: 80, host: 8080 # machines.each do | hostname, attrs| # config.vm.define hostname do |machine| # machine.vm.hostname = hostname # machine.vm.network :private_network, :ip => attrs[:ip] # machine.vm.provider "virtualbox" do | v | config.vm.provider "virtualbox" do | v | #v.memory = "4096" #v.memory = "8192" v.memory = "9216" v.cpus = "2" end #end # end end “”” I think that the vm has enough RAM, although there is minimal swap being used, but I think that this is not significant as there is much more RAM used for caching files. Any obvious hints as to why the spin up fails to create the NIC - or somewhere to look for further and better details? Tim

4 6

[manila] CephFS NFS high availability cluster
by CHANU ROMAIN 13 Jun '22

13 Jun '22

Hello, I added Manila service with CephFS NFS driver to my openstack cluster. Everything works fine but I would like to add 2 nfs-ganesha servers to ensure high availability to the service. I configured haproxy to forward 2049 to ganesha backend but Manila cephFS NFS provides only IP restriction and see only haproxy's IP address. To make it work you have to add haproxy to allowed ip but it means everyone can access the share. So currently the only way I found out is to use pacemaker to set public vip to a running nfs-ganesha node. Could you confirm that is not possible to provide an active/active nfs-ganesha cluster with manila cephfs NFS driver? Best regards, Romain

2 1

[Magnum resize Master node root volume/docker volume or resize flavor]
by Nabeel Tariq Khan 13 Jun '22

13 Jun '22

Hi, Is it possible to resize the root volume or docker volume or resize the flavor of master or minion node on openstack magnum? regards, Nabeel Tariq Khan

1 0

Regarding Policy.json entries for glance image update not working for a user
by Adivya Singh 13 Jun '22

13 Jun '22

Hi Team, I have a use case where I have to give a user restriction on updating the image properties as a member. I have created a policy Json file and give the modify_image rule to the particular role, but still it is not working "modify_image": "role:user", This role is created in OpenStack. but still it is failing while updating properties with a particular user assigned to a role as "access denied" and unauthorized access Regards Adivya Singh

3 3

[ironic] No weekly meeting today (June 13)
by Iury Gregory 13 Jun '22

13 Jun '22

Hello Ironicers, I'm still on my way back home from the OIS, I'll not be able to run the upstream meeting I've asked in the irc and people are ok with skipping today's meeting, see you next week! -- *Att[]'s* *Iury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Ironic PTL * *Senior Software Engineer at Red Hat Brazil* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory(a)gmail.com <iurygregory(a)gmail.com>*

1 0

[neutron][networking-l2gw][puppet] Status of networking-l2gw project
by Takashi Kajinami 13 Jun '22

13 Jun '22

Hello team, # I understand networking-l2gw is not under neutron stadium # but adding [neutron] tag hoping it would attract more attention. We found a recent change in neutron[1] broke networking-l2gw[2], and some jobs in puppet repos and rdo, with l2gw enabled, are failing. I proposed a potential fix for the bug[3], but I noticed the project looks unmaintained according to the following points. - No change has been merged for more than 1 year - No release was created since Wallaby - The zuul setting still points the victoria job template (openstack-python3-victoria-jobs-neutron) I'm wondering if anybody is actually interested in maintaining the project. If not, I'll propose retiring the plugin support from puppet (and TripleO because it requires puppet implementation). I appreciate any feedback about this. [1] https://review.opendev.org/c/openstack/neutron/+/837392 [2] https://bugs.launchpad.net/networking-l2gw/+bug/1977980 [3] https://review.opendev.org/c/x/networking-l2gw/+/845141 Thank you, Takashi

2 1

Re: snapshot problem
by Eugen Block 12 Jun '22

12 Jun '22

Have you tried with debug logs? Has it worked with live snapshots before for other instances or did it never work and all snapshots were "cold"? Zitat von Parsa Aminian <p.aminian.server(a)gmail.com>: > Hi > kolla-ansible victoria version with ceph backend without volume > > On Sun, Jun 12, 2022 at 12:45 PM Eugen Block <eblock(a)nde.ag> wrote: > >> Hi, >> >> can you share more details about your environment? Which openstack >> version is it? What is the storage backend? In earlier releases there >> was an option: >> >> #disable_libvirt_livesnapshot = false >> >> but this option has been deprecated. But if you're on an older version >> that could explain it. >> >> Zitat von Parsa Aminian <p.aminian.server(a)gmail.com>: >> >> > When I snapshot from the instance , server will gone away and its not >> > reachable until the snapshot is complete here is the logs : >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) >> > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the >> instance >> > has a pending task (image_pending_upload). Skip. >> > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) >> > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver >> > [req-e79e4177-4712-4795-91da-853bc524fac0 >> 93fb420b3c604d4fae408b81135b58e9 >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, beginning image >> > upload >> > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) >> >> >> >> >>

1 0

[nova][openstack-ansible] "No Valid Host Found" when running on VirtualBox
by Dave Hall 12 Jun '22

12 Jun '22

Hello, I'm trying to run a test/learning cluster on VirtualBox VMs deployed with Openstack-Ansible prior to deploying on real hardware. The playbooks succeed, and the basic functional checks all look good. However, I am unable to deploy an instance due to 'No Valid Host Found'. I have seen that this might be because VirtualBox does not support KVM. But I am also perplexed - I am able to deploy instances on an AIO running in VirtualBox on the same physical host. If there is an easy fix for this please point me in the right direction. If I need to destroy and re-clone my VMs I can try that as well. In an effort to learn, though, I'd really like to learn how to adjust the current deployment. Thanks. -Dave -- Dave Hall Binghamton University kdhall(a)binghamton.edu

2 5

snapshot problem
by Parsa Aminian 12 Jun '22

12 Jun '22

When I snapshot from the instance , server will gone away and its not reachable until the snapshot is complete here is the logs : 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting 2022-06-12 09:42:34.755 7 INFO nova.compute.manager [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) 2022-06-12 09:42:34.995 7 INFO nova.compute.manager [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the instance has a pending task (image_pending_upload). Skip. 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver [req-e79e4177-4712-4795-91da-853bc524fac0 93fb420b3c604d4fae408b81135b58e9 ef940663426b4c87a751afaf13b758e0 - default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, beginning image upload 2022-06-12 09:43:08.778 7 INFO nova.compute.manager [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event)

2 1

[all][tc] What's happening in Technical Committee: summary 10 June 2022: Reading: 5 min
by Ghanshyam Mann 11 Jun '22

11 Jun '22

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We cancelled this week's meeting. The next TC weekly meeting will be on 16 June Thursday at 15:00 UTC, feel free to add the topic to the agenda[1] by 15 June. 2. What we completed this week: ========================= * Added community infrastructure contributor help in upstream investment[2] 3. Activities In progress: ================== TC Tracker for Zed cycle ------------------------------ * Zed tracker etherpad includes the TC working items[3], many items are in-progress. Open Reviews ----------------- * Five open reviews for ongoing activities[4]. New release cadence "SLURP" open items -------------------------------------------------- 1. release notes strategy: Brian proposal for ""SLURP release notes approach is up for review[5]. Improve project governance --------------------------------- Slawek has the framework proposal up and is under review[6]. New ELK service dashboard: e-r service ----------------------------------------------- No update this week. Consistent and Secure Default RBAC ------------------------------------------- We called for the operator's feedback on RBAC new directions, I sent an email about it[7] and also created etherpad to collect feedback in a central place[8]. My humble request is to use this central etherpad for feedback/discussion otherwise it might be difficult for me to collect/know feedback conclusions from a lot of 100s places. We have feedback from the berlin summit but we still encourage operators/local events or so keep adding the feedback and your view on RBAC. Please note that operator feedback is very important for us to proceed on 'scope' in SRBAC. If you are operator or know anyone around you, we request you to provide feedback. 2021 User Survey TC Question Analysis ----------------------------------------------- No update on this. The survey summary is up for review[9]. Feel free to check and provide feedback. Zed cycle Leaderless projects ---------------------------------- No updates on this. Only Adjutant project is leaderless/maintainer-less. We will check Adjutant's the situation again on ML and hope Braden will be ready with their company side permission[10]. Fixing Zuul config error ---------------------------- Requesting projects with zuul config error to look into those and fix them which should not take much time[11][12]. Project updates ------------------- * None 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[13]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [14] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/843749 [3] https://etherpad.opendev.org/p/tc-zed-tracker [4] https://review.opendev.org/q/projects:openstack/governance+status:open [5] https://review.opendev.org/c/openstack/project-team-guide/+/843457 [6] https://review.opendev.org/c/openstack/governance/+/839880 [7] http://lists.openstack.org/pipermail/openstack-discuss/2022-June/028878.html [8] https://etherpad.opendev.org/p/rbac-operator-feedback [9] https://review.opendev.org/c/openstack/governance/+/836888 [10] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027626.ht… [11] https://etherpad.opendev.org/p/zuul-config-error-openstack [12] http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028603.html [13] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [14] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann

1 0

Openstack-Ansible vs Kolla-Ansible -Xena or later
by Dave Hall 10 Jun '22

10 Jun '22

Hello, My question is about OpenStack-Ansible vs. Kolla-Ansible. While I am sensitive to the effort that has been put into both of these projects, what I really need right now is the most fool-proof way to deploy and manage a small production cluster for academic instructional use. I have been working for a couple months with OpenStack-Ansible to deploy a small (3-node) test cluster on VirtualBox VMs in preparation for a larger deployment on real hardware - 6 to 10 nodes. (Details: Debian 11 deployment, Debian 10 infra, compute, and storage nodes) It has been slow going, at least partially due to some issues and limitations with VirtualBox. However, deploying a test cluster on VMs still seems preferable to just diving deployment on real hardware and going through multiple scrubs/reinstalls. Today I saw that Mark asked a question about Kolla-Ansible. So, as a 'beginner', should I shift and look at Kolla-Ansible, or should I stay the course and continue with Openstack-Ansible? What are the pros and cons of each? Thanks. -Dave -- Dave Hall Binghamton University kdhall(a)binghamton.edu

6 10

[TripleO] move stable/victoria tripleo to End Of Life OK?
by Marios Andreou 10 Jun '22

10 Jun '22

Hello TripleO the tripleo-ci team proposes that we transition the stable/victoria branch across all tripleo repos [1] to End Of Life [2]. Hopefully this is not a suprise to anyone as it has been discussed for a while (most recent discussion I can point to is at Zed PTG [3]). The stable/victoria branch was moved to Extended Maintenance with [4] so we can no longer make any releases for these repos. Victoria is not an active branch for TripleO (not an FFU branch) so we are just merging patches there on the way back to train. Are there any concerns, objections or any other comment? I'll give this a few days and if there are no further comments I'll post the EOL proposal and update here towards the end of next week, regards, marios [1] https://releases.openstack.org/teams/tripleo.html#victoria [2] https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… [3] https://etherpad.opendev.org/p/tripleo-zed-ci-load [4] https://review.opendev.org/c/openstack/releases/+/837982

1 2

[doc] question/enhancement-request on Sphinx theme used by openstack docs
by Waines, Greg 10 Jun '22

10 Jun '22

Hey ... I am on the technical steering committee for the openstack starlingx team and also provide primary technical review for our docs at https://docs.starlingx.io/ . docs.starlingx.io basically uses the openstack docs team's Sphinx theme for building our html docs ... see https://opendev.org/starlingx/docs . docs.starlingx.io is a very large doc suite. We are interested in improvements to the Table-of-Contents (TOC) navigation and presentation ... but we have limited Sphinx expertise. Specifically we are looking for the TOC support to have - more subsection header levels in TOC // such that user has full view of where he currently is in the doc hierarchy - highlighting to indicate currently selected section - separate scroll bar for TOC // ... so you don't lose sight of selected TOC selection when you scroll down on the main page which has large content. Have you ever considered such doc improvements? Did you have Sphinx expertise to do such improvements ? let us know, Greg.

2 1

[nova] local LVM volume on compute hosts
by Paul Harrison 10 Jun '22

10 Jun '22

I have installed a small test cluster using openstack-ansible (xena) and would like to use local LVM volumes for storing instance volumes - and following https://docs.openstack.org/nova/latest/configuration/config.html and (e.g. https://cloudnull.io/2017/12/nova-lvm-an-iop-love-story/) I have the libvirt section of my /etc/nova/nova.conf as below. -8<--- [libvirt] inject_partition = -2 inject_password = False inject_key = False virt_type = kvm live_migration_with_native_tls = true live_migration_scheme = tls live_migration_inbound_addr = 172.29.236.5 hw_disk_discard = ignore disk_cachemodes = none images_type = lvm images_volume_group = nova volume_clear = zero -8<—— however, the nova service continues to use cinder volumes from my storage host to store instances. Is there something else that needs to be configured to make this work, or is it obsolete?

2 7

[magnum] Docker Swarm errors on Yoga Ubuntu using Install Guide cut-n-paste
by Vince Mulhollon 10 Jun '22

10 Jun '22

I'm trying to use Magnum on a fresh Yoga install on Ubuntu to run some non-load balanced docker swarm workloads. Its a long story but the parts of the swarm workload are heterogeneous so a cluster would just get in the way. Everything else about the cluster is working fine, all is well, etc. I'm a small time VMware and OS admin with years of experience but absolute zero experience with Magnum. Also I have "some" experience with Docker Swarm clusters mostly created and assembled with Ansible. So I know quite well how to look at log files or configs in a general sense, but I have no idea what to expect from Magnum specifically as a "glue" technology. I've followed every detail of the "Magnum Installation Guide" for Yoga as seen at: https://docs.openstack.org/magnum/yoga/install/index.html When provisioning a docker swarm, I very literally cut and paste from: https://docs.openstack.org/magnum/yoga/install/launch-instance.html#provisi… cluster creation fails every time with "Property allowed_cidrs not assigned" exactly as seen on this mailing list post from 2021-August-20 about ten months ago on a Wallaby install, they were also using Ubuntu. http://lists.openstack.org/pipermail/openstack-discuss/2021-August/024273.h… Advice at that point ranged from, try installing Octavia, which I was trying to avoid and I definitely do NOT want a load balancer in front of that particular workload, to Docker swarm is no longer tested for Magnum and it's in practice a K8S exclusive product. I played around for an afternoon and got nowhere WRT configuring the allowed cidrs for a k8s load balancer on a config that does not use a load balancer and does not use k8s. Heat works fine, its super fun, and one alternative is I could give up on Magnum and use some Heat templates, which I know how to do and seems simpler than getting Magnum to work at this point. Or I could brave the scary looking mostly-undocumented world of kuryr and zun, which I was attempting to avoid by using Magnum which is at least simple and easy to install. Or just keep spinning up manual images and letting ansible turn them into docker swarms for me. But, ideally, I'd like to use Magnum to spin up some Docker Swarms for personal curiosity reasons, if nothing else, it would be very convenient if it worked. So the point of this long ramble is, can anyone successfully running Docker Swarm on Yoga on Ubuntu using a very typical "Install Guide" installation provide a pointer or link or github or something that Google searches is not finding to get it to work? Also I hate to piggyback multiple questions but this paragraph also interests me, does anyone have any information and experience with this: https://docs.openstack.org/magnum/yoga/user/index.html#heat-stack-templates "Sample cluster driver To help developers in creating new COE drivers, a minimal cluster driver is provided as an example. The ‘docker’ cluster driver will simply deploy a single VM running Ubuntu with the latest Docker version installed. It is not a true cluster, but the simplicity will help to illustrate the key concepts. To be filled in" That "To be filled in" part would also be very interesting to me as worst case I could shove the Docker Swarm's workload onto a single Docker host, although then I'm back to orchestrating with Heat and presumably Ansible but via a different path.

1 0

[kolla-ansible][xena] Cell database HA/redundancy?
by Norrie, Andrew 10 Jun '22

10 Jun '22

Hi, We are currently planning some large OpenStack deployments utilizing kolla-ansible and I'm curious what folks are doing with the cell database HA/redundancy. With kolla-ansible (xena) it appears that a loadbalancer setup is only allowed for the default database shard (shard 0)... reference: https://docs.openstack.org/kolla-ansible/latest/reference/databases/mariadb… If we are setting up separate cell database shards with Galera, I'm wondering if there is a convenient work-around or configuration for implementation of HA for these cell databases. In the inventory group_vars directory you can specify the group variables (for each cell database) as: nova_cell_database_address: nova_cell_database_group: but these aren't virtual IP hosts/addresses (non default db shard). This works perfectly fine with a single server cell database but not Galera. If that db server goes down the cell instance information is lost. Many thanks ... Andrew ________________________________ "This e-mail and any accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of the email by others is strictly prohibited. If you are not the intended recipient, you must not review, disclose, copy, distribute or use this e-mail; please delete it from your system and notify the sender immediately."

3 3

Error creating octavia amphora
by Russell Stather 10 Jun '22

10 Jun '22

Hi I am seeing the below error when creating a load balancer. The security group does exist, and it is the correct group (tagged with charm-octavia) What am I missing here to resolve this? 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server octavia.common.exceptions.ComputeBuildException: Failed to build compute instance due to: {'code': 500, 'created': '2022-06-09T10:09:59Z', 'message': 'Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.', 'details': 'Traceback (most recent call last):\n File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 654, in build_instances\n scheduler_utils.populate_retry(\n File "/usr/lib/python3/dist-packages/nova/scheduler/utils.py", line 989, in populate_retry\n raise exception.MaxRetriesExceeded(reason=msg)\nnova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.\n'} 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server

4 6

[ops] details for Ops Meetup, Berlin, Tomorrow, Friday 2022-6-10
by Chris Morgan 10 Jun '22

10 Jun '22

A brief twitter thread here https://twitter.com/osopsmeetup/status/1535004027032940544?s=20&t=AgLzV6Qvg… has some reminders about tomorrow's event. Looking forward to seeing everyone. Chris -- Chris Morgan <mihalis68(a)gmail.com>

1 0

[CLOUDKITTY][OpenInfra summit 2022][June 9th] Presentation on Ceilometer, Gnocchi, and CloudKitty
by Rafael Weingärtner 09 Jun '22

09 Jun '22

Hello CloudKitters! It was a great opportunity to be able to present this talk for you all guys. It was a pleasure to see that so many people are interested in CloudKitty, Ceilometer, and Gnocchi. I really hope that we can all collaborate together to keep improving CloudKitty, and other related systems. Attached is the presentation we used during the presentation, so you all can consult this material. Also, as always, if you need some guidance, let us know. Furthermore, as I mentioned in the presentation, I will provide a step-by-step process on how to create a new metric using Ceilometer Dynamic pollster that can collect/generate/derive (however you want to call it) IOPs information for volumes. The new metric name is going to be called "dynamic_pollster.volume.services.volume.iops". Also, the idea is that we collect some attributes, and we use the attribute called "bootable", which indicates if the volume is bootable or not. Then, if the volume is bootable, we will charge for its IOPS; otherwise, we do not charge it. As I mentioned, this is just a very simple use case. It is not targeting anyone's need in production, but it can be used to get you going quite easily with the pipeline. Requirements: - Ceilometer up and running. Version 15.0.0 (or higher is recommended); - Gnocchi up and running. Version 4.3.0 (or higher). If you want to run this in production, this patch might be very important ( https://github.com/gnocchixyz/gnocchi/pull/1059) it has not been merged yet; - CloudKitty up and running. Version 10.0.0 (or higher). With all systems up and running, you will need to do the following: - Ceilometer - Add the new metric collection in the "polling.yaml" file. You can add it in any of the sections you already have there. Or, you can create its own, as follows: ``` - name: volume_iops_metric interval: 600 meters: - dynamic_pollster.volume.services.volume.iops ``` - After declaring the metrics there, you can create the pollster in the "pollster.d" folder in Ceilometer. The name of the file can be anything, as long as it ends with ".yaml"; for instance, "pollsters.d/volume-iops.yaml". Then, you will need to add the following content. Remember to replace the contents of "<>" with the values of your environment. ``` - name: "dynamic_pollster.volume.services.volume.iops" response_entries_key: "volumes" next_sample_url_attribute: "volumes_links | filter(lambda v: v.get('rel') == 'next', value) | list(value) | value[0] | value.get('href')" sample_type: "gauge" unit: "GB" value_attribute: "volume_type" endpoint_type: "volume" url_path: "/v3/<SERVICE_UUID_FOR_CEILOMETER_USER>/volumes/detail?all_tenants=true&limit=10" headers: "Openstack-API-Version": "latest" project_id_attribute: "os-vol-tenant-attr:tenant_id" metadata_fields: - "status" - "name" - "volume_type" - "availability_zone" - "user_id" - "bootable | str(value)" - "attachments | value or [ { 'server_id': '' } ] | value[0] | value['server_id']" metadata_mapping: "availability_zone": "dynamic_availability_zone" "name": "dynamic_display_name" "volume_type": "dynamic_volume_type" "attachments | value or [{ 'server_id': '' }] | value[0] | value['server_id']": "dynamic_instance_id" "bootable | str(value)": "bootable" value_mapping: "<VOLUME_TYPE_UUID_OR_VOLUME_TYPE_NAME>": "<THE IOPS THAT THIS VOLUME TYPE REPRESENTS>" ``` - After creating that, you will need to customize the file "gnocchi_resources.yaml", which is the file that maps metrics to resources, and attributes that are pushed to the backend. You can also create your own "gnocchi_resources.yaml" file, and then use it in the pipeline configuration in the "pipeline.yaml" file, in the configuration of "publishers". Then, you would have something like "gnocchi://?resources_definition_file=%2Fetc%2Fceilometer%2Fcustom_gnocchi_resources.yaml", assuming the file is called "custom_gnocchi_resources.yaml". In this file, you will need to add the new metric and attribute to be pushed to the metrics backend. The following is an example of "volume" resource declaration in the "gnocchi_resource.yaml" file with the new metric and attribute that we want to capture. ``` - resource_type: volume metrics: <MANY OTHER METRICS HERE> dynamic_pollster.volume.services.volume.iops: attributes: display_name: resource_metadata.(dynamic_display_name|notification_display_name) bootable: resource_metadata.bootable <MANY OTHER ENTRIES HERE> ``` - After the above configuration, you will need to create the new attribute in the Gnocchi resource type. To achieve that, you can issue the following command. ``` gnocchi resource-type update -a bootable:string:false volume ``` - Then, you will need to restart the Ceilometer. - Now, you can issue "gnocchi resource show <VOLUME ID>", and you should see the new metric in the resource, and the new attribute as well. With Ceilometer and Gnocchi configured for the new metric, we can move on to CloudKitty. - The new metric needs to be declared in the "metrics.yml" file. The content should be something similar to the following (adapt as you need). ``` dynamic_pollster.volume.services.volume.iops: unit: IOPS alt_name: disk-volume-iops-usage-hours groupby: - id - user_id - project_id - volume_type - display_name - revision_start - instance_id - availability_zone - bootable metadata: [] extra_args: aggregation_method: max resource_type: volume use_all_resource_revisions: false ``` - Restart CloudKitty processor - With the new metric introduced in CloudKitty processor, you will need to create the rating configuration. In our case, we wanted to rate the volumes marked as bootable for IOPs. The volumes that are not bootable would not be rated. The following are the steps one needs to execute to create the hashmap rating rules. - Create the service ``` openstack rating hashmap service create disk-volume-iops-usage-hours ``` - Then, create the field ``` openstack rating hashmap field create <SERVICE_ID> <FIELD NAME WE WANT TO RATE, IN OUR CASE, IT IS bootable> ``` - After that, we can create the rating rule ``` openstack rating hashmap mapping create --field <FIELD UUID THAT WE CREATE> --value 'true' 1 --type flat ``` - Having done that, you should be able to rate all volumes by IOPS if they have the flag "bootable" set to true. Otherwise, they will not be rated. That is all that one needs to execute to replicate the example I showed during the presentation. If you guys have any doubts or difficulties when using CloudKitty, do not hesitate to ping me. -- Rafael Weingärtner

1 0