November 2021 - openstack-discuss

Windows imaging process
by Gaël THEROND 02 Dec '21

02 Dec '21

Hi everyone, On one of our Openstack platforms, we maintain windows based workloads. We currently have the following build process: 1°/- We download windows ISO image. 2°/- Use hyper-v to create a gold image that bring virtio drivers. 3°/- We upload this image on glance and publish it for our users. 4°/- Our users use packer to create their own windows custom image from our gold image. This workflow works pretty fine, it’s simple enough and as Microsoft isn’t releasing a new major Windows every two months it’s pretty ok not having all the steps automated until now. However, I’m wondering if there is a way to automate my first two steps using Openstack? So far, from my early tests, I didn’t managed to get all of appropriate gears to work together, did I missed something? I need to create a VM that at least can use the Windows ISO plus a virtio ISO as a second cdrom device and an additional user-data payload that will instruct the Windows installer to go automatically through the installation steps plus loads missing drivers from the virtio iso. This is perfectly working on a native kvm as you can add multiple cdrom device but I didn’t find a way to replicate that on Openstack. Starting from an unaltered Microsoft originated ISO image is a mandatory requirement for this project (Because of security constraints that I can’t have any impact on). Help from anyone that already had to deal with such situation would be very appreciated ! Thanks!

2 3

[kuryr] Propose to EOL stable branches (Stein, Rocky and Queens) for all the kuryr scope
by Maysa De Macedo Souza 02 Dec '21

02 Dec '21

Hello, We have plenty of stable branches in Kuryr repositories still open with the oldest being stable/queens. We haven't seen many backports[0] proposed to Stein, Rocky and Queens, so I would like to propose retiring these branches. Let me know if anybody is interested in keeping any of those branches. [0] https://review.opendev.org/q/(project:%255Eopenstack/kuryr.*)AND((branch:st… Thank you, Maysa Macedo.

2 3

[all][tc] Technical Committee next weekly meeting on Dec 2nd at 1500 UTC
by Ghanshyam Mann 01 Dec '21

01 Dec '21

Hello Everyone, Technical Committee's next weekly meeting is scheduled for Dec 2nd at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, Dec 1st, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[cinder] : SAN migration
by Gauurav Sabharwal1 01 Dec '21

01 Dec '21

Hi Experts , I need some expert advise of one of the scenario, I have multiple isolated OpenStack cluster running with train & rocky edition. Each OpenStack cluster environment have it's own isolated infrastructure of SAN ( CISCO fabric ) & Storage ( HP, EMC & IBM). Now company planning to refresh their SAN infrastructure. By procuring new Brocade SAN switches. But there are some migration relevant challenges we have. As we understand under one cinder instance only one typer of FC zone manager is supported . Currently customer configured & managing CISCO . Is it possible to configure two different vendor FC Zone manager under one cinder instance. Migration of SAN zoning is supposedly going to be happen offline way from OpenStack point of view. We will be migrating all ports of each existing cisco fabric to Brocade with zone configuration using brocade CLI. Our main concern is that after migration How CINDER DB update new zone info & path via Brocade SAN. Regards Gauurav Sabharwal IBM India Pvt. Ltd. IBM towers Ground floor, Block -A , Plot number 26, Sector 62, Noida Gautam budhnagar UP-201307. Email:gauurav.sabharwal@in.ibm.com Mobile No.: +91-9910159277

3 3

[neutron][networking][ipv6][dns][ddi] Upcoming OpenInfra Edge Computing Group sessions
by Ildiko Vancsa 30 Nov '21

30 Nov '21

Hi, I’m reaching out to you to draw your attention to the amazing lineup of discussion topics for the OpenInfra Edge Computing Group weekly calls up until the end of this year with industry experts to present and participate in the discussions! I would like to invite and encourage you to join the working group sessions to discuss edge related challenges and solutions in the below areas and more! Some of the sessions to highlight will be continuing the discussions we started at the recent PTG: * November 29th - Networking and DNS discussion with Cricket Liu and Andrew Wertkin * December 6th - Networking and IPv6 discussion with Ed Horley Beyond the above topics we will have presentations from groups and projects such as Smart Edge and the Industry IoT Consortium (Formerly Industrial Internet Consortium). For the full schedule please see the edge WG’s wiki: https://wiki.openstack.org/wiki/Edge_Computing_Group#Upcoming_Topics Our calls happen every Monday at 6am Pacific Time / 1400 UTC on Zoom. You can find all the meeting details here: https://wiki.openstack.org/wiki/Edge_Computing_Group#Meetings Please let me know if you have any questions about the working group or any of the upcoming sessions. Thanks and Best Regards, Ildikó

1 3

[Neutron][OVN] nova-compute timeout while waiting for VIF activation confirmed by Neutron
by Tony Liu 30 Nov '21

30 Nov '21

Hi, I see such problem from time to time. It's not consistently reproduceable. ====================== 2021-11-23 22:16:28.532 7 INFO nova.compute.manager [req-50817b01-e7ae-4991-94fe-e4c5672c481b - - - - -] [instance: 7021bf37-348d-4dfa-b0c5-6a251fceb6cd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-11-23 22:21:28.342 7 WARNING nova.virt.libvirt.driver [req-814c98a4-3fd9-4607-9abb-5fbe5cef8650 3a4c320d64664d9cb6784b7ea52d618a 21384fe21ebb4339b1a5f2a8f9000ea3 - default default] [instance: 7021bf37-348d-4dfa-b0c5-6a251fceb6cd] Timeout waiting for [('network-vif-plugged', '735d97a3-db7c-42ed-b3be-1596d4cc7f4b')] for instance with vm_state building and task_state spawning.: eventlet.timeout.Timeout: 300 seconds ====================== The VIF/port is activated by OVN ovn-controller to ovn-sb. It seems that, either Neutron didn't capture the update or didn't send message back to nova-compute. Is there any known fix for this problem? Thanks! Tony

5 9

[neutron][nova-compute] Received unexpected event network-vif-plugged-<uuid>
by Tony Liu 30 Nov '21

30 Nov '21

Hi, With Ussuri, when launch VM, I see this from nova-compute. ======== Received unexpected event network-vif-plugged-32b7af8a-ef81-4785-bd79-3a5619823638 ======== That caused VM to be destroyed, rescheduled and relaunched. And that recreation worked fine. >From user POV, it just takes a bit longer to launch the VM. What's the event nova-compute expects from Neutron for vif-plug? Any hints where I can look into it? More logs here. ======== 2021-11-29 18:28:10.064 7 INFO os_vif [req-2a89f605-6972-41dc-83f1-1ef6a444e486 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f1:94:a0,bridge_name='br-int',has_traffic_filtering=True,id=32b7af8a-ef81-4785-bd79-3a5619823638,network=Network(27ea27c6-3d1e-4d37-ad4a-de2ae1578f1e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap32b7af8a-ef') 2021-11-29 18:28:10.515 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384-1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] VM Started (Lifecycle Event) 2021-11-29 18:28:10.544 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384 -1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] VM Paused (Lifecycle Event) 2021-11-29 18:28:10.604 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384 -1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] During sync_power_state the instance has a pending task (spawning). Skip. 2021-11-29 18:28:25.441 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384 -1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] VM Res umed (Lifecycle Event) 2021-11-29 18:28:25.445 7 INFO nova.virt.libvirt.driver [-] [instance: a8eb2a2a- 8c83-46b2-bfd4-fd3a7d00f2ce] Instance spawned successfully. 2021-11-29 18:28:25.445 7 INFO nova.compute.manager [req-2a89f605-6972-41dc-83f1-1ef6a444e486 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] Took 18.75 seconds to spawn the instance on the hypervisor. 2021-11-29 18:28:25.502 7 INFO nova.compute.manager [req-01cbb5ac-d453-4484-9384-1a2173b532a9 - - - - -] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] During sync_power_state the instance has a pending task (spawning). Skip. 2021-11-29 18:28:25.514 7 INFO nova.compute.manager [req-2a89f605-6972-41dc-83f1-1ef6a444e486 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] Took 19.46 seconds to build instance. 2021-11-29 18:28:27.466 7 WARNING nova.compute.manager [req-82314965-fe71-4993-bbf6-f0aa9e708529 f0c1375066214e24ab7942d72d829097 3ec2940fc10d4e57bcf929d1fe678c79 - default default] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] Received unexpected event network-vif-plugged-32b7af8a-ef81-4785-bd79-3a5619823638 for instance with vm_state active and task_state None. 2021-11-29 18:28:55.829 7 INFO nova.compute.manager [req-e36e4a81-4755-4d7f-9ee4-d43d00629765 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] [instance: a8eb2a2a-8c83-46b2-bfd4-fd3a7d00f2ce] Terminating instance ======== Thanks! Tony

2 1

[neutron] CI meeting 30.11.2021 - cancelled
by Slawek Kaplonski 29 Nov '21

29 Nov '21

Hi, Due to some personal reasons I will not be able to drive today's Neutron CI meeting. Let's cancel it this week. Remember, next week we will have CI meeting on Jitsi and we will discuss all ideas about how to improve stability of our CI. -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[Neutron][OVN] AttributeError: 'NoneType' object has no attribute 'get_lrouter'
by Tony Liu 29 Nov '21

29 Nov '21

Hi, It happens from time to time, not consistently, with the latest kolla/centos-binary-neutron-server:ussuri container image. >From the bottome of traceback, it seems self._ovn is None. Any clues how this can happen? I don't see other suspicious logging in Neutron and OVN. ===================== File "/usr/lib/python3.6/site-packages/neutron/services/ovn_l3/plugin.py", line 477, in get_router_availability_zones ERROR neutron.api.v2.resource lr = self._ovn.get_lrouter(router['id']) ERROR neutron.api.v2.resource AttributeError: 'NoneType' object has no attribute 'get_lrouter' ===================== Here is the full traceback. ===================== neutron.api.v2.resource [req-bf2230aa-0491-4caa-b5f7-169d37009672 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] index failed: No details.: AttributeError: 'NoneType' object has no attribute 'get_lrouter' neutron.api.v2.resource Traceback (most recent call last): neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/api/v2/resource.py", line 98, in resource neutron.api.v2.resource result = method(request=request, **args) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped neutron.api.v2.resource return f(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper neutron.api.v2.resource ectxt.value = e.inner_exc neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper neutron.api.v2.resource return f(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped neutron.api.v2.resource LOG.debug("Retry wrapper got retriable exception: %s", e) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped neutron.api.v2.resource return f(*dup_args, **dup_kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 369, in index neutron.api.v2.resource return self._items(request, True, parent_id) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 304, in _items neutron.api.v2.resource obj_list = obj_getter(request.context, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 233, in wrapped neutron.api.v2.resource return method(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped neutron.api.v2.resource return f(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper neutron.api.v2.resource ectxt.value = e.inner_exc neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper neutron.api.v2.resource return f(*args, **kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped neutron.api.v2.resource LOG.debug("Retry wrapper got retriable exception: %s", e) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ neutron.api.v2.resource self.force_reraise() neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise neutron.api.v2.resource raise value neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped neutron.api.v2.resource return f(*dup_args, **dup_kwargs) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/db/l3_db.py", line 547, in get_routers neutron.api.v2.resource page_reverse=page_reverse) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/model_query.py", line 317, in get_collection neutron.api.v2.resource for c in query neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/model_query.py", line 317, in <listcomp> neutron.api.v2.resource for c in query neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/db/l3_db.py", line 221, in _make_router_dict neutron.api.v2.resource resource_extend.apply_funcs(l3_apidef.ROUTERS, res, router) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/resource_extend.py", line 84, in apply_funcs neutron.api.v2.resource resolved_func(response, db_object) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/db/availability_zone/router.py", line 40, in _add_az_to_response neutron.api.v2.resource l3_plugin.get_router_availability_zones(router_db)) neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/services/ovn_l3/plugin.py", line 477, in get_router_availability_zones neutron.api.v2.resource lr = self._ovn.get_lrouter(router['id']) neutron.api.v2.resource AttributeError: 'NoneType' object has no attribute 'get_lrouter' neutron.api.v2.resource neutron.wsgi [req-bf2230aa-0491-4caa-b5f7-169d37009672 b8de8d37a30147fd98681c22fc515874 b5938e7ebb0645e396b19e99bb7f9868 - default default] 10.91.1.7,10.250.20.26 "GET /v2.0/routers HTTP/1.1" status: 500 len: 344 time: 0.2660162 ===================== Thanks! Tony

1 0

RE: Nova Local Disk Storage (Seena Fallah)
by Steven Relf 29 Nov '21

29 Nov '21

Hi Seena, Do you mean passing an entire disk to an instance, or moving where nova stores its ephemeral disks? (e.g. /var/lib/nova/instances?) Rgds Steve. The future has already arrived. It's just not evenly distributed yet - William Gibson Hi, Is there any support for creating instances on a whole separate disk, not an LVM?

1 0