May 2021 - openstack-discuss - lists.openstack.org

[openstack-helm] Meeting Cancelled
by Gage Hugo 25 May '21

25 May '21

Hey team, Since there are no agenda items [0] for the IRC meeting today, the meeting is cancelled. Our next meeting will be June 1st. Thanks [0] https://etherpad.opendev.org/p/openstack-helm-weekly-meeting

1 0

[cinder] reminder: this week's meeting in video+IRC
by Brian Rosmaita 25 May '21

25 May '21

Quick reminder that this week's Cinder team meeting on Wednesday 26 May, being the final meeting of the month, will be held in both videoconference and IRC at the regularly scheduled time of 1400 UTC. These are the video meeting rules we've agreed to: * Everyone will keep IRC open during the meeting. * We'll take notes in IRC to leave a record similar to what we have for our regular IRC meetings. * Some people are more comfortable communicating in written English. So at any point, any attendee may request that the discussion of the current topic be conducted entirely in IRC. * The meeting will be recorded. connection info: https://bluejeans.com/3228528973 meeting agenda: https://etherpad.opendev.org/p/cinder-xena-meetings cheers, brian

1 0

[victoria] Reboot problem
by Albert Shih 25 May '21

25 May '21

Hi I'm running (trying) a openstack (version victoria) with a Dell Unity storage unit. When I'm create a instance, everything work fine, the block volume are created on the Unity, the mount (with iscsi) work on the compute and the instance boot normaly. But if I shutdown a instance and try to restart it, it's failed. It's just like the block volume cannot be mount. I check on the unity eveyrthing are ok In the log of the compute /var/log/nova/nova-compute.log I can see : Any idea ? Regards 2021-05-21 21:39:15.849 2663 WARNING os_brick.initiator.connectors.iscsi [req-42f8147b-b006-4af7-8dfb-31c39b46b728 868dde297576ce232570ea549928d02d58f543007825d199eb48f30a10139c96 416ac2de9ed74316920e9cbe3d376bb3 - 223c116cac324ae19fd74877a6c06d27 223c116cac324ae19fd74877a6c06d27] LUN 5 on iSCSI portal 10.15.23.252:3260 not found on sysfs after logging in. 2021-05-21 21:39:15.952 2663 WARNING os_brick.initiator.connectors.iscsi [req-42f8147b-b006-4af7-8dfb-31c39b46b728 868dde297576ce232570ea549928d02d58f543007825d199eb48f30a10139c96 416ac2de9ed74316920e9cbe3d376bb3 - 223c116cac324ae19fd74877a6c06d27 223c116cac324ae19fd74877a6c06d27] Couldn't find iSCSI nodes because iscsiadm err: iscsiadm: No records found : os_brick.exception.VolumeDeviceNotFound: Volume device not found at . 2021-05-21 21:39:15.960 2663 WARNING os_brick.initiator.connectors.iscsi [req-42f8147b-b006-4af7-8dfb-31c39b46b728 868dde297576ce232570ea549928d02d58f543007825d199eb48f30a10139c96 416ac2de9ed74316920e9cbe3d376bb3 - 223c116cac324ae19fd74877a6c06d27 223c116cac324ae19fd74877a6c06d27] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. : os_brick.exception.VolumeDeviceNotFound: Volume device not found at . 2021-05-21 21:39:16.231 2663 INFO nova.compute.manager [req-42f8147b-b006-4af7-8dfb-31c39b46b728 868dde297576ce232570ea549928d02d58f543007825d199eb48f30a10139c96 416ac2de9ed74316920e9cbe3d376bb3 - 223c116cac324ae19fd74877a6c06d27 223c116cac324ae19fd74877a6c06d27] [instance: ec79c0d6-259d-466f-94da-def4ce318d3a] Successfully reverted task state from powering-on on failure for instance. 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server [req-42f8147b-b006-4af7-8dfb-31c39b46b728 868dde297576ce232570ea549928d02d58f543007825d199eb48f30a10139c96 416ac2de9ed74316920e9cbe3d376bb3 - 223c116cac324ae19fd74877a6c06d27 223c116cac324ae19fd74877a6c06d27] Exception during message handling: os_brick.exception.VolumeDeviceNotFound: Volume device not found at . 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 196, in _do_dispatch 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/exception_wrapper.py", line 77, in wrapped 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server _emit_exception_notification( 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server self.force_reraise() 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server raise value 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/exception_wrapper.py", line 69, in wrapped 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 188, in decorated_function 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server LOG.warning("Failed to revert task state for instance. " 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server self.force_reraise() 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server raise value 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 159, in decorated_function 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/compute/utils.py", line 1456, in decorated_function 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 216, in decorated_function 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server compute_utils.add_instance_fault_from_exc(context, 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server self.force_reraise() 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server raise value 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 205, in decorated_function 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 3152, in start_instance 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server self._power_on(context, instance) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 3120, in _power_on 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server self.driver.power_on(context, instance, 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 3429, in power_on 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server self._hard_reboot(context, instance, network_info, block_device_info, 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 3294, in _hard_reboot 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server xml = self._get_guest_xml(context, instance, network_info, disk_info, 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6364, in _get_guest_xml 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server conf = self._get_guest_config(instance, network_info, image_meta, 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6006, in _get_guest_config 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server storage_configs = self._get_guest_storage_config(context, 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 4750, in _get_guest_storage_config 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server self._connect_volume(context, connection_info, instance) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 1623, in _connect_volume 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server vol_driver.connect_volume(connection_info, instance) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/virt/libvirt/volume/iscsi.py", line 64, in connect_volume 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server device_info = self.connector.connect_volume(connection_info['data']) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/os_brick/utils.py", line 137, in trace_logging_wrapper 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py", line 359, in inner 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/iscsi.py", line 519, in connect_volume 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server self._cleanup_connection(connection_properties, force=True) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server self.force_reraise() 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server raise value 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/iscsi.py", line 513, in connect_volume 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server return self._connect_single_volume(connection_properties) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/os_brick/utils.py", line 61, in _wrapper 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server return r.call(f, *args, **kwargs) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/retrying.py", line 212, in call 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server raise attempt.get() 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/retrying.py", line 247, in get 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server six.reraise(self.value[0], self.value[1], self.value[2]) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server raise value 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/retrying.py", line 200, in call 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server attempt = Attempt(fn(*args, **kwargs), attempt_number, False) 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/iscsi.py", line 591, in _connect_single_volume 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server raise exception.VolumeDeviceNotFound(device='') 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server os_brick.exception.VolumeDeviceNotFound: Volume device not found at . 2021-05-21 21:39:16.237 2663 ERROR oslo_messaging.rpc.server 2021-05-21 21:43:45.556 2663 WARNING nova.compute.manager [req-6d91e265-b6b2-4522-826d-176e0d24a2d3 - - - - -] While synchronizing instance power states, found 1 instances in the database and 0 instances on the hypervisor. 2021-05-21 21:43:45.598 2663 INFO nova.compute.manager [-] [instance: ec79c0d6-259d-466f-94da-def4ce318d3a] During _sync_instance_power_state the DB power_state (4) does not match the vm_power_state from the hypervisor (0). Updating power_state in the DB to match the hypervisor. -- Albert SHIH Observatoire de Paris France xmpp: jas(a)obspm.fr Heure local/Local time: Fri May 21 11:43:30 PM CEST 2021

2 2

[telemetry] Retire panko
by Matthias Runge 25 May '21

25 May '21

Hi there, over the past couple of cycles, we have seen decreasing interest on panko. Also it has some debts, which were just carried over from the early days. We discussed over at the PTG and didn't really found a reason to keep it alive or included under OpenStack. With that, it also makes sense to retire puppet-panko. I'll wait for 1-2 weeks and propose appropriate patches to get it retired then, if I don't hear anything against it or if there are any takers. Matthias

1 2

[automation-sig][freezer][i18n-sig][powervmstacker-sig][public-cloud-sig] IRC channel cleanup
by Jeremy Stanley 25 May '21

25 May '21

I've proposed a pair of changes removing the OpenDev Collaboratory IRC bots (including logging) from inactive channels. Some of these channels correspond to current groups within the OpenStack community, so it's only fair to give them a heads up and let them know. The following channels have had no human discussion at all (no comments from anything other than our bots) for all of 2021 so far: * #openstack-auto-scaling * #openstack-fr * #openstack-freezer * #openstack-powervm * #openstack-publiccloud * #openstack-self-healing It's fine if these efforts aren't utilizing IRC, but there's probably little point in us logging or reporting information in them if they're unused. The patches for removing our services from those channels are available for review here (they also include cleanup of channels for other known defunct efforts or vestiges of channels which have since move to new names): https://review.opendev.org/792301 https://review.opendev.org/792302 If you're concerned and would like to resume using one or more of these channels, feel free to follow up here on the mailing list or with review comments some time in the next few days. If there are no objections I plan to merge these no later than Monday, May 24. Of course it's quite easy to reinstate services on a channel at any time, so if you don't see this until the changes have already merged, please follow up anyway and we can restore the bots as needed. -- Jeremy Stanley

1 1

[neutron] Bug deputy report (week starting on 2021-05-17)
by Bernard Cafarelli 24 May '21

24 May '21

Hey neutrinos, we are starting a new bug deputy rotation (new dates available as usual at [0]), here are the bugs that were reported last week. Skipping invalid/duplicate bugs, this was relatively quiet, most bugs have fixes or assignees. OVN eyes are welcome on https://bugs.launchpad.net/bugs/1929197 (unconfirmed section). In details: Critical * Fullstack test TestUninterruptedConnectivityOnL2AgentRestart failing often with LB agent - https://bugs.launchpad.net/bugs/1928764 Causing FT failures, lajos looking into it in https://review.opendev.org/c/openstack/neutron/+/792507 * Devstack "fixup_ubuntu" method removed - https://bugs.launchpad.net/bugs/1928805 Fixed with https://review.opendev.org/c/openstack/neutron/+/791983 * Docs job is broken in neutron - https://bugs.launchpad.net/bugs/1928913 pyroute2 issue with newer versions, https://review.opendev.org/c/openstack/neutron/+/792180 fixes it for older branches, and https://review.opendev.org/c/openstack/neutron/+/792077 in master (dependency will be added to pyroute2 directly soon) Medium * "test_reserve_provider_segment_without_physical_network" failing randomly - https://bugs.launchpad.net/bugs/1929190 Proposed test fix https://review.opendev.org/c/openstack/neutron/+/792559 Low * Missing packages in openSuse installation steps - https://bugs.launchpad.net/bugs/1929012 Small suggested doc fix on packages to install for L3 agent Unconfirmed * [OVN] SB connection unreliable - https://bugs.launchpad.net/bugs/1929197 Needs some OVN eyes, error on "AttributeError: 'NoneType' object has no attribute 'chassis_exists'" Incomplete * [stein][neutron] l3 agent error - https://bugs.launchpad.net/bugs/1928675 After a migration from queens, controllers go OOM with HA routers "Respawning ip_monitor" logs Mentioned env was cleaned, we may get more information on next env update [0] https://wiki.openstack.org/wiki/Network/Meetings#Bug_deputy -- Bernard Cafarelli

1 0

[TripleO] Opting out of global-requirements.txt
by James Slagle 24 May '21

24 May '21

I'd like to propose that TripleO opt out of dependency management by removing tripleo-common from global-requirements.txt. I do not feel that the global dependency management brings any advantages or anything needed for TripleO. I can't think of any reason to enforce the ability to be globally pip installable with the rest of OpenStack. Two of our most critical projects, tripleoclient and tripleo-common do not even put many of their data files in the right place where our code expects them when they are pip installed. So, I feel fairly confident that no one is pip installing TripleO and relying on global requirements enforcement. One potential advantage of not being in global-requirements.txt is that our unit tests and functional tests could actually test the same code. As things stand today, our unit tests in projects that depend on tripleo-common are pinned to the version in global-requirements.txt, while our functional tests currently run with tripleo-common from master (or included depends-on). The changes needed would be (aiui): - Remove tripleo repos from projects.txt - Remove check-requirements jobs from those same repos - Remove tripleo-common from global-requirements.txt I think we should also plan to backport these changes to Wallaby. Let me know any concerns or feedback, or anything I might be overlooking. Thanks. -- -- James Slagle --

9 14

[kolla] Draft release process
by Mark Goddard 24 May '21

24 May '21

Hi, At the Xena PTG [1] we discussed changing the early phases of our release process to provide more stability while developing features for Kolla projects. We also agreed to firm up timing and improve documentation around some of the steps of the release process nearer to the release. I have drafted a new release process/schedule [2] based on these ideas. Please share your comments, thoughts and concerns either here or in the Etherpad. Thanks, Mark [1] https://etherpad.opendev.org/p/kolla-xena-ptg [2] https://etherpad.opendev.org/p/kolla-release-process-draft

1 0

[kolla] master branch open for development
by Mark Goddard 24 May '21

24 May '21

Hi, Apologies, this email should have been sent some time ago. I wanted to notify the community that the master branch of Kolla projects is open for development. Note that review focus will still be on patches relevant to Wallaby until it is released. Thanks, Mark

1 0

[wallaby][trove] Instance Volume Resize
by Ammad Syed 24 May '21

24 May '21

Hi, I am using wallaby / trove on ubuntu 20.04. I am trying to extend volume of database instance. Its having trouble that instance cannot exceed volume size of 10GB. My flavor has 2vcpus 4GB RAM and 10GB disk. I created a database instance with 5GB database size and mysql datastore. The deployment has created 10GB root and 5GB /var/lib/mysql. I have tried to extend volume to 11GB, it failed with error that "Volume 'size' cannot exceed maximum of 10 GB, 11 cannot be accepted". I want to keep root disk size to 10GB and only want to extend /var/lib/mysql keeping the same flavor. Is it possible or should I need to upgrade flavor as well ? -- Regards, Syed Ammad Ali

2 4