March 2021 - openstack-discuss - lists.openstack.org

How to customize the xml used in libvirt from GUI/opestack command line?
by Evan Zhao 01 Apr '21

01 Apr '21

Hi there, I googled this question and found two major answers: 1. ssh to the compute node and use `virsh dumpxml` and `virsh undefine/define`, etc. 2. edit nova/virt/libvrit/config.py directly. However, it's trivial to ssh to each node and do the modification, and I prefer not to touch the nova source code, is there any better ways to achieve this? I expect to edit the namespace of a certain element and append an additional element <qemu:command> to the xml file. Any information will be appreciated.

2 1

[cinder][nova][requirements] RFE requested for os-brick
by Brian Rosmaita 01 Apr '21

01 Apr '21

Hello Requirements Team, The Cinder team recently became aware of a potential data-loss bug [0] that has been fixed in os-brick master [1] and backported to os-brick stable/wallaby [2]. We've proposed a release of os-brick 4.4.0 from stable/wallaby [3] and are petitioning for an RFE to include 4.4.0 in the wallaby release. We have three jobs running tempest with os-brick source in master that have passed with [1]: os-brick-src-devstack-plugin-ceph [4], os-brick-src-tempest-lvm-lio-barbican [5],and os-brick-src-tempest-nfs [6]. The difference between os-brick master (at the time the tests were run) and stable/wallaby since the 4.3.0 tag is as follows: master: d4205bd 3 days ago iSCSI: Fix flushing after multipath cfg change (Gorka Eguileor) 0e63fe8 2 weeks ago Merge "RBD: catch read exceptions prior to modifying offset" (Zuul) 28545c7 4 months ago RBD: catch read exceptions prior to modifying offset (Jon Bernard) 99b2c60 2 weeks ago Merge "Dropping explicit unicode literal" (Zuul) 7cfdb76 6 weeks ago Dropping explicit unicode literal (tushargite96) 9afa1a0 3 weeks ago Add Python3 xena unit tests (OpenStack Release Bot) ab57392 3 weeks ago Update master for stable/wallaby (OpenStack Release Bot) 91a1cca 3 weeks ago (tag: 4.3.0) Merge "NVMeOF connector driver connection information compatibility fix" (Zuul) stable/wallaby: f86944b 3 days ago Add release note prelude for os-brick 4.4.0 (Brian Rosmaita) c70d70b 3 days ago iSCSI: Fix flushing after multipath cfg change (Gorka Eguileor) 6649b8d 3 weeks ago Update TOX_CONSTRAINTS_FILE for stable/wallaby (OpenStack Release Bot) f3f93dc 3 weeks ago Update .gitreview for stable/wallaby (OpenStack Release Bot) 91a1cca 3 weeks ago (tag: 4.3.0) Merge "NVMeOF connector driver connection information compatibility fix" (Zuul) This gives us very high confidence that the results of the tests run against master also apply to stable/wallaby at f86944b. Thank you for considering this request. (I've included Nova here because the bug occurs when the configuration option that enables multipath connections on a compute is changed while volumes are attached, so if this RFE is approved, nova might want to raise the minimum version of os-brick in wallaby to 4.4.0.) [0] https://launchpad.net/bugs/1921381 [1] https://review.opendev.org/c/openstack/os-brick/+/782992 [2] https://review.opendev.org/c/openstack/os-brick/+/783207 [3] https://review.opendev.org/c/openstack/releases/+/783641 [4] https://zuul.opendev.org/t/openstack/build/30a103668e4c4a8cb6f1ef907ef3edcb [5] https://zuul.opendev.org/t/openstack/build/bb11eef737d34c41bb4a52f8433850b0 [6] https://zuul.opendev.org/t/openstack/build/3ad3359ca712432d9ef4261d72c787fa

4 9

[tc] Technical Committee weekly meeting
by Ghanshyam Mann 31 Mar '21

31 Mar '21

Hello Everyone, Technical Committee's next weekly meeting is scheduled for April 1st at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, March 31st, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[ironic] PTL on vacation - Week of April 12th
by Julia Kreger 31 Mar '21

31 Mar '21

Greetings folks! I'm going to take a little time off before we get to the Project Teams Gathering, so aside from the two calls I've committed myself to during that week, I'm going to be sitting on a mountain enjoying a change of scenery. If someone will volunteer to run the weekly meeting during the week of the 12th, I will be greatly appreciative. I'm hoping to have a rough PTG schedule up in advance of that meeting as well as the topic list refined, so it will just be links in the weekly agenda. Thanks, -Julia

2 1

[neutron] oslo.privsep migration in Neutron
by Rodolfo Alonso Hernandez 31 Mar '21

31 Mar '21

Hello Neutrinos: During the last cycles we have been migrating the Neutron code from oslo.rootwrap to oslo.privsep. Those efforts are aimed at reaching the goal defined in [1] and are tracked in [2]. At this point, starting Xena developing cycle, we can state that we have migrated all short lived commands from oslo.rootwrap to oslo.privsep or to a native implementation (that could also use oslo.privsep to elevate the permissions if needed). The problem are the daemons or services (long lived processes) that Neutron spawns using "ProcessManager"; this is why "ProcessManager.enable" is the only code calling "utils.execute" without "privsep_exec" parameter. Those process cannot be executed using oslo.privsep because the privsep root daemon has a limited number of executing threads. The remaining processes are [3]. Although we didn't reach the Completion Criteria defined in [1], that is remove the oslo.rootwrap dependency, I think we don't have an alternative to run those services and we should keep rootwrap for them. If there are no objections, once [3] is merged we can consider that Neutron (not other Stadium projects) finished the efforts on [1]. Please, any feedback is always welcome. Regards. [1]https://review.opendev.org/c/openstack/governance/+/718177 [2]https://storyboard.openstack.org/#!/story/2007686 [3] https://review.opendev.org/c/openstack/neutron/+/778444/2/etc/neutron/rootw…

5 6

[ironic] How to move nodes from a 'clean failed' state into 'Available'
by Igal Katzir 31 Mar '21

31 Mar '21

Hello Team, I had a situation where my undercloud-node had a problem with it’s disk and has disconnected from overcloud. I couldn’t restore the undercloud controller and ended up re-installing it (running 'openstack undercloud install’). The installation ended successfully but now I’m in a situation where Cleanup of the overcloud deployed nodes fails: (undercloud) [stack@interop010 ~]$ openstack baremetal node list +--------------------------------------+------------+---------------+-------------+--------------------+-------------+ | UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance | +--------------------------------------+------------+---------------+-------------+--------------------+-------------+ | 97b9a603-f64f-47c1-9fb4-6c68a5b38ff6 | interop025 | None | power on | clean failed | True | | 4b02703a-f765-4ebb-85ed-75e88b4cbea5 | interop026 | None | power on | clean failed | True | +--------------------------------------+------------+---------------+-------------+--------------------+-------------+ I’ve tried to move node to available state but cannot: (undercloud) [stack@interop010 ~]$ openstack baremetal node provide 97b9a603-f64f-47c1-9fb4-6c68a5b38ff6 The requested action "provide" can not be performed on node "97b9a603-f64f-47c1-9fb4-6c68a5b38ff6" while it is in state "clean failed". (HTTP 400) My question is: How do I make the nodes available again? as the deployment of overcloud fails with: ERROR due to "Message: No valid host was found. , Code: 500” Thanks, Igal

3 9

[ironic] PTG and topics oh my
by Julia Kreger 31 Mar '21

31 Mar '21

As everyone hopefully knows by now, the PTG[0] is coming up. We've had an etherpad[1] posted for about a month. During this time, the etherpad has collected 22 topics. I'm 99.95% sure there is some topic overlap, but what we need to make a schedule... or at least begin to form an agenda which requires us understanding what community members perceive as priorities or important topics. So if everyone over the next couple days could do the following two tasks, it would help tremendously for consensus building. 1) Add any missing topics you feel to be important 2) Add +1's to topics you feel are important for yourself OR the community. As always, pressing topics and topics which have the most votes will tend to have more time allocated. -Julia [0]: https://www.eventbrite.com/e/project-teams-gathering-april-2021-tickets-143… [1]: https://etherpad.opendev.org/p/ironic-xena-ptg

1 0

How to debug silent live migration errors
by Linebarger, John 31 Mar '21

31 Mar '21

How would I debug silent (or mostly silent) live migration errors? We're using the Stein release of Canonical's Charmed OpenStack. I have configured it for live migration per the instructions at this link: https://docs.openstack.org/nova/pike/admin/configuring-migrations.html#sect… Specifically: 1. I did not specify vncserver_listen=0.0.0.0 in nova.conf because we are not running VNC on our instances 2. instances_path is /var/lib/nova/instances on all compute nodes 3. I believe that MAAS is "the sole provider of DHCP and DNS for the network hosting the MAAS cluster", per https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/late… 4. Identical authorized_keys files are present on all compute nodes with keys from all compute nodes by default 5. I manually configured the firewalls on all compute nodes to allow libvirt to communicate between compute hosts with: sudo ufw allow 49152:49261/tcp 6. The following settings are specified in nova.conf on each compute node: live_migration_downtime = 500 live_migration_downtime_steps = 10 live_migration_downtime_delay = 75 live_migration_permit_post_copy=true Here's what happens when I try to Live Migrate from the Horizon Dashboard: 1. As admin, in the Admin --> Instances menu, I select the dropdown arrow to the right of the instance. Live Migrate Instance appears (but in black, unlike Migrate Instance, which appears in red). I select Live Migrate Instance, and whether or not I Automatically schedule new host or manually select a new host the Task column says "Migrating" and then it stops and reverts to None. The server never changes. The Action Log shows the live migration request but the Message column is blank. 2. I do the very same thing but this time select Disk Over Commit. Same results. Migrating reverts back to None and the server never changes. 3. I do the very same thing but this time select Block Migration. This time I do get an error: "Failed to live migrate instance to host 'AUTO_SCHEDULE'". And this time the Action Log has "Error" in the Message column. Same behavior with the CLI. For example, this CLI command below completes silently, yet the server for the instance never changes. john@vm-dev-john:~/bin$ openstack server migrate <instanceID> --live <newServerName> [Silent failure] john@vm-dev-john:~/bin$ openstack server show <instancID> [Still running on original server] Note that I *can* successfully Migrate, both using the Horizon Dashboard and the CLI. What fails is Live Migration. I just have no idea why, and no error is displayed in the Action Log for the instance. For reference, the instance is an m1.small with 2GB of RAM, 1 VCPU, and a 20GB Cinder disk volume attached on /dev/vda. Any and all debugging ideas would be most welcome. Without logs I am simply guessing in the dark at this point. Thanks! Enjoy! John M. Linebarger, PhD, MBA Principal Member of Technical Staff Sandia National Laboratories (Office) 505-845-8282 (Cell) 505-681-4879 [cid:image002.jpg@01D72535.958BCAE0]<http://www.sandia.gov/>[AWS Certified Solutions Architect - Professional]<https://www.youracclaim.com/badges/24fe4e43-2f72-4ecc-a11a-783d21dada0f>[AWS Certified Solutions Architect - Associate]<https://www.youracclaim.com/badges/707a5f6f-d919-4daa-bbc4-81b5a779a6cd>[AWS Certified Developer - Associate]<https://www.youracclaim.com/badges/b52ac093-5415-4493-85c3-b77e055211f5>[cid:image003.png@01D72531.072F13F0]<https://www.youracclaim.com/badges/d6e5a453-0e61-4dbe-9716-6af51f364710/pub…>[cid:image005.png@01D72535.958BCAE0]<https://www.youracclaim.com/badges/035d4967-f77d-476e-915b-1061bb789ec3/pub…>

3 3

[cinder] Bug deputy report for week of 2021-03-31
by Sofia Enriquez 31 Mar '21

31 Mar '21

Hello, This is a bug report from 2021-03-16 to 2021-03-24. You're welcome to join the next Cinder Bug Meeting later today. Weekly on Wednesday at 1500 UTC in #openstack-cinder Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Critical: - https://bugs.launchpad.net/os-brick/+bug/1921381 "iSCSI: Flushing issues when multipath config has changed". Stable/ussuri backport assigned to Luigi Toscano (ltoscano) High: - Medium: - https://bugs.launchpad.net/cinder/+bug/1922013 "IBM SVF driver: Add vols to a group (consitent_group_replication enabled) fails sometimes if rccg and rcrel are in different states" Unassigned. Low: - https://bugs.launchpad.net/cinder/+bug/1921584: "backup/drivers/s3.py uses vendored requests from botocore". Unassigned. Incomplete: - Regards Sofi -- L. Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

[kolla] does someone uses AArch64 images?
by Marcin Juszkiewicz 31 Mar '21

31 Mar '21

AArch64 support in Kolla(-Ansible) project is present since Ocata (iirc). For most of that time we supported all three distributions: CentOS, Debian and Ubuntu. # Distribution coverage CentOS has binary (with packages from RDO project) and source images. We supported version 7, 8 and now we are using CentOS Stream 8 in Wallaby. Debian is source only as there never was any source of up-to-date OpenStack packages (Debian OpenStack team builds only x86-64 packages and they do it after release). Ubuntu has both binary (packages from UCA) and source images. # My interest I take care of Debian/source ones as we use them at Linaro in our developer cloud setups. It often means backporting of fixes when we upgrade from one previous release to something newer. # External repos problem The problem in all cases is external repositories and their lack of support for AArch64 architecture. ## RabbitMQ Biggest problem is with RabbitMQ and Erlang. For x86-64 RMQ developers build packages for CentOS/Debian/Ubuntu. We use them. On AArch64 it differ: - CentOS uses erlang from CentOS distribution - Debian uses erlang from my Linaro OBS repository (where I do builds of backport of Debian 'testing' package). - Ubuntu uses erlang from Ubuntu distribution Ubuntu one is too old - 22.2 while current rabbitmq needs 22.3 or newer. I do not plan to work on getting it working. # Final question I would like to know does anyone use AArch64 images built with Kolla. Please respond which branches you use, distribution used as host OS and as images OS.

3 9