- openstack-discuss - lists.openstack.org

[nova] Ussuri feature scrub
by Eric Fried 20 Feb '20

20 Feb '20

Nova maintainers and contributors- { Please refer to this ML thread [1][2] for background. } Now that spec freeze has passed, I would like to assess the Design:Approved blueprints and understand how many we could reasonably expect to land in Ussuri. We completed 25 blueprints in Train. However, mriedem is gone, and it is likely that I will drop off the radar after Q1. Obviously all blueprints/releases/reviews/etc. are not created equal, but using stackalytics review numbers as a rough heuristic, I expect about 20 blueprints to get completed in Ussuri. If we figure that 5-ish of the incompletes will be due to factors beyond our control, that would mean we should Direction:Approve about 25. As of this writing: - 30 blueprints are targeted for ussuri [3]. Of these, - 7 are already implemented. Of the remaining 23, - 2 are not yet Design:Approved. These will need an exception if they are to proceed. And - 19 (including the unapproved ones) have code in various stages. I would like to see us cut 5-ish of the 30. I have made an etherpad [4] with the unimplemented blueprints listed with owners and code links. I made notes on some of the ones I would like to see prioritized, and a couple on which I'm more meh. If you have a stake in Nova/Ussuri, I encourage you to weigh in. How will we ultimately decide? Will we actually cut anything? I don't have the answers yet. Let's go through this exercise and see if anything obvious falls out, and then we can figure out the next steps. Thanks, efried [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-September/00983… [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/thread.… [3] https://blueprints.launchpad.net/nova/ussuri [4] https://etherpad.openstack.org/p/nova-ussuri-planning

2 2

[neutron]After update neutron-lbaas, How to not restart neutron-server to let new neutron-lbaas code work?
by Zhengyu Pan 20 Feb '20

20 Feb '20

Hi, I modified neutron-lbaas load balancer code to support our special need. I have updated neutron-lbaas code in the controller node. But i don't want to restart neutron-server in case the customer's business was interrupted. How can i let netron-lbaas code work in the case of not restarting neutrn-sever in controller node? -- Best! ZhengYu！

1 0

[all][neutron][neutron-fwaas] Maintainers needed
by Slawek Kaplonski 20 Feb '20

20 Feb '20

Hi, Over the past couple of cycles we have noticed that new contributions and maintenance efforts for neutron-fwaas project were almost non existent. This impacts patches for bug fixes, new features and reviews. The Neutron core team is trying to at least keep the CI of this project healthy, but we don’t have enough knowledge about the details of the neutron-fwaas code base to review more complex patches. During the PTG in Shanghai we discussed that with operators and TC members during the forum session [1] and later within the Neutron team during the PTG session [2]. During these discussions, with the help of operators and TC members, we reached the conclusion that we need to have someone responsible for maintaining project. This doesn’t mean that the maintainer needs to spend full time working on this project. Rather, we need someone to be the contact person for the project, who takes care of the project’s CI and review patches. Of course that’s only a minimal requirement. If the new maintainer works on new features for the project, it’s even better :) If we don’t have any new maintainer(s) before milestone Ussuri-2, which is Feb 10 - Feb 14 according to [3], we will need to mark neutron-fwaas as deprecated and in “V” cycle we will propose to move the project from the Neutron stadium, hosted in the “openstack/“ namespace, to the unofficial projects hosted in the “x/“ namespace. So if You are using this project now, or if You have customers who are using it, please consider the possibility of maintaining it. Otherwise, please be aware that it is highly possible that the project will be deprecated and moved out from the official OpenStack projects. [1] https://etherpad.openstack.org/p/PVG-Neutron-stadium-projects-the-path-forw… [2] https://etherpad.openstack.org/p/Shanghai-Neutron-Planning-restored - Lines 379-421 [3] https://releases.openstack.org/ussuri/schedule.html -- Slawek Kaplonski Senior software engineer Red Hat

8 14

[nova] context being introduced to nova.virt.driver.ComputeDriver.extend_volume
by Lee Yarwood 20 Feb '20

20 Feb '20

Hello all, I'm looking to introduce the request context to the signature of extend_volume in the following change: virt: Pass request context to extend_volume https://review.opendev.org/#/c/706899/ Any out of tree drivers implementing extend_volume will also need to add the request context to their implementations once this lands. This is part of a wider bugfix series below: https://review.opendev.org/#/q/topic:bug/1861071 Also discussed on the ML in the following thread: http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012551… If there are any concerns or issues with this then please let me know! Cheers, -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76

1 0

[kolla-ansible] External Ceph keyring encryption
by Jason Anderson 20 Feb '20

20 Feb '20

Hi all, My understanding is that KA has dropped support for provisioning Ceph directly, and now requires an external Ceph cluster (side note: we should update the docs[1], which state it is only "sometimes necessary" to use an external cluster--I will try to submit something today). I think this works well, but the handling of keyrings cuts a bit against the grain of KA. The keyring files must be dropped in to the node_custom_config directory. This means that operators who prefer to keep their KA configuration in source control must have some mechanism for securing that, as it is unencrypted. What does everybody think about storing Ceph keyring secrets in passwords.yml instead, similar to how SSH keys are handled? Thanks, /Jason [1]: https://docs.openstack.org/kolla-ansible/latest/reference/storage/external-… -- Jason Anderson Chameleon DevOps Lead Consortium for Advanced Science and Engineering, The University of Chicago Mathematics & Computer Science Division, Argonne National Laboratory

4 6

VPNaaS with multiple endpoint groups
by Eugen Block 20 Feb '20

20 Feb '20

Hi all, is anyone here able to help with a vpn issue? It's not really my strong suit but I'll try to explain. In a Rocky environment (using openvswitch) a customer has setup a VPN service successfully, but that only seems to work if there's only one local and one peer endpoint group. According to the docs it should work with multiple endpoint groups, as far as I could tell the setup looks fine and matches the docs (don't create the subnet when creating the vpn service but use said endpoint groups). What we're seeing is that as soon as the vpn site connection is created with multiple endpoints only one of the destination IPs is reachable. And it seems as if it's always the first in the list of EPs (see below). This seems to be reflected in the iptables where we also only see one of the required IP ranges. Also neutron reports duplicate rules if we try to use both EPs: 2020-02-12 14:14:27.638 16275 WARNING neutron.agent.linux.iptables_manager [req-92ff6f06-3a92-4daa-aeea-9c02dc9a31c3 ba9bf239530d461baea2f6f60bd301e6 850dad648ce94dbaa5c0ea2fb450bbda - - -] Duplicate iptables rule detected. This may indicate a bug in the iptables rule generation code. Line: -A neutron-l3-agent-POSTROUTING -s X.X.252.0/24 -d Y.Y.0.0/16 -m policy --dir out --pol ipsec -j ACCEPT These are the configured endpoints: root@control:~ # openstack vpn endpoint group list +--------------------------------------+-----------------+--------+------------------------------------------------------------------------------------+ | ID | Name | Type | Endpoints | +--------------------------------------+-----------------+--------+------------------------------------------------------------------------------------+ | 0f853567-e4bf-4019-9290-4cd9f94a9793 | peer-ep-group-1 | cidr | [u'X.X.253.0/24'] | | 152a0f9e-ce49-4769-94f1-bc0bebedd3ec | peer-ep-group-2 | cidr | [u'X.X.253.0/24', u'Y.Y.0.0/16'] | | 791ab8ef-e150-4ba0-ac2c-c044659f509e | local-ep-group1 | subnet | [u'38efad5e-0f1e-4e36-8995-74a611bfef41'] | | 810b0bf2-d258-459b-9b57-ae5b491ea612 | local-ep-group2 | subnet | [u'38efad5e-0f1e-4e36-8995-74a611bfef41', u'9e35d80f-029e-4cc1-a30b-1753f7683e16'] | | b5c79e08-41e4-441c-9ed3-9b02c2654173 | peer-ep-group-3 | cidr | [u'Y.Y.0.0/16'] | +--------------------------------------+-----------------+--------+------------------------------------------------------------------------------------+ Has anyone experience with this and could help me out? Another follow-up question: how can we gather some information regarding the ipsec status? Althoug there is a active tunnel we don't see anything with 'ipsec statusall', I've checked all namespaces on the control node. Any help is highly appreciated! Best regards, Eugen

1 1

[ironic]: Failed to create config drive on disk
by fsbiz＠yahoo.com 19 Feb '20

19 Feb '20

Hi, I'd very much appreciate some help on this. We have a medium-large ironic installation where the baremetal nodes are constantly doing provisionings and deprovisionings.We test a variety of images (Ubuntu, RedHat, Windows, etc.) in both BIOS and UEFI modes. Our approach so far is to configure all the baremetal nodes in CSM UEFI mode so that both BIOS and UEFI images can be run. And things have worked fairly well with this. Lately, I'm having this weird "Failed to create config drive on disk" issue and it is happening only with BIOS images on certain baremetal nodes. Here are the important snippets from the ironic conductor and the IPA that I've managed to narrow down. ================================Ironic conductor (before power-on)2020-02-18 13:06:33.261 DEBUG iDeploy boot mode is uefi for ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 ================================Ironic conductor (power-on)2020-02-18 13:07:18.541 INFO Successfully set node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 power state to power on by rebooting. 020-02-18 13:07:18.560 INFO Node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 moved to provision state "wait call-back" from state "deploying"; target provision state is "active" ================================== Ironic Python Agent: (I can provide the full log on request. Only relevant logs provided here for the sake of brevity). wipefs --force --all Feb 18 21:08:17 ironic-python-agent: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): wipefs --force --all /dev/sda Feb 18 21:08:17 ironic-python-agent: CMD "wipefs --force --all /dev/sda" returned: 0 in 0.023s Feb 18 21:08:17 ironic-python-agentt: Execution completed, command line is "wipefs --force --all /dev/sda" Feb 18 21:08:17 ironic-python-agent: Command stdout is: "" execute /usr/share/ironic-python-agent/venv/lib/python2.7/site-packages/ironic_lib/utils.py:103Feb 18 21:08:17 ironic-python-agent: Command stderr is: "" execute /usr/share/ironic-python-agent/venv/lib/python2.7/site-packages/ironic_lib/utils.py:104 sgdisk -ZFeb 18 21:08:17 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:17.304 15063 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): sgdisk -Z /dev/sda Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.321 15063 DEBUG oslo_concurrency.processutils [-] CMD "sgdisk -Z /dev/sda" returned: 0 in 1.017s Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.323 15063 DEBUG ironic_lib.utils [-] Execution completed, command line is "sgdisk -Z /dev/sda" Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.323 15063 DEBUG ironic_lib.utils [-] Command stdout is: "Creating new GPT entries in memory.Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: GPT data structures destroyed! You may now partition the disk using fdisk or other utilities. fuser /dev/sda Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.326 15063 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): fuser /dev/sda Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.332 15063 DEBUG oslo_concurrency.processutils [-] CMD "fuser /dev/sda" returned: 0 in 0.006s Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.332 15063 DEBUG ironic_lib.utils [-] Execution completed, command line is "fuser /dev/sda"Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.332 15063 DEBUG ironic_lib.utils [-] Command stdout is: " 15221" Feb 18 21:08:18 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:18.332 15063 DEBUG ironic_lib.utils [-] Command stderr is: "/dev/sda: fuser /dev/sda Feb 18 21:08:19 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:19.324 15063 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): fuser /dev/sda Feb 18 21:08:19 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:19.336 15063 DEBUG oslo_concurrency.processutils [-] CMD "fuser /dev/sda" returned: 1 in 0.012s Feb 18 21:08:19 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:19.336 15063 DEBUG ironic_lib.utils [-] Execution completed, command line is "fuser /dev/sda" Feb 18 21:08:19 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:19.336 15063 INFO ironic_lib.disk_utils [-] Disk metadata on /dev/sda successfully destroyed for node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 start iscsi:Feb 18 21:08:19 host-10-33-23-71 ironic-python-agent[15063]: 2020-02-18 21:08:19.336 15063 DEBUG ironic_python_agent.extensions.iscsi [-] Starting ISCSI target with iqn iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 on device /dev/sda start_iscsi_targetFeb 18 21:08:19 host-10-33-23-71 kernel: Rounding down aligned max_sectors from 4294967295 to 4294967288Feb 18 21:08:19 host-10-33-23-71 kernel: db_root: cannot open: /etc/targetFeb 18 21:08:19 host-10-33-23-71 WARNING ironic_python_agent.extensions.iscsi [-] Linux-IO is not available, falling back to TGT. Error: Cannot set dbroot to /etc/target. Please check if this directory exists..: RTSLibError: Cannot set dbroot to /etc/target. Please check if this directory exists.Feb 18 21:08:19 host-10-33-23-71 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tgtd Feb 18 21:08:19 host-10-33-23-71 DEBUG oslo_concurrency.processutils [-] CMD "tgtadm --lld iscsi --mode target --op show" returned: 0 in 0.002s Feb 18 21:08:19 host-10-33-23-71 DEBUG ironic_lib.utils [-] Execution completed, command line is "tgtadm --lld iscsi --mode target --op show" Feb 18 21:08:19 host-10-33-23-71 DEBUG ironic_lib.utils [-] Command stdout is: "" Feb 18 21:08:19 host-10-33-23-71 DEBUG ironic_lib.utils [-] Command stderr is: "" Feb 18 21:08:19 host-10-33-23-71 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tgtadm --lld iscsi --mode target --op new --tid 1 --targetname iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 Feb 18 21:08:19 host-10-33-23-71 DEBUG oslo_concurrency.processutils [-] CMD "tgtadm --lld iscsi --mode target --op new --tid 1 --targetname iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97" returned: 0 in 0.002s Feb 18 21:08:19 host-10-33-23-71 DEBUG ironic_lib.utils [-] Execution completed, command line is "tgtadm --lld iscsi --mode target --op new --tid 1 --targetname iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97" Feb 18 21:08:19 host-10-33-23-71 DEBUG ironic_lib.utils [-] Command stdout is: "" Feb 18 21:08:19 host-10-33-23-71 DEBUG ironic_lib.utils [-] Command stderr is: ""Feb 18 21:08:19 host-10-33-23-71 INFO root [-] Command iscsi.start_iscsi_target completed: Command name: start_iscsi_target, params: {u'wipe_disk_metadata': True, u'iqn': u'iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97', u'portal_port': 3260}, status: SUCCEEDED, result: {'iscsi_target_iqn': u'iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97'}.Feb 18 21:08:19 host-10-33-23-71 ironic-python-agent[15063]: ::ffff:10.33.24.87 - - [18/Feb/2020 21:08:19] "POST /v1/commands?wait=true HTTP/1.1" 200 386 ====================================Back to Ironic conductor020-02-18 13:08:25.953 346408 DEBUG RPC heartbeat called for node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 heartbeat 2020-02-18 13:08:25.997 346408 DEBUG Heartbeat from node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 heartbeat 2020-02-18 13:08:26.033 346408 Node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 moved to provision state "deploying" from state "wait call-back"; target provision state is "active" Starting iscsi:2020-02-18 13:08:28.385 Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 10.33.23.71:3260 -T iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 --login 2020-02-18 13:08:28.613 CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf iscsiadm -m node -p 10.33.23.71:3260 -T iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 --login" returned: 0 in 0.229s 2020-02-18 13:08:28.615 Execution completed, command line is "iscsiadm -m node -p 10.33.23.71:3260 -T iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 --login" 2020-02-18 13:08:28.615 Command stdout is: "Logging in to [iface: default, target: iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97, portal: 10.33.23.71,3260] (multiple) Successful login to iSCSI:Login to [iface: default, target: iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97, portal: 10.33.23.71,3260] successful. qemu-img info2020-02-18 13:08:29.005 346408 DEBUG Running cmd (subprocess): /usr/bin/python2 -m oslo_concurrency.prlimit --as=1073741824 -- env LC_ALL=C LANG=C qemu-img info /var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/disk 2020-02-18 13:08:29.072 346408 DEBUG CMD "/usr/bin/python2 -m oslo_concurrency.prlimit --as=1073741824 -- env LC_ALL=C LANG=C qemu-img info /var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/disk" returned: 0 in 0.067s 2020-02-18 13:08:29.073 346408 DEBUG Execution completed, command line is "env LC_ALL=C LANG=C qemu-img info /var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/disk" 2020-02-18 13:08:29.074 346408 DEBUG Command stdout is: "image: /var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/diskfile format: rawvirtual size: 9.8G (10485760000 bytes)disk size: 5.4G copying image via dd: 2020-02-18 13:08:29.075 Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf dd if=/var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/disk of=/dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 bs=1M oflag=direct 2020-02-18 13:08:49.020 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf dd if=/var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/disk of=/dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 bs=1M oflag=direct" returned: 0 in 19.945s 2020-02-18 13:08:49.021 346408 DEBUG Execution completed, command line is "dd if=/var/lib/ironic/images/ea6f8eda-402c-4fb0-a2c4-61c83bc73f97/disk of=/dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 bs=1M oflag=direct" 2020-02-18 13:08:49.022 346408 DEBUG Command stdout is: "" execute /usr/lib/python2.7/site-packages/ironic_lib/utils.py:1032020-02-18 13:08:49.022 346408 DEBUG Command stderr is: "10000+0 records in 10000+0 records out 10485760000 bytes (10 GB) copied, 19.7712 s, 530 MB/s partprobe2020-02-18 13:08:49.023 346408 DEBUGRunning cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf partprobe /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:03.341 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf partprobe /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 14.318s 2020-02-18 13:09:03.341 346408 DEBUG Execution completed, command line is "partprobe /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:03.342 346408 DEBUG Command stdout is: "" 2020-02-18 13:09:03.342 346408 DEBUG Command stderr is: "" lsblk:2020-02-18 13:09:03.343 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf lsblk -Po name,label /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:03.519 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf lsblk -Po name,label /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.176s 2020-02-18 13:09:03.519 346408 DEBUG Execution completed, command line is "lsblk -Po name,label /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:03.519 346408 DEBUG Command stdout is: "NAME="sdb" LABEL="" NAME="sdb1" LABEL="" " execute /usr/lib/python2.7/site-packages/ironic_lib/utils.py:1032020-02-18 13:09:03.520 346408 DEBUG iCommand stderr is: "" Adding configDrive partition:2020-02-18 13:09:03.772 346408 DEBUG Adding config drive partition 64 MiB to device: /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 for node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 create_config_drive_partition blkid2020-02-18 13:09:03.773 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf blkid -p -o value -s PTTYPE /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:03.959 346408 DEBUG oslo_concurrency.processutils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf blkid -p -o value -s PTTYPE /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.186s 2020-02-18 13:09:03.960 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Execution completed, command line is "blkid -p -o value -s PTTYPE /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:03.960 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stdout is: "dos 2020-02-18 13:09:03.960 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stderr is: "" parted2020-02-18 13:09:03.961 346408 DEBUG oRunning cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -s -m /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 unit MiB print execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:3722020-02-18 13:09:04.136 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -s -m /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 unit MiB print" returned: 0 in 0.175s 2020-02-18 13:09:04.137 346408 DEBUG Execution completed, command line is "parted -s -m /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 unit MiB print" 2020-02-18 13:09:04.137 346408 DEBUG Command stdout is: "BYT;/dev/sdb:953870MiB:scsi:512:512:msdos:IET VIRTUAL-DISK:;1:1.00MiB:9999MiB:9998MiB:ext4::boot; blkid2020-02-18 13:09:04.138 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf blkid -p -o value -s PTTYPE /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:04.321 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf blkid -p -o value -s PTTYPE /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.182s 2020-02-18 13:09:04.321 346408 DEBUG iExecution completed, command line is "blkid -p -o value -s PTTYPE /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:04.321 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stdout is: "dos2020-02-18 13:09:04.322 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stderr is: "" partprobe2020-02-18 13:09:04.322 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf partprobe -d -s /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:04.510 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf partprobe -d -s /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.188s 2020-02-18 13:09:04.511 346408 DEBUG Execution completed, command line is "partprobe -d -s /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:04.512 346408 DEBUG Command stdout is: "/dev/sdb: msdos partitions 1 " 2020-02-18 13:09:04.512 346408 DEBUG Command stderr is: "" blockdev --getsize642020-02-18 13:09:04.513 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf blockdev --getsize64 /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:05.045 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf blockdev --getsize64 /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.532s 2020-02-18 13:09:05.046 346408 DEBUG Execution completed, command line is "blockdev --getsize64 /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:05.046 346408 DEBUG Command stdout is: "1000204886016" 2020-02-18 13:09:05.046 346408 DEBUG Command stderr is: "" parted/mkpart2020-02-18 13:09:05.047 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s -- /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 mkpart primary fat32 -64MiB -0 2020-02-18 13:09:05.241 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -a optimal -s -- /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 mkpart primary fat32 -64MiB -0" returned: 0 in 0.194s 2020-02-18 13:09:05.241 346408 DEBUG Execution completed, command line is "parted -a optimal -s -- /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 mkpart primary fat32 -64MiB -0" 2020-02-18 13:09:05.242 346408 DEBUG Command stdout is: "" 2020-02-18 13:09:05.242 346408 DEBUG Command stderr is: "Warning: The resulting partition is not properly aligned for best performance. partprobe2020-02-18 13:09:05.329 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf partprobe /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:05.516 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf partprobe /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.187s2020-02-18 13:09:05.517 346408 DEBUG Execution completed, command line is "partprobe /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" 2020-02-18 13:09:05.517 346408 DEBUG Command stdout is: "" 2020-02-18 13:09:05.518 346408 DEBUG Command stderr is: "" sgdisk -v: IS THERE ANY ISSUE WITH THE BELOW OUTPUT2020-02-18 13:09:05.518 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf sgdisk -v /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 2020-02-18 13:09:05.707 346408 DEBUG oslo_concurrency.processutils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf sgdisk -v /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" returned: 0 in 0.189s 2020-02-18 13:09:05.708 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Execution completed, command line is "sgdisk -v /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1" execute /usr/lib/python2.7/site-packages/ironic_lib/utils.py:101 2020-02-18 13:09:05.708 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stdout is: "***************************************************************Found invalid GPT and valid MBR; converting MBR to GPT formatin memory.***************************************************************Warning! Secondary partition table overlaps the last partition by33 blocks!You will need to delete this partition or resize it in another utility.Warning! Secondary partition table overlaps the last partition by33 blocks!You will need to delete this partition or resize it in another utility.Identified 1 problems! 2020-02-18 13:09:05.709 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stderr is: "" parted2020-02-18 13:09:05.709 346408 DEBUG Running cmd (subprocess): sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -s -m /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 unit MiB print 2020-02-18 13:09:05.899 346408 DEBUG CMD "sudo ironic-rootwrap /etc/ironic/rootwrap.conf parted -s -m /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 unit MiB print" returned: 0 in 0.189s 2020-02-18 13:09:05.899 346408 DEBUG Execution completed, command line is "parted -s -m /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1 unit MiB print" 2020-02-18 13:09:05.900 346408 DEBUG Command stdout is: "BYT;/dev/sdb:953870MiB:scsi:512:512:msdos:IET VIRTUAL-DISK:;1:1.00MiB:9999MiB:9998MiB:ext4::boot;2:953806MiB:953870MiB:64.0MiB:::lba;" execute /usr/lib/python2.7/site-packages/ironic_lib/utils.py:1032020-02-18 13:09:05.900 346408 DEBUG ironic_lib.utils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Command stderr is: "" 2020-02-18 13:09:05.927 346408 DEBUG Waiting for the config drive partition /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1-part2 on node ea6f8eda-402c-4fb0-a2c4-61c83bc73f97 to be ready for writing. create_config_drive_partition /usr/lib/python2.7/site-packages/ironic_lib/disk_utils.py:8692020-02-18 13:09:05.927 346408 DEBUG oslo_concurrency.processutils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] Running cmd (subprocess): test -e /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1-part2 execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:3722020-02-18 13:09:05.945 346408 DEBUG oslo_concurrency.processutils [req-34f931af-69e5-4ff4-b37b-6b6df8d7f560 - - - - -] u'test -e /dev/disk/by-path/ip-10.33.23.71:3260-iscsi-iqn.2008-10.org.openstack:ea6f8eda-402c-4fb0-a2c4-61c83bc73f97-lun-1-part2' failed. Retrying. execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:461 The code retries several times followed by the config drive failure error.Again, this happens on a few nodes only and happens only when I try to run BIOS based images. UEFI based images provision just fine. Any help will be appreciated.thanks,Fred.

1 1

[oslo] Core reviewer changes
by Stephen Finucane 19 Feb '20

19 Feb '20

Hi all, Just an update on some recent changes that have been made to oslo-core after discussion among current, active cores. Added: scmcginnis Removed: ChangBo Guo(gcb) Davanum Srinivas (dims) Flavio Percoco Joshua Harlow Mehdi Abaakouk (sileht) Michael Still Victor Stinner lifeless Please let me know if anyone has any concerns about these changes. Cheers, Stephen

5 4

[operators][cinder][nova][glance] possible data loss situation (bug 1852106)
by Brian Rosmaita 19 Feb '20

19 Feb '20

tl;dr: If you are running the OpenStack Train release, add the following to the [DEFAULT]non_inheritable_image_properties configuration option [0] in your nova.conf: cinder_encryption_key_id,cinder_encryption_key_deletion_policy About Your nova.conf ==================== - if you already have a value set for non_inheritable_image_properties, add the above to what you currently have - if non_inheritable_image_properties is *not* set in your current nova.conf, you must set its value to include BOTH the above properties AND the default values (which you can see when you generate a sample nova configuration file [1]) NOTE: At a minimum, the non_inheritable_image_properties list should contain: - the properties used for image-signature-validation (these are not transferrable from image to image): * img_signature_hash_method * img_signature * img_signature_key_type * img_signature_certificate_uuid - the properties used to manage the keys for images of cinder encrypted volumes: * cinder_encryption_key_id * cinder_encryption_key_deletion_policy - review the documentation to determine whether you want to include the other default properties: * cache_in_nova * bittorrent Details ======= This issue is being tracked as Launchpad bug 1852106 [2]. This is probably a low-occurrence situation, because in order for the issue to occur, all of the following must happen: (0) using the OpenStack Train release (or code from master (Ussuri development)) (1) cinder_encryption_key_id and cinder_encryption_key_deletion_policy are NOT included in the non_inheritable_image_properties setting in nova.conf (which they aren't, by default) (2) a user has created a volume of an encrypted volume-type in the Block Storage service (cinder). Call this Volume-1 (3) using the Block Storage service, the user has uploaded the encrypted volume as an image to the Image service (glance). Call this Image-1 (4) using the Compute service (nova), the user has attempted to directly boot a server from the image. (Note: this is an unsupported action, the supported workflow is to use the image to boot-from-volume.) (5) although an unsupported action, if a user does (4), it currently results in a server in status ACTIVE but which is unusable because the operating system can't be found (6) using the Compute service, the user requests the createImage action on the unusable (yet ACTIVE) server, resulting in Image-2 (7) using the Image service, the user deletes Image-2 (which has inherited the cinder_encryption_key_* properties from Image-1) upon which the encryption key is deleted, thereby rendering Image-1 non-decryptable so that it can no longer be used in the normal boot-from-volume workflow NOTE 1: the cinder_encryption_key_deletion_policy image property was introduced in Train. In pre-Train releases, deleting the useless Image-2 in step (7) does NOT result in encryption key deletion. NOTE 2: Volume-1 created in step (2) has a *different* encryption key ID than the one associated with Image-1. Thus, even in the scenario where Image-1 becomes non-decryptable, Volume-1 is not affected. Workaround ========== When cinder_encryption_key_id,cinder_encryption_key_deletion_policy are added to the non_inheritable_image_properties setting in nova.conf, the useless Image-2 created in step (6) above will not have the image properties on it that enable Glance to delete the encryption key still in use by Image-1. This does not, however, protect images for which steps (4)-(6) have been performed before the deployment of this workaround. The safest way to deal with images created before the workaround is deployed is to remove the cinder_encryption_key_deletion_policy image property from any image that has it (or to change its value to 'do_not_delete'). While it is possible to use other image properties to identify images created by Nova as opposed to images created by Cinder, this is not guaranteed to be reliable because image properties may have been modified or removed by the image owner. Proposed Longer-term Fixes ========================== - In the Ussuri release, the unsupported action in step (4) above will result in a 400 rather than an active yet unusable server. Hence it will no longer be possible to create the image of the unusable server that causes the issue. [3] - Additionally, given that the image properties associated with cinder encrypted volumes and image signature validation are specific to a single image and should not be inherited by server snapshots under any circumstances, in Ussuri these "absolutely non-inheritable image properties" will no longer be required to appear in the non_inheritable_image_properties configuration setting in order to prevent them from being propagated to server snapshots. [4] References ========== [0] https://docs.openstack.org/nova/train/configuration/config.html#DEFAULT.non… [1] https://docs.openstack.org/nova/train/configuration/sample-config.html [2] https://bugs.launchpad.net/nova/+bug/1852106 [3] https://review.opendev.org/#/c/707738/ [4] https://review.opendev.org/#/c/708126/

2 1

[operators][cinder] driver removal policy update
by Brian Rosmaita 19 Feb '20

19 Feb '20

The Cinder driver removal policy has been updated [0]. Drivers eligible for removal, at the discretion of the team, may remain in the code repository as long as they continue to pass OpenStack CI testing. When such a driver blocks the CI check or gate, it will be removed immediately. The intent of the policy revision is twofold. First, it gives vendors a longer grace period in which to make the necessary changes to have their drivers reinstated as ‘supported’. Second, keeping these drivers in-tree longer should make life easier for operators who have deployed storage backends with drivers that have been marked as ‘unsupported’. Please see the full statement of the policy [0] for details. [0] https://docs.openstack.org/cinder/latest/drivers-all-about.html#driver-remo…

1 0

[cinder] volume-local-cache spec - last call
by Brian Rosmaita 19 Feb '20

19 Feb '20

The volume-local-cache spec [0], which was granted a Cinder spec-freeze exception, currently has two +2s. It's been reviewed by a lot of people, and as far as I can tell, everyone's concerns have been addressed at this point. However, I'd like to give people one last opportunity to verify this for themselves. Unless a substantive objection is raised, I intend to approve the spec around 12:30 UTC tomorrow (Thursday) so that it can be merged before the Nova meeting at 14:00 UTC. cheers, brian [0] https://review.opendev.org/#/c/684556/

1 0

[kolla] Following Nova with XenAPI deprecation
by Radosław Piliszek 19 Feb '20

19 Feb '20

Hello fellow OpenStackers! There exists some logic in Kolla Ansible that handles XenAPI-specific overrides and bootstrapping with os-xenapi. It is one of those murky, untested features that we prefer to remove as soon as possible. :-) Hence we follow Nova in deprecating this functionality for removal. We will do the removal in Victoria. If you are using XenAPI with recent Kolla-Ansible, please let us know. I will be proposing deprec changes later today. As far as other deployment projects are concerned, DevStack, OpenStack-Helm and Puppet-OpenStack seem to still handle XenAPI specifics. -yoctozepto

1 0

[OSSA-2020-001] Nova can leak consoleauth token into log files (CVE-2015-9543)
by Jeremy Stanley 19 Feb '20

19 Feb '20

============================================================= OSSA-2020-001: Nova can leak consoleauth token into log files ============================================================= :Date: February 19, 2020 :CVE: CVE-2015-9543 Affects ~~~~~~~ - Nova: <18.2.4,>=19.0.0<19.1.0,>=20.0.0<20.1.0 Description ~~~~~~~~~~~ Paul Carlton from HP reported a vulnerability in Nova. An attacker with read access to the serviceâ€™s logs may obtain tokens used for console access. All Nova setups using novncproxy are affected. Patches ~~~~~~~ - https://review.opendev.org/707845 (Queens) - https://review.opendev.org/704255 (Rocky) - https://review.opendev.org/702181 (Stein) - https://review.opendev.org/696685 (Train) - https://review.opendev.org/220622 (Ussuri) Credits ~~~~~~~ - Paul Carlton from HP (CVE-2015-9543) References ~~~~~~~~~~ - https://launchpad.net/bugs/1492140 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-9543 Notes ~~~~~ - The stable/queens branch is under extended maintenance and will receive no new point releases, but a patch for it is provided as a courtesy. -- Jeremy Stanley, on behalf of OpenStack Vulnerability Management

1 0

[oslo][release] oslo.limit mistakenly released as 1.0.0
by Ben Nemec 19 Feb '20

19 Feb '20

Hi, We seem to have created a bit of a problem with the latest oslo.limit release. In keeping with our general policy of bumping the major version when we release libraries without py2 support, oslo.limit got bumped to 1.0. Unfortunately, being a pre-1.0 library it should have only had the minor version bumped. This puts us in an awkward situation since the library is still under heavy development and we expect the API will change, possibly multiple times, before we're ready to commit to a stable API. We also need the ability to continue doing releases during development so we can test the library with consuming projects. I can think of a few options, although I won't guarantee all of these are even possible: * Unpublish 1.0.0 and do further pre-1.0 development on a feature branch cut from before we released 1.0.0. Once we're ready for "1.0", we merge the feature branch to master and release it as 2.0.0. * Stick a big disclaimer in the 1.0.0 docs that it is still under development and proceed to treat 1.0 the same as we would have treated a pre-1.0 library. Again, when ready for "1.0" we tag it 2.0.0. * Make our breaking changes as needed and just continue bumping the major version every release. This unfortunately makes it hard to communicate via the version when the library is ready for use. :-/ * [some better idea that you suggest :-)] Any input on the best way to handle this is greatly appreciated. Thanks. -Ben

9 22

[vmware][hyperv][kolla] Is there any interest? (or should we deprecate and remove?)
by Radosław Piliszek 19 Feb '20

19 Feb '20

Hello Fellow Stackers, In Kolla and Kolla-Ansible we have some support for VMware (both hypervisor and networking controller stuff) and Hyper-V. The issue is the relevant code is in pretty bad shape and we had no recent reports about these being used nor working at all for that matter and we are looking into dropping support for these. Please respond if you are interested in these. Long term we would require access to some CI running these to really keep things in shape. -yoctozepto

1 1

Re: [openstack-dev][kuryr] Not working in ARM64 (as Node)
by mdulko＠redhat.com 19 Feb '20

19 Feb '20

Hi, So from this run you need the kuryr-controller logs. Apparently the pod never got annotated with an information about the VIF. Thanks, Michał On Fri, 2020-02-07 at 05:14 +0000, VeeraReddy wrote: > Hi mdulko, > Thanks for your support. > > As you mention i removed readinessProbe and > livenessProbe from Kuryr pod definitions. Still i am facing issue , unable to create pod. > > > > > Attached kubelet and kuryr-cni logs. > > > > Regards, > Veera. > > > On Thursday, 6 February, 2020, 05:19:12 pm IST, <mdulko(a)redhat.com> wrote: > > > Hm, nothing too troubling there too, besides Kubernetes not answering > on /healthz endpoint. Are those full logs, including the moment you > tried spawning a container there? It seems like you only pasted the > fragments with tracebacks regarding failures to read /healthz endpoint > of kube-apiserver. That is another problem you should investigate - > that causes Kuryr pods to restart. > > At first I'd disable the healthchecks (remove readinessProbe and > livenessProbe from Kuryr pod definitions) and try to get fresh set of > logs. > > On Thu, 2020-02-06 at 10:46 +0000, VeeraReddy wrote: > > Hi mdulko, > > Please find kuryr-cni logs > > http://paste.openstack.org/show/789209/ > > > > > > Regards, > > Veera. > > > > > > On Thursday, 6 February, 2020, 04:08:35 pm IST, <mdulko(a)redhat.com> wrote: > > > > > > Hi, > > > > The logs you provided doesn't seem to indicate any issues. Please > > provide logs of kuryr-daemon (kuryr-cni pod). > > > > Thanks, > > Michał > > > > On Thu, 2020-02-06 at 07:09 +0000, VeeraReddy wrote: > > > Hi, > > > I am trying to run kubelet in arm64 platform > > > 1. Generated kuryr-cni successfullly. using kur-cni Dockerfile > > > 2. Generated kuryr-cni-arm64 container. > > > 3. my kube-kuryr-arm64.yml (http://paste.openstack.org/show/789208/) > > > > > > My master node in x86 installed successfully using devstack > > > > > > While running kubelet in arm platform , not able to create kubelet interface (kubelet logs: http://paste.openstack.org/show/789206/) > > > > > > COntroller logs: http://paste.openstack.org/show/789209/ > > > > > > Please help me to fix the issue > > > > > > Veera. > > > > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > Veera. > > > > > >

2 6

Virtio memory balloon driver
by Albert Braden 18 Feb '20

18 Feb '20

When we build a Centos 7 VM with 1.4T RAM it fails with "[ 17.797177] BUG: unable to handle kernel paging request at ffff988b19478000" I asked in #centos and they asked me to show a list of devices from a working VM (if I use 720G RAM it works). This is the list: [root@alberttest1 ~]# lspci 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 00:01.2 USB controller: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton II] (rev 01) 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 00:03.0 Ethernet controller: Red Hat, Inc. Virtio network device 00:04.0 SCSI storage controller: Red Hat, Inc. Virtio block device 00:05.0 SCSI storage controller: Red Hat, Inc. Virtio block device 00:06.0 SCSI storage controller: Red Hat, Inc. Virtio block device 00:07.0 Unclassified device [00ff]: Red Hat, Inc. Virtio memory balloon [root@alberttest1 ~]# lsusb Bus 001 Device 002: ID 0627:0001 Adomax Technology Co., Ltd Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub They suspect that the "Virtio memory balloon" driver is causing the problem, and that we should disable it. I googled around and found this: http://www.linux-kvm.org/page/Projects/auto-ballooning It looks like memory ballooning is deprecated. How can I get rid of the driver? Also they complained about my host bridge device; they say that we should have a newer one: 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) Where can I specify the host bridge? <bugs_> ok ozzzo one of the devices is called "virtio memory balloon" [13:18:12] <bugs_> do you see that? [13:18:21] <ozzzo> yes [13:18:47] <bugs_> i suggest you google that and read about what it does - i think it would [13:19:02] <bugs_> be worth trying to disable that device on your larger vm to see what happens [13:19:18] <ozzzo> ok I will try that, thank you [13:19:30] * Altiare (~Altiare@unaffiliated/altiare) has quit IRC (Quit: Leaving) [13:21:45] * Sheogorath[m] (sheogora1(a)gateway/shell/matrix.org/x-uiiwpoddodtgrwwz) joins #centos [13:22:06] <@TrevorH> I also notice that the VM seems to be using the very old 440FX and there's a newer model of hardware available that might be worth checking [13:22:21] <@TrevorH> 440FX chipset is the old old pentium Pro chipset! [13:22:32] <@TrevorH> I had one of those in about 1996

4 11

ops meetups team meeting 2020-2-18
by Chris Morgan 18 Feb '20

18 Feb '20

The OpenStack Ops Meetups team meeting was held today on IRC, minutes linked below. Key links vancouver opendev+ptg : https://www.openstack.org/events/opendev-ptg-2020/ South Korea Ops Meetup proposal : https://etherpad.openstack.org/p/ops-meetup-2nd-korea-2020 OpenInfra summit 2020 announced for Berlin, Oct 19-23 : https://superuser.openstack.org/articles/inside-open-infrastructure-the-lat… The meetups team will be proposing content for Vancouver and will shortly solicit feedback on preferred dates for the Ops Meetups even in South Korea. Please follow https://twitter.com/osopsmeetup for more announcements. Minutes: <•openstack> Meeting ended Tue Feb 18 15:37:42 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) 10:37 AM Minutes: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2020/ops_meetup_tea… 10:37 AM Minutes (text): http://eavesdrop.openstack.org/meetings/ops_meetup_team/2020/ops_meetup_tea… 10:37 AM Log: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2020/ops_meetup_tea… Chris Morgan - on behalf of the OpenStack Ops Meetups team ( https://wiki.openstack.org/wiki/Ops_Meetups_Team) -- Chris Morgan <mihalis68(a)gmail.com>

1 0

Re: [Release-job-failures] Release of openstack/puppet-keystone for ref refs/tags/16.1.0 failed
by Thierry Carrez 18 Feb '20

18 Feb '20

We had a series of failures uploading Puppet artifacts to the forge: Forge API failed to upload tarball with code: 500 errors: An error was encountered while processing your request, please try again later. This affected: openstack/puppet-ceilometer (16.1.0) https://zuul.opendev.org/t/openstack/build/5b0b890d777545c098ea88b70eb89b52 openstack/puppet-panko (16.1.0) https://zuul.opendev.org/t/openstack/build/073cbb3b72ac4241aa1802ccbaf4ada5 openstack/puppet-murano (16.1.0) https://zuul.opendev.org/t/openstack/build/b07a78bd59c84176bb7d9d2f1e0f635c openstack/puppet-keystone (16.1.0) https://zuul.opendev.org/t/openstack/build/be7510a717d3413ab008ab3d34db3672 As a result the tarballs were not uploaded to tarballs.openstack.org either. I think we should reenqueue the tag reference to retrigger the release-openstack-puppet job for those ? -- Thierry Carrez (ttx)

2 1

[neutron] Can br-ex and br-tun use the same interface?
by Zhengyu Pan 18 Feb '20

18 Feb '20

hi, I have only two physical interfaces. In my deploying, network node and compute node are the same. Can Bridge br-tun and br-ex use the same interface when using vxlan network type ? management network use the other interface?

3 5

[nova][cinder] What should the behaviour of extend_volume be with attached encrypted volumes?
by Lee Yarwood 18 Feb '20

18 Feb '20

Hello all, The following bug was raised recently regarding a failure to extend attached encrypted volumes: Failing to extend an attached encrypted volume https://bugs.launchpad.net/nova/+bug/1861071 I've worked up a series below that resolves this for LUKSv1 volumes by taking the LUKSv1 header into account before calling Libvirt to resize the block device within the instance: https://review.opendev.org/#/q/topic:bug/1861071 This results in the instance visable block device being resized to a size just smaller than that requested through Cinder's API. My question to the list is if that behaviour is acceptable given the same call to extend an attached unencrypted volume *will* grow the instance visable block device to the requested size? Many thanks in advance, -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76

1 1

[kolla][zun] Zun image source
by Hongbin Lu 18 Feb '20

18 Feb '20

Hi Kolla team, I was looking into the CentOS Zun image downloaded from DockerHub. It looks the source code is from stable/train branch: $ docker run kolla/centos-source-zun-compute:master cat zun/zun.egg-info/pbr.json {"git_version": "25e56636", "is_release": false} The git_version points to the stable/train branch, but I think it should point to the master branch. FWIW, I also checked the Ubuntu image and the git version in there looks correct: $ docker run kolla/ubuntu-source-zun-compute:master cat zun/zun.egg-info/pbr.json {"git_version": "6fbf52ae", "is_release": false} Any suggestion? Best regards, Hongbin

2 1

[tripleo] deep-dive containers & tooling
by Emilien Macchi 17 Feb '20

17 Feb '20

Hi folks, On Friday I'll do a deep-dive on where we are with container tools. It's basically an update on the removal of Paunch, what will change etc. I'll be on Bluejeans at 2pm UTC, anyone is welcome to join and ask questions or give feedback. https://bluejeans.com/6007759543 Link of the slides: https://docs.google.com/presentation/d/111sEwyIKxx2NCqTIQizdPNVaRbv0HAT5YeM… -- Emilien Macchi

2 4

Re: [Ask OpenStack] 7 updates about "galera", "rabbiitmq", "auth_token", "keysone", "swift-proxy" and more
by Gaurav Goyal 17 Feb '20

17 Feb '20

Dear Openstack Experts, We are using a big openstack setup in our organization and different operation teams adds new computes nodes to it. We as system admin of this Openstack environment wants to audit the configuration parameters on all the nodes. Can you please help to suggest a better way (Tools) to audit this environment? Awaiting your kind reply. Regards Gaurav Goyal On Wed, Feb 12, 2020 at 10:26 PM <communitymngr(a)openstack.org> wrote: > Dear Gaurav Goyal, > > Ask OpenStack has these updates, please have a look: > > - Kubernetes cluster created with Magnum doesn't work > <https://ask.openstack.org/en/question/126164/kubernetes-cluster-created-wit…> (new > question) > - how do i list the top objects consuming more space in the swift > container > <https://ask.openstack.org/en/question/126161/how-do-i-list-the-top-objects-…> (new > question) > - action requests is empty > <https://ask.openstack.org/en/question/126160/action-requests-is-empty/?sort…> (new > question) > - How to Migrate OpenStack Instance from one Tenant to another > <https://ask.openstack.org/en/question/126159/how-to-migrate-openstack-insta…> (new > question) > - Architecture of keystone, swift and memcached together > <https://ask.openstack.org/en/question/126158/architecture-of-keystone-swift…> (3 > rev) > - why is octavia not using keystone public endpoint to validate tokens? > <https://ask.openstack.org/en/question/126156/why-is-octavia-not-using-keyst…> (new > question) > - Error while launching instance on openstack > <https://ask.openstack.org/en/question/126150/error-while-launching-instance…> (2 > rev, 1 ans, 3 ans rev) > > To change frequency and content of these alerts, please visit your user > profile > <https://ask.openstack.org/en/users/18141/gaurav-goyal/subscriptions/>. > To unsubscribe, visit this page > <https://ask.openstack.org/en/users/unsubscribe/?key=89e52fdededaf7e8066737b…> > If you believe that this message was sent in an error, please email about > it the forum administrator at communitymngr(a)openstack.org. > ------------------------------ > > Sincerely, > Ask OpenStack Administrator > > To unsubscribe: there's a big "Stop Email" button in the link to your user > profile above. >

1 0

[neutron] bug deputy report (Feb 10-17)
by Akihiro Motoki 17 Feb '20

17 Feb '20

Hi, Here is a neutron bug deputy report last week. While 15 new bugs were reported, several bugs needs further investigation and are not triaged yet. I will look into them but it would be great if they need attentions. --- Undecided - https://bugs.launchpad.net/neutron/+bug/1862611 Neutron try to register invalid host to nova aggregate for ironic routed network Undecided It was covered by the last week report, but it needs more eyes familiar with routed network. - https://bugs.launchpad.net/neutron/+bug/1862851 update_floatingip_statuses: StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. Undecided Needs to be checked by folks familiar with DVR. It looks like a race condition. - https://bugs.launchpad.net/neutron/+bug/1862932 [neutron-bgp-dragent] passive agents send wrong number of routes Undecided Needs attention by folks familiar with dynamic-routing stuff. - https://bugs.launchpad.net/neutron/+bug/1863068 Dublicated Neutron Meter Rules in different projects kills metering Undecided - https://bugs.launchpad.net/neutron/+bug/1863091 IPVS setup fails with openvswitch firewall driver, works with iptables_hybrid Undecided - https://bugs.launchpad.net/neutron/+bug/1863110 2/3 snat namespace transitions to master Undecided - https://bugs.launchpad.net/neutron/+bug/1863201 stein regression listing security group rules Undecided - https://bugs.launchpad.net/neutron/+bug/1863213 Spawning of DHCP processes fail: invalid netcat options Undecided ralonsoh assigned himself. Any update on this? Incomplete - https://bugs.launchpad.net/neutron/+bug/1863206 Port is reported with 'port_security_enabled=True' without port-security extension Incomplete I think the bug author misunderstood the default of port-security extension behavior. Double check would be appreciated. Confirmed - https://bugs.launchpad.net/neutron/+bug/1863577 [ovn] tempest.scenario.test_network_v6.TestGettingAddress tests failing 100% times Confirmed, High In Progress - https://bugs.launchpad.net/neutron/+bug/1862618 [OVN] functional test test_virtual_port_delete_parents is unstable In Progress, Medium - https://bugs.launchpad.net/neutron/+bug/1862703 Neutron remote security group does not work In Progress, High - https://bugs.launchpad.net/neutron/+bug/1862648 [OVN] Reduce the number of tables watched by MetadataProxyHandler High, Fix Released - https://bugs.launchpad.net/neutron/+bug/1862893 [OVN]Updating a QoS policy for a port will cause a KeyEerror In Progress, Low Fix Released - https://bugs.launchpad.net/neutron/+bug/1862927 "ncat" rootwrap filter is missing Fix Released The root cause turned out that we need to specify rootwrap filter config by abspath.

1 0