September 2019 - openstack-discuss

OSA upgrading Xenial Queens to Bionic Rocky
by Grant Morley 10 Sep '19

10 Sep '19

Hi all, I was wondering if there was a guide for upgrading OpenStack Ansible from Ubuntu 16.04 Queens to Ubuntu 18.04 Rocky? I remember a long time ago there was an etherpad set up for upgrading from 14.04 -> 16.04 but I can't seem to find anything similar for going to 18.04. Annoyingly as we don't have lots of hardware, we are going to have to upgrade in place. If there are any guides that would be much appreciated. Many thanks. -- Grant Morley Cloud Lead, Civo Ltd www.civo.com <https://www.civo.com/>| Signup for an account! <https://www.civo.com/signup>

2 1

[neutron][vmware][vsphere] integration
by Ionut Biru 10 Sep '19

10 Sep '19

Hello guys, I'm trying to integrate openstack stein with an already running vmware vsphere cluster. All the documentation that I found explain how do it with distributed switches or port groups but currently in my setup, vmware is using Standard Network. Openstack Stein was deployed using OSA , neutron was configured using ovs and i configured the integrated_bridge to br-int. I tried first to deployed using linux-bridge but when I tried to deployed an instance, neutron returned that only ovs or dvs method is supported. Now when ovs when I'm deploying an instance with network, nova returns an error message: 2019-09-10 10:15:19.010 22443 ERROR nova.compute.manager [instance: d8d1cbb8-5c1c-4b98-9739-bea0668cfaa5] VimFaultException: An error occurred during host configuration. 2019-09-10 10:15:19.010 22443 ERROR nova.compute.manager [instance: d8d1cbb8-5c1c-4b98-9739-bea0668cfaa5] Faults: ['PlatformConfigFault'] How do you guys integrate neutron with vmware vsphere with standard network? Is there a driver that I need to use? -- Ionut Biru - https://fleio.com

1 0

Thank you Stackers for five amazing years!
by Chris Hoge 10 Sep '19

10 Sep '19

Hi everyone, After more than nine years working in cloud computing and on OpenStack, I've decided that it is time for a change and will be moving on from the OpenStack Foundation. For the last five years I've had the honor of helping to support this vibrant community, and I'm going to deeply miss being a part of it. OpenStack has been a central part of my life for so long that it's hard to imagine a work life without it. I'm proud to have helped in some small way to create a lasting project and community that has, and will continue to, transform how infrastructure is managed. September 12 will officially be my last day with the OpenStack Foundation. As I make the move away from my responsibilities, I'll be working with community members to help ensure continuity of my efforts. Thank you to everyone for building such an incredible community filled with talented, smart, funny, and kind people. You've built something special here, and we're all better for it. I'll still be involved with open source. If you ever want to get in touch, be it with questions about work I've been involved with or to talk about some exciting new tech or to just catch up over a tasty meal, I'm just a message away in all the usual places. Sincerely, Chris chris(a)hogepodge.com Twitter/IRC/everywhere else: @hogepodge

10 9

[keystone] cannot use 'openstack trust list' without admin role
by Francois Scheurer 10 Sep '19

10 Sep '19

Dear Keystone Experts, I have an issue with the openstack client in stage (using Rocky), using a user 'fsc' without 'admin' role and with password auth. 'openstack trust create/show' works. 'openstack trust list' is denied. But keystone policy.json says: "identity:create_trust": "user_id:%(trust.trustor_user_id)s", "identity:list_trusts": "", "identity:list_roles_for_trust": "", "identity:get_role_for_trust": "", "identity:delete_trust": "", "identity:get_trust": "", So "openstack list trusts" is always allowed. In keystone log (I replaced the uid's by names in the ouput below) I see that 'identity:list_trusts()' was actually granted but just after that a_*admin_required()*_ is getting checked and fails... I wonder why... There is also a flag*is_admin_project=True* in the rbac creds for some reason... Any clue? Many thanks in advance! Cheers Francois #openstack --os-cloud stage-fsc trust create --project fscproject --role creator fsc fsc #=> fail because of the names and policy rules, but using uid's it works openstack --os-cloud stage-fsc trust create --project aeac4b07d8b144178c43c65f29fa9dac --role 085180eeaf354426b01908cca8e82792 3e9b1a4fe95048a3b98fb5abebd44f6c 3e9b1a4fe95048a3b98fb5abebd44f6c +--------------------+----------------------------------+ | Field | Value | +--------------------+----------------------------------+ | deleted_at | None | | expires_at | None | | id | e74bcdf125e049c69c2e0ab1b182df5b | | impersonation | False | | project_id | fscproject | | redelegation_count | 0 | | remaining_uses | None | | roles | creator | | trustee_user_id | fsc | | trustor_user_id | fsc | +--------------------+----------------------------------+ openstack --os-cloud stage-fsc trust show e74bcdf125e049c69c2e0ab1b182df5b +--------------------+----------------------------------+ | Field | Value | +--------------------+----------------------------------+ | deleted_at | None | | expires_at | None | | id | e74bcdf125e049c69c2e0ab1b182df5b | | impersonation | False | | project_id | fscproject | | redelegation_count | 0 | | remaining_uses | None | | roles | creator | | trustee_user_id | fsc | | trustor_user_id | fsc | +--------------------+----------------------------------+ #this fails: openstack --os-cloud stage-fsc trust list *You are not authorized to perform the requested action: admin_required. (HTTP 403)*

3 3

[ironic] Ironic tempest jobs hitting retry_limit failures
by Clark Boylan 09 Sep '19

09 Sep '19

Hello Ironic, We've noticed that your tempest jobs have been hitting retry_limit failure recently. What this means is we attempted to run the job 3 times but each time the job failed due to "network" problems and Zuul eventually gave up. On further investigation I found that this is happening because the ironic tempest jobs are filling the root disk on rackspace nodes (which have a smaller root / + ephemeral drive mounted at /opt) with libvirt qcow2 images. This seems to cause ansible to fail to operate because it needs to write to /tmp and it thinks there is a "network" error. I've thrown my investigation into a bug for you [0]. It would be great if you could take a look at this as we are effectively spinning our wheels for about 9 hours every time this happens. I did hold the node I used to investigate. If you'd like to dig in yourselves just ask the infra team for access to nodepool node ubuntu-bionic-rax-ord-0011007873. Finally, to help debug these issues in the future I've started adding a cleanup-run playbook [1] which should give us network and disk info (can be expanded if necessary too) for every job when it is done running. Even if the disk is full. [0] https://storyboard.openstack.org/#!/story/2006520 [1] https://review.opendev.org/#/c/681100/ Clark

1 0

[devstack][qa][python3] "also install the Python 2 dev library" - still needed?
by Peter Penchev 09 Sep '19

09 Sep '19

Hi, When devstack's `setup_dev_lib` function is invoked and USE_PYTHON3 has been specified, this function tries to also install the development library for Python 2.x, I guess just in case some package has not declared proper Python 3 support or something. It then proceeds to install the Python 3 version of the library and all its dependencies. Unfortunately there is a problem with that, and specifically with script files installed in the system's executable files directory, e.g. /usr/local/bin. The problem appears when some Python library has already been installed for Python 3 (and has installed its script files), but is now installed for Python 2 (overwriting the script files) and is then not forcefully reinstalled for Python 3, since it is already present. Thus, the script files are last modified by the Python 2 library installation and they have a hashbang line saying `python2.x` - so if something then tries to execute them, they will run and use modules and libraries for Python 2 only. We experienced this problem when running the cinderlib tests from Cinder's `playbooks/cinderlib-run.yaml` file - it finds a unit2 executable (installed by the unittest2 library) and runs it, hoping that unit2 will be able to discover and collect the cinderlib tests and load the cinderlib modules. However, since unittest2 has last been installed as a Python 2 library, unit2 runs with Python 2 and fails to locate the cinderlib modules. (Yes, we know that there are other ways to run the cinderlib tests; this message is about the problem exposed by this way of running them) The obvious solution would be to instruct the Python 2 pip to not install script (or other shared) files at all; unfortunately, https://github.com/pypa/pip/issues/3980 ("Option to exclude scripts on install"), detailing a very similar use case ("need it installed for Python 2, but want to use it with Python 3") has been open for almost exactly three years now with no progress. I wonder if I could try to help, but even if this issue is resolved, there will be some time before OpenStack can actually depend on a recent enough version of pip. A horrible workaround would be to find the binary directory before installing the Python 2 library (using something like `pip3.7 show somepackage` and then running some heuristics on the "Location" field), tar'ing it up and then restoring it... but I don't know if I even want to think about this. Another possible way forward would be to consider whether we still want the Python 2 libraries installed - is OpenStack's Python 3 transition reached a far enough stage to assume that any projects that still require Python 2 *and* fail to declare their Python 2 dependencies properly are buggy? To be honest, this seems the most reasonable path for me - drop the "also install the Python 2 libs" code and see what happens. I could try to make this change in a couple of test runs in our third-party Cinder CI system and see if something breaks. Here is a breakdown of what happens, with links to the log of the StorPool third-party CI system for Cinder: https://spfactory.storpool.com/logs/80/639180/35/check/cinder-storpool-temp… `stack.sh` invokes `pip_install` for `os-testr` https://spfactory.storpool.com/logs/80/639180/35/check/cinder-storpool-temp… `pip_install` sees that we want a Python 3 installation and invokes `pip3.7` to install os-testr. https://spfactory.storpool.com/logs/80/639180/35/check/cinder-storpool-temp… `pip3.7` wants to install `unittest2` https://spfactory.storpool.com/logs/80/639180/35/check/cinder-storpool-temp… `pip3.7` has installed `unittest2` - now `/usr/local/bin/unit2` has a hashbang line saying `python3.7` Now this is where it gets, uhm, interesting: https://spfactory.storpool.com/logs/80/639180/35/check/cinder-storpool-temp… `setup_dev_lib` is invoked for `os-brick` https://spfactory.storpool.com/logs/80/639180/35/check/cinder-storpool-temp… `setup_dev_lib`, seeing that we really want a Python 3 installation, decides to install `os-brick` for Python 2 just in case. https://spfactory.storpool.com/logs/80/639180/35/check/cinder-storpool-temp… `pip2.7` is invoked to install `os-brick` and its dependencies. https://spfactory.storpool.com/logs/80/639180/35/check/cinder-storpool-temp… `pip2.7` decides it wants to install `unittest2`, too. https://spfactory.storpool.com/logs/80/639180/35/check/cinder-storpool-temp… `pip2.7` has installed `unittest2`, and now `/usr/local/bin/unit2` has a hasbang line saying `python2.7` https://spfactory.storpool.com/logs/80/639180/35/check/cinder-storpool-temp… `setup_dev_lib` turns the Python 3 flag back on. https://spfactory.storpool.com/logs/80/639180/35/check/cinder-storpool-temp… `pip3.7` is invoked to install `os-brick` https://spfactory.storpool.com/logs/80/639180/35/check/cinder-storpool-temp… `pip3.7` decides (correctly) that it has already installed `unittest2`, so (only partially correctly) it does not need to install it again. Thus `/usr/local/bin/unit2` is left with a hashbang line saying `python2.7`. Thanks for reading this far, I guess :) G'luck, Peter

4 5

Bug Deputy September 2 - September 9
by Nate Johnston 09 Sep '19

09 Sep '19

Neutrinos, Here is the bug deputy report for this past week. It was an exciting week of CI problems; thanks to everyone who pitched in to get us past those thorny issues. I would call out specifically the two bugs rated as High that have no assignee yet; they are both gate related - one for Neutron, and the other affecting the Patrole project. There is also one bug left Untriaged because I was unable to validate the bug; I would appreciate a triaging look if you can. Thanks! Nate ---- Critical - https://bugs.launchpad.net/bugs/1842482 "test_get_devices_info_veth_different_namespaces" fails because veth1_1 interface has a link device in the same namespace Status: Fix Committed (slaweq) https://review.opendev.org/680001 - https://bugs.launchpad.net/bugs/1842517 neutron-sanity-check command fails if netdev datapath is used Status: In Progress (deepak.tiwari) No change registered in LP bug - https://bugs.launchpad.net/bugs/1842657 Job networking-ovn-tempest-dsvm-ovs-release is failing 100% times Status: Fix Committed (maciej.josefczyk) https://review.opendev.org/661065 - https://bugs.launchpad.net/bugs/1842659 Funtional tests of start and restart services failing 100% times Status: Fix Committed (slaweq) https://review.opendev.org/680001 High - https://bugs.launchpad.net/bugs/1843285 Trunk scenario test test_subport_connectivity failing with iptables_hybrid fw driver Status: Unassigned - https://bugs.launchpad.net/bugs/1842666 Bulk port creation with supplied security group also adds default security group Status: In Progress (njohnston) https://review.opendev.org/679852 - https://bugs.launchpad.net/bugs/1843025 FWaaS v2 fails to add ICMPv6 rules via horizon Status: In Progress (haleyb) https://review.opendev.org/680753 - https://bugs.launchpad.net/bugs/1843282 Rally CI not working since jsonschema version bump Status: Fix Committed (ralonsoh) https://review.opendev.org/681001 - https://bugs.launchpad.net/bugs/1843290 Remove network flavor profile fails Status: Unassigned Note: Currently breaking the gate for the Patrole project Medium - https://bugs.launchpad.net/bugs/1842327 Report in logs when FIP associate and disassociate Status: In progress (ralonsoh) https://review.opendev.org/680976 Low - https://bugs.launchpad.net/bugs/1842934 multicast scenario test failing when guest image don't have python3 installed Status: In Progress (slaweq) https://review.opendev.org/680428 - https://bugs.launchpad.net/bugs/1842937 Some ports assigned to routers don't have the correspondent routerport register Status: In Progress (ralonsoh) No change registered in LP bug - https://bugs.launchpad.net/bugs/1843269 Nova notifier called even if set to False Status: In Progress (haleyb) https://review.opendev.org/681016 RFE - https://bugs.launchpad.net/bugs/1843218 allow to create record on default zone from tenants Untriaged - https://bugs.launchpad.net/bugs/1843211 network-ip-availabilities' result is not correct when the subnet has no allocation-pool Status: Unassigned

1 1

[edge] Edge whitepaper and tutorial authors needed
by Ildiko Vancsa 09 Sep '19

09 Sep '19

Hi, I’m reaching out to point you to two mail threads on the edge-computing mailing list. The edge working group is looking into writing up a second whitepaper with a few detailed use cases and information about the reference architecture work the group has been doing. If you are interested in this work please __reach out to me or check out this mail thread__: http://lists.openstack.org/pipermail/edge-computing/2019-September/000632.h… The other work item is writing up edge tutorials about frameworks to a German magazine. __This is a short deadline activity, please reach out to me if you are interested in participating.__ For further information please see this mail thread: http://lists.openstack.org/pipermail/edge-computing/2019-September/000633.h… Please let me know if you have questions to any of the above. Thanks, Ildikó

1 0

[nova] Offline Installation of Openstack
by Premdeep S 09 Sep '19

09 Sep '19

Hi Team, Requesting your help on below. We have a requirement to setup Openstack in an isolated infra. We will not be provided with Internet. How can we set it up? 1. Can we have a local repository (Rocky, Universal, etc)created? If so how do we manage it? 2. We have noticed lot of package dependencies while setting up Openstack Infra, so will creating a local repository help in an implementation when we do not have an internet. What is the success rate? Thanks Prem

2 2

[manila][ops] Shanghai Forum - Manila Topic Planning
by Tom Barron 09 Sep '19

09 Sep '19

As mentioned several times in the Manila Community Meeting, we have posted an etherpad to brainstorm and gauge interest in topics for the Forum at the upcoming OpenInfra Summit in Shanghai. The point of the Forum sesssions is to get feedback from operators and users on things that need fixing, improvements and enhancements, and more generally about the strategic direction for Manila. So please take a look and update this etherpad with topic ideas and indicate your interest in topics already present if you have an interest in Manila. It doesn't matter whether you contribute to the project or not, or whether you will yourself be attending the Forum: https://etherpad.openstack.org/p/manila-shanghai-forum-brainstorming We will review this etherpad in our community meeting at 1500 UTC on 19 September in #openstack-meeting-alt on Freenode, one day before the Forum proposal deadline. Please feel free to join that meeting to discuss, and in any case please add to the brainstorming deadline before then. Cheers, -- Tom Barron

1 0