October 2019 - openstack-discuss - lists.openstack.org

[monasca] Ussuri planning meeting summary
by Witek Bedyk 17 Oct '19

17 Oct '19

Hello everyone, yesterday we held a planning meeting for the next release cycle (Ussuri). You can find the summary of the meeting in the etherpad [1]. Please discuss with your teams and vote for the proposed topics in this spreadsheet [2] until next Wednesday. We meet next week again to agree on priorities during the regular Team Meeting hour, Wednesday, 15 UTC [3]. Thanks Witek [1] https://etherpad.openstack.org/p/monasca-planning-ussuri [2] https://docs.google.com/spreadsheets/d/17PLO8PJr28jXuPaFJ7efZCwgdgMpinZyDRT… [3] https://global.gotomeeting.com/join/817873693

1 0

[nova][gate] Hold off on rechecks until https://review.opendev.org/689152/ merges
by Matt Riedemann 17 Oct '19

17 Oct '19

The gate is blocked for nova changes until [1] is merged so hold off on rechecking anything until then. [1] https://review.opendev.org/#/c/689152/ -- Thanks, Matt

1 0

[openstacksdk][RelMgmt] release-post job for openstack/releases for ref refs/heads/master failed
by Sean McGinnis 17 Oct '19

17 Oct '19

----- Forwarded message from zuul(a)openstack.org ----- Date: Thu, 17 Oct 2019 10:19:24 +0000 From: zuul(a)openstack.org To: release-job-failures(a)lists.openstack.org Subject: [Release-job-failures] release-post job for openstack/releases for ref refs/heads/master failed Reply-To: openstack-discuss(a)lists.openstack.org Build failed. - tag-releases https://zuul.opendev.org/t/openstack/build/571c0f0e43cb469d8727121b77a3c92b : FAILURE in 3m 14s - publish-tox-docs-static https://zuul.opendev.org/t/openstack/build/None : SKIPPED _______________________________________________ Release-job-failures mailing list Release-job-failures(a)lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/release-job-failures ----- End forwarded message ----- The tagging of the new stable/queens OpenStackSDK release failed. It appears to be due to the .gitreview file in that branch still referring to the python-openstacksdk name. That caused the automation to fail on trying to interact with that repo, but it has since been renamed to openstack/openstacksdk. Since this wasn't able to process the release, even though the patch had merged, I think the fix is actually fairly easy. I've proposed a backport of the repo name changes with: https://review.opendev.org/#/c/689131/ If we were to do another release to pick up that change, I believe it would actually fail validation because it would not find the current 0.11.4 tag and therefore think that two releases were needed instead of just one. So since it was not able to process anything to actually perform the release, I believe the easy path forward is to just revert the release patch of this last release: https://review.opendev.org/#/c/689132/ Then wait for the .gitreview patch to merge, then propose the 0.11.4 release again with the hash updated to include that rename fix. Please let me know if there are any questions or if I've overlooked anything. Thanks, Sean

1 0

CentOS 8 nodes available now
by Ian Wienand 17 Oct '19

17 Oct '19

Hello, I'm happy to say that CentOS 8 images are now live in OpenDev infra. Using a node label of "centos-8" will get you started. --- The python environment setup on these images is different to our other images. Firstly, some background: currently during image build we go to some effort to: a) install the latest pip/virtualenv/setuptools b) ensure standard behaviour: /usr/bin/python -> python2 /usr/bin/pip -> python2 install /usr/bin/pip3 -> python3 install /usr/bin/virtualenv -> creates python2 environment by default; python3 virtualenv package in sync This means a number of things; hosts always have python2, and because we overwrite the system pip/virtualenv/setuptools we put these packages on "hold" (depending on the package manager) so jobs don't re-install them and create (even more of) a mess. This made sense in the past, when we had versions of pip/setuptools in distributions that couldn't understand syntax in requirements files (and other bugs) and didn't have the current fantastic job inheritance and modularity that Zuul (v3) provides. However, it also introduces a range of problems for various users, and has a high maintenance overhead. Thus these new images are, by default, python3 only, and have upstream pip and virtualenv packages installed. You will have a default situation: /usr/bin/python -> not provided /usr/bin/pip -> not provided, use /usr/bin/pip3 or "python3 -m pip" /usr/bin/virtualenv -> create python3 environment; provided by upstream python3-virtualenv package /usr/bin/pyvenv -> not provided (is provided by Ubuntu python3-venv), use /usr/bin/pyvenv-3 or "python3 -m venv". Ergo, the "standard behaviour" is not so standard any more. I would suggest if you wish to write somewhat portable jobs/roles etc., you do the following: * in general don't rely on "unversioned" calls of tools at all (python/pip/virtualenv) -- they can all mean different things on different platforms. * scripts should always be #!/usr/bin/python3 * use "python3 -m venv" for virtual environments (if you really need "virtualenv" because of one of the features it provides, use "-m virtualenv") * use "python3 -m pip" to install global pip packages; but try not too -- mixing packages and pip installs never works that well. * if you need python2 for some reason, use a bindep file+role to install it (don't assume it is there) --- For any Zuul admins, note that to use python3-only images similar to what we make, you'll need to set "python-path" to python3 in nodepool so that Ansible calls the correct remote binary. Keep an eye on [1] which will automate this for Ansible >=2.8 after things are merged and released. --- Most of the job setup has been tested (network configs, setting mirrors, adding swap etc.) but there's always a chance of issues with a new platform. Please bring up any issues in #openstack-infra and we'll be sure to get them fixed. --- If you're interested in the images, they are exported at https://nb01.openstack.org/images/ although they are rather large, because we pre-cache a lot. If you'd like to build your own, [2] might help with: DISTRO=centos-minimal DIB_RELEASE=8 --- Thanks, -i [1] https://review.opendev.org/#/c/682797/ [2] https://opendev.org/openstack/project-config/src/branch/master/tools/build-…

5 5

[placement][ptg] Ussuri Placement Topics
by Tetsuro Nakamura 17 Oct '19

17 Oct '19

Hi Placementers, We won't have a specific meeting space for Shanghai PTG, but I'd like to have retrospective on Stein, and gather and note work items we have. Please put your ideas on the etherpad [1]. [1] https://etherpad.openstack.org/p/placement-shanghai-ptg -- Tetsuro Nakamura <tetsuro.nakamura.bc(a)hco.ntt.co.jp> NTT Network Service Systems Laboratories TEL:0422 59 6914(National)/+81 422 59 6914(International) 3-9-11, Midori-Cho Musashino-Shi, Tokyo 180-8585 Japan

1 0

[Octavia] Amphora build issues
by Eric K. Miller 17 Oct '19

17 Oct '19

Hi, It seems that every build I have attempted of an amphora fails in some way. I have tried CentOS 7, Ubuntu Bionic, Xenial, and Trusty. Note that we are running Stein. I will concentrate on Ubuntu issues for now. I first create a fresh VM that is used to install the diskimage-create tool, then run (after sudo'ing to root): apt update apt -y upgrade apt-get -y install qemu qemu-system-common uuid-runtime curl kpartx git jq python-pip debootstrap libguestfs-tools pip install 'networkx==2.2' pip install argparse Babel dib-utils PyYAML git clone -b stable/stein https://github.com/openstack/octavia.git git clone https://git.openstack.org/openstack/diskimage-builder.git cd diskimage-builder pip install -r requirements.txt cd ../octavia/diskimage-create/ pip install -r requirements.txt # And finally, I run the diskimage-create script, specifying the image's OS, so ONE of these, depending on the OS: ./diskimage-create.sh -d bionic # or to use Xenial: ./diskimage-create.sh -d xenial # Note that when selecting Trusty, diskimage-create.sh error's, and so never finishes successfully. # Somewhat expected since it is quite old and unsupported. ./diskimage-create.sh -d trusty The amphorae launch when creating a load balancer, but the amphora agent fails to start, and thus is not responsive on TCP Port 9443. The log from inside the amphora is below. Has anyone successfully created an image? Am I missing something? Thanks! Eric Amphora agent fails to start inside amphora - this is logged when running the agent from the command line: 2019-10-16 03:41:04.835 1119 INFO octavia.common.config [-] /usr/local/bin/amphora-agent version 5.1.0.dev20 2019-10-16 03:41:04.835 1119 DEBUG octavia.common.config [-] command line: /usr/local/bin/amphora-agent --config-file /etc/octavia/amphora-agent.conf setup_logging /opt/amphora-agent-venv/lib/python3.5/site-packages/octavia/common/confi g.py:779 2019-10-16 03:41:05.036 1124 INFO octavia.amphorae.backends.health_daemon.health_daemon [-] Health Manager Sender starting. 2019-10-16 03:41:05.084 1119 CRITICAL octavia [-] Unhandled error: FileNotFoundError: [Errno 2] No such file or directory 2019-10-16 03:41:05.084 1119 ERROR octavia Traceback (most recent call last): 2019-10-16 03:41:05.084 1119 ERROR octavia File "/usr/local/bin/amphora-agent", line 8, in <module> 2019-10-16 03:41:05.084 1119 ERROR octavia sys.exit(main()) 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/octavia/cmd/agent.p y", line 89, in main 2019-10-16 03:41:05.084 1119 ERROR octavia AmphoraAgent(server_instance.app, options).run() 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/gunicorn/app/base.p y", line 72, in run 2019-10-16 03:41:05.084 1119 ERROR octavia Arbiter(self).run() 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/gunicorn/arbiter.py ", line 60, in __init__ 2019-10-16 03:41:05.084 1119 ERROR octavia self.setup(app) 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/gunicorn/arbiter.py ", line 95, in setup 2019-10-16 03:41:05.084 1119 ERROR octavia self.log = self.cfg.logger_class(app.cfg) 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/gunicorn/glogging.p y", line 200, in __init__ 2019-10-16 03:41:05.084 1119 ERROR octavia self.setup(cfg) 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/gunicorn/glogging.p y", line 227, in setup 2019-10-16 03:41:05.084 1119 ERROR octavia self.error_log, cfg, self.syslog_fmt, "error" 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/gunicorn/glogging.p y", line 449, in _set_syslog_handler 2019-10-16 03:41:05.084 1119 ERROR octavia facility=facility, socktype=socktype) 2019-10-16 03:41:05.084 1119 ERROR octavia File "/usr/lib/python3.5/logging/handlers.py", line 806, in __init__ 2019-10-16 03:41:05.084 1119 ERROR octavia self._connect_unixsocket(address) 2019-10-16 03:41:05.084 1119 ERROR octavia File "/usr/lib/python3.5/logging/handlers.py", line 823, in _connect_unixsocket 2019-10-16 03:41:05.084 1119 ERROR octavia self.socket.connect(address) 2019-10-16 03:41:05.084 1119 ERROR octavia FileNotFoundError: [Errno 2] No such file or directory

3 11

OpenStack Train is officially released!
by Sean McGinnis 17 Oct '19

17 Oct '19

The official OpenStack Train release announcement has been sent out: http://lists.openstack.org/pipermail/openstack-announce/2019-October/002024… Thanks to all who were part of making the Train series a success! This marks the official opening of the releases repo for Ussuri, and freezes are now lifted. Train is now a full stable branch. Thanks! Sean

4 3

[PTL] PTG Team Photos
by Kendall Nelson 16 Oct '19

16 Oct '19

Hello Everyone! We are excited to see you in a few weeks at the PTG and wanted to share that we will be taking team photos again! Here is an ethercalc signup for the available time slots [1]. We will be providing time on Thursday Morning/Afternoon and Friday morning to come as a team to get your photo taken. Slots are only ten minutes so its *important that everyone be on time*! The location is TBD at this point, but it will likely be in the prefunction space near registration. Thanks, -Kendall Nelson (diablo_rojo) [1] https://ethercalc.openstack.org/lnupu1sx6ljl

1 1

[nova][ptg] Ussuri scope containment
by Eric Fried 16 Oct '19

16 Oct '19

Nova developers and maintainers- Every cycle we approve some number of blueprints and then complete a low percentage [1] of them. Which blueprints go unfinished seems to be completely random (notably, it appears to have nothing to do with our declared cycle priorities). This is especially frustrating for consumers of a feature, who (understandably) interpret blueprint/spec approval as a signal that they can reasonably expect the feature to land [2]. The cause for non-completion usually seems to fall into one of several broad categories: == Inadequate *developer* attention == - There's not much to be done about the subset of these where the contributor actually walks away. - The real problem is where the developer thinks they're ready for reviewers to look, but reviewers don't. Even things that seem obvious to experienced reviewers, like failing CI or "WIP" in the commit title, will cause patches to be completely ignored -- but unseasoned contributors don't necessarily understand even that, let alone more subtle issues. Consequently, patches will languish, with each side expecting the other to take the next action. This is a problem of culture: contributors don't understand nova reviewer procedures and psychology. == Inadequate *reviewer* attention == - Upstream maintainer time is limited. - We always seem to have low review activity until the last two or three weeks before feature freeze, when there's a frantic uptick and lots gets done. - But there's a cultural rift here as well. Getting maintainers to care about a blueprint is hard if they don't already have a stake in it. The "squeaky wheel" concept is not well understood by unseasoned contributors. The best way to get reviews is to lurk in IRC and beg. Aside from not being intuitive, this can also be difficult logistically (time zone pain, knowing which nicks to ping and how) as well as interpersonally (how much begging is enough? too much? when is it appropriate?). == Multi-release efforts that we knew were going to be multi-release == These may often drag on far longer than they perhaps should, but I'm not going to try to address that here. ======== There's nothing new or surprising about the above. We've tried to address these issues in various ways in the past, with varying degrees of effectiveness. I'd like to try a couple more. (A) Constrain scope, drastically. We marked 25 blueprints complete in Train [3]. Since there has been no change to the core team, let's limit Ussuri to 25 blueprints [4]. If this turns out to be too few, what's the worst thing that happens? We finish everything, early, and wish we had done more. If that happens, drinks are on me, and we can bump the number for V. (B) Require a core to commit to "caring about" a spec before we approve it. The point of this "core liaison" is to act as a mentor to mitigate the cultural issues noted above [5], and to be a first point of contact for reviews. I've proposed this to the spec template here [6]. Thoughts? efried [1] Like in the neighborhood of 60%. This is anecdotal; I'm not aware of a good way to go back and mine actual data. [2] Stuff happens, sure, and nobody expects 100%, but 60%? Come on, we have to be able to do better than that. [3] https://blueprints.launchpad.net/nova/train [4] Recognizing of course that not all blueprints are created equal, this is more an attempt at a reasonable heuristic than an actual expectation of total size/LOC/person-hours/etc. The theory being that constraining to an actual number, whatever the number may be, is better than not constraining at all. [5] If you're a core, you can be your own liaison, because presumably you don't need further cultural indoctrination or help begging for reviews. [6] https://review.opendev.org/685857

11 21

[oslo] On PTO rest of the week
by Ben Nemec 16 Oct '19

16 Oct '19

Hey, I'm not working for the rest of the week. \o/ I also realized that I haven't announced that here. /o\ Things are pretty quiet in Oslo right now so I don't anticipate that I'll be needed in the interim, but I should have internet the whole time I'm out and I'll _probably_ be checking email. If something comes up that can't wait until Monday just holler and hopefully I'll see it. Thanks. -Ben

1 0