April 2019 - openstack-discuss - lists.openstack.org

Announcing Gertty 1.6.0
by corvus＠inaugust.com 24 Apr '19

24 Apr '19

Announcing Gertty 1.6.0 ======================= Gertty is a console-based interface to the Gerrit Code Review system. Gertty is designed to support a workflow similar to reading network news or mail. It syncs information from Gerrit to local storage to support disconnected operation and easy manipulation of local git repos. It is fast and efficient at dealing with large numbers of changes and projects. The full README may be found here: https://opendev.org/ttygroup/gertty/src/branch/master/README.rst Changes since 1.5.0: ==================== * The default config file location is now ~/.config/gertty/gertty.yaml (but Gertty will fall back on the previous default of ~/.gertty.yaml if it is not found) * The source code is now hosted in OpenDev (https://opendev.org) * Gertty supports (and should be run with) python3 * Added tab display support * Reviewkeys are enabled in diffs * Added a new option (off by default) to close the change after reviewing * Added inline comments to the change overview screen * The size column graph is more configurable * Reviewkeys now support sending a message * Added more example configurations As well as several bug fixes and stability improvements. Thanks to the following people whose changes are included in this release: Clint Byrum Dominique Martinet Doug Wiegley Emilien Macchi Ian Wienand John L. Villalovos Logan V Major Hayden Masayuki Igawa Matthias Runge Natal Ngétal Nate Johnston Nguyen Hung Phuong Robert Collins Tobias Henkel Tristan Cacqueray

1 0

[First Contact] [SIG] [PTL] Project Liaisons Update
by Matthew Oliver 24 Apr '19

24 Apr '19

Hey PTLs, Now that the new PTL election season is over and you all should be coming to grips with your life choices. It's time to update the First contact project liaison list[0]. What are first contact liaisons do I hear you ask? We at the first contact SIG strive to be a point of call for new contributors. When they do contact or we find a new contributors first patch, we try and connect them with the project. This could be as simple a adding the liaison to the review or providing and introduction. Anyone can be a liaison for a project, so you're welcome to delegate, however we do default to PTLs. Further given that we live on a globe, tracking and having multiple liaisons on each project from different timezones would also be a big win. For those of you who have already delegated previously and/or are continuing to PTL, you don't get off too easily. The world has finished it's summertime/daylight saving dance, so is time to update your timezone's too. I know the summit/PTG is just around the corner, so while your updating your etherpad and wiki's what's one more ;) Regards, Matt [0] https://wiki.openstack.org/wiki/First_Contact_SIG#Project_Liaisons

1 0

The python black project.
by Natal Ngétal 24 Apr '19

24 Apr '19

Hi everyone, Black is project to format Python code. It's more and more used by the python community. That can save time and harmonize the code. For example, simple quote versus double quote, that was not checked by pep8.i It's just an example of what make black With a pre-hook commit hook this will be also avoid to lose time with pep8 errors. Two links to see more: https://github.com/ambv/black https://www.mattlayman.com/blog/2018/python-code-black/ I think it can be very interesting to start to use Black on OpenStack. What do you thinks about that? For example, we can introduce it on some projects, I would volunteer to try on TripleO. My best regards

21 46

[openstack-dev] [neutron] Cancelling the L3 meeting on April 24
by Miguel Lavalle 24 Apr '19

24 Apr '19

Hi Neutrinos, Since we are meeting next week in Denver to discuss, among other things, L3 topics, we are cancelling this week's meeting. Safe travels to everybody! Miguel

1 0

[nova][ptg] Draft Agenda
by Eric Fried 24 Apr '19

24 Apr '19

Nova PTGers- I have drafted an agenda at the bottom of the main etherpad [1]. It is subject to change for the next week for various reasons, but I expect mainly a) for scheduling conflicts and the like (see below) and b) things falling off because resolved (e.g. I expect the SEV spec to merge in the next couple of days). Please review the agenda, **especially if your schedule isn't "I'm present all three days, and glued to the nova team"** and contact me [2] if you: - have a scheduling conflict for a topic you own **or** a topic you care about and want to be present for. - feel I've prioritized your topic unjustly. - feel I've allocated too little or too much time [3] to your topic. - think I've committed some other glaring error or omission. Because we're short on time, I'll be strict about sticking to the schedule in whatever form it solidifies into. I've allocated some time at the end [4] to resume discussions that have to be chopped off early as a result. Thanks, efried P.S. Please be gentle with me. I've tried to be fair, accommodate scheduling issues I know about, and allocate reasonable chunks of time for topics even when I barely understand them. If I've made a mistake, assume it's out of ignorance, not malice or prejudice. [1] https://etherpad.openstack.org/p/nova-ptg-train [2] via email or IRC. Please do not edit the agenda on the etherpad. [3] seriously, including "we don't actually need to talk about this at the PTG". [4] I know it sucks that it's on Saturday afternoon; my working theory is that that's better than some topics not getting any floor time at all.

1 0

Openstack LXC
by Alejandro Ruiz Bermejo 24 Apr '19

24 Apr '19

Hi everyone, I have a fresh Openstack installation and i need to use LXC to launch instances I made the deployment manually, currently i have one controller node, one compute node, one block storage node and one object storage node. I already made the necessary configurations in the nova.conf file as appears on the official documentation site compute_driver = libvirt.LibvirtDriver [libvirt]virt_type = lxc and i have installed the nova-compute-lxc package and the nbd-server package. i created a new ubuntu 18 image for the container with the following command: openstack image create --public --container-format=bare --disk-format=raw --file bionic-server-cloudimg-amd64.tar.gz ubuntu18 and the flavor: openstack flavor create --ram 2048 --vcpus 1 --disk 5 lxc.test then when i try to launch an instance: openstack server create --flavor lxc.test --image ubuntu18 --nic net-id=d0f6cc99-565b-484f-a5ab-fcb1c3af456d --security-group 1160755e-1312-43c6-8ebb-b12b92d35e51 --key-name mykey lxc6 I get the following Error cat /var/log/nova/nova-*.log | grep 269d2480-4477-4382-b09c-9c686ab7a17c 2019-04-23 17:59:26.846 3949 INFO nova.osapi_compute.wsgi.server [req-1dbeb432-67f0-4305-b91c-6548dde49bee a16c6ef0319b4643a4ec8e56a1d025cb 59065d8f970b467aa94ef7b35f1edab5 - default default] 10.8.9.54 "GET /v2.1/servers/269d2480-4477-4382-b09c-9c686ab7a17c HTTP/1.1" status: 200 len: 1737 time: 0.6656921 2019-04-23 17:59:33.076 3944 INFO nova.api.openstack.compute.server_external_events [req-206e8ace-90e3-4a95-8274-4a9091c174c0 c2657fa5083f4d5c932d53d6bf094604 18337c70d397424f8895cd4f1ae7450e - default default] Creating event network-changed:818d6adc-252e-4d47-9275-009cff55c716 for instance 269d2480-4477-4382-b09c-9c686ab7a17c on compute 2019-04-23 17:59:37.663 3926 *ERROR* nova.scheduler.utils [req-abd32ef2-3a19-4154-a7da-893f3d2b2aef a16c6ef0319b4643a4ec8e56a1d025cb 59065d8f970b467aa94ef7b35f1edab5 - default default] [instance: 269d2480-4477-4382-b09c-9c686ab7a17c] Error from last host: compute (node compute.cujae.edu.cu): [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1841, in _do_build_and_run_instance\n filter_properties, request_spec)\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2121, in _build_and_run_instance\n instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance 269d2480-4477-4382-b09c-9c686ab7a17c was re-scheduled: mount() takes exactly 4 arguments (3 given)\n'] Is this error a bug or something fixable?? Has anyone accomplished this kind of environment or used lxc without problems?

3 4

[keystone] No meeting next two weeks
by Colleen Murphy 24 Apr '19

24 Apr '19

As discussed in today's meeting[1], we'll be cancelling next week's meeting (30 April) due to the summit, as well as the following week's meeting (7 May) for recovery. Colleen [1] http://eavesdrop.openstack.org/meetings/keystone/2019/keystone.2019-04-23-1…

1 0

[Security][Security SIG] Weekly Meeting Schedule
by Gage Hugo 24 Apr '19

24 Apr '19

With the Summit/PTG next week, the Security SIG meeting for May 2nd will be cancelled due to people attending the Summit/PTG. The following week I will not be available to chair, so the meeting for May 9th will be preemptively canceled, however if there is a need for discussion, please feel free to host at the usual time. We will be back to regularly scheduled meetings again on the 16th, Safe travels everyone attending the Summit/PTG!

1 0

[Security][Security SIG] Weekly Meeting Schedule
by Gage Hugo 24 Apr '19

24 Apr '19

With the Summit/PTG next week, the Security SIG meeting for May 2nd will be cancelled due to people attending the Summit/PTG. The following week I will not be available to chair, so the meeting for May 9th will be preemptively canceled, however if there is a need for discussion, please feel free to host at the usual time. We will be back to regularly scheduled meetings again on the 16th, Safe travels everyone attending the Summit/PTG! - Gage

1 0

[dev] oslosphinx Sphinx theme is EOL
by Stephen Finucane 24 Apr '19

24 Apr '19

Just a quick pointer that the oslosphinx theme does not work with Sphinx 2.0, which was released yesterday, failing with the following error: Exception occurred: File "/home/zuul/src/git.openstack.org/openstack/nova-specs/.tox/docs/lib/python3.6/site-packages/oslosphinx/__init__.py", line 76, in builder_inited app.info('Using openstack theme from %s' % theme_dir) AttributeError: 'Sphinx' object has no attribute 'info' The oslo team have decided not to fix these issues, given that oslosphinx is EOL since Pike, and we suggest that people instead migrate to openstackdocstheme [1] (for projects that are able to use the OpenStack brand [2]) or something else (for projects that are not). For the former, I suggest follow the openstackdocstheme [1] or look at what we've done in nova-specs [3] for an existing example. If you have issues, please reach out on #openstack-doc and one of us will try our best to help you migrate. I'll work with infra to get the repo shuttered and avoid any further confusion with this theme going forward. Stephen [1] https://docs.openstack.org/openstackdocstheme/latest/ [2] https://www.openstack.org/brand [3] https://review.openstack.org/#/c/648670/

1 1

[FYI][all] PySnooper, a poor man's debugger
by Artom Lifshitz 24 Apr '19

24 Apr '19

tl;dr Instead of debugging with LOG.debug and/or print() statements all over the place, use the pysnooper decorator [1]. I came across this on Hacker News this morning - if you're like me and are too lazy to invest in learning a real debugger, and instead do it with LOG.debug/print()s all over the place, pysnooper might be useful to you. This line from the README feels particularly relevant to me ;) > What makes PySnooper stand out from all other code intelligence tools? > You can use it in your sh***y, sprawling enterprise codebase without having > to do any setup. I tried it out with a random Nova unit test by decorating the function being tested, and it gave me the expected analysis of how the function executed. Just sharing this, I hope it might be useful to someone :) [1] https://github.com/cool-RR/pysnooper

5 4

[api-sig][neutron] Question on oslo policy assertion when supplied attribute equals the default
by Nate Johnston 24 Apr '19

24 Apr '19

API SIG and Neutron communities, TL;DR ----------------------- For API requests that specify boolean attributes to the request, we check "are you allowed to specify this attribute" before we check "would specifying this argument make any difference". I am proposing to change that so users can specify things forbidden by policy as long as the attribute specified equals the default and therefore the result would be the same as if it had not been specified. Are there any security problems with that? Long-winded explanation ----------------------- I would like your opinion on the potential security impact of a change I am proposing to the Neutron mechanism for assertion of oslo.policy. In response to an operator concern I proposed a change [1] that would exempt oslo.policy checks under one specific condition: if the request specifies a boolean attribute that matches a configured default. The way Neutron is currently coded, API requests have the oslo.policy check - "are you allowed to ask for X" before anything else about the request is processed. For those not familiar with it, oslo.polcy additionally provides the ability to control arguments within that request, so that you can permit only some users to use those arguments. For example, usually permissions to create a network are pretty broad but you must me admin to create a network and specify if it will be shared. The feedback that I got was that this was frustrating for users and some operators. The distinction that we make as programmers - asserting the policy check before the processing of the request - is counterintuitive from a user perspective. The preferred behavior is to say "if supplying the argument results in the same end condition as not supplying any argument (because the argument supplied is the default anyway) then allow it". Proposed change ----------------------- The proposed change I submitted to neutron is to bypass oslo.policy checks for method arguments when all boolean method arguments equal the configured default. If any boolean arguments differ from the default, or if there are any other arguments specified that are controlled by policy, then execute the oslo.policy checks as normal. Here are a few examples: - Regular project user creates a network specifying shared=false. - Current behavior: regular user cannot specify the shared attribute, denied. - Proposed behavior: "shared=false" is the default when creating networks, so allow it. - Regular project user creates a router specifying that it be non-HA. - Current behavior: regular user cannot specify the "ha" attribute, denied. - Proposed behavior: "ha=false" is the default when creating routers, so allow it. In neutron the oslo.policy check is done in the API base [2] almost immediately upon receipt of an API request, not as a decorator in the keystone style. So comparison of supplied boolean attributes against defaults could be done at that point centrally. The Concern ----------------------- The concern raised - and what I hope to get the API SIG's guidance on - is whether this raises any kind of security concern. I don't think it will. Since we are comparing equivalence of the user supplied attribute to the default, the end result of a request permitted by this proposal should always be the same as a request where the user did not supply that argument. But your wisdom would be appreciated in validating that assumption. Thanks for your feedback, Nate [1] https://bugs.launchpad.net/neutron/+bug/1821208 [2] https://opendev.org/openstack/neutron/src/branch/master/neutron/api/v2/base…

6 9

[nova] Running multinode resize tests in the gate
by Artom Lifshitz 24 Apr '19

24 Apr '19

Yo Novans, In trying to understand why my partial fix to bug 1813789 [1] - specifically the part where reverting a resize fails because we get the network-vif-plugged event from Neutron before we start listening for it - I noticed that we don't test multinode resize (and revert) in the gate. As far as I can tell, the only place where the revert tests are run are in tempest-full-py3, and that's on an allinone single node. In order to test a theory of mine about revert resize on the same node vs source to dest and back again, I do have a DNM patch up [2] to run the resize tests in the nova-live-migration job - I think that's the only multinode tempest job we have, right? I think that job also runs evacuation tests [3], correct? So, should we just rename it nova-multinode and add resize to the list of tests it runs? [1] https://bugs.launchpad.net/nova/+bug/1813789 [2] https://review.openstack.org/#/c/653498/ [3] https://github.com/openstack/nova/blob/54d8b40471e6f7b36dbe45966c1c605f94d4…

2 2

[nova][cinder] An API to replace volume-update (aka swap-volume)
by Matthew Booth 23 Apr '19

23 Apr '19

Consider this a straw-man proposal for the upcoming cross-project discussion at PTG. I'm not (currently, at any rate) proposing to do this work myself, which is also why I haven't put this in a spec. Volume-update is a problematic interface. There are likely many more than this, but problems which come to mind are: * It shares state between cinder and nova while running. A failure of either leaves an inconsistent state which is not easily recoverable. * It is fragile: it requires an instance is running. Stopping an instance while it is in progress is undefined at best. It will fail if an instance is not running. * It is mis-used by end-users to copy data between different volumes, which results in an unsupportable strange instance state. * It is slow. * It results in data corruption for multiattached volumes I'd like to propose the following api as a replacement. I'll describe the contract up top and put some example implementations below. This new flow for volume migration would be driven entirely by cinder, with no callbacks from nova, so all of the following are new nova apis. * volume-migration-start(src_attachment_id) After returning, nova will not write any data to src until completion or cancelled. Nova will return an error if it is not able to suspend writes to src. Start is a synchronous operation. Nova will not return until it can guarantee that there will be no further writes to src. * volume-migration-cancel(src_attachment_id) Nova will resume writing to src. A subsequent complete call will return an error. cancel is an asynchronous operation. Nova assumes it can resume writing to src immediately, but does not guarantee when it will finish the cancel operation. This has no impact on cinder, but nova instance operations may continue to be restricted until the cancel completes. * volume-migration-complete(src_attachment_id, dst_attachment_id) Nova expects that dst is a copy of src at the time that volume-migration-start() was called. Nova will detach from src and attach dst in its place. Nova will resume reading and writing to dst immediately. Nova will not return until it is safe for cinder to remove src, but the complete operation is not guaranteed to have finished at this time. This has no impact on cinder, but nova instance operations may continue to be restricted until it completes asynchronously. The implementation on the cinder side would be: volume-migration-start(src_attachment_id) copy(src, dst) volume-migration-complete(src, dst) Cinder doesn't need to be concerned with whether the instance or running or not, and it always does the copy itself. Using driver-specific optimisations this has the potential to be very fast, but even the fallback implementation of a full copy between 2 arrays should be substantially faster than a qemu block job on the compute host. It is entirely up to cinder to ensure this operation is safe wrt multiattach. I expect 'safe' here to mean it will refuse to do the operation. There may also be details to be worked out wrt who creates the dst attachment. If at all possible, I don't want to expose to nova the quirk that under cinder's covers a new volume is created which temporarily has the wrong volume id. I'd prefer that, from nova's POV, the operation is just a managed switch between 2 attachments to apparently the same volume. 2 possible implementations on the Nova side: 1. Hypervisor does not support live storage migration Summary: instance must be shutdown. volume-migration-start() returns success iff the instance is shutdown. Nova will set a task state such that instance cannot be started until cancel or complete. volume-migration-complete() reconfigures the specific volume attachment and unsets the task state so the instance can be started. 2. libvirt driver Summary: libvirt allows writes to continue during the copy by sending them to a local snapshot, which it commits to dst on complete(). volume-migration-start() sets task state such that instance cannot be started or stopped until cancel or completion. If instance is not running, does nothing else. If instance is running, creates local writeable qcow2 backed by read-only src volume. Swaps volume to local qcow2. volume-migration-cancel() does nothing if instance is not running except unset task state. If instance is running returns success to cinder immediately before commiting qcow2 to src volume and then unsetting task state. volume-migration-complete() updates instance to dst if it's not running, unsets task state and returns immediately. If instance is running, rebases local qcow2 from src to dst (cinder guarantees us that these are identical), then returns success to cinder because src can now be released. Nova starts a job to commit qcow2 to dst. On completion it updates instance to use dst and unsets task state. This API does not handle the case where the hypervisor is only capable of live storage migration where it manages the entire copy itself. If we had a requirement to support this we might change volume-migration-start() to return something indicating that nova will do the copy, or perhaps indicating that cinder should try again with a different entry point. However, I believe this approach will always be slower than a cinder-native copy, and I can't see any way around it requiring a callback to cinder on completion of the copy. I'd very much like some feedback from non-libvirt hypervisor folks around whether they're interested in this functionality at all, and if so what their capabilities are. Matt -- Matthew Booth Red Hat OpenStack Engineer, Compute DFG Phone: +442070094448 (UK)

2 3

Scuba Diving at the Denver Summit
by Monty Taylor 23 Apr '19

23 Apr '19

Hey everybody! tl;dr - Let me know if you'd like to go diving in the Rocky Mountains. Of all of the places we've had a Summit where one might reasonably expect there to be a possibility of Scuba Diving, the Denver Summit is probably REALLY low on the list. But it turns out that there is a Dive Shop that does dives at the Denver Aquarium [0] (with all the fish and sharks and whatnot) The Denver Aquarium is also very close to the Convention Center. I'm going to see about putting together a private group dive event for those of us who think that sounds like a fun idea. Due to cross-contamination concerns for the health and safety of the fish, the dive shop provides all equipment - so you don't have to bring gear. They will allow bringing your own mask - and I'm assuming if a well-scrubbed mask is ok then a well-scrubbed dive computer would be ok - but I'll be checking with them when I call to arrange things. If you are Open Water certified and would like to join, please send me a response back (reply directly to me rather than the list is fine) so I can judge general interest level when talking to the lovely humans. If you are **NOT** Open Water certified but are thinking to yourself "wow, that sounds like fun" - the dive shop does certification dives at the aquarium on the weekends. That's a bit out of scope for me to arrange, but if you want to show up in Denver early and do an Open Water class then join us, that's awesome. You can also get certified at home before coming ... but you'll need an Open Water certification to join us. BUT WAIT - THERE'S MORE! There is another interesting diving opportunity in the general Mountain West area - although not immediately **close** to Denver - that Sandy and I are going to be going to before the Summit - Homestead Crater. We're arranging to meet a Scuba instructor there to do our second Altitude cert dive, and also to do a 2-dive "Hot Springs Diver" specialty. We'd be happy to have folks join us. Homestead Crater [1] is outside of Salt Lake City. It's a hot spring - the water is 96F/35.5C and it's also an altitude dive. If people are interested, we can meet at the crater on Thursday morning pre-summit, dive the crater, then drive to Denver on Friday/Saturday. ** If only a small number of people are interested in any of this - we'll likely just do the Denver dives the weekend before during one of the normally scheduled dive times instead of arranging a group outing, and shift Homestead, if anyone is interested, a few days earlier ** I will not have space enough in my car to get you from the crater to Denver, so if you want to fly in to SLC ahead of time and meet there, awesome. Unlike Denver, the crater will require you have equipment, including scuba tanks. For anyone interested in coming, there are dive shops in Salt Lake where you can rent gear. There is a dive shop at the crater to do air fills. Anyway - if anyone is interested in some or all of these things, let me know this week so I can get a headcount and then we can talk more off-list about logistics and figure out how to deal with things for people coming from far off (carpools, equipment, etc) Monty PS. If any of you aren't already using Subsurface for dive logging, I'll be happy to help you get set up. [0] https://divedowntown.com/ [1] https://homesteadresort.com/utah-resort-things-to-do/homestead-crater/

4 5

[magnum] kubernests cinder deploying wordpress
by Ionut Biru 23 Apr '19

23 Apr '19

Hello, I'm trying to deploy a simple wordpress installation following the documentation from kubernets. https://kubernetes.io/docs/tutorials/stateful-application/mysql-wordpress-p… kubernets was deployed using magnum on openstack rocky using the following template: https://paste.xinu.at/sRk/ Then i'm using the cinder storage class that is described in kubernets documentation: https://kubernetes.io/docs/concepts/storage/storage-classes/#openstack-cind… and I also made it default. I did the following test: cinder volume created before hand and used in pod directly with the id, the volume is attached and mounted correctly like is described here: https://docs.openstack.org/magnum/latest/user/#storage Now if I try to deploy wordpress using the first link or other helms that use volume claim, the volume is created automatically by kubernets but they are never attached or mounted to the instances. On master I see the following error: pv_protection_controller.go:116] PV pvc-48fed935-5b61-11e9-9901-fa163e5ebeae failed with : Operation cannot be fulfilled on persistentvolumes "pvc-48fed935-5b61-11e9-9901-fa163e5ebea> the object has been modified; please apply your changes to the latest version and try again Is there something I miss? I'm using the wrong version of fedora-atomic? It's the latest version available for download. I did find that someone else had this error but not with cinder: https://github.com/kubernetes/kubernetes/issues/72905

1 1

[octavia][openstack-ansible] network layout example
by Ionut Biru 23 Apr '19

23 Apr '19

Hello guys, I have hard time understanding how to create the right layout infrastructure of octavia. Does anyone have octavia running in production with OSA? For now I have created a new tagged interface. Is this the right approach? auto bond0.40 iface bond0.40 inet manual vlan-raw-device bond0 auto br-lbaas iface br-lbaas inet manual bridge_stp off bridge_waitport 0 bridge_fd 0 pre-up ip link add br-lbaas-veth type veth peer name eth14 || true pre-up ip link set br-lbaas-veth up pre-up ip link set eth14 up post-down ip link del br-lbaas-veth || true bridge_ports bond0.40 br-lbaas-veth

1 0

[Third Party CI][Cinder] Changes from review.openstack.org to review.opedev.org
by Sean McGinnis 23 Apr '19

23 Apr '19

Attention operators of third party CI. It may have been easy to miss some of the changes that have been done last week to migrate from our Gerrit instance from review.openstack.org to review.opendev.org. For a lot of things, this should be seamless as there are pretty robust HHTP(s) redirects in place, making it easy to not even notice right away that things have changed. One thing that may impact third party CI operators though, is there is no way to automatically redirect SSH traffic. http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005423.ht… I think many CI systems SSH in and watch the event stream with something like: >> ssh -p 24198 review.openstack.org gerrit stream-events That will now fail with the move to opendev.org. Please make sure your CI configuration is updated to change that to review.opendev.org to continue to capture the event stream for triggering job execution. Thanks! Sean

3 3

[Glance] Weekly meetings
by Erno Kuvaja 23 Apr '19

23 Apr '19

Hi all, It's the Summit/PTG disruptions again. So This week Glance will be having weekly meeting normally at Thu (25th) 14:00, next week we will be having our meeting face to face in Denver so IRC will be quiet, the week after Summit, I might not be available to chair meeting if one is needed but I'll see if Abhishek or Brian can cover me. From Thu 16th of May you can again expect business as usual from us. Best, Erno

1 0

[Watcher] tomorrows meeting and agenda
by info＠dantalion.nl 23 Apr '19

23 Apr '19

Hello everyone, Tomorrow Watcher will have a meeting again at 08:00 UTC in the #openstack-meeting-alt channel. This is a reminder since we only recently started to have meetings again. The agenda is available on https://wiki.openstack.org/wiki/Watcher_Meeting_Agenda feel free to add any additional items. Kind Regards, Corne Lukken (Dantali0n)

1 0

Re: [Release-job-failures] Release of openstack/solum failed
by Thierry Carrez 23 Apr '19

23 Apr '19

zuul(a)openstack.org wrote: > Build failed. > > - release-openstack-python http://logs.openstack.org/19/19a7574237f44807b16c37e0983223ff57340ba3/relea… : POST_FAILURE in 4m 40s > - announce-release announce-release : SKIPPED > - propose-update-constraints propose-update-constraints : SKIPPED Solum release post-job (triggered by stable/pike 5.4.1 tagging) failed with the following error: Ensure twine is installed: [ "python3", "-m", "pip", "install", "twine!=1.12.0", "readme_renderer[md]!=23.0", "requests-toolbelt!=0.9.0", "--user" ] Can not perform a '--user' install. User site-packages are not visible in this virtualenv. http://logs.openstack.org/19/19a7574237f44807b16c37e0983223ff57340ba3/relea… This seems to be a new error hitting us recently (also happened for networking-arista, which the release management team does not directly handle). It is weird since the job (exact same command) worked in subsequent runs: http://logs.openstack.org/d0/d0d2089455efb6d01a1158bad597f42cf2306c30/relea… -- Thierry Carrez (ttx)

1 1

[release-openstack-python][release pipeline] POST_FAILURE since last couple of hours, already known and being fixed?
by Yatin Karel 23 Apr '19

23 Apr '19

Hi all, release-openstack-python[1] job is failing since last couple of hours, is this already known and worked upon? If it's not being fixed yet good to hold other releases until it's fixed. The job failing at ansible task: ensure-twine : Ensure twine is installed with Error:- Can not perform a '--user' install. User site-packages are not visible in this virtualenv. Example log:- http://logs.openstack.org/c0/c09c5676b549b6705e7286ee6d1d7e000b9e756f/relea… Actually i tried to reach via IRC at #openstack-release but seems release team in other time zone, posting on mailing list so it can be followed up. Noticed the issue when found tarballs missing when building the released projects in RDO [2]. [1] http://zuul.openstack.org/builds?job_name=release-openstack-python [2] https://logs.rdoproject.org/97/20397/1/check/distgit-cbs-validate/7bd5d70/j… Thanks and Regards Yatin Karel

2 1

[Neutron][TripleO] Avoid dangling sidecars containers - add hook support
by Cédric Jeanneret 23 Apr '19

23 Apr '19

Dear List, While doing some validations on an "in-use" node, we detected a container being Exited with status 137. After some researches and discussions, it appears this container is launched from a Neutron container, as a sidecar, and then the service is killed: https://github.com/openstack/neutron/blob/master/neutron/agent/linux/extern… While the "kill" is fine outside of a container, it leads to some issues in a containerized world: - dangling container is here, with a failed state - validating container state is therefore complicated - might lead to false assumption if we don't know that "kill" process - might lead to disk space issues In order to sort that bad situation out, and prevent disk space issues and the like, I propose to discuss about some "hook" addition in that "external_process.py" (or wherever is more suited for that usage) There is apparently something like that for the service launch, since there are wrapper scripts in /var/lib/neutron. In my idea, that wrapper script could generate a new wrapper used in order to actually delete the container (instead of kill -9 <pid>). The neutron code could be modified in a simple way, something like: ############### if os.path.exists(<wrapper-file>): utils.execute(['bash', <wrapper-file>]) else: # current way to handle things ############### The <wrapper-file> might have "something" in its name in order to ensure we're actually killing the right process/container. self.pid, or maybe some specific string that neutron is aware of... you get the idea. Even outside of a container, that might be used in order to clean temporary configurations/files, and ensure we're actually facing a clean environment. It's really something more like "hooks" than "container-centric thingy" Would you consider that kind of new feature? Thank you for your time and consideration! Cheers, C. -- Cédric Jeanneret Software Engineer DFG:DF

2 2

vm missing psi-passthrough device if cpu pinning is in place
by Manuel Sopena Ballesteros 23 Apr '19

23 Apr '19

Dear Openstack community, I am trying to setup a vm with bith pci-passthrough and cpu pinning. This is my flavor: [root@openstack-deployment ~]# openstack flavor show kudu.test.dedicated.tiny +----------------------------+--------------------------------------------------------------+ | Field | Value | +----------------------------+--------------------------------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | None | | disk | 10 | | id | 7e241e94-1c50-4a92-9ce6-8abb272d79d8 | | name | kudu.test.dedicated.tiny | | os-flavor-access:is_public | True | | properties | hw:cpu_policy='dedicated', hw:pci_passthrough:alias='nvme:1' | | ram | 4096 | | rxtx_factor | 1.0 | | swap | | | vcpus | 2 | +----------------------------+--------------------------------------------------------------+ And this is how I create the vm: [root@openstack-deployment ~]# openstack server create --flavor kudu.test.dedicated.tiny --image centos7.5-image --nic net-id=hpc --security-group admin --key-name mykey --availability-zone nova:zeus-53.localdomain kudu-test-3-1 The problem is that the vm does not have the nvme pci device attached to it. [root@zeus-53 test]# docker exec -it nova_libvirt virsh dumpxml instance-000000ad <domain type='kvm' id='9'> <name>instance-000000ad</name> <uuid>5d8f6cc1-e716-4f4d-ac55-df24d4cffd38</uuid> <metadata> <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0"> <nova:package version="18.2.0-1.el7"/> <nova:name>kudu-test-3-1</nova:name> <nova:creationTime>2019-04-17 05:17:02</nova:creationTime> <nova:flavor name="kudu.test.dedicated.tiny"> <nova:memory>4096</nova:memory> <nova:disk>10</nova:disk> <nova:swap>0</nova:swap> <nova:ephemeral>0</nova:ephemeral> <nova:vcpus>2</nova:vcpus> </nova:flavor> <nova:owner> <nova:user uuid="91e83343e9834c8ba0172ff369c8acac">admin</nova:user> <nova:project uuid="b91520cff5bd45c59a8de07c38641582">admin</nova:project> </nova:owner> <nova:root type="image" uuid="5ec18792-05f2-4bce-b5d8-cb3480cbb35a"/> </nova:instance> </metadata> <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <vcpu placement='static'>2</vcpu> <cputune> <shares>2048</shares> <vcpupin vcpu='0' cpuset='9'/> <vcpupin vcpu='1' cpuset='37'/> <emulatorpin cpuset='9,37'/> </cputune> <numatune> <memory mode='strict' nodeset='0'/> <memnode cellid='0' mode='strict' nodeset='0'/> </numatune> <resource> <partition>/machine</partition> </resource> <sysinfo type='smbios'> <system> <entry name='manufacturer'>RDO</entry> <entry name='product'>OpenStack Compute</entry> <entry name='version'>18.2.0-1.el7</entry> <entry name='serial'>00000000-0000-0000-0000-0cc47aa482cc</entry> <entry name='uuid'>5d8f6cc1-e716-4f4d-ac55-df24d4cffd38</entry> <entry name='family'>Virtual Machine</entry> </system> </sysinfo> <os> <type arch='x86_64' machine='pc-i440fx-rhel7.6.0'>hvm</type> <boot dev='hd'/> <smbios mode='sysinfo'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Skylake-Client-IBRS</model> <vendor>Intel</vendor> <topology sockets='1' cores='1' threads='2'/> <feature policy='require' name='ss'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='tsc_adjust'/> <feature policy='require' name='stibp'/> <feature policy='require' name='ssbd'/> <feature policy='require' name='pdpe1gb'/> <feature policy='disable' name='mpx'/> <feature policy='disable' name='xsavec'/> <feature policy='disable' name='xgetbv1'/> <numa> <cell id='0' cpus='0-1' memory='4194304' unit='KiB'/> </numa> </cpu> <clock offset='utc'> <timer name='pit' tickpolicy='delay'/> <timer name='rtc' tickpolicy='catchup'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='/var/lib/nova/instances/5d8f6cc1-e716-4f4d-ac55-df24d4cffd38/disk'/> <backingStore type='file' index='1'> <format type='raw'/> <source file='/var/lib/nova/instances/_base/4cc6eebe175e35178cb81853818a1eb103cea937'/> <backingStore/> </backingStore> <target dev='vda' bus='virtio'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <controller type='usb' index='0' model='piix3-uhci'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <interface type='bridge'> <mac address='fa:16:3e:45:ba:a1'/> <source bridge='qbr2a0abb35-29'/> <target dev='tap2a0abb35-29'/> <model type='virtio'/> <mtu size='1500'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/14'/> <log file='/var/lib/nova/instances/5d8f6cc1-e716-4f4d-ac55-df24d4cffd38/console.log' append='off'/> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/14'> <source path='/dev/pts/14'/> <log file='/var/lib/nova/instances/5d8f6cc1-e716-4f4d-ac55-df24d4cffd38/console.log' append='off'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <input type='tablet' bus='usb'> <alias name='input0'/> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'> <alias name='input1'/> </input> <input type='keyboard' bus='ps2'> <alias name='input2'/> </input> <graphics type='vnc' port='5901' autoport='yes' listen='192.168.1.53'> <listen type='address' address='192.168.1.53'/> </graphics> <video> <model type='cirrus' vram='16384' heads='1' primary='yes'/> <alias name='video0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <memballoon model='virtio'> <stats period='10'/> <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </memballoon> </devices> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+42436:+42436</label> <imagelabel>+42436:+42436</imagelabel> </seclabel> </domain> However I do can see the nvme if I use a flavor without cpu pinning. Has anyone seen this before? Thank you NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed.

1 0

[Tripleo][Octavia][OVN] Overcloud deployment Issues with Octavia
by Rabi Mishra 23 Apr '19

23 Apr '19

Hi All, For some Octavia testing with heat, I tried deploying 3+1 node overcloud with Octavia environment and THT master. As expected, deployment finished successfully. However, when trying to do some Octavia API calls (ex. list laodbalancers etc), it was returning (503) Service Unavailable error. Checking the logs of octavia_api container (though the container was running), it was in a continous loop. I have captured some of these issues (as described below) in https://bugs.launchpad.net/tripleo/+bug/1825146 too. The intent of this mail is to bring it to the notice of the respective teams and understand the best possible way to fix these. 1. Incorrect config setting for provider_drivers. As of https://github.com/openstack/puppet-tripleo/commit/97c46ca76803fb238279d245…, we added OVN provider driver support for Octavia. However, the config option seems to be set incorrectly which resulted in octavia-api failing to start. 2. ovn_nb_connection not in octavia config It seems ovn provider driver (part of networking-ovn) connects to ovn nb database as part of initialization. However, we don't seem to configure connection string in octavia.conf and it tries to connect to 127.0.0.1 and fails. I've already proposed a few patches[1] to fix the above two issues. 3. Missing Octavia Driver Agent Octavia had added a driver-agent controller process in Stein (for provider drivers to communicate with the octavia for status and stats update). This process is expected to be collocated with octavia-api and creates a pair of unix domain sockets for communication that the drivers use. We don't seem to have this implemented in THT yet. Is this something planned and someone working on it? AFAICT, we can't enable/use the OVN provider driver without adding this service/container? Adding support for this in THT also seems little complex and probably can be done in few different ways. Is there a standard design pattern we use for these kind of requirements in Tripleo? If we already have an agreed plan on how to do it, please ignore below. I can only think of the few options below. Probably there are better ones too:) a. New OctivaDriverAgent Service. As the process is expected to be co-llocated with octavia-api to share the domain sockets (we can share /var/run/octavia from host on both containers), I'm not sure if this would be a good design though. b. As a sidecar of octavia_api Not a new service, but as a sidecar container for octavia-api (started when api starts, do we've something like this already there for other services?). Ideas/comments welcome:) [1] https://review.opendev.org/#/q/status:open+branch:master+topic:bug/1825146 -- Regards, Rabi Mishra

2 1