January 2019 - openstack-discuss - lists.openstack.org

[neutron]How to ease congestion at neutron server nodes?
by Cody 18 Jan '19

18 Jan '19

Hi Stackers, What solution(s) other than DVR could I use to avoid north-south traffic congestion at the neutron server nodes? Basically, I wish to let VMs with floating IPs to route directly from their respective hypervisor hosts to the Internet. Thank you very much. Regards, Cody

5 5

[ironic] Mid-cycle call times
by Julia Kreger 18 Jan '19

18 Jan '19

Greetings everyone, During our ironic team meeting in December, we discussed if we should go ahead and have a "mid-cycle" call in order to try sync up on where we are at during this cycle, and the next steps for us to take as a team. With that said, I have created a doodle poll[1] in an attempt to identify some days that might work. Largely the days available on the poll are geared around my availability this month. Ideally, I would like to find three days where we can schedule some 2-4 hour blocks of time. I've gone ahead and started an etherpad[2] to get us started on brainstorming. Once we have some ideas, we will be able to form a schedule and attempt to identify the amount of time required. -Julia [1]: https://doodle.com/poll/uqwywaxuxsiu7zde [2]: https://etherpad.openstack.org/p/ironic-stein-midcycle

2 5

Retiring openstack-infra/puppet-storyboard
by Andreas Jaeger 18 Jan '19

18 Jan '19

The repo openstack-infra/puppet-storyboard has been created with the intent to host storyboard by the OpenStack Infra team - and this did not happen. We will therefore retire the repository, I'm proposing patches now with topic retire-puppet-storyboard, Andreas -- Andreas Jaeger aj(a){suse.com,opensuse.org} Twitter: jaegerandi SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126

1 1

[placement] update 19-02
by Chris Dent 18 Jan '19

18 Jan '19

HTML: https://anticdent.org/placement-update-19-02.html Hi! It's a placement update! The main excitement this week is we had a meeting to check in on the state of extraction and figure out the areas that need the most attention. More on that in the extraction section within. # Most Important Work to complete and review changes to deployment to support extracted placement is the main thing that matters. # What's Changed * Placement is now able to publish release notes. * Placement is running python 3.7 unit tests in the gate, but not functional (yet). * We had that meeting and [Matt made some notes](http://lists.openstack.org/pipermail/openstack-discuss/2019-January/…. # Bugs * Placement related [bugs not yet in progress](https://goo.gl/TgiPXb): 15. +1. * [In progress placement bugs](https://goo.gl/vzGGDQ) 16. Stable. # Specs Last week was spec freeze so I'll not list all the specs here, but for reference, there were 16 specs listed last week and all 16 of them are neither merged nor abandoned. # Main Themes The reshaper work was restarted after discussion at the meeting surfaced its stalled nature. The libvirt side of things is due some refactoring while the xenapi side is waiting for a new owner to come up to speed. Gibi has proposed a related functional test. All of that at: * <https://review.openstack.org/#/q/topic:bp/reshape-provider-tree+status:open> Also making use of nested is this spectacular stack of code at bandwidth-resource-provider: * <https://review.openstack.org/#/q/topic:bp/bandwidth-resource-provider> Eric's in the process of doing lots of cleanups to how often the ProviderTree in the resource tracker is checked against placement, and a variety of other "let's make this more right" changes in the same neighborhood: * Stack at: <https://review.openstack.org/#/c/615677> That stuff is very close to ready and will make lots of people happy when it merges. One of the main areas of concern is making sure it doesn't break things for Ironic. ## Extraction As noted above, there was a meeting which resulted in [Matt's Notes](http://lists.openstack.org/pipermail/openstack-discuss/2019-January/…, an updated [extraction etherpad](https://etherpad.openstack.org/p/placement-extract-stein-5), and an improved understanding of where things stand. The critical work to ensure a healthy extraction is with getting deployment tools working. Here are _some_ of the links to that work: * [TripleO](https://review.openstack.org/#/q/topic:tripleo-placement-extractio… * [OpenStack Ansible](https://review.openstack.org/#/q/project:openstack/openstack-ansib… * [Kolla and Kolla Ansible](https://review.openstack.org/#/q/topic:split-placement) and [kolla upgrade](https://review.openstack.org/#/q/topic:upgrade-placement) We also worked out that getting the online database migrations happening on the placement side of the world would help: * [in placement](https://review.openstack.org/#/c/624942/) * [use in grenade](https://review.openstack.org/#/c/631614/) Documentation is mostly in-progress, but needs some review from packagers. A change to [openstack-manuals](https://review.openstack.org/#/c/628324/) depends on the [initial placement install docs](https://review.openstack.org/628220). There is a patch to [delete placement](https://review.openstack.org/#/c/618215/) from nova on which we've put an administrative -2 until it is safe to do the delete. # Other There are 13 [open changes](https://review.openstack.org/#/q/project:openstack/placement+statu… in placement itself. Several of those are easy win cleanups. Of those placement changes, the [online-migration-related](https://review.openstack.org/#/q/topic:bug/180392… ones are the most important. Outside of placement (I've decided to trim this list to just stuff that's seen a commit in the last two months): * <https://review.openstack.org/#/q/topic:minimum-bandwidth-allocation-placeme…> Neutron minimum bandwidth implementation * <https://review.openstack.org/#/c/621645/> WIP: add Placement aggregates tests (in tempest) * <https://review.openstack.org/627326> blazar: Consider the number of reservation inventory * <https://review.openstack.org/622316> Add placement client for basic GET operations (to tempest) # End Because I wanted to see what it might look like, I made a toy VM scheduler and placer, using etcd and placement. Then I wrote a [blog post](https://anticdent.org/etcd-placement-virt-install-compute.html). I wish there was more time for this kind of educational and exploratory playing. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

1 0

[sig][upgrades] 2019 reboot/irc meeting on Monday
by James Page 18 Jan '19

18 Jan '19

Hi All During the PTG in Denver we agreed to move the Upgrades SIG IRC meeting to a 4 week cadence. I completed the re-scheduling shortly after the PTG, but we've failed to actually hold a meeting to date! The next scheduled meetings are 0900 and 1600 UTC this coming Monday; we'll go with the standing agenda that we had before: https://etherpad.openstack.org/p/upgrades-sig-meeting Have a great weekend and look forward to chatting with interested parties on Monday! Cheers James

1 0

[dev] [infra] [devstack] [qa] [kuryr] DevStack's etcd performance on gate VM's
by Michał Dulko 18 Jan '19

18 Jan '19

Hi, In Kuryr-Kubernetes we're using the DevStack-installed etcd as a backend store for Kubernetes that we run on our gates. For some time we can see its degraded performance manifesting like this [1] in the logs. Later on this leads to various K8s errors [2], [3], up to missing notifications from the API, which causes failures in Kuryr-Kubernetes tempest tests. From what I've seen those etcd warnings normally mean that disk latency is high. This seems to be mostly happening on OVH and RAX hosts. I've looked at this with OVH folks and there isn't anything immediately alarming about their hosts running gate VM's. Upgrading the etcd version doesn't seem to help, as well as patch [4] which increases IO priority for etcd process. Any ideas of what I can try next? I think we're the only project that puts so much pressure on the DevStack's etcd. Help would really be welcomed, getting rid of this issue will greatly increase our gates stability. Thanks, Michał [1] http://logs.openstack.org/49/624749/1/check/kuryr-kubernetes-tempest-daemon… [2] http://logs.openstack.org/49/624749/1/check/kuryr-kubernetes-tempest-daemon… [3] http://logs.openstack.org/49/624749/1/check/kuryr-kubernetes-tempest-daemon… [4] https://review.openstack.org/#/c/624730/

3 5

[tripleo] LP bug bash
by Juan Antonio Osorio Robles 17 Jan '19

17 Jan '19

Hello folks! It has come to our attention that our Launchpad bug list has been growing and some bugs have gone stale. We have decided to go through the list weekly and triage or update bugs as needed in order to address this. This would be done one hour before our weekly meeting (so Tuesday at 13:00 UTC). Volunteers or people interested are welcome! There will be more details on the medium (video or IRC) on that day in the #tripleo channel. Best regards!

2 2

Neutron tagging
by cmnfths 17 Jan '19

17 Jan '19

Hi everyone! Recently I've encountered a neutron issue related to port tagging mechanism, and wonder if anyone else ever did. The initial source of the issue was, as it seems, identical numeration of external VLANs' tags and OVS (exactly br-int) port tags. The VLAN tagged 222 was added to bridge br-ex and everything worked, but at some point firewall reported unusual traffic. Thus I'd found that there was a loop and 2 qr-* ports with the same tag 222 attached to br-int. After that I deleted them and the loop was gone. So the question is what exactly these tags are? The same ethernet frame bits as 802.1Q tags? Where is a pool or something they're taken from? Is there any possibility to form such pool via configs or patches? Has anyone ever encountered such situation? Thanks, Andrew Th.

2 1

[oslo] Parallel Privsep is Proposed for Release
by Ben Nemec 17 Jan '19

17 Jan '19

Yay alliteration! :-) I wanted to draw attention to this release[1] in particular because it includes the parallel privsep change[2]. While it shouldn't have any effect on the public API of the library, it does significantly affect how privsep will process calls on the back end. Specifically, multiple calls can now be processed at the same time, so if any privileged code is not reentrant it's possible that new race bugs could pop up. While this sounds scary, it's a necessary change to allow use of privsep in situations where a privileged call may take a non-trivial amount of time. Cinder in particular has some privileged calls that are long-running and can't afford to block all other privileged calls on them. So if you're a consumer of oslo.privsep please keep your eyes open for issues related to this new release and contact the Oslo team if you find any. Thanks. -Ben 1: https://review.openstack.org/628019 2: https://review.openstack.org/#/c/593556/

5 14

Infra team upgrading review.openstack.org Gerrit from 2.13.9 to 2.13.12 January 18 at about 1700UTC
by Clark Boylan 17 Jan '19

17 Jan '19

We will be performing a minor Gerrit upgrade to version 2.13.12 tomorrow (January 18, 2019) at 1700UTC. We've tested this upgrade on our dev server, https://review-dev.openstack.org, and expect it to be a quick upgrade. Any outage shouldn't last more than 10 minutes. We will let our configuration management tooling manage the upgrade so we won't have an exact time, but will try to get it as close to 1700UTC as possible. Feel free to test out the new version on the dev server. We are happy to answer any questions you might have as well. Sorry for the short notice and the cross post, Clark

1 0