February 2019 - openstack-discuss

[nova][qa] Should we make legacy-grenade-dsvm-neutron-multinode-live-migration voting?
by Matt Riedemann 06 Mar '19

06 Mar '19

The legacy-grenade-dsvm-neutron-multinode-live-migration job was recently fixed [1][2] but had been broken for awhile because it's non-voting so no one really noticed. The unique thing about this job is it runs live migration tests in a grenade setting where one of the computes is n-1 and it does the live migration back and forth between the nodes so we have old->new and new->old coverage, and it does this across local disk and shared storage with ceph. Now that it's working again, should we make it voting? It came up while discussing regression test coverage for Artom's NUMA live migration series. It seems like a no-brainer to me at least that we should be gating on this. Before doing that, I'd like to move the job config from openstack-zuul-jobs to nova so we "own" any changes to the job and then might also consider converting it to zuul v3, but that's not a blocker to make it voting. [1] https://review.openstack.org/#/c/634962/ [2] https://review.openstack.org/#/c/637231/ -- Thanks, Matt

5 5

[senlin] Senlin Monthly(ish) Newsletter Jan/Feb 2019
by Duc Truong 06 Mar '19

06 Mar '19

HTML: https://dkt26111.wordpress.com/2019/02/21/senlin-monthlyish-newsletter-janu… This is the January/February edition of the Senlin monthly(ish) newsletter. The goal of the newsletter is to highlight happenings in the Senlin project. If you have any feedback or questions regarding the contents, please feel free to reach out to me in the #senlin IRC channel. News ---- * We are almost at Stein-3 milestone which coincides with the feature freeze during the week of March 4. Please submit your changes for review before then. Blueprint Status ---------------- * Fail fast locked resource - https://blueprints.launchpad.net/senlin/+spec/fail-fast-locked-resource - Working on documentation and release notes. * Multiple detection modes - https://blueprints.launchpad.net/senlin/+spec/multiple-detection-modes - Working on documentation and release notes. Community Goal Status --------------------- * Python 3 - All patches by Python 3 goal champions for zuul migration, documentation and unit test changes have been merged. - We have set py35 functional tests as voting in gate. * Upgrade Checkers - I have added a patch set to check for unsupported health policies: https://review.openstack.org/#/c/638284/ Reviews Needed -------------- * Improve Health Manager to avoid duplicate health checks: https://review.openstack.org/#/c/634811/ * Fixing devstack tempest jobs for master and stable/rocky: https://review.openstack.org/#/c/637664/ https://review.openstack.org/#/c/635638/

2 1

[nova] NUMA live migration - mostly how it's tested
by Artom Lifshitz 05 Mar '19

05 Mar '19

Hey all, There won't be much new here for those who've reviewed the patches [1] already, but I wanted to address the testing situation. Until recently, the last patch was WIP because I had functional tests but no unit tests. Even without NUMA anywhere, the claims part of the new code could be tested in functional tests. With the new and improved implementation proposed by Dan Smith [2], this is no longer the case. Any test more involved than unit testing will need "real" NUMA instances on "real" NUMA hosts to trigger the new code. Because of that, I've dropped functional testing altogether, have added unit tests, and have taken the WIP tag off. What I've been using for testing is this: [3]. It's a series of patches to whitebox_tempest_plugin, a Tempest plugin used by a bunch of us Nova Red Hatters to automate testing that's outside of Tempest's scope. Same idea as the intel-nfv-ci plugin [4]. The tests I currently have check that: * CPU pin mapping is updated if the destination has an instance pinned to the same CPUs as the incoming instance * emulator thread pins are updated if the destination has a different cpu_shared_set value and the instance has the hw:emulator_threads_policy set to `share` * NUMA node pins are updated for a hugepages instance if the destination has a hugepages instances consuming the same NUMA node as the incoming instance It's not exhaustive by any means, but I've made sure that all iterations pass those 3 tests. It should be fairly easy to add new tests, as most of the necessary scaffolding is already in place. [1] https://review.openstack.org/#/c/634606/ [2] https://review.openstack.org/#/c/634828/28/nova/virt/driver.py@1147 [3] https://review.rdoproject.org/r/#/c/18832/ [4] https://github.com/openstack/intel-nfv-ci-tests/

5 14

[neutron] Multi-segment per host support for routed networks
by David G. Bingham 05 Mar '19

05 Mar '19

Neutron land, Problem: Neutron currently only allows a single network segment per host. This becomes a problem when networking teams want to limit the number of IPs it supports on a segment. This means that at times the number of IPs available to the host is the limiting factor for the number of instances we can deploy on a host. Ref: https://bugs.launchpad.net/neutron/+bug/1764738 Ongoing Work: We are excited in our work add "multi-segment support for routed networks". We currently have a proof of concept here https://review.openstack.org/#/c/623115 that for routed networks effectively: * Removes validation preventing multiple segments. * Injects segment_id into fixed IP records. * Uses the segment_id when creating a bridge (rather than network_id). In effect, it gives each segment its own bridge. It works pretty well for new networks and deployments. For existing routed networks, however, it breaks networking. Please use *caution* if you decide to try it. TODOs: Things TODO before this before it is fully baked: * Need to add code to handle ensuring bridges are also updated/deleted using the segment_id (rather than network_id). * Need to add something (a feature flag?) that prevents this from breaking routed networks when a cloud admin updates to master and is configured for routed networks. * Need to create checker and upgrade migration code that will convert existing bridges from network_id based to segment_id based (ideally live or with little network traffic downtime). Once converted, the feature flag could enable the feature and start using the new code. Need: 1. How does one go about adding a migration tool? Maybe some examples? 2. Will nova need to be notified/upgraded to have bridge related files updated? 3. Is there a way to migrate without (or minimal) downtime? 4. How to repeatably test this migration code? Grenade? Looking for any ideas that can keep this moving :) Thanks a ton, David Bingham (wwriverrat on irc) Kris Lindgren (klindgren on irc) Cloud Engineers at GoDaddy

2 2

[docs] Implementation of the api-ref consolidation under doc/source/
by Luigi Toscano 04 Mar '19

04 Mar '19

Hi all, During the last PTG it was decided to move forward with the migration of the api-ref documentation together with the rest of the documentation [1]. This is one of the item still open after the (not so recent anymore) massive documentation restructuring [2]. (most likely anything below applies to releasenotes/ as well.) I asked about this item few weeks ago on the documentation channels. So far no one seems against moving forward with this, but, you know, resources :) I think that the process itself shouldn't be too complicated on the technical side, but more on the definition of the desired outcome. I can help with the technical part (the moving), but it would be better if someone from the doc team with the required knowledge and background on the doc process would start with at least a draft of a spec, which can be used to start the discussion. If implemented, this change would also fix an inconsistency in the guidelines [2]: the content of reference/ seems to overlap with the content of api- ref("Library projects should place their automatically generated class documentation here."), but then having api-ref there would allow us to always use api-ref. That's where the entire discussion started in the QA session [3]: some client libraries document their API in different places. [1] https://etherpad.openstack.org/p/docs-i18n-ptg-stein line 144 [2] https://docs.openstack.org/doc-contrib-guide/project-guides.html [3] https://etherpad.openstack.org/p/clean-up-the-tempest-documentation Ciao -- Luigi

8 15

[fenix] starting bi-weekly meetings
by Juvonen, Tomi (Nokia - FI/Espoo) 04 Mar '19

04 Mar '19

Hi, Now reaching a point with the Fenix project where we should start to have meetings in #openstack-fenix. I propose to have a bi-weekly meeting at 6 AM UTC, starting from 11th March. Please give your +1 or indicate if the time does not suit to you. For those who have not heard about the project, you can find more from [1]. Currently working to have the rolling OpenStack upgrade implemented before ONS summit and then make it a job to be run on each patch later on. What is awesome is that I use multinode DevStack environment and I am modifying the scripts to do most of the node upgrade for me just by passing "UPGRADE=True" in local.conf. Let's see how it goes, might be a quite generic solution for anybody to test upgrade with own changes to projects. [1] https://wiki.openstack.org/wiki/Fenix Regards, Tomi Fenix PTL

2 3

[all][ops] Ops Meetup Agenda Planning - Berlin Edition
by Erik McCormick 03 Mar '19

03 Mar '19

Hello All, The time is rapidly approaching for the Ops Meetup in Berlin. In preparation, we need your help developing the agenda. i put an [all] tag on this because I'm hoping that anyone, not just ops, looking for discussion and feedback on particular items might join in and suggest sessions. It is not required that you attend the meetup to post session ideas. If there is sufficient interest, we will hold the session and provide feedback and etherpad links following the meetup. Please insert your session ideas into this etherpad, add subtopics to already proposed sessions, and +1 those that you are interested in. Also please put your name, and maybe some contact info, at the bottom. If you'd be willing to moderate a session, please add yourself to the moderators list. https://etherpad.openstack.org/p/BER-ops-meetup I'd like to give a big shout out to Deutsche Telekom for hosting us and providing the catering. I look forward to seeing many of you in Berlin! Cheers, Erik

3 3

sriov bonding
by Manuel Sopena Ballesteros 03 Mar '19

03 Mar '19

Hi, Is there a documentation that explains how to setup bonding on SR-IOV neutron? Thank you Manuel NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed.

6 7

[ops] berlin ops meetup content thread
by Chris Morgan 01 Mar '19

01 Mar '19

Hello Everyone, Here's the first etherpad for next week's Ops Meetup in Berlin, hosted by Deutsche Telekom: https://etherpad.openstack.org/p/BER19-OPS-DUDES This one is for a proposed group discussion amongst the openstack operators attending about all the new (or newly top-level) projects under the openstack foundation umbrella, or as I've nicknamed it "All the young dudes", kata,zuul, airship, starlingx. I also added a section about the changes to the foundation, stackforge, the summits etc.I haven't seen a prior effort to get feedback from openstack operators on all these changes. Feedback welcome Chris p.s. still tickets available, come join us if you can - especially if it's nearby to you there's still time! https://www.eventbrite.com/e/openstack-ops-meetup-berlin-tickets-55034908894 -- Chris Morgan <mihalis68(a)gmail.com>

1 2

[nova] Updates about Detaching/Attaching root volumes
by Zhenyu Zheng 01 Mar '19

01 Mar '19

Hi Nova, I'm working on a blueprint to support Detach/Attach root volumes. The blueprint has been proposed for quite a while since mitaka[1] in that version of proposal, we only talked about instances in shelved_offloaded status. And in Stein[2] the status of stopped was also added. But now we realized that support detach/attach root volume on a stopped instance could be problemastic since the underlying image could change which might invalidate the current host.[3] So Matt and Sean suggested maybe we could just do it for shelved_offloaded instances, and I have updated the patch according to this comment. And I will update the spec latter, so if anyone have thought on this, please let me know. Another thing I wanted to discuss is that in the proposal, we will reset some fields in the root_bdm instead of delete the whole record, among those fields, the tag field could be tricky. My idea was to reset it too. But there also could be cases that the users might think that it would not change[4]. Thoughts, BR, [1] http://specs.openstack.org/openstack/nova-specs/specs/mitaka/approved/detac… [2] http://specs.openstack.org/openstack/nova-specs/specs/stein/approved/detach… [3] https://review.openstack.org/#/c/614750/34/nova/compute/manager.py@5467 [4] https://review.openstack.org/#/c/614750/37/nova/objects/block_device.py

5 18