- openstack-discuss - lists.openstack.org

[placement] update 18-49
by Chris Dent 07 Dec '18

07 Dec '18

HTML: https://anticdent.org/placement-update-18-49.html This will be the last placement update of the year. I'll be travelling next Friday and after that we'll be deep in the December lull. I'll catch us up next on January 4th. # Most Important As last week, progress continues on the work in ansible, puppet/tripleo, kolla, loci to package placement up and establish upgrade processes. All of these things need review (see below). Work on GPU reshaping in virt drivers is getting close. # What's Changed * The perfload jobs which used to run in the nova-next job now has its [own job](https://review.openstack.org/#/c/619248/), running on each change. This may be of general interest because it runs placement "live" but without devstack. Results in job runs that are less than 4 minutes. * We've decided to go ahead with the simple os-resource-classes idea, so a repo is [being created](https://review.openstack.org/#/c/621666/). # Slow Reviews (Reviews which need additional attention because they have unresolved questions.) * <https://review.openstack.org/#/c/619126/> Set root_provider_id in the database This has some indecision because it does a data migration within schema migrations. For this particular case this is safe and quick, but there's concern that it softens a potentially useful boundary between schema and data migrations. # Bugs * Placement related [bugs not yet in progress](https://goo.gl/TgiPXb): 17. -2. * [In progress placement bugs](https://goo.gl/vzGGDQ) 13. -1 ## Interesting Bugs (Bugs that are sneaky and interesting and need someone to pick them up.) * <https://bugs.launchpad.net/nova/+bug/1805858> placement/objects/resource_provider.py missing test coverage for several methods This is likely the result of the extraction. Tests in nova's test_servers and friends probably covered some of this stuff, but now we need placement-specific tests. * <https://bugs.launchpad.net/nova/+bug/1804453> maximum recursion possible while setting aggregates in placement This can only happen under very heavy load with a very low number of placement processes, but the code that fails should probably change anyway: it's a potentially infinite loop with no safety breakout. # Specs Spec freeze is milestone 2, the week of January 7th. There was going to be a spec review sprint next week but it was agreed that people are already sufficiently busy. This will certainly mean that some of these specs do not get accepted for this cycle. None of the specs listed last week have merged. * <https://review.openstack.org/#/c/544683/> Account for host agg allocation ratio in placement (Still in rocky/) * <https://review.openstack.org/#/c/595236/> Add subtree filter for GET /resource_providers * <https://review.openstack.org/#/c/597601/> Resource provider - request group mapping in allocation candidate * <https://review.openstack.org/#/c/549067/> VMware: place instances on resource pool (still in rocky/) * <https://review.openstack.org/#/c/555081/> Standardize CPU resource tracking * <https://review.openstack.org/#/c/599957/> Allow overcommit of dedicated CPU (Has an alternative which changes allocations to a float) * <https://review.openstack.org/#/c/591037/> Modelling passthrough devices for report to placement * <https://review.openstack.org/#/c/603955/> Nova Cyborg interaction specification. * <https://review.openstack.org/#/c/601596/> supporting virtual NVDIMM devices * <https://review.openstack.org/#/c/603352/> Spec: Support filtering by forbidden aggregate * <https://review.openstack.org/#/c/552924/> Proposes NUMA topology with RPs * <https://review.openstack.org/#/c/569011/> Count quota based on resource class * <https://review.openstack.org/#/c/141219/> Adds spec for instance live resize * <https://review.openstack.org/#/c/612497/> Provider config YAML file * <https://review.openstack.org/#/c/509042/> Propose counting quota usage from placement and API database * <https://review.openstack.org/603545> Resource modeling in cyborg. * <https://review.openstack.org/#/c/609960/> Support filtering of allocation_candidates by forbidden aggregates # Main Themes ## Making Nested Useful Progress continues on gpu-reshaping for libvirt and xen: * <https://review.openstack.org/#/q/topic:bp/reshape-provider-tree+status:open> Also making use of nested is bandwidth-resource-provider: * <https://review.openstack.org/#/q/topic:bp/bandwidth-resource-provider> Eric's in the process of doing lots of cleanups to how often the ProviderTree in the resource tracker is checked against placement, and a variety of other "let's make this more right" changes in the same neighborhood: * Stack at: <https://review.openstack.org/#/c/615677> ## Extraction The [extraction etherpad](https://etherpad.openstack.org/p/placement-extract-stein-4) is starting to contain more strikethrough text than not. Progress is being made. The main tasks are the reshaper work mentioned above and the work to get deployment tools operating with an extracted placement: * [TripleO](https://review.openstack.org/#/q/topic:tripleo-placement-extractio… * [OpenStack Ansible](https://review.openstack.org/#/q/project:openstack/openstack-ansib… * [Kolla](https://review.openstack.org/#/c/613589/) * [Kolla Ansible](https://review.openstack.org/#/c/613629/) * [Loci](https://review.openstack.org/#/c/617273/) Documentation tuneups: * Release-notes: <https://review.openstack.org/#/c/618708/> This is blocked until we refactor the release notes to reflect _now_ better. * The main remaining task here is participating in [openstack-manuals](https://docs.openstack.org/doc-contrib-guide/doc-index.h…. The functional tests in nova that use [extracted placement](https://review.openstack.org/#/c/617941/) are working but not yet merged. A child of that patch [removes the placement code](https://review.openstack.org/#/c/618215/). Further work will be required to tune up the various pieces of documentation in nova that reference placement. # Other There are currently only 8 [open changes](https://review.openstack.org/#/q/project:openstack/placement+statu… in placement itself. Most of the time critical work is happening elsewhere (notably the deployment tool changes listed above). Of those placement changes the [database-related](https://review.openstack.org/#/q/owner:nakamura.tetsuro%2… ones from Tetsuro are the most important. Outside of placement: * <https://review.openstack.org/#/q/topic:bp/initial-allocation-ratios> Improve handling of default allocation ratios * <https://review.openstack.org/#/q/topic:minimum-bandwidth-allocation-placeme…> Neutron minimum bandwidth implementation * <https://review.openstack.org/#/c/602160/> Add OWNERSHIP $SERVICE traits * <https://review.openstack.org/#/c/586960/> zun: Use placement for unified resource management * <https://review.openstack.org/#/q/project:openstack/blazar+topic:bp/placemen…> Blazar using the placement-api * <https://review.openstack.org/619626> Tenks doing some node management, with a bit of optional placement. * <https://review.openstack.org/#/c/620485/> Sync placement database to the current version (in grenade) * <https://review.openstack.org/#/c/621645/> WIP: add Placement aggregates tests (in tempest) # End In case it hasn't been clear: things being listed here is an explicit invitation (even plea) for _you_ to help out by reviewing or fixing. Thank you. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

2 1

[dev][keystone] Keystone Team Update - Week of 3 December 2018
by Colleen Murphy 07 Dec '18

07 Dec '18

# Keystone Team Update - Week of 3 December 2018 ## News ### New Outreachy Interns The keystone team is lucky enough to have two Outreachy interns this round: welcome to Erus (erus) and Islam (imus)! Erus will be helping us with our federation implementation and Islam will be working on improving our API unit tests. We're happy to have you! ### Keystone as IdP Proxy (Broker) Morgan started working on a diagram to explain the proposed architecture for keystone as an identity provider proxy/broker[1][2]. It is a good starting point for understanding the general idea of what the proposed design is. Please have a look at it and bring your questions to Morgan. [1] https://usercontent.irccloud-cdn.com/file/Au4e3DXb/Keystone%20IDP%20(initia… [2] http://eavesdrop.openstack.org/meetings/keystone/2018/keystone.2018-12-04-1… ## Open Specs Stein specs: https://bit.ly/2Pi6dGj Ongoing specs: https://bit.ly/2OyDLTh ## Recently Merged Changes Search query: https://bit.ly/2pquOwT We merged 25 changes this week. ## Changes that need Attention Search query: https://bit.ly/2RLApdA There are 91 changes that are passing CI, not in merge conflict, have no negative reviews and aren't proposed by bots. ## Bugs This week we opened 5 new bugs and closed 4. Bugs opened (5) Bug #1806713 (keystone:Medium) opened by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1806713 Bug #1806762 (keystone:Medium) opened by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1806762 Bug #1806195 (keystone:Undecided) opened by nigel cook https://bugs.launchpad.net/keystone/+bug/1806195 Bug #1806377 (keystone:Undecided) opened by kirandevraaj https://bugs.launchpad.net/keystone/+bug/1806377 Bug #1807184 (oslo.policy:Medium) opened by Brian Rosmaita https://bugs.launchpad.net/oslo.policy/+bug/1807184 Bugs fixed (4) Bug #1806109 (keystoneauth:Medium) fixed by Monty Taylor https://bugs.launchpad.net/keystoneauth/+bug/1806109 Bug #1800259 (oslo.policy:Undecided) fixed by Lance Bragstad https://bugs.launchpad.net/oslo.policy/+bug/1800259 Bug #1803722 (oslo.policy:Undecided) fixed by Corey Bryant https://bugs.launchpad.net/oslo.policy/+bug/1803722 Bug #1804073 (oslo.policy:Undecided) fixed by John Dennis https://bugs.launchpad.net/oslo.policy/+bug/1804073 ## Milestone Outlook https://releases.openstack.org/stein/schedule.html We have just a month left until our spec freeze. A month after that is our feature proposal freeze, so if you are planning significant feature work, best to get started on it soon. ## Help with this newsletter Help contribute to this newsletter by editing the etherpad: https://etherpad.openstack.org/p/keystone-team-newsletter Dashboard generated using gerrit-dash-creator and https://gist.github.com/lbragstad/9b0477289177743d1ebfc276d1697b67

1 0

[nova] Moving nova-cells-v1 job to the experimental queue
by Matt Riedemann 07 Dec '18

07 Dec '18

I have changes up which move the nova-cells-v1 CI job to the experimental queue: https://review.openstack.org/#/q/topic:bug/1807407 We continue to see intermittent failures in that job and given the deprecated nature of cells v1, CERN is using cells v2, the job no longer runs nova-network, and its flakiness (plus the general state of the gate lately), it is time to move this further out of our daily lives and down the road to its eventual removal. Keeping it as an option to run on-demand in the experimental queue at least allows us to leverage it if needed should some major cells v1 fix be needed, but that is doubtful (we haven't had a cells v1 specific bug fix in a long time). -- Thanks, Matt

1 0

[ironic] Time to discuss clean/deploy steps
by Julia Kreger 07 Dec '18

07 Dec '18

All, I've looked at the doodle poll results and it looks like the best available time is 3:00 PM UTC on Friday December 7th. I suggest we use bluejeans[2] as that has worked fairly well for us thus far. The specification documented related to the discussion can be found in review[3]. Thanks, -Julia [1] https://doodle.com/poll/yan4wyvztf7mpq46 [2] https://bluejeans.com/u/jkreger/ [3] https://review.openstack.org/#/c/606199/

2 2

[nova] What to do with the legacy notification interface?
by Balázs Gibizer 07 Dec '18

07 Dec '18

Hi, Last decision from the Denver PTG [1] was to * finish the remaining transformations [2] * change the default value of notification_format conf option from 'both' to 'versioned' * communicate that the legacy notification interface has been deprecated, unsupported but the code will not be removed Since the PTG the last two transformation patches [2] has been approved and the deprecation patch has been prepared [3]. However recently we got a bug report [4] where the performance impact of emiting both legacy and versioned notification caused unnecessary load on the message bus. (Btw this performance impact was raised in the original spec [5]). The original reason we emited both type of notifications was to keep backward compatibility with consumers only understanding legacy notifications. It worked so well that most of the consumers still depend on the legacy notifications (detailed status is in [1]). I see three options to move forward: A) Do nothing. By default, emit both type of notifications. This means that deployers seeing the performance impact of [4] needs to reconfigure nova. Also this is agains our decision to move away from supporting legacy notifications. B) Follow the plan from the PTG and change the default value of the config to emit only versioned notifications. This solves the immediate effect of the bug [4]. This is following our original plan. BUT this is backward incompatible change which in the current situation means most of the deployments (e.g. those, deploying notification consumer modules like ceilometer) need to change this config to 'unversioned' or 'both'. There was discussion about helping out notification consumers to convert their code using the new versioned interface but I have no spare time at the moment and I failed to drum up resources internally for this work. Also I don't see others volunteering for such work. C) Change the default to 'legacy'. This solves the bug [4]. BUT sends a really mixed message. As our original goal was to stop supporting the legacy notification interface of nova. Matt stated on IRC recently, there is no real bug inflow for the legacy code. Also looking at changes in the code I see that last cycle we had couple of blueprints extending the new versioned notifications with extra fields but that inflow stopped in this cycle too. So we most probably can keep the legacy code in place and supported forever without too much pain and too much divergence from the versioned interface. -- Personally, I spent enough time implementing and reviewing the versioned interface that I'm biased towards option B). But I do understand that our original goals setting in 2015 was made in such environment that changed significantly in the past cycles. So I can accept option C) as well if we can agree. Cheers, gibi [1] https://etherpad.openstack.org/p/nova-ptg-stein L765 [2] https://review.openstack.org/#/q/topic:bp/versioned-notification-transforma… [3] https://review.openstack.org/#/c/603079/ [4] https://bugs.launchpad.net/nova/+bug/1805659 [5] https://review.openstack.org/#/c/224755/11/specs/mitaka/approved/versioned-…

2 2

[tc] Technical Committee status update for 6 December
by Doug Hellmann 06 Dec '18

06 Dec '18

This is the (alegedly) weekly summary of work being done by the Technical Committee members. The full list of active items is managed in the wiki: https://wiki.openstack.org/wiki/Technical_Committee_Tracker We also track TC objectives for the cycle using StoryBoard at: https://storyboard.openstack.org/#!/project/923 == Recent Activity == It has been 4 weeks since the last update email, in part due to the Summit, then a holiday, then my absense due to illness. Project updates: * Add tenks under Ironic: https://review.openstack.org/#/c/600411/ * Add os_placement role to OpenStack Ansible: https://review.openstack.org/#/c/615187/ * Retired openstack-ansible-os_monasca-ui https://review.openstack.org/#/c/617322/ * Chris Hoge was made the new Loci team PTL https://review.openstack.org/#/c/620370/ Other updates: * Thierry has been working on a series of changes on behalf of the release management team to record the release management style used for each deliverable listed in the governance repository. https://review.openstack.org/#/c/613268/ * Zane updated our guidelines for new projects to clarify the scoping requirements: https://review.openstack.org/#/c/614799/ * Zane also added a Vision for OpenStack Clouds document, describing what we see as the scope of OpenStack overall: https://review.openstack.org/#/c/592205/ * I copied some code we had in a couple of other places into the governance repository to make it easier for the release, goal, and election tools to consume the governance data using the new openstack-governance library from PyPI. https://review.openstack.org/#/c/614599/ * I started a document describing the responsibilities of the TC chair. https://review.openstack.org/#/c/618810/ == TC Meetings == In order to fulfill our obligations under the OpenStack Foundation bylaws, the TC needs to hold meetings at least once each quarter. We agreed to meet monthly, and to emphasize agenda items that help us move initiatives forward while leaving most of the discussion of those topics to the mailing list. The agendas for all of our meetings will be sent to the openstack-dev mailing list in advance, and links to the logs and summary will be sent as a follow up after the meeting. Our most recent meeting was held on 6 Dec 2018. * http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000696… The next meeting will be 3 December @ 1400 UTC in #openstack-tc == Ongoing Discussions == We have several governance changes up for review related to deciding how we will manage future Python 3 upgrades (including adding 3.7 and possibly dropping 3.5 during Stein). These are close to being approved, with some final wording adjustments being made now. * Explicitly declare stein supported runtimes: https://review.openstack.org/#/c/611080/ * Resolution on keeping up with Python 3 releases: https://review.openstack.org/#/c/613145/ I proposed an update to our house rules to allow faster approval of release management metadata maintained in the governance repository. * https://review.openstack.org/#/c/622989/ Thierry and Chris have been working on a description of the role of the TC. * https://review.openstack.org/#/c/622400/ Ghanshyam has proposed an enhancement to the Vision for OpenStack Clouds to cover feature discovery. * https://review.openstack.org/#/c/621516/ Lance has proposed an update to the charter to clarify how we will handle PTL transitions in the middle of a development cycle. * https://review.openstack.org/#/c/620928/ == TC member actions/focus/discussions for the coming week(s) == We have several significant changes up for review. Please take time to consider all of the open patches, then comment and vote on them. It's very difficult to tell if we have reached consensus if everyone waits for the conversation to settle before commenting. == Contacting the TC == The Technical Committee uses a series of weekly "office hour" time slots for synchronous communication. We hope that by having several such times scheduled, we will have more opportunities to engage with members of the community from different timezones. Office hour times in #openstack-tc: - 09:00 UTC on Tuesdays - 01:00 UTC on Wednesdays - 15:00 UTC on Thursdays If you have something you would like the TC to discuss, you can add it to our office hour conversation starter etherpad at: https://etherpad.openstack.org/p/tc-office-hour-conversation-starters Many of us also run IRC bouncers which stay in #openstack-tc most of the time, so please do not feel that you need to wait for an office hour time to pose a question or offer a suggestion. You can use the string "tc-members" to alert the members to your question. You will find channel logs with past conversations at http://eavesdrop.openstack.org/irclogs/%23openstack-tc/ If you expect your topic to require significant discussion or to need input from members of the community other than the TC, please start a mailing list discussion on openstack-discuss at lists.openstack.org and use the subject tag "[tc]" to bring it to the attention of TC members. -- Doug

3 3

Re: [sdk] Establishing SDK Validation Baseline
by Melvin Hillsman 06 Dec '18

06 Dec '18

On Thu, Dec 6, 2018 at 12:30 PM Artem Goncharov <artem.goncharov(a)gmail.com> wrote: > > > On Thu, 6 Dec 2018, 19:12 Melvin Hillsman, <mrhillsman(a)gmail.com> wrote: > >> Hi everyone, >> >> We have spent some time working to get an idea of what official SDKs >> would look like. We had some sessions during the Berlin summit[0][1] and >> there was a lot of great feedback. >> >> Currently the following SDKs are generally considered usable for their >> respective language; there are others of course: >> >> openstacksdk (Python) >> gophercloud (Go) >> pkgcloud (JavaScript) >> openstack4j (Java) >> rust-openstack (Rust) >> fog-openstack (Ruby) >> php-opencloud (PHP) >> >> After many discussions it seems SDK validation essentially should be >> about confirming cloud state pre/post SDK interaction rather than API >> support. An example is that when I use pkgcloud and ask that a VM be >> created, does the VM exist, in the way I asked it exist, rather than are >> there specific API calls that are being hit along the way to creating my VM. >> >> I am putting this email out to keep the community informed of what has >> been discussed in this space but also and most importantly to get feedback >> and support for this work. It would be great to get a set of official and >> community SDKs, get them setup with CI testing for validation (not changing >> their current CI for unit/functional/acceptance testing; unless asked to >> help do this), and connect the results to the updated project navigator SDK >> section. A list of scenarios has been provided as a good starting point for >> cloud state checks.[2] >> >> Essentially the proposal is to deploy OpenStack from upstream (devstack >> or other), stand up a VM within the cloud, grab all the SDKs, run >> acceptance tests, report pass/fail results, update project navigator. Of >> course there are details to be worked out and I do have a few questions >> that I hope would help get everyone interested on the same page via this >> thread. >> >> >> 1. Does this make sense? >> >> > As a representative of a public cloud operator I say yes. It definitely > makes sense to let users know, which SDKs are certified to work to have > understanding what they can use. However then comes the question, that each > public operator has own "tricks" and possibly even extension services, > which make SDK behave not as expected. I do really see a need to provide a > regular way in each of those SDKs to add and (I really don't want this, but > need unfortunately) sometimes override services implementation if those are > changed in some way. > So the certification make sense, but the CI question probably goes in the > direction of OpenLab > Yes this is definitely something to consider; public cloud tricks and extensions. An idea that was floated is to have an upstream cloud to run against that passes all interop powered programs to run the SDKs against. Even though SDKs provide more coverage an SDK that works against a service that is certified should work in any cloud that is certified? > >> 1. Would folks be interested in a SDK SIG or does it make more sense >> to request an item on the API SIG's agenda? >> >> Both has pro and cons, but I am neutral here > > Same, just wanted to get it out there and hopefully get API SIG members feedback so as not to hijack any of their agendas or creep on their scope without consent. > > >> 1. Bi-weekly discussions a good cadence? >> >> Yes. The regular communication is a requirement. > > > >> 1. Who is interested in tackling this together? >> >> > I do, but not alone > Haha, agreed, but we definitely should work on it and be consistent even if the initial set of folks is small. We have a large community to lean on and should rather than simply going at it alone. > > >> >> >> [0] https://etherpad.openstack.org/p/BER-better-expose-what-we-produce >> [1] https://etherpad.openstack.org/p/BER-sdk-certification >> [2] >> https://docs.google.com/spreadsheets/d/1cdzFeV5I4Wk9FK57yqQmp5JJdGfKzEOdB3V… >> >> >> -- >> Kind regards, >> >> Melvin Hillsman >> mrhillsman(a)gmail.com >> mobile: (832) 264-2646 >> > > Regards, > Artem Goncharov > gtema > OpenTelekomCloud > >> -- Kind regards, Melvin Hillsman mrhillsman(a)gmail.com mobile: (832) 264-2646

1 0

[openstack-dev] [puppet] [stable] Deprecation of newton branches
by Tobias Urdin 06 Dec '18

06 Dec '18

Hello, We've been talking for a while about the deprecation and removal of the stable/newton branches. I think it's time now that we get rid of them, we have no open patches and Newton is considered EOL. Could cores get back with a quick feedback and then the stable team can get rid of those whenever they have time. Best regards Tobias __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

6 8

[dev][nova] time for another spec review day?
by melanie witt 06 Dec '18

06 Dec '18

Hi All, Our spec freeze is milestone 2 January 10 and I was thinking, because of holiday time coming up, it might be a good idea to have another spec review day ahead of the freeze early next year. I was thinking maybe Tuesday next week December 11, to allow the most amount of time before holiday PTO starts. Please let me know what you think. Cheers, -melanie

1 1

[ops] Disk QoS in Cinder and Nova flavour
by Jean-Philippe Méthot 06 Dec '18

06 Dec '18

Hi, This is something that’s not exactly clear in the documentation and so I thought I’d ask here. There are currently two ways to set disk IO limits, or QoS: through Cinder QoS values or through Nova flavours. My question is, what’s the difference exactly? I assume that Cinder QoS only applies on Cinder volume, but is the same true for flavour QoS? In other words, is QoS through flavour supposed to apply to both Cinder volumes and ephemeral storage or is it only for ephemeral storage? Additionally, I'm under the assumption that once you provision a block device of a certain type with QoS associated to it, it becomes impossible to modify the values afterwards. Is that correct? This is a strange behaviour, considering that technically, you should be able to change QoS when it’s set through flavour using a simple resize. Please correct me if I’m wrong. I feel that this could be better explained in the current documentation. Jean-Philippe Méthot Openstack system administrator Administrateur système Openstack PlanetHoster inc.

1 0