July 2019 - openstack-discuss - lists.openstack.org

[keystone] Keystone Team Update - Week of 1 July 2019
by Colleen Murphy 06 Jul '19

06 Jul '19

# Keystone Team Update - Week of 1 July 2019 ## News ### Midcycle Planning If you have not already done so, please participate in the scheduling poll[1] and contribute topic ideas[2] for the virtual midcycle. We will select a date next week and start preparing the agenda. [1] https://doodle.com/poll/wr7ct4uhpw82sysg [2] https://etherpad.openstack.org/p/keystone-train-midcycle-topics ### Oslo.limit Walkthrough We used this week's office hours session to walk through the new proposals for the oslo.limit interface, which Lance nicely summarized[3]. [3] http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007487.html ## Open Specs Train specs: https://bit.ly/2uZ2tRl Ongoing specs: https://bit.ly/2OyDLTh We still have three Train specs that needs to be updated or reviewed prior to the Milestone 2 deadline. ## Recently Merged Changes Search query: https://bit.ly/2pquOwT We merged 14 changes this week. ## Changes that need Attention Search query: https://bit.ly/2tymTje There are 43 changes that are passing CI, not in merge conflict, have no negative reviews and aren't proposed by bots. ## Bugs This week we opened 5 new bugs and closed none. Bugs opened (5) Bug #1835299 (keystone:Low) opened by Colleen Murphy https://bugs.launchpad.net/keystone/+bug/1835299 Bug #1835303 (keystoneauth:Undecided) opened by Ben Nemec https://bugs.launchpad.net/keystoneauth/+bug/1835303 Bug #1835103 (oslo.limit:Low) opened by Lance Bragstad https://bugs.launchpad.net/oslo.limit/+bug/1835103 Bug #1835104 (oslo.limit:Low) opened by Lance Bragstad https://bugs.launchpad.net/oslo.limit/+bug/1835104 Bug #1835106 (oslo.limit:Low) opened by Lance Bragstad https://bugs.launchpad.net/oslo.limit/+bug/1835106 ## Milestone Outlook https://releases.openstack.org/train/schedule.html Spec freeze is in 3 weeks, closely followed by feature proposal freeze. If you are working on a feature, even if the spec is not approved yet, don't hesitate to propose some code ahead of the deadline. ## Help with this newsletter Help contribute to this newsletter by editing the etherpad: https://etherpad.openstack.org/p/keystone-team-newsletter

1 0

[all][release] PTL on Vacation July 6-14
by Tony Breeds 06 Jul '19

06 Jul '19

Hi All, Just a very quick note to say I'll be AFK for the next week. The only visible change could be that stable releases are delayed a little. As always if it's urgent nothing stops the release team from approving things in my absence Yours Tony.

1 0

[release] Release countdown for week R-14, July 8-12
by Tony Breeds 06 Jul '19

06 Jul '19

Hi folks, Development Focus ----------------- The Train-2 milestone will happen in three weeks, on July 25. Train-related specs should now be finalized so that teams can move to implementation ASAP. Some teams observe specific deadlines on the second milestone (mostly spec freezes): please refer to https://releases.openstack.org/train/schedule.html for details. General Information ------------------- Please remember that libraries need to be released at least once per milestone period. At milestone 2, the release team will propose releases for any library that has not been otherwise released since milestone 1. Other non-library deliverables that follow the cycle-with-intermediary release model should have an intermediary release before milestone-2. Those who haven't will be proposed to switch to the cycle-with-rc model, which is more suited to deliverables that are released only once per cycle. At milestone-2 we also freeze the contents of the final release. If you have a new deliverable that should be included in the final release, you should make sure it has a deliverable file in https://opendev.org/openstack/releases/src/branch/master/deliverables/train . You should request a beta release (or intermediary release) for those new deliverables by milestone-2. We understand some may not be quite ready for a full release yet, but if you have something minimally viable to get released it would be good to do a 0.x release to exercise the release tooling for your deliverables. See the MembershipFreeze description for more details: https://releases.openstack.org/train/schedule.html#t-mf Upcoming Deadlines & Dates -------------------------- Train-2 Milestone: July 25 (R-12 week) Yours Tony.

1 0

[goal][python3] Train unit tests weekly update (goal-10)
by Corey Bryant 06 Jul '19

06 Jul '19

This is the goal-10 weekly update for the "Update Python 3 test runtimes for Train" goal [1]. There are 10 weeks remaining for completion of Train community goals [2]. == What's the Goal? == To ensure (in the Train cycle) that all official OpenStack repositories with Python 3 unit tests are exclusively using the 'openstack-python3-train-jobs' Zuul template or one of its variants (e.g. 'openstack-python3-train-jobs-neutron') to run unit tests, and that tests are passing. This will ensure that all official projects are running py36 and py37 unit tests in Train. For complete details please see [1]. == Ongoing Work == Patches have been submitted for all applicable projects except for: OpenStack Charms, Puppet OpenStack, Quality Assurance, Release Management, tripleo. Open patches needing reviews: https://review.openstack.org/#/q/topic:python3-train+is:open Failing patches: https://review.openstack.org/#/q/topic:python3-train+status:open+(+label:Ve… Patch automation scripts needing review: https://review.opendev.org/#/c/666934 == Completed Work == Merged patches: https://review.openstack.org/#/q/topic:python3-train+is:merged == How can you help? == Please take a look at the failing patches and help fix any failing unit tests for your projects. Python 3.7 unit tests will be self-testing in Zuul. If you're interested in helping submit patches, please let me know. == Reference Material == [1] Goal description: https://governance.openstack.org/tc/goals /train/python3-updates.html [2] Train release schedule: https://releases.openstack.org/train/schedule.html (see R-5 for "Train Community Goals Completed") Storyboard: https://storyboard.openstack.org/#!/story/2005924 Porting to Python 3.7: https://docs.python.org/3/whatsnew/3.7.html#porting-to-python-3-7 Python Update Process: https://opendev.org/openstack/governance/src/branch/master/resolutions/2018… Train runtimes: https://opendev.org/openstack/governance/src/branch/master/reference/runtim… Thanks, Corey

1 0

[nova] consoleauth token ttl
by William Lima Reiznautt 05 Jul '19

05 Jul '19

Hello folks, Some known about this configuration token_ttl on [consoleauth] section: Is it on second or minutes ? At documentation is not information. Sorry the question here. -- -- William Lima Reiznautt IC/UNICAMP - Telefone: +55 (19) 3521-5915 --- chavepgp: http://www.ic.unicamp.br/~william/william.pgp

2 1

[nova] type-PCI vs type-PF vs type-VF
by Mauricio Tavares 05 Jul '19

05 Jul '19

Quick questions: 1. Is there some official documentation comparing them? Only thing I found so far was https://mohamede.com/2019/02/07/pci-passthrough-type-pf-type-vf-and-type-pc… 2. Am I correct to assume (I seem to do a lot of that; be afraid) that when nova starts populating the database entry for a given physical node (say a computer node or a ironic baremetal node) it looks through the pci list and decides what type of device (type-PCI, type-PF, or type-VF) it is? What is the criteria? 3. https://mohamede.com/2019/02/07/pci-passthrough-type-pf-type-vf-and-type-pc… implies that nova sees the 3 different types, well, differently where it will try hard to virtualize type-PF and type-VF and really really wants me to setup sr-iov on them ( https://docs.openstack.org/nova/latest/admin/pci-passthrough.html seem to also want sr-iov) really . Is that correct?

2 1

[placement] db query analysis
by Chris Dent 05 Jul '19

05 Jul '19

One of the queries that has come up recently with placement performance is whether there may be opportunities to gain some improvement by making fewer queries to the db. That is: * are there redundant queries * are there places where data is gathered in multiple queries that could be in one (or at least fewer) * are there queries that are not doing what we think to that end I've done some analysis of logs produced when [placement_database]/connection_debug is set to 50 (which dumps SQL queries to the INFO log). The collection of queries made during a single request to GET /allocation_candidates is at http://paste.openstack.org/show/753183/ The data set is a single resource resource provider of the same form as used in the placement-perfload job (where 1000 providers are used). Only 1 is used in this case as several queries use 'IN' statements that list all the resource provider ids currently in play, and that gets dumped to the log making it inscrutable. I've noted in the paste where this happens. Each block of SQL is associated with the method that calls it. The queries are in the order they happen. One query that happens three times (once for each resource class requested) is listed once. Observations: * The way we use IN could be improved using a bindparam: https://docs.sqlalchemy.org/en/13/core/sqlelement.html?highlight=expanding%… * That we use IN in that fashion at all, where we are carrying lists of rp ids around and making multiple queries, instead of one giant one, might be an area worth exploring. * There are a couple of places where get get a trait id (via name) in a separate query from using the trait id. * What can you see? Please have a look to see if anything looks odd, wrong, etc. Basically what we're after is trying to find things that violate our expectations. Note that this is just one of several paths through the database. When there are sharing or nested providers things change. I didn't bother to do a more complex set of queries at this time as it seemed starting simple would help us tease out how best to communicate these sorts of things. Related to that, I've started working on a nested-perfload at https://review.opendev.org/665695 -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

2 2

[placement] update 19-26
by Chris Dent 05 Jul '19

05 Jul '19

HTML: https://anticdent.org/placement-update-19-26.html Pupdate 19-26. Next week is R-15, two weeks until Train milestone 2. # Most Important The [spec for nested magic](https://docs.openstack.org/placement/latest/specs/train/approved/200… merged and significant progress has been made in the implementation. That work is nearly ready to merge (see below), after a few more reviews. Once that happens one of our most important tasks will be experimenting with that code to make sure it fully addresses the uses cases, has proper documentation (including "how do I use this?"), and is properly evaluated for performance and maintainability. # What's Changed * The implementation for mappings in allocation candidates had a bug which Eric [found](https://review.opendev.org/668302) and fixed and then I realized there was a [tidier way to do it](https://review.opendev.org/668724). This then led to the `same_subtree` work needing to manage less information, because it was already there. * The spec for [Consumer Types](https://docs.openstack.org/placement/latest/specs/train/approved/200… merged and work [has started](https://review.opendev.org/669170). * We're using os-traits 0.15.0 now. * There's a [framework in place](https://review.opendev.org/665695) for nested resource provider peformance testing. We need to update the provider topology to reflect real world situations (more on that below). * The `root_required` query parameter on `GET /allocation_candidates` has been merged as [microversion 1.35](https://docs.openstack.org/placement/latest/placement-api-microversio…. * I've sent [an email](http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007… announcing my intent to not go to the Shangai (or any other) summit, and what changes that could imply for how Placement does the PTG. # Specs/Features All placement specs have merged. Thanks to everyone for the frequent reviews and quick followups. We've been maintaining some good velocity. Some non-placement specs are listed in the Other section below. # Stories/Bugs (Numbers in () are the change since the last pupdate.) There are 23 (3) stories in [the placement group](https://storyboard.openstack.org/#!/project_group/placement). 0 (0) are [untagged](https://storyboard.openstack.org/#!/worklist/580). 2 (-2) are [bugs](https://storyboard.openstack.org/#!/worklist/574). 5 (0) are [cleanups](https://storyboard.openstack.org/#!/worklist/575). 11 (0) are [rfes](https://storyboard.openstack.org/#!/worklist/594). 4 (1) are [docs](https://storyboard.openstack.org/#!/worklist/637). If you're interested in helping out with placement, those stories are good places to look. * Placement related nova [bugs not yet in progress](https://goo.gl/TgiPXb) on launchpad: 16 (0). * Placement related nova [in progress bugs](https://goo.gl/vzGGDQ) on launchpad: 4 (-1). # osc-placement osc-placement is currently behind by 11 microversions. * <https://review.opendev.org/666542> Add support for multiple member_of. # Main Themes ## Nested Magic These are the features required by modern nested resource provider use cases. We've merged mappings in allocation candidates and `root_required`. `same_subtree` and resourceless request groups are what's left and they are in: * <https://review.opendev.org/668376> Support `same_subtree` queryparam ## Consumer Types Adding a type to consumers will allow them to be grouped for various purposes, including quota accounting. * <https://review.opendev.org/#/q/topic:bp/support-consumer-types> ## Cleanup Cleanup is an overarching theme related to improving documentation, performance and the maintainability of the code. The changes we are making this cycle are fairly complex to use and are fairly complex to write, so it is good that we're going to have plenty of time to clean and clarify all these things. As mentioned above, one of the important cleanup tasks that is not yet in progress is updating the [gabbit](https://opendev.org/openstack/placement/src/branch/master/gate/gabb… that creates the nested topology that's used in nested performance testing. The topology there is simple, unrealistic, and doesn't sufficiently exercise the several features that may be used during a query that desires a nested response. Recently I've been seeing that the `placement-perfload` job is giving results that vary between `N` and `N*2` (usually .5 and 1 seconds) and the difference that I can discern is the type of CPUs being presented by the host (same number of CPUs (8) but different type). This supports something we've been theorizing for a while: when dealing with large result sets we are CPU bound processing the several large result sets returned by the database. Further profiling required… Another cleanup that needs to start is satisfying the community wide goal of [PDF doc generation](https://storyboard.openstack.org/#!/story/2006110). # Other Placement Miscellaneous changes can be found in [the usual place](https://review.opendev.org/#/q/project:openstack/placement+status:op…. There are three [os-traits changes](https://review.opendev.org/#/q/project:openstack/os-traits+status:… being discussed. And one [os-resource-classes change](https://review.opendev.org/#/q/project:openstack/os-resource-classe…. # Other Service Users New discoveries are added to the end. Merged stuff is removed. Anything that has had no activity in 4 weeks has been removed. * <https://review.openstack.org/#/q/topic:bug/1819923> Nova: nova-manage: heal port allocations * <https://review.openstack.org/650188> nova-spec: Allow compute nodes to use DISK_GB from shared storage RP * <https://review.opendev.org/659233> Cyborg: Placement report * <https://review.opendev.org/662229> helm: WIP: add placement chart * <https://review.opendev.org/656023> Nova: Use OpenStack SDK for placement * <https://review.opendev.org/612497> Nova: Spec: Provider config YAML file * <https://review.opendev.org/634551> libvirt: report pmem namespaces resources by provider tree * <https://review.opendev.org/660852> Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI * <https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports> Nova: support move ops with qos ports * <https://review.opendev.org/664689> Nova: get_ksa_adapter: nix by-service-type confgrp hack * <https://review.opendev.org/664862> OSA: Add nova placement to placement migration * <https://review.opendev.org/657796> Nova: Defaults missing group_policy to 'none' * <https://review.opendev.org/666202> Blazar: Create placement client for each request * <https://review.opendev.org/668930> tempest: Define the Integrated-gate-networking gate template * <https://review.opendev.org/669309> tempest: Define the Integrated-gate-placement gate template * <https://review.opendev.org/668263> Nova: Restore RT.old_resources if ComputeNode.save() fails * <https://review.opendev.org/669188> Remove assumption of http error if consumer not exists * <https://review.opendev.org/669253> TripleO: Add new parameter NovaSchedulerLimitTenantsToPlacementAggregate * <https://review.opendev.org/669252> puppet-nova: Expose limit_tenants_to_placement_aggregate parameter * <https://review.opendev.org/667952> nova: Support filtering of hosts by forbidden aggregates * <https://review.opendev.org/669079> blazar: Send global_request_id for tracing calls * <https://review.opendev.org/667417> nova: Implement update_provider_tree for hyperv * <https://review.opendev.org/668639> watcher: Improve Compute Data Model * <https://review.opendev.org/#/q/topic:bp/pre-filter-disabled-computes> Nova: pre filter disable computes * <https://review.opendev.org/668252> Nova: Update HostState.\*\_allocation_ratio earlier # End This space left intentionally blank. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

1 0

[Mistral] Guaranteed notification delivery
by Лапин Артем 05 Jul '19

05 Jul '19

Good day, please look at https://blueprints.launchpad.net/mistral/+spec/guaranteed-notifies. I would like to know your opinion about this feature.

1 0

[horizon] PTL on vacation
by Ivan Kolodyazhny 05 Jul '19

05 Jul '19

Hi team, I'll be on a vacation until July, 19th with a very limited internet connection. During the last weekly meeting [1] we agreed to skip the next two. I'll try to check my mail during the PTO but for any issues, I recommend to ask our great core team [2] via emails or IRC. [1] http://eavesdrop.openstack.org/meetings/horizon/2019/horizon.2019-07-03-15.… [2] https://review.opendev.org/#/admin/groups/43,members Regards, Ivan Kolodyazhny, http://blog.e0ne.info/

1 0