May 2019 - openstack-discuss - lists.openstack.org

[nova] dropping 2013.1 tag on pypi
by Mohammed Naser 31 May '19

31 May '19

Hi everyone, I've recently had some folks reach out who were quite confused by the fact that searching for 'nova' on pypi shows a 2013 release, and then hitting the correct path: https://pypi.org/project/nova/19.0.0/ shows that there is a new version available. I know usually deleting releases is a bit of a 'don't do this' thing, but in this case, that's a 6 year old unsupported/never-to-be-used again release and I think to make life easier for consumers, we should maybe drop it. Thoughts? Regards, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

4 3

[ironic][edge]: Recap of PTG discussions
by Csatari, Gergely (Nokia - HU/Budapest) 31 May '19

31 May '19

Hi, There was a one hour discussion with Julia from Ironic with the Edge Computing Group [1]. In this mail I try to conclude what was discussed and ask some clarification questions. Current Ironic uses DHCP for hardware provisioning, therefore it requires DHCP relay enabled on the whole path to the edge cloud instances. There are two alternatives to solve this: 1) Virtual media support [2] where the ip configuration is embedded into a virtual image what is booted via the board management interface 2) Redfish support, however the state and support of redfish for host management is not clear. Is there already a specification has been added for redfish support? Upgrade of edge cloud infrastructures: - Firmware upgrade should be supported by Ironic. Is this something on its way or is this a new need? - Operating system and infra update can be solved using Fenix [3], however handling several edge cloud instances from a central location needs new features. Handling of failed servers: - A monitoring system or the operator should provide the input to mark a server as failed - Ironic can power down the failed servers and have the definition of a maintenance state - Discussed in [4] Additional ideas what we half discussed: - Running Ironic containers in a switch with the images hosted by Swift somewhere else. Are there any concerns about this idea? Any missing features from somewhere? [1]: https://etherpad.openstack.org/p/edge-wg-ptg-preparation-denver-2019 [2]: https://specs.openstack.org/openstack/ironic-specs/specs/not-implemented/L3… [3]: https://wiki.openstack.org/wiki/Fenix [4]: http://lists.openstack.org/pipermail/edge-computing/2019-May/000582.html Br, Gerg0

4 4

[placement] update 19-21
by Chris Dent 31 May '19

31 May '19

HTML: https://anticdent.org/placement-update-19-21.html And here we have placement update 19-21. # Most Important The [spec for nested magic](https://review.opendev.org/658510) has been split. The [second half](https://review.opendev.org/662191) contains the parts which should be relatively straightforward. The original retains functionality that may be too complex. An email has been [addressed to operators](http://lists.openstack.org/pipermail/openstack-discuss/2019-May/… asking for feedback. Those two specs represent a significant portion of the work planned this cycle. Getting them reviewed and merged is a good thing to do. # What's Changed * A few small refactorings plus the removal of [null provider protections](https://review.opendev.org/657716) has improved performance when retrieving 10,000 allocation candidates significantly: from around 36 seconds to 6 seconds. * We've chosen to switch to office hours. Ed has started an [email thread](http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006… to determine when they should be. * Tetsuro's changes to add a RequestGroupSearchContext have merged. These simplify state management throughout the processing of individual request groups, and help avoid redundant queries. * Most of the code for [counting (nova) quota usage from placement](https://review.opendev.org/#/q/topic:bp/count-quota-usage-from-p… has merged. * [Microversion 1.33](https://docs.openstack.org/placement/latest/placement-api-microversio… of the placement API has merged. This allows more expressive suffixes on granular request groups (e.g., `resources_COMPUTE` in addition to `resources1`). # Specs/Features * <https://review.opendev.org/654799> Support Consumer Types. This is very close with a few details to work out on what we're willing and able to query on. It's two weeks later and still only has reviews from me. * <https://review.opendev.org/658510> Spec for can_split part of nested magic. We're unlikely to do this at this point, unless response to the [email asking for input](http://lists.openstack.org/pipermail/openstack-discuss/2019-May/0067… is large and insisting. * <https://review.opendev.org/662191> Spec for nested magic 1. The easier parts of nested magic: same_subtree, resource request groups, verbose suffixes (already merged as 1.33). * <https://review.opendev.org/657582> Resource provider - request group mapping in allocation candidate. There's a [WIP](https://review.opendev.org/#/c/662245/) that looks promising, but we ought to make sure the spec is presenting the desired outcome. These and other features being considered can be found on the [feature worklist](https://storyboard.openstack.org/#!/worklist/594). Some non-placement specs are listed in the Other section below. Note that nova will be having a spec-review-sprint this coming Tuesday. if you're doing that, spending a bit of time on the placement specs would be great too. # Stories/Bugs (Numbers in () are the change since the last pupdate.) There are 19 (-1) stories in [the placement group](https://storyboard.openstack.org/#!/project_group/placement). 0 (0) are [untagged](https://storyboard.openstack.org/#!/worklist/580). 2 (0) are [bugs](https://storyboard.openstack.org/#!/worklist/574). 4 (-1) are [cleanups](https://storyboard.openstack.org/#!/worklist/575). 11 (0) are [rfes](https://storyboard.openstack.org/#!/worklist/594). 2 (0) are [docs](https://storyboard.openstack.org/#!/worklist/637). If you're interested in helping out with placement, those stories are good places to look. * Placement related nova [bugs not yet in progress](https://goo.gl/TgiPXb) on launchpad: 16 (0). * Placement related nova [in progress bugs](https://goo.gl/vzGGDQ) on launchpad: 7 (0). # osc-placement osc-placement is currently behind by 12 microversions. No change since the last report. _Note_: Based on conversations that we see on reviews, explicitly trying to chase microversions when patching the plugin may not be aligned with the point of OSC. We're trying to make a humane interface to getting stuff done with placement. Different microversions allow different takes on stuff. It's the stuff that matters, not the microversion. Pending Changes: * <https://review.openstack.org/#/c/640898/> Add 'resource provider inventory update' command (that helps with aggregate allocation ratios). * <https://review.openstack.org/#/c/651783/> Add support for 1.22 microversion. (So, for example, what matters here is that support for forbidden traits is being added. That it is in microversion 1.22 ought to be incidental (to the user of the client).) * <https://review.openstack.org/586056> Provide a useful message in the case of 500-error # Main Themes ## Nested Magic The overview of the features encapsulated by the term "nested magic" are in a [story](https://storyboard.openstack.org/#!/story/2005575). There is some in progress code, some of it WIPs to expose issues: * <https://review.opendev.org/657510> WIP: Allow RequestGroups without resources * <https://review.opendev.org/657463> Add NUMANetworkFixture for gabbits * <https://review.opendev.org/658192> Gabbi test cases for can_split * <https://review.opendev.org/662245> WIP: Implement allocation candidate mappings ## Consumer Types Adding a type to consumers will allow them to be grouped for various purposes, including quota accounting. A [spec](https://review.opendev.org/654799) has started. There are some questions about request and response details that need to be resolved, but the overall concept is sound. ## Cleanup We continue to do cleanup work to lay in reasonable foundations for the nested work above. As a nice bonus, we keep eking out additional performance gains too. * <https://review.opendev.org/643269> Optionally run a wsgi profiler when asked. This has proven useful to find issues. The change has been unwipped and augumented with some documentation. * <https://review.opendev.org/661769> Add olso.middleware.cors to conf generator * <https://review.opendev.org/661922> Modernize CORS config and setup. Ed Leafe has also been doing some intriguing work on using graph databases with placement. It's not yet clear if or how it could be integrated with mainline placement, but there are likely many things to be learned from the experiment. # Other Placement Miscellaneous changes can be found in [the usual place](https://review.opendev.org/#/q/project:openstack/placement+status:op…. There are several [os-traits changes](https://review.opendev.org/#/q/project:openstack/os-traits+status:… being discussed. # Other Service Users New discoveries are added to the end. Merged stuff is removed. As announced last week, anything that has had no activity in 4 weeks has been removed (many have been removed). * <https://review.openstack.org/601596> Nova: spec: support virtual persistent memory * <https://review.openstack.org/645316> Nova: Pre-filter hosts based on multiattach volume support * <https://review.openstack.org/647396> Nova: Add flavor to requested_resources in RequestSpec * <https://review.openstack.org/633204> Blazar: Retry on inventory update conflict * <https://review.openstack.org/#/q/topic:bp/count-quota-usage-from-placement> Nova: count quota usage from placement * <https://review.openstack.org/#/q/topic:bug/1819923> Nova: nova-manage: heal port allocations * <https://review.openstack.org/629142> Tempest: Add QoS policies and minimum bandwidth rule client * <https://review.openstack.org/650188> nova-spec: Allow compute nodes to use DISK_GB from shared storage RP * <https://review.opendev.org/659233> Cyborg: Placement report * <https://review.opendev.org/657884> Nova: Spec to pre-filter disabled computes with placement * <https://review.opendev.org/657801> rpm-packaging: placement service * <https://review.opendev.org/657016> Delete resource providers for all nodes when deleting compute service * <https://review.opendev.org/654066> nova fix for: Drop source node allocations if finish_resize fails * <https://review.opendev.org/656885> nova: WIP: Hey let's support routed networks y'all! * <https://review.opendev.org/662281> Fixes to keystoneauth to deal with the style of error responses that placement uses. * <https://review.opendev.org/662371> starlingx: Add placement chart patch to openstack-helm * <https://review.opendev.org/662229> helm: WIP: add placement chart * <https://review.opendev.org/661679> starlingx: Add stx-placement docker image directives files * <https://review.opendev.org/662164> kolla-ansible: Add a explanatory note for "placement_api_port" * <https://review.opendev.org/658451> neutron-spec: L3 agent capacity and scheduling # End That's a lot of reviewing. Please help out where you can. Your reward will be brief but sincere moments of joy. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

1 0

Integration of osprofiler in rally results page
by Slawomir Kaplonski 31 May '19

31 May '19

Hi, Recently thx to Andrey Kurilin and Ilya Shakhat we integrated in neutron-rally-task job results from osprofiler that profiler results are available in “Scenario data” tab for every iteration of every scenario. Example of how it looks is available at [1]. Patch with changes required to enable it in rally job is in [2]. It doesn’t require many changes and job isn’t run much longer so maybe it will be useful for other projects also :) [1] http://logs.openstack.org/50/615350/44/check/neutron-rally-task/e453164/res… [2] https://review.opendev.org/#/c/615350/ — Slawek Kaplonski Senior software engineer Red Hat

2 1

[openstack-helm] external ceph
by Robson Ramos Barreto 31 May '19

31 May '19

Hi All, I'd like to know how can I integrate openstack deployment with my existing ceph cluster using openstack helm to be used for both k8s dynamic provision (PVC) and as storage backend to cinder, glance and nova. I'm trying deploying openstack with ceph[1] to understanding the needs but it's a little painful I'd appreciate any help Thank you [1] https://docs.openstack.org/openstack-helm/latest/install/developer/deploy-w…

2 1

[ops][nova][placement] NUMA topology vs non-NUMA workloads
by Chris Dent 31 May '19

31 May '19

This message is primarily addressed at operators, and of those, operators who are interested in effectively managing and mixing workloads that care about NUMA with workloads that do not. There are some questions within, after some background to explain the issue. At the PTG, Nova and Placement developers made a commitment to more effectively manage NUMA topologies within Nova and Placement. On the placement side this resulted in a spec which proposed several features that would enable more expressive queries when requesting allocation candidates (places for workloads to go), resulting in fewer late scheduling failures. At first there was one spec that discussed all the features. This morning it was split in two because one of the features is proving hard to resolve. Those two specs can be found at: * https://review.opendev.org/658510 (has all the original discussion) * https://review.opendev.org/662191 (the less contentious features split out) After much discussion, we would prefer to not do the feature discussed in 658510. Called 'can_split', it would allow specified classes of resource (notably VCPU and memory) to be split across multiple numa nodes when each node can only contribute a portion of the required resources and where those resources are modelled as inventory on the NUMA nodes, not the host at large. While this is a good idea in principle it turns out (see the spec) to cause many issues that require changes throughout the ecosystem, for example enforcing pinned cpus for workloads that would normally float. It's possible to make the changes, but it would require additional contributors to join the effort, both in terms of writing the code and understanding the many issues. So the questions: * How important, in your cloud, is it to co-locate guests needing a NUMA topology with guests that do not? A review of documentation (upstream and vendor) shows differing levels of recommendation on this, but in many cases the recommendation is to not do it. * If your answer to the above is "we must be able to do that": How important is it that your cloud be able to pack workloads as tight as possible? That is: If there are two NUMA nodes and each has 2 VCPU free, should a 4 VCPU demanding non-NUMA workload be able to land there? Or would you prefer that not happen? * If the answer to the first question is "we can get by without that" is it satisfactory to be able to configure some hosts as NUMA aware and others as not, as described in the "NUMA topology with RPs" spec [1]? In this set up some non-NUMA workloads could end up on a NUMA host (unless otherwise excluded by traits or aggregates), but only when there was contiguous resource available. This latter question articulates the current plan unless responses to this message indicate it simply can't work or legions of assistance shows up. Note that even if we don't do can_split, we'll still be enabling significant progress with the other features described in the second spec [2]. Thanks for your help in moving us in the right direction. [1] https://review.opendev.org/552924 [2] https://review.opendev.org/662191 -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

3 2

[keystone][placement][neutron][api-sig] http404 to NotFound, or how should a http json error body look like?
by Lajos Katona 31 May '19

31 May '19

Hi, Recently I planned to add tests to the neutron feature routed provider networks, which actually uses placement API to make possible the scheduling based on neutron segment information. I realized that the feature last worked on Queens. We can discuss separately the lack of test coverage, and similar things, but I would like to narrow or view now. The neutron bug report, with some background information: https://bugs.launchpad.net/neutron/+bug/1828543 After some debugging I realized that the problem comes from the fact that at a point neutron uses the information received in http404, like this: try: return self._get(url).json() except keystoneauth1.exceptions.NotFound as e: if 'foo' in e.details: do_something() see: https://opendev.org/openstack/neutron-lib/src/branch/master/neutron_lib/pla… keystoneauth1 expects the http body in case of for example http404 to be something like this (python dict comes now): body= {'error': {'message': 'Foooooo', 'details': 'Baaaaar'}} see: https://opendev.org/openstack/keystoneauth/src/branch/master/keystoneauth1/… But placement started to adopt one suggestion from the API-SIG: http://specs.openstack.org/openstack/api-wg/guidelines/errors.html, and that is something like this (again python): body={'errors': [{'status': 404, 'title': 'Not Found', 'detail': 'The resource could not be found.... ', 'request_id': '...'}]} Shall I ask for help how I should make this bug in neutron fixed? As a quick and dirty solution in neutron I can use the response from the exception, but I feel that there should be a better way to fix this:-) Thanks in advance for the help Regards LAjos

5 7

[Glance] Removal of glance-replicator
by Erno Kuvaja 30 May '19

30 May '19

Hi all, The replicator has been purely depending of the Images API v1 and thus not operational at all for good couple of cycles now. As a part of the v1 cleanup we are going to remove the replicator command as well. If the replicator is needed (currently doesn't seem to as no-one even noticed that it deasn't work since we removed v1 api), lets not port the original just to v2 but open a spec for it and take the opportunity to come up with robust solution that will serve us well into the future. Thanks, Erno "jokke" Kuvaja

2 1

[nova] libvirt+virtuozzo CI is broken
by Matt Riedemann 30 May '19

30 May '19

I noticed this today: http://openstack-3rd-party-virtuozzo-ci-logs.virtuozzo.com/24/638324/30/che… 2019-05-24 02:16:51.833 | + :pre_test_hook:L13: git fetch https://review.opendev.org/p/openstack/nova refs/changes/86/506686/4 2019-05-24 02:17:01.100 | From https://review.opendev.org/p/openstack/nova 2019-05-24 02:17:01.102 | * branch refs/changes/86/506686/4 -> FETCH_HEAD 2019-05-24 02:17:01.109 | + :pre_test_hook:L14: git cherry-pick FETCH_HEAD 2019-05-24 02:17:01.311 | error: could not apply 789a1f2... don't add device address if there is no any units 2019-05-24 02:17:01.313 | hint: after resolving the conflicts, mark the corrected paths 2019-05-24 02:17:01.315 | hint: with 'git add <paths>' or 'git rm <paths>' 2019-05-24 02:17:01.318 | hint: and commit the result with 'git commit' Looks like the CI is broken because this change is in merge conflict: https://review.opendev.org/#/c/506686/ Does anyone plan on working on that? It's had a -1 on it since August of 2018. Is that change really needed for the CI to work? -- Thanks, Matt

3 3

[nova] Quality warning patch for xenapi driver
by Matt Riedemann 30 May '19

30 May '19

As discussed during today's nova meeting [1] I've posted a change [2] to the xenapi driver such that it logs a warning on startup saying its not tested (no 3rd party CI) and as such may be subject to deprecation in the future. This is kind of the first step in formally deprecating the driver. If you have a vested interest in maintaining the xenapi driver please speak up here and/or on the change. [1] http://eavesdrop.openstack.org/meetings/nova/2019/nova.2019-05-30-14.00.log… [2] https://review.opendev.org/#/c/662295/ -- Thanks, Matt

1 0