March 2019 - openstack-discuss - lists.openstack.org

[kuryr] PTG and Forum planning etherpad
by Daniel Mellado 02 Mar '19

02 Mar '19

Hi Kuryrs, As the time for the PTG and the Forum is coming and my paternity leave is about to end as well (sleepless nights have just started xD) I set up an etherpad for PTG planning [1]. Please add topics with your nick or email alongside. As a reminder we also have a Forum topics brainstorming etherpad [2]. The deadline for submitting topics is next week so don't delay! -- Daniel Mellado

1 1

[security][barbican][manila] hiding back end and service user credentials
by Tom Barron 02 Mar '19

02 Mar '19

In manila -- and so far as I can tell, other projects -- service user and back end (storage devices, security service) credentials appear plaintext in configuration files and in database tables. These are not accessible to ordinary OpenStack users but some cloud deployers nonetheless have concerns about this exposure and have asked us to tighten things up. So I want to check for best practices from other projects. I doubt this is a manila-specific concern -- e.g. is barbican already being used today by some projects to protect information of this sort? Thanks, -- Tom Barron

3 4

[sahara][heat][qa][murano] Heat integration tests run by Grenade
by Luigi Toscano 01 Mar '19

01 Mar '19

Hi, Heat is a requirement for Sahara, which means that the Heat plugin for DevStack and Grenade are enabled in the is enabled in the Sahara DevStack and Grenade jobs respectively. I noticed that the Grenade plugin for Heat unconditionally runs few tests both after the deployment of the base environment and after the upgrade phase. While I understand that the Heat team want to always run them when Heat is tested, I would argue that whenever Heat is deployed as requirement by other projects (for example, Sahara :) it should be possible to disable those tests. This way the execution time of the Grenade job would be shortened by ~20 minutes, from 1h30m to 1h10m. This change should benefit also the other projects which depend on Heat. Howoever, even if the change was implemented now, its effects would not be noticeable immediately, because it seems to be just one additional project which both a) depends on Heat and b) provides a Grenade plugin. That project is Murano, which, on the other hand, it does not enable the Heat plugin for Grenade in its Grenade job. I think that this should be fixed, otherwise (IMHO) the upgrade test is not really complete IMHO, but that's a different story (hence the [murano] tag in this email). So my question for the Heat team is: can I propose an Heat patch which allows users to disable the Heat integration tests executed by Grenade? The default behavior will be unchanged. Ciao -- Luigi

2 2

[ops] berlin ops meetup content thread
by Chris Morgan 01 Mar '19

01 Mar '19

Hello Everyone, Here's the first etherpad for next week's Ops Meetup in Berlin, hosted by Deutsche Telekom: https://etherpad.openstack.org/p/BER19-OPS-DUDES This one is for a proposed group discussion amongst the openstack operators attending about all the new (or newly top-level) projects under the openstack foundation umbrella, or as I've nicknamed it "All the young dudes", kata,zuul, airship, starlingx. I also added a section about the changes to the foundation, stackforge, the summits etc.I haven't seen a prior effort to get feedback from openstack operators on all these changes. Feedback welcome Chris p.s. still tickets available, come join us if you can - especially if it's nearby to you there's still time! https://www.eventbrite.com/e/openstack-ops-meetup-berlin-tickets-55034908894 -- Chris Morgan <mihalis68(a)gmail.com>

1 2

[loci] Cleaning fetch_wheels.py
by Jean-Philippe Evrard 01 Mar '19

01 Mar '19

Hello everyone, scripts/fetch_wheels.py could deserve more love, as it's not really DRY, and to be honest, I still wonder the value of some of it. For example, I've seen multiple failures in the protocol_detection method. And, in the face of ambiguity, I would prefer to avoid the temptation to guess :) We've discussed with Chris about the removal at our weekly meeting, and he was not opposed to it. Would you be okay if I remove this protocol_detection method, instead relying on user provided information in the `REGISTRY_PROTOCOL` argument? (Which would default to http). If no answer within a week, I would consider that nobody cares or everyone agrees, and therefore I will go on with the cleanup :p Regards, Jean-Philippe Evrard (evrardjp)

1 0

[dev][keystone] Keystone Team Update - Week of 25 February 2019
by Colleen Murphy 01 Mar '19

01 Mar '19

# Keystone Team Update - Week of 25 February 2019 ## News ### Releases and Feature Freeze Yesterday we released keystonemiddleware, keystoneauth, and python-keystoneclient, and then the release team was kind enough to let us release keystoneauth again today. Next week will be the feature freeze for keystone, so we should aim for having all of the domain limits changes[1] and ideally the default roles and system scope changes[2] merged by then. We'll hold off on application credentials access rules (capabilities) changes[3] until the next cycle opens for development. [1] https://review.openstack.org/#/q/status:open+topic:bp/domain-level-limit [2] https://review.openstack.org/#/q/status:open+topic:implement-default-roles [3] https://review.openstack.org/#/q/status:open+topic:bp/whitelist-extension-f… ### Forum/PTG prep As a reminder, we're brainstorming for Forum and PTG topics[4]. If you are prepared to moderate a Forum topic don't hesitate to speak up and mark it on etherpad. [4] https://etherpad.openstack.org/p/DEN-keystone-forum-sessions ### Rate-limiting in keystoneauth Lance and Morgan heroically walked through[5], tested and landed Monty's threaded rate limiting feature in keystoneauth[6]. They found the interface for it to be slightly but unavoidably unintuitive, but docs will be forthcoming. We released keystoneauth again to include this feature. [5] http://eavesdrop.openstack.org/irclogs/%23openstack-keystone/%23openstack-k… [6] https://review.openstack.org/605043 ### Domain admin in tempest There are issues with the assumptions tempest makes the scope requirements for a deployment administrator role that prevent us from landing one of the reader role changes[7][8]. A workaround is proposed[9] that will help us move forward, but we need to unwind the tempest/devstack implementation with the QA team to ensure that tempest can understand and validate scope the way we intend for it to work. It's also an opportunity to consider what interesting workarounds operators may have come up with for managing the admin role so that we can come up with migration plans. [7] http://eavesdrop.openstack.org/irclogs/%23openstack-keystone/%23openstack-k… [8] http://eavesdrop.openstack.org/irclogs/%23openstack-qa/%23openstack-qa.2019… [9] https://review.openstack.org/624218 ## Recently Merged Changes Search query: https://bit.ly/2pquOwT We merged 30 changes this week. ## Changes that need Attention Search query: https://bit.ly/2tymTje There are 46 changes that are passing CI, not in merge conflict, have no negative reviews and aren't proposed by bots. ## Bugs This week we opened 9 new bugs and closed 14. Bugs opened (9) Bug #1816833 (keystone:High) opened by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1816833 Bug #1817313 (keystone:High) opened by Colleen Murphy https://bugs.launchpad.net/keystone/+bug/1817313 Bug #1816927 (keystone:Low) opened by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1816927 Bug #1817047 (keystone:Low) opened by André Luis Penteado https://bugs.launchpad.net/keystone/+bug/1817047 Bug #1817780 (keystone:Low) opened by Patrick Lepak https://bugs.launchpad.net/keystone/+bug/1817780 Bug #1818085 (keystone:Wishlist) opened by Colleen Murphy https://bugs.launchpad.net/keystone/+bug/1818085 Bug #1817623 (keystone:Undecided) opened by Rastislav Fukas https://bugs.launchpad.net/keystone/+bug/1817623 Bug #1817769 (keystone:Undecided) opened by prashkre https://bugs.launchpad.net/keystone/+bug/1817769 Bug #1817953 (oslo.policy:Low) opened by Ben Nemec https://bugs.launchpad.net/oslo.policy/+bug/1817953 Bugs closed (1) Bug #1817623 (keystone:Undecided) https://bugs.launchpad.net/keystone/+bug/1817623 Bugs fixed (13) Bug #1815539 (keystone:High) fixed by Guang Yee https://bugs.launchpad.net/keystone/+bug/1815539 Bug #1757000 (keystone:Medium) fixed by erus https://bugs.launchpad.net/keystone/+bug/1757000 Bug #1804292 (keystone:Medium) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1804292 Bug #1804463 (keystone:Medium) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1804463 Bug #1804516 (keystone:Medium) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1804516 Bug #1804517 (keystone:Medium) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1804517 Bug #1804519 (keystone:Medium) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1804519 Bug #1804521 (keystone:Medium) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1804521 Bug #1804523 (keystone:Medium) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1804523 Bug #1805369 (keystone:Medium) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1805369 Bug #1805402 (keystone:Medium) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1805402 Bug #1806713 (keystone:Medium) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1806713 Bug #1817953 (oslo.policy:Low) fixed by Ben Nemec https://bugs.launchpad.net/oslo.policy/+bug/1817953 ## Milestone Outlook https://releases.openstack.org/stein/schedule.html Yesterday was the non-client library release deadline. Next week will be the final client release deadline as well as feature freeze. After that we'll being prepping the RC. ## Help with this newsletter Help contribute to this newsletter by editing the etherpad: https://etherpad.openstack.org/p/keystone-team-newsletter

1 0

[manila] PTG planning etherpad available and Forum topics reminder
by Tom Barron 01 Mar '19

01 Mar '19

Dear Zorillas, I set up an etherpad for PTG planning [1]. Please add topics with your nick or email alongside. As a reminder we also have a Forum topics brainstorming etherpad [2]. The deadline for submitting topics is next week so don't delay! -- Tom Barron [1] https://etherpad.openstack.org/p/manila-denver-train-ptg-planning [2] https://etherpad.openstack.org/p/DEN-train-forum-manila-brainstorming

1 0

[nova] Updates about Detaching/Attaching root volumes
by Zhenyu Zheng 01 Mar '19

01 Mar '19

Hi Nova, I'm working on a blueprint to support Detach/Attach root volumes. The blueprint has been proposed for quite a while since mitaka[1] in that version of proposal, we only talked about instances in shelved_offloaded status. And in Stein[2] the status of stopped was also added. But now we realized that support detach/attach root volume on a stopped instance could be problemastic since the underlying image could change which might invalidate the current host.[3] So Matt and Sean suggested maybe we could just do it for shelved_offloaded instances, and I have updated the patch according to this comment. And I will update the spec latter, so if anyone have thought on this, please let me know. Another thing I wanted to discuss is that in the proposal, we will reset some fields in the root_bdm instead of delete the whole record, among those fields, the tag field could be tricky. My idea was to reset it too. But there also could be cases that the users might think that it would not change[4]. Thoughts, BR, [1] http://specs.openstack.org/openstack/nova-specs/specs/mitaka/approved/detac… [2] http://specs.openstack.org/openstack/nova-specs/specs/stein/approved/detach… [3] https://review.openstack.org/#/c/614750/34/nova/compute/manager.py@5467 [4] https://review.openstack.org/#/c/614750/37/nova/objects/block_device.py

5 18

[placement] update 19-08
by Chris Dent 01 Mar '19

01 Mar '19

HTML: https://anticdent.org/placement-update-19-08.html Welcome back to the placement update. If I've read the signs correctly, I should now be back to this as a regular thing. Apologies for the gap, I had to attend to some other responsibilities. # Most Important A lot has changed in the past few months, so it's hard to extract out a most important. It will depend on who is reading. Review what's changed for a summary of important stuff. # What's Changed * Placement is now its own official project. Until elections are held (it looks like nominations start this coming Tuesday), Mel is the PTL. * [Setting up storyboard](https://review.openstack.org/#/c/639445/) for placement-related projects is in progress. For the time being we are continuing to use launchpad for most tracking. See a [related email thread](http://lists.openstack.org/pipermail/openstack-discuss/2019-Februar…. * Deleting placement code from nova has been put on hold until Train to make it easier for certain types of upgrades to happen. New installs should prefer the extracted code, as the nova-side is frozen, but the placement side is not. * A large stack of code to remove oslo.versionedobjects from placement has merged. This has resulted in a significant change in performance on the `perfload` test that runs in the gate. While not a complete representation of the entire system, it's enough to say "yeah, that was worth it": A request for allocation candidates that used to take around 2.5 seconds now takes 1.2. That refactoring continues (see below), seeking additional simplifications. * Microversion 1.31 adds `in_tree` and `in_treeN` query parameters to GET /allocation_candidates. This is useful in a variety of nested resource provider scenarios, including the big bandwidth QoS changes that are in progress in nova and neutron. * Placement is now publishing [install docs](https://docs.openstack.org/placement/latest/install/) but it is important to note that those docs have not been validated (as far as I'm aware) by the packagers. That's a thing that needs to happen, presumably by the packagers. * os-resource-classes 0.3.0 has been [released](https://pypi.org/p/os-resource-classes) with a `normalize_name` function. * There are some pending specs from nova which are primarily placement feature specs. We'll continue with those as is (see below), but come the next cycle the plan is to manage specs in the placement repo, not have a separate repo, and not have separate spec cores. # Specs/Blueprints/Features ## Near to Done * [Filter Allocation Candidates by Provider Tree](http://specs.openstack.org/openstack/nova-specs/specs/stein/approved/… has been mostly completed by Tetsuro, but there's a [pending update to the spec](https://review.openstack.org/639033). ## Not yet Done * [Support filtering by forbidden aggregate membership](http://specs.openstack.org/openstack/nova-specs/specs/stein/app… * [Support any traits in allocation_candidates query](http://specs.openstack.org/openstack/nova-specs/specs/stein/approved… * [Support mixing required traits with any traits](http://specs.openstack.org/openstack/nova-specs/specs/stein/approve… ## Not yet Approved * [Update alloc-candidates-in-tree](https://review.openstack.org/#/c/639033/) updates the in-tree spec above to reflect what was learned while doing the actual implementation. Notably how numbered `in_tree` parameters impact results. * [Resource provider - request group mapping in allocation candidate](https://review.openstack.org/#/c/597601/) has had a recent resurgence in attention. # Bugs * Placement related [bugs not yet in progress](https://goo.gl/TgiPXb): 15. * [In progress placement bugs](https://goo.gl/vzGGDQ) 17. # osc-placement osc-placement is currently behind by 14 microversions. Code for 1.18 is [under review](https://review.openstack.org/#/c/639738/). # Main Themes This section now overlaps a bit with the Specs/Features bit above. This will settle out with a bit more clarity as we move along. ## Nested * Reshaper handing in nova keeps exposing additional things that need to be remembered on the nova-side, so there are a few patches remaining related to [vgpu reshaping](https://review.openstack.org/#/q/topic:bp/reshape-provider-tree+… but it is mostly ready. * The bandwidth-resource-provider topic has merged a vast amount of code but there is still [plenty left](https://review.openstack.org/#/q/topic:bp/bandwidth-resource-provider). Related to all this nested stuff: The complex hardware models that drove the development of the nested resource provider system are challenging to test. The cloud hardware provided to OpenStack infrastructure does not expose the hardware that would allow real integration tests. If anyone reading this is in a position to provide third party CI with fancy hardware for NUMA, NFV, FPGA, and GPU related integration testing with nova, there's a significant need for that. ## Refactoring (I think refactoring should be a constant theme. To reflect that, I'm going to have a section here. Editorial privilege or something.) There's a collection of patches in progress, currently under the topic [scrub-Lists](https://review.openstack.org/#/q/topic:scrub-Lists) that is a follow up to the patches that removed oslo versioned objects. That work pointed out some opportunities to DRY-up the List classes (e.g., UsageList) to remove some duplication and simplify. Then, after looking at that, it became clear that entirely removing the List classes, in favor of using python native lists, would further simplify the code. Apart from the previously mentioned performance and simplicity benefits of these changes, it's also managed to expose and fix a few bugs, simple because we were looking at things and moving them around. If you pick up rocks, you can see the bugs and squash them. If you don't, they breed. # Other Placement * <https://review.openstack.org/#/q/topic:improve-debug-log> A series of improvements leading to a better debug log when retrieving allocation candidates. * <https://review.openstack.org/#/c/639628/> Docs: extract testing info to own sub-page * <https://review.openstack.org/#/q/topic:cd/gabbi-tempest-job> Gabbi-based integration tests of placement. These recently found a bug that none of the functional, grenade, nor tempest tests did. * <https://review.openstack.org/#/c/619050/> Optionally migrate database at service startup (so you don't have to run `placement-manage db sync` if you don't want to). * <https://review.openstack.org/#/c/630216/> Add a vision-reflection (of the Technical Vision doc). # Other Service Users ## Nova See also the several links above for more nova changes. Also, I'm a bit behind on my tracking in this area, so there is likely plenty of other stuff too. This will improve over time. * <https://review.openstack.org/538498> Convert driver supported capabilities to compute node provider traits * <https://review.openstack.org/621494> Add descriptions of numbered resource classes and traits * <https://review.openstack.org/636412> Make move_allocations handle empty source allocations (Part of a series on cross-cell resize) * <https://review.openstack.org/#/q/topic:bp/count-quota-usage-from-placement> Using placement (from nova) for counting (some of) quota. ## Not Nova * <https://review.openstack.org/#/q/topic:tripleo-nova-placement-removal> * <https://review.openstack.org/#/q/topic:tripleo-placement-extraction> * <https://review.openstack.org/#/q/topic:minimum-bandwidth-allocation-placeme…> Neutron side of minimum bandwidth. * <https://review.openstack.org/#/q/topic:puppet-placement-extraction> * <https://review.openstack.org/#/q/bp/no-affinity-instance-reservation> Blazar reservation handling, including some manipulation of inventory in placement. * <https://review.openstack.org/633204> Blazar: Retry on inventory update conflict # End Though this is long, it doesn't really bring us fully up to date. If something is missing that you think is important please let me know. Once I'm back in the flow it should become increasingly complete. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

2 1

Re: [scientific-sig] IRC meeting 1100 UTC: Edge computing and Scientific use cases
by Bogdan Dobrelya 01 Mar '19

01 Mar '19

On 27.02.2019 14:33, Adrien Lebre wrote: > Hi, > > Getting a look to this post could also make sense: > http://beyondtheclouds.github.io/blog/openstack/cockroachdb/2018/06/04/eval… Thank you. The latter links give a lot of insights and sources of information to be consumed by me in a bunch of years perhaps :) In case someone would love to collaborate for the aforementioned positioning paper to help to make it into some future events hopefully, please consider contributing your ideas directly to the draft [0]. By the way, I don't mind if anyone's willing to handover it and being a main presenter/driver of it for the future events (once/if the paper is ever finished ofc). [0] https://github.com/bogdando/papers-ieee/tree/master/ICFC-2019/LaTeX > > Best regards, > Adrien > PS: Thanks Pierre. > -- > Prof. IMT Atlantique / Inria / LS2N > Head of the STACK Research Group > www.emn.fr/x-info/alebre08 <http://www.emn.fr/x-info/alebre08> > >> On 27 Feb 2019, at 14:18, Pierre Riteau <pierre(a)stackhpc.com >> <mailto:pierre@stackhpc.com>> wrote: >> >> Hello Bogdan, >> >> In the context of data management in edge computing, you may be >> interested by research done by the Discovery Initiative: >> http://beyondtheclouds.github.io/publications.html >> I am adding Adrien Lèbre to the thread, he may be able to point you to >> the publications most relevant to your whitepaper draft. >> >> Best wishes, >> Pierre Riteau >> >> On Wed, 27 Feb 2019 at 10:21, Bogdan Dobrelya <bdobreli(a)redhat.com> wrote: >>> >>> On 27.02.2019 10:51, Stig Telfer wrote: >>>> Hi All - >>>> >>>> We have an IRC meeting at 1100 UTC (about an hour’s time) in channel >>>> #openstack-meeting. Everyone is welcome. >>>> >>>> Today’s agenda is here: >>>> >>>> https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_February_27th_20… >>>> >>>> Today’s main event is that we have Ildikó Vancsa joining us to talk >>>> about edge computing use cases and find overlap with Scientific SIG >>>> use cases. >>>> >>>> Cheers, >>>> Stig >>>> >>>> >>> >>> Edge computing is a *great* topic for lot of future research work indeed. >>> Thank you for this useful announcement, appreciated! >>> I'll attend at least the first half of it, and in case I'll miss the >>> agenda item that I placed at the end of the list, please be kind to >>> review that research request with my absence as well. Thanks! >>> >>> >>> -- >>> Best regards, >>> Bogdan Dobrelya, >>> Irc #bogdando >>> > -- Best regards, Bogdan Dobrelya, Irc #bogdando

1 0