September 2019 - openstack-discuss

[keystone] Pre-feature-freeze update
by Colleen Murphy 09 Sep '19

09 Sep '19

I won't be writing a team report since I'm still figuring out which way is up after a week in the desert, but with feature freeze next week I wanted to give a status update on all the in-flight work that is due next week: * System Scope and Default Roles All documented scope[1] and role[2] migrations are in progress. Some are closer to done than others. Since enforce_scope cannot be set to true in keystone.conf until all of them are completed, and since leaving deprecation warnings in the logs for more than two cycles is a very undesirable operator experience, it's essential we complete these by next week. * Application Credential Access Rules This implementation[3] for keystone has been completed for months but the last few patches in the stack are lacking reviews. Client support has been proposed but with the final client release happening next week we will likely not land it until next cycle. * Resource Options and Immutable Resources Resource options[4] and immutable resources[5] are intertwined and the finishing touches are still being applied. Hope to have this completed early next week. * Federated Attributes for Users Support for federated attributes for users[6] is passing CI but needs reviews, it's unclear to me how much has changed since those patches were originally proposed two years ago so it's unfortunate that we're only left with a week to look at them. * Expiring Group Membership There is only a partial implementation proposed for expiring group membership[7] and neither patch is passing CI. This seems to have effectively missed the feature proposal freeze deadline which was a few weeks ago and will not likely make it in this cycle. * CI After skimming the meeting logs I saw the unit test timeout problem was discussed and a temporary workaround was proposed[8]. This sounded like a great idea but it seems that no one implemented it, so I did[9]. Unfortunately this will conflict with all the system-scope/default-roles patches in flight. With how many changes need to go in and how slow it will be with all of them needing to be rechecked and continually making the problem even worse, I propose we go ahead and merge the workaround ASAP and update all the in-flight changes to move the protection tests to the new location. It also appears that the non-voting federation CI broke recently, this will hopefully be fixed by updating the opensuse nodeset[10]. [1] https://bugs.launchpad.net/keystone/+bugs?field.tag=system-scope [2] https://bugs.launchpad.net/keystone/+bugs?field.tag=default-roles [3] https://review.opendev.org/#/q/topic:bp/whitelist-extension-for-app-creds [4] https://review.opendev.org/678322 [5] https://review.opendev.org/#/q/topic:immutable-resources [6] https://review.opendev.org/#/q/topic:bp/support-federated-attr [7] https://review.opendev.org/#/q/topic:bug/1809116 [8] http://eavesdrop.openstack.org/meetings/keystone/2019/keystone.2019-08-27-1… [9]https://review.opendev.org/680788 [10] https://review.opendev.org/680799 Colleen

2 2

[nova][telemetry] does Telemetry still use the Nova server usage audit log API?
by melanie witt 09 Sep '19

09 Sep '19

Howdy all, TL;DR: I have a question, does the Telemetry service (or any other service) still make use of the server usage audit log API in Nova [1]? Recently I was investigating customer issues where the nova.task_log database table grows infinitely and is never cleaned up [2]. I asked about it today in #openstack-nova [3] and Matt Riedemann explained that the API is toggled via config option [4] and that the Telemetry service is/was the consumer of the API. I found through code inspection that there are no methods for deleting nova.task_log records and am trying to determine what is the best way forward for handling cleanup. Matt mentioned the possibility of deprecating the server usage audit log API altogether, which we might be able to do if no one is using it anymore. So, I was thinking: * If Telemetry is no longer using the server usage audit log API, we deprecate it in Nova and notify deployment tools to stop setting [DEFAULT]/instance_usage_audit = true to prevent further creation of nova.task_log records and recommend manual cleanup by users or * If Telemetry is still using the server usage audit log API, we create a new 'nova-manage db purge_task_log --before <date>' (or similar) command that will hard delete nova.task_log records before a specified date or all if --before is not specified Can anyone shed any light on whether Telemetry, or any other service, still uses the server usage audit log API in Nova? Would we be able to deprecate it? If we can't, what do you think of the nova-manage command idea? I would appreciate hearing your thoughts about it. Cheers, -melanie [1] https://docs.openstack.org/api-ref/compute/#server-usage-audit-log-os-insta… [2] https://bugzilla.redhat.com/show_bug.cgi?id=1726256 [3] http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.… [4] https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.in…

4 5

[nova][neutron][all][openstack-devs] studying and analysing Openstack developers
by Farida El Zanaty 09 Sep '19

09 Sep '19

Hi! I am Farida El-Zanaty from McGill University. Under the supervision of Prof. Shane McIntosh, my research aims to study design discussions that occur between developers during code reviews. Last year, we published a study about the frequency and types of such discussions that occur in OpenStack Nova and Neutron (http://rebels.ece.mcgill.ca/papers/esem2018_elzanaty.pdf) We are reaching out to OpenStack developers to better understand their perspectives on design discussions during code reviews. Those who are interested can start by participating in our 10-minute survey about their experiences as both the code reviewer and author. Survey participants will be entered into a raffle for a $50 Amazon gift card. Survey: https://forms.gle/Hhn191f6cxF5hVgG8 Thanks for your time, Farida El-Zanaty

1 0

Re: [devstack][qa][python3] "also install the Python 2 dev library" - still needed?
by Peter Penchev 09 Sep '19

09 Sep '19

On Mon, Sep 9, 2019 at 5:40 PM Peter Penchev <pp(a)storpool.com> wrote: > On Mon, Sep 9, 2019 at 2:55 PM Sean Mooney <smooney(a)redhat.com> wrote: > >> On Mon, 2019-09-09 at 11:22 +0300, Peter Penchev wrote: >> > Hi, >> > >> > When devstack's `setup_dev_lib` function is invoked and USE_PYTHON3 has >> > been specified, this function tries to also install the development >> library >> > for Python 2.x, I guess just in case some package has not declared >> proper >> > Python 3 support or something. It then proceeds to install the Python 3 >> > version of the library and all its dependencies. >> > >> > Unfortunately there is a problem with that, and specifically with script >> > files installed in the system's executable files directory, e.g. >> > /usr/local/bin. The problem appears when some Python library has already >> > been installed for Python 3 (and has installed its script files), but is >> > now installed for Python 2 (overwriting the script files) and is then >> not >> > forcefully reinstalled for Python 3, since it is already present. Thus, >> the >> > script files are last modified by the Python 2 library installation and >> > they have a hashbang line saying `python2.x` - so if something then >> tries >> > to execute them, they will run and use modules and libraries for Python >> 2 >> > only. >> yes this is a long standing issue. we discovered it a year ago but it was >> never fix. >> >> in Ussrui i guess one of the first changes to devstack to make it python >> 3 only >> will be to chagne that behavior. im not sure if we will be able to change >> it before then. >> >> whenever you us libs_from_git in your local.conf on a python 3 install it >> will install >> them twice both with python 2 and python 3. i hope more distros elect to >> symlink /usr/bin/python to python 3 >> some distros have chosen to do that on systems that are python only and i >> believe that is the correct >> approch. >> >> when i encountered this it was always resuliting on the script header >> being #!/usr/bin/python with no >> version suffix i >> gues on a system where that points to python 3 the python 2.7 install >> might write python2.7 >> there instead? >> > > It depends on what version of pip is invoked; I think that the way > devstack invokes it nowadays it will always provide a version on the > shebang line. > > >> >> > >> > We experienced this problem when running the cinderlib tests from >> Cinder's >> > `playbooks/cinderlib-run.yaml` file - it finds a unit2 executable >> > (installed by the unittest2 library) and runs it, hoping that unit2 >> will be >> > able to discover and collect the cinderlib tests and load the cinderlib >> > modules. However, since unittest2 has last been installed as a Python 2 >> > library, unit2 runs with Python 2 and fails to locate the cinderlib >> > modules. (Yes, we know that there are other ways to run the cinderlib >> > tests; this message is about the problem exposed by this way of running >> > them) >> > >> > The obvious solution would be to instruct the Python 2 pip to not >> install >> > script (or other shared) files at all; unfortunately, >> > https://github.com/pypa/pip/issues/3980 ("Option to exclude scripts on >> > install"), detailing a very similar use case ("need it installed for >> Python >> > 2, but want to use it with Python 3") has been open for almost exactly >> > three years now with no progress. I wonder if I could try to help, but >> even >> > if this issue is resolved, there will be some time before OpenStack can >> > actually depend on a recent enough version of pip. >> well the obvious solution is to stop doing this entirly. >> it was added as a hack to ensure if you use LIB_FROM_GIT in you >> local.conf that those >> libs would always be install from the git checkout that you specified in >> you local.conf >> for train we are technically requireing all project to run under python 3 >> so we could remove >> the fallback mechanium of in stalling under python 2. it was there incase >> a service installed >> under python 2 to ensure it used the same version of the lib and did not >> use a version form >> pypi instead. i wanted to stop doing this last year but we could not >> becase not all project >> could run under python 3. but now that they should be able to we dont >> need this hack anymore. >> we should change it to respec the python version you have selected. that >> will speed >> up stacking speed as we wont have to install everything twice and fix the >> issue you have encountered. >> > > Yeah, thanks for confirming my thoughts that this might be the right > solution. I've proposed https://review.opendev.org/681029/ (and set > workflow -1) to wait for the Ussuri cycle. > > >> > >> > A horrible workaround would be to find the binary directory before >> > installing the Python 2 library (using something like `pip3.7 show >> > somepackage` and then running some heuristics on the "Location" field), >> > tar'ing it up and then restoring it... but I don't know if I even want >> to >> > think about this. >> > >> > Another possible way forward would be to consider whether we still want >> the >> > Python 2 libraries installed - is OpenStack's Python 3 transition >> reached a >> > far enough stage to assume that any projects that still require Python 2 >> > *and* fail to declare their Python 2 dependencies properly are buggy? >> To be >> > honest, this seems the most reasonable path for me - drop the "also >> install >> > the Python 2 libs" code and see what happens. I could try to make this >> > change in a couple of test runs in our third-party Cinder CI system and >> see >> > if something breaks. >> > > G'luck, > Peter > > Argh, I sent this from the wrong account, did I not... G'luck, Peter

1 0

Re: [tc] Campaign Question: Treating the Problem, not just the symptoms- Burnout, No Polling, etc
by Chris Dent 09 Sep '19

09 Sep '19

On Mon, 9 Sep 2019, Adam Spiers wrote: > Chris Dent <cdent+os(a)anticdent.org> wrote: >> On Fri, 6 Sep 2019, Jeremy Stanley wrote: >> >>> I'm disappointed that you don't think the software you're making is >>> open source. I think the software I'm making is open source, and if >>> I didn't I wouldn't be here. > > I wouldn't either. I'd be very worried to live in a world where there > was no serious open source rival to AWS, Azure, GCE etc. (One possible tl;dr of the below is: For at least some people working on OpenStack the more direct and immediate cause and result of their work (whatever their intent) is the enablement of corporate profit (through sales and support) not individual humans using the software.) >From some standpoints I would guess that OpenStack looks and behaves like open source: people work on it collaboratively and the code is available for anyone to change. And I would agree that from that standpoint it is open source, and the four opens are good both in letter and in spirit. I would also agree that the academic and other non-profit use of a OpenStack that Jeremy is very compelling and motivating. But the context of much of this thread has been about the experience of the developers making OpenStack. How they come to be in this situation, how they manage their work, who they work with, what drives decisions, etc. (What follows is a ramble. A apologize for not being able to write less. But you did ask so here it goes.) In the context of daily developer experience, things are less clear. They would be more clear and would feel more like open source if I was more frequently collaborating in the creation of code with people who were using OpenStack. But I don't. Most frequently I'm collaborating with people who instead of using OpenStack are helping to make something for other people (with whom they have infrequent collaborative contact) to use OpenStack. For some people this is not the case. For example, many of the people who have been deeply involved with OpenStack infra use OpenStack all the time and also work hard to improve the code of OpenStack. But on a daily basis that isn't my experience. Nor does it feel like the experience of most of the people I tend to collaborate with. Yes, sometimes I will collaborate with someone from CERN to create a feature, but this is rare. Usually I collaborate with people from Intel, VMware, Red Hat, and a variety of Telco vendors. Doing a thing to help an existing customer or hoped for notional customer, both of whom are abstractions at a distance, not humans. This isn't a bad thing. Organizations collaborating in any way is great. But it doesn't _feel_ like "open source" to me. And that feeling is an important factor (I think) in analyzing the motivations people experience when working on OpenStack and the choices they make with regard to how they act in the environment. As someone who has done what could be called open source since long before the term was invented, the common failure of corporate patrons to give maintainability and quality (of product and (critically) the experience of creating it) sufficient attention is a source of a great deal of resentment and internal conflict. I am far too conscious of the necessity to compensate for that failure if I want to feel a sense of well being with what I'm helping to create (both in terms of product and the environment it is being created in). That is: I care enough to try to do what I think is right. In this thread, and the one that started it, we've put forward the "maybe we should just chill" as a bit of an antidote to burnout and overcommitment. While I rationally think that's the right idea, emotionally it is very hard to do and the source of that difficulty is this: OpenStack has constituted itself over the years as the domain of contributing corporations. Many paid contributors for whom working on OpenStack is their job. At the same time we have also been very vocal about being not just open source, but a source of good wisdom (the four opens) on how to do open source well. The latter creates a community I want to believe in. A source of pride. The former creates a conflict of interest, a frequent inability to do the actually right thing for the long term health of the community. A source of shame. Continued pleas to get the corporates to do "open source" well -- that is with correct attention to: * developer experience * maintainability * architectural integrity * deeper/closer ties to user engagement and their satisfaction and thus some akin to "actually open source" -- have fallen on what, if actions speak louder than words, are deaf ears. This creates a conundrum. I've tried a variety of ways out of it. One I'm experimenting with now is realizing that OpenStack really isn't, now, proper open source. And if it is not, then I don't have to care because they don't. > Again I'd be very interested to learn more about your take on what we > can do better. There are two directions to go: Maintain the mode of corporate-contribution-driven development. If this is to be healthy then the corps doing that contribution need to invest far more heavily in general, but especially in the items I've listed above at "correct attention". This would grant the community sufficient resources to evolve out of its aging models for development and governance. You have to have some free space to have the head space to get to new spaces. Start breaking down the corporate-contribution-driven development. Encourage professional openstack devs (like me) to age out of the system and discourage new ones coming in. Encourage feature development from and via users. Feature velocity might drop drastically but they might be features individuals actually use within a few weeks of their release rather than a few years. Some of this latter is already happening. Especially in what some people call the non-core projects; things associated with deployment for example. But in projects like nova we're heavily driven by trying to create a feature base which is predicted to drive sales, either directly or indirectly. And, though opinions and experiences differ, my opinion and experience is that driving sales as a direct factor is anathema to "open source". Indirect? Sure, whatever, if that floats your boat. The proper direct factor is humans. There's a lot more to this than I've stated here, but I hope that gives at least something in answer to the question. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

1 0

[ansible-sig] weekly meetings
by Mohammed Naser 09 Sep '19

09 Sep '19

Hi everyone, For those interested in getting involved, the ansible-sig meetings will be held weekly on Fridays at 2:00 pm UTC starting next week (13 September 2019). Looking forward to discussing details and ideas with all of you! Regards, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

3 3

[all][ptl][tc][docs] Develope a code-review practices document
by Trinh Nguyen 09 Sep '19

09 Sep '19

Hi all, I find it's hard sometimes to handle situations in code-review, something likes solving conflicts while not upsetting developers, or suggesting a change to a patchset while still encouraging the committer, etc. I know there are already documents that guide us on how to do a code-review [2] and even projects develope their own procedures but I find they're more about technical issues rather than human communication. Currently reading Google's code-review practices [1] give me some inspiration to develop more human-centric code-review guidelines for OpenStack projects. IMO, it could be a great way to help project teams develop stronger relationship as well as encouraging newcomers. When the document is finalized, I then encourage PTLs to refer to that document in the project's docs. Let me know what you think and I will put a patchset after one or two weeks. [1] https://google.github.io/eng-practices/review/ [2] https://docs.openstack.org/project-team-guide/review-the-openstack-way.html [3] https://docs.openstack.org/doc-contrib-guide/docs-review.html [4] https://docs.openstack.org/nova/rocky/contributor/code-review.html [5] https://docs.openstack.org/neutron/pike/contributor/policies/code-reviews.h… Bests, -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

2 1

[winstackers][powervmstackers][tc] removing winstackers and PowerVMStackers from TC governance
by Doug Hellmann 09 Sep '19

09 Sep '19

Following the U cycle election there was no candidate for either the winstackers or powervmstackers team PTL role. This is the second cycle in a row where that problem has occurred for both teams, which indicates that the teams are not active in the community. During the TC meeting today [1] we discussed removing the teams from governance, so I have proposed the patches to do that [2][3]. Doug [1] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-09-05-14.00.log.htm… [2] https://review.opendev.org/680438 remove powervmstackers team [3] https://review.opendev.org/680439 remove winstackers team

5 5

[placement] "now" worklist
by Chris Dent 09 Sep '19

09 Sep '19

As we near the end of a cycle it can be a bit unclear what tasks are relevant or a priority for the placement project. I've made a worklist in storyboard https://storyboard.openstack.org/#!/worklist/754 called "placement now". It gathers stories from the placement group (placement, osc-placement, os-resource-classes, os-traits) that I've tagged with 'pnow' to mean "these are the things we should be concerned with in the near future". This helps to take off the radar anything from the following groups: * Features that will not be considered this cycle. * Anything related to osc-placement (which has already seen its likely last release for this cycle) This leaves placement (the service) bug fixes, and docs. Not yet there is an item for "documenting the new nested provider features", mostly because the story for that has not solidified. Anything that is currently on that list we should finish before the end of the cycle. I hope that having a focused list can help drive that. Thanks. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

1 0

[mistral] Publish field in workflow tasks
by Ali Abdelal 09 Sep '19

09 Sep '19

Hello, Currently, there are two "publish" fields, one in the task(regular "publish")-the scope is branch and not global, and another under "on-success", “on-error” or “on-complete”. In the current behavior, regular "publish" is ignored if there is "publish" under "on-success", “on-error” or “on-complete” [1]. For example:- (a) version: '2.0' wf1: tasks: t1: publish: res_x1: 1 on-success: publish: branch: res_x2: 2 (b) version: '2.0' wf2: tasks: t1: publish: res_x1: 1 "res_x1" won't be published in (a), but it will in (b). We can either:- 1) Invalidate such syntax. 2) Merge the two publishes together and if there are duplicate keys, there are two options:- a) What takes priority is what's in publish under "on-success" or “on-error” or “on-complete. b) Not allow having a duplicate. What is your opinion? And please tell us if you have other suggestions. [1] https://bugs.launchpad.net/mistral/+bug/1791449

2 1