September 2019 - openstack-discuss

Re: [openstack-dev] Are we ready to put stable/ocata into extended maintenance mode?
by Alex Schultz 03 Mar '21

03 Mar '21

On Tue, Sep 18, 2018 at 1:30 PM Alex Schultz <aschultz(a)redhat.com> wrote: > > On Tue, Sep 18, 2018 at 1:27 PM, Matt Riedemann <mriedemos(a)gmail.com> wrote: > > The release page says Ocata is planned to go into extended maintenance mode > > on Aug 27 [1]. There really isn't much to this except it means we don't do > > releases for Ocata anymore [2]. There is a caveat that project teams that do > > not wish to maintain stable/ocata after this point can immediately end of > > life the branch for their project [3]. We can still run CI using tags, e.g. > > if keystone goes ocata-eol, devstack on stable/ocata can still continue to > > install from stable/ocata for nova and the ocata-eol tag for keystone. > > Having said that, if there is no undue burden on the project team keeping > > the lights on for stable/ocata, I would recommend not tagging the > > stable/ocata branch end of life at this point. > > > > So, questions that need answering are: > > > > 1. Should we cut a final release for projects with stable/ocata branches > > before going into extended maintenance mode? I tend to think "yes" to flush > > the queue of backports. In fact, [3] doesn't mention it, but the resolution > > said we'd tag the branch [4] to indicate it has entered the EM phase. > > > > 2. Are there any projects that would want to skip EM and go directly to EOL > > (yes this feels like a Monopoly question)? > > > > I believe TripleO would like to EOL instead of EM for Ocata as > indicated by the thead > http://lists.openstack.org/pipermail/openstack-dev/2018-September/134671.ht… > Bringing this backup to see what we need to do to get the stable/ocata branches ended for the TripleO projects. I'm bringing this up because we have https://review.openstack.org/#/c/647009/ which is for the upcoming rename but CI is broken and we have no interest in continue to keep the stable/ocata branches alive (or fix ci for them). Thanks, -Alex > Thanks, > -Alex > > > [1] https://releases.openstack.org/ > > [2] > > https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… > > [3] > > https://docs.openstack.org/project-team-guide/stable-branches.html#extended… > > [4] > > https://governance.openstack.org/tc/resolutions/20180301-stable-branch-eol.… > > > > -- > > > > Thanks, > > > > Matt > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

5 11

[Syntribos] Does anyone still use this?
by Gage Hugo 07 May '20

07 May '20

The Security SIG has recently been looking into updating several sites/documentation and one part we discussed the last couple meetings was the security tools listings[0]. One of the projects listed there, Syntribos, doesn't appear to have much activity[1] outside of zuul/python/docs maintenance and the idea of retiring the project was mentioned. However, there does seem to be an occasional bug-fix submitted, so we are sending this email out to see if anyone is still utilizing Syntribos. If so, please either respond here, reach out to us in the #openstack-security irc channel, or fill out the section in the Security SIG Agenda etherpad[2]. Thanks! [0] https://security.openstack.org/#security-tool-development [1] https://review.opendev.org/#/q/project:openstack/syntribos [2] https://etherpad.openstack.org/p/security-agenda

2 2

[openstack-ansible] dropping suse support
by Mohammed Naser 05 May '20

05 May '20

Hi everyone, I've been trying to avoid writing this email for the longest time ever. It's really painful to have to reach this state, but I think we've hit a point where we can no longer maintain support for SUSE within OpenStack Ansible. Unfortunately, with the recent layoffs, the OSA team has taken a huge hit in terms of contributors which means that there is far less contributors within the project. This means that we have less resources to go around and it forces us to focus on a more functional, reliable and well tested set of scenarios. Over the past few cycles, there has been effort to add SUSE support to OpenStack Ansible, during the time that we had maintainers, it was great and SUSE issues were being fixed promptly. In addition, due to the larger team at the time, we found ourselves having some extra time where we can help unbreak other gates. Jesse used to call this a "labour of love", which I admired at the time and hoped we continue to do as much as we can of. However, the lack of a committed maintainer for OpenSUSE has resulted in constantly failing jobs[1][2] (which were moved to non-voting, wasting CI resources as no one fixed them). In addition, it's causing several gate blocks for a few times with no one really finding the time to clean them up. We are resource constrained at this point and we need the resource to go towards making the small subset of supported features functional (i.e. CentOS/Ubuntu). We struggle with that enough, and there seems to be no deployers that are running SUSE in real life at the moment based on bugs submitted. With that, I propose that we drop SUSE support this cycle. If anyone would like to volunteer to maintain it, we can review that option, but that would require a serious commitment as we've had maintainers step off and it hurts the velocity of the project as no one can merge code anymore. ..Really wish I didn't have to write this email Mohammed [1]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct… [2]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct…

6 9

[cinder] consistency group not working
by Francois Scheurer 16 Jan '20

16 Jan '20

Dear Cinder Experts We are running the rocky release. |We can create a consistency group: openstack consistency group create --volume-type b9f67298-cf68-4cb2-bed2-c806c5f83487 fsc-consgroup Bug 1: but adding volumes is not working: openstack consistency group add volume c3f49ef0-601e-4558-a75a-9b758304ce3b b48752e3-641f-4a49-a892-6cb54ab6b74d c0022411-59a4-4c7c-9474-c7ea8ccc7691 0f4c6493-dbe2-4f75-8e37-5541a267e3f2 => Invalid volume: Volume is not local to this node. (HTTP 400) (Request-ID: req-7f67934a-5835-40ef-b25c-12591fd79f85) Bug 2: deleting consistency group is also not working (silently failing): openstack consistency group delete c3f49ef0-601e-4558-a75a-9b758304ce3b |||=> AttributeError: 'RBDDriver' object has no attribute 'delete_consistencygroup'| See details below. Using the --force option makes no difference and the consistency group is not deleted. Do you think this is a bug or a configuration issue? Thank you in advance. | Cheers Francois |Details: ==> /var/lib/docker/volumes/kolla_logs/_data/cinder/cinder-api-access.log <== 10.0.129.17 - - [27/Sep/2019:12:16:24 +0200] "POST /v3/f099965b37ac41489e9cac8c9d208711/consistencygroups/3706bbab-e2df-4507-9168-08ef811e452c/delete HTTP/1.1" 202 - 109720 "-" "python-cinderclient" ==> /var/lib/docker/volumes/kolla_logs/_data/cinder/cinder-volume.log <== 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server [req-9010336e-d569-47ad-84e2-8dd8b729939c b141574ee71f49a0b53a05ae968576c5 f099965b37ac41489e9cac8c9d208711 - default default] Exception during message handling: AttributeError: 'RBDDriver' object has no attribute 'delete_consistencygroup' 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/osprofiler/profiler.py", line 159, in wrapper 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/volume/manager.py", line 3397, in delete_group 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server vol_obj.save() 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server self.force_reraise() 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/volume/manager.py", line 3362, in delete_group 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server self.driver.delete_consistencygroup(context, cg, 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server AttributeError: 'RBDDriver' object has no attribute 'delete_consistencygroup' 2019-09-27 12:16:24.491 30 ERROR oslo_messaging.rpc.server| -- EveryWare AG François Scheurer Senior Systems Engineer Zurlindenstrasse 52a CH-8003 Zürich tel: +41 44 466 60 00 fax: +41 44 466 60 10 mail: francois.scheurer(a)everyware.ch web: http://www.everyware.ch

2 2

[ospurge] looking for project owners / considering adoption
by Adam Harwell 04 Nov '19

04 Nov '19

I haven't seen much activity on this project in a while, and it's been moved to opendev/x since the opendev migration... Who is the current owner of this project? Is there anyone who actually is maintaining it, or would mind if others wanted to adopt the project to move it forward? Thanks, --Adam Harwell

5 10

[nova]review guide for the policy default refresh spec
by Ghanshyam Mann 29 Oct '19

29 Oct '19

Hello Everyone, As many of you might know that in Train, we are doing Nova policy changes to adopt the keystone's new defaults and scope type[1]. There are multiple changes required per policy as mentioned in spec. I am writing this review guide for the patch sequence and at the end how each policy will look like. I have prepared the first set of patches. I would like to get the feedback on those so that we can modify the other policy also on the same line. My plan is to start the other policy work after we merge the first set of policy changes. Patch sequence: Example: os-services API policy: ------------------------------------------------------------- 1. Cover/Improve the test coverage for existing policies: This will be the first patch. We do not have good test coverage of the policy, current tests are not at all useful and do not perform the real checks. Idea is to add the actual tests coverage for each policy as the first patch. new tests try to access the API with all possible context and check for positive and negative cases. - https://review.opendev.org/#/c/669181/ 2. Introduce scope_types: This will add the scope_type for policy. It will be either 'system', 'project' or 'system and project'. In the same patch, along with existing test working as it is, new tests of scope type will be added which will run with [oslo_policy] enforce_scope=True so that we can capture the real scope checks. - https://review.opendev.org/#/c/645427/ 3. Add new default roles: This will add new defaults which can be SYSTEM_ADMIN, SYSTEM_READER, PROJECT_MEMBER_OR_SYSTEM_ADMIN, PROJECT_READER_OR_SYSTEM_READER etc depends on Policy. Test coverage of new defaults, as well as deprecated defaults, are covered in same patch. This patch will add the granularity in policy if needed. Without policy granularity, we cannot add new defaults per rule. - https://review.opendev.org/#/c/648480/ (I need to add more tests for deprecated rules) 4. Pass actual Targets in policy: This is to pass the actual targets in context.can(). Main goal is to remove the defaults targets which is nothing but context'user_id,project_id only. It will be {} if no actual target data needed in check_str. - https://review.opendev.org/#/c/676688/ Patch sequence: Example: Admin Action API policy: 1. https://review.opendev.org/#/c/657698/ 2. https://review.opendev.org/#/c/657823/ 3. https://review.opendev.org/#/c/676682/ 4. https://review.opendev.org/#/c/663095/ There are other patches I have posted in between for common changes or fix or framework etc. [1] https://specs.openstack.org/openstack/nova-specs/specs/train/approved/polic… -gmann

1 2

[cinder][tooz]Lock-files are remained
by Rikimaru Honjo 19 Oct '19

19 Oct '19

Hi, I'm using Queens cinder with the following setting. --------------------------------- [coordination] backend_url = file://$state_path --------------------------------- As a result, the files like the following were remained under the state path after some operations.[1] cinder-63dacb3d-bd4d-42bb-88fe-6e4180164765-delete_volume cinder-32c426af-82b4-41de-b637-7d76fed69e83-delete_snapshot In my understanding, these are lock-files created for synchronization by tooz. But, these lock-files were not deleted after finishing operations. Is this behaviour correct? [1] e.g. Delete volume, Delete snapshot -- _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ Rikimaru Honjo E-mail:honjo.rikimaru@ntt-tx.co.jp

7 15

[metrics] [telemetry] [stein] cpu_util
by Budai Laszlo 18 Oct '19

18 Oct '19

Hello all, the release release announce of ceilometer rocky is deprecating the cpu_util and *.rate metrics "* cpu_util and *.rate meters are deprecated and will be removed in future release in favor of the Gnocchi rate calculation equivalent." so we don't have them in Stein. Can you direct me to some document that describes how to achieve these with Gnocchi rate calculation? Thank you, Laszlo

5 4

[aodh] [heat] Stein: How to create alarms based on rate metrics like CPU utilization?
by Bernd Bausch 18 Oct '19

18 Oct '19

Prior to Stein, Ceilometer issued a metric named /cpu_util/, which I could use to trigger alarms and autoscaling when CPU utilization was too high. cpu_util doesn't exist anymore. Instead, we are asked to use Gnocchi's /rate/ feature. However, when using rates, alarms on a group of resources require more parameters than just one metric: Both an aggregation and a reaggregation method are needed. For example, a group of instances that implement "myapp": gnocchi measures aggregation -m cpu --reaggregation mean --aggregation rate:mean --query server_group=myapp --resource-type instance Actually, this command uses a deprecated API (but from what I can see, Aodh still uses it). The new way is like this: gnocchi aggregates --resource-type instance '(aggregate rate:mean (metric cpu mean))' server_group=myapp If rate:mean is in the archive policy, it also works the other way around: gnocchi aggregates --resource-type instance '(aggregate mean (metric cpu rate:mean))' server_group=myapp Without reaggregation, I get quite unexpected numbers, including negative CPU rates. If you want to understand why, see this discussion with one of the Gnocchi maintainers [1]. *My problem*: Aodh allows me to set an aggregation method, but not a reaggregation method. How can I create alarms based on rates? The problem extends to Heat and autoscaling. Thanks much, Bernd. [1] https://github.com/gnocchixyz/gnocchi/issues/1044

4 3

[nova][ptg] Ussuri scope containment
by Eric Fried 16 Oct '19

16 Oct '19

Nova developers and maintainers- Every cycle we approve some number of blueprints and then complete a low percentage [1] of them. Which blueprints go unfinished seems to be completely random (notably, it appears to have nothing to do with our declared cycle priorities). This is especially frustrating for consumers of a feature, who (understandably) interpret blueprint/spec approval as a signal that they can reasonably expect the feature to land [2]. The cause for non-completion usually seems to fall into one of several broad categories: == Inadequate *developer* attention == - There's not much to be done about the subset of these where the contributor actually walks away. - The real problem is where the developer thinks they're ready for reviewers to look, but reviewers don't. Even things that seem obvious to experienced reviewers, like failing CI or "WIP" in the commit title, will cause patches to be completely ignored -- but unseasoned contributors don't necessarily understand even that, let alone more subtle issues. Consequently, patches will languish, with each side expecting the other to take the next action. This is a problem of culture: contributors don't understand nova reviewer procedures and psychology. == Inadequate *reviewer* attention == - Upstream maintainer time is limited. - We always seem to have low review activity until the last two or three weeks before feature freeze, when there's a frantic uptick and lots gets done. - But there's a cultural rift here as well. Getting maintainers to care about a blueprint is hard if they don't already have a stake in it. The "squeaky wheel" concept is not well understood by unseasoned contributors. The best way to get reviews is to lurk in IRC and beg. Aside from not being intuitive, this can also be difficult logistically (time zone pain, knowing which nicks to ping and how) as well as interpersonally (how much begging is enough? too much? when is it appropriate?). == Multi-release efforts that we knew were going to be multi-release == These may often drag on far longer than they perhaps should, but I'm not going to try to address that here. ======== There's nothing new or surprising about the above. We've tried to address these issues in various ways in the past, with varying degrees of effectiveness. I'd like to try a couple more. (A) Constrain scope, drastically. We marked 25 blueprints complete in Train [3]. Since there has been no change to the core team, let's limit Ussuri to 25 blueprints [4]. If this turns out to be too few, what's the worst thing that happens? We finish everything, early, and wish we had done more. If that happens, drinks are on me, and we can bump the number for V. (B) Require a core to commit to "caring about" a spec before we approve it. The point of this "core liaison" is to act as a mentor to mitigate the cultural issues noted above [5], and to be a first point of contact for reviews. I've proposed this to the spec template here [6]. Thoughts? efried [1] Like in the neighborhood of 60%. This is anecdotal; I'm not aware of a good way to go back and mine actual data. [2] Stuff happens, sure, and nobody expects 100%, but 60%? Come on, we have to be able to do better than that. [3] https://blueprints.launchpad.net/nova/train [4] Recognizing of course that not all blueprints are created equal, this is more an attempt at a reasonable heuristic than an actual expectation of total size/LOC/person-hours/etc. The theory being that constraining to an actual number, whatever the number may be, is better than not constraining at all. [5] If you're a core, you can be your own liaison, because presumably you don't need further cultural indoctrination or help begging for reviews. [6] https://review.opendev.org/685857

11 21