October 2020 - openstack-discuss - lists.openstack.org

[placement][nova][cinder][neutron][blazar] Placement governance switch(back)
by Stephen Finucane 03 Nov '20

03 Nov '20

Placement has been a separate project with its own governance since Stein [1]. Since then, the main drivers behind the separation have moved onto pastures new and with Tetsuro sadly declaring his non- candidacy for the PTL position for Wallaby [2], we're left in the unenviable position of potentially not having a PTL for the Wallaby cycle. As such, it's probably time to discuss the future of Placement governance. Assuming no one steps forward for the Placement PTL role, it would appear to me that we have two options. Either we look at transitioning Placement to a PTL-less project, or we move it back under nova governance. To be honest, given how important placement is to nova and other projects now, I'm uncomfortable with the idea of not having a point person who is ultimately responsible for things like cutting a release (yes, delegation is encouraged but someone needs to herd the cats). At the same time, I do realize that placement is used by more that nova now so nova cores and what's left of the separate placement core team shouldn't be the only ones making this decision. So, assuming the worst happens and placement is left without a PTL for Victoria, what do we want to do? Stephen PS: Apologies if I missed other projects with an interest in placement. I did try to catch them all /o\ [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-February/002575… [2] http://lists.openstack.org/pipermail/openstack-discuss/2020-September/01749…

8 10

[tripleo] please avoid creating ansible playbooks that call modules for trivial tasks
by Bogdan Dobrelya 02 Nov '20

02 Nov '20

Hi, In today TripleO client development we invoke ansible playbooks instead of Mistral workflows. I believe this may only be justified for non-trivial things. Otherwise please just import python modules into the client command actions without introducing additional moving parts in the form of ansible modules (that import the same python modules as well). Ansible brings declarative DSL that sometimes can better and simpler and faster describes things to do and shape inputs data, than if the same would be done in python, in the client. What should be the trivial thing and non-trivial then? Well, I think that example [0] illustrates such a trivial thing. A new playbook [1] would only install a few packages then "returns" back into python and calls a new ansible module [2]. And that new module only wraps [3] the same code being removed from the client: from tripleo_common.image import build ... manager = build.ImageBuildManager(<args>) I see no point of adding a new playbook and a new module for that trivial example. Those 4 packages could (and should) be as well installed from the client caller code without any ansible involved in the middle IMO. [0] https://review.opendev.org/#/c/759954/ [1] https://review.opendev.org/#/c/759953/3/tripleo_ansible/playbooks/cli-overc… [2] https://review.opendev.org/#/c/759953/3/tripleo_ansible/ansible_plugins/mod… -- Best regards, Bogdan Dobrelya, Irc #bogdando

6 9

[tripleo] stable/victoria for tripleo repos
by Marios Andreou 02 Nov '20

02 Nov '20

Hello all, I got some pings about tripleo stable/victoria so here is an update. The tripleo-ci team is busy preparing and we are almost ready. Basically we've blocked branching until we have adequate CI coverage. This means check & gate jobs but also promotion jobs so that check & gate aren't running with stale content. tl;dr We are aiming to branch victoria tripleoclient & common today and everything else next week - with more details below but please note: PLEASE AVOID MERGING to the stable/victoria tripleoclient/common that will appear after [4] until we are confident that our ci coverage is working well. Status: Most of our 'core' (standalone/scenarios/undercloud etc) check/gate jobs match on branch like [1] so we expect those to run OK once there is a stable/victoria available. There is ongoing work for branchful, upgrade, 3rd party and ovb jobs but we will not block branching on those. The victoria integration pipeline [2] (which produces current-tripleo) is in good shape. It was blocked on a nit in the component pipeline but it is resolved with [3]. Once we have a victoria promotion then we should be all clear to branch stable/victoria. Actions: A). today: make stable/victoria for python-tripleoclient & tripleo-common. The change at [4] will do this once merged. We will use this to see our check/gate jobs run against victoria & fix issues. B). next week: once we are happy with A). AND we've had a promotion, we release and make stable/victoria for everything else. Hopefully B). will happen end of next week but that depends on how well the above goes (plus inevitable delays cos PTG), I'll repeat this cos it's important: PLEASE AVOID MERGING to stable/victoria tripleoclient/common for now until we are confident that our jobs are doing the right thing! As always grateful for a sanity check or any thoughts about any of the above - I'm pretty new to this (weshay has been a great help so far thank you weshay++), thanks, marios [1] https://opendev.org/openstack/tripleo-ci/src/commit/d5514028452f9d427949f5a… [2] https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integ… [3] https://review.rdoproject.org/r/#/c/30654/ [4] https://review.opendev.org/#/c/759449/

1 2

Python 3.9 is here in Debian Sid
by Thomas Goirand 02 Nov '20

02 Nov '20

Hi there! Matthias Klose decided that uploading Python 3.9 on the 14th of this month (yes, the same day of the OpenStack release) would be fun. I am now really having fun ... fixing bugs! :) What's not so funny for me, is that some key packages like Greenlet or lxml aren't built yet for Python 3.9, even though they should. As a consequence, I just need to wait to be able to upload to Unstable. So this will delay my usual announcement for the release GA in Debian (even though the unofficial Victoria repo for Buster is ready and I could spawn VMs on Buster + Victoria). Anyways, all of this will settle slowly. Though Python 3.9 is here to stay, and to soon reach Ubuntu as well (it's planned for the next release). It'd be nice if projects were starting to investigate gating on Python 3.9 as early as possible. As much as I can tell, there's not so many issues to fix. The first one I fixed is here: https://review.opendev.org/758237 but I don't expect much more, and hopefully, I'll be able to propose more patch as I find the issues. Must we wait until Python 3.9 reaches Ubuntu to enable gating in the OpenStack projects? FYI, I do expect Victoria to run over Python 3.9 in Bullseye. Please don't reply with the usual "Victoria doesn't support it", because that's irrelevant. I'm fine with backporting patches, as long as they also exist in Wallaby. Cheers, Thomas Goirand (zigo)

7 9

[magnum] hyperkube deprecation situation
by Ionut Biru 02 Nov '20

02 Nov '20

Hi guys, I was hoping that in Victoria there will be a decision regarding this topic. Currently, if I use vanilla magnum, I'm stuck on 1.18.6 being the latest version, even so 1.18.10 was released. We are kinda stuck on a version that maybe in the near future is pruned to security issues. How do other operators are using magnum (vanilla) and keeping their kubernetes up to date? Are operators using a forked version of https://review.opendev.org/#/c/752254/ ? -- Ionut Biru - https://fleio.com

2 2

Fwd: [MURANO] DevStack Murano "yaql_function" Error
by İzzettin Erdem 01 Nov '20

01 Nov '20

Thanks Andy. Regards. Forwarded Conversation Subject: [MURANO] DevStack Murano "yaql_function" Error ------------------------ Gönderen: Andy Botting <andy(a)andybotting.com> Date: 30 Nis 2020 Per, 14:13 To: İzzettin Erdem <root.mch(a)gmail.com> Cc: Rong Zhu <aaronzhu1121(a)gmail.com>, < openstack-discuss(a)lists.openstack.org> Hi İzzettin, Thanks for your interest. Complete log link is at below. It is happening > when i try to deploy or update an environment. > > http://paste.openstack.org/show/792444/ > I've encountered the error AttributeError: 'method' object has no attribute '__yaql_function__' in our environment recently. I believe it started after we upgraded to the Stein release. I haven't tracked it down yet, but calls to the YAQL function: std:Project.getEnvironmentOwner() in a Murano app seem to generate it. It's good to know that at least someone else has reproduced this. cheers, Andy ---------- Gönderen: İzzettin Erdem <root.mch(a)gmail.com> Date: 2 May 2020 Cmt, 16:08 To: Andy Botting <andy(a)andybotting.com> Hi Andy, I solved this problem, it is happening when you try to create or update an environment on existing network. If you change the network with "create new" parameter, it stuck on "creating -or updating- environment" state and after that throws connection error. So i searched Murano network errors and i found this: https://docs.oracle.com/cd/E73172_01/E73173/html/issue-21976631.html. I applied these steps and i created successfully an environment just only with "create new" network parameter. After all these, i realized my devstack environment has no internet connection and because of this murano-agent could not install. So i tried to install Murano on my OSA stable/train test environment and i came with success until last step. Murano-agent installed but this time agent could not connect to rabbitmq instance. I discussed this with OSA IRC channel and i think i have to struggle with this for a long time. Apologize for this long mail but i wanted to inform you. Thanks. Regards. ---------- Gönderen: Andy Botting <andy(a)andybotting.com> Date: 3 May 2020 Paz, 14:24 To: İzzettin Erdem <root.mch(a)gmail.com> Hi İzzettin, Glad you got it sorted. I think you've probably just worked around the issue. Using your new method, you're probably just avoiding the std:Project.getEnvironmentOwner YAQL call now. I've also worked around it in our environment, but I'd like to solve it properly. When I get some time to look into it properly, I'll let you know. cheers, Andy

3 3

[tc][all][ Wallaby Virtual PTG Summary
by Ghanshyam Mann 31 Oct '20

31 Oct '20

Hello Everyone, I am summarizing the TC PTG discussion and action items. Wallaby Leaderless assignment --------------------------------------- Four projects are still pending for leadership assignment. We discussed the next step. If you would like to help any of these or using in your cloud/distro, this is the right time to step up: 1. Karbor - Action item: diablo_rojo will start the retirement process. 2. Qinling - Action item: gmann to start the retirement process. 3. Searchlight -Action item: gmann to start the retirement process. 4. Placement - Action item: gibi will update TC about the final discussion and accordingly mnaser to propose the patch in governance. Victoria Retrospective: ----------------------------- There are a couple of things TC finished in last cycle, few of them are: * Reduced size of TC to speed up the things. * Updated policy to add projects faster. * Outlined Distributed Project Leadership Model as an alternate leadership solution of PTL. Oslo will be the first project to try this new model. * Normalized retirement processes. All the retired repos cleanup is finished. * Tag cleanups & additions This is not complete yet but started with tc-approved release tag in the Victoria cycle. * Merged UC with TC. This is one of the key things finished and the best strategy to make users-developers work more closely. TC Meeting time: ----------------- TC is going to try (re-try) the weekly meeting every Thursday 15:00 UTC. * Action Items: mnaser: propose patch. TC policy regarding OpenStack client: ----------------------------------------------- Forum Discussion: https://etherpad.opendev.org/p/vSummit2020_FeatureGap This is one of the important items and still no good progress on this. From TC perspective, we talked about how to ask or motivate projects to migrate to OSC. TC needs to be very clear on whether it's a strict policy or just guidelines. After long discussion on this we are going with the below strategy: These strategies will be documented as TC resolution. * All user docs should have `openstack` client commands. * Focus on user docs first and then ops docs later * All ci-tooling should have `openstack` client usage * Use the process to find what is missing so we can prioritize it. * Action items: - diablo_rojo to work on initial resolution proposal[1] Finalize the Wallaby cycle goal selection: --------------------------------------------------- There are two proposals for the Wallaby community-wide goals. First is 'oslo.rootwrap to oslo.privsep' which is already being discussed in the last cycle also and all set to be selected[2]. 2nd proposal came up during PTG itself. During policy-popup PTG[3], it came up that deprecating the JSON formatted of policy file will be a good advance step before the projects move to new RBAC policies. This will help operators to smoothly migrate to new policies. This does not involve much work and ok to select as 2nd goal for Wallaby cycle, TC agreed to have below goals for Wallaby cycle: * Migrate from oslo.rootwrap to oslo.privsep [2] * Migrate Default Policy Format from JSON to YAML [4] *Action: - TC to merge the patches which selects both goals for Wallaby cycle. TC tag cleanup: ------------------- In the Victoria cycle, we started the process to audit all the TC tags and start cleanup those. We removed the 'tc:approved-release' tag in the Victoria cycle[5]. In this PTG we discussed two more tags. 1. assert:supports-zero-downtime-upgrade: Currently, there is no project who has this tag and also no testing framework available. Testing for zero downtime is not so easy in upstream testing. We decided to remove this tag as it is advertising something we aren't doing. If anyone interested to spend time on this in the future then we can add it after projects start testing it and document it. 2. assert:supports-api-interoperability: This tag is whether project API is interoperable also or not and this is important from the interop trademark program also. We only have Nova having this tag. Our goal is to promote more projects to apply for this tag. During the discussion, we found that we need to clarify this tag more clearly. For example, this tag is not about implementing the microversion but any versioning schema which provides the feature (API changes) discoverability. And also it is about how we change the API not how our APIs are currently. As long as services have some versioning mechanism to discover the changes and follow the API SIG guidelines for interoperability, and test it via branchless testing way, that service is applicable to apply for this tag. TC will document this tag in a more clear way and encourage each project to start working on applying this tag. * Action: - Graham to put up a change to update the wording to allow for discovery mechanisms[6] - Should include the release in which they started to assert this tag. - also documents that it's about following the guidelines for future changes not existing one. - gmann - Should socialize this out to the projects after we get the documentation improvements landed. - gmann k8s Community Bridging : -------------------------------- This is one of the exciting discussion for everyone. We hosted this cross-community meeting with Kubernetes steering committee teams. Bob and dims from the Kubernetes steering committee joined us in this discussion. It was started with a quick introduction from both teams and then started the discussion on the below topics [11] * Sharing governance models: k8s governance hierarchy is LF->CNCF->k8s steering commitee->various SIG/working group. k8s Steering the committee consists of 7 elected members and doesn't actually influence the direction of the code instead leave it to SIG and arch committee. There is no chair in this committee and host biweekly private as well as public meetings. Each SIG (repos) team has approver and reviewer roles where reviewer review the code and approver are responsible for merging the code. Naser explained the OpenStack governance model. * What are your 3 biggest issues that are barriers to growth/sustainability as a community? as a project? - Going up the contribution ladder is hard. - Stale/emeritus membership in reviewer/approver level groups. - Mono repo makes it so that each SIG involved needs to review which can slow things down. * Area/Domain mapping vs service/repo centered teams - mono repo forces people to work together and it's a little more diverse who is reviewing - mono repo is bad because sometimes things need to be in multiple places like documentation - mono repo struggles with libraries especially on testing the external one. * Related Ecosystem projects comparisons The SIG lead has to sign off and accept that they are willing to take ownership + handle maintenance + releasing etc. It is general consensus that we try to distribute much work as possible to the subgroup and keep it out of k/k. There is general consensus across k8s leadership that work should be delegated out to subgroups. For API interoperability challenge in a distributed model, k8s has conformance tests which exercise the API performance and vendors try and upload conformance results every release or dot release. * Common FOSS topics release and CI part was discussed. and also on how COVID things are impacting community health. k8s community almost lost their independents or part-time contributors. also, the k8s community is doing 3 releases per year compared to 4. * Staying connected going forward To stay connected it will be a good idea if we extend an invite for K8s to join PTG. Upstream Investment Opportunities: ----------------------------------------------- We have three upstream opportunities defined for 2020[1] but there is no help on any of these, even in previous (2018, 2019) upstream opportunities also. We started the discussion if we need to continue this for 2021 years also or just stop defining it and decide the area to help when we have someone interested. Before deciding anything on this mnaser will discuss this with the board of directors and get their opinion. * Action Items: - mnaser to find ways to engage with members and talk with them. - mnaser to talk to board about infrastructure - mnaser to talk to board about platinum members contribution levels + what are they doing to drive engagement. Pop Up Team Check In: ----------------------------- Currently, we have two active popup teams 1. policy 2. Encryption. TC checked the progress on both teams. The policy team is very active and finished some work in Voctoria cycle (cyborg finished it and Barbican started) and also hosted forum[8], PTG sessions[9], and discussed Wallaby development plan. This team host biweekly meeting to discuss and review the progress. The encryption team is also active. Josephine explained the progress on this. Glance spec is merged. Both teams will continue in Wallaby cycle also. Completing the Gerrit breach audit : ---------------------------------------------- There are still 19 teams pending to finish this audit. Those are listed in etherpad - https://etherpad.opendev.org/p/code-audit-gerrit-breach-tracker We encourage all those pending teams to finish the audit, TC members will start following with those projects every week. * Action: - TC to follow up every week on progress Other topics: ----------------- We ran out of time and all the pending (below) topics will be discussed in TC regular meetings. TC will skip this month's meeting[10] but will have weekly meetings on 12th Nov onwards. *Better way to test defined testing runtime *Monitoring in OpenStack: Ceilometer + Telemetry + Gnocchi state *Stable core team maintainer and process to recruit the new members and adding core team members in stable core list. Detailed discussions are in this Etherpad: https://etherpad.opendev.org/p/tc-wallaby-ptg [1] https://review.opendev.org/759904 [2] https://review.opendev.org/#/c/755590/ [3] https://etherpad.opendev.org/p/policy-popup-wallaby-ptg [4] https://review.opendev.org/#/c/759881/ [5] http://lists.openstack.org/pipermail/openstack-discuss/2020-October/017786.… [6] https://review.opendev.org/760562 [7] https://governance.openstack.org/tc/reference/upstream-investment-opportuni… [8] https://etherpad.opendev.org/p/consistent-and-secure-default-policies-walla… [9] https://etherpad.opendev.org/p/policy-popup-wallaby-ptg [10] http://lists.openstack.org/pipermail/openstack-discuss/2020-October/018439.… [11] https://etherpad.opendev.org/p/kubernetes-cross-community-topics -gmann

1 0

devstack installation error - [qa-octavia]
by Ratnopam Chakrabarti 31 Oct '20

31 Oct '20

I tried installing devstack in ubuntu 20.04 with no luck. Are the instructions > at https://docs.openstack.org/devstack/latest/ and > https://docs.openstack.org/devstack/latest/guides/devstack-with-lbaas-v2.ht… > still valid? Could anyone please confirm?

2 3

Weird iptables problem
by frommel＠gmail.com 31 Oct '20

31 Oct '20

Hi, so we deployed a test cluster with 8 total nodes. Openstack ussuri, Ubuntu 20.04 , opevswitch, octavia. Everything works more or less as it should , however when deploying a loadbalancer (which works perfectly), instances on the same node as the amphora deployment loose all access via public up after open switch logs a message : modified security group “long uuid”, which is a sec group belonging to octavia. What effect does that have with the rest? Questions: Should ufw be on and started? How can we troubleshoot this more in depth? Why does the lb work perfectly whole instance with floating up loos all access from the internet but the lb works fine passing thru the web services. One more thing. Does anyone have a recommended sysctl.conf Settings list for Ubuntu/ openstack ussuri? Thanks already and have a nice weekend, //f

1 0

[magnum] Keystone trust credentials failing authentication (after previously working)
by Paul Browne 30 Oct '20

30 Oct '20

Hello Magnum users and devs, I was wondering if any may be able to provide some clues on a mystifying problem we've been seeing with some Magnum-deployed k8s clusters, of older deployment date (v1.17, deployed ~6 months ago) and Keystone trusts used to manage OpenStack cloud resources. I've attached a copy of the Magnum template for the k8s cluster to give an idea of its initial environment. The issue we've been seeing is that the Keystone trust generated at Magnum cluster creation time seems no longer be usable to successfully authenticate to the OpenStack APIs, and so operations such as creation of Manila shares used as k8s Persistent Volume Claims then get stuck in Pending state, forever. The strange thing is that this doesn't happen for newly created Magnum clusters of the same template (in attached file, I use trust credentials for a newly deployed and old cluster; OpenStack API calls for new credentials succeed, whereas old cluster trust fails). Unfortunately following debug-level Keystone logging for rejected trust auth attempts hasn't led me much further to understanding the root issue; the Magnum generated trust exists and has no expiry, but it still seems to be rejected. Keystone debug logging doesn't seem to give much indication as to *why* it is being rejected or what policy violations may be involved, but perhaps there's details hidden in there I'm not seeing yet. Does anyone have any clues as to why this may be happening, or any advice on how we may be able to replace or refresh the trust that the Magnum-created k8s cluster is expecting to use in calling OpenStack APIs? Many thanks, Paul -- ******************* Paul Browne Research Computing Platforms University Information Services Roger Needham Building JJ Thompson Avenue University of Cambridge Cambridge United Kingdom E-Mail: pfb29(a)cam.ac.uk Tel: 0044-1223-746548 *******************

1 0