December 2020 - openstack-discuss

[all][interop] Reforming the refstack maintainers team
by Ghanshyam Mann 12 Feb '21

12 Feb '21

Hello Everyone, As Goutham mentioned in a separate ML thread[2] that there is no active maintainer for refstack repo which we discussed in today's interop meeting[1]. We had a few volunteers who can help to maintain the refstack and other interop repo which is good news. I would like to call for more volunteers (new or existing ones), if you are interested to help please do reply to this email. The role is to maintain the source code of the below repos. I will propose the ACL changes in infra sometime next Friday (18th dec) or so. For easy maintenance, we thought of merging the below repo core group into a single group called 'refstack-core' - openstack/python-tempestconf - openstack/refstack - openstack/refstack-client - x/ansible-role-refstack-client (moving to osf/ via https://review.opendev.org/765787) Current Volunteers: - martin (mkopec(a)redhat.com) - gouthamr (gouthampravi(a)gmail.com) - gmann (gmann(a)ghanshyammann.com) - Vida (vhariria(a)redhat.com) - interop-core (we will add this group also which has interop WG chairs so that it will be easy to maintain in the future changes) NOTE: there is no change in the 'interop' repo group which has interop guidelines and doc etc. [1] https://etherpad.opendev.org/p/interop [2] http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019263… -gmann

8 12

[largescale-sig] OpenStack DB Archiver
by Arnaud Morin 10 Feb '21

10 Feb '21

Hello large-scalers! TLDR: we opensource a tool to help reducing size of databases. See https://github.com/ovh/osarchiver/ Few months ago, we released a tool, name osarchiver, which we are using on our production environment (at OVH) to help reduce the size of our tables in mariadb (or mysql) In fact, some tables are well know to grow very quickly. We use it, for example, to clean the OpenStack mistral database from old tasks, actions and executions which are older than a year. Another use case could be to archive some data in another table (e.g. with _archived as suffix) if they are 6 months old, and delete this data after 1 year. The source code of this tool is available here: https://github.com/ovh/osarchiver/ We were wondering if some other users would be interested in using the tool, and maybe move it under the opendev governance? Feel free to contact us and/or answer this thread. Cheers, -- Arnaud, Pierre-Samuel and OVH team

12 18

bindep, pacman, and PyPI release
by Jakob Lykke Andersen 06 Feb '21

06 Feb '21

Dear developers, I'm trying to use bindep on Arch but hitting a problem with the output handling. It seems that 'pacman -Q' may output warning lines if you have a file with the same name as the package. E.g.,: $ mkdir cmake $ pacman -Q cmake error: package 'cmake' was not found warning: 'cmake' is a file, you might want to use -p/--file. The current detection assumes the "was not found" is the last part of the output. Applying the patch below seems to fix it. After applying this patch, or whichever change you deem reasonable to fix the issue, it would be great if you could make a new release on PyPI. Thanks, Jakob diff --git a/bindep/depends.py b/bindep/depends.py index bb9553f..cacc863 100644 --- a/bindep/depends.py +++ b/bindep/depends.py @@ -558,7 +558,8 @@ class Pacman(Platform): stderr=subprocess.STDOUT).decode(getpreferredencoding(False)) except subprocess.CalledProcessError as e: eoutput = e.output.decode(getpreferredencoding(False)) - if e.returncode == 1 and eoutput.strip().endswith('was not found'): + s = "error: package '{}' was not found".format(pkg_name) + if e.returncode == 1 and s in eoutput: return None raise # output looks like

2 2

[magnum] [neutron] [ovn] No inter-node pod-to-pod communication due to missing ACLs in OVN
by Krzysztof Klimonda 04 Feb '21

04 Feb '21

Hi, This email is a follow-up to a discussion I've openened on ovs-discuss ML[1] regarding lack of TCP/UDP connectivity between pods deployed on magnum-managed k8s cluster with calico CNI and IPIP tunneling disabled (calico_ipv4pool_ipip label set to a default value of Off). As a short introduction, during magnum testing in ussuri deployment with ml2/ovn neutron driver I've noticed lack of communication between pods deployed on different nodes as part of magnum deployment with calico configured to *not* encapsulate traffic in IPIP tunnel, but route it directly between nodes. In theory, magnum configures adds defined pod network to k8s nodes ports' allowed_address_pairs[2] and then security group is created allowing for ICMP and TCP/UDP traffic between ports belonging to that security group[3]. This doesn't work with ml2/ovn as TCP/UDP traffic between IP addresses in pod network is not matching ACLs defined in OVN. I can't verify this behaviour under ml2/ovs for the next couple of weeks, as I'm taking them off for holidays, but perhaps someone knows if that specific usecase (security group rules with remote groups used with allowed address pairs) is supposed to be working, or should magnum use pod network cidr to allow traffic between nodes instead. [1] https://mail.openvswitch.org/pipermail/ovs-discuss/2020-December/050836.html [2] https://github.com/openstack/magnum/blob/c556b8964fab129f33e766b1c33908b2eb… [3] https://github.com/openstack/magnum/blob/c556b8964fab129f33e766b1c33908b2eb… -- Best Regards, - Chris

5 9

[tc][all] Encouraging projects to apply for tag 'assert:supports-api-interoperability'
by Ghanshyam Mann 30 Jan '21

30 Jan '21

Hello Everyone, TC defined a tag for API interoperability (cover both stable and compatible APIs) called 'assert:supports-api-interoperability' which assert on API won’t break any users when they upgrade a cloud or start using their code on a new OpenStack cloud. Basically, Projects will not change (or remove) an API in a way that will break existing users of an API. We have updated the tag documentation to clarify its definition and requirements. If your projects follow the API interoperability guidelines[1] and some API versioning mechanism that does not need to be microversion then you should start thinking to apply for this tag. The complete requirement can be found here[2]. Currently, only nova has this tag but I am sure many projects are eligible for this, and TC encourage them to apply for this. [1] https://specs.openstack.org/openstack/api-wg/guidelines/api_interoperabilit… [2] https://governance.openstack.org/tc/reference/tags/assert_supports-api-inte… -gmann

2 3

Secure RBAC work
by Lance Bragstad 27 Jan '21

27 Jan '21

Hey everyone, I wanted to take an opportunity to clarify some work we have been doing upstream, specifically modifying the default policies across projects. These changes are the next phase of an initiative that’s been underway since Queens to fix some long-standing security concerns in OpenStack [0]. For context, we have been gradually improving policy enforcement for years. We started by improving policy formats, registering default policies into code [1], providing better documentation for policy writers, implementing necessary identity concepts in keystone [2], developing support for those concepts in libraries [3][4][5][6][7][8], and consuming all of those changes to provide secure default policies in a way operators can consume and roll out to their users [9][10]. All of this work is in line with some high-level documentation we started writing about three years ago [11][12][13]. There are a handful of services that have implemented the goals that define secure RBAC by default, but a community-wide goal is still out-of-reach. To help with that, the community formed a pop-up team with a focused objective and disbanding criteria [14]. The work we currently have in progress [15] is an attempt to start applying what we have learned from existing implementations to other projects. The hope is that we can complete the work for even more projects in Wallaby. Most deployers looking for this functionality won't be able to use it effectively until all services in their deployment support it. I hope this helps clarify or explain the patches being proposed. As always, I'm happy to elaborate on specific concerns if folks have them. Thanks, Lance [0] https://bugs.launchpad.net/keystone/+bug/968696/ [1] https://governance.openstack.org/tc/goals/selected/queens/policy-in-code.ht… [2] https://specs.openstack.org/openstack/keystone-specs/specs/keystone/queens/… [3] https://review.opendev.org/c/openstack/keystoneauth/+/529665 [4] https://review.opendev.org/c/openstack/python-keystoneclient/+/524415 [5] https://review.opendev.org/c/openstack/oslo.context/+/530509 [6] https://review.opendev.org/c/openstack/keystonemiddleware/+/564072 [7] https://review.opendev.org/c/openstack/oslo.policy/+/578995 [8] https://review.opendev.org/q/topic:%22system-scope%22+(status:open%20OR%20s… [9] https://review.opendev.org/q/status:merged+topic:bp/policy-defaults-refresh… [10] https://review.opendev.org/q/topic:%22implement-default-roles%22+(status:op… [11] https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ongoing… [12] https://docs.openstack.org/keystone/latest/admin/service-api-protection.html [13] https://docs.openstack.org/keystone/latest/contributor/services.html#author… [14] https://governance.openstack.org/tc/reference/popup-teams.html#secure-defau… [15] https://review.opendev.org/q/topic:%2522secure-rbac%2522+status:open

6 10

[all] CI test result table in the new gerrit review UI
by Balázs Gibizer 27 Jan '21

27 Jan '21

Hi, I understand that adapting the old CI test result table to the new gerrit review UI is not a simple task. As a temporary measure I hacked together a tampermonkey user script[1] that does a limited version of the CI table injection from the browser side. I tested it with recent chrome and firefox. Let me know if you have problems with it and I'll try to help. Also feel free to adapt it to your own needs. Cheers, gibi [1] https://gist.github.com/gibizer/717e0cb5b0da0d60d5ecf1e18c0c171a

15 34

[all][tc][goals] Migrate RBAC Policy Format from JSON to YAML: Week R-16 Update
by Ghanshyam Mann 21 Jan '21

21 Jan '21

Hello Everyone, Please find the week's R-16 updates on 'Migrate RBAC Policy Format from JSON to YAML' community-wide goals. Tracking: https://etherpad.opendev.org/p/migrate-policy-format-from-json-to-yaml Gerrit Topic: https://review.opendev.org/q/topic:%22policy-json-to-yaml%22+(status:open%2… Progress: ======= * Projects completed: 5 * Projects left to merge the patches: 25 * Projects left to push the patches: 2 (horizon and Openstackansible) * Projects do not need any work: 17 Updates: ======= * I have pushed the patches for all the required service projects. ** Because of many services gate is already broken for lower constraints job, these patches might not be green in the test results. I request projects to fix the gate so that we can merge this goal work before m-2. ** There are many project tests where CONF object was not fully initialized before the policy is init. This was working till now as policy init did not use the CONF object but oslo_policy 3.6.0 onwards it needs fully initialized CONF object during init only. ** Aodh work for this goal is blocked because it needs oslo_policy 3.6.0 but gnocchi is capped for oslo_policy 3.4.0 [1] - https://review.opendev.org/c/openstack/aodh/+/768499 * Horizon and Openstackansible work is pending to use/deploy the YAML formatted policy file. I will start exploring this next week or so. [1] https://github.com/gnocchixyz/gnocchi/blob/e19fda590c7f7f07f1df0ba93177df07… Merry Christmas and Happy Holidays! -gmann

6 11

[TripleO] moving stable/rocky for tripleo repos to unmaintained (+ then EOL) OK?
by Marios Andreou 20 Jan '21

20 Jan '21

Hello TripleO I would like to propose that we move all tripleo stable/rocky repos [1] to "unmaintained", with a view to tagging as end-of-life in due course. This will allow us to focus our efforts on keeping the check and gate queues green and continue to deliver weekly promotions for the more recent and active stable/* branches train ussuri victoria and master. The stable/rocky repos have not had much action in the last few months - I collected some info at [2] about the most recent stable/rocky commits for each of the tripleo repos. For many of those there are no commits in the last 6 months and for some even longer. The tripleo stable/rocky repos were tagged as "extended maintenance" (rocky-em) [2] in April 2020 with [3]. We have already reduced our CI commitment for rocky - these [4] are the current check/gate jobs and these [5] are the jobs that run for promotion to current-tripleo. However maintaining this doesn’t make sense if we are not even using it e.g. merging things into tripleo-* stable/rocky. Please raise your objections or any other comments or thoughts about this. Unless there are any blockers raised here, the plan is to put this into motion early in January. One still unanswered question I have is that since there is no ‘unmaintained’ tag, in the same way as we have the <release>-em or <release-eol> for extended maintenance and end-of-life, do we simply _declare_ that the repos are unmaintained? Then after a period of “0 to 6 months” per [6] we can tag the tripleo repos with rocky-eol. If any one reading this knows please tell us! Thanks for reading! regards, marios [1] https://releases.openstack.org/teams/tripleo.html#rocky [2] http://paste.openstack.org/raw/800464/ [3] https://review.opendev.org/c/openstack/releases/+/709912 [4] http://dashboard-ci.tripleo.org/d/3-DYSmOGk/jobs-exploration?orgId=1&var-in… [5] http://dashboard-ci.tripleo.org/d/3-DYSmOGk/jobs-exploration?orgId=1&fullsc… [6] https://docs.openstack.org/project-team-guide/stable-branches.html#maintena…

3 5

[ops][nova][designate] Does anyone rely on fully-qualified instance names?
by Stephen Finucane 14 Jan '21

14 Jan '21

When attaching a port to an instance, nova will check for DNS support in neutron and set a 'dns_name' attribute if found. To populate this attribute, nova uses a sanitised version of the instance name, stored in the instance.hostname attribute. This sanitisation simply strips out any unicode characters and replaces underscores and spaces with dashes, before truncating to 63 characters. It does not currently replace periods and this is the cause of bug 1581977 [1], where an instance name such as 'ubuntu20.04' will fail to schedule since neutron identifies '04' as an invalid TLD. The question now is what to do to resolve this. There are two obvious paths available to us. The first is to simply catch these invalid hostnames and replace them with an arbitrary hostname of format 'Server-{serverUUID}'. This is what we currently do for purely unicode instance names and is what I've proposed at [2]. The other option is to strip all periods, or rather replace them with hyphens, when sanitizing the instance name. This is more predictable but breaks the ability to use the instance name as a FQDN. Such usage is something I'm told we've never supported, but I'm concerned that there are users out there who are relying on this all the same and I'd like to get a feel for whether this is the case first. So, the question: does anyone currently rely on this inadvertent "feature"? Cheers, Stephen [1] https://launchpad.net/bugs/1581977 [2] https://review.opendev.org/c/openstack/nova/+/764482

9 23