- openstack-discuss - lists.openstack.org

On pre-commit, black, ruff, ruff-format and everything else $linter related
by Stephen Finucane 13 Sep '24

13 Sep '24

o/ Seeing as we've been doing quite a bit of experimentation on this topic in the SDK team recently, and it's come up in random discussions a few times over recent days, I thought it might be helpful to share a few tips and tricks on how our linting stored has evolved across SDK deliverables. Hopefully it's helpful to someone. Historically, teams have provided a `pep8` tox target to run their linters. Originally this would have run flake8 or the hacking wrapper, but over the years these targets have expanded to cover other linters like bandit, doc8 and mypy, as well as various flake8 plugins. In recent times, we have been slowly introducing use of the pre-commit tool [1]. Initially this was purely additive, as it's a nice help for users working in a terminal (like me!) and mostly duplicated what was defined in the `pep8` target. This duplication led to issues where pre-commit was doing slightly different things from the `pep8` target, so we have evolved this so that the `pep8` target now simply calls pre-commit. This is pretty easy to do now since most if not all of the linters in common use across OpenStack - flake8, hacking, doc8, bandit, etc. - provides pre-commit hooks. I recommend anyone using pre-commit does the same thing otherwise these thing can drift. For example, your pep8 target can become: [testenv:pep8] description = Run style checks. skip_install = true deps = pre-commit commands = pre-commit run --all-files --show-diff-on-failure We've since evolved this further to start changing the linters we've used in general. We first introduced black [2], the automatic code formatting tool. The rationale behind this was to remove the need to even think about code style and focus on the actual semantic change. We're already overburdened, and going back and forth on style issues is a waste of everyone's time. Contributors (particularly new ones) can simply install pre-commit or use the `pep8` tox target and their code will be automatically restyled in a generally non- offensive manner. Is it perfect? No, but it's close enough that we've decided we don't care. When we have introduced black, we have typically kept the changes introduced by black separate from the changes to e.g. the `.pre-commit- config.yaml` or `tox.ini` file. This allows us to introduce a `.git-blame- ignore-revs` file afterwards which some software forges (like GitHub, though sadly not Gittea (yet!)) can use to ignore these bulk update changes when using `git-blame`. You can also configure git locally to ignore this, and I've made an effort to place comments in the files whenever I've added them. In addition to black, we also added other linters like bandit [3] and pyupgrade [4], including these in pre-commit so that they would run automatically on commit and in CI. However, as anyone reading the orange site or surveying the broader Python landscape might be aware, the ruff [5] linter and formatter has been gaining steam and is pretty mature at this point. In recent weeks, we have been migrating pretty much all our linters and formatters - black, pyupgrade, bandit and (most of) flake8 - to ruff and ruff-format. We have kept hacking but only enable the H (hacking) ruleset plus any custom rules for now, but at some point we will explore getting these few hacking rules into ruff or writing them off. In any case, this migration has resulted in a significantly trimmer pre- commit-config.yaml file (and much faster runtime to boot). You can see the end result on a repo like e.g. osc-lib [6]. All in all, I think I can recommend this approach for anyone interested in pursuing it. Linters are far from the most important aspect of development, but I've found these tools to yield a nice improvement in developer experience. Top tips: * Have your 'pep8' target (and any other linter-related target run 'pre-commit' rather than the linters themselves), and don't forget to remove linters from test-requirements.txt once you do (with the exception of hacking if you have custom hacking rules) * Consider using ruff and ruff-format in place of all non-hacking linters/formatters: it's seriously fast and provides rules to cover most of these. * When applying ruff-format or the UP (pyupgrade) ruff rules, do it separately so you can add the commit ID to a .git-blame-ignore-revs file. You should be able to do this pre-merge since Gerrit/Zuul merge patches rather than rebasing them so the commit ID won't change. Other points: * When we first introduced pre-commit, some people noted that installing from Git rather than PyPI introduced a possible attack vector since tags can be changed. Given we only use a small, curated set of linters from trusted sources, I don't know how much of an issue this is but I'll raise it here. * Difficulty backporting code has been highlighted as an blocker for running a formatter like black or ruff-format. We don't do many backports in SDK, OSC etc. so this hasn't been a huge issue. If it were though, we would simply apply the formatter to the stable branch first before backporting. Hope this helps, Stephen [1] https://pre-commit.com/ [2] https://github.com/psf/black [3] https://github.com/PyCQA/bandit [4] https://github.com/asottile/pyupgrade [5] https://github.com/astral-sh/ruff [6] https://opendev.org/openstack/osc-lib/

3 3

[horizon] Weekly meeting cancelled Sept 18
by Tatiana Ovchinnikova 12 Sep '24

12 Sep '24

Hello team, I'm on PTO next week, so I'm cancelling the weekly meeting on September 18. See you on September 25! Regards, Tatiana

1 0

[kolla] Proposing Sven Kieske as Kolla and Kolla-Ansible core
by Michał Nasiadka 12 Sep '24

12 Sep '24

Hello Koalas, I would like to propose Sven Kieske as a core reviewer for Kolla and Kolla-Ansible Sven has been very active and providing insightful feedback to reviews as well as code contributions. If I won’t receive any negative votes until 11th of September - I’ll add Sven in Gerrit on the Kolla weekly meeting on that day. Thanks! Michał Nasiadka mnasiadka(a)gmail.com

7 7

[API] Gitea API return 404
by tzing0013＠gmail.com 11 Sep '24

11 Sep '24

Hi everyone， I'm testing Gitea API, I visit the web[1], and click "Try is out", no matter what I enter there, it returns 404, and I also test "/contents" [2], that's ok. [1] https://opendev.org/api/swagger#/repository/repoListPullRequests [2] https://opendev.org/api/swagger#/repository/repoGetContentsList

2 4

[openstack-ansible]: Move one control node to different chasis
by Ram Ram 10 Sep '24

10 Sep '24

Hi All We are currently running 3 control nodes with LXC containers deployed using openstack-ansible release ussuri and need to move 1 control node on HP blade from one chasis to another chasis. Is there any recommended way/procedure to do that? can we directly move the HP blade from one chasis to another? or we need to install new control node and decommission the older one? Regards Rambo

3 3

[kolla][vpn-as-a-service] docker container missing
by garcetto 10 Sep '24

10 Sep '24

good afternoon, installed all-in-one ubuntu 22.04 based kolla-ansible version 2024.1, so far so good. tried to enable vpnaas, folllowing the instructions here https://docs.openstack.org/kolla-ansible/2024.1/reference/networking/neutro… but still missing the container "neutron_vpnaas_agent". i checked the ansible playbook but see no evidence of deployment of this container, what am i doing wrong? thank you.

3 2

[charms] Proposing Rodrigo Barbieri for OpenStack Charms stable core
by Edward Hope-Morley 10 Sep '24

10 Sep '24

Hi everyone, I'd like to propose Rodrigo Barbieri for the OpenStack Charms stable maintenance team. Rodrigo has been a contributor to the charms for many years and has an impressive history of contributions to across all the OpenStack charms. He always demonstrates an attention to detail and provides insightful feedback in reviews and as such I believe he would make a valuable addition to our charms-stable maintenance core team. Thank you for your contributions to the project Rodrigo! Regards, Ed

2 1

Can only live migrate between hypervisors with same kernel
by sinan86polat＠gmail.com 10 Sep '24

10 Sep '24

Is it normal behavior that instances can only be live migrated between hypervisors running the same kernel? Like this I am not able to patch my hypervisors without downtime on the instances. When I live migrate all instances from hypervisor A to hypervisor B, patch hypervisor A, I cannot migrate the instances back to hypervisor A since it is running a newer kernel. The error is: guest CPU doesn't match specification: missing features: xsaves My source and target hypervisors have both the same CPU. Running OpenStack Rocky.

4 3

Clarification on Custom Role and Sub-User Management in OpenStack
by Thamanna Farhath 10 Sep '24

10 Sep '24

Dear OpenStack Community, I hope this message finds you well. I am reaching out to seek clarification and guidance on a scenario involving custom roles and sub-user management within OpenStack. In our deployment, we have the standard admin role as well as a custom role called super_user, which grants privileges for CRUD operations on projects, role assignments, and user management. We currently have two users with the super_user role, named Alice and John. Each of these super_user accounts manages their own sub-users: Alice manages the ABC sub-users, and John manages the XYZ sub-users. I would like to implement a restriction where Alice, as a super_user, can only manage the A,B,C sub-users and should not be able to delete or manage John's X,Y,Z sub-users. Similarly, John should not be able to manage or delete Alice's A,B,C sub-users. My questions are as follows: How can I identify and ensure that ABC sub-users belong to Alice and X,Y,Z sub-users belong to John? What is the best approach to implement this restriction within OpenStack, preventing cross-management of sub-users between Alice and John? Will using the multi-domain concept help me achieve this? Any insights, best practices, or suggestions on how to configure this within OpenStack would be greatly appreciated. Thank you for your assistance and support. Thanks & Regards, Thamanna Farhath | Associate Engineer-R&D Ph No: (+91) 9344093591 email: mailto:thamanna.f@zybisys.com Zybisys Consulting LLP | NO.1B 2nd floor, NSA Tower, Akila Nagar, Main Rd, Ganapathy Nagar, Thiruvanaikoil, Tiruchirappalli, Tamil Nadu 620005 zybisys.com Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

1 0

[swift] Ideas: Object Locking
by David Zanetti 10 Sep '24

10 Sep '24

Hello everyone! We've added an entry to the Swift ideas page about implementing Object Locks. The full description of what this would be and how we expect it to work is in the ideas page here: https://wiki.openstack.org/wiki/Swift/ideas/object-lock I have two questions of the community. The first is whether work on this would be accepted by the Swift project, as we wouldn't want to expend efforts on something we'd have to carry as our own patches to upstream. The second is if there is any interest from other people to collaborate on design/implementation of the locking feature. Happy to take replies off-list if people prefer, and thanks for reading this far! (Ob-reply: Yes, we know Ceph RGW does this already. We use Swift for Object for $REASONS, which we'd like to keep doing.) -- David Zanetti Chief Technology Officer [1]Aotearoa New Zealand's cloud provider e david.zanetti(a)catalystcloud.nz w catalystcloud.nz Follow us on LinkedIn [2] Level 9, 33 Federal Street, Auckland 1010 [1] https://catalystcloud.nz/ [2] Follow us on LinkedIn https://www.linkedin.com/company/catalyst-cloud-limited/

2 1