Those of you who work on a fairly large project will have noticed that running flake8 over all of it takes some time, and that this slows down development.
Nova (at least) has a solution to this, in the form of a "fast8" tox environment that runs flake8 only against the files that have changed in the latest patch + the working directory. This is *much* faster, but that approach has some limitations: the script is buggy, it only tests the top-most patch, it creates a second tox environment (which is slow) that can then get out of sync with your regular pep8 environment, and of course it requires the project to add it explicitly.
If you're interested in a solution with none of those limitations, here is a script that I've been using:
https://gist.github.com/zaneb/7a8c752bfd97dd8972756d296fc5e41f
It tests all changes on the branch, using your existing pep8 tox environment, handles deleted files and changes to non-python files correctly, and should be usable for every OpenStack project.
I hope this is helpful to someone.
(Note that the pep8 environment on many projects includes other test commands in addition to flake8 - such as bandit - so you should still run the pep8 tox tests once before submitting a patch.)
cheers, Zane.
On Thu, 2019-06-20 at 13:37 -0400, Zane Bitter wrote:
Those of you who work on a fairly large project will have noticed that running flake8 over all of it takes some time, and that this slows down development.
Nova (at least) has a solution to this, in the form of a "fast8" tox environment that runs flake8 only against the files that have changed in the latest patch + the working directory. This is *much* faster, but that approach has some limitations: the script is buggy, it only tests the top-most patch, it creates a second tox environment (which is slow) that can then get out of sync with your regular pep8 environment, and of course it requires the project to add it explicitly.
If you're interested in a solution with none of those limitations, here is a script that I've been using:
https://gist.github.com/zaneb/7a8c752bfd97dd8972756d296fc5e41f
Neat :) There's also the opportunity of integrating flake8 (and other things) as a pre-commit hook, which is something I'm trying to adopt within nova and the maybe oslo and further over time.
http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007151.html
That requires some project-level work though (including backports, if you want it on stable branches) whereas your script can be used anywhere. Both useful.
Stephen
It tests all changes on the branch, using your existing pep8 tox environment, handles deleted files and changes to non-python files correctly, and should be usable for every OpenStack project.
I hope this is helpful to someone.
(Note that the pep8 environment on many projects includes other test commands in addition to flake8 - such as bandit - so you should still run the pep8 tox tests once before submitting a patch.)
cheers, Zane.
Yeah, use pre-commit, the tool, not the git hook. At this moment it just rocks and was adopted by more and more openstack projects.
See http://codesearch.openstack.org/?q=pre-commit&i=nope&files=.pre-comm... http://codesearch.openstack.org/?q=pre-commit&i=nope&files=.pre-commit-config.yaml&repos= for examples of projects using it (tox -e lint)
Mainly pre-commit does something similar but with almost any linter, including flake8.
Also has the ability to run only on changed files, runs nice under tox and has a much lower footprint (disk&cpu) than tox.
And a bit of shameless advertising about my 1year old article explaining it: https://medium.com/@sbarnea/embracing-pre-commit-hooks-4ef1f4e72914 https://medium.com/@sbarnea/embracing-pre-commit-hooks-4ef1f4e72914
On 21 Jun 2019, at 17:47, Stephen Finucane sfinucan@redhat.com wrote:
On Thu, 2019-06-20 at 13:37 -0400, Zane Bitter wrote:
Those of you who work on a fairly large project will have noticed that running flake8 over all of it takes some time, and that this slows down development.
Nova (at least) has a solution to this, in the form of a "fast8" tox environment that runs flake8 only against the files that have changed in the latest patch + the working directory. This is *much* faster, but that approach has some limitations: the script is buggy, it only tests the top-most patch, it creates a second tox environment (which is slow) that can then get out of sync with your regular pep8 environment, and of course it requires the project to add it explicitly.
If you're interested in a solution with none of those limitations, here is a script that I've been using:
https://gist.github.com/zaneb/7a8c752bfd97dd8972756d296fc5e41f
Neat :) There's also the opportunity of integrating flake8 (and other things) as a pre-commit hook, which is something I'm trying to adopt within nova and the maybe oslo and further over time.
http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007151.html http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007151.html
That requires some project-level work though (including backports, if you want it on stable branches) whereas your script can be used anywhere. Both useful.
Stephen
It tests all changes on the branch, using your existing pep8 tox environment, handles deleted files and changes to non-python files correctly, and should be usable for every OpenStack project.
I hope this is helpful to someone.
(Note that the pep8 environment on many projects includes other test commands in addition to flake8 - such as bandit - so you should still run the pep8 tox tests once before submitting a patch.)
cheers, Zane.
participants (3)
-
Sorin Sbarnea
-
Stephen Finucane
-
Zane Bitter