[hacking][all] Expanding set of import exceptions
o/ I've been slowly adding type hints to oslo and SDK projects of late as time has allowed. One thing that I've noticed is that strict adherence to hacking's H301 (Do not import more than one module per line) and H303 (Do not use wildcard * import) rules results in both stupidly long groups of imports and overly verbose and difficult to read signatures [1]. I've therefore proposed a change to these rules that exempt some typing-related libraries - collections.abc, types, and typing - from these rules. This is proposed at [2]. As noted there, exceptions already exist for the sqlalchemy and projects' i18n modules for the same reason so this should be nothing new. I am instead bringing this up so projects currently using e.g. 'import typing as ty' imports can pivot to something sooner once this is merged. Cheers, Stephen [1] If you've already started writing a reply about how type hints *themselves* make function signatures more verbose and difficult to read, (a) is your name Sean Mooney and (b) stop right there, go enable an LSP in your editor, then come back to me in a month once you've seen the quality of life improvement hints provide. Then we'll talk 😄 [2] https://review.opendev.org/c/openstack/hacking/+/967719
On Wed, Nov 19, 2025, at 8:59 AM, Stephen Finucane wrote:
o/
I've been slowly adding type hints to oslo and SDK projects of late as time has allowed. One thing that I've noticed is that strict adherence to hacking's H301 (Do not import more than one module per line) and H303 (Do not use wildcard * import) rules results in both stupidly long groups of imports and overly verbose and difficult to read signatures [1]. I've therefore proposed a change to these rules that exempt some typing-related libraries - collections.abc, types, and typing - from these rules. This is proposed at [2]. As noted there, exceptions already exist for the sqlalchemy and projects' i18n modules for the same reason so this should be nothing new. I am instead bringing this up so projects currently using e.g. 'import typing as ty' imports can pivot to something sooner once this is merged.
Isn't the concern with H303 that using wildcard imports makes you susceptible to unexpected name collisions either due to updates in your imports or changes to your local code? The example in [2] actually points at an extremely common case of this: `_`. A single underscore is commonly used as a throwaway result in code like: result, _ = some_fun() Which if you expect `_()` to perform gettext actions could suddenly become problematic. Type checking probably does alleviate some of this as it may be able to detect where `_` is no longer a function. Similarly if you're using some name as a type in a definition and that name has been shadowed the type checker will likely complain. Is it possible to relax the rules only when there is an expectation that we'll catch the rules through another process like type checking? It is also worth noting that the current rules allow us to take advantage of lazy import behaviors when Python adds support for them without making major changes. This is particularly relevant to type checking as I believe one of the benefits of lazy imports is the mitigation of cost for type checking imports and code at runtime. So we may want to be careful making changes here even if we are type checking.
Cheers, Stephen
[1] If you've already started writing a reply about how type hints *themselves* make function signatures more verbose and difficult to read, (a) is your name Sean Mooney and (b) stop right there, go enable an LSP in your editor, then come back to me in a month once you've seen the quality of life improvement hints provide. Then we'll talk 😄
On Wed, 2025-11-19 at 09:49 -0800, Clark Boylan wrote:
On Wed, Nov 19, 2025, at 8:59 AM, Stephen Finucane wrote:
o/
I've been slowly adding type hints to oslo and SDK projects of late as time has allowed. One thing that I've noticed is that strict adherence to hacking's H301 (Do not import more than one module per line) and H303 (Do not use wildcard * import) rules results in both stupidly long groups of imports and overly verbose and difficult to read signatures [1]. I've therefore proposed a change to these rules that exempt some typing-related libraries - collections.abc, types, and typing - from these rules. This is proposed at [2]. As noted there, exceptions already exist for the sqlalchemy and projects' i18n modules for the same reason so this should be nothing new. I am instead bringing this up so projects currently using e.g. 'import typing as ty' imports can pivot to something sooner once this is merged.
Isn't the concern with H303 that using wildcard imports makes you susceptible to unexpected name collisions either due to updates in your imports or changes to your local code? The example in [2] actually points at an extremely common case of this: `_`. A single underscore is commonly used as a throwaway result in code like:
result, _ = some_fun()
Which if you expect `_()` to perform gettext actions could suddenly become problematic. Type checking probably does alleviate some of this as it may be able to detect where `_` is no longer a function. Similarly if you're using some name as a type in a definition and that name has been shadowed the type checker will likely complain.
Sorry, Clark: I meant to call out the 'Do not import objects, only modules (*)' rule (which doesn't actually have a hacking (the tool) rule associated with since it would presumably be very difficult/impossible to enforce). I agree with the '[H303] Do not use wildcard * import (*)' rule and wildcard imports are actually discourage from a typing perspective.
Is it possible to relax the rules only when there is an expectation that we'll catch the rules through another process like type checking?
It is also worth noting that the current rules allow us to take advantage of lazy import behaviors when Python adds support for them without making major changes. This is particularly relevant to type checking as I believe one of the benefits of lazy imports is the mitigation of cost for type checking imports and code at runtime. So we may want to be careful making changes here even if we are type checking.
Yes, for expensive imports you can work around this by placing imports behind the 'TYPE_CHECKING' guard, e.g.: if TYPE_CHECKING: from collections.abc import Sequence But you'd have to either using '__future__.annotations' or wrap these in strings to prevent them being evaluated at runtime. I don't believe the cost of imports of objects from three proposed stdlib libraries is high enough to justify the cognitive load/tooling costs of dealing with deferred evaluation, but we should definitely evaluate whether that's the case for other libraries and modules. I've experimented with doing this for things like libvirt in Nova, for example. Stephen
Cheers, Stephen
[1] If you've already started writing a reply about how type hints *themselves* make function signatures more verbose and difficult to read, (a) is your name Sean Mooney and (b) stop right there, go enable an LSP in your editor, then come back to me in a month once you've seen the quality of life improvement hints provide. Then we'll talk 😄
On 19/11/2025 18:22, Stephen Finucane wrote:
On Wed, 2025-11-19 at 09:49 -0800, Clark Boylan wrote:
On Wed, Nov 19, 2025, at 8:59 AM, Stephen Finucane wrote:
o/
I've been slowly adding type hints to oslo and SDK projects of late as time has allowed. One thing that I've noticed is that strict adherence to hacking's H301 (Do not import more than one module per line) and H303 (Do not use wildcard * import) rules results in both stupidly long groups of imports and overly verbose and difficult to read signatures [1]. I've therefore proposed a change to these rules that exempt some typing-related libraries - collections.abc, types, and typing - from these rules. This is proposed at [2]. As noted there, exceptions already exist for the sqlalchemy and projects' i18n modules for the same reason so this should be nothing new. I am instead bringing this up so projects currently using e.g. 'import typing as ty' imports can pivot to something sooner once this is merged. i dont think we should move away for `import typing as ty` that is the conventional way to use it out side openstack as well
Isn't the concern with H303 that using wildcard imports makes you susceptible to unexpected name collisions either due to updates in your imports or changes to your local code? The example in [2] actually points at an extremely common case of this: `_`. A single underscore is commonly used as a throwaway result in code like:
result, _ = some_fun()
Which if you expect `_()` to perform gettext actions could suddenly become problematic. Type checking probably does alleviate some of this as it may be able to detect where `_` is no longer a function. Similarly if you're using some name as a type in a definition and that name has been shadowed the type checker will likely complain. so the use of _ as a sential value in many languages is very old and the fact that gettext conflicts with that and that the i18n module build on
i also don't believe we should allow expction for h301 h303 or h304 for any module but more on that later. that is a very unfortunate collision. i don't know which is the more common expectation but for me i know of the _ usage as a throwaway value long before i learned of the _() usage. to this day is still fine that surprising and confusing and i wish we had a better way to express that.
Sorry, Clark: I meant to call out the 'Do not import objects, only modules (*)' rule (which doesn't actually have a hacking (the tool) rule associated with since it would presumably be very difficult/impossible to enforce). I agree with the '[H303] Do not use wildcard * import (*)' rule and wildcard imports are actually discourage from a typing perspective. i stongly agree wtih the 'Do not import objects, only modules (*)' rule
i would much rather give up oru import order rule then that. i have spent quite a lot of tiem trying to make ai generate code be a little stricter then hacking currently requires https://github.com/SeanMooney/openstack-ai-style-guide/blob/master/docs/comp... i actually just realized i don't currently have non module imports in my anti-pattern list but its one of the things i explicit review to make sure are not introduced when looking at ai code. i really dislike polluting the current scope with non namespaces symbols form other module. i particularity dislike the example for sqlalchamy given in the commit message as a reason to allow this.
Is it possible to relax the rules only when there is an expectation that we'll catch the rules through another process like type checking?
It is also worth noting that the current rules allow us to take advantage of lazy import behaviors when Python adds support for them without making major changes. This is particularly relevant to type checking as I believe one of the benefits of lazy imports is the mitigation of cost for type checking imports and code at runtime. So we may want to be careful making changes here even if we are type checking. Yes, for expensive imports you can work around this by placing imports behind the 'TYPE_CHECKING' guard, e.g.:
if TYPE_CHECKING: from collections.abc import Sequence
so im not a fan of allowing importing Sequence like that ^ or any non module based import, alias are fine but i do not belive we should start allowing importing classes function constant other non module imports. why not just do ``` import typeing as ty if ty.TYPE_CHECKING: from collections import abc ``` and use abc.Sequence instead of Sequence in the type hints im not a huge fan of the if ty.TYPE_CHECKING pattern but as you note there is a usecase for it today.
But you'd have to either using '__future__.annotations' or wrap these in strings to prevent them being evaluated at runtime.
i would prefer either of those options my only concern with |"from __future__ import annotations" is that now that| it is now deprecate for removal because they the defered sematics are now part of py3.14 as part of https://peps.python.org/pep-0749/ and https://peps.python.org/pep-0649/ 0749 is the deprecation and the commitment to not remove `from __future__ import annotaitons` until 3.13 is eol. if we were to sue that more widely we woudl eventually need to remove it when we raise our min version to 3.14 in 2028
I don't believe the cost of imports of objects from three proposed stdlib libraries is high enough to justify the cognitive load/tooling costs of dealing with deferred evaluation, but we should definitely evaluate whether that's the case for other libraries and modules. I've experimented with doing this for things like libvirt in Nova, for example. i think having any exception to this is increasing the cognitive load. i would actually prefer to remove the current exception not extend them so the opposite of what your proposing in
[2] https://review.opendev.org/c/openstack/hacking/+/967719 i was not aware of the sqlachemy exception until now but its not something i would use in code i write in nova or other project personally.
Stephen
Cheers, Stephen
[1] If you've already started writing a reply about how type hints *themselves* make function signatures more verbose and difficult to read, (a) is your name Sean Mooney and (b) stop right there, go enable an LSP in your editor, then come back to me in a month once you've seen the quality of life improvement hints provide. Then we'll talk 😄
IF you want me not to reply tell me up front :P also i like type hint and want to use them more in all the code we write. they are awesome, they help us find bugs, they help lsp, llm and humans understand what the code is doing with out going on an expedition but i don't see a need to change hacking for them. with that said i am interested to see what other think. i think you need to motivate why we should expand the set of exception or relax the existing rule better
On Wed, 2025-11-19 at 20:41 +0000, Sean Mooney wrote:
On 19/11/2025 18:22, Stephen Finucane wrote:
On Wed, 2025-11-19 at 09:49 -0800, Clark Boylan wrote:
On Wed, Nov 19, 2025, at 8:59 AM, Stephen Finucane wrote:
o/
I've been slowly adding type hints to oslo and SDK projects of late as time has allowed. One thing that I've noticed is that strict adherence to hacking's H301 (Do not import more than one module per line) and H303 (Do not use wildcard * import) rules results in both stupidly long groups of imports and overly verbose and difficult to read signatures [1]. I've therefore proposed a change to these rules that exempt some typing-related libraries - collections.abc, types, and typing - from these rules. This is proposed at [2]. As noted there, exceptions already exist for the sqlalchemy and projects' i18n modules for the same reason so this should be nothing new. I am instead bringing this up so projects currently using e.g. 'import typing as ty' imports can pivot to something sooner once this is merged. i dont think we should move away for `import typing as ty` that is the conventional way to use it out side openstack as well
This isn't true. See end.
Isn't the concern with H303 that using wildcard imports makes you susceptible to unexpected name collisions either due to updates in your imports or changes to your local code? The example in [2] actually points at an extremely common case of this: `_`. A single underscore is commonly used as a throwaway result in code like:
result, _ = some_fun()
Which if you expect `_()` to perform gettext actions could suddenly become problematic. Type checking probably does alleviate some of this as it may be able to detect where `_` is no longer a function. Similarly if you're using some name as a type in a definition and that name has been shadowed the type checker will likely complain. so the use of _ as a sential value in many languages is very old and the fact that gettext conflicts with that and that the i18n module build on
i also don't believe we should allow expction for h301 h303 or h304 for any module but more on that later. that is a very unfortunate collision.
i don't know which is the more common expectation but for me i know of the _ usage as a throwaway value long before i learned of the _() usage. to this day is still fine that surprising and confusing and i wish we had a better way to express that.
Sorry, Clark: I meant to call out the 'Do not import objects, only modules (*)' rule (which doesn't actually have a hacking (the tool) rule associated with since it would presumably be very difficult/impossible to enforce). I agree with the '[H303] Do not use wildcard * import (*)' rule and wildcard imports are actually discourage from a typing perspective. i stongly agree wtih the 'Do not import objects, only modules (*)' rule
i would much rather give up oru import order rule then that.
i have spent quite a lot of tiem trying to make ai generate code be a little stricter then hacking currently requires https://github.com/SeanMooney/openstack-ai-style-guide/blob/master/docs/comp... i actually just realized i don't currently have non module imports in my anti-pattern list but its one of the things i explicit review to make sure are not introduced when looking at ai code. i really dislike polluting the current scope with non namespaces symbols form other module. i particularity dislike the example for sqlalchamy given in the commit message as a reason to allow this.
Is it possible to relax the rules only when there is an expectation that we'll catch the rules through another process like type checking?
It is also worth noting that the current rules allow us to take advantage of lazy import behaviors when Python adds support for them without making major changes. This is particularly relevant to type checking as I believe one of the benefits of lazy imports is the mitigation of cost for type checking imports and code at runtime. So we may want to be careful making changes here even if we are type checking. Yes, for expensive imports you can work around this by placing imports behind the 'TYPE_CHECKING' guard, e.g.:
if TYPE_CHECKING: from collections.abc import Sequence
so im not a fan of allowing importing Sequence like that ^ or any non module based import, alias are fine but i do not belive we should start allowing importing classes function constant other non module imports.
why not just do ``` import typeing as ty
if ty.TYPE_CHECKING: from collections import abc ```
and use abc.Sequence instead of Sequence in the type hints
im not a huge fan of the if ty.TYPE_CHECKING pattern but as you note there is a usecase for it today.
But you'd have to either using '__future__.annotations' or wrap these in strings to prevent them being evaluated at runtime.
i would prefer either of those options my only concern with |"from __future__ import annotations" is that now that| it is now deprecate for removal because they the defered sematics are now part of py3.14 as part of https://peps.python.org/pep-0749/ and https://peps.python.org/pep-0649/
0749 is the deprecation and the commitment to not remove `from __future__ import annotaitons` until 3.13 is eol.
if we were to sue that more widely we woudl eventually need to remove it when we raise our min version to 3.14 in 2028
I don't believe the cost of imports of objects from three proposed stdlib libraries is high enough to justify the cognitive load/tooling costs of dealing with deferred evaluation, but we should definitely evaluate whether that's the case for other libraries and modules. I've experimented with doing this for things like libvirt in Nova, for example. i think having any exception to this is increasing the cognitive load. i would actually prefer to remove the current exception not extend them so the opposite of what your proposing in
[2] https://review.opendev.org/c/openstack/hacking/+/967719
i was not aware of the sqlachemy exception until now but its not something i would use in code i write in nova or other project personally.
Stephen
Cheers, Stephen
[1] If you've already started writing a reply about how type hints *themselves* make function signatures more verbose and difficult to read, (a) is your name Sean Mooney and (b) stop right there, go enable an LSP in your editor, then come back to me in a month once you've seen the quality of life improvement hints provide. Then we'll talk 😄
IF you want me not to reply tell me up front :P
also i like type hint and want to use them more in all the code we write. they are awesome, they help us find bugs, they help lsp, llm and humans understand what the code is doing with out going on an expedition but i don't see a need to change hacking for them.
with that said i am interested to see what other think. i think you need to motivate why we should expand the set of exception or relax the existing rule better
I had a much longer response written to this email, but it just consisted of rebuttals and disagreements, and I don't know how much it would do to move the ball forward on this. Instead, I would encourage you or anyone else that disagrees with this proposal to pick a reasonably complicated project or module of a project that doesn't have hints yet, and to go add them. If you end up having to make use of type aliases or formalize dicts as TypedDicts, all the better. Once done, save the a copy of the file, modify the original to use object imports for types instead, and go compare the two. Come back to me when you've done so and tell me what you think. I used aliases module imports ('import typing as ty') for years, and was initially happy with them. However, I came around to this take based on practical experience (aligning with the significant majority of other Python developers in the process [1][2]). Stephen [1] https://github.com/search?q=%22from+typing+import%22+language%3APython+&type=code [2] https://github.com/search?q=%22import+typing+as+%22+language%3APython+&type=code
hi! If my voice counts, I prefer keeping the rule as is, or even reducing the existing list of exceptions, as Sean mentioned. Importing individual names from modules makes it harder to understand what is actually being used when reading a small fragment of code. Why `Any` should refer to `typing.Any` and not to `mock.ANY`? The shared GitHub search results are inaccurate; they do not illustrate the issue. They are not about typing specifically, they are about general Python "best practices". If you compare search results for import os vs from os import, or any other builtin/third-party module, you will see a similar difference in stats. чт, 20 лист. 2025 р. о 12:11 Stephen Finucane <stephenfin@redhat.com> пише:
On Wed, 2025-11-19 at 20:41 +0000, Sean Mooney wrote:
On 19/11/2025 18:22, Stephen Finucane wrote:
On Wed, 2025-11-19 at 09:49 -0800, Clark Boylan wrote:
On Wed, Nov 19, 2025, at 8:59 AM, Stephen Finucane wrote:
o/
I've been slowly adding type hints to oslo and SDK projects of
time has allowed. One thing that I've noticed is that strict adherence to hacking's H301 (Do not import more than one module per line) and H303 (Do not use wildcard * import) rules results in both stupidly long groups of imports and overly verbose and difficult to read signatures [1]. I've therefore proposed a change to these rules that exempt some typing-related libraries - collections.abc, types, and typing - from these rules. This is proposed at [2]. As noted there, exceptions already exist for the sqlalchemy and projects' i18n modules for the same reason so this should be nothing new. I am instead bringing
late as this
up so projects currently using e.g. 'import typing as ty' imports can pivot to something sooner once this is merged. i dont think we should move away for `import typing as ty` that is the conventional way to use it out side openstack as well
This isn't true. See end.
i also don't believe we should allow expction for h301 h303 or h304 for any module but more on that later.
Isn't the concern with H303 that using wildcard imports makes you
result, _ = some_fun()
Which if you expect `_()` to perform gettext actions could suddenly
become problematic. Type checking probably does alleviate some of this as it may be able to detect where `_` is no longer a function. Similarly if you're using some name as a type in a definition and that name has been shadowed the type checker will likely complain. so the use of _ as a sential value in many languages is very old and the fact that gettext conflicts with that and that the i18n module build on that is a very unfortunate collision.
i don't know which is the more common expectation but for me i know of the _ usage as a throwaway value long before i learned of the _() usage. to this day is still fine that surprising and confusing and i wish we had a better way to express that.
Sorry, Clark: I meant to call out the 'Do not import objects, only modules (*)' rule (which doesn't actually have a hacking (the tool) rule associated with since it would presumably be very difficult/impossible to enforce). I agree with the '[H303] Do not use wildcard * import (*)' rule and wildcard imports are actually discourage from a typing perspective. i stongly agree wtih the 'Do not import objects, only modules (*)' rule
i would much rather give up oru import order rule then that.
i have spent quite a lot of tiem trying to make ai generate code be a
susceptible to unexpected name collisions either due to updates in your imports or changes to your local code? The example in [2] actually points at an extremely common case of this: `_`. A single underscore is commonly used as a throwaway result in code like: little stricter then hacking currently requires
i actually just realized i don't currently have non module imports in my anti-pattern list but its one of the things i explicit review to make sure are not introduced when looking at ai code. i really dislike polluting the current scope with non namespaces symbols
i particularity dislike the example for sqlalchamy given in the commit message as a reason to allow this.
Is it possible to relax the rules only when there is an expectation
It is also worth noting that the current rules allow us to take
advantage of lazy import behaviors when Python adds support for them without making major changes. This is particularly relevant to type checking as I believe one of the benefits of lazy imports is the mitigation of cost for type checking imports and code at runtime. So we may want to be careful making changes here even if we are type checking. Yes, for expensive imports you can work around this by placing imports behind the 'TYPE_CHECKING' guard, e.g.:
if TYPE_CHECKING: from collections.abc import Sequence so im not a fan of allowing importing Sequence like that ^ or any non module based import, alias are fine but i do not belive we should start allowing importing classes function constant other non module imports.
why not just do ``` import typeing as ty
if ty.TYPE_CHECKING: from collections import abc ```
and use abc.Sequence instead of Sequence in the type hints
im not a huge fan of the if ty.TYPE_CHECKING pattern but as you note
today.
But you'd have to either using '__future__.annotations' or wrap these in strings to prevent them being evaluated at runtime.
i would prefer either of those options my only concern with |"from __future__ import annotations" is that now
https://github.com/SeanMooney/openstack-ai-style-guide/blob/master/docs/comp... form other module. that we'll catch the rules through another process like type checking? there is a usecase for it that|
it is now deprecate for removal because they the defered sematics are now part of py3.14 as part of https://peps.python.org/pep-0749/ and https://peps.python.org/pep-0649/
0749 is the deprecation and the commitment to not remove `from __future__ import annotaitons` until 3.13 is eol.
if we were to sue that more widely we woudl eventually need to remove it when we raise our min version to 3.14 in 2028
I don't believe the cost of imports of objects from three proposed stdlib libraries is high enough to justify the cognitive load/tooling costs of dealing with deferred evaluation, but we should definitely evaluate whether that's the case for other libraries and modules. I've experimented with doing this for things like libvirt in Nova, for example. i think having any exception to this is increasing the cognitive load. i would actually prefer to remove the current exception not extend them so the opposite of what your proposing in
[2] https://review.opendev.org/c/openstack/hacking/+/967719
i was not aware of the sqlachemy exception until now but its not something i would use in code i write in nova or other project personally.
Stephen
Cheers, Stephen
[1] If you've already started writing a reply about how type hints *themselves* make function signatures more verbose and difficult to read, (a) is your name Sean Mooney and (b) stop right there, go
enable
an LSP in your editor, then come back to me in a month once you've seen the quality of life improvement hints provide. Then we'll talk 😄 IF you want me not to reply tell me up front :P
also i like type hint and want to use them more in all the code we write. they are awesome, they help us find bugs, they help lsp, llm and humans understand what the code is doing with out going on an expedition but i don't see a need to change hacking for them.
with that said i am interested to see what other think. i think you need to motivate why we should expand the set of exception or relax the existing rule better
I had a much longer response written to this email, but it just consisted of rebuttals and disagreements, and I don't know how much it would do to move the ball forward on this.
Instead, I would encourage you or anyone else that disagrees with this proposal to pick a reasonably complicated project or module of a project that doesn't have hints yet, and to go add them. If you end up having to make use of type aliases or formalize dicts as TypedDicts, all the better. Once done, save the a copy of the file, modify the original to use object imports for types instead, and go compare the two. Come back to me when you've done so and tell me what you think. I used aliases module imports ('import typing as ty') for years, and was initially happy with them. However, I came around to this take based on practical experience (aligning with the significant majority of other Python developers in the process [1][2]).
Stephen
[1] https://github.com/search?q=%22from+typing+import%22+language%3APython+&type=code [2] https://github.com/search?q=%22import+typing+as+%22+language%3APython+&type=code
-- Best regards, Andriy Kurilin.
hi!
If my voice counts, I prefer keeping the rule as is, or even reducing the existing list of exceptions, as Sean mentioned. Importing individual names from modules makes it harder to understand what is actually being used when reading a small fragment of code. Why `Any` should refer to `typing.Any` and not to `mock.ANY`? Because it's a type annotation? The fact that it's a type annotation tells us to expect a type. mock.ANY is not a type: it's a singleton instance of a class. More to the point, I'm asking for 3 very specific
On Thu, 2025-11-20 at 17:55 +0100, Andriy Kurilin wrote: libraries, so chosen because no one is going to be confused about where 'Any' comes from [*], or where 'MutableMapping' comes from. Heck, is people were confused about these things we wouldn't have direct namedtuple imports in what appears to be a significant number of OpenStack projects... [1]
The shared GitHub search results are inaccurate; they do not illustrate the issue. They are not about typing specifically, they are about general Python "best practices". If you compare search results for import os vs from os import, or any other builtin/third-party module, you will see a similar difference in stats. This was really secondary to my main point and I wanted to note that what Sean had said was not true. However, for the os case I believe you're seeing the inverse? I said there are significantly more examples of direct imports from typing that there is aliasing of typing (so 'from typing import <foo>' is more common than 'import typing as <foo>'). This doesn't appear to be the case from os ('import os' > 'from os import <foo>', where <foo> != path).
More specific evidence of this behavior being preferrable: the docs for mypy [2], pyright [3], pyre [4] and ty [5] all use direct imports in their example code. mypy specifically calls this convention out [6] and highlights the advantage of brevity (the same thing I'm saying): When adding types, the convention is to import types using the form from typing import <name> (as opposed to doing just import typing or import typing as t or from typing import *). I firmly believe this is a best practice for typing large code bases, and I'd like to see that recognised. Stephen [*] I wouldn't be surprised to see 'Any' become a soft keyword in future Python versions. As more and more types become subscriptable, [1] https://codesearch.opendev.org/?q=import%20namedtuple&i=nope&literal=nope&files=&excludeFiles=&repos= [2] https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html [3] https://microsoft.github.io/pyright/#/type-concepts-advanced [4] https://docs.astral.sh/ty/reference/rules/#invalid-generic-class [5] https://pyre-check.org/docs/types-in-python/ [6] https://mypy.readthedocs.io/en/stable/getting_started.html
чт, 20 лист. 2025 р. о 12:11 Stephen Finucane <stephenfin@redhat.com> пише:
On Wed, 2025-11-19 at 20:41 +0000, Sean Mooney wrote:
On 19/11/2025 18:22, Stephen Finucane wrote:
On Wed, 2025-11-19 at 09:49 -0800, Clark Boylan wrote:
On Wed, Nov 19, 2025, at 8:59 AM, Stephen Finucane wrote:
o/
I've been slowly adding type hints to oslo and SDK projects
time has allowed. One thing that I've noticed is that strict adherence to hacking's H301 (Do not import more than one module per
of late as line) and
H303 (Do not use wildcard * import) rules results in both stupidly long groups of imports and overly verbose and difficult to read signatures [1]. I've therefore proposed a change to these rules that exempt some typing-related libraries - collections.abc, types, and typing - from these rules. This is proposed at [2]. As noted there, exceptions already exist for the sqlalchemy and projects' i18n modules for the same reason so this should be nothing new. I am instead bringing this up so projects currently using e.g. 'import typing as ty' imports can pivot to something sooner once this is merged. i dont think we should move away for `import typing as ty` that is the conventional way to use it out side openstack as well
This isn't true. See end.
i also don't believe we should allow expction for h301 h303 or
any module but more on that later.
Isn't the concern with H303 that using wildcard imports makes you susceptible to unexpected name collisions either due to updates in your imports or changes to your local code? The example in [2] actually points at an extremely common case of this: `_`. A single underscore is commonly used as a throwaway result in code like:
result, _ = some_fun()
Which if you expect `_()` to perform gettext actions could suddenly become problematic. Type checking probably does alleviate some of this as it may be able to detect where `_` is no longer a function. Similarly if you're using some name as a type in a definition and that name has been shadowed the type checker will
h304 for likely complain.
so the use of _ as a sential value in many languages is very old and the fact that gettext conflicts with that and that the i18n module build on that is a very unfortunate collision.
i don't know which is the more common expectation but for me i know of the _ usage as a throwaway value long before i learned of the _() usage. to this day is still fine that surprising and confusing and i wish we had a better way to express that.
Sorry, Clark: I meant to call out the 'Do not import objects, only modules (*)' rule (which doesn't actually have a hacking (the tool) rule associated with since it would presumably be very difficult/impossible to enforce). I agree with the '[H303] Do not use wildcard * import (*)' rule and wildcard imports are actually discourage from a typing perspective. i stongly agree wtih the 'Do not import objects, only modules (*)' rule
i would much rather give up oru import order rule then that.
i have spent quite a lot of tiem trying to make ai generate code be a little stricter then hacking currently requires
i actually just realized i don't currently have non module imports in my anti-pattern list but its one of the things i explicit review to make sure are not introduced when looking at ai code. i really dislike polluting the current scope with non namespaces symbols form other module. i particularity dislike the example for sqlalchamy given in the commit message as a reason to allow this.
Is it possible to relax the rules only when there is an
expectation that we'll catch the rules through another process like type checking?
It is also worth noting that the current rules allow us to
take advantage of lazy import behaviors when Python adds support for them without making major changes. This is particularly relevant to type checking as I believe one of the benefits of lazy imports is the mitigation of cost for type checking imports and code at runtime. So we may want to be careful making changes here even if we are type checking. Yes, for expensive imports you can work around this by placing imports behind the 'TYPE_CHECKING' guard, e.g.:
if TYPE_CHECKING: from collections.abc import Sequence so im not a fan of allowing importing Sequence like that ^ or any non module based import, alias are fine but i do not belive we should start allowing importing classes function constant other non module imports.
why not just do ``` import typeing as ty
if ty.TYPE_CHECKING: from collections import abc ```
and use abc.Sequence instead of Sequence in the type hints
im not a huge fan of the if ty.TYPE_CHECKING pattern but as you note there is a usecase for it today.
But you'd have to either using '__future__.annotations' or wrap
in strings to prevent them being evaluated at runtime. i would prefer either of those options my only concern with |"from __future__ import annotations" is
it is now deprecate for removal because they the defered sematics are now part of py3.14 as part of https://peps.python.org/pep-0749/ and https://peps.python.org/pep-0649/
0749 is the deprecation and the commitment to not remove `from __future__ import annotaitons` until 3.13 is eol.
if we were to sue that more widely we woudl eventually need to remove it when we raise our min version to 3.14 in 2028
I don't believe the cost of imports of objects from three proposed stdlib
high enough to justify the cognitive load/tooling costs of dealing with deferred evaluation, but we should definitely evaluate whether
the case for other libraries and modules. I've experimented with doing this for things like libvirt in Nova, for example. i think having any exception to this is increasing the cognitive load. i would actually prefer to remove the current exception not extend them so the opposite of what your proposing in
[2] https://review.opendev.org/c/openstack/hacking/+/967719
i was not aware of the sqlachemy exception until now but its not something i would use in code i write in nova or other project personally.
Stephen
Cheers, Stephen
[1] If you've already started writing a reply about how
type hints
*themselves* make function signatures more verbose and difficult to read, (a) is your name Sean Mooney and (b) stop right
https://github.com/SeanMooney/openstack-ai-style-guide/blob/master/docs/comp... these that now that| libraries is that's there, go enable
an LSP in your editor, then come back to me in a month once you've seen the quality of life improvement hints provide. Then we'll talk 😄 IF you want me not to reply tell me up front :P
also i like type hint and want to use them more in all the code we write. they are awesome, they help us find bugs, they help lsp, llm and humans understand what the code is doing with out going on an expedition but i don't see a need to change hacking for them.
with that said i am interested to see what other think. i think you need to motivate why we should expand the set of exception or relax the existing rule better
I had a much longer response written to this email, but it just consisted of rebuttals and disagreements, and I don't know how much it would do to move the ball forward on this.
Instead, I would encourage you or anyone else that disagrees with this proposal to pick a reasonably complicated project or module of a project that doesn't have hints yet, and to go add them. If you end up having to make use of type aliases or formalize dicts as TypedDicts, all the better. Once done, save the a copy of the file, modify the original to use object imports for types instead, and go compare the two. Come back to me when you've done so and tell me what you think. I used aliases module imports ('import typing as ty') for years, and was initially happy with them. However, I came around to this take based on practical experience (aligning with the significant majority of other Python developers in the process [1][2]).
Stephen
[1] https://github.com/search?q=%22from+typing+import%22+language%3APython+&type=code [2] https://github.com/search?q=%22import+typing+as+%22+language%3APython+&type=code
-- Best regards, Andriy Kurilin.
On 20/11/2025 18:07, Stephen Finucane wrote:
On Thu, 2025-11-20 at 17:55 +0100, Andriy Kurilin wrote:
hi!
If my voice counts, I prefer keeping the rule as is, or even reducing the existing list of exceptions, as Sean mentioned.
it does. regardless of if the over all preference is to relax the rule or tighten it i want to here all voices that care. what i said separately to Stephen is while i prefer only importing modules not non module symbols like function, types or constants the part i find personally confusing is that we have exceptions at all. i would almost prefer to have project skip the rule entirely if they want to allow non mondule import but if most folks prefer to relax it or keep it as is we should still discuss it. Stephen also shared that in oslo and some other project the already use the ablity to extend the list of excption vai tox configuration https://codesearch.opendev.org/?q=import_exceptions&i=nope&literal=nope&files=&excludeFiles=&repos= most project only use it for there i18n module for a project like oslo if they wanted to exclude the same set of module form this rule they would have to copy the typing related excpetion to each repo which would be tedious. thre might be a middle ground here, this is the current code that govenrs this behvior https://github.com/openstack/hacking/blob/822b93c4cc08c0918f3b6f22c43a32288f... ``` DEFAULT_IMPORT_EXCEPTIONS = [ 'sqlalchemy', 'migrate', ] IMPORT_EXCEPTIONS = CONF.get_multiple('import_exceptions', default=[]) IMPORT_EXCEPTIONS += DEFAULT_IMPORT_EXCEPTIONS ``` hacking already config option supprort for this rule. we could intoduce the idea fo a profile and 3 profiles strict, default, relaxed. strict would have no exceptions, default woudl be todays list and relaxed woudl have the new typing related exceptions that stephen wants to add. if we did this we could also perhaps automatically allow all ".*i18n*." modules as exception in the default and relaxed profile so that we don't have to keep adding that exception in every new repo that support translation. with that said i think i have been conviced by the other argment in this respocne to accpet your propsoal more on that below.
Importing individual names from modules makes it harder to understand what is actually being used when reading a small fragment of code. Why `Any` should refer to `typing.Any` and not to `mock.ANY`?
Because it's a type annotation? The fact that it's a type annotation tells us to expect a type. mock.ANY is not a type: it's a singleton instance of a class. More to the point, I'm asking for 3 very specific libraries, so chosen because no one is going to be confused about where 'Any' comes from [*], or where 'MutableMapping' comes from. Heck, is people were confused about these things we wouldn't have direct namedtuple imports in what appears to be a significant number of OpenStack projects... [1]
The shared GitHub search results are inaccurate; they do not illustrate the issue. They are not about |typing| specifically, they are about general Python "best practices". If you compare search results for |import os| vs |from os import|, or any other builtin/third-party module, you will see a similar difference in stats.
This was really secondary to my main point and I wanted to note that what Sean had said was not true. However, for the os case I believe you're seeing the inverse? I said there are significantly more examples of direct imports from typing that there is aliasing of typing (so 'from typing import <foo>' is more common than 'import typing as <foo>'). This doesn't appear to be the case from os ('import os' > 'from os import <foo>', where <foo> != path).
More specific evidence of this behavior being preferrable: the docs for mypy [2], pyright [3], pyre [4] and ty [5] all use direct imports in their example code. mypy specifically calls this convention out [6] and highlights the advantage of brevity (the same thing I'm saying):
When adding types, the convention is to import types using the form from typing import <name> (as opposed to doing just import typing or import typing as t or from typing import *).
I firmly believe this is a best practice for typing large code bases, and I'd like to see that recognised.
Stephen
[*] I wouldn't be surprised to see 'Any' become a soft keyword in future Python versions. As more and more types become subscriptable, [1] https://codesearch.opendev.org/?q=import%20namedtuple&i=nope&literal=nope&files=&excludeFiles=&repos= <https://codesearch.opendev.org/?q=import%20namedtuple&i=nope&literal=nope&files=&excludeFiles=&repos=>
i feel like this is more pointing to the fact that we are not actually enforcing the rule cosntienty today and that is actullly a better argument for relaxing it then the others you put forward. or making it configurable. it also a documented rule that has never actully been implemted as a check https://github.com/openstack/hacking/blob/822b93c4cc08c0918f3b6f22c43a32288f... openstack style guied (hacking) was orginally derived form google python style guide as an agmentaiton beyond pep8 https://google.github.io/styleguide/pyguide.html#22-imports google however have also change there style guide to accommodate the additon of typing more widely to the python ecosystem to add the 3 modules you are askign for https://google.github.io/styleguide/pyguide.html#2241-exemptions so with that and the situation you give below i find https://review.opendev.org/c/openstack/hacking/+/967719/1/hacking/core.py actully to be more accpable i asked you to motivate your usecase better and i can accept the evidence you presented as enough motivation to change the rule by adding the 3 module to the default exception list project are still free to review for that based on there preference but i withdraw my objection to your proposal.
[2] https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html [3] https://microsoft.github.io/pyright/#/type-concepts-advanced [4] https://docs.astral.sh/ty/reference/rules/#invalid-generic-class [5] https://pyre-check.org/docs/types-in-python/ [6] https://mypy.readthedocs.io/en/stable/getting_started.html
чт, 20 лист. 2025 р. о 12:11 Stephen Finucane <stephenfin@redhat.com> пише:
On Wed, 2025-11-19 at 20:41 +0000, Sean Mooney wrote:
On 19/11/2025 18:22, Stephen Finucane wrote:
On Wed, 2025-11-19 at 09:49 -0800, Clark Boylan wrote:
On Wed, Nov 19, 2025, at 8:59 AM, Stephen Finucane wrote: > o/ > > I've been slowly adding type hints to oslo and SDK projects
> time has allowed. One thing that I've noticed is that strict adherence > to hacking's H301 (Do not import more than one module per
> H303 (Do not use wildcard * import) rules results in both stupidly long > groups of imports and overly verbose and difficult to read signatures > [1]. I've therefore proposed a change to these rules that exempt some > typing-related libraries - collections.abc, types, and typing - from > these rules. This is proposed at [2]. As noted there, exceptions > already exist for the sqlalchemy and projects' i18n modules for the > same reason so this should be nothing new. I am instead bringing this > up so projects currently using e.g. 'import typing as ty' imports can > pivot to something sooner once this is merged. i dont think we should move away for `import typing as ty` that is
of late as line) and the
conventional way to use it out side openstack as well
This isn't true. See end.
i also don't believe we should allow expction for h301 h303 or
any module but more on that later.
Isn't the concern with H303 that using wildcard imports makes you susceptible to unexpected name collisions either due to updates in your imports or changes to your local code? The example in [2] actually points at an extremely common case of this: `_`. A single underscore is commonly used as a throwaway result in code like:
result, _ = some_fun()
Which if you expect `_()` to perform gettext actions could suddenly become problematic. Type checking probably does alleviate some of this as it may be able to detect where `_` is no longer a function. Similarly if you're using some name as a type in a definition and that name has been shadowed the type checker will
h304 for likely complain.
so the use of _ as a sential value in many languages is very old and the fact that gettext conflicts with that and that the i18n module build on that is a very unfortunate collision.
i don't know which is the more common expectation but for me i know of the _ usage as a throwaway value long before i learned of the _() usage. to this day is still fine that surprising and confusing and i wish we had a better way to express that.
Sorry, Clark: I meant to call out the 'Do not import objects, only modules (*)' rule (which doesn't actually have a hacking (the tool) rule associated with since it would presumably be very difficult/impossible to enforce). I agree with the '[H303] Do not use wildcard * import (*)' rule and wildcard imports are actually discourage from a typing perspective. i stongly agree wtih the 'Do not import objects, only modules (*)' rule
i would much rather give up oru import order rule then that.
i have spent quite a lot of tiem trying to make ai generate code be a little stricter then hacking currently requires
i actually just realized i don't currently have non module imports in my anti-pattern list but its one of the things i explicit review to make sure are not introduced when looking at ai code. i really dislike polluting the current scope with non namespaces symbols form other module. i particularity dislike the example for sqlalchamy given in the commit message as a reason to allow this.
Is it possible to relax the rules only when there is an
expectation that we'll catch the rules through another process like type checking?
It is also worth noting that the current rules allow us to
take advantage of lazy import behaviors when Python adds support for
Yes, for expensive imports you can work around this by placing imports behind the 'TYPE_CHECKING' guard, e.g.:
if TYPE_CHECKING: from collections.abc import Sequence so im not a fan of allowing importing Sequence like that ^ or any non module based import, alias are fine but i do not belive we should start allowing importing classes function constant other non module imports.
why not just do ``` import typeing as ty
if ty.TYPE_CHECKING: from collections import abc ```
and use abc.Sequence instead of Sequence in the type hints
im not a huge fan of the if ty.TYPE_CHECKING pattern but as you note there is a usecase for it today.
But you'd have to either using '__future__.annotations' or wrap
in strings to prevent them being evaluated at runtime. i would prefer either of those options my only concern with |"from __future__ import annotations" is that now that| it is now deprecate for removal because they the defered sematics are now part of py3.14 as part of https://peps.python.org/pep-0749/ and https://peps.python.org/pep-0649/
0749 is the deprecation and the commitment to not remove `from __future__ import annotaitons` until 3.13 is eol.
if we were to sue that more widely we woudl eventually need to remove it when we raise our min version to 3.14 in 2028
I don't believe the cost of imports of objects from three proposed stdlib
high enough to justify the cognitive load/tooling costs of dealing with deferred evaluation, but we should definitely evaluate whether
the case for other libraries and modules. I've experimented with doing this for things like libvirt in Nova, for example. i think having any exception to this is increasing the cognitive load. i would actually prefer to remove the current exception not extend
https://github.com/SeanMooney/openstack-ai-style-guide/blob/master/docs/comp... them without making major changes. This is particularly relevant to type checking as I believe one of the benefits of lazy imports is the mitigation of cost for type checking imports and code at runtime. So we may want to be careful making changes here even if we are type checking. these libraries is that's them
so the opposite of what your proposing in
[2] https://review.opendev.org/c/openstack/hacking/+/967719
i was not aware of the sqlachemy exception until now but its not something i would use in code i write in nova or other project personally.
Stephen
> Cheers, > Stephen > > [1] If you've already started writing a reply about how type
hints
> *themselves* make function signatures more verbose and difficult to > read, (a) is your name Sean Mooney and (b) stop right there, go enable > an LSP in your editor, then come back to me in a month once you've seen > the quality of life improvement hints provide. Then we'll talk 😄 IF you want me not to reply tell me up front :P
also i like type hint and want to use them more in all the code we write. they are awesome, they help us find bugs, they help lsp, llm and humans understand what the code is doing with out going on an expedition but i don't see a need to change hacking for them.
with that said i am interested to see what other think. i think you need to motivate why we should expand the set of exception or relax the existing rule better
I had a much longer response written to this email, but it just consisted of rebuttals and disagreements, and I don't know how much it would do to move the ball forward on this.
Instead, I would encourage you or anyone else that disagrees with this proposal to pick a reasonably complicated project or module of a project that doesn't have hints yet, and to go add them. If you end up having to make use of type aliases or formalize dicts as TypedDicts, all the better. Once done, save the a copy of the file, modify the original to use object imports for types instead, and go compare the two. Come back to me when you've done so and tell me what you think. I used aliases module imports ('import typing as ty') for years, and was initially happy with them. However, I came around to this take based on practical experience (aligning with the significant majority of other Python developers in the process [1][2]).
Stephen
[1] https://github.com/search?q=%22from+typing+import%22+language%3APython+&type=code <https://github.com/search?q=%22from+typing+import%22+language%3APython+&type=code> [2] https://github.com/search?q=%22import+typing+as+%22+language%3APython+&type=code <https://github.com/search?q=%22import+typing+as+%22+language%3APython+&type=code>
> > [2] https://review.opendev.org/c/openstack/hacking/+/967719
-- Best regards, Andriy Kurilin.
participants (4)
-
Andriy Kurilin
-
Clark Boylan
-
Sean Mooney
-
Stephen Finucane