__name__ attribute in typing module #88690
Comments
|
I noticed some (perhaps intentional) oddities with the __name__ attribute:
I have written a function to show presence/absence if the name __name__ attribute: def split_module_names(module):
unnamed, named = set(), set()
for name in dir(module):
if not name.startswith('_'):
attr = getattr(module, name)
try:
if hasattr(attr, '__name__'):
named.add(name)
else:
unnamed.add(name)
except TypeError:
pass
return named, unnamed
import typing
import collectionstyping_named, typing_unnamed = split_module_names(typing) print("typing_unnamed:", typing_unnamed)
print("collec_named & typing_unnamed:", collec_named & typing_unnamed)Is this intentional? It seems a little inconsistent. I also found something that sometimes the __name__ attribute does resolve: class S(typing.Sized):
def __len__(self):
return 0
print("'Sized' in typing_unnamed:", 'Sized' in typing_unnamed)
print("[t.__name__ for t in S.__mro__]:", [t.__name__ for t in S.__mro__]) # here __name__ is resolved!
print("getattr(typing.Sized, '__name__', None):", getattr(typing.Sized, '__name__', None))printing: 'Sized' in typing_unnamed: True |
farcat
mannequin
added
3.9
The Sorry, I don't really understand what this issue is requesting. Do you want to add the |
|
I was not aware the __name__ attribute is an implementation detail. It is described in the docs: https://docs.python.org/3/reference/datamodel.html. I have been using it since python 2.7, for example for logging. The function “split_module_names” is just a function to see what items in a module have and do not have a __name__ attribute; thought it might help proceedings. If I were to suggest an improvement, it would be that all classes and types (or minimally the abc’s) would have a __name__ attribute, being the name under which it can be imported. |
|
Lars, yes you're right that __name__ is documented in datamodel, sorry I wasn't clear in my original message. What I meant was that specifically for the typing module, it's not exposed anywhere in its docs https://docs.python.org/3/library/typing.html.
I think this makes sense. It should be as simple as adding self.__name__ = name or some variant. Note that some types hack their names, such as TypeVar or ParamSpec. So it's not always that __name__ == type/class name.
|
|
It sounds reasonable to add the __name__ attribute. Since these objects |
|
I have been doing some research, but note that I don't have much experience with the typing module. That said, there seem to be 2 main cases:
I think '_SpecialForm' can be enhanced to have '__name__' by replacing the '_name' attribute with '__name__'. Maybe add '__qualname__' as well. I cannot say whether there are many more attributes that could be implemented to have the same meaning as in 'type'. The meaning of attributes like '__mro__' seem difficult to define. def __getattr__(self, attr):
return getattr(self._getitem, attr)'_BaseGenericAlias''_SpecialGenericAlias' the '__getattr__' method could perhaps be adapted (or overridden in '_SpecialGenericAlias') as follows, from: def __getattr__(self, attr):
# We are careful for copy and pickle.
# Also for simplicity we just don't relay all dunder names
if '__origin__' in self.__dict__ and not _is_dunder(attr):
return getattr(self.__origin__, attr)
raise AttributeError(attr)to: def __getattr__(self, attr):
if '__origin__' in self.__dict__:
return getattr(self.__origin__, attr)
raise AttributeError(attr)or perhaps: def __getattr__(self, attr):
if '__origin__' in self.__dict__ and hasattr(type, attr):
return getattr(self.__origin__, attr)
raise AttributeError(attr)to forward unresolved attribute names to the original class. I have written some tools and tested some with the above solutions and this seems to solve the missing '__name__' issue and make the typing abc's much more in line with the collection abc's. However I did not do any unit/regression testing (pull the repo, etc.) tools are attached. |
|
Sorry for the slow progress. I don’t think it is important for Any orUnion to have these attributes, but the ones that match ABCs or concrete classes (e.g. MutableSet, Counter) should probably have __name__, __qualname__, and __module__, since the originals have those. I think __module__ should be set to ‘typing’, and __qualname__ to ‘typing.WhatEver’. |
|
Happy to see progress on this issue and I can see that adding these attributes to the ABC's in typing makes the most sense. However for my direct use-case (simplified: using Any in a type checking descriptor) it would be really practical to have the __name__ (and perhaps __qualname__ and __module__) attributes in the Any type. This is mainly for consistent logging/printing purposes. Since Any already has a _name attribute, changing this to __name__ might achieve this. |
PEP-3155 specifies that Rather, it's for nested classes and classes created in local scopes. |
|
Yurii has a working PR for __name__ in _BaseGenericAlias, but not for _SpecialForm yet. Guido and/or Lukasz, do y'all think we should support __name__ and __qualname__ for special forms too? Personally I don't see how it'd hurt and I'm +1 for this. |
|
I see this as part of a trend to improve runtime introspection of complex |
|
Thanks! ✨ 🍰 ✨ |
ambv
added
3.10
|
This PRs herein have created a situation wherein the |
|
Serhiy or Ken-Jin? |
The affected objects are special forms which can hold types, so Union[], TypeGuard[], and Concatenate[]. Personally, I don't really understand the meaning of __name__ for special forms. From the docs https://docs.python.org/3/library/stdtypes.html#definition.\_\_name__, __name__ refers to "The name of the class, function, method, descriptor, or generator instance.". __name__ make sense for GenericAlias because it's supposed to be an almost transparent proxy to the original class (eg. typing.Callable -> collections.abc.Callable). A special form is not really any one of those things listed in the docs (though I'm aware it's implemented using GenericAlias internally). OTOH, Python's definition of a type and typing's are wildly different. Union[X, Y] is called a "Union type" (despite being an object), and all types ought to have __name__ ;). @Lars, would it help your use case for Union[X, Y] and friends to have __name__ too? Note that this would mean the builtin union object (int | str) will need to support __name__ too. It looks a bit strange to me, but if it's useful I'm a +0.5 on this. CC-ed Serhiy for his opinion too. |
|
I do agree that it's nice to have a Whether we should have this feature is a distinct "problem" from its It seems to be easy to fix though (see below for a |
|
Unfortunately PR27614 and its backport has introduced reference leaks: ❯ ./python -m test test_typing -R : == Tests result: FAILURE == 1 test failed: 1 re-run test: Total duration: 1.2 sec |
|
Unfortunately given that the all refleak buildbots will start to fail and the fact that this got into the release candidate, per our buildbot policy (https://discuss.python.org/t/policy-to-revert-commits-on-buildbot-failure/404) we will be forced to revert 8bdf12e and its backport to avoid masking other issues if this is not fixed in 24 hours. |
|
Wow, turns out the reference leak has been here since forever! I opened https://bugs.python.org/issue44856? to tackle it |
|
Curiously, while the root cause for the refleaks is in BPO-44856, while hunting down how test_typing.py triggered them, I found that for a while now this exception has been kind of broken: >>> class C(Union[int, str]): ...
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: __init__() takes 2 positional arguments but 4 were given
>>>It's still a TypeError but the message is cryptic. This regressed in Python 3.9. In Python 3.8 and before, this used to be more descriptive: >>> class C(Union[int, str]): ...
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/ambv/.pyenv/versions/3.8.9/lib/python3.8/typing.py", line 317, in __new__
raise TypeError(f"Cannot subclass {cls!r}")
TypeError: Cannot subclass <class 'typing._SpecialForm'>Interestingly, after the Bas' last change, the exception is now yet different: >>> class C(Union[int, str]): ...
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its basesThis makes sense, the conflict is due to bases being (typing.Union, <class 'typing.Generic'>) where "typing.Union" is really a _UnionGenericAlias since this is a subscripted Union (unlike bare "typing.Union" which is an instance of _SpecialForm). And in _GenericAlias' __mro_entries__ we're finding: Lines 1089 to 1090 in a40675c Clearly Ivan only intended _name to be used for shadowing builtins and ABCs. BTW, the "__init__() takes 2 positional arguments but 4 were given" is about _SpecialForm's __init__. It's called with 4 arguments through here in builtin___build_class__: Lines 223 to 224 in a40675c This isn't high priority since the end result is a TypeError anyway, but it's something I will be investigating to make the error message sensible again. |
|
There are still cryptic TypeError messages for Annotated: >>> class X(Annotated[int | float, "const"]): pass
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases |
|
There are some side effects of setting _name. In 3.9: >>> class X(Annotated[int, (1, 10)]): pass
...
>>> X.__mro__
(<class '__main__.X'>, <class 'int'>, <class 'object'>)In 3.10: >>> class X(Annotated[int, (1, 10)]): pass
...
>>> X.__mro__
(<class '__main__.X'>, <class 'int'>, <class 'typing.Generic'>, <class 'object'>)Now a subclass of an Annotated alias is a generic type. Should it be? |
I'm unsure if Annotated should be subclassable in the first place, but if I understand PEP-593 correctly, FWIW, the other special forms don't allow subclassing, so we don't need to think about this problem for them. Annotated is a special cookie. I propose we just drop the _name hack temporarily in Annotated. A real fix requires fixing up __mro_entries__, but I am uncomfortable with us backporting to 3.10 anything that touches __mro_entries__ due to the numerous edge cases it has and how close we are to 3.10 final. |
|
I don't think we need to support Annotated as a base class. PEP-593 is titled "Flexible function and variable annotations", and base classes are neither of those things. None of the examples in the PEP or the implementation use Annotated as a base class either. On the other hand, subclassing Annotated[T, ...] does work at runtime in 3.9, so maybe we're bound by backward compatibility now. |
|
Is there anything left to do here, or can this now be closed? |
|
This issue has gone through a bit of a journey, but the original complaint was that |
_GenericAlias._namewas not properly set for specialforms #27614_GenericAlias._namewas not properly set for specialforms (GH-27614) #27632typing(GH-27710) #27815Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: