Merge branch 'main' into supermon

* main: (27 commits)
  pythongh-87849: fix SEND specialization family definition (pythonGH-104268)
  pythongh-101819: Adapt _io.IOBase.seek and _io.IOBase.truncate to Argument Clinic (python#104384)
  pythongh-101819: Adapt _io._Buffered* methods to Argument Clinic (python#104367)
  pythongh-101819: Refactor `_io` futher in preparation for module isolation (python#104369)
  pythongh-101819: Adapt _io.TextIOBase methods to Argument Clinic (python#104383)
  pythongh-101117: Improve accuracy of sqlite3.Cursor.rowcount docs (python#104287)
  pythonGH-92184: Convert os.altsep to '/' in filenames when creating ZipInfo objects (python#92185)
  pythongh-104357: fix inlined comprehensions that close over iteration var (python#104368)
  pythonGH-90208: Suppress OSError exceptions from `pathlib.Path.glob()` (pythonGH-104141)
  pythonGH-102181: Improve specialization stats for SEND (pythonGH-102182)
  pythongh-103000: Optimise `dataclasses.asdict` for the common case (python#104364)
  pythongh-103538: Remove unused TK_AQUA code (pythonGH-103539)
  pythonGH-87695: Fix OSError from `pathlib.Path.glob()` (pythonGH-104292)
  pythongh-104263: Rely on Py_NAN and introduce Py_INFINITY (pythonGH-104202)
  pythongh-104010: Separate and improve docs for `typing.get_origin` and `typing.get_args` (python#104013)
  pythongh-101819: Adapt _io._BufferedIOBase_Type methods to Argument Clinic (python#104355)
  pythongh-103960: Dark mode: invert image brightness (python#103983)
  pythongh-104252: Immortalize Py_EMPTY_KEYS (pythongh-104253)
  pythongh-101819: Clean up _io windows console io after pythongh-104197 (python#104354)
  pythongh-101819: Harden _io init (python#104352)
  ...

Author: carljm

Doc/howto/logging.rst

@@ -418,6 +418,7 @@ The flow of log event information in loggers and handlers is illustrated in the
 following diagram.
 
 .. image:: logging_flow.png
+   :class: invert-in-dark-mode
 
 Loggers
 ^^^^^^^

Doc/library/hashlib.rst

@@ -430,6 +430,7 @@ Constructor functions also accept the following tree hashing parameters:
 
 .. figure:: hashlib-blake2-tree.png
    :alt: Explanation of tree mode parameters.
+   :class: invert-in-dark-mode
 
 See section 2.10 in `BLAKE2 specification
 <https://www.blake2.net/blake2_20130129.pdf>`_ for comprehensive review of tree

Doc/library/http.client.rst

@@ -264,7 +264,10 @@ HTTPConnection Objects
             encode_chunked=False)
 
    This will send a request to the server using the HTTP request
-   method *method* and the selector *url*.
+   method *method* and the request URI *url*. The provided *url* must be
+   an absolute path to conform with :rfc:`RFC 2616 §5.1.2 <2616#section-5.1.2>`
+   (unless connecting to an HTTP proxy server or using the ``OPTIONS`` or
+   ``CONNECT`` methods).
 
    If *body* is specified, the specified data is sent after the headers are
    finished.  It may be a :class:`str`, a :term:`bytes-like object`, an
@@ -279,7 +282,10 @@ HTTPConnection Objects
    iterable are sent as is until the iterable is exhausted.
 
    The *headers* argument should be a mapping of extra HTTP headers to send
-   with the request.
+   with the request. A :rfc:`Host header <2616#section-14.23>`
+   must be provided to conform with :rfc:`RFC 2616 §5.1.2 <2616#section-5.1.2>`
+   (unless connecting to an HTTP proxy server or using the ``OPTIONS`` or
+   ``CONNECT`` methods).
 
    If *headers* contains neither Content-Length nor Transfer-Encoding,
    but there is a request body, one of those
@@ -298,6 +304,16 @@ HTTPConnection Objects
    HTTPConnection object assumes that all encoding is handled by the
    calling code.  If it is ``True``, the body will be chunk-encoded.
 
+   For example, to perform a ``GET`` request to ``https://docs.python.org/3/``::
+
+      >>> import http.client
+      >>> host = "docs.python.org"
+      >>> conn = http.client.HTTPSConnection(host)
+      >>> conn.request("GET", "/3/", headers={"Host": host})
+      >>> response = conn.getresponse()
+      >>> print(response.status, response.reason)
+      200 OK
+
    .. note::
       Chunked transfer encoding has been added to the HTTP protocol
       version 1.1.  Unless the HTTP server is known to handle HTTP 1.1,

Doc/library/pathlib.rst

@@ -21,6 +21,7 @@ inherit from pure paths but also provide I/O operations.
 
 .. image:: pathlib-inheritance.png
    :align: center
+   :class: invert-in-dark-mode
 
 If you've never used this module before or just aren't sure which class is
 right for your task, :class:`Path` is most likely what you need. It instantiates

Doc/library/sqlite3.rst

@@ -1694,7 +1694,10 @@ Cursor objects
       ``INSERT``, ``UPDATE``, ``DELETE``, and ``REPLACE`` statements;
       is ``-1`` for other statements,
       including :abbr:`CTE (Common Table Expression)` queries.
-      It is only updated by the :meth:`execute` and :meth:`executemany` methods.
+      It is only updated by the :meth:`execute` and :meth:`executemany` methods,
+      after the statement has run to completion.
+      This means that any resulting rows must be fetched in order for
+      :attr:`!rowcount` to be updated.
 
    .. attribute:: row_factory
 

Doc/library/typing.rst

@@ -2876,24 +2876,37 @@ Introspection helpers
       if a default value equal to ``None`` was set.
       Now the annotation is returned unchanged.
 
-.. function:: get_args(tp)
 .. function:: get_origin(tp)
 
-   Provide basic introspection for generic types and special typing forms.
-
-   For a typing object of the form ``X[Y, Z, ...]`` these functions return
-   ``X`` and ``(Y, Z, ...)``. If ``X`` is a generic alias for a builtin or
+   Get the unsubscripted version of a type: for a typing object of the form
+   ``X[Y, Z, ...]`` return ``X``. If ``X`` is a generic alias for a builtin or
    :mod:`collections` class, it gets normalized to the original class.
+   If ``X`` is an instance of :class:`ParamSpecArgs` or :class:`ParamSpecKwargs`,
+   return the underlying :class:`ParamSpec`.
+   Return ``None`` for unsupported objects.
+   Examples::
+
+      assert get_origin(str) is None
+      assert get_origin(Dict[str, int]) is dict
+      assert get_origin(Union[int, str]) is Union
+      P = ParamSpec('P')
+      assert get_origin(P.args) is P
+      assert get_origin(P.kwargs) is P
+
+   .. versionadded:: 3.8
+
+.. function:: get_args(tp)
+
+   Get type arguments with all substitutions performed: for a typing object
+   of the form ``X[Y, Z, ...]`` return ``(Y, Z, ...)``.
    If ``X`` is a union or :class:`Literal` contained in another
    generic type, the order of ``(Y, Z, ...)`` may be different from the order
    of the original arguments ``[Y, Z, ...]`` due to type caching.
-   For unsupported objects return ``None`` and ``()`` correspondingly.
+   Return ``()`` for unsupported objects.
    Examples::
 
-      assert get_origin(Dict[str, int]) is dict
+      assert get_args(int) == ()
       assert get_args(Dict[int, str]) == (int, str)
-
-      assert get_origin(Union[int, str]) is Union
       assert get_args(Union[int, str]) == (int, str)
 
    .. versionadded:: 3.8

Include/cpython/genobject.h

@@ -77,6 +77,8 @@ PyAPI_FUNC(PyObject *) PyAsyncGen_New(PyFrameObject *,
 
 #define PyAsyncGen_CheckExact(op) Py_IS_TYPE((op), &PyAsyncGen_Type)
 
+#define PyAsyncGenASend_CheckExact(op) Py_IS_TYPE((op), &_PyAsyncGenASend_Type)
+
 
 #undef _PyGenObject_HEAD
 

Include/internal/pycore_dict_state.h

@@ -38,6 +38,11 @@ struct _Py_dict_state {
     PyDict_WatchCallback watchers[DICT_MAX_WATCHERS];
 };
 
+#define _dict_state_INIT \
+    { \
+        .next_keys_version = 2, \
+    }
+
 
 #ifdef __cplusplus
 }

Include/internal/pycore_dtoa.h

@@ -64,8 +64,6 @@ PyAPI_FUNC(double) _Py_dg_strtod(const char *str, char **ptr);
 PyAPI_FUNC(char *) _Py_dg_dtoa(double d, int mode, int ndigits,
                         int *decpt, int *sign, char **rve);
 PyAPI_FUNC(void) _Py_dg_freedtoa(char *s);
-PyAPI_FUNC(double) _Py_dg_stdnan(int sign);
-PyAPI_FUNC(double) _Py_dg_infinity(int sign);
 
 #endif // _PY_SHORT_FLOAT_REPR == 1
 

Include/internal/pycore_runtime_init.h

@@ -107,9 +107,7 @@ extern PyTypeObject _PyExc_MemoryError;
             }, \
         }, \
         .dtoa = _dtoa_state_INIT(&(INTERP)), \
-        .dict_state = { \
-            .next_keys_version = 2, \
-        }, \
+        .dict_state = _dict_state_INIT, \
         .func_state = { \
             .next_version = 1, \
         }, \

Include/pymath.h

@@ -39,27 +39,24 @@
 // Return 1 if float or double arg is neither infinite nor NAN, else 0.
 #define Py_IS_FINITE(X) isfinite(X)
 
-/* HUGE_VAL is supposed to expand to a positive double infinity.  Python
- * uses Py_HUGE_VAL instead because some platforms are broken in this
- * respect.  We used to embed code in pyport.h to try to worm around that,
- * but different platforms are broken in conflicting ways.  If you're on
- * a platform where HUGE_VAL is defined incorrectly, fiddle your Python
- * config to #define Py_HUGE_VAL to something that works on your platform.
+// Py_INFINITY: Value that evaluates to a positive double infinity.
+#ifndef Py_INFINITY
+#  define Py_INFINITY ((double)INFINITY)
+#endif
+
+/* Py_HUGE_VAL should always be the same as Py_INFINITY.  But historically
+ * this was not reliable and Python did not require IEEE floats and C99
+ * conformity.  Prefer Py_INFINITY for new code.
  */
 #ifndef Py_HUGE_VAL
 #  define Py_HUGE_VAL HUGE_VAL
 #endif
 
-// Py_NAN: Value that evaluates to a quiet Not-a-Number (NaN).
+/* Py_NAN: Value that evaluates to a quiet Not-a-Number (NaN).  The sign is
+ * undefined and normally not relevant, but e.g. fixed for float("nan").
+ */
 #if !defined(Py_NAN)
-#  if _Py__has_builtin(__builtin_nan)
-     // Built-in implementation of the ISO C99 function nan(): quiet NaN.
-#    define Py_NAN (__builtin_nan(""))
-#else
-     // Use C99 NAN constant: quiet Not-A-Number.
-     // NAN is a float, Py_NAN is a double: cast to double.
 #    define Py_NAN ((double)NAN)
-#  endif
 #endif
 
 #endif /* Py_PYMATH_H */

Lib/dataclasses.py

@@ -1324,11 +1324,18 @@ def _asdict_inner(obj, dict_factory):
     if type(obj) in _ATOMIC_TYPES:
         return obj
     elif _is_dataclass_instance(obj):
-        result = []
-        for f in fields(obj):
-            value = _asdict_inner(getattr(obj, f.name), dict_factory)
-            result.append((f.name, value))
-        return dict_factory(result)
+        # fast path for the common case
+        if dict_factory is dict:
+            return {
+                f.name: _asdict_inner(getattr(obj, f.name), dict)
+                for f in fields(obj)
+            }
+        else:
+            result = []
+            for f in fields(obj):
+                value = _asdict_inner(getattr(obj, f.name), dict_factory)
+                result.append((f.name, value))
+            return dict_factory(result)
     elif isinstance(obj, tuple) and hasattr(obj, '_fields'):
         # obj is a namedtuple.  Recurse into it, but the returned
         # object is another namedtuple of the same type.  This is

Lib/pathlib.py

@@ -142,25 +142,21 @@ def _select_from(self, parent_path, scandir):
             # avoid exhausting file descriptors when globbing deep trees.
             with scandir(parent_path) as scandir_it:
                 entries = list(scandir_it)
+        except OSError:
+            pass
+        else:
             for entry in entries:
                 if self.dironly:
                     try:
-                        # "entry.is_dir()" can raise PermissionError
-                        # in some cases (see bpo-38894), which is not
-                        # among the errors ignored by _ignore_error()
                         if not entry.is_dir():
                             continue
-                    except OSError as e:
-                        if not _ignore_error(e):
-                            raise
+                    except OSError:
                         continue
                 name = entry.name
                 if self.match(name):
                     path = parent_path._make_child_relpath(name)
                     for p in self.successor._select_from(path, scandir):
                         yield p
-        except PermissionError:
-            return
 
 
 class _RecursiveWildcardSelector(_Selector):
@@ -175,28 +171,25 @@ def _iterate_directories(self, parent_path, scandir):
             # avoid exhausting file descriptors when globbing deep trees.
             with scandir(parent_path) as scandir_it:
                 entries = list(scandir_it)
+        except OSError:
+            pass
+        else:
             for entry in entries:
                 entry_is_dir = False
                 try:
-                    entry_is_dir = entry.is_dir()
-                except OSError as e:
-                    if not _ignore_error(e):
-                        raise
-                if entry_is_dir and not entry.is_symlink():
+                    entry_is_dir = entry.is_dir(follow_symlinks=False)
+                except OSError:
+                    pass
+                if entry_is_dir:
                     path = parent_path._make_child_relpath(entry.name)
                     for p in self._iterate_directories(path, scandir):
                         yield p
-        except PermissionError:
-            return
 
     def _select_from(self, parent_path, scandir):
-        try:
-            successor_select = self.successor._select_from
-            for starting_point in self._iterate_directories(parent_path, scandir):
-                for p in successor_select(starting_point, scandir):
-                    yield p
-        except PermissionError:
-            return
+        successor_select = self.successor._select_from
+        for starting_point in self._iterate_directories(parent_path, scandir):
+            for p in successor_select(starting_point, scandir):
+                yield p
 
 
 class _DoubleRecursiveWildcardSelector(_RecursiveWildcardSelector):

Lib/test/test_cmath.py

@@ -166,6 +166,11 @@ def test_infinity_and_nan_constants(self):
         self.assertEqual(cmath.nan.imag, 0.0)
         self.assertEqual(cmath.nanj.real, 0.0)
         self.assertTrue(math.isnan(cmath.nanj.imag))
+        # Also check that the sign of all of these is positive:
+        self.assertEqual(math.copysign(1., cmath.nan.real), 1.)
+        self.assertEqual(math.copysign(1., cmath.nan.imag), 1.)
+        self.assertEqual(math.copysign(1., cmath.nanj.real), 1.)
+        self.assertEqual(math.copysign(1., cmath.nanj.imag), 1.)
 
         # Check consistency with reprs.
         self.assertEqual(repr(cmath.inf), "inf")

Lib/test/test_complex.py

@@ -529,6 +529,12 @@ class complex2(complex):
                     self.assertFloatsAreIdentical(z.real, x)
                     self.assertFloatsAreIdentical(z.imag, y)
 
+    def test_constructor_negative_nans_from_string(self):
+        self.assertEqual(copysign(1., complex("-nan").real), -1.)
+        self.assertEqual(copysign(1., complex("-nanj").imag), -1.)
+        self.assertEqual(copysign(1., complex("-nan-nanj").real), -1.)
+        self.assertEqual(copysign(1., complex("-nan-nanj").imag), -1.)
+
     def test_underscores(self):
         # check underscores
         for lit in VALID_UNDERSCORE_LITERALS:
@@ -569,6 +575,7 @@ def test(v, expected, test_fn=self.assertEqual):
         test(complex(NAN, 1), "(nan+1j)")
         test(complex(1, NAN), "(1+nanj)")
         test(complex(NAN, NAN), "(nan+nanj)")
+        test(complex(-NAN, -NAN), "(nan+nanj)")
 
         test(complex(0, INF), "infj")
         test(complex(0, -INF), "-infj")

Lib/test/test_float.py

@@ -1040,11 +1040,8 @@ def test_inf_signs(self):
         self.assertEqual(copysign(1.0, float('inf')), 1.0)
         self.assertEqual(copysign(1.0, float('-inf')), -1.0)
 
-    @unittest.skipUnless(getattr(sys, 'float_repr_style', '') == 'short',
-                         "applies only when using short float repr style")
     def test_nan_signs(self):
-        # When using the dtoa.c code, the sign of float('nan') should
-        # be predictable.
+        # The sign of float('nan') should be predictable.
         self.assertEqual(copysign(1.0, float('nan')), 1.0)
         self.assertEqual(copysign(1.0, float('-nan')), -1.0)
 

Lib/test/test_importlib/extension/test_loader.py

@@ -262,15 +262,16 @@ def test_reload(self):
 
     def test_try_registration(self):
         # Assert that the PyState_{Find,Add,Remove}Module C API doesn't work.
-        module = self.load_module()
-        with self.subTest('PyState_FindModule'):
-            self.assertEqual(module.call_state_registration_func(0), None)
-        with self.subTest('PyState_AddModule'):
-            with self.assertRaises(SystemError):
-                module.call_state_registration_func(1)
-        with self.subTest('PyState_RemoveModule'):
-            with self.assertRaises(SystemError):
-                module.call_state_registration_func(2)
+        with util.uncache(self.name):
+            module = self.load_module()
+            with self.subTest('PyState_FindModule'):
+                self.assertEqual(module.call_state_registration_func(0), None)
+            with self.subTest('PyState_AddModule'):
+                with self.assertRaises(SystemError):
+                    module.call_state_registration_func(1)
+            with self.subTest('PyState_RemoveModule'):
+                with self.assertRaises(SystemError):
+                    module.call_state_registration_func(2)
 
     def test_load_submodule(self):
         # Test loading a simulated submodule.

Lib/test/test_listcomps.py

@@ -163,6 +163,16 @@ def test_inner_cell_shadows_outer(self):
         outputs = {"y": [4, 4, 4, 4, 4], "i": 20}
         self._check_in_scopes(code, outputs)
 
+    def test_inner_cell_shadows_outer_no_store(self):
+        code = """
+            def f(x):
+                return [lambda: x for x in range(x)], x
+            fns, x = f(2)
+            y = [fn() for fn in fns]
+        """
+        outputs = {"y": [1, 1], "x": 2}
+        self._check_in_scopes(code, outputs)
+
     def test_closure_can_jump_over_comp_scope(self):
         code = """
             items = [(lambda: y) for i in range(5)]