gh-126654: Fix crash in several functions in `_interpreters` module (GH-126678)
(cherry picked from commit 9fc2808eaf)
Co-authored-by: sobolevn <mail@sobolevn.me>
pickle.Pickler.persistent_id and pickle.Unpickler.persistent_load can
again be overridden as instance attributes.
(cherry picked from commit 223d3dc554)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-126455: Disallow _ssl.SSLSocket instantiation (GH-126481)
Prevent creation of incomplete/invalid _ssl.SSLSocket objects when
created directly.
(cherry picked from commit b1c4ffc205)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-126425: Refactor `_lsprof_Profiler_enable` (GH-126426)
- Explicit memory management for `None` objects (since we still try to treat immortal objects as regular objects)
- Respect possible errors of `sys.monitoring.register_callback` call
(cherry picked from commit 75872605aa)
Co-authored-by: sobolevn <mail@sobolevn.me>
gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (GH-126120)
(cherry picked from commit 0e8665554b)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (GH-126103)
(cherry picked from commit d07dcce693)
Co-authored-by: Nico-Posada <102486290+Nico-Posada@users.noreply.github.com>
Yak-shave in preparation for Argument Clinic adaption in gh-125999.
(cherry picked from commit 9b14083497)
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
gh-125966: fix use-after-free on `fut->fut_callback0` due to an evil callback's `__eq__` in asyncio (GH-125967)
(cherry picked from commit ed5059eeb1)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-125984: fix use-after-free on `fut->fut_{callback,context}0` due to an evil `loop.__getattribute__` (GH-126003)
(cherry picked from commit f819d4301d)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-125969: fix OOB in `future_schedule_callbacks` due to an evil `call_soon` (GH-125970)
(cherry picked from commit c5b99f5c2c)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
GH-125789: fix `fut._callbacks` to always return a copy of callbacks (#125922)
Fix `asyncio.Future._callbacks` to always return a copy of the internal list of callbacks to avoid mutation from user code affecting the internal state.
(cherry picked from commit cae853e3b4)
This includes a drive-by cleanup in _queues_init() and _queues_fini().
This change also applies to the _interpchannels module.
(cherry picked from commit 4848b0b92c, AKA gh-125803)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
The fix applies to the _interpchannels module as well.
I've also included a drive-by typo fix for _interpqueues.
(cherry picked from commit 44f841f01a, AKA gh-125802)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
This is a follow up of GH-124974. Only Glibc needed a fix.
Now the returned value is a string consisting of semicolon-separated
symbols on all Posix platforms.
(cherry picked from commit dcc4fb2c90)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-125667: Statically Initialize the Arg Converter Data Values in _interpqueuesmodule.c (gh-125668)
(cherry picked from commit 7cf2dbc3cb)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Fix time.strftime(), the strftime() method and formatting of the
datetime classes datetime, date and time.
* Characters not encodable in the current locale are now acceptable in
the format string.
* Surrogate pairs and sequence of surrogatescape-encoded bytes are no
longer recombinated.
* Embedded null character no longer terminates the format string.
This fixes also gh-78662 and gh-124531.
(cherry picked from commit ad3eac1963)
Lock `ZoneInfoType` to protect accesses to `ZONEINFO_STRONG_CACHE`.
Refactor the `tp_new` handler to use Argument Clinic so that we can just
use `@critical_section` annotations on the relevant functions.
Also use `PyDict_SetDefaultRef` instead of `PyDict_SetDefault` when
inserting into the `TIMEDELTA_CACHE`.
(cherry picked from commit f1d33dbddd)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-125235: Keep `_tkinter` TCL paths pointing to base installation on Windows (GH-125250)
(cherry picked from commit b3aa1b5fe2)
Signed-off-by: y5c4l3 <y5c4l3@proton.me>
Co-authored-by: Y5 <124019959+y5c4l3@users.noreply.github.com>
Returns a tuple of up to 100 strings for ALT_DIGITS lookup (an empty tuple on most locales).
Previously it returned the first item of that tuple or an empty string.
(cherry picked from commit 21c04e1a97)
memcopy'ing arbitrary values to _Bool variable triggers undefined
behaviour. Avoid this.
We assume that `false` is represented by all zero bytes.
Credits to Alex Gaynor.
(cherry picked from commit 87d7315ac5)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
gh-120378: Fix crash caused by integer overflow in `curses` (GH-124555)
This is actually an upstream problem in curses, and has been reported
to them already:
https://lists.gnu.org/archive/html/bug-ncurses/2024-09/msg00101.html
This is a nice workaround in the meantime to prevent the segfault.
(cherry picked from commit c2ba931318)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-116810: fix memory leak in ssl module (GH-123249)
Resolve a memory leak introduced in CPython 3.10's :mod:`ssl` when the :attr:`ssl.SSLSocket.session` property was accessed. Speeds up read and write access to said property by no longer unnecessarily cloning session objects via serialization.
(cherry picked from commit 7e7223e18f)
Co-authored-by: Jeffrey R. Van Voorst <jeff.vanvoorst@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Revert the incremental GC in 3.13, since it's not clear that without further turning, the benefits outweigh the costs.
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Buildbot failure on Windows 10 with MSC v.1916 64 bit (AMD64):
FAIL: testFmod (test.test_math.MathTests.testFmod)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\buildarea\3.x.bolen-windows10\build\Lib\test\test_math.py", line 605, in testFmod
self.ftest('fmod(-10, 1)', math.fmod(-10, 1), -0.0)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\buildarea\3.x.bolen-windows10\build\Lib\test\test_math.py", line 258, in ftest
self.fail("{}: {}".format(name, failure))
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: fmod(-10, 1): expected -0.0, got 0.0 (zero has wrong sign)
Here Windows loose sign of the result; if y is nonzero, the result
should have the same sign as x.
This amends commit 28aea5d07d.
(cherry picked from commit f4dd440210)
gh-123797: Check for runtime availability of `ptsname_r` on macos (GH-123806)
(cherry picked from commit 3e36e5aef1)
Co-authored-by: sobolevn <mail@sobolevn.me>
gh-124248: Fix crash in struct when processing 0p fields (GH-124251)
(cherry picked from commit 63f196090f)
Co-authored-by: Brian Schubert <brianm.schubert@gmail.com>
gh-123017: Add Android to the list of platforms where `strftime` doesn't support negative years (GH-124467)
Add Android to the list of platforms where `strftime` doesn't support negative years
(cherry picked from commit 0a3577bdfc)
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Switch more _Py_IsImmortal(...) assertions to _Py_IsImmortalLoose(...)
The remaining calls to _Py_IsImmortal are in free-threaded-only code,
initialization of core objects, tests, and guards that fall back to
code that works with mortal objects.
(cherry picked from commit 57c471a688)
gh-123431: Harmonize extension code checks in pickle (GH-123434)
This checks are redundant in normal circumstances and can only work if
the extension registry was intentionally broken.
* The Python implementation now raises exception for the extension code
with false boolean value.
* Simplify the C code. RuntimeError is now raised in explicit checks.
* Add many tests.
(cherry picked from commit 0c3ea30238)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-123213: Fixed xml.etree.ElementTree.Element.extend and assignment to no longer hide exceptions (GH-123214)
(cherry picked from commit 90b6d0e0f8)
Co-authored-by: Bar Harel <bharel@barharel.com>
* Parameters after the var-positional parameter are now keyword-only
instead of positional-or-keyword.
* Correctly calculate min_kw_only.
* Raise errors for invalid combinations of the var-positional parameter
with "*", "/" and deprecation markers.
(cherry picked from commit 8393608dd9)
gh-100256: Skip inaccessible registry keys in the WinAPI mimetype implementation (GH-122047)
(cherry picked from commit 0bd93755f3)
Co-authored-by: Lucas Esposito <LucasEsposito@users.noreply.github.com>
gh-116622: Rename build variable MODULE_LDFLAGS back to LIBPYTHON (GH-122764)
(LIBPYTHON was renamed MODULE_LDFLAGS in commit 7f5e3f04f.)
(cherry picked from commit 2f5c3b09e4)
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Fix _PyArg_UnpackKeywordsWithVararg for the case when argument for
positional-or-keyword parameter is passed by keyword.
There was only one such case in the stdlib -- the TypeVar constructor.
(cherry picked from commit 540fcc62f5)
gh-122728: Fix SystemError in PyEval_GetLocals() (GH-122735)
Fix PyEval_GetLocals() to avoid SystemError ("bad argument to
internal function"). Don't redefine the 'ret' variable in the if
block.
Add an unit test on PyEval_GetLocals().
(cherry picked from commit 4767a6e31c)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-120974: Make asyncio `swap_current_task` safe in free-threaded build (GH-122317)
* gh-120974: Make asyncio `swap_current_task` safe in free-threaded build
(cherry picked from commit b5e6fb39a2)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Update datetime module and test_type_cache.py to not call PyType_Modified.
(cherry picked from commit e55b05f29e, AKA gh--122182)
Co-authored-by: Mark Shannon <mark@hotpy.org>
gh-121489: Export private _PyBytes_Join() again (GH-122267)
(cherry picked from commit aef95eb107)
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Serializing objects with complex __qualname__ (such as unbound methods and
nested classes) by name no longer involves serializing parent objects by value
in pickle protocols < 4.
(cherry picked from commit dc07f65a53)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-120974: Make _asyncio._leave_task atomic in the free-threaded build (GH-122139)
* gh-120974: Make _asyncio._leave_task atomic in the free-threaded build
Update `_PyDict_DelItemIf` to allow for an argument to be passed to the
predicate.
(cherry picked from commit a15feded71)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-120974: Make _asyncio._enter_task atomic in the free-threaded build (GH-122138)
Use `PyDict_SetDefaultRef` to set the current task in a single operation
under the dictionary's lock.
(cherry picked from commit 47847aa8ef)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Relatedly, emit the `cpython.run_startup` event from the Python version of
`PYTHONSTARTUP` handling.
(cherry picked from commit dc93d1125f)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
The futureobj freelist isn't thread-safe. We intend to re-enable the
freelist in a thread-safe way for 3.14 (but not 3.13).
(cherry picked from commit 97248204a1)
This is a small refactoring to the current design that allows us to
avoid manually iterating over threads.
This should also fix gh-118490.
(cherry picked from commit e059aa6b01)
Co-authored-by: mpage <mpage@meta.com>
gh-121621: Move asyncio_running_loop to private struct (GH-121939)
This avoids changing the ABI and keeps the field in the private struct.
(cherry picked from commit 81fd625b5c)
Co-authored-by: Sam Gross <colesbury@gmail.com>
* Switch PyUnicode_InternInPlace to _PyUnicode_InternMortal, clarify docs
* Document immortality in some functions that take `const char *`
This is PyUnicode_InternFromString;
PyDict_SetItemString, PyObject_SetAttrString;
PyObject_DelAttrString; PyUnicode_InternFromString;
and the PyModule_Add convenience functions.
Always point out a non-immortalizing alternative.
* Don't immortalize user-provided attr names in _ctypes
(cherry picked from commit b4aedb23ae)
gh-121791: Check for `NULL` in `MethodDescriptor2_new` in `_testcapi` (GH-121792)
(cherry picked from commit 8b6d475581)
Co-authored-by: sobolevn <mail@sobolevn.me>
See 6b98b274b6 for an explanation of the problem and solution. Here I've applied the solution to channels.
(cherry picked from commit 8b209fd4f8, AKA gh-121805)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Any cross-interpreter mechanism for passing objects between interpreters must be very careful to respect isolation, even when the object is effectively immutable (e.g. int, str). Here this especially relates to when an interpreter sends one of its objects, and then is destroyed while the inter-interpreter machinery (e.g. queue) still holds a reference to the object.
When I added interpreters.Queue, I dealt with that case (using an atexit hook) by silently removing all items from the queue that were added by the finalizing interpreter.
Later, while working on concurrent.futures.InterpreterPoolExecutor (gh-116430), I noticed it was somewhat surprising when items were silently removed from the queue when the originating interpreter was destroyed. (See my comment on that PR.)
It took me a little while to realize what was going on. I expect that users, which much less context than I have, would experience the same pain.
My approach, here, to improving the situation is to give users three options:
1. return a singleton (interpreters.queues.UNBOUND) from Queue.get() in place of each removed item
2. raise an exception (interpreters.queues.ItemInterpreterDestroyed) from Queue.get() in place of each removed item
3. existing behavior: silently remove each item (i.e. Queue.get() skips each one)
The default will now be (1), but users can still explicitly opt in any of them, including to the silent removal behavior.
The behavior for each item may be set with the corresponding Queue.put() call. and a queue-wide default may be set when the queue is created. (This is the same as I did for "synconly".)
(cherry picked from commit 6b98b274b6, AKA gh-116431)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
On POSIX systems, excluding macOS framework installs, the lib directory
for the free-threaded build now includes a "t" suffix to avoid conflicts
with a co-located default build installation.
(cherry picked from commit e8c91d90ba)
Co-authored-by: Sam Gross <colesbury@gmail.com>
This makes select.poll() and kqueue() objects thread-safe in the
free-threaded build. Note that calling close() concurrently with other
functions is still not thread-safe due to races on file descriptors
(gh-121544).
(cherry picked from commit 44937d11a6)
Co-authored-by: Sam Gross <colesbury@gmail.com>
This fixes a mistake in gh-113012 and adds a test that verifies the fix.
(cherry picked from commit 35a67e36aa, AKA gh-121597)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
The `_PySeqLock_EndRead` function needs an acquire fence to ensure that
the load of the sequence happens after any loads within the read side
critical section. The missing fence can trigger bugs on macOS arm64.
Additionally, we need a release fence in `_PySeqLock_LockWrite` to
ensure that the sequence update is visible before any modifications to
the cache entry.
(cherry picked from commit 1d3cf79a50)
Co-authored-by: Sam Gross <colesbury@gmail.com>
* [3.13] gh-120782: Update internal type cache when reloading datetime
When reloading _datetime module, the single-phase version did not invoke the PyInit__datetime function, whereas the current multi-phase version updates the static types through the module init. The outdated static type cache in the interpreter state needs to be invalidated at the end of reloading the multi-phase module.
gh-116181: Remove Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE in rotatingtree.c (GH-121260)
(cherry picked from commit 705a123898)
Co-authored-by: AN Long <aisk@users.noreply.github.com>
Refactor the fast Unicode hash check into `_PyObject_HashFast` and use relaxed
atomic loads in the free-threaded build.
After this change, the TSAN doesn't report data races for this method.
(cherry picked from commit 294e724964)
Co-authored-by: AN Long <aisk@users.noreply.github.com>
1. Use pkg-config to check for ncursesw/panelw. If that fails, use
pkg-config to check for ncurses/panel.
2. Regardless of pkg-config output, search for curses/panel headers, so
we're sure we have all defines in pyconfig.h.
3. Regardless of pkg-config output, check if libncurses or libncursesw
contains the 'initscr' symbol; if it does _and_ pkg-config failed
earlier, add the resulting -llib linker option to CURSES_LIBS.
Ditto for 'update_panels' and PANEL_LIBS.
4. Wrap the rest of the checks with WITH_SAVE_ENV and make sure we're
using updated LIBS and CPPFLAGS for those.
Add the PY_CHECK_CURSES convenience macro.
(cherry picked from commit f80376b129)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Miss Islington (bot)