Some fields in PyCodeObject are lazily initialized. Use atomics and
critical sections to make their initializations and accesses thread-safe.
(cherry picked from commit 3926842117)
Co-authored-by: Sam Gross <colesbury@gmail.com>
* gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (gh-124865)
Fix a crash caused by immortal interned strings being shared between
sub-interpreters that use basic single-phase init. In that case, the string
can be used by an interpreter that outlives the interpreter that created and
interned it. For interpreters that share obmalloc state, also share the
interned dict with the main interpreter.
This is an un-revert of gh-124646 that then addresses the Py_TRACE_REFS
failures identified by gh-124785.
(cherry picked from commit f2cb399470)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* [3.13] gh-125286: Share the Main Refchain With Legacy Interpreters (gh-125709)
They used to be shared, before 3.12. Returning to sharing them resolves a failure on Py_TRACE_REFS builds.
---------
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-116938: Fix `dict.update` docstring and remove erraneous full stop from `dict` documentation (GH-125421)
(cherry picked from commit 5527c4051c)
Co-authored-by: Prometheus3375 <35541026+Prometheus3375@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Dictionary watchers on an object's attributes dictionary
(`object.__dict__`) were not triggered when the managed dictionary used
the object's inline values.
(cherry picked from commit 5989eb7446)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-125590: Allow FrameLocalsProxy to delete and pop keys from extra locals (GH-125616)
(cherry picked from commit 5b7a872b26)
Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
gh-125221: Fix free-threading data race in `object.__reduce_ex__` (GH-125267)
(cherry picked from commit b12e99261e)
Co-authored-by: Sam Gross <colesbury@gmail.com>
* Setting the __module__ attribute for a class now removes the
__firstlineno__ item from the type's dict.
* The _collections_abc and _pydecimal modules now completely replace the
collections.abc and decimal modules after importing them. This
allows to get the source of classes and functions defined in these
modules.
* inspect.findsource() now checks whether the first line number for a
class is out of bound.
(cherry picked from commit 69a4063ca5)
gh-123826: Fix unused function warnings in mimalloc on NetBSD (GH-123827)
(cherry picked from commit 4a6b1f1796)
Co-authored-by: Furkan Onder <furkanonder@protonmail.com>
gh-77894: Fix a crash when the GC breaks a loop containing a memoryview (GH-123898)
Now a memoryview object can only be cleared if there are no buffers
that refer it.
(cherry picked from commit a1dbf2ea69)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-124513: Check args in framelocalsproxy_new() (GH-124515)
Fix a crash in FrameLocalsProxy constructor: check the number of
arguments.
(cherry picked from commit d6954b6421)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-124498: Fix `TypeAliasType` not to be generic, when `type_params=()` (GH-124499)
(cherry picked from commit abe5f799e6)
Co-authored-by: sobolevn <mail@sobolevn.me>
gh-119004: fix a crash in equality testing between `OrderedDict` (GH-121329)
(cherry picked from commit 38a887dc3e)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-116510: Fix crash due to shared immortal interned strings. (gh-124646)
(cherry picked from commit 98b2ed7e23)
Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
gh-124538: Fix crash when using `gc.get_referents` on an untracked capsule object (GH-124559)
(cherry picked from commit f923605658)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Switch more _Py_IsImmortal(...) assertions to _Py_IsImmortalLoose(...)
The remaining calls to _Py_IsImmortal are in free-threaded-only code,
initialization of core objects, tests, and guards that fall back to
code that works with mortal objects.
(cherry picked from commit 57c471a688)
gh-123091: Use _Py_IsImmortalLoose() (#123511)
Use _Py_IsImmortalLoose() in bytesobject.c, typeobject.c
and ceval.c.
(cherry picked from commit f1a0d96f41)
* Parameters after the var-positional parameter are now keyword-only
instead of positional-or-keyword.
* Correctly calculate min_kw_only.
* Raise errors for invalid combinations of the var-positional parameter
with "*", "/" and deprecation markers.
(cherry picked from commit 8393608dd9)
gh-122527: Fix a crash on deallocation of `PyStructSequence` (GH-122577)
The `PyStructSequence` destructor would crash if it was deallocated after
its type's dictionary was cleared by the GC, because it couldn't compute
the "real size" of the instance. This could occur with relatively
straightforward code in the free-threaded build or with a reference
cycle involving the type in the default build, due to differing orders
in which `tp_clear()` was called.
Account for the non-sequence fields in `tp_basicsize` and use that,
along with `Py_SIZE()`, to compute the "real" size of a
`PyStructSequence` in the dealloc function. This avoids the accesses to
the type's dictionary during dealloc, which were unsafe.
(cherry picked from commit 4b63cd170e)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-122982: Extend the deprecation period for bool inversion by two years (GH-123306)
(cherry picked from commit 249b083ed8)
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Check that the current default heap is initialized in
`_mi_os_get_aligned_hint` and `mi_os_claim_huge_pages`.
The mimalloc function `_mi_os_get_aligned_hint` assumes that there is an
initialized default heap. This is true for our main thread, but not for
background threads. The problematic code path is usually called during
initialization (i.e., `Py_Initialize`), but it may also be called if the
program allocates large amounts of memory in total.
The crash only affected the free-threaded build.
(cherry picked from commit d061ffea7b)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-120974: Make asyncio `swap_current_task` safe in free-threaded build (GH-122317)
* gh-120974: Make asyncio `swap_current_task` safe in free-threaded build
(cherry picked from commit b5e6fb39a2)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-116622: Fix testPyObjectPrintOSError on Android (GH-122487)
Adds extra handling for way BSD/Android return errors from calls to fwrite.
(cherry picked from commit 82db572813)
Co-authored-by: Malcolm Smith <smith@chaquo.com>
The tests were only checking cases where the slot wrapper was present in the initial case. They were missing when the slot wrapper was added in the additional initializations. This fixes that.
(cherry-picked from commit 490e0ad83a, AKA gh-122248)
Update datetime module and test_type_cache.py to not call PyType_Modified.
(cherry picked from commit e55b05f29e, AKA gh--122182)
Co-authored-by: Mark Shannon <mark@hotpy.org>
In gh-121602, I applied a fix to a builtin types initialization bug.
That fix made sense in the context of some broader future changes,
but introduced a little bit of extra complexity. For earlier versions
those future changes are not relevant; we can avoid the extra complexity.
Thus we can revert that earlier change and replace it with this one,
which is more focused and conceptually simpler. This is essentially
the implementation of an idea that @markshannon pointed out to me.
Note that this change would be much smaller if we didn't have to deal
with repr compatibility for builtin types that explicitly inherit tp slots
(see expect_manually_inherited()). The alternative is to stop
*explicitly* inheriting tp slots in static PyTypeObject values,
which is churn that we can do separately.
gh-120974: Make _asyncio._leave_task atomic in the free-threaded build (GH-122139)
* gh-120974: Make _asyncio._leave_task atomic in the free-threaded build
Update `_PyDict_DelItemIf` to allow for an argument to be passed to the
predicate.
(cherry picked from commit a15feded71)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-121266: Remove Py_ALWAYS_INLINE in dictobject.c (GH-121493)
compare_unicode_generic(), compare_unicode_unicode() and
compare_generic() are callbacks used by do_lookup(). When enabling
assertions, it's not possible to inline these functions.
(cherry picked from commit c5a6b9afd8)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-121266: Change dict check_lookup() return type to int (GH-121581)
(cherry picked from commit 51da3dfbf3)
Co-authored-by: Victor Stinner <vstinner@python.org>
* Switch PyUnicode_InternInPlace to _PyUnicode_InternMortal, clarify docs
* Document immortality in some functions that take `const char *`
This is PyUnicode_InternFromString;
PyDict_SetItemString, PyObject_SetAttrString;
PyObject_DelAttrString; PyUnicode_InternFromString;
and the PyModule_Add convenience functions.
Always point out a non-immortalizing alternative.
* Don't immortalize user-provided attr names in _ctypes
(cherry picked from commit b4aedb23ae)
* The result has type Py_ssize_t, not intptr_t.
* Type cast between unsigned and signed integer types should be explicit.
* Downcasting should be explicit.
* Fix integer overflow check in sum().
(cherry picked from commit 1801545)
The object's inline values may be marked invalid if the materialized
dict was already initialized and then deleted.
(cherry picked from commit 162b41f577)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-113993: For string interning, do not rely on (or assert) _Py_IsImmortal (GH-121358)
Older stable ABI extensions are allowed to make immortal objects mortal.
Instead, use `_PyUnicode_STATE` (`interned` and `statically_allocated`).
(cherry picked from commit 956270d08d)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
We should maintain the invariant that a zero `ob_tid` implies the
refcount fields are merged.
* Move the assignment in `_Py_MergeZeroLocalRefcount` to immediately
before the refcount merge.
* Update `_PyTrash_thread_destroy_chain` to set `ob_ref_shared` to
`_Py_REF_MERGED` when setting `ob_tid` to zero.
Also check this invariant with assertions in the GC in debug builds.
That uncovered a bug when running out of memory during GC.
(cherry picked from commit d23be3947c)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-121660: Fix `ga_getitem` by explicitly checking for `NULL` result (GH-121661)
(cherry picked from commit bb802db8cf)
Co-authored-by: sobolevn <mail@sobolevn.me>
The `allocate_weakref` may return NULL when out of memory. We need to
handle that case and propagate the error.
(cherry picked from commit a640a605a8)
Co-authored-by: Sam Gross <colesbury@gmail.com>
When builtin static types are initialized for a subinterpreter, various "tp" slots have already been inherited (for the main interpreter). This was interfering with the logic in add_operators() (in Objects/typeobject.c), causing a wrapper to get created when it shouldn't. This change fixes that by preserving the original data from the static type struct and checking that.
(cherry picked from commit 5250a03133, AKA gh-121602)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
The only remaining race in dictobject.c was in _PyDict_CheckConsistency
when the dictionary has shared keys.
(cherry picked from commit 3ec719fabf)
Co-authored-by: Sam Gross <colesbury@gmail.com>
`munmap(NULL)` is not noop, like `free(NULL)` is.
Fixes an observed testsuite hang on 32-bit ARM systems.
(cherry picked from commit a802277914, AKA gh-121491)
Co-authored-by: Stefano Rivera <stefano@rivera.za.net>
GH-121439: Allow PyTupleObjects with an ob_size of 20 in the free_list to be reused (gh-121428)
(cherry picked from commit 9585a1a2a2)
Co-authored-by: satori1995 <132636720+satori1995@users.noreply.github.com>
The `used` field must be written using atomic stores because `set_len`
and iterators may access the field concurrently without holding the
per-object lock.
(cherry picked from commit 9c08f40a61)
Co-authored-by: Sam Gross <colesbury@gmail.com>
The `_PySeqLock_EndRead` function needs an acquire fence to ensure that
the load of the sequence happens after any loads within the read side
critical section. The missing fence can trigger bugs on macOS arm64.
Additionally, we need a release fence in `_PySeqLock_LockWrite` to
ensure that the sequence update is visible before any modifications to
the cache entry.
(cherry picked from commit 1d3cf79a50)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Refactor the fast Unicode hash check into `_PyObject_HashFast` and use relaxed
atomic loads in the free-threaded build.
After this change, the TSAN doesn't report data races for this method.
(cherry picked from commit 294e724964)
Co-authored-by: AN Long <aisk@users.noreply.github.com>
PyDict_Next no longer locks the dictionary in the free-threaded build. Locking
around individual PyDict_Next calls is not sufficient because the function
returns borrowed references and because it allows concurrent modifications
during the iteraiton loop.
The internal locking also interferes with correct external synchronization
because it may suspend outer critical sections created by the caller.
(cherry picked from commit 375b723d58)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Moves the logic to update the type's dictionary to its own function in order
to make the lock scoping more clear.
Also, ensure that `name` is decref'd on the error path.
(cherry picked from commit dee63cb359)
Co-authored-by: Sam Gross <colesbury@gmail.com>
* Add an InternalDocs file describing how interning should work and how to use it.
* Add internal functions to *explicitly* request what kind of interning is done:
- `_PyUnicode_InternMortal`
- `_PyUnicode_InternImmortal`
- `_PyUnicode_InternStatic`
* Switch uses of `PyUnicode_InternInPlace` to those.
* Disallow using `_Py_SetImmortal` on strings directly.
You should use `_PyUnicode_InternImmortal` instead:
- Strings should be interned before immortalization, otherwise you're possibly
interning a immortalizing copy.
- `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
`SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
backports, as they are now part of public API and version-specific ABI.
* Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.
* Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
- `_Py_ID`
- `_Py_STR` (including the empty string)
- one-character latin-1 singletons
Now, when you intern a singleton, that exact singleton will be interned.
* Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).
* Intern `_Py_STR` singletons at startup.
* For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup.
* Beef up the tests. Cover internal details (marked with `@cpython_only`).
* Add lots of assertions
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
- gh-119521: Rename IncompleteInputError to _IncompleteInputError and remove from public API/ABI (GH-119680)
(cherry picked from commit ce1064e4c9)
- gh-119521: Use `PyAPI_DATA`, not `extern`, for `_PyExc_IncompleteInputError` (GH-120955)
(cherry picked from commit ac61d58db0)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
This makes the following macros public as part of the non-limited C-API for
locking a single object or two objects at once.
* `Py_BEGIN_CRITICAL_SECTION(op)` / `Py_END_CRITICAL_SECTION()`
* `Py_BEGIN_CRITICAL_SECTION2(a, b)` / `Py_END_CRITICAL_SECTION2()`
The supporting functions and structs used by the macros are also exposed for
cases where C macros are not available.
(cherry picked from commit 8f17d69b7b)
This exposes `PyUnstable_Object_ClearWeakRefsNoCallbacks` as an unstable
C-API function to provide a thread-safe mechanism for clearing weakrefs
without executing callbacks.
Some C-API extensions need to clear weakrefs without calling callbacks,
such as after running finalizers like we do in subtype_dealloc.
Previously they could use `_PyWeakref_ClearRef` on each weakref, but
that's not thread-safe in the free-threaded build.
(cherry picked from commit e8752d7b80)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
gh-120182 added new global state (interp_count), but didn't add thread-safety for it. This change eliminates the possible race.
(cherry picked from commit 2c66318cdc, AKA gh-120529)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-117657: Make PyType_HasFeature (exported version) atomic (GH-120484)
Make PyType_HasFeature (exported version) atomic
(cherry picked from commit 6f63dfff6f)
Co-authored-by: Ken Jin <kenjin@python.org>
In gh-120009 I used an atexit hook to finalize the _datetime module's static types at interpreter shutdown. However, atexit hooks are executed very early in finalization, which is a problem in the few cases where a subclass of one of those static types is still alive until the final GC collection. The static builtin types don't have this probably because they are finalized toward the end, after the final GC collection. To avoid the problem for _datetime, I have applied a similar approach here.
Also, credit goes to @mgorny and @neonene for the new tests.
FYI, I would have liked to take a slightly cleaner approach with managed static types, but wanted to get a smaller fix in first for the sake of backporting. I'll circle back to the cleaner approach with a future change on the main branch.
(cherry picked from commit b2e71ff4f8, AKA gh-120182)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-117657: Make Py_TYPE and Py_SET_TYPE thread safe (GH-120165)
(cherry picked from commit e16aed63f6)
Co-authored-by: Ken Jin <kenjin@python.org>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
We need to write to `ob_ref_local` and `ob_tid` before `ob_ref_shared`.
Once we mark `ob_ref_shared` as merged, some other thread may free the
object because the caller also passes in `-1` as `extra` to give up its
only reference.
(cherry picked from commit 4055577221)
Co-authored-by: Sam Gross <colesbury@gmail.com>
We make use of the same mechanism that we use for the static builtin types. This required a few tweaks.
This change is the final piece needed to make _datetime support multiple interpreters. I've updated the module slot accordingly.
(cherry picked from commit 105f22ea46, AKA gh-119929)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
The free-threaded build currently immortalizes objects that use deferred
reference counting (see gh-117783). This typically happens once the
first non-main thread is created, but the behavior can be suppressed for
tests, in subinterpreters, or during a compile() call.
This fixes a race condition involving the tracking of whether the
behavior is suppressed.
(cherry picked from commit 47fb4327b5)
gh-117657: Fix data races report by TSAN unicode-hash (gh-119907)
(cherry picked from commit 0594a27e5f)
Co-authored-by: Donghee Na <donghee.na@python.org>
* Add docs for new APIs
* Add soft-deprecation notices
* Add What's New porting entries
* Update comments referencing `PyFrame_LocalsToFast()` to mention the proxy instead
* Other related cleanups found when looking for refs to the deprecated APIs
(cherry picked from commit 3859e09e3d)
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
* Remove the equivalence with real+imag*1j which can be incorrect in corner
cases (non-finite numbers, the sign of zeroes).
* Separately document the three roles of the constructor: parsing a string,
converting a number, and constructing a complex from components.
* Document positional-only parameters of complex(), float(), int() and bool()
as positional-only.
* Add examples for complex() and int().
* Specify the grammar of the string for complex().
* Improve the grammar of the string for float().
* Describe more explicitly the behavior when real and/or imag arguments are
complex numbers. (This will be deprecated in future.)
(cherry picked from commit ec1ba26460)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
The deadlock only affected the free-threaded build and only occurred
when the GIL was enabled at runtime. The `Py_DECREF(old_name)` call
might temporarily release the GIL while holding the type seqlock.
Another thread may spin trying to acquire the seqlock while holding the
GIL.
The deadlock occurred roughly 1 in ~1,000 runs of `pool_in_threads.py`
from `test_multiprocessing_pool_circular_import`.
(cherry picked from commit c22323cd1c)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Add `Py_BEGIN_CRITICAL_SECTION_SEQUENCE_FAST` and
`Py_END_CRITICAL_SECTION_SEQUENCE_FAST` macros and update `str.join` to use
them. Also add a regression test that would crash reliably without this
patch.
(cherry picked from commit baf347d916)
Co-authored-by: Josh {*()} Rosenberg <26495692+MojoVampire@users.noreply.github.com>
gh-119053: Implement the fast path for list.__getitem__ (gh-119112)
(cherry picked from commit ab4263a82a)
Co-authored-by: Donghee Na <donghee.na@python.org>
Miss Islington (bot)