Commit Graph

9746 Commits

Author SHA1 Message Date
Sam Gross cc581f32bf
gh-135099: Only wait on `_PyOS_SigintEvent()` in main thread (GH-135100)
On Windows, the `_PyOS_SigintEvent()` event handle is used to interrupt
the main thread when Ctrl-C is pressed. Previously, we also waited on
the event from other threads, but ignored the result. However, this can
race with interpreter shutdown because the main thread closes the handle
in `_PySignal_Fini` and threads may still be running and using mutexes
during interpreter shtudown.

Only use `_PyOS_SigintEvent()` in the main thread in parking_lot.c, like
we do in other places in the CPython codebase.
2025-06-04 09:35:56 +02:00
stratakis 485b499610
gh-128605: Add branch protections for x86_64 in asm_trampoline.S (#128606)
Apply Intel Control-flow Technology for x86-64 on asm_trampoline.S.

Required for mitigation against return-oriented programming (ROP)
and Call or Jump Oriented Programming (COP/JOP) attacks.

Manual application is required for the assembly files.

See also: https://sourceware.org/annobin/annobin.html/Test-cf-protection.html
2025-06-03 09:09:43 +02:00
Eric Snow 52deabefd0
gh-132775: Expand the Capability of Interpreter.call() (gh-133484)
It now supports most callables, full args, and return values.
2025-05-30 09:15:00 -06:00
Mark Shannon ce6a6371a2
GH-134879: Fix `INSTRUMENT_FOR_ITER` for list/tuple (#134897)
Fix INSTRUMENT_FOR_ITER for list/tuple
2025-05-30 07:11:42 -07:00
Victor Stinner ebf6d13567
gh-134745: Change PyThread_allocate_lock() implementation to PyMutex (#134747)
Co-authored-by: Sam Gross <colesbury@gmail.com>
2025-05-30 10:15:47 +00:00
Victor Stinner f49a07b531
gh-133968: Add PyUnicodeWriter_WriteASCII() function (#133973)
Replace most PyUnicodeWriter_WriteUTF8() calls with
PyUnicodeWriter_WriteASCII().

Unrelated change to please the linter: remove an unused
import in test_ctypes.

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-05-29 14:54:30 +00:00
Serhiy Storchaka bac3fcba5b
gh-108512: Add and use new replacements for PySys_GetObject() (GH-111035)
Add functions PySys_GetAttr(), PySys_GetAttrString(),
PySys_GetOptionalAttr() and PySys_GetOptionalAttrString().
2025-05-28 20:11:09 +03:00
Petr Viktorin 0d499c7e93
gh-128629: Add _Py_PACK_VERSION for CPython's own definitions (GH-134247)
Add _Py_PACK_VERSION for CPython's own definitions

Py_PACK_VERSION was added to limited API in 3.14, so if
Py_LIMITED_API is lower, the macro can't be used.
Add a private version that can be used in CPython headers
for checks like `Py_LIMITED_API+0 >= _Py_PACK_VERSION(3, 14)`.
2025-05-28 15:24:40 +02:00
Neil Schemenauer fbbbc10055
gh-127266: avoid data races when updating type slots (gh-133177)
In the free-threaded build, avoid data races caused by updating type
slots or type flags after the type was initially created.  For those
(typically rare) cases, use the stop-the-world mechanism.  Remove the
use of atomics when reading or writing type flags.
2025-05-27 18:27:41 -07:00
Noam Cohen 79d81f7cba
gh-131798: Optimize `_ITER_CHECK_TUPLE` (GH-134803) 2025-05-28 02:30:17 +08:00
Kumar Aditya ac539e7e0d
gh-132917: fix data race on `last_mem` in free-threading gc (#134692) 2025-05-27 22:42:08 +05:30
Eric Snow 9b5e80000e
gh-132775: Always Set __builtins__ In _PyFunction_FromXIData() (gh-134758)
This is a small follow-up to gh-133481.  There's a corner case
in the behavior of PyImport_ImportModuleAttrString(), where
it expects __builtins__ to be set if __globals__ is set.
2025-05-27 15:42:24 +00:00
Mark Shannon f6f4e8a662
GH-132554: "Virtual" iterators (GH-132555)
* FOR_ITER now pushes either the iterator and NULL or leaves the iterable and pushes tagged zero

* NEXT_ITER uses the tagged int as the index into the sequence or, if TOS is NULL, iterates as before.
2025-05-27 15:59:45 +01:00
neonene c60f39ada6
gh-134557: Suppress immortalization in _PyCode_GetScriptXIData under free-threading (gh-134686)
Disable immortalization around Py_CompileString*().

The same approach as 332356b that fixed the refleaks in compile() and eval().

E: 09e72cf can pass test_capi, test_sys and test__interpchannels with this patch for me.
2025-05-26 11:46:49 -06:00
Pablo Galindo Salgado 0909d6d8e8
gh-91048: Add better error messages for remote debugging for CI builds (#134682) 2025-05-26 15:31:47 +01:00
Russell Keith-Magee 965662ee4a
gh-91048: Correct Apple platform includes for iOS. (#134712)
Correct Apple platform includes for iOS.
2025-05-26 12:44:41 +00:00
Pablo Galindo Salgado 7b1a700231
Heavily comment Python/perf_jit_trampoline.c to improve maintainability (#134527)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2025-05-25 21:37:15 +01:00
Pablo Galindo Salgado 42b25ad4d3
gh-91048: Refactor and optimize remote debugging module (#134652)
Completely refactor Modules/_remote_debugging_module.c with improved
code organization, replacing scattered reference counting and error
handling with centralized goto error paths. This cleanup improves
maintainability and reduces code duplication throughout the module while
preserving the same external API.

Implement memory page caching optimization in Python/remote_debug.h to
avoid repeated reads of the same memory regions during debugging
operations. The cache stores previously read memory pages and reuses
them for subsequent reads, significantly reducing system calls and
improving performance.

Add code object caching mechanism with a new code_object_generation
field in the interpreter state that tracks when code object caches need
invalidation. This allows efficient reuse of parsed code object metadata
and eliminates redundant processing of the same code objects across
debugging sessions.

Optimize memory operations by replacing multiple individual structure
copies with single bulk reads for the same data structures. This reduces
the number of memory operations and system calls required to gather
debugging information from the target process.

Update Makefile.pre.in to include Python/remote_debug.h in the headers
list, ensuring that changes to the remote debugging header force proper
recompilation of dependent modules and maintain build consistency across
the codebase.

Also, make the module compatible with the free threading build as an extra :)

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2025-05-25 20:19:29 +00:00
Tomas R. 71dea74865
gh-131798: Small improvements to `remove_unneeded_uops` (GH-134554)
Improve remove_unneeded_uops
2025-05-23 20:48:45 +08:00
Brett Cannon ad42dc1909
GH-130397: remove special-casing of C stack depth for WASI (#134469)
Removed special-casing for WASI when setting C stack depth limits. Since WASI has its own C stack checking this isn't a security risk.

Also disabled some tests that stopped passing. They all happened to have already been disabled under Emscripten.
2025-05-22 14:08:44 -07:00
Eric Snow ac06b534ee
gh-132775: Fix Recently Introduced Warnings (gh-134530) 2025-05-22 17:24:09 +00:00
Tomas R. 484e00379b
GH-131798: Optimize away isinstance calls in the JIT (GH-134369) 2025-05-22 12:52:47 -04:00
Eric Snow 4a4ac3ab4d
gh-132775: Make _PyXI_session Opaque (gh-134452)
This is mostly a refactor to clean things up a bit, most notably the "XI namespace" code.

Making the session opaque requires adding the following internal-only functions:

* _PyXI_NewSession()
* _PyXI_FreeSession()
* _PyXI_GetMainNamespace()
2025-05-22 10:14:04 -06:00
Brandt Bucher ec736e7dae
GH-131798: Optimize cached class attributes and methods in the JIT (GH-134403) 2025-05-22 11:15:03 -04:00
Nadeshiko Manju 8c5e5557c6
GH-131798: Turn _LOAD_SMALL_INT into _LOAD_CONST_INLINE_BORROW in the JIT (GH-134406) 2025-05-22 09:54:57 -04:00
Eric Snow d0eedfa10e
gh-132775: Use _PyObject_GetXIData (With Fallback) (gh-134440)
This change includes some semi-related refactoring of queues and channels.
2025-05-22 06:50:06 -06:00
Eric Snow 88f8102a8f
gh-132775: Support Fallbacks in _PyObject_GetXIData() (gh-133482)
It now supports a "full" fallback to _PyFunction_GetXIData() and then `_PyPickle_GetXIData()`.  There's also room for other fallback modes if that later makes sense.
2025-05-21 07:23:48 -06:00
Peter Bierma b8998fe2d8
gh-131185: Use a proper thread-local for cached thread states (gh-132510)
Switches over to a _Py_thread_local in place of autoTssKey, and also fixes a few other checks regarding PyGILState_Ensure after finalization.

Note that this doesn't fix concurrent use of PyGILState_Ensure with Py_Finalize; I'm pretty sure zapthreads doesn't work at all, and that needs to be fixed seperately.
2025-05-21 07:01:25 -06:00
Brandt Bucher 2f0570caf4
GH-131798: Narrow types more aggressively in the JIT (GH-134373) 2025-05-20 18:09:51 -04:00
Nadeshiko Manju e1c0c451a2
GH-131798: Narrow the return type of _GET_LEN to int (GH-133345) 2025-05-20 18:02:50 -04:00
Hood Chatham 3b7888bf3d
gh-106213: Shorten Emscripten wasm-gc trampoline by a little (#133984)
Using the if instruction results in slightly shorter trampoline code.
2025-05-20 14:22:49 -04:00
Mark Shannon 6dcb0fdfe0
GH-134282: Always borrow references LOAD_CONST (GH-134284) 2025-05-20 11:24:11 -04:00
Tomas R. a7f317d730
GH-131798: Add _POP_CALL_TWO_LOAD_CONST_INLINE_BORROW (GH-134268) 2025-05-19 18:00:53 -04:00
Diego Russo 42d03f3933
GH-131798: Split CALL_LIST_APPEND into several uops (GH-134240) 2025-05-19 15:48:55 -04:00
Peter Bierma 27bd08273c
Revert "gh-128639: Don't assume one thread in subinterpreter finalization (gh-128640)" (gh-134256)
This reverts commit 9859791f9e.

The original change broke the iOS and android buildbots, where the tests are run single-process.
2025-05-19 12:22:05 -06:00
Victor Stinner e79f640eb6
Simplify interp_look_up_id() (#134257)
Don't use PyInterpreterState_GetID() but get directly the interpreter
'id' member which cannot fail.
2025-05-19 18:09:10 +00:00
Tomas R. 8d490b3687
GH-131798: Narrow the return type of isinstance for some known arguments in the JIT (GH-133172) 2025-05-19 13:19:24 -04:00
Peter Bierma 9859791f9e
gh-128639: Don't assume one thread in subinterpreter finalization (gh-128640)
Incidentally, this also fixed the warning not showing up if a subinterpreter wasn't
cleaned up via _interpreters.destroy. I had to update some of the tests as a result.
2025-05-19 10:24:08 -06:00
b-pass f2de1e6861
gh-134144: Fix use-after-free in zapthreads() (#134145) 2025-05-18 20:32:29 +05:30
Nico-Posada 4e9005d32f
gh-134100: Fix use-after-free in `PyImport_ImportModuleLevelObject` (#134117) 2025-05-18 12:41:38 +05:30
Victor Stinner 009e7b3698
gh-134064: Fix sys.remote_exec() error checking (#134067) 2025-05-18 00:24:40 +02:00
Max Bachmann 20095fb29a
Fix GetNamedPipeHandleStateW on non-desktop Windows API partitions (GH-134049) 2025-05-15 21:12:10 +01:00
Max Bachmann 5d118d0a92
gh-133580: Add missing exception to _sys_getwindowsversion_from_kernel32 (GH-133574) 2025-05-14 14:10:35 +01:00
Serhiy Storchaka c09cec5d69
gh-133886: Fix sys.remote_exec() for non-UTF-8 paths (GH-133887)
It now supports non-ASCII paths in non-UTF-8 locales and
non-UTF-8 paths in UTF-8 locales.
2025-05-13 11:55:24 +03:00
Eric Snow 8cf4947b0f
gh-132775: Add _PyFunction_GetXIData() (gh-133481) 2025-05-12 22:10:56 +00:00
Lauta c838e21fda
gh-133590: ensure that `TableEntry.linenumber_borrow` is initialized (#133681) 2025-05-11 10:16:28 +02:00
Bénédikt Tran dc191d2484
gh-133336: Remove comment about reserved -J in ``initconfig.c`` (#133821) 2025-05-10 22:31:45 +01:00
Jelle Zijlstra 7dddb4e667
gh-133783: Fix __replace__ on AST nodes for optional attributes (#133797) 2025-05-10 09:17:38 -07:00
Stan Ulbrych dbca27cfca
gh-133379: Fix misuse of the term "arguments" in error messages (GH-133382)
The right term is "parameters".
2025-05-10 15:00:43 +03:00
Bénédikt Tran 5044e85265
gh-133644: Remove deprecated Python initialization getter functions (#133661)
Remove functions:

* Py_GetExecPrefix()
* Py_GetPath()
* Py_GetPrefix()
* Py_GetProgramFullPath()
* Py_GetProgramName()
* Py_GetPythonHome()
2025-05-09 11:39:23 +00:00