Commit Graph

1581 Commits

Author SHA1 Message Date
Hood Chatham f3e275f1a9
GH-130396: Work around for broken `pthread_get_stackaddr_np` on Emscripten (#131088)
Tests / Ubuntu SSL tests with OpenSSL (3.1.7, ubuntu-24.04) (push) Blocked by required conditions Details
Tests / Ubuntu SSL tests with OpenSSL (3.2.3, ubuntu-24.04) (push) Blocked by required conditions Details
Tests / Ubuntu SSL tests with OpenSSL (3.3.2, ubuntu-24.04) (push) Blocked by required conditions Details
Tests / Ubuntu SSL tests with OpenSSL (3.4.0, ubuntu-24.04) (push) Blocked by required conditions Details
Tests / Hypothesis tests on Ubuntu (push) Blocked by required conditions Details
Tests / Address sanitizer (ubuntu-24.04) (push) Blocked by required conditions Details
Tests / All required checks pass (push) Blocked by required conditions Details
mypy / Run mypy on ${{ matrix.target }} (Lib/_pyrepl) (push) Successful in 14s Details
mypy / Run mypy on ${{ matrix.target }} (Lib/test/libregrtest) (push) Successful in 13s Details
mypy / Run mypy on ${{ matrix.target }} (Tools/build) (push) Successful in 11s Details
mypy / Run mypy on ${{ matrix.target }} (Tools/cases_generator) (push) Successful in 12s Details
mypy / Run mypy on ${{ matrix.target }} (Tools/clinic) (push) Successful in 11s Details
mypy / Run mypy on ${{ matrix.target }} (Tools/jit) (push) Successful in 11s Details
mypy / Run mypy on ${{ matrix.target }} (Tools/peg_generator) (push) Successful in 12s Details
Lint / lint (push) Failing after 1m32s Details
mypy / Run mypy on ${{ matrix.target }} (Tools/wasm) (push) Successful in 11s Details
Tests / Change detection (push) Successful in 26m17s Details
Tests / Docs (push) Has been skipped Details
Tests / Windows ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} (Win32, false, windows-latest) (push) Has been skipped Details
Tests / Windows ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} (arm64, true, windows-latest) (push) Has been skipped Details
Tests / Windows ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} (arm64, false, windows-latest) (push) Has been skipped Details
Tests / Windows ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} (x64, false, windows-latest) (push) Has been skipped Details
Tests / Windows ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} (x64, true, windows-latest) (push) Has been skipped Details
Tests / Windows MSI${{ '' }} (arm64) (push) Has been skipped Details
Tests / Windows MSI${{ '' }} (x64) (push) Has been skipped Details
Tests / Windows MSI${{ '' }} (x86) (push) Has been skipped Details
Tests / macOS ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} (true, ${{ github.repository_owner != 'python' }}, ghcr.io/cirruslabs/macos-runner:sonoma) (push) Failing after 24m9s Details
Tests / macOS ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} (false, ${{ github.repository_owner != 'python' }}, macos-13) (push) Failing after 33m1s Details
Tests / macOS ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} (true, ${{ github.repository_owner != 'python' }}, macos-14) (push) Failing after 33m1s Details
Tests / macOS ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} (false, ${{ github.repository_owner != 'python' }}, macos-14) (push) Failing after 33m3s Details
Tests / macOS ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} (false, ${{ github.repository_owner != 'python' }}, ghcr.io/cirruslabs/macos-runner:sonoma) (push) Failing after 33m5s Details
Tests / Ubuntu ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} ${{ fromJSON(matrix.bolt) && '(bolt)' || '' }} (false, false, ubuntu-24.04-arm) (push) Failing after 26m22s Details
Tests / Ubuntu ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} ${{ fromJSON(matrix.bolt) && '(bolt)' || '' }} (false, false, ubuntu-24.04) (push) Failing after 38m7s Details
Tests / WASI (push) Failing after 29m5s Details
Tests / Ubuntu ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} ${{ fromJSON(matrix.bolt) && '(bolt)' || '' }} (true, false, ubuntu-24.04) (push) Failing after 30m29s Details
Tests / CIFuzz (address) (push) Has been skipped Details
Tests / CIFuzz (memory) (push) Has been skipped Details
Tests / CIFuzz (undefined) (push) Has been skipped Details
Tests / Ubuntu ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} ${{ fromJSON(matrix.bolt) && '(bolt)' || '' }} (false, true, ubuntu-24.04) (push) Failing after 36m25s Details
Tests / Cross build Linux (push) Successful in 7m48s Details
Tests / Ubuntu ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} ${{ fromJSON(matrix.bolt) && '(bolt)' || '' }} (false, true, ubuntu-24.04-arm) (push) Failing after 39m37s Details
Tests / Thread sanitizer ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} (false) (push) Failing after 30m37s Details
Tests / Thread sanitizer ${{ fromJSON(matrix.free-threading) && '(free-threading)' || '' }} (true) (push) Successful in 49m39s Details
Mark stale pull requests / stale (push) Has been skipped Details
Tail calling interpreter / ${{ matrix.target }} (aarch64, 19, macos-14, aarch64-apple-darwin/clang) (push) Has been cancelled Details
Tail calling interpreter / ${{ matrix.target }} (aarch64, 19, ubuntu-24.04-arm, aarch64-unknown-linux-gnu/gcc) (push) Has been cancelled Details
Tail calling interpreter / ${{ matrix.target }} (x64, 19, windows-latest, x86_64-pc-windows-msvc/msvc) (push) Has been cancelled Details
Tail calling interpreter / ${{ matrix.target }} (x86_64, 19, macos-13, x86_64-apple-darwin/clang) (push) Has been cancelled Details
Tail calling interpreter / ${{ matrix.target }} (x86_64, 19, ubuntu-24.04, free-threading) (push) Has been cancelled Details
Tail calling interpreter / ${{ matrix.target }} (x86_64, 19, ubuntu-24.04, x86_64-unknown-linux-gnu/gcc) (push) Has been cancelled Details
Implements a workaround implementation of `pthread_get_stackaddr_np` for Emscripten.
This will be replaced by an implementation that will be included in Emscripten 4.0.6.
2025-03-14 07:04:40 +08:00
Tomasz Pytel c5abded099
gh-130382: add missing `_PyReftracerTrack` to ceval `Py_DECREF` (#130689) 2025-03-13 12:31:49 -04:00
Sam Gross 4162bc133b
gh-130396: Fix thread sanitizer crashes on stack overflow tests (gh-130966)
Thread sanitizer will often crash if a test uses more than half the
stack.
2025-03-11 10:33:23 -04:00
Mark Shannon 2bef8ea8ea
GH-127705: Use `_PyStackRef`s in the default build. (GH-127875) 2025-03-10 14:06:56 +00:00
Serhiy Storchaka 0ef4ffeefd
gh-130163: Fix crashes related to PySys_GetObject() (GH-130503)
The use of PySys_GetObject() and _PySys_GetAttr(), which return a borrowed
reference, has been replaced by using one of the following functions, which
return a strong reference and distinguish a missing attribute from an error:
_PySys_GetOptionalAttr(), _PySys_GetOptionalAttrString(),
_PySys_GetRequiredAttr(), and _PySys_GetRequiredAttrString().
2025-02-25 23:04:27 +02:00
Mark Shannon 014223649c
GH-130396: Use computed stack limits on linux (GH-130398)
* Implement C recursion protection with limit pointers for Linux, MacOS and Windows

* Remove calls to PyOS_CheckStack

* Add stack protection to parser

* Make tests more robust to low stacks

* Improve error messages for stack overflow
2025-02-25 09:24:48 +00:00
Petr Viktorin ef29104f7d
GH-91079: Revert "GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)" for now (GH130413)
Revert "GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)" for now

Unfortunatlely, the change broke some buildbots.

This reverts commit 2498c22fa0.
2025-02-24 11:16:08 +01:00
Mark Shannon 2498c22fa0
GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)
* Implement C recursion protection with limit pointers

* Remove calls to PyOS_CheckStack

* Add stack protection to parser

* Make tests more robust to low stacks

* Improve error messages for stack overflow
2025-02-19 11:44:57 +00:00
Ken Jin 46ac85e4d9
gh-129989: Change Py_TAIL_CALL_INTERP ifndef to ! (#130269)
Change Py_TAIL_CALL_INTERP ifndef to !
2025-02-18 15:48:49 +00:00
Ken Jin 359c7dde3b
gh-129989: Properly disable tailcall interp in configure (GH-129991)
Co-authored-by: Zanie Blue <contact@zanie.dev>
2025-02-16 03:01:24 +08:00
Mark Shannon 72f56654d0
GH-128682: Account for escapes in `DECREF_INPUTS` (GH-129953)
* Handle escapes in DECREF_INPUTS

* Mark a few more functions as escaping

* Replace DECREF_INPUTS with PyStackRef_CLOSE where possible
2025-02-12 17:44:59 +00:00
Michael Droettboom 00ec781877
gh-129244: Only remove the workaround when MSVC has the bugfix (#130011) 2025-02-11 14:49:42 -05:00
Ken Jin 5cdd6e5e75
gh-130004: Disable PGO for ceval.c on MSVC for default build (GH-130009)
Disable PGO for ceval.c on MSVC on default build
2025-02-12 01:13:05 +08:00
Ken Jin 247b50dec8
gh-130004: Revert commit 9e52e55 (GH-130005)
Revert commit 9e52e55
2025-02-11 23:23:58 +08:00
Irit Katriel a1417b211f
gh-100239: replace BINARY_SUBSCR & family by BINARY_OP with oparg NB_SUBSCR (#129700) 2025-02-07 22:39:54 +00:00
Brandt Bucher 70e387c990
GH-129709: Clean up tier two (GH-129710) 2025-02-07 09:52:49 -08:00
Brandt Bucher fbaa6c8ff0
GH-129763: Remove the LLTRACE macro (GH-129764) 2025-02-07 08:49:51 -08:00
Ken Jin cb640b659e
gh-128563: A new tail-calling interpreter (GH-128718)
Co-authored-by: Garrett Gu <garrettgu777@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2025-02-06 23:21:57 +08:00
Mark Shannon 96ff4c2486
GH-128682: Mark two more macros as escaping. (GH-129645)
Expand out SETLOCAL so that code generator can see the decref. Mark Py_CLEAR as escaping
2025-02-04 14:00:51 +00:00
Mark Shannon 54f74b80ae
GH-128563: Move some labels, to simplify implementing tailcalling interpreter. (GH-129525) 2025-01-31 17:13:20 +00:00
Mark Shannon c3ae5c9e4a
GH-128563: Simplify recursion check in `_PyEval_EvalFrameDefault` (GH-129481)
Simplify recursion check in _PyEval_EvalFrameDefault
2025-01-31 12:12:24 +00:00
Ken Jin 87fb8b198c
gh-128563: Move labels in ceval.c to bytecodes.c (GH-129112) 2025-01-27 18:30:20 +08:00
Irit Katriel c39ae8922b
gh-128799: Add frame of except* to traceback when wrapping a naked exception (#128971) 2025-01-25 13:00:23 +00:00
Michael Droettboom 9e52e553f4
gh-129244: Remove workaround for MSVC compiler crash (#129263)
* Remove compiler workaround

* Remote _Py_USING_PGO
2025-01-25 07:56:13 -05:00
Chris Eibl 8fecb9fa0b
Remove unused DPRINTF in ceval.c (GH-129282)
remove unused DPRINTF in ceval.c
2025-01-25 18:33:26 +08:00
Mark Shannon 470a0a68eb
GH-128682: Change a couple of functions to only steal references on success. (GH-129132)
Change PyTuple_FromStackRefSteal and PyList_FromStackRefSteal to only steal on success to avoid escaping
2025-01-22 10:51:37 +00:00
Ken Jin 5809b25909
gh-128563: Move lltrace into the frame struct (GH-129113) 2025-01-21 22:17:15 +08:00
Mark Shannon f5b6356a11
GH-128563: Add new frame owner type for interpreter entry frames (GH-129078)
Add new frame owner type for interpreter entry frames
2025-01-21 10:15:02 +00:00
mpage 13c4def692
gh-128955: Fix goto if tlbc creation fails when throwing into a generator (#128957)
We don't have the correct copy of the bytecode and can't update next_instr
appropriately, so just unwind.
2025-01-17 12:53:29 -08:00
Peter Bierma f6c61bf2d7
gh-128717: Stop-the-world when setting the recursion limit (#128741) 2025-01-12 18:34:30 +05:30
Nico-Posada 3879ca0100
gh-128049: Fix type confusion bug with the return value of a custom ExceptionGroup split function (#128079) 2024-12-20 19:20:31 +00:00
Mark Shannon 128cc47fbd
GH-127705: Add debug mode for `_PyStackRef`s inspired by HPy debug mode (GH-128121) 2024-12-20 16:52:20 +00:00
Shantanu 45e6dd63b8
gh-128030: Avoid error from PyModule_GetFilenameObject for non-module (#128047)
I missed the extra `PyModule_Check` in #127660 because I was looking at
3.12 as the base implementation for import from. This meant that I
missed the `PyModuleCheck` introduced in #112661.
2024-12-20 00:22:26 -08:00
Mark Shannon e62e1ca455
GH-126833: Dumps graphviz representation of executor graph. (GH-126880) 2024-12-13 11:00:00 +00:00
Shantanu 3983527c3a
gh-127651: Use __file__ in diagnostics if origin is missing (#127660)
See the left hand side in https://github.com/python/cpython/pull/123929/files#diff-c22186367cbe20233e843261998dc027ae5f1f8c0d2e778abfa454ae74cc59deL2840-L2849

---------

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-12-09 20:55:20 -08:00
Neil Schemenauer 0cb5222079
gh-115999: Specialize `LOAD_SUPER_ATTR` in free-threaded builds (gh-127128)
Use existing helpers to atomically modify the bytecode.  Add unit tests
to ensure specializing is happening as expected.  Add test_specialize.py
that can be used with ThreadSanitizer to detect data races.  
Fix thread safety issue with cell_set_contents().
2024-12-03 09:32:26 -08:00
Mark Shannon a8dd821d5b
GH-126491: GC: Mark objects reachable from roots before doing cycle collection (GH-127110)
* Mark almost all reachable objects before doing collection phase

* Add stats for objects marked

* Visit new frames before each increment

* Update docs

* Clearer calculation of work to do.
2024-12-02 10:12:17 +00:00
Sam Gross d3da04bfc9
gh-127022: Remove `_PyEvalFramePushAndInit_UnTagged` (gh-127168)
The interpreter now handles `_PyStackRef`s pointing to immortal objects
without the deferred bit set, so `_PyEvalFramePushAndInit_UnTagged` is
no longer necessary.
2024-11-25 12:24:37 -05:00
Eric Snow 9dabace39d
gh-114940: Add _Py_FOR_EACH_TSTATE_UNLOCKED(), and Friends (gh-127077)
This is a precursor to the actual fix for gh-114940, where we will change these macros to use the new lock.  This change is almost entirely mechanical; the exceptions are the loops in codeobject.c and ceval.c, which now hold the "head" lock.  Note that almost all of the uses of _Py_FOR_EACH_TSTATE_UNLOCKED() here will change to _Py_FOR_EACH_TSTATE_BEGIN() once we add the new per-interpreter lock.
2024-11-21 11:08:38 -07:00
Pablo Galindo Salgado 30aeb00d36
gh-126076: Account for relocated objects in tracemalloc (#126077) 2024-11-19 10:35:17 +00:00
Hugo van Kemenade 899fdb213d
Revert "GH-126491: GC: Mark objects reachable from roots before doing cycle collection (GH-126502)" (#126983) 2024-11-19 11:25:09 +02:00
Mark Shannon b0fcc2c47a
GH-126491: GC: Mark objects reachable from roots before doing cycle collection (GH-126502)
* Mark almost all reachable objects before doing collection phase

* Add stats for objects marked

* Visit new frames before each increment

* Remove lazy dict tracking

* Update docs

* Clearer calculation of work to do.
2024-11-18 14:31:26 +00:00
mpage 2e95c5ba3b
gh-115999: Implement thread-local bytecode and enable specialization for `BINARY_OP` (#123926)
Each thread specializes a thread-local copy of the bytecode, created on the first RESUME, in free-threaded builds. All copies of the bytecode for a code object are stored in the co_tlbc array on the code object. Threads reserve a globally unique index identifying its copy of the bytecode in all co_tlbc arrays at thread creation and release the index at thread destruction. The first entry in every co_tlbc array always points to the "main" copy of the bytecode that is stored at the end of the code object. This ensures that no bytecode is copied for programs that do not use threads.

Thread-local bytecode can be disabled at runtime by providing either -X tlbc=0 or PYTHON_TLBC=0. Disabling thread-local bytecode also disables specialization.

Concurrent modifications to the bytecode made by the specializing interpreter and instrumentation use atomics, with specialization taking care not to overwrite an instruction that was instrumented concurrently.
2024-11-04 11:13:32 -08:00
Shantanu 500f5338a8
gh-123930: Better error for "from imports" when script shadows module (#123929) 2024-10-24 12:11:12 -07:00
Sam Gross 3c4a7fa617
gh-124218: Avoid refcount contention on builtins module (GH-125847)
This replaces `_PyEval_BuiltinsFromGlobals` with
`_PyDict_LoadBuiltinsFromGlobals`, which returns a new reference
instead of a borrowed reference. Internally, the new function uses
per-thread reference counting when possible to avoid contention on the
refcount fields on the builtins module.
2024-10-24 12:44:38 -04:00
Pablo Galindo Salgado 3d1df3d84e
gh-125703: Correctly honour tracemalloc hooks on more PyDECREF specialized paths (#125712) 2024-10-21 15:39:05 +01:00
Eric Snow 6d93690954
gh-125604: Move _Py_AuditHookEntry, etc. Out of pycore_runtime.h (gh-125605)
This is essentially a cleanup, moving a handful of API declarations to the header files where they fit best, creating new ones when needed.

We do the following:

* add pycore_debug_offsets.h and move _Py_DebugOffsets, etc. there
* inline struct _getargs_runtime_state and struct _gilstate_runtime_state in _PyRuntimeState
* move struct _reftracer_runtime_state to the existing pycore_object_state.h
* add pycore_audit.h and move to it _Py_AuditHookEntry , _PySys_Audit(), and _PySys_ClearAuditHooks
* add audit.h and cpython/audit.h and move the existing audit-related API there
*move the perfmap/trampoline API from cpython/sysmodule.h to cpython/ceval.h, and remove the now-empty cpython/sysmodule.h
2024-10-18 09:26:08 -06:00
Michael Droettboom 37986e830b
gh-123153: Fix PGO builds with free-threading on Windows (#125607)
* gh-123153: Fix PGO builds with free-threading

* Redo how the #define works
2024-10-17 08:20:30 -04:00
Michael Droettboom 51410d8bdc
gh-125217: Turn off optimization around_PyEval_EvalFrameDefault to avoid MSVC crash (#125477) 2024-10-16 12:51:15 +00:00
Victor Stinner b9a8ca0a6a
gh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_STR) (#125194)
Replace PyUnicode_New(0, 0), PyUnicode_FromString("")
and PyUnicode_FromStringAndSize("", 0)
with Py_GetConstant(Py_CONSTANT_EMPTY_STR).
2024-10-09 17:15:23 +02:00