Commit Graph

4697 Commits

Author SHA1 Message Date
Bogdan Romanyuk 365cf5fc23
gh-117657: Fix data race in `new_reference` for free threaded build (gh-129665) 2025-02-06 15:35:37 -05:00
sobolevn 8b2fb62933
gh-129707: Check `Tools/build/compute-changes.py` with `mypy` (#129708)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2025-02-06 15:47:29 +00:00
Ken Jin cb640b659e
gh-128563: A new tail-calling interpreter (GH-128718)
Co-authored-by: Garrett Gu <garrettgu777@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2025-02-06 23:21:57 +08:00
Adam Turner 7d9a22f509
Convert change detection to a Python script (#129627)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
2025-02-05 16:39:42 +00:00
Tomas R. e41ec8e18b
gh-104400: pygettext: Prepare to replace TokenEater with a NodeVisitor (#129672)
* Update the module docstring
* Move ``key_for`` inside the class
* Move ``write_pot_file`` outside the class
2025-02-04 22:59:23 +00:00
Mark Shannon 96ff4c2486
GH-128682: Mark two more macros as escaping. (GH-129645)
Expand out SETLOCAL so that code generator can see the decref. Mark Py_CLEAR as escaping
2025-02-04 14:00:51 +00:00
Mark Shannon 2effea4dab
GH-128682: Spill the stack pointer in labels, as well as instructions (GH-129618) 2025-02-04 12:18:31 +00:00
Mark Shannon 75b628adeb
GH-128563: Generate `opcode = ...` in instructions that need `opcode` (GH-129608)
* Remove support for GO_TO_INSTRUCTION
2025-02-03 15:09:21 +00:00
Mark Shannon 808071b994
GH-128682: Make `PyStackRef_CLOSE` escaping. (GH-129404) 2025-02-03 12:41:32 +00:00
Diego Russo 567394517a
GH-128842: Collect JIT memory stats (GH-128941) 2025-02-02 15:17:53 -08:00
Adam Turner 237f186da4
gh-104400: Remove ``fintl.gettext`` from pygettext (#129580)
The ``fintl`` module is never installed or tested, meaning that the
fallback identity function is unconditionally used for ``_()``.
This means we can simplify, converting the docstring to a real
docstring, and converting some other strings to f-strings.

We also convert the module to UTF-8, sort imports,
and remove the history comment, which was last updated in 2002.
Consult the git history for a more accurate summary of changes.
2025-02-02 14:30:34 +00:00
Mark Shannon 54f74b80ae
GH-128563: Move some labels, to simplify implementing tailcalling interpreter. (GH-129525) 2025-01-31 17:13:20 +00:00
Irit Katriel 5ab9604683
clearer error and suggestion when c-analyzer cannot read ignored.tsv (#129423) 2025-01-30 11:38:52 +00:00
Stan Ulbrych 652f66ac38
gh-129438: Update ``--enable-experimental-jit`` section with install requirements (#129450)
Add a note to the `JIT` docs that building CPython with `JIT` requires Python 3.11 or newer.

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2025-01-30 12:16:05 +02:00
Pieter Eendebak a29221675e
gh-127119: Faster check for small ints in long_dealloc (GH-127620) 2025-01-29 15:22:18 +00:00
Irit Katriel 4815131910
gh-100239: specialize bitwise logical binary ops on ints (#128927) 2025-01-29 09:28:21 +00:00
🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) 03d9cdb729
Merge TSAN test matrices in CI (#123278) 2025-01-29 11:16:51 +02:00
Brandt Bucher 828b27680f
GH-126599: Remove the PyOptimizer API (GH-129194) 2025-01-28 16:10:51 -08:00
T. Wouters 5c930a26fb
gh-115999: Enable free-threaded specialization of LOAD_CONST (#129365)
Enable free-threaded specialization of LOAD_CONST.
2025-01-29 01:07:56 +01:00
Sam Gross d23f5701ad
gh-128844: Make `_Py_TryIncref` public as an unstable API. (#128926)
This exposes `_Py_TryIncref` as `PyUnstable_TryIncref()` and the helper
function `_PyObject_SetMaybeWeakref` as `PyUnstable_EnableTryIncRef`.

These are helpers for dealing with unowned references in a safe way,
particularly in the free threading build.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2025-01-28 19:32:27 +00:00
Ken Jin 87fb8b198c
gh-128563: Move labels in ceval.c to bytecodes.c (GH-129112) 2025-01-27 18:30:20 +08:00
sobolevn cc4f0a2770
Replace `strict_concatenate = True` with `extra_checks = True` (#126391) 2025-01-25 12:44:23 +03:00
Sam Gross a10f99375e
Revert "GH-128914: Remove conditional stack effects from `bytecodes.c` and the code generators (GH-128918)" (GH-129202)
The commit introduced a ~2.5-3% regression in the free threading build.

This reverts commit ab61d3f430.
2025-01-23 09:26:25 +00:00
Yury Selivanov 188598851d
GH-91048: Add utils for capturing async call stack for asyncio programs and enable profiling (#124640)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Savannah Ostrowski <savannahostrowski@gmail.com>
Co-authored-by: Jacob Coffee <jacob@z7x.org>
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2025-01-22 17:25:29 +01:00
Mark Shannon 470a0a68eb
GH-128682: Change a couple of functions to only steal references on success. (GH-129132)
Change PyTuple_FromStackRefSteal and PyList_FromStackRefSteal to only steal on success to avoid escaping
2025-01-22 10:51:37 +00:00
Ken Jin 86c1a60d5a
gh-128563: Move GO_TO_INSTRUCTION and PREDICT to cases generator (GH-129115) 2025-01-22 09:22:25 +08:00
Mark Shannon f5b6356a11
GH-128563: Add new frame owner type for interpreter entry frames (GH-129078)
Add new frame owner type for interpreter entry frames
2025-01-21 10:15:02 +00:00
Petr Viktorin d3b1bb228c
gh-128156: Guard use of `ffi_type_complex_double` on macOS system libffi (GH-128680)
* Determine ffi complex support at runtime
* Also, generate SIMPLE_TYPE_CHARS once at runtime
2025-01-21 10:59:18 +01:00
Mark Shannon ab61d3f430
GH-128914: Remove conditional stack effects from `bytecodes.c` and the code generators (GH-128918) 2025-01-20 17:09:23 +00:00
Mark Shannon f0f7b978be
GH-128939: Refactor JIT optimize structs (GH-128940) 2025-01-20 15:49:15 +00:00
Erlend E. Aasland 537296cdcd
gh-111178: Generate correct signature for most self converters (#128447) 2025-01-20 12:40:18 +01:00
Mark Shannon b5558cd63c
Refactor code generators a bit (GH-128920)
Refactor code generators a bit to avoid passing stack property around all over the place
2025-01-17 16:59:30 +00:00
Xuanteng Huang b44ff6d0df
GH-126599: Remove the "counter" optimizer/executor (GH-126853) 2025-01-16 15:57:04 -08:00
Irit Katriel 3893a92d95
gh-100239: specialize long tail of binary operations (#128722) 2025-01-16 15:22:13 +00:00
mpage b5ee0258bf
gh-115999: Specialize `LOAD_ATTR` for instance and class receivers in free-threaded builds (#128164)
Finish specialization for LOAD_ATTR in the free-threaded build by adding support for class and instance receivers.
2025-01-14 11:56:11 -08:00
Mark Shannon f49a1df6f3
GH-128682: Convert explicit loops closing arrays into `DECREF_INPUTS`. (GH-128822)
* Mark Py_DECREF and Py_XDECREF as escaping

* Remove explicit loops for clearing array inputs
2025-01-14 15:08:56 +00:00
Mark Shannon 517dc65ffc
GH-128682: Stronger checking of `PyStackRef_CLOSE` and `DEAD`. (GH-128683) 2025-01-13 12:37:48 +00:00
Mark Shannon 39fc7ef4fe
GH-124483: Mark `Py_DECREF`, etc. as escaping for the JIT (GH-128678) 2025-01-13 11:42:45 +00:00
Brandt Bucher 65ae3d5a73
GH-127809: Fix the JIT's understanding of ** (GH-127844) 2025-01-07 17:25:48 -08:00
Mark Shannon f826beca0c
GH-128375: Better instrument for `FOR_ITER` (GH-128445) 2025-01-06 17:54:47 +00:00
Mark Shannon b9c693dcca
GH-128073: Include `EXIT_IF` when checking for escaping calls (GH-128537) 2025-01-06 14:16:22 +00:00
Erlend E. Aasland a4e773c540
gh-128152: Argument Clinic: ignore pre-processor directives inside C comments (#128464) 2025-01-04 11:46:04 +01:00
Sam Gross f1574859d7
gh-125985: Fix `cmodule_function()` scaling benchmark (#128460)
Add a separate benchmark that measures the effect of
`_PyObject_LookupSpecial()` on scaling.

In the process of cleaning up the scaling benchmarks for inclusion, I
unintentionally changed the "cmodule_function" benchmark to pass an
`int` to `math.floor()` instead of a `float`, which causes it to use the
`_PyObject_LookupSpecial()` code path. `_PyObject_LookupSpecial()` has
its own scaling issues that we want to measure separately from calling a
function on a C module.
2025-01-03 21:48:47 +00:00
Erlend E. Aasland 8abd6cef68
gh-115765: Upgrade to GNU Autoconf 2.72 (#128411) 2025-01-03 11:37:54 +00:00
Kumar Aditya e389d6c650
gh-128277: make globals variables thread safe in socket module (#128286) 2024-12-31 19:10:06 +05:30
Mark Shannon d2f1d917e8
GH-122548: Implement branch taken and not taken events for sys.monitoring (GH-122564) 2024-12-19 16:59:51 +00:00
Peter Bierma b5d1e4552f
gh-128083: Fix macro redefinition warning in clinic. (GH-127950) 2024-12-19 15:00:30 +01:00
Pablo Galindo Salgado 47c5a0f307
gh-125588: Allow to regenerate the parser with Python < 3.12 (#127969)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2024-12-15 23:17:01 +00:00
Stan Ulbrych 7b8bd3b2b8
gh-119786: Fix miscellaneous typos in `InternalDocs/interpreter_definition.md` (#127957) 2024-12-15 19:11:50 +02:00
mpage 2de048ce79
gh-115999: Specialize loading attributes from modules in free-threaded builds (#127711)
We use the same approach that was used for specialization of LOAD_GLOBAL in free-threaded builds:

_CHECK_ATTR_MODULE is renamed to _CHECK_ATTR_MODULE_PUSH_KEYS; it pushes the keys object for the following _LOAD_ATTR_MODULE_FROM_KEYS (nee _LOAD_ATTR_MODULE). This arrangement avoids having to recheck the keys version.

_LOAD_ATTR_MODULE is renamed to _LOAD_ATTR_MODULE_FROM_KEYS; it loads the value from the keys object pushed by the preceding _CHECK_ATTR_MODULE_PUSH_KEYS at the cached index.
2024-12-13 10:17:16 -08:00
Victor Stinner 6446408d42
gh-102471, PEP 757: Add PyLong import and export API (#121339)
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
Co-authored-by: Steve Dower <steve.dower@microsoft.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-12-13 14:24:48 +01:00
Hood Chatham 3b18af964d
gh-127629: Add ctypes to the Emscripten build (#127683)
Adds tooling to build libffi and add ctypes to the stdlib for Emscripten.
2024-12-10 10:32:58 +08:00
Hood Chatham d8d12b37b5
gh-127503: Fix realpath handling in emscripten cli (#127632)
Corrects the handling of realpath on Linux.
2024-12-09 10:03:11 +08:00
Hood Chatham 5876063d06
gh-127503 Don't propagate native PATH to Emscripten Python (#127633)
Modifies the handling of PATH to ensure that native executables aren't picked
up when running under node.
2024-12-09 10:01:37 +08:00
Stan Ulbrych 2367759212
[doc] Fix typos in `interpreter_definition.md` (#127742) 2024-12-08 18:01:55 +00:00
Brett Cannon 0fc4063747
GH-127652: stop using `--wasi preview2` in `wasi.py` (GH-127704)
It's only to use WASI 0.2 code to back preview1 APIs and is considered experimental anyway.
2024-12-06 18:42:05 +00:00
Hood Chatham 87faf0a9c4
gh-127503: Emscripten make Python.sh function as proper Python CLI (#127506)
Modifies the python.sh script to work on macOS, and adapt to recent emscripten changes.
2024-12-05 08:44:50 +08:00
Hood Chatham 2f1cee8477
gh-127111: Apply prettier formatter to Emscripten web example (#127551)
Cleaned up formatting (and a stray closing tag) of the web example HTML and JS.
2024-12-05 08:25:06 +08:00
Diego Russo 7c5a6f67c7
Enable native AArch64 Ubuntu CI jobs (#127584)
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2024-12-04 14:12:06 -08:00
Hood Chatham bfb0788bfc
gh-127111: Emscripten Make web example work again (#127113)
Moves the Emscripten web example into a standalone folder, and updates 
Makefile targets to build the web example. Instructions for usage have
also been added.
2024-12-03 07:30:24 +08:00
Mark Shannon a8dd821d5b
GH-126491: GC: Mark objects reachable from roots before doing cycle collection (GH-127110)
* Mark almost all reachable objects before doing collection phase

* Add stats for objects marked

* Visit new frames before each increment

* Update docs

* Clearer calculation of work to do.
2024-12-02 10:12:17 +00:00
alm 4e0a4cafe8
summarize: Fix typo in stats (#127450) 2024-11-30 18:07:54 +02:00
Peter Bierma 99490913a0
gh-127341: Argument Clinic: fix compiler warnings for getters with docstrings (#127310)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2024-11-29 11:12:13 +01:00
Илья Любавский dd3a87d2a8
gh-127303: Add docs for token.EXACT_TOKEN_TYPES (#127304)
---------

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-11-29 04:00:50 -05:00
Petr Viktorin db5c5763f3
gh-127330: Update for OpenSSL 3.4 & document+improve the update process (GH-127331)
- Add `git describe` output to headers generated by `make_ssl_data.py`

  This info is more important than the date when the file was generated.
  It does mean that the tool now requires a Git checkout of OpenSSL,
  not for example a release tarball.

- Regenerate the older file to add the info.
  To the other older file, add a note about manual edits.

- Add notes on how to add a new OpenSSL version

- Add 3.4 error messages and multissl tests
2024-11-28 13:29:27 +01:00
mpage 193890c1cc
gh-126612: Include stack effects of uops when computing maximum stack depth (#126894) 2024-11-26 00:53:49 +00:00
Brandt Bucher 17c16aea66
GH-115869: Make jit_stencils.h reproducible (GH-127166) 2024-11-24 14:42:50 -08:00
Filipe Laíns 🇵🇸 a264637654
GH-89435: os.path should not be a frozen module (#126924) 2024-11-22 18:50:30 +00:00
Sam Gross 4759ba6eec
gh-127022: Simplify `PyStackRef_FromPyObjectSteal` (#127024)
This gets rid of the immortal check in `PyStackRef_FromPyObjectSteal()`.
Overall, this improves performance about 2% in the free threading
build.

This also renames `PyStackRef_Is()` to `PyStackRef_IsExactly()` because
the macro requires that the tag bits of the arguments match, which is
only true in certain special cases.
2024-11-22 12:55:33 -05:00
Tomas R. 0a1944cda8
gh-126700: pygettext: Support more gettext functions (GH-126912)
Support multi-argument gettext functions: ngettext(), pgettext(), dgettext(), etc.
2024-11-22 16:52:16 +02:00
Peter Bierma 89125e9f9f
Allow local use of `static PyMutex` in the C analyzer (#127102) 2024-11-22 08:48:39 +01:00
mpage 09c240f20c
gh-115999: Specialize `LOAD_GLOBAL` in free-threaded builds (#126607)
Enable specialization of LOAD_GLOBAL in free-threaded builds.

Thread-safety of specialization in free-threaded builds is provided by the following:

A critical section is held on both the globals and builtins objects during specialization. This ensures we get an atomic view of both builtins and globals during specialization.
Generation of new keys versions is made atomic in free-threaded builds.
Existing helpers are used to atomically modify the opcode.
Thread-safety of specialized instructions in free-threaded builds is provided by the following:

Relaxed atomics are used when loading and storing dict keys versions. This avoids potential data races as the dict keys versions are read without holding the dictionary's per-object lock in version guards.
Dicts keys objects are passed from keys version guards to the downstream uops. This ensures that we are loading from the correct offset in the keys object. Once a unicode key has been stored in a keys object for a combined dictionary in free-threaded builds, the offset that it is stored in will never be reused for a different key. Once the version guard passes, we know that we are reading from the correct offset.
The dictionary read fast-path is used to read values from the dictionary once we know the correct offset.
2024-11-21 11:22:21 -08:00
Victor Stinner 3c2bd66e21
gh-126316: Make grp.getgrall() thread-safe: add a mutex (#127055)
grpmodule.c is no longer built with the limited C API, since PyMutex
is excluded from the limited C API.
2024-11-21 15:47:24 +01:00
Hood Chatham 1629d2ca56
gh-126898: Emscripten support: Use es6 modules (#126903)
Modify Emscripten support to use ES6 modules.
2024-11-21 15:10:46 +11:00
CoderTCY 88dc84bcf9
gh-125729: Makes the installation of the turtle module dependent on the Tcl/Tk install option (GH-126176) 2024-11-19 17:19:33 +00:00
Hugo van Kemenade 899fdb213d
Revert "GH-126491: GC: Mark objects reachable from roots before doing cycle collection (GH-126502)" (#126983) 2024-11-19 11:25:09 +02:00
Mark Shannon b0fcc2c47a
GH-126491: GC: Mark objects reachable from roots before doing cycle collection (GH-126502)
* Mark almost all reachable objects before doing collection phase

* Add stats for objects marked

* Visit new frames before each increment

* Remove lazy dict tracking

* Update docs

* Clearer calculation of work to do.
2024-11-18 14:31:26 +00:00
Hood Chatham 544b001b23
gh-126691: Remove --with-emscripten-target (#126787)
This unifies the code for nodejs and the code for the browser. After this
commit, the browser example doesn't work; this will be fixed in a
subsequent update.
2024-11-16 09:44:05 +08:00
Steve Dower 9332a6f825
gh-118973: Add _tkinter to freethreaded MSI (GH-126768) 2024-11-15 12:22:56 +00:00
Tomas R. 9a456383be
gh-126807: pygettext: Do not attempt to extract messages from function definitions. (GH-126808)
Fixes a bug where pygettext would attempt
to extract a message from a code like this:

def _(x): pass

This is because pygettext only looks at one
token at a time and '_(x)' looks like a
function call.

However, since 'x' is not a string literal,
it would erroneously issue a warning.
2024-11-14 22:17:42 +00:00
Hood Chatham 142104ce78
gh-89640: Pull in update to float word order detection in autoconf-archive (#126747) 2024-11-13 21:57:33 +01:00
Seth Michael Larson 3c99969094
gh-126623: Update libexpat to 2.6.4, make future updates easier (GH-126792)
Update libexpat to 2.6.4, make future updates easier.
2024-11-13 18:31:20 +00:00
sobolevn f223efb2a2
gh-126525: Fix `makeunicodedata.py` output on macOS and Windows (#126526) 2024-11-12 13:23:57 +03:00
Ken Jin 6293d00e72
gh-120619: Strength reduce function guards, support 2-operand uop forms (GH-124846)
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2024-11-09 11:35:33 +08:00
Hood Chatham f8276bf5f3
gh-126187 Add emscripten.py script to automate emscripten build (#126190)
Add emscripten.py script to automate emscripten build.

This is modeled heavily on `Tools/wasm/wasi.py`. This will form the basis of an Emscripten build bot.
2024-11-09 10:12:55 +08:00
Steve Dower fd5580cd15
gh-126497: Add missing venv redirectors to freethreaded installer (GH-126556) 2024-11-08 16:09:34 +00:00
Serhiy Storchaka 061e50f196
gh-122943: Add the varpos parameter in _PyArg_UnpackKeywords (GH-126564)
Remove _PyArg_UnpackKeywordsWithVararg.
Add comments for integer arguments of _PyArg_UnpackKeywords.
2024-11-08 14:23:50 +02:00
Serhiy Storchaka ee0746af7d
gh-122943: Move code generation for var-positional parameter to converters (GH-126575) 2024-11-08 14:12:15 +02:00
Serhiy Storchaka 06a8b0bb5e
gh-122943: Remove the object converter for var-positional parameter (GH-126560) 2024-11-08 05:41:54 +00:00
Brett Cannon bbe9b21d06
GH-123877: default to `wasm32-wasip1` instead of `wasm32-wasi` to be more specific (GH-126552)
Eventually wasm32-wasi will represent WASI 1.0, and so it's currently deprecated so it can be used for that eventual purpose. wasm32-wasip1 is also more specific to what version of WASI is currently supported.

---------

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-11-07 13:40:56 -08:00
Serhiy Storchaka 1f777396f5
gh-122943: Rework support of var-positional parameter in Argument Clinic (GH-122945)
Move creation of a tuple for var-positional parameter out of
_PyArg_UnpackKeywordsWithVararg().
Merge _PyArg_UnpackKeywordsWithVararg() with _PyArg_UnpackKeywords().
Add a new parameter in _PyArg_UnpackKeywords().

The "parameters" and "converters" attributes of ParseArgsCodeGen no
longer contain the var-positional parameter. It is now available as the
"varpos" attribute. Optimize code generation for var-positional
parameter and reuse the same generating code for functions with and without
keyword parameters.

Add special converters for var-positional parameter. "tuple" represents it as
a Python tuple and "array" represents it as a continuous array of PyObject*.
"object" is a temporary alias of "tuple".
2024-11-07 23:40:03 +02:00
Eric Snow 9357fdcaf0
gh-76785: Minor Cleanup of "Cross-interpreter" Code (gh-126457)
The primary objective here is to allow some later changes to be cleaner. Mostly this involves renaming things and moving a few things around.

* CrossInterpreterData -> XIData
* crossinterpdatafunc -> xidatafunc
* split out pycore_crossinterp_data_registry.h
* add _PyXIData_lookup_t
2024-11-07 09:32:42 -07:00
Mark Shannon 85036c8d61
GH-126222: Fix `_PyUop_num_popped` (GH-126507) 2024-11-07 10:48:27 +00:00
Savannah Ostrowski c3a12ae13e
GH-125911: Rename big trampoline to "shim" (GH-126339) 2024-11-05 15:26:46 -08:00
T. Wouters bbfd9c92fa
gh-115999: Fix gdb support for libpython.so after thread-local bytecode change (#126440)
Fix the gdb pretty printer in the face of --enable-shared by delaying the attempt to load the _PyInterpreterFrame definition until after .so files are loaded.
2024-11-05 15:49:27 +01:00
Peter Bierma 1371295e67
gh-126366: Fix crash if `__iter__` raises an exception during `yield from` (#126369) 2024-11-05 15:26:36 +05:30
mpage 2e95c5ba3b
gh-115999: Implement thread-local bytecode and enable specialization for `BINARY_OP` (#123926)
Each thread specializes a thread-local copy of the bytecode, created on the first RESUME, in free-threaded builds. All copies of the bytecode for a code object are stored in the co_tlbc array on the code object. Threads reserve a globally unique index identifying its copy of the bytecode in all co_tlbc arrays at thread creation and release the index at thread destruction. The first entry in every co_tlbc array always points to the "main" copy of the bytecode that is stored at the end of the code object. This ensures that no bytecode is copied for programs that do not use threads.

Thread-local bytecode can be disabled at runtime by providing either -X tlbc=0 or PYTHON_TLBC=0. Disabling thread-local bytecode also disables specialization.

Concurrent modifications to the bytecode made by the specializing interpreter and instrumentation use atomics, with specialization taking care not to overwrite an instruction that was instrumented concurrently.
2024-11-04 11:13:32 -08:00
sobolevn bfc1d2504c
gh-109413: Add more type hints to `libregrtest` (#126352) 2024-11-04 13:15:57 +03:00
Bénédikt Tran 38a604fd90
gh-99108: Cleanup references to inexisting `Modules/_blake2`. (GH-126270)
* Remove references to `Modules/_blake2`.

* Remove `Modules/_blake2` entry from CODEOWNERS

The folder does not exist anymore.

* Remove `Modules/_blake2` entry from `Tools/c-analyzer/TODO`
2024-11-01 16:18:44 +00:00
mpage 821759d631
gh-126211: Exclude preprocessor directives from statements containing escaping calls (#126213)
The cases generator inserts code to save and restore the stack pointer around
statements that contain escaping calls. To find the beginning of such statements,
we would walk backwards from the escaping call until we encountered a token that
was treated as a statement terminator. This set of terminators should include
preprocessor directives.
2024-11-01 08:53:03 -07:00
Sergey B Kirpichev 8c22eba877
gh-90370: Argument Clinic: avoid temporary tuple creation for varargs (#126064)
Avoid temporary tuple creation when all arguments either positional-only
or vararg.

Objects/setobject.c and Modules/gcmodule.c adapted. This fixes slight
performance regression for set methods, introduced by gh-115112.
2024-10-31 11:37:03 +01:00
Savannah Ostrowski c29bbe2101
GH-125498: Update JIT builds to use LLVM 19 and preserve_none (GH-125499) 2024-10-30 12:03:31 -07:00
Pablo Galindo Salgado 9dfef4e5f4
gh-125588: Teach the python PEG generator the new f-string tokens (#125589)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2024-10-29 23:40:12 +00:00
Tomas R. aab58a93ef
gh-118423: Add `INSTRUCTION_SIZE` macro to code generator (GH-125467) 2024-10-29 17:25:05 +00:00
Hood Chatham dc2552d429
gh-124928: Emscripten node support: Clean up old node <= 16 flags (#124929)
Clean up configure flags for old node versions

These flags are only needed for node <= 16. Node 16 has been end of life since
October of 2023.
2024-10-29 06:28:54 +00:00
Sam Gross 00ea179879
gh-125985: Add free threading scaling micro benchmarks (#125986)
These consist of a number of short snippets that help identify scaling
bottlenecks in the free threaded interpreter.

The current bottlenecks are in calling functions in benchmarks that call
functions (due to `LOAD_ATTR` not yet using deferred reference counting)
and when accessing thread-local data.
2024-10-28 17:47:23 -04:00
Mark Shannon 25441592db
GH-125515: Reduce number of compiler warnings in generated code (GH-125697) 2024-10-28 10:30:31 +00:00
Erlend E. Aasland 26d627779f
gh-89640: properly detect float word ordering on Linux (#125571)
autoconf-archive patch by Dan Amelang.
2024-10-26 15:46:46 +00:00
Mark Shannon 57e3c59bb6
GH-125521: Remove `if (true)` from generated output to reduce C compiler warnings (GH-125700) 2024-10-22 10:11:29 +01:00
Mikhail Efimov c1bdbe84c8
gh-124889: Rework Python generator cache (#125816) 2024-10-22 09:42:56 +01:00
Adam Turner f36d37bbaf
gh-125741: Update `build.yml` for the new check_autoconf_regen job (#125772) 2024-10-21 11:54:54 +01:00
Michael Droettboom c8fd4b12e3
gh-125207: Fix MSVC 1935 build with JIT (#125209)
* gh-125207: Use {0} array initializers

* Simplify, as suggested in PR

* Revert change to explicitly specify length
2024-10-18 15:51:29 -04:00
Wulian cda0ec8e7c
gh-124102: Clean up unsupported VS and WiX detections (GH-124784) 2024-10-18 13:48:18 +01:00
Kirill Podoprigora d83fcf8371
CI: Bump Python to 3.13 and mypy to 1.12 in mypy workflow (#125592)
* Bump mypy to 1.12 & Python to 3.13

* Remove unnecessary `type: ignore`
2024-10-16 14:27:19 +00:00
Kirill Podoprigora bee112a94d
gh-124872: Replace enter/exit events with "switched" (#125532)
Users want to know when the current context switches to a different
context object.  Right now this happens when and only when a context
is entered or exited, so the enter and exit events are synonymous with
"switched".  However, if the changes proposed for gh-99633 are
implemented, the current context will also switch for reasons other
than context enter or exit.  Since users actually care about context
switches and not enter or exit, replace the enter and exit events with
a single switched event.

The former exit event was emitted just before exiting the context.
The new switched event is emitted after the context is exited to match
the semantics users expect of an event with a past-tense name.  If
users need the ability to clean up before the switch takes effect,
another event type can be added in the future.  It is not added here
because YAGNI.

I skipped 0 in the enum as a matter of practice.  Skipping 0 makes it
easier to troubleshoot when code forgets to set zeroed memory, and it
aligns with best practices for other tools (e.g.,
https://protobuf.dev/programming-guides/dos-donts/#unspecified-enum).

Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-10-16 13:53:21 +02:00
Kirill Podoprigora d3c82b9cce
gh-125512: Revert "gh-124872: Replace enter/exit events with "switched" (#124776)" (#125513) 2024-10-15 17:42:16 +03:00
Richard Hansen 843d28f59d
gh-124872: Replace enter/exit events with "switched" (#124776)
Users want to know when the current context switches to a different
context object.  Right now this happens when and only when a context
is entered or exited, so the enter and exit events are synonymous with
"switched".  However, if the changes proposed for gh-99633 are
implemented, the current context will also switch for reasons other
than context enter or exit.  Since users actually care about context
switches and not enter or exit, replace the enter and exit events with
a single switched event.

The former exit event was emitted just before exiting the context.
The new switched event is emitted after the context is exited to match
the semantics users expect of an event with a past-tense name.  If
users need the ability to clean up before the switch takes effect,
another event type can be added in the future.  It is not added here
because YAGNI.

I skipped 0 in the enum as a matter of practice.  Skipping 0 makes it
easier to troubleshoot when code forgets to set zeroed memory, and it
aligns with best practices for other tools (e.g.,
https://protobuf.dev/programming-guides/dos-donts/#unspecified-enum).
2024-10-14 12:28:41 -07:00
Mikhail Efimov 187580d95c
gh-119786: [doc] broken link and typo fix in interpreter_definition.md (#125455) 2024-10-14 17:24:54 +00:00
Mark Shannon 06ca33020e
GH-125323: Convert DECREF_INPUTS_AND_REUSE_FLOAT into a function that takes PyStackRefs. (GH-125439) 2024-10-14 14:18:57 +01:00
Ken Jin 4b358ee647
gh-125323: Remove some unsafe Py_DECREFs in bytecodes.c, replacing them with PyStackRef_CLOSEs (GH-125324) 2024-10-14 09:17:51 +01:00
Donghee Na 08f6bf7171
gh-124612: Update autoconf container image (#125320) 2024-10-12 00:27:26 +09:00
Mark Shannon c9014374c5
GH-125174: Make immortal objects more robust, following design from PEP 683 (GH-125251) 2024-10-10 18:19:08 +01:00
Tomas R. 6b533a659b
gh-125039: Make `this_instr`/`prev_instr` const in cases generator (GH-125071) 2024-10-09 13:54:39 +01:00
Donghee Na b502573f7f
gh-124612: Use ghcr.io/python/autoconf instead of public image (#124657)
* gh-124612: Use ghcr.io/python/autoconf instead of public image

* Update
2024-10-09 09:33:18 +09:00
Cody Maloney 7dca7322cc
gh-125084: Resolve paths in generator common code (GH-125085)
In out of tree builds, the paths can contain `../ which needs to be
resolved for the relative path calculation to work.
2024-10-08 12:16:02 -05:00
Bénédikt Tran e4292c0410
gh-123961: Convert _curses to a multi-phase init module (PEP-489) (#124965) 2024-10-08 13:42:44 +02:00
Mark Shannon da071fa3e8
GH-119866: Spill the stack around escaping calls. (GH-124392)
* Spill the evaluation around escaping calls in the generated interpreter and JIT. 

* The code generator tracks live, cached values so they can be saved to memory when needed.

* Spills the stack pointer around escaping calls, so that the exact stack is visible to the cycle GC.
2024-10-07 14:56:39 +01:00
efimov-mikhail 1f9025a4e7
gh-124889: Remove redundant artificial rules in PEG parser (#124893)
Cache in C PEG-generator reworked:
we save artificial rules in cache by Node string representation as a key instead of Node object itself.
As a result total count of artificial rules in parsers.c is lowered from 283 to 170.
More natural number ordering is used for the names of artificial rules.

Auxiliary method CCallMakerVisitor._generate_artificial_rule_call is added.
Its purpose is abstracting work with artificial rules cache.

Explicit using of "is_repeat1" kwarg is added to visit_Repeat0 and visit_Repeat1 methods.
Its slightly improve code readabitily.
2024-10-03 13:58:56 +01:00
Diego Russo b85923a0fe
GH-119726: Deduplicate AArch64 trampolines within a trace (GH-123872) 2024-10-02 12:07:20 -07:00
Savannah Ostrowski 6737333ac5
GH-124820: Move `-mno-outline-atomics` flag to aarch64 Linux JIT builds only (#124821) 2024-10-02 11:40:59 +02:00
Wulian da1e5526ae
gh-124487: Updated installer messages and internal docs regarding minimum Windows version (GH-124822) 2024-10-01 15:44:18 +01:00
Steve Dower fac5e7aa17
gh-124487: Require at least Windows 10 SDK and update install check (GH-124672) 2024-09-30 18:50:46 +01:00
Bénédikt Tran 4d8e7c40a0
gh-123961: Add a global state to _curses (#124729) 2024-09-29 15:17:20 +00:00
Bénédikt Tran c00964ecd5
gh-124665: Add `_PyCodec_UnregisterError` and `_codecs._unregister_error` (#124677) 2024-09-29 02:25:23 +02:00
Jelle Zijlstra 25189188bf
generate_global_objects.py: Fix name of macro in error message (#124464)
_PyID does not exist but _Py_ID does.
2024-09-26 21:37:21 -07:00
Savannah Ostrowski 65f1237098
GH-123516: Improve JIT memory consumption by invalidating cold executors (GH-124443)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-09-27 00:35:42 +00:00
Brett Cannon 66cc6d4c50
Remove some unused files related to WASM/WASI (GH-124635) 2024-09-26 21:23:41 +00:00
Irit Katriel 78aeb38f7d
gh-124285: Fix bug where bool() is called multiple times for the same part of a boolean expression (#124394) 2024-09-25 15:51:25 +01:00
Savannah Ostrowski b6471f4a39
GH-124408: Temporarily skip test_strftime_y2k for emulated Linux JIT CI (GH-124466) 2024-09-24 18:09:50 -07:00
Sam Gross f4997bb3ac
gh-123923: Defer refcounting for `f_funcobj` in `_PyInterpreterFrame` (#124026)
Use a `_PyStackRef` and defer the reference to `f_funcobj` when
possible. This avoids some reference count contention in the common case
of executing the same code object from multiple threads concurrently in
the free-threaded build.
2024-09-24 20:08:18 +00:00
Jason Fried d87482bc4e
gh-119333: Add C api to have contextvar enter/exit callbacks (#119335)
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-09-23 20:40:17 -07:00
Hugo van Kemenade e256a7590a
Lint and format Tools/build/check-warnings.py (#124382) 2024-09-24 02:47:05 +03:00
Wulian 27a62e7371
gh-124102: Update internal PCbuild docs to accurately list build dependencies (GH-124103) 2024-09-23 23:09:22 +00:00
Mark Shannon c87b0e4a46
GH-124284: Add stats for refcount operations on immortal objects (GH-124288) 2024-09-23 19:10:55 +01:00
Steve Dower df7228ce14
gh-124254: Detect freethreaded MSI component when doing an upgrade on Windows (GH-124279) 2024-09-23 16:27:46 +01:00
Petr Viktorin aee219f455
gh-123880: Allow recursive import of single-phase-init modules (GH-123950)
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Co-authored-by: Brett Cannon <brett@python.org>
2024-09-20 10:27:34 +02:00
Nate Ohlson 81480e6edb
gh-124190: Ignore files directories check warning tooling (#124193)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2024-09-18 10:49:43 +03:00
Nate Ohlson 29a1a6e3ed
gh-124191: Disable -Wconversion in enable safety (gh-124192) 2024-09-18 12:55:09 +09:00
Victor Stinner 3aff1d0260
gh-124064: Fix -Wconversion warnings in Parser/pegen.c (#124181) 2024-09-17 15:58:43 +00:00
Victor Stinner ec08aa1fe4
gh-124064: Fix -Wconversion warnings in pycore_{long,object}.h (#124177)
Change also the fix for pycore_gc.h and pycore_stackref.h:
declare constants as uintptr_t, rather than casting constants.
2024-09-17 15:35:40 +00:00