4ed7d1d6ac
GH-123996: Explicitly mark 'self_or_null' as an array of size 1 to ensure that it is kept in memory for calls (GH-124003)
2024-09-12 15:32:45 +01:00
1fbc118c5d
GH-123545: Remove duplicate Py_DECREF when handling _PyOptimizer_Optimize errors (GH-123546)
2024-09-05 10:56:07 -07:00
f1a0d96f41
gh-123091: Use _Py_IsImmortalLoose() ( #123511 )
...
Use _Py_IsImmortalLoose() in bytesobject.c, typeobject.c
and ceval.c.
2024-09-02 14:25:19 +02:00
54a05a4600
GH-123232: Factor BINARY_SLICE and STORE_SLICE to handle stats properly for tier 2. (GH-123381)
2024-08-27 10:49:39 +01:00
67f2c84bff
gh-123205: `Python/bytecodes.c`: Fix compiler warning ( #123206 )
...
Fix MSVC warning "conversion from '__int64' to 'int'"
2024-08-23 15:35:25 -04:00
0b0f7befad
GH-123232: Fix "not specialized" stats (GH-123236)
2024-08-23 10:46:03 +01:00
297f2e093e
gh-123083: Fix a potential use-after-free in ``STORE_ATTR_WITH_HINT`` (gh-123092)
2024-08-22 23:49:09 +09:00
a4fd7aa4a6
GH-115776: Allow any fixed sized object to have inline values (GH-123192)
2024-08-21 15:52:04 +01:00
1eba8bae92
GH-123185: Check for `NULL` after calling `_PyEvalFramePushAndInit` (GH-123194)
2024-08-21 12:44:56 +01:00
bb1d30336e
GH-118093: Make `CALL_ALLOC_AND_ENTER_INIT` suitable for tier 2. (GH-123140)
...
* Convert CALL_ALLOC_AND_ENTER_INIT to micro-ops such that tier 2 supports it
* Allow inexact arguments for CALL_ALLOC_AND_ENTER_INIT.
2024-08-20 16:52:58 +01:00
c13e7d98fb
GH-118093: Specialize `CALL_KW` (GH-123006)
2024-08-16 17:11:24 +01:00
f84754b705
GH-118093: Turn some DEOPT_IFs into EXIT_IFs (GH-122998)
2024-08-14 07:54:42 -07:00
eec7bdaf01
GH-120024: Remove `CHECK_EVAL_BREAKER` macro. (GH-122968)
...
* Factor some instructions into micro-ops to isolate CHECK_EVAL_BREAKER for escape analysis
* Eliminate CHECK_EVAL_BREAKER macro
2024-08-14 12:04:05 +01:00
9621a7d017
GH-118093: Handle some polymorphism before requiring progress in tier two (GH-122843)
2024-08-12 12:39:31 -07:00
ab094d1b2b
gh-117139: Replace _PyList_FromArraySteal with stack ref variant ( #122830 )
...
This replaces `_PyList_FromArraySteal` with `_PyList_FromStackRefSteal`.
It's functionally equivalent, but takes a `_PyStackRef` array instead of
an array of `PyObject` pointers.
Co-authored-by: Ken Jin <kenjin@python.org>
2024-08-12 14:49:49 -04:00
736fe4d23e
gh-117139: Fix a few `_PyStackRef` related bugs ( #122831 )
...
`BUILD_SET` should use a borrow instead of a steal. The cleanup in `_DO_CALL`
`CONVERSION_FAILED` was incorrect.
Co-authored-by: Ken Jin <kenjin@python.org>
2024-08-12 14:49:33 -04:00
3e753c689a
gh-118926: Spill deferred references to stack in cases generator ( #122748 )
...
This automatically spills the results from `_PyStackRef_FromPyObjectNew`
to the in-memory stack so that the deferred references are visible to
the GC before we make any possibly escaping call.
Co-authored-by: Ken Jin <kenjin@python.org>
2024-08-07 13:23:53 -04:00
674a50ef2f
gh-117139: Fix an incorrect borrow in bytecodes.c ( #122318 )
...
`_PyDict_SetItem_Take2` steals both the key (i.e., `sub`) and the value.
2024-08-07 19:06:19 +05:30
4c31791848
GH-120024: Move three more escaping calls out of conditional statements (GH-122734)
2024-08-06 14:14:52 +01:00
a8be8fc6c4
GH-120024: Refactor code a bit so that escaping calls can be wrapped in spill code in code generator (GH-122693)
2024-08-06 08:40:39 +01:00
5bd72912a1
GH-122616: Simplify LOAD_ATTR_WITH_HINT and STORE_ATTR_WITH_HINT (GH-122620)
2024-08-05 16:27:48 +01:00
7aca84e557
GH-117224: Move the body of a few large-ish micro-ops into helper functions (GH-122601)
2024-08-02 16:31:17 +01:00
df13a1821a
GH-118095: Add tier two support for BINARY_SUBSCR_GETITEM (GH-120793)
2024-08-01 16:19:05 -07:00
a9d56e38a0
GH-122155: Track local variables between pops and pushes in cases generator (GH-122286)
2024-08-01 09:27:26 +01:00
15d4cd0967
GH-116090: Fire RAISE events from _FOR_ITER_TIER_TWO (GH-122413)
2024-07-29 12:17:47 -07:00
64857d849f
GH-122294: Burn in the addresses of side exits (GH-122295)
2024-07-26 09:40:15 -07:00
95a73917cd
GH-122029: Break INSTRUMENTED_CALL into micro-ops, so that its behavior is consistent with CALL (GH-122177)
2024-07-26 14:35:57 +01:00
afb0aa6ed2
GH-121131: Clean up and fix some instrumented instructions. (GH-121132)
...
* Add support for 'prev_instr' to code generator and refactor some INSTRUMENTED instructions
2024-07-26 12:24:12 +01:00
d9efa45d74
GH-118093: Add tier two support for BINARY_OP_INPLACE_ADD_UNICODE (GH-122253)
2024-07-25 14:45:07 -07:00
5f6001130f
GH-118093: Add tier two support for LOAD_ATTR_PROPERTY (GH-122283)
2024-07-25 10:45:28 -07:00
5e686ff57d
GH-122034: Add StackRef variants of type checks to reduce the number of PyStackRef_AsPyObjectBorrow calls (GH-122037)
2024-07-25 18:32:43 +01:00
2e14a52cce
GH-122160: Remove BUILD_CONST_KEY_MAP opcode. (GH-122164)
2024-07-25 16:24:29 +01:00
794546fd53
GH-118093: Remove invalidated executors from side exits (GH-121885)
2024-07-24 09:16:30 -07:00
7b36b67b1e
GH-118093: Add tier two support to several instructions (GH-121884)
2024-07-18 14:24:58 -07:00
3eacfc1a4d
GH-121784: Generate an error during code gen if a variable is marked `unused`, but is used and thus cached in a prior uop. ( #121788 )
...
* Reject uop definitions that declare values as 'unused' that are already cached by prior uops
* Track which variables are defined and only load from memory when needed
* Support explicit `flush` in macro definitions.
* Make sure stack is flushed in where needed.
2024-07-18 12:49:24 +01:00
8ad6067bd4
GH-121012: Set index to -1 when list iterators become exhausted in tier 2 (GH-121483)
2024-07-08 14:20:13 +01:00
8e8d202f55
gh-117139: Add _PyTuple_FromStackRefSteal and use it ( #121244 )
...
Avoids the extra conversion from stack refs to PyObjects.
2024-07-02 12:30:14 -04:00
33903c53db
GH-116017: Get rid of _COLD_EXITs (GH-120960)
2024-07-01 13:17:40 -07:00
e6543daf12
gh-117139: Fix a few wrong steals in bytecodes.c (GH-121127)
...
Fix a few wrong steals in bytecodes.c
2024-06-29 02:14:48 +08:00
22b0de2755
gh-117139: Convert the evaluation stack to stack refs ( #118450 )
...
This PR sets up tagged pointers for CPython.
The general idea is to create a separate struct _PyStackRef for everything on the evaluation stack to store the bits. This forces the C compiler to warn us if we try to cast things or pull things out of the struct directly.
Only for free threading: We tag the low bit if something is deferred - that means we skip incref and decref operations on it. This behavior may change in the future if Mark's plans to defer all objects in the interpreter loop pans out.
This implies a strict stack reference discipline is required. ALL incref and decref operations on stackrefs must use the stackref variants. It is unsafe to untag something then do normal incref/decref ops on it.
The new incref and decref variants are called dup and close. They mimic a "handle" API operating on these stackrefs.
Please read Include/internal/pycore_stackref.h for more information!
---------
Co-authored-by: Mark Shannon <9448417+markshannon@users.noreply.github.com>
2024-06-27 03:10:43 +08:00
8f5a01707f
GH-120982: Add stack check assertions to generated interpreter code (GH-120992)
2024-06-25 16:42:29 +01:00
a47abdb45d
GH-117062: Make _JUMP_TO_TOP a general absolute jump (GH-120854)
2024-06-24 08:35:10 -07:00
65a12c559c
gh-120834: fix type of *_iframe field in _PyGenObject_HEAD declaration ( #120835 )
2024-06-24 10:23:38 +01:00
9cefcc0ee7
GH-120507: Lower the `BEFORE_WITH` and `BEFORE_ASYNC_WITH` instructions. ( #120640 )
...
* Remove BEFORE_WITH and BEFORE_ASYNC_WITH instructions.
* Add LOAD_SPECIAL instruction
* Reimplement `with` and `async with` statements using LOAD_SPECIAL
2024-06-18 12:17:46 +01:00
274f844830
GH-120619: Clean up `RETURN_VALUE` instruction (GH-120624)
...
* Rename _POP_FRAME to _RETURN_VALUE as it returns a value as well as popping a frame.
* Remove remaining _POP_FRAMEs
2024-06-17 14:40:11 +01:00
9b8611eeea
gh-119180: PEP 649 compiler changes ( #119361 )
2024-06-11 13:06:49 +00:00
3859e09e3d
gh-74929: PEP 667 C API documentation (gh-119379)
...
* Add docs for new APIs
* Add soft-deprecation notices
* Add What's New porting entries
* Update comments referencing `PyFrame_LocalsToFast()` to mention the proxy instead
* Other related cleanups found when looking for refs to the deprecated APIs
2024-06-01 13:59:35 +10:00
80a4e38994
gh-119821: Support non-dict globals in LOAD_FROM_DICT_OR_GLOBALS ( #119822 )
...
Support non-dict globals in LOAD_FROM_DICT_OR_GLOBALS
The implementation basically copies LOAD_GLOBAL. Possibly it could be deduplicated,
but that seems like it may get hairy since the two operations have different operands.
This is important to fix in 3.14 for PEP 649, but it's a bug in earlier versions too,
and we should backport to 3.13 and 3.12 if possible.
2024-05-31 14:05:24 -07:00
5cd3ffd6b7
GH-119258: Handle STORE_ATTR_WITH_HINT in tier two (GH-119481)
2024-05-28 12:47:54 -07:00
cfcc054dee
GH-119476: Split _CHECK_FUNCTION_VERSION out of _CHECK_FUNCTION_EXACT_ARGS (GH-119510)
2024-05-28 12:45:11 -07:00
98e855fcc1
gh-119180: Add LOAD_COMMON_CONSTANT opcode ( #119321 )
...
The PEP 649 implementation will require a way to load NotImplementedError
from the bytecode. @markshannon suggested implementing this by converting
LOAD_ASSERTION_ERROR into a more general mechanism for loading constants.
This PR adds this new opcode. I will work on the rest of the implementation
of the PEP separately.
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2024-05-22 00:46:39 +00:00
0d9148823d
gh-118414: Fix assertion in YIELD_VALUE when tracing lines or instrs ( #118683 )
2024-05-06 21:22:59 -07:00
1ab6356ebe
GH-118095: Use broader specializations of CALL in tier 1, for better tier 2 support of calls. (GH-118322)
...
* Add CALL_PY_GENERAL, CALL_BOUND_METHOD_GENERAL and call CALL_NON_PY_GENERAL specializations.
* Remove CALL_PY_WITH_DEFAULTS specialization
* Use CALL_NON_PY_GENERAL in more cases when otherwise failing to specialize
2024-05-04 12:11:11 +01:00
da2cfc4cb6
GH-113464: Remove the extra jump via `_SIDE_EXIT` in `_EXIT_TRACE` (GH-118545)
2024-05-04 08:50:24 +01:00
9c14ed0618
gh-107674: Improve performance of `sys.settrace` (GH-117133)
...
* Check tracing in RESUME_CHECK
* Only change to RESUME_CHECK if not tracing
2024-05-03 19:49:24 +01:00
72867c962c
GH-118095: Unify the behavior of tier 2 FOR_ITER branch micro-ops (GH-118420)
...
* Target _FOR_ITER_TIER_TWO at POP_TOP following the matching END_FOR
* Modify _GUARD_NOT_EXHAUSTED_RANGE, _GUARD_NOT_EXHAUSTED_LIST and _GUARD_NOT_EXHAUSTED_TUPLE so that they also target the POP_TOP following the matching END_FOR
2024-05-02 16:17:59 +01:00
67bba9dd0f
GH-117442: Check eval-breaker at start (rather than end) of tier 2 loops (GH-118482)
2024-05-02 13:10:31 +01:00
5b05d452cd
GH-118095: Add tier 2 support for YIELD_VALUE (GH-118380)
2024-04-30 11:33:13 +01:00
ab6eda0ee5
GH-118095: Allow a variant of RESUME_CHECK in tier 2 (GH-118286)
2024-04-29 07:54:05 +01:00
3e06c7f719
GH-118095: Add dynamic exit support and FOR_ITER_GEN support to tier 2 (GH-118279)
2024-04-26 18:08:50 +01:00
f180b31e76
GH-118095: Handle `RETURN_GENERATOR` in tier 2 (GH-118180)
2024-04-25 11:32:47 +01:00
83235f7791
GH-115419: Move setting the instruction pointer to error exit stubs (GH-118088)
2024-04-24 14:41:30 +01:00
a6647d16ab
GH-115480: Reduce guard strength for binary ops when type of one operand is known already (GH-118050)
2024-04-22 13:34:06 +01:00
8b541c017e
gh-112075: Make instance attributes stored in inline "dict" thread safe ( #114742 )
...
Make instance attributes stored in inline "dict" thread safe on free-threaded builds
2024-04-21 22:57:05 -07:00
07525c9a85
gh-116818: Make `sys.settrace`, `sys.setprofile`, and monitoring thread-safe ( #116775 )
...
Makes sys.settrace, sys.setprofile, and monitoring generally thread-safe.
Mostly uses a stop-the-world approach and synchronization around the code object's _co_instrumentation_version. There may be a little bit of extra synchronization around the monitoring data that's required to be TSAN clean.
2024-04-19 14:47:42 -07:00
7e6fa5fced
GH-116202: Incorporate invalidation check into _START_EXECUTOR. (GH-118044)
2024-04-19 09:26:42 +01:00
757b62493b
gh-117457: Regen executor cases post PR #117477 ( #117559 )
2024-04-05 10:13:00 +00:00
0edde64a41
GH-117457: Correct pystats uop "miss" counts (GH-117477)
2024-04-04 15:49:18 -07:00
434bc593df
gh-112075: Make _PyDict_LoadGlobal thread safe ( #117529 )
...
Make _PyDict_LoadGlobal threadsafe
2024-04-04 12:26:07 -07:00
060a96f1a9
gh-116968: Reimplement Tier 2 counters ( #117144 )
...
Introduce a unified 16-bit backoff counter type (``_Py_BackoffCounter``),
shared between the Tier 1 adaptive specializer and the Tier 2 optimizer. The
API used for adaptive specialization counters is changed but the behavior is
(supposed to be) identical.
The behavior of the Tier 2 counters is changed:
- There are no longer dynamic thresholds (we never varied these).
- All counters now use the same exponential backoff.
- The counter for ``JUMP_BACKWARD`` starts counting down from 16.
- The ``temperature`` in side exits starts counting down from 64.
2024-04-04 15:03:27 +00:00
1c43468886
gh-116168: Remove extra `_CHECK_STACK_SPACE` uops ( #117242 )
...
This merges all `_CHECK_STACK_SPACE` uops in a trace into a single `_CHECK_STACK_SPACE_OPERAND` uop that checks whether there is enough stack space for all calls included in the entire trace.
2024-04-03 17:14:18 +00:00
c32dc47aca
GH-115776: Embed the values array into the object, for "normal" Python objects. (GH-116115)
2024-04-02 11:59:21 +01:00
19c1dd60c5
gh-117323: Make `cell` thread-safe in free-threaded builds ( #117330 )
...
Use critical sections to lock around accesses to cell contents. The critical sections are no-ops in the default (with GIL) build.
2024-03-29 13:35:43 -04:00
bf82f77957
GH-116422: Tier2 hot/cold splitting (GH-116813)
...
Splits the "cold" path, deopts and exits, from the "hot" path, reducing the size of most jitted instructions, at the cost of slower exits.
2024-03-26 09:35:11 +00:00
950667ed07
GH-115802: Reduce the size of _INIT_CALL_PY_EXACT_ARGS. (GH-116856)
2024-03-15 17:16:30 +00:00
2cf18a4430
GH-116422: Modify a few uops so that they can be supported by tier 2 with hot/cold splitting (GH-116832)
2024-03-15 10:48:00 +00:00
61e54bfcee
GH-116422: Factor out eval breaker checks at end of calls into its own micro-op. (GH-116817)
2024-03-14 16:31:47 +00:00
4e5df2013f
GH-116468: Use constants instead of `oparg` in stack effects when `oparg` is known to be a constant. (GH-116469)
2024-03-11 09:30:15 +00:00
0003285c8d
GH-113710: Fix optimization of globals using `_CHECK_FUNCTION` (GH-116460)
2024-03-08 09:47:41 +00:00
41457c7fdb
gh-116381: Remove bad specializations, add fail stats (GH-116464)
...
* Remove bad specializations, add fail stats
2024-03-08 00:21:21 +08:00
72d3cc94cd
gh-116437: Use new C API PyDict_Pop() to simplify the code (GH-116438)
2024-03-07 11:21:08 +02:00
7114cf20c0
gh-116381: Specialize CONTAINS_OP (GH-116385)
...
* Specialize CONTAINS_OP
* 📜 🤖 Added by blurb_it.
* Add PyAPI_FUNC for JIT
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2024-03-07 03:30:11 +08:00
27858e2a17
GH-113710: Tier 2 optimizer: check the function instead of checking globals. (GH-116410)
2024-03-06 13:12:23 +00:00
23db9c6227
GH-115685: Split `_TO_BOOL_ALWAYS_TRUE` into micro-ops (GH-116352)
2024-03-05 15:23:08 +00:00
7895a61168
gh-116098: Revert "gh-107674: Improve performance of `sys.settrace` (GH-114986)" (GH-116178)
...
Revert "gh-107674: Improve performance of `sys.settrace` (GH-114986)"
This reverts commit 0a61e23700
2024-03-01 07:46:33 +01:00
d01886c5c9
gh-115685: Type/values propagate for TO_BOOL in tier 2 (GH-115686)
2024-03-01 06:13:38 +08:00
f0df35eeca
GH-115802: JIT "small" code for Windows (GH-115964)
2024-02-29 08:11:28 -08:00
0a61e23700
gh-107674: Improve performance of `sys.settrace` (GH-114986)
2024-02-28 15:21:42 +00:00
e4561e0501
gh-115778: Add `tierN` annotation for instruction definitions ( #115815 )
...
This replaces the old `TIER_{ONE,TWO}_ONLY` macros. Note that `specialized` implies `tier1`.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-02-23 17:31:57 +00:00
b348313e7a
GH-115651: Convert `LOAD_MODULE_ATTR` into `LOAD_INLINE_CONST` when the module is itself a constant. (GH-115711)
2024-02-22 14:48:25 +00:00
520403ed4c
gh-115733: Fix crash involving exhausted list iterator ( #115740 )
...
* gh-115733: Fix crash involving exhausted iterator
* Add blurb
2024-02-21 05:18:44 +09:00
0749244d13
gh-112175: Add `eval_breaker` to `PyThreadState` ( #115194 )
...
This change adds an `eval_breaker` field to `PyThreadState`. The primary
motivation is for performance in free-threaded builds: with thread-local eval
breakers, we can stop a specific thread (e.g., for an async exception) without
interrupting other threads.
The source of truth for the global instrumentation version is stored in the
`instrumentation_version` field in PyInterpreterState. Threads usually read the
version from their local `eval_breaker`, where it continues to be colocated
with the eval breaker bits.
2024-02-20 09:57:48 -05:00
dcba21f905
gh-115687: Split up guards from COMPARE_OP (GH-115688)
2024-02-20 11:30:49 +00:00
626c414995
GH-115457: Support splitting and replication of micro ops. (GH-115558)
2024-02-20 10:50:59 +00:00
7b21403ccd
GH-112354: Initial implementation of warm up on exits and trace-stitching (GH-114142)
2024-02-20 09:39:55 +00:00
9e3729bbd7
gh-114626: add PyCFunctionFast and PyCFunctionFastWithKeywords (GH-114627)
...
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-02-15 11:05:20 +01:00
a2d4281415
gh-112087: Make __sizeof__ and listiter_{len, next} to be threadsafe (gh-114843)
2024-02-15 02:00:50 +09:00
681778c56a
GH-113710: Improve `_SET_IP` and `_CHECK_VALIDITY` (GH-115248)
2024-02-13 16:28:19 +00:00
7cce857622
gh-114058: Foundations of the Tier2 redundancy eliminator (GH-115085)
...
---------
Co-authored-by: Mark Shannon <9448417+markshannon@users.noreply.github.com>
Co-authored-by: Jules <57632293+JuliaPoo@users.noreply.github.com>
Co-authored-by: Guido van Rossum <gvanrossum@users.noreply.github.com>
2024-02-13 21:24:48 +08:00
0e71a295e9
GH-113710: Add a "globals to constants" pass (GH-114592)
...
Converts specializations of `LOAD_GLOBAL` into constants during tier 2 optimization.
2024-02-02 12:14:34 +00:00
Mark Shannon