c1e9647107
gh-119689: generate stack effect metadata for pseudo instructions ( #119691 )
2024-05-29 09:47:56 +00:00
5cd3ffd6b7
GH-119258: Handle STORE_ATTR_WITH_HINT in tier two (GH-119481)
2024-05-28 12:47:54 -07:00
cfcc054dee
GH-119476: Split _CHECK_FUNCTION_VERSION out of _CHECK_FUNCTION_EXACT_ARGS (GH-119510)
2024-05-28 12:45:11 -07:00
ae9140f32a
gh-119676: remove several pseudo instructions which are use only in codegen ( #119677 )
2024-05-28 19:05:38 +00:00
98e855fcc1
gh-119180: Add LOAD_COMMON_CONSTANT opcode ( #119321 )
...
The PEP 649 implementation will require a way to load NotImplementedError
from the bytecode. @markshannon suggested implementing this by converting
LOAD_ASSERTION_ERROR into a more general mechanism for loading constants.
This PR adds this new opcode. I will work on the rest of the implementation
of the PEP separately.
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2024-05-22 00:46:39 +00:00
6e9863d7a3
gh-118692: Avoid creating unnecessary StopIteration instances for monitoring ( #119216 )
2024-05-21 20:42:51 +00:00
0d9148823d
gh-118414: Fix assertion in YIELD_VALUE when tracing lines or instrs ( #118683 )
2024-05-06 21:22:59 -07:00
616b745b89
GH-115709: Invalidate executors when a local variable is changed via frame.f_locals ( #118639 )
...
Also fix unrelated assert in debug Tier2/JIT builds.
2024-05-06 21:21:06 +00:00
1ab6356ebe
GH-118095: Use broader specializations of CALL in tier 1, for better tier 2 support of calls. (GH-118322)
...
* Add CALL_PY_GENERAL, CALL_BOUND_METHOD_GENERAL and call CALL_NON_PY_GENERAL specializations.
* Remove CALL_PY_WITH_DEFAULTS specialization
* Use CALL_NON_PY_GENERAL in more cases when otherwise failing to specialize
2024-05-04 12:11:11 +01:00
da2cfc4cb6
GH-113464: Remove the extra jump via `_SIDE_EXIT` in `_EXIT_TRACE` (GH-118545)
2024-05-04 08:50:24 +01:00
9c14ed0618
gh-107674: Improve performance of `sys.settrace` (GH-117133)
...
* Check tracing in RESUME_CHECK
* Only change to RESUME_CHECK if not tracing
2024-05-03 19:49:24 +01:00
72867c962c
GH-118095: Unify the behavior of tier 2 FOR_ITER branch micro-ops (GH-118420)
...
* Target _FOR_ITER_TIER_TWO at POP_TOP following the matching END_FOR
* Modify _GUARD_NOT_EXHAUSTED_RANGE, _GUARD_NOT_EXHAUSTED_LIST and _GUARD_NOT_EXHAUSTED_TUPLE so that they also target the POP_TOP following the matching END_FOR
2024-05-02 16:17:59 +01:00
67bba9dd0f
GH-117442: Check eval-breaker at start (rather than end) of tier 2 loops (GH-118482)
2024-05-02 13:10:31 +01:00
f6fab21721
GH-118095: Make invalidating and clearing executors memory safe (GH-118459)
2024-05-01 11:34:50 +01:00
7d83f7bcc4
gh-118335: Configure Tier 2 interpreter at build time ( #118339 )
...
The code for Tier 2 is now only compiled when configured
with `--enable-experimental-jit[=yes|interpreter]`.
We drop support for `PYTHON_UOPS` and -`Xuops`,
but you can disable the interpreter or JIT
at runtime by setting `PYTHON_JIT=0`.
You can also build it without enabling it by default
using `--enable-experimental-jit=yes-off`;
enable with `PYTHON_JIT=1`.
On Windows, the `build.bat` script supports
`--experimental-jit`, `--experimental-jit-off`,
`--experimental-interpreter`.
In the C code, `_Py_JIT` is defined as before
when the JIT is enabled; the new variable
`_Py_TIER2` is defined when the JIT *or* the
interpreter is enabled. It is actually a bitmask:
1: JIT; 2: default-off; 4: interpreter.
2024-04-30 18:26:34 -07:00
7fabcc727d
gh-117657: Don't specialize RESUME_CHECK when specialization is disabled (GH-118349)
2024-04-30 21:51:59 +00:00
4a1cf66c5c
gh-117657: Fix small issues with instrumentation and TSAN ( #118064 )
...
Small TSAN fixups for instrumentation
2024-04-30 11:38:05 -07:00
5b05d452cd
GH-118095: Add tier 2 support for YIELD_VALUE (GH-118380)
2024-04-30 11:33:13 +01:00
ab6eda0ee5
GH-118095: Allow a variant of RESUME_CHECK in tier 2 (GH-118286)
2024-04-29 07:54:05 +01:00
3e06c7f719
GH-118095: Add dynamic exit support and FOR_ITER_GEN support to tier 2 (GH-118279)
2024-04-26 18:08:50 +01:00
f180b31e76
GH-118095: Handle `RETURN_GENERATOR` in tier 2 (GH-118180)
2024-04-25 11:32:47 +01:00
83235f7791
GH-115419: Move setting the instruction pointer to error exit stubs (GH-118088)
2024-04-24 14:41:30 +01:00
77cd0428b6
GH-118095: Convert DEOPT_IFs on likely side exits to EXIT_IFs (GH-118106)
...
Covert DEOPT_IFs on likely side exits to EXIT_IFs
2024-04-24 14:37:55 +01:00
a6647d16ab
GH-115480: Reduce guard strength for binary ops when type of one operand is known already (GH-118050)
2024-04-22 13:34:06 +01:00
8b541c017e
gh-112075: Make instance attributes stored in inline "dict" thread safe ( #114742 )
...
Make instance attributes stored in inline "dict" thread safe on free-threaded builds
2024-04-21 22:57:05 -07:00
07525c9a85
gh-116818: Make `sys.settrace`, `sys.setprofile`, and monitoring thread-safe ( #116775 )
...
Makes sys.settrace, sys.setprofile, and monitoring generally thread-safe.
Mostly uses a stop-the-world approach and synchronization around the code object's _co_instrumentation_version. There may be a little bit of extra synchronization around the monitoring data that's required to be TSAN clean.
2024-04-19 14:47:42 -07:00
7e6fa5fced
GH-116202: Incorporate invalidation check into _START_EXECUTOR. (GH-118044)
2024-04-19 09:26:42 +01:00
434bc593df
gh-112075: Make _PyDict_LoadGlobal thread safe ( #117529 )
...
Make _PyDict_LoadGlobal threadsafe
2024-04-04 12:26:07 -07:00
060a96f1a9
gh-116968: Reimplement Tier 2 counters ( #117144 )
...
Introduce a unified 16-bit backoff counter type (``_Py_BackoffCounter``),
shared between the Tier 1 adaptive specializer and the Tier 2 optimizer. The
API used for adaptive specialization counters is changed but the behavior is
(supposed to be) identical.
The behavior of the Tier 2 counters is changed:
- There are no longer dynamic thresholds (we never varied these).
- All counters now use the same exponential backoff.
- The counter for ``JUMP_BACKWARD`` starts counting down from 16.
- The ``temperature`` in side exits starts counting down from 64.
2024-04-04 15:03:27 +00:00
1c43468886
gh-116168: Remove extra `_CHECK_STACK_SPACE` uops ( #117242 )
...
This merges all `_CHECK_STACK_SPACE` uops in a trace into a single `_CHECK_STACK_SPACE_OPERAND` uop that checks whether there is enough stack space for all calls included in the entire trace.
2024-04-03 17:14:18 +00:00
c32dc47aca
GH-115776: Embed the values array into the object, for "normal" Python objects. (GH-116115)
2024-04-02 11:59:21 +01:00
19c1dd60c5
gh-117323: Make `cell` thread-safe in free-threaded builds ( #117330 )
...
Use critical sections to lock around accesses to cell contents. The critical sections are no-ops in the default (with GIL) build.
2024-03-29 13:35:43 -04:00
bf82f77957
GH-116422: Tier2 hot/cold splitting (GH-116813)
...
Splits the "cold" path, deopts and exits, from the "hot" path, reducing the size of most jitted instructions, at the cost of slower exits.
2024-03-26 09:35:11 +00:00
950667ed07
GH-115802: Reduce the size of _INIT_CALL_PY_EXACT_ARGS. (GH-116856)
2024-03-15 17:16:30 +00:00
59e30f41ed
gh-116735: Use `MISSING` for `CALL` event if argument is absent (GH-116737)
2024-03-15 14:46:18 +00:00
2cf18a4430
GH-116422: Modify a few uops so that they can be supported by tier 2 with hot/cold splitting (GH-116832)
2024-03-15 10:48:00 +00:00
61e54bfcee
GH-116422: Factor out eval breaker checks at end of calls into its own micro-op. (GH-116817)
2024-03-14 16:31:47 +00:00
cef0ec1a3c
gh-116760: Fix pystats for trace attempts (GH-116761)
...
There are now at least two bytecodes that may attempt to optimize,
JUMP_BACK, and more recently, COLD_EXIT.
Only the JUMP_BACK was counting the attempt in the stats.
This moves that counter to uop_optimize itself so it should
always happen no matter where it is called from.
2024-03-13 22:13:33 +00:00
8332e85b2f
gh-116626: Emit `CALL` events for all `INSTRUMENTED_CALL_FUNCTION_EX` (GH-116627)
2024-03-13 08:28:01 +00:00
4e5df2013f
GH-116468: Use constants instead of `oparg` in stack effects when `oparg` is known to be a constant. (GH-116469)
2024-03-11 09:30:15 +00:00
0003285c8d
GH-113710: Fix optimization of globals using `_CHECK_FUNCTION` (GH-116460)
2024-03-08 09:47:41 +00:00
41457c7fdb
gh-116381: Remove bad specializations, add fail stats (GH-116464)
...
* Remove bad specializations, add fail stats
2024-03-08 00:21:21 +08:00
72d3cc94cd
gh-116437: Use new C API PyDict_Pop() to simplify the code (GH-116438)
2024-03-07 11:21:08 +02:00
7114cf20c0
gh-116381: Specialize CONTAINS_OP (GH-116385)
...
* Specialize CONTAINS_OP
* 📜 🤖 Added by blurb_it.
* Add PyAPI_FUNC for JIT
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2024-03-07 03:30:11 +08:00
27858e2a17
GH-113710: Tier 2 optimizer: check the function instead of checking globals. (GH-116410)
2024-03-06 13:12:23 +00:00
23db9c6227
GH-115685: Split `_TO_BOOL_ALWAYS_TRUE` into micro-ops (GH-116352)
2024-03-05 15:23:08 +00:00
7895a61168
gh-116098: Revert "gh-107674: Improve performance of `sys.settrace` (GH-114986)" (GH-116178)
...
Revert "gh-107674: Improve performance of `sys.settrace` (GH-114986)"
This reverts commit 0a61e23700
2024-03-01 07:46:33 +01:00
d01886c5c9
gh-115685: Type/values propagate for TO_BOOL in tier 2 (GH-115686)
2024-03-01 06:13:38 +08:00
f0df35eeca
GH-115802: JIT "small" code for Windows (GH-115964)
2024-02-29 08:11:28 -08:00
0a61e23700
gh-107674: Improve performance of `sys.settrace` (GH-114986)
2024-02-28 15:21:42 +00:00
e4561e0501
gh-115778: Add `tierN` annotation for instruction definitions ( #115815 )
...
This replaces the old `TIER_{ONE,TWO}_ONLY` macros. Note that `specialized` implies `tier1`.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-02-23 17:31:57 +00:00
b348313e7a
GH-115651: Convert `LOAD_MODULE_ATTR` into `LOAD_INLINE_CONST` when the module is itself a constant. (GH-115711)
2024-02-22 14:48:25 +00:00
520403ed4c
gh-115733: Fix crash involving exhausted list iterator ( #115740 )
...
* gh-115733: Fix crash involving exhausted iterator
* Add blurb
2024-02-21 05:18:44 +09:00
0749244d13
gh-112175: Add `eval_breaker` to `PyThreadState` ( #115194 )
...
This change adds an `eval_breaker` field to `PyThreadState`. The primary
motivation is for performance in free-threaded builds: with thread-local eval
breakers, we can stop a specific thread (e.g., for an async exception) without
interrupting other threads.
The source of truth for the global instrumentation version is stored in the
`instrumentation_version` field in PyInterpreterState. Threads usually read the
version from their local `eval_breaker`, where it continues to be colocated
with the eval breaker bits.
2024-02-20 09:57:48 -05:00
dcba21f905
gh-115687: Split up guards from COMPARE_OP (GH-115688)
2024-02-20 11:30:49 +00:00
626c414995
GH-115457: Support splitting and replication of micro ops. (GH-115558)
2024-02-20 10:50:59 +00:00
7b21403ccd
GH-112354: Initial implementation of warm up on exits and trace-stitching (GH-114142)
2024-02-20 09:39:55 +00:00
9e3729bbd7
gh-114626: add PyCFunctionFast and PyCFunctionFastWithKeywords (GH-114627)
...
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-02-15 11:05:20 +01:00
a2d4281415
gh-112087: Make __sizeof__ and listiter_{len, next} to be threadsafe (gh-114843)
2024-02-15 02:00:50 +09:00
681778c56a
GH-113710: Improve `_SET_IP` and `_CHECK_VALIDITY` (GH-115248)
2024-02-13 16:28:19 +00:00
f9f6156c5a
GH-113710: Backedge counter improvements. (GH-115166)
2024-02-13 14:16:37 +00:00
7cce857622
gh-114058: Foundations of the Tier2 redundancy eliminator (GH-115085)
...
---------
Co-authored-by: Mark Shannon <9448417+markshannon@users.noreply.github.com>
Co-authored-by: Jules <57632293+JuliaPoo@users.noreply.github.com>
Co-authored-by: Guido van Rossum <gvanrossum@users.noreply.github.com>
2024-02-13 21:24:48 +08:00
235cacff81
GH-114695: Add `sys._clear_internal_caches` (GH-115152)
2024-02-12 09:04:36 +00:00
0e71a295e9
GH-113710: Add a "globals to constants" pass (GH-114592)
...
Converts specializations of `LOAD_GLOBAL` into constants during tier 2 optimization.
2024-02-02 12:14:34 +00:00
191531f352
Update outdated comment in ``Python/bytecodes.c`` ( #114522 )
2024-01-24 09:14:15 -08:00
981d172f7f
GH-112354: `END_FOR` instruction to only pop one value. (GH-114247)
...
* Compiler emits END_FOR; POP_TOP instead of END_FOR. To support tier 2 side exits in loops.
2024-01-24 15:10:17 +00:00
384429d1c0
GH-113710: Add a tier 2 peephole optimization pass. (GH-114487)
...
* Convert _LOAD_CONST to inline versions
* Remove PEP 523 checks
2024-01-24 12:08:31 +00:00
2f2ddabd1a
gh-113102: Fix typo in INSTRUMENTED_RESUME (GH-114349)
2024-01-22 11:56:28 +00:00
ac10947ba7
GH-112354: `_GUARD_IS_TRUE_POP` side-exits to target the next instruction, not themselves. (GH-114078)
2024-01-15 11:41:06 +00:00
ac92527c08
gh-113710: Add types to the interpreter DSL ( #113711 )
...
Co-authored-by: Jules <57632293+JuliaPoo@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2024-01-13 01:30:27 +08:00
30e6cbdba2
GH-113860: Get rid of `_PyUOpExecutorObject` (GH-113954)
2024-01-12 11:58:23 +00:00
55824d01f8
GH-113853: Guarantee forward progress in executors (GH-113854)
2024-01-11 18:20:42 +00:00
a0c9cf9456
GH-113860: All executors are now defined in terms of micro ops. Convert counter executor to use uops. (GH-113864)
2024-01-10 15:44:34 +00:00
35ef8cb259
GH-113689: Fix broken handling of invalid executors (GH-113694)
2024-01-04 11:14:15 +00:00
dc8df6e840
GH-113595: Don't enter invalid executor (GH-113596)
2024-01-03 11:01:13 +00:00
a2dd0e7038
gh-111375: Use `NULL` rather than `None` in the exception stack to indicate that an exception was handled ( #113302 )
2023-12-21 01:46:41 +00:00
e96f26083b
GH-111485: Generate instruction and uop metadata (GH-113287)
2023-12-20 14:27:25 +00:00
737d23ffcd
GH-111485: Mark some instructions as `TIER_ONE_ONLY` (GH-113155)
2023-12-15 13:03:17 +00:00
6873555955
GH-112354: Treat _EXIT_TRACE like an unconditional side exit (GH-113104)
2023-12-14 14:26:44 +00:00
956023826a
GH-108866: Guarantee forward progress in executors. (GH-113006)
2023-12-12 19:02:24 +00:00
0c55f27060
GH-111485: Factor out tier 2 code generation from the rest of the interpreter code generator (GH-112968)
2023-12-12 12:12:17 +00:00
b449415b2f
GH-111485: Separate out parsing, analysis and code-gen phases of tier 1 code generator (GH-112299)
2023-12-07 12:49:40 +00:00
07ebd46f9e
gh-112519: Make it possible to specify instruction flags for pseudo instructions in bytecodes.c ( #112520 )
2023-11-30 11:03:30 +00:00
e723700190
Rename ...Uop... to ...UOp... (uppercase O) for consistency ( #112327 )
...
* Rename _PyUopExecute to _PyUOpExecute (uppercase O) for consistency
* Also rename _PyUopName and _PyUOp_Replacements, and some output strings
2023-11-28 17:10:11 -08:00
1995955173
gh-106529: Make FOR_ITER a viable uop ( #112134 )
...
This uses the new mechanism whereby certain uops
are replaced by others during translation,
using the `_PyUop_Replacements` table.
We further special-case the `_FOR_ITER_TIER_TWO` uop
to update the deoptimization target to point
just past the corresponding `END_FOR` opcode.
Two tiny code cleanups are also part of this PR.
2023-11-20 10:08:53 -08:00
eb3c94ea66
gh-110319: Assert type_version != 0 before using it ( #112226 )
...
- Ensure that `assert(type_version != 0);` always comes *before* using `type_version`
Also:
- In cases_generator, rename `-v` to from `--verbose` to `--viable`
2023-11-17 20:58:13 -08:00
be0bd54c6b
gh-106529: Cleanups split off gh-112134 ( #112214 )
...
- Double max trace size to 256
- Add a dependency on executor_cases.c.h for ceval.o
- Mark `_SPECIALIZE_UNPACK_SEQUENCE` as `TIER_ONE_ONLY`
- Add debug output back showing the optimized trace
- Bunch of cleanups to Tools/cases_generator/
2023-11-17 11:49:42 -08:00
a519b87958
GH-111848: Convert remaining jumps to deopts into tier 2 code. (GH-112045)
2023-11-14 15:30:33 +00:00
b11c443bb2
gh-111789: Simplify bytecodes.c by using PyDict_GetItemRef() (GH-111978)
2023-11-14 15:38:49 +02:00
34a03e951b
GH-111843: Tier 2 exponential backoff (GH-111850)
2023-11-09 13:49:51 +00:00
25c4956488
GH-109369: Exit tier 2 if executor is invalid (GH-111657)
2023-11-09 11:19:51 +00:00
06efb60264
GH-111848: Tidy up tier 2 handling of FOR_ITER specialization by using DEOPT_IF instead of jumps. (GH-111849)
2023-11-08 13:31:55 +00:00
931f4438c9
GH-111485: Allow arbitrary annotations on instructions and micro-ops. (GH-111697)
2023-11-07 09:42:39 +00:00
3e99c9cbf6
GH-111485: Make BEFORE_WITH a uop (GH-111812)
2023-11-06 16:42:49 -08:00
d49aba5a7a
gh-111354: Simplify _PyGen_yf by moving some of its work to the compiler and frame state ( #111648 )
2023-11-03 10:01:36 +00:00
3a1b09e6d0
gh-111654: remove redundant decref in LOAD_FROM_DICT_OR_DEREF ( #111655 )
2023-11-02 21:06:51 -07:00
52cc4af6ae
gh-111354: simplify detection of RESUME after YIELD_VALUE at except-depth 1 ( #111459 )
2023-11-02 10:18:43 +00:00
e4b37835ef
GH-111485: Silence warnings in Python/executor_cases.c.h ( #111619 )
2023-11-01 14:24:52 -07:00
7e135a48d6
gh-111520: Integrate the Tier 2 interpreter in the Tier 1 interpreter ( #111428 )
...
- There is no longer a separate Python/executor.c file.
- Conventions in Python/bytecodes.c are slightly different -- don't use `goto error`,
you must use `GOTO_ERROR(error)` (same for others like `unused_local_error`).
- The `TIER_ONE` and `TIER_TWO` symbols are only valid in the generated (.c.h) files.
- In Lib/test/support/__init__.py, `Py_C_RECURSION_LIMIT` is imported from `_testcapi`.
- On Windows, in debug mode, stack allocation grows from 8MiB to 12MiB.
- **Beware!** This changes the env vars to enable uops and their debugging
to `PYTHON_UOPS` and `PYTHON_LLTRACE`.
2023-11-01 13:13:02 -07:00
5697fc2d4b
GH-111537: Avoid using `this_instr` in asserts. (GH-111600)
2023-11-01 12:59:08 +00:00
Irit Katriel