It doesn't make sense to use a deprecation for evaluate_forward_ref,
as it is a new function in Python 3.14 and doesn't have compatibility
guarantees.
I considered making it throw an error if type_params it not passed and
there is no owner. However, I think this is too unfriendly for users. The
case where this param is really needed is fairly esoteric and I don't think
this case is worth the pain of forcing users to write "type_params=()".
Optimize `LOAD_FAST` opcodes into faster versions that load borrowed references onto the operand stack when we can prove that the lifetime of the local outlives the lifetime of the temporary that is loaded onto the stack.
* Rename 'defined' attribute to 'in_local' to more accurately reflect how it is used
* Make death of variables explicit even for array variables.
* Convert in_memory from boolean to stack offset
* Don't apply liveness analyis to optimizer generated code
* Add 'out' parameter to stack.pop
We fix a use-after-free in the `find`, `findtext` and `findall` methods of `xml.etree.ElementTree.Element`
objects that can be triggered when the tag to find implements an `__eq__` method that mutates the
element being queried.
`email.message.Message` objects now validate header names specified via `__setitem__`
or `add_header` according to RFC 5322, §2.2 [1].
In particular, callers should expect a ValueError to be raised for invalid header names.
[1]: https://datatracker.ietf.org/doc/html/rfc5322#section-2.2
---------
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: R. David Murray <rdmurray@bitdance.com>
The bin tag is 3.0.16.1 because we rebuilt without uplink support to fix gh-131804.
This PR also prevents making calls that are now unsafe without uplink, and updates
the tests to property interpret these failures as unsupported.
In the free threaded build, the `_PyObject_LookupSpecial()` call can lead to
reference count contention on the returned function object becuase it
doesn't use stackrefs. Refactor some of the callers to use
`_PyObject_MaybeCallSpecialNoArgs`, which uses stackrefs internally.
This fixes the scaling bottleneck in the "lookup_special" microbenchmark
in `ftscalingbench.py`. However, the are still some uses of
`_PyObject_LookupSpecial()` that need to be addressed in future PRs.
Concurrent accesses from multiple threads to the same `cell` object did not
scale well in the free-threaded build. Use `_PyStackRef` and optimistically
avoid locking to improve scaling.
With the locks around cell reads gone, some of the free threading tests were
prone to starvation: the readers were able to run in a tight loop and the
writer threads weren't scheduled frequently enough to make timely progress.
Adjust the tests to avoid this.
Co-authored-by: Donghee Na <donghee.na@python.org>
* Rename 'defined' attribute to 'in_local' to more accurately reflect how it is used
* Make death of variables explicit even for array variables.
* Convert in_memory from boolean to stack offset
* Don't apply liveness analysis to optimizer generated code
* Fix RETURN_VALUE in optimizer
When `pathlib._os.magic_open()` is called to open a path in binary mode,
raise `ValueError` if any of the *encoding*, *errors* or *newline*
arguments are given. This matches the `open()` built-in.
- Restore max field size to sys.maxsize, as in Python 3.13 & below
- PyCField: Split out bit/byte sizes/offsets.
- Expose CField's size/offset data to Python code
- Add generic checks for all the test structs/unions, using the newly exposed attrs
add a set of asserts to test.test_capi.test_bytearray
1. Assert empty bytearray object for PyByteArray_Check.
2. Assert empty bytearray object for PyByteArray_CheckExact.
3. Assert 0-size bytearray object for PyByteArray_Size.
4. Assert empty bytearray object for PyByteArray_AsString.
5. Assert concatenation of the bytearray object with itself for PyByteArray_Concat.
Adjust the tests for the `pathlib.types` module so that they can be run
against the `pathlib-abc` PyPI package, which is a backport of the module
for older Python versions.
Specifically, we add a `.support.is_pypi` switch that is false in the
stdlib and true in the pathlib-abc package. This controls which package
we import, and whether or not we run tests against `PurePath` and `Path`.
For compatibility with older Python versions, we stop using
`zipfile.ZipFile.mkdir()` and `zipfile.ZipInfo._for_archive()`.
The subinterpreter tests have data races (see gh-129824).
TSAN attempts to intercept some of the fatal signals, which can lead to
bogus reports. We could possibly handle these via TSAN_OPTIONS, but it's
simpler to just skip those tests -- they're not multithreaded anyways.
Call `urllib.request.pathname2url()` from `pathlib.Path.as_uri()`, and
deprecate the duplicate implementation in `PurePath`.
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Deprecate the `nturl2path` module. Its functionality is merged into
`urllib.request`.
Add `tests.test_nturl2path` to exercise `nturl2path`, as it's no longer
covered by `test_urllib`.
* Make musl test skips smarter (fixes Alpine errors)
A relatively small number of tests fail when the underlying c library is
provided by musl. This was originally reported in bpo-46390 by
Christian Heimes. Among other changes, these tests were marked for
skipping in gh-31947/ef1327e3 as part of bpo-40280 (emscripten support),
but the skips were conditioned on the *platform* being emscripten (or
wasi, skips for which ere added in 9b50585e02).
In gh-131071 Victor Stinner added a linked_to_musl function to enable
skipping a test in test_math that fails under musl, like it does on a
number of other platforms. This check can successfully detect that
python is running under musl on Alpine, which was the original problem
report in bpo-46390.
This PR replaces Victor's solution with an enhancement to
platform.libc_ver that does the check more cheaply, and also gets the
version number. The latter is important because the math test being
skipped is due to a bug in musl that has been fixed, but as of this
checkin date has not yet been released. When it is, the test skip can
be fixed to check for the minimum needed version.
The enhanced version of linked_to_musl is also used to do the skips of
the other tests that generically fail under musl, as opposed to
emscripten or wasi only failures. This will allow these tests to be
skipped automatically on Alpine.
This PR does *not* enhance libc_ver to support emscripten and wasi, as
I'm not familiar with those platforms; instead it returns a version
triple of (0, 0, 0) for those platforms. This means the musl tests will
be skipped regardless of musl version, so ideally someone will add
support to libc_ver for these platforms.
* Platform tests and bug fixes.
In adding tests for the new platform code I found a bug in the old code:
if a valid version is passed for version and it is greater than the
version found for an so *and* there is no glibc version, then the
version from the argument was returned. The code changes here fix
that.
* Add support docs, including for some preexisting is_xxx's.
* Add news item about libc_ver enhancement.
* Prettify platform re expression using re.VERBOSE.
Closes#130914: Make graphlib.TopologicalSorter.prepare() idempotent
Relax the rules so that `.prepare()` can be called multiple times, provided that no work has been passed out by `.get_ready()` yet.
Email generators using email.policy.default may convert an RFC 2047
encoded-word to unencoded form during header refolding. In a structured
header, this could allow 'specials' chars outside a quoted-string,
leading to invalid address headers and enabling spoofing. This change
ensures a parsed encoded-word that contains specials is kept as an
encoded-word while the header is refolded.
[Better fix from @bitdancer.]
---------
Co-authored-by: R David Murray <rdmurray@bitdance.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
New features:
* refactor `hashlib_helper.requires_hashdigest` in prevision of a future
`hashlib_helper.requires_builtin_hashdigest` for built-in hashes only
* add `hashlib_helper.requires_openssl_hashdigest` to request OpenSSL
hashes, assuming that `_hashlib` exists.
Refactoring:
* split hmac.copy() test by implementation
* update how algorithms are discovered for RFC test cases
* simplify how OpenSSL hash digests are requested
* refactor hexdigest tests for RFC test vectors
* typo fix: `assert_hmac_hexdigest_by_new` -> `assert_hmac_hexdigest_by_name`
Improvements:
* strengthen contract on `hmac_new_by_name` and `hmac_digest_by_name`
* rename mixin classes to better match their responsibility
Remove the `test.test_pathlib.test_pathlib_abc` test module, which was
hollowed out in previous commits. Its few remaining tests are most relevant
to `PurePath` and `Path`, so we move them into `test_pathlib`.
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
The implementation of `PyLong_FromLong()`, `PyLong_FromLongLong()` and `PyLong_FromSsize_t()`
are now handled by a common macro `PYLONG_FROM_INT` which contains fast paths depending on the
size of the integer to convert. Consequently, `PyLong_FromSsize_t()` for medium-sized integers is faster
by roughly 25%.
---------
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
`concurrent.futures.Executor.map` now supports limiting the number of submitted
tasks whose results have not yet been yielded via the new `buffersize` parameter.
---------
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Modifies the behavior of the interpreter on crash under Emscripten:
1. No Python traceback shown on segfault/trap
2. The JavaScript source line is shown
The JavaScript source line is super long and completely unenlightening,
whereas the Python traceback is very helpful.
Check would fail if value would create a pseudo-member, but that member
had not yet been created. We now attempt to create a pseudo-member for
a passed-in value first.
Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
Test `pathlib.types._WritablePath` in a dedicated test module. These tests
cover `WritableZipPath`, `WritableLocalPath` and `Path`, where the former
two classes are implementations of `_WritablePath` for use in tests.
Add free-threaded versions of existing specialization for FOR_ITER (list, tuples, fast range iterators and generators), without significantly affecting their thread-safety. (Iterating over shared lists/tuples/ranges should be fine like before. Reusing iterators between threads is not fine, like before. Sharing generators between threads is a recipe for significant crashes, like before.)
Test `pathlib.types._ReadablePath` in a dedicated test module. These tests
cover `ReadableZipPath`, `ReadableLocalPath` and `Path`, where the former
two classes are implementations of `_ReadablePath` for use in tests.
Test Windows-flavoured `pathlib.types._JoinablePath` in a dedicated test
module. These tests cover `LexicalWindowsPath`, `PureWindowsPath` and
`WindowsPath`, where `LexicalWindowsPath` is a simple implementation of
`_JoinablePath` for use in tests.
* gh-130959: Reject whitespace in fractions, in pure Python `fromisoformat()`
Fix the pure Python implementation of `fromisoformat()` to reject any
non-digit characters, including whitespace, in the fractional part
of time specification. This makes the behavior consistent with the C
implementation, and prevents incorrect parsing of these fractions
(e.g. `.400 ` would be misinterpreted as `.04`).
* Add the news entry
* Use a different example to fix Sphinx lint
* Apply suggestions from code review
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com>
* Try fixing `:func:` ref.
---------
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com>
Test Posix-flavoured `pathlib.types._JoinablePath` in a dedicated test
module. These tests cover `LexicalPosixPath`, `PurePosixPath` and
`PosixPath`, where `LexicalPosixPath` is a simple implementation of
`_JoinablePath` for use in tests.
Remove the *case_sensitive* argument from `_JoinablePath.full_match()` and
`_ReadablePath.glob()`. Using a non-native case sensitivity forces the use
of "case-pedantic" globbing, where we `iterdir()` even for non-wildcard
pattern segments. But it's hard to know when to enable this mode, as
case-sensitivity can vary by directory, so `_PathParser.normcase()` doesn't
always give the full picture. The `Path.glob()` implementation is forced to
make an educated guess, but we can avoid the issue in the ABCs by dropping
the *case_sensitive* argument.
(I probably shouldn't have added these arguments in `PurePath` and `Path`
in the first place!)
Also drop support for `_ReadablePath.glob(recurse_symlinks=False)`, which
makes recursive globbing much slower.
Test `pathlib.types._JoinablePath` in a dedicated test module. These tests
cover `LexicalPath`, `PurePath` and `Path`, where `LexicalPath` is defined
in a new `test.test_pathlib.support` package.
The test could deadlock trying join on the worker processes.
Apply the same technique as gh-130933.
Join the process before the test ends in `test_notify` as well.
The new methods simply delegate to the underlying buffer, much like the existing GzipFile.read[1] methods. This avoids extra allocations caused by the BufferedIOBase.readinto implementation previously used.
This commit also factors out a common readability check rather than copying it an additional two times.
The workload to advance the virtual timeout is too lightweight for some
platforms. As result the test goes in timeout as it never reaches the
end of the timer. By having a heavier workload, the virtual timer
advances rapidly and the SIGVTALRM is sent before the timeout.
The test could deadlock trying join on the worker processes due to a
combination of behaviors:
* The use of `assertReachesEventually` did not ensure that workers
actually woken.release() because the SyncManager's Semaphore does not
implement get_value.
* This mean that the test could finish and the variable "sleeping" would
got out of scope and be collected. This unregisters the proxy leading
to failures in the worker or possibly the manager.
* The subsequent call to `p.join()` during cleanUp therefore never
finished.
This takes two approaches to fix this:
1) Use woken.acquire() to ensure that the workers actually finish
calling woken.release()
2) At the end of the test, wait until the workers are finished, while `cond`,
`sleeping`, and `woken` are still valid.
I chose to not raise an exception here because I think it would be
confusing for module attribute access to start raising something other
than AttributeError if e.g. the cwd goes away
Without the change in moduleobject.c
```
./python.exe -m unittest test.test_import.ImportTests.test_script_shadowing_stdlib_cwd_failure
...
Assertion failed: (PyErr_Occurred()), function _PyObject_SetAttributeErrorContext, file object.c, line 1253.
```
Fix `test_list.ListTest.test_no_memory` under trace refs build
Memory allocation ends up failing in _PyRefchainTrace(), which produces
different output. Assert that we don't segfault, which is the thing
we want to test and is less brittle than checking output.
This adds two new methods to `multiprocessing`'s `ProcessPoolExecutor`:
- **`terminate_workers()`**: forcefully terminates worker processes using `Process.terminate()`
- **`kill_workers()`**: forcefully kills worker processes using `Process.kill()`
These methods provide users with a direct way to stop worker processes without `shutdown()` or relying on implementation details, addressing situations where immediate termination is needed.
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Sam Gross @colesbury
Commit-message-mostly-authored-by: Claude Sonnet 3.7 (because why not -greg)
* Fix use after free in list objects
Set the items pointer in the list object to NULL after the items array
is freed during list deallocation. Otherwise, we can end up with a list
object added to the free list that contains a pointer to an already-freed
items array.
* Mark `_PyList_FromStackRefStealOnSuccess` as escaping
I think technically it's not escaping, because the only object that
can be decrefed if allocation fails is an exact list, which cannot
execute arbitrary code when it is destroyed. However, this seems less
intrusive than trying to special cases objects in the assert in `_Py_Dealloc`
that checks for non-null stackpointers and shouldn't matter for performance.
The bytecode compiler only generates a few different types of constants,
like str, int, tuple, slices, etc. Users can construct code objects with
various unusual constants, including ones that are not hashable or not
even constant.
The free threaded build previously crashed with a fatal error when
confronted with these constants. Instead, treat distinct objects of
otherwise unhandled types as not equal for the purposes of deduplication.
Fix a race condition in test_check_output_timeout() of
test_subprocess. Don't write into stdout anymore, since there is no
reliable way to synchronize the parent and the child processes.
Change the timeout from 3 seconds to 0.1 seconds, and remove
@requires_resource('walltime') decorator.
Tomas R.