gh-117174: Add a new route in linecache to fetch interactive source code (GH-117500)
(cherry picked from commit a931a8b324)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
gh-125331: Allow the parser to activate future imports on the fly (GH-125482)
(cherry picked from commit 3bd3e09588)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
gh-124927: Fix conversion issue between coordinates and position in REPL (GH-125001)
(cherry picked from commit 6ab5c4aa05)
Co-authored-by: FeH2 <i@feh2.im>
gh-131020: py.exe launcher does not correctly detect a BOM when searching for the shebang (GH-131021)
(cherry picked from commit 36ef3bfe39)
Co-authored-by: Chris Eibl <138194463+chris-eibl@users.noreply.github.com>
The test could deadlock trying join on the worker processes.
Apply the same technique as gh-130933.
Join the process before the test ends in `test_notify` as well.
(cherry picked from commit edd1eca336)
Co-authored-by: Sam Gross <colesbury@gmail.com>
The workload to advance the virtual timeout is too lightweight for some
platforms. As result the test goes in timeout as it never reaches the
end of the timer. By having a heavier workload, the virtual timer
advances rapidly and the SIGVTALRM is sent before the timeout.
(cherry picked from commit 7879081198)
Co-authored-by: Diego Russo <diego.russo@arm.com>
The test could deadlock trying join on the worker processes due to a
combination of behaviors:
* The use of `assertReachesEventually` did not ensure that workers
actually woken.release() because the SyncManager's Semaphore does not
implement get_value.
* This mean that the test could finish and the variable "sleeping" would
got out of scope and be collected. This unregisters the proxy leading
to failures in the worker or possibly the manager.
* The subsequent call to `p.join()` during cleanUp therefore never
finished.
This takes two approaches to fix this:
1) Use woken.acquire() to ensure that the workers actually finish
calling woken.release()
2) At the end of the test, wait until the workers are finished, while `cond`,
`sleeping`, and `woken` are still valid.
(cherry picked from commit c476410dc5)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-130932: Fix incorrect exception handling in _PyModule_IsPossiblyShadowing (GH-130934)
I chose to not raise an exception here because I think it would be
confusing for module attribute access to start raising something other
than AttributeError if e.g. the cwd goes away
Without the change in moduleobject.c
```
./python.exe -m unittest test.test_import.ImportTests.test_script_shadowing_stdlib_cwd_failure
...
Assertion failed: (PyErr_Occurred()), function _PyObject_SetAttributeErrorContext, file object.c, line 1253.
```
(cherry picked from commit 0a9ae5ed48)
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
The bytecode compiler only generates a few different types of constants,
like str, int, tuple, slices, etc. Users can construct code objects with
various unusual constants, including ones that are not hashable or not
even constant.
The free threaded build previously crashed with a fatal error when
confronted with these constants. Instead, treat distinct objects of
otherwise unhandled types as not equal for the purposes of deduplication.
(cherry picked from commit 2905690a91)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-116742: Fix subprocess test_check_output_timeout() (GH-130836)
Fix a race condition in test_check_output_timeout() of
test_subprocess. Don't write into stdout anymore, since there is no
reliable way to synchronize the parent and the child processes.
Change the timeout from 3 seconds to 0.1 seconds, and remove
@requires_resource('walltime') decorator.
(cherry picked from commit 67a942d427)
Co-authored-by: Victor Stinner <vstinner@python.org>
(cherry picked from commit 90130807d9)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
gh-130730: Fix multiprocessing test_active_children() (GH-130837)
Replace a sleep with an event: sleep is not a reliable
synchronization primitive.
(cherry picked from commit 3dd3675492)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-130727: Retry test_wmi on TimeoutError (GH-130832)
Use sleeping_retry() in test_wmi to retry multiple times on
TimeoutError. Wait up to LONG_TIMEOUT seconds (5 minutes by default).
(cherry picked from commit f67ff9e820)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-130737: Fix multiprocessing test_notify() (GH-130797)
Replace hardcoded delay (100 ms) with a loop awaiting until a
condition is true: replace assertReturnsIfImplemented() with
assertReachesEventually().
Use sleeping_retry() in assertReachesEventually() to tolerate slow
buildbots and raise an exception on timeout (30 seconds).
(cherry picked from commit 8a64a62002)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-130379: Fix incorrect zipapp logic to avoid including the target in itself (gh-130509)
(cherry picked from commit 64ccbbbf36)
Co-authored-by: Paul Moore <p.f.moore@gmail.com>
gh-130637: Add validation for numeric response data in `stat()` method (GH-130646)
(cherry picked from commit a42168d316)
Co-authored-by: Kanishk Pachauri <itskanishkp.py@gmail.com>
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
Revert "gh-128364: Fix flaky `test_timeout` test (gh-130724)" (gh-130732)
Change broke Android and iOS buildbots that do not have multiprocessing.
This reverts commit cfa0b1dc37.
(cherry picked from commit 5221d9ce0e)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Extend IPv6 tests and made little syntax refactoring
(cherry picked from commit 9f0879baf1)
Co-authored-by: Ilya Bazhenov <31971067+bazhil@users.noreply.github.com>
gh-129726: Break `gzip.GzipFile` reference loop (GH-130055)
A reference loop was resulting in the `fileobj` held by the `GzipFile`
being closed before the `GzipFile`.
The issue started with gh-89550 in 3.12, but was hidden in most cases
until 3.13 when gh-62948 made it more visible.
(cherry picked from commit 7f39137662)
Co-authored-by: Cody Maloney <cmaloney@users.noreply.github.com>
Fix a few thread-safety bugs to enable test_opcache when run with TSAN:
* Use relaxed atomics when clearing `ht->_spec_cache.getitem`
(gh-115999)
* Add temporary suppression for type slot modifications (gh-127266)
* Use atomic load when reading `*dictptr`
(cherry picked from commit f151d27159)
The use of PySys_GetObject() and _PySys_GetAttr(), which return a borrowed
reference, has been replaced by using one of the following functions, which
return a strong reference and distinguish a missing attribute from an error:
_PySys_GetOptionalAttr(), _PySys_GetOptionalAttrString(),
_PySys_GetRequiredAttr(), and _PySys_GetRequiredAttrString().
(cherry picked from commit 0ef4ffeefd)
* gh-128714: Fix function object races in `__annotate__`, `__annotations__` and `__type_params__` in free-threading build (#129016)
(cherry picked from commit 55f17b77c3)
---------
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Add test checking value of a TypedDict's __total__ attribute when there is an assignment in the class body. (GH-130460)
In relation to GH-109544 which changed this behavior.
(cherry picked from commit d8ce092fe4)
Signed-off-by: Daniel Sperber <github.blurry@9ox.net>
Co-authored-by: Daraan <github.blurry@9ox.net>
gh-129363: Change regrtest sequential mode output (GH-129476)
First, write the test name without color. Then, write the test name
and the result with color. Each test is displayed twice.
(cherry picked from commit f1b81c408f)
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
The `test_trace.test_coverage_ignore` test would fail if you had
`setuptools` installed, such as in `~/.local/lib/python3.xxx/site-packages/`.
Ignore everything in `sys.path` when running the test.
(cherry picked from commit 35925e9529)
gh-130250: use support.swap_attr() in test and don't assume we're beginning with a clean system state (GH-130342)
(cherry picked from commit 048ee2d5ec)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Ensure colorize tests will run on dumb terminals (or environment with TERM=dumb set)
(cherry picked from commit 1b070060c0)
Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
[3.13] gh-128396: Fix a crash when inline comprehension has the same local variable as the outside scope (GH-130235)
(cherry picked from commit ccf17323c2)
We had the definition of what makes a character "printable" documented in three places, giving two different definitions.
The definition in the comment on `_PyUnicode_IsPrintable` was inverted; correct that.
With that correction, the two definitions turn out to be equivalent -- but to confirm that, you have to go look up, or happen to know, that those are the only five "Other" categories and only three "Separator" categories in the Unicode character database. That makes it hard for the reader to tell whether they really are the same, or if there's some subtle difference in the intended semantics.
Fix that by cutting the C API docs' and the C comment's copies of the subtle details, in favor of referring to the Python-level docs. That ensures it's explicit that these are all meant to agree, and also lets us concentrate improvements to the wording in one place.
Speaking of which, borrow some ideas from the C comment, along with other tweaks, to hopefully add a bit more clarity to that one newly-centralized copy in the docs.
Also add a thorough test that the implementation agrees with this definition.
Author: Greg Price <gnprice@gmail.com>
Co-authored-by: Greg Price <gnprice@gmail.com>
(cherry picked from commit 3402e133ef)
gh-130185: Fix unintentionally skipped tests in `test_functools` (GH-130186)
(cherry picked from commit 73d03005b0)
Co-authored-by: Tomas R <tomas.roun8@gmail.com>
gh-129693: Suppress `SyntaxWarning` in test_fstring (GH-129830)
Suppress SyntaxWarning in test_fstring
(cherry picked from commit 2dd018848c)
Co-authored-by: Tomas R <tomas.roun8@gmail.com>
The call to `PySequence_List()` could temporarily unlock and relock the
set, allowing the items to be cleared and return the incorrect
notation `{}` for a empty set (it should be `set()`).
(cherry picked from commit a7427f2db9)
Co-authored-by: T. Wouters <thomas@python.org>
gh-126554: correct detection of `gcc` for `TestNullDlsym.test_null_dlsym` (GH-129872)
In case gcc is not available, the test will fail with FileNotFoundError.
So catch the exception to skip the test correctly.
(cherry picked from commit 978211c8a8)
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Co-authored-by: Peter Marko <peter.marko@siemens.com>
Test that the trailing pathname separator is preserved.
Multiple trailing pathname separators are only preserved if the pattern
does not contain metacharacters, otherwise only one trailing pathname
separator is preserved. This is rather an implementation detail.
(cherry picked from commit 8b5c8508c7)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-127975: Avoid reusing quote types in ast.unparse if not needed (GH-127980)
(cherry picked from commit 8df5193d37)
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
* Do not recreate unnamed section on every read in ConfigParser
* Remove duplicate section creation code
(cherry picked from commit 914c232e93)
Co-authored-by: Andrey Efremov <duxus@yandex.ru>
gh-115514: Fix incomplete writes after close while using ssl in asyncio(GH-128037)
(cherry picked from commit 4e38eeafe2)
Co-authored-by: Vojtěch Boček <vbocek@gmail.com>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
gh-105704: Disallow square brackets (`[` and `]`) in domain names for parsed URLs (GH-129418)
* gh-105704: Disallow square brackets ( and ) in domain names for parsed URLs
* Use Sphinx references
* Add mismatched bracket test cases, fix news format
* Add more test coverage for ports
---------
(cherry picked from commit d89a5f6a6e)
Co-authored-by: Seth Michael Larson <seth@python.org>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
gh-119461: Restore the testSocket VSOCK skipUnless removed by PR GH-119465 (GH-129561)
Restore the skipUnless removed by GH-119465.
This test can only pass on virtual machines, not actual machines.
actual machines see:
```
self.cli.connect((cid, VSOCKPORT))
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
OSError: [Errno 19] No such device
```
Reproduced on (Linux) Ubuntu 24.04.1 running 6.8.0-52-generic.
(cherry picked from commit e1006ce1de)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
gh-112064: Fix incorrect handling of negative read sizes in `HTTPResponse.read()` (GH-128270)
The parameter `amt` of `HTTPResponse.read()`, which could be a negative integer,
has not been handled before and led to waiting for the connection to close
for `keep-alive connections`. Now, this has been fixed, and passing negative values
to `HTTPResponse().read()` works the same as passing `None` value.
(cherry picked from commit 4d0d24f6e3)
Co-authored-by: Yury Manushkin <manushkin@gmail.com>
gh-119511: Fix a potential denial of service in imaplib (GH-119514)
The IMAP4 client could consume an arbitrary amount of memory when trying
to connect to a malicious server, because it read a "literal" data with a
single read(size) call, and BufferedReader.read() allocates the bytes
object of the specified size before reading. Now the IMAP4 client reads data
by chunks, therefore the amount of used memory is limited by the
amount of the data actually been sent by the server.
(cherry picked from commit 735f25c5e3)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
gh-129061: Fix `FORCE_COLOR` and `NO_COLOR` when empty strings (GH-129140)
(cherry picked from commit 9546fe2ef2)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
gh-123987: Fix NotADirectoryError in NamespaceReader when sentinel present (GH-124018)
(cherry picked from commit b543b32eff)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
(cherry picked from commit 7907203bc0)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
gh-118878: Pyrepl: show completions menu below the current line (GH-118939)
(cherry picked from commit 29caec62ee)
Co-authored-by: Daniel Hollas <daniel.hollas@bristol.ac.uk>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
gh-129185: Fix PyTraceMalloc_Untrack() at Python exit (#129191)
Support calling PyTraceMalloc_Track() and PyTraceMalloc_Untrack()
during late Python finalization.
* Call _PyTraceMalloc_Fini() later in Python finalization.
* Test also PyTraceMalloc_Untrack() without the GIL
* PyTraceMalloc_Untrack() now gets the GIL.
* Test also PyTraceMalloc_Untrack() in test_tracemalloc_track_race().
(cherry picked from commit 46c7e13c05)
gh-124363: Treat debug expressions in f-string as raw strings (GH-128399)
(cherry picked from commit 60a3a0dd6f)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
gh-128636: Fix crash in PyREPL when `os.environ` is overwritten with an invalid value (GH-128653)
(cherry picked from commit ba9a4b6215)
Co-authored-by: Tomas R <tomas.roun8@gmail.com>
gh-129093: Fix f-string debug text sometimes getting cut off when expression contains `!` (GH-129159)
(cherry picked from commit 767cf70844)
Co-authored-by: Tomas R <tomas.roun8@gmail.com>
gh-125553: Fix backslash continuation in `untokenize` (GH-126010)
(cherry picked from commit 7ad793e5db)
Co-authored-by: Tomas R <tomas.roun8@gmail.com>
Add a mix-in class ExtraAssertions containing the following methods:
* assertHasAttr() and assertNotHasAttr()
* assertIsSubclass() and assertNotIsSubclass()
* assertStartsWith() and assertNotStartsWith()
* assertEndsWith() and assertNotEndsWith()
(cherry picked from commit 06cad77a5b)
gh-80222: Fix email address header folding with long quoted-string (GH-122753)
Email generators using email.policy.default could incorrectly omit the
quote ('"') characters from a quoted-string during header refolding,
leading to invalid address headers and enabling header spoofing. This
change restores the quote characters on a bare-quoted-string as the
header is refolded, and escapes backslash and quote chars in the string.
(cherry picked from commit 5aaf416858)
Co-authored-by: Mike Edmunds <medmunds@gmail.com>
tracemalloc_alloc(), tracemalloc_realloc(), PyTraceMalloc_Track(),
PyTraceMalloc_Untrack() and _PyTraceMalloc_TraceRef() now check
tracemalloc_config.tracing after calling TABLES_LOCK().
_PyTraceMalloc_Stop() now protects more code with TABLES_LOCK(),
especially setting tracemalloc_config.tracing to 1.
Add a test using PyTraceMalloc_Track() to test tracemalloc.stop()
race condition.
Call _PyTraceMalloc_Init() at Python startup.
* Fix a deprecation warning for using importlib.resources.abc.ResourceReader.
* Fix an import warning when importing readline (if it has not yet been imported).
(cherry picked from commit 599be687ec)
Co-authored-by: Thomas Grainger <tagrain@gmail.com>
gh-109959: Skip test_glob.test_selflink() flaky test (GH-128812)
(cherry picked from commit 1153e66e20)
Co-authored-by: Victor Stinner <vstinner@python.org>
[3.13] gh-128717: Stop-the-world when setting the recursion limit (GH-128741)
(cherry picked from commit f6c61bf2d7)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
gh-125997: Increase test coverage for `time.sleep()` (GH-128751)
- Add tests for durations of invalid types.
- Add tests for `int` and `float` durations, including signed zeroes durations.
- Add tests for nonzero very small durations and durations close to the clock resolution.
---------
(cherry picked from commit b70a567575)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
There were possible conflicts if the widget class name ends with a digit.
(cherry picked from commit da8825ea95)
Co-authored-by: Zhikang Yan <2951256653@qq.com>
gh-128302: Fix bugs in xml.dom.xmlbuilder (GH-128284)
* Allow DOMParser.parse() to correctly handle DOMInputSource instances
that only have a systemId attribute set.
* Fix DOMEntityResolver.resolveEntity(), which was broken by the
Python 3.0 transition.
* Add Lib/test/test_xml_dom_xmlbuilder.py with few tests.
(cherry picked from commit 6ea04da270)
Co-authored-by: Stephen Morton <git@tungol.org>
gh-127196: Fix crash in `_interpreters`, when `shared` had invalid encodings (GH-127220)
(cherry picked from commit 087bb48aca)
Co-authored-by: sobolevn <mail@sobolevn.me>
It now supports docstrings with single quotes, escape sequences,
raw string literals, and other Python syntax.
(cherry picked from commit 474e419792)
Co-authored-by: Srinivas Reddy Thatiparthy (తాటిపర్తి శ్రీనివాస్ రెడ్డి) <thatiparthysreenivas@gmail.com>
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-98188: Fix EmailMessage.get_payload to decode data when CTE value has extra text (GH-127547)
Up to this point message handling has been very strict with regards to content encoding values: mixed case was accepted, but trailing blanks or other text would cause decoding failure, even if the first token was a valid encoding. By Postel's Rule we should go ahead and decode as long as we can recognize that first token. We have not thought of any security or backward compatibility concerns with this fix.
This fix does introduce a new technique/pattern to the Message code: we look to see if the header has a 'cte' attribute, and if so we use that. This effectively promotes the header API exposed by HeaderRegistry to an API that any header parser "should" support. This seems like a reasonable thing to do. It is not, however, a requirement, as the string value of the header is still used if there is no cte attribute.
The full fix (ignore any trailing blanks or blank-separated trailing text) applies only to the non-compat32 API. compat32 is only fixed to the extent that it now ignores trailing spaces. Note that the HeaderRegistry parsing still records a HeaderDefect if there is extra text.
(cherry picked from commit a62ba52f14)
Co-authored-by: RanKKI <hliu86.me@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-127903: Fix a crash on debug builds when calling `Objects/unicodeobject::_copy_characters`` (GH-127876)
(cherry picked from commit 46cb6340d7)
Co-authored-by: Alexander Shadchin <shadchin@yandex-team.com>
Union now uses the instance checks against its parameters instead of
the subclass checks.
(cherry picked from commit b2ac70a62a)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-127586: multiprocessing.Pool does not properly restore blocked signals (try 2) (GH-128011)
Correct pthread_sigmask in resource_tracker to restore old signals
Using SIG_UNBLOCK to remove blocked "ignored signals" may accidentally
cause side effects if the calling parent already had said signals
blocked to begin with and did not intend to unblock them when
creating a pool. Use SIG_SETMASK instead with the previous mask of
blocked signals to restore the original blocked set.
(cherry picked from commit aeb9b65aa2)
Co-authored-by: Stephen Hansen <stephen.paul.hansen@gmail.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
gh-127537: Add __class_getitem__ to the python implementation of functools.partial (GH-127537)
(cherry picked from commit 401bba6b58)
Co-authored-by: CF Bolz-Tereick <cfbolz@gmx.de>
gh-127847: Fix position in the special-cased zipfile seek (GH-127856)
---------
(cherry picked from commit 7ed6c5c696)
Co-authored-by: Dima Ryazanov <dima@bucket.xxx>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
gh-128049: Fix type confusion bug with the return value of a custom ExceptionGroup split function (GH-128079)
(cherry picked from commit 3879ca0100)
Co-authored-by: Nico-Posada <102486290+Nico-Posada@users.noreply.github.com>
gh-128030: Avoid error from PyModule_GetFilenameObject for non-module (GH-128047)
I missed the extra `PyModule_Check` in GH-127660 because I was looking at
3.12 as the base implementation for import from. This meant that I
missed the `PyModuleCheck` introduced in GH-112661.
(cherry picked from commit 45e6dd63b8)
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
gh-126742: Avoid checking for library filename in test_ctypes (GH-128034)
Avoid checking for library filename in `dlerror()` error messages of test_ctypes.
(cherry picked from commit 335e24fb0a)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
- Add a helper to set an error from locale-encoded `char*`
- Use the helper for gdbm & dlerror messages
(cherry picked from commit 7303f06846)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-127353: Allow to force color output on Windows V2 (GH-127926)
(cherry picked from commit 0ac40acec0)
Co-authored-by: Andrey Efremov <duxus@yandex.ru>
gh-127906: Test the limited C API in test_cppext (GH-127916)
(cherry picked from commit d05a4e6a0d)
Co-authored-by: Victor Stinner <vstinner@python.org>
This backports the *test* from GH-126938, with changed limit and exception class.
Co-authored-by: Melissa0x1f992 <70096546+Melissa0x1f992@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Adds a `use_system_log` config item to enable stdout/stderr redirection for
Apple platforms. This log streaming is then used by a new iOS test runner
script, allowing the display of test suite output at runtime. The iOS test
runner script can be used by any Python project, not just the CPython test
suite.
(cherry picked from commit 2041a95e68)
gh-127655: Ensure `_SelectorSocketTransport.writelines` pauses the protocol if needed (GH-127656)
Ensure `_SelectorSocketTransport.writelines` pauses the protocol if it reaches the high water mark as needed.
(cherry picked from commit e991ac8f20)
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
gh-122431: Disallow negative values in `readline.append_history_file` (GH-122469)
(cherry picked from commit 208b0fb645)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-127421: Fix race in test_start_new_thread_failed (GH-127549)
Fix race in test_start_new_thread_failed
When we succeed in starting a new thread, for example if setrlimit
was ineffective, we must wait for the newly spawned thread to exit.
Otherwise, we run the risk that the newly spawned thread will race
with runtime finalization and access memory that has already been
clobbered/freed.
`_thread.start_new_thread()` only spawns daemon threads, which the runtime
does not wait for at shutdown, and does not return a handle. Use
`_thread.start_joinable_thread()` and join the resulting handle when
the thread is started successfully.
(cherry picked from commit 13b68e1a61)
Co-authored-by: mpage <mpage@meta.com>
Make IPv4-mapped IPv6 address properties consistent with IPv4.
(cherry picked from commit 76a1c5d183)
Co-authored-by: Seth Michael Larson <seth@python.org>
This approach eliminates the originally reported race. It also gets rid of the deadlock reported in gh-96071, so we can remove the workaround added then.
This is mostly a cherry-pick of 1c0a104 (AKA gh-126989). The difference is we add PyInterpreterState.threads_preallocated at the end of PyInterpreterState, instead of adding PyInterpreterState.threads.preallocated. That avoids ABI disruption.
gh-126876: Fix socket internal_select() for large timeout (GH-126968)
If the timeout is larger than INT_MAX, replace it with INT_MAX, in
the poll() code path.
Add an unit test.
(cherry picked from commit b3687ad454)
Co-authored-by: Victor Stinner <vstinner@python.org>
Note, that transformed expression is not an equivalent for original one (1/exp(-x) != exp(x) in general for floating-point numbers). Though, the difference seems to be ~1ULP for good libm implementations.
It's more interesting why division was used from beginning. Closest algorithm I've found (no error checks, of course;)) - it's Algorithm 190 from ACM: https://dl.acm.org/doi/10.1145/366663.366679. It uses subtraction in the exponent.
(cherry picked from commit f7bb658124)
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
gh-126699: allow AsyncIterator to be used as a base for Protocols (GH-126702)
(cherry picked from commit feb3e0b19c)
Co-authored-by: Stephen Morton <github@tungol.org>
gh-125666: Avoid PyREPL exiting when a null byte is in input (GH-125732)
(cherry picked from commit 44becb8cba)
Co-authored-by: devdanzin <74280297+devdanzin@users.noreply.github.com>
Make SSL objects thread safe in Free Theaded build by
using critical sections.
(cherry picked from commit 4c53b25775)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-127190: Fix local_setattro() error handling (GH-127366)
Don't make the assumption that the 'name' argument is a string. Use
repr() to format the 'name' argument instead.
(cherry picked from commit 20657fbdb1)
Co-authored-by: Victor Stinner <vstinner@python.org>
Since MultiByteToWideChar()/WideCharToMultiByte() is not reversible if
the data contains invalid UTF-8 sequences, use binary search to
calculate the number of written bytes from the number of written
characters.
Also fix writing incomplete UTF-8 sequences.
Also fix handling of memory allocation failures.
(cherry picked from commit 3cf83d91a5)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-127208: Reject null character in _imp.create_dynamic() (GH-127400)
_imp.create_dynamic() now rejects embedded null characters in the
path and in the module name.
(cherry picked from commit b14fdadc6c)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-124873: Skip timerfd tests on Android (GH-127279)
* Revert "[3.13] gh-124873: Tolerate 100 ms in TimerfdTests on Android (GH-127101) (GH-127105)"
This reverts commit c09366b1fe.
* Skip timerfd tests on Android.
(cherry picked from commit 4ca2c82862)
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-127258: Fix asyncio test_staggered_race_with_eager_tasks() (GH-127358)
Replace the sleep(2) with a task which is blocked forever.
(cherry picked from commit bfabf96b50)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-127182: Fix `io.StringIO.__setstate__` crash when `None` is the first value (GH-127219)
(cherry picked from commit a2ee899682)
Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Victor Stinner <vstinner@python.org>
If Python fails to start newly created thread
due to failure of underlying PyThread_start_new_thread() call,
its state should be removed from interpreter' thread states list
to avoid its double cleanup.
(cherry picked from commit ca3ea9ad05)
Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
Threads are gone after fork, so clear the queues too. Otherwise the
child process (here created via multiprocessing.Process) crashes on
interpreter exit.
(cherry picked from commit 1848ce61f3)
Co-authored-by: Andrei Bodrov <Drino@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-109413: Fix libregrtest get_running() (GH-127153)
Skip threads which are not running.
(cherry picked from commit 0cb20177d6)
Co-authored-by: Victor Stinner <vstinner@python.org>
* Name without a PATHEXT extension is only searched if the mode does not
include X_OK.
* Support multi-component PATHEXT extensions (e.g. ".foo.bar").
* Support files without extensions in PATHEXT contains dot-only extension
(".", "..", etc).
* Support PATHEXT extensions that end with a dot (e.g. ".foo.").
(cherry picked from commit 8899e85de1)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
GH-127078: `url2pathname()`: handle extra slash before UNC drive in URL path (GH-127132)
Decode a file URI like `file://///server/share` as a UNC path like
`\\server\share`. This form of file URI is created by software the simply
prepends `file:///` to any absolute Windows path.
(cherry picked from commit 8c98ed846a)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
GH-126766: `url2pathname()`: handle 'localhost' authority (GH-127129)
Discard any 'localhost' authority from the beginning of a `file:` URI. As a
result, file URIs like `//localhost/etc/hosts` are correctly decoded as
`/etc/hosts`.
(cherry picked from commit ebf564a1d3)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
Some fields in PyCodeObject are lazily initialized. Use atomics and
critical sections to make their initializations and accesses thread-safe.
(cherry picked from commit 3926842117)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-124873: Tolerate 100 ms in TimerfdTests on Android (GH-127101)
On Android, TimerfdTests of test_os now uses 100 ms accuracy instead
of 10 ms.
(cherry picked from commit bab4b0462e)
Co-authored-by: Victor Stinner <vstinner@python.org>
It now returns multiple era description segments separated by semicolons.
Previously it only returned the first segment on platforms with Glibc.
(cherry picked from commit 4803cd0244)
* Fix support of STRING and GLOBAL opcodes with non-ASCII arguments.
* dis() now outputs non-ASCII bytes in STRING, BINSTRING and
SHORT_BINSTRING arguments as escaped (\xXX).
(cherry picked from commit eaf2171082)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
GH-85168: Use filesystem encoding when converting to/from `file` URIs (GH-126852)
Adjust `urllib.request.url2pathname()` and `pathname2url()` to use the
filesystem encoding when quoting and unquoting file URIs, rather than
forcing use of UTF-8.
No changes are needed in the `nturl2path` module because Windows always
uses UTF-8, per PEP 529.
(cherry picked from commit c9b399fbdb)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
gh-126909: Fix running xattr tests on systems with lower limits (GH-126930)
Modify the extended attribute tests to write fewer and smaller extended
attributes, in order to fit within filesystems with total xattr limit
of 1 KiB (e.g. ext4 with 1 KiB blocks). Previously, the test would
write over 2 KiB, making it fail with ENOSPC on such systems.
(cherry picked from commit 2c0a21c1aa)
Co-authored-by: Michał Górny <mgorny@gentoo.org>
If SRE(match) function terminates abruptly, either because of a signal
or because memory allocation fails, allocated SRE_REPEAT blocks might
be never released.
Co-authored-by: <wjssz@users.noreply.github.com>
(cherry picked from commit 7538e7f569)
gh-124452: Fix header mismatches when folding/unfolding with email message (GH-125919)
The header-folder of the new email API has a long standing known buglet where
if the first token is longer than max_line_length, it puts that token on the next
line. It turns out there is also a *parsing* bug when parsing such a header:
the space prefixing that first, non-empty line gets preserved and tacked on to
the start of the header value, which is not the expected behavior per the RFCs.
The bug arises from the fact that the parser assumed that there would be at
least one token on the line with the header, which is going to be true for
probably every email producer other than the python email library with its
folding buglet. Clearly, though, this is a case that needs to be handled
correctly. The fix is simple: strip the blanks off the start of the whole
value, not just the first physical line of the value.
(cherry picked from commit ed81971e6b)
Co-authored-by: RanKKI <hliu86.me@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
For dlsym(), a return value of NULL does not necessarily indicate
an error [1].
Therefore, to avoid using stale (or NULL) dlerror() values, we must:
1. clear the previous error state by calling dlerror()
2. call dlsym()
3. call dlerror()
If the return value of dlerror() is not NULL, an error occured.
In ctypes we choose to treat a NULL return value from dlsym()
as a "not found" error. This is the same as the fallback
message we use on Windows, Cygwin or when getting/formatting
the error reason fails.
[1]: https://man7.org/linux/man-pages/man3/dlsym.3.html
(cherry picked from commit 8717f792f7)
Signed-off-by: Georgios Alexopoulos <grgalex@ba.uoa.gr>
Co-authored-by: George Alexopoulos <giorgosalexo0@gmail.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
gh-126476: Raise IllegalMonthError for calendar.formatmonth() when the input month is not correct (GH-126484)
(cherry picked from commit 3be7498d24)
Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
GH-126766: `url2pathname()`: handle empty authority section. (GH-126767)
Discard two leading slashes from the beginning of a `file:` URI if they
introduce an empty authority section. As a result, file URIs like
`///etc/hosts` are correctly parsed as `/etc/hosts`.
(cherry picked from commit cae9d9d20f)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
Fixes a bug where pygettext would attempt
to extract a message from a code like this:
def _(x): pass
This is because pygettext only looks at one
token at a time and '_(x)' looks like a
function call.
However, since 'x' is not a string literal,
it would erroneously issue a warning.
(cherry picked from commit 9a456383be)
Co-authored-by: Tomas R <tomas.roun8@gmail.com>
In strict mode, raise `NotADirectoryError` if we encounter a non-directory
while we still have path parts left to process.
We use a `part_count` variable rather than `len(rest)` because the `rest`
stack also contains markers for unresolved symlinks.
(cherry picked from commit fd4b5453df)
gh-104745: Limit starting a patcher more than once without stopping it (GH-126649)
Previously, this would cause an `AttributeError` if the patch stopped more than once after this, and would also disrupt the original patched object.
---------
(cherry picked from commit 1e40c5ba47)
Co-authored-by: Red4Ru <39802734+Red4Ru@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
GH-120423: `pathname2url()`: handle forward slashes in Windows paths (GH-126593)
Adjust `urllib.request.pathname2url()` so that forward slashes in Windows
paths are handled identically to backward slashes.
(cherry picked from commit bf224bd7ce)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
* upper-case non-BMP character was ignored
* the ASCII flag was ignored when matching a character range whose
upper bound is beyond the BMP region
(cherry picked from commit 819830f34a)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-126500: test_ssl: Don't stop ThreadedEchoServer on OSError in ConnectionHandler; rely on __exit__ (GH-126503)
If `read()` in the ConnectionHandler thread raises `OSError` (except `ConnectionError`),
the ConnectionHandler shuts down the entire ThreadedEchoServer,
preventing further connections.
It also does that for `EPROTOTYPE` in `wrap_conn`.
As far as I can see, this is done to avoid the server thread getting stuck,
forgotten, in its accept loop. However, since 2011 (5b95eb90a7)
the server is used as a context manager, and its `__exit__` does `stop()` and `join()`.
(I'm not sure if we *always* used `with` since that commit, but currently we do.)
Make sure that the context manager *is* used, and remove the `server.stop()`
calls from ConnectionHandler.
(cherry picked from commit c9cda1608e)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
gh-126499: test_ssl: Don't assume err.reason is a string (GH-126501)
The skipping machinery called `getattr(err, "reason", "")` on an arbitrary
exception. As intermittent Buildbot failures show, sometimes it's set
to None.
Convert it to string for this specific check.
(cherry picked from commit 78ad7e6322)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
gh-126654: Fix crash in several functions in `_interpreters` module (GH-126678)
(cherry picked from commit 9fc2808eaf)
Co-authored-by: sobolevn <mail@sobolevn.me>
gh-117378: Only run the new multiprocessing SysPath test when appropriate (GH-126635)
The first version had it running two forkserver and one spawn tests underneath each of the _fork, _forkserver, and _spawn test suites that build off the generic one.
This adds to the existing complexity of the multiprocessing test suite by offering BaseTestCase classes another attribute to control which suites they are invoked under. Practicality vs purity here. :/
Net result: we don't over-run the new test and their internal logic is simplified.
(cherry picked from commit ca878b6e45)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Skip test in test_socket.py if `sys.getrefcount` isn't available (GH-126640)
Skip `testMakefileCloseSocketDestroy` test if `sys.getrefcount` isn't available. This is necessary for PyPy and other Python implementations that do not have `sys.getrefcount`.
(cherry picked from commit 0f6bb28ff3)
Co-authored-by: CF Bolz-Tereick <cfbolz@gmx.de>
gh-117378: Fix multiprocessing forkserver preload sys.path inheritance. (GH-126538)
`sys.path` was not properly being sent from the parent process when launching
the multiprocessing forkserver process to preload imports. This bug has been
there since the forkserver start method was introduced in Python 3.4. It was
always _supposed_ to inherit `sys.path` the same way the spawn method does.
Observable behavior change: A `''` value in `sys.path` will now be replaced in
the forkserver's `sys.path` with an absolute pathname
`os.path.abspath(os.getcwd())` saved at the time that `multiprocessing` was
imported in the parent process as it already was when using the spawn start
method. **This will only be observable during forkserver preload imports**.
The code invoked before calling things in another process already correctly sets `sys.path`.
Which is likely why this went unnoticed for so long as a mere performance issue in
some configurations.
A workaround for the bug on impacted Pythons is to set PYTHONPATH in the
environment before multiprocessing's forkserver process was started. Not perfect
as that is then inherited by other children, etc, but likely good enough for many
people's purposes.
(cherry picked from commit 9d08423b6e)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
GH-126212: Fix removal of slashes in file URIs on Windows (GH-126214)
Adjust `urllib.request.pathname2url()` and `url2pathname()` so that they
don't remove slashes from Windows DOS drive paths and URLs. There was no
basis for this behaviour, and it conflicts with how UNC and POSIX paths are
handled.
(cherry picked from commit 54c63a32d0)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
pickle.Pickler.persistent_id and pickle.Unpickler.persistent_load can
again be overridden as instance attributes.
(cherry picked from commit 223d3dc554)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-70764: inspect.getclosurevars now identifies global variables with LOAD_GLOBAL (GH-120143)
(cherry picked from commit 83ba8c2bba)
Co-authored-by: blhsing <blhsing@gmail.com>
gh-125783: Add more tests to prevent regressions with the combination of ctypes and metaclasses. (GH-126126)
(cherry picked from commit 6c67446a6e)
Co-authored-by: Jun Komoda <45822440+junkmd@users.noreply.github.com>
gh-125522: Remove bare except in test_zlib.test_flushes (gh-126321)
(cherry picked from commit cfb1b2f0cb)
Co-authored-by: simple-is-great <103080930+simple-is-great@users.noreply.github.com>
GH-126205: Fix conversion of UNC paths to file URIs (GH-126208)
File URIs for Windows UNC paths should begin with two slashes, not four.
(cherry picked from commit 951cb2c369)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (GH-126103)
(cherry picked from commit d07dcce693)
Co-authored-by: Nico-Posada <102486290+Nico-Posada@users.noreply.github.com>
gh-60712: Include the "object" type in the lists of documented types (GH-103036)
* add test for the predefined object's attributes
* Include the "object" type in the lists of documented types
* remove 'or' from augment tuple
* 📜🤖 Added by blurb_it.
* Add cross-reference to news
* Fix format for the function parameter
* Add space
* add reference for the 'object'
* add reference for NotImplemented
* Change ref:`string <textseq>` as class:`str`
* remove hyphen from `newly-created`
* Update Doc/reference/datamodel.rst
'dictionaries' to 'dict'
* Update predefined attribute types in testPredefinedAttrs
* Change `universal type` as `top type`
* Don't mention about the top type
* Update the description of richcmpfuncs
* Update Doc/library/stdtypes.rst
* Revert: Hierarchy Section in Data Model Documentation
* Revert to original explanations of __new__ and __init__ methods in datamodel.rst for improved clarity.
* Update Doc/reference/datamodel.rst
* Remove blank line
* Use ref:`str <textseq>` instead of :class:`str
* Revert changes the description of Other Built-in Types in stdtypes.rst
* Update Doc/reference/datamodel.rst
---------
(cherry picked from commit 4f826214b3)
Co-authored-by: Furkan Onder <furkanonder@protonmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
GH-125866: Improve tests for `pathname2url()` and `url2pathname()` (GH-125993)
Merge `URL2PathNameTests` and `PathName2URLTests` test cases (which test
only the Windows-specific implementations from `nturl2path`) into the main
`Pathname_Tests` test case for these functions.
Copy/port some test cases for `pathlib.Path.as_uri()` and `from_uri()`.
(cherry picked from commit 6742f14dfd)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
gh-126105: Fix crash in `ast` module, when `._fields` is deleted (GH-126115)
Previously, if the `ast.AST._fields` attribute was deleted, attempts to create a new `as`t node would crash due to the assumption that `_fields` always had a non-NULL value. Now it has been fixed by adding an extra check to ensure that `_fields` does not have a NULL value (this can happen when you manually remove `_fields` attribute).
(cherry picked from commit b2eaa75b17)
Co-authored-by: sobolevn <mail@sobolevn.me>
gh-126014: Ignore `__pycache__`-only folders in makefile tests (GH-126066)
(cherry picked from commit aeafaf4cda)
Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
gh-125783: Add tests to prevent regressions with the combination of `ctypes` and metaclasses. (GH-125881)
(cherry picked from commit 1384409460)
Co-authored-by: Jun Komoda <45822440+junkmd@users.noreply.github.com>
gh-125966: fix use-after-free on `fut->fut_callback0` due to an evil callback's `__eq__` in asyncio (GH-125967)
(cherry picked from commit ed5059eeb1)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-125984: fix use-after-free on `fut->fut_{callback,context}0` due to an evil `loop.__getattribute__` (GH-126003)
(cherry picked from commit f819d4301d)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-125593: Use colors to highlight error locations in tracebacks from exception group (GH-125681)
(cherry picked from commit 51b012b2a8)
Co-authored-by: Bogdan Romanyuk <65823030+wrongnull@users.noreply.github.com>
gh-118950: Fix SSLProtocol.connection_lost not being called when OSError is thrown (GH-118960)
(cherry picked from commit 3f24bde0b6)
Co-authored-by: Javad Shafique <javadshafique@hotmail.com>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
gh-125969: fix OOB in `future_schedule_callbacks` due to an evil `call_soon` (GH-125970)
(cherry picked from commit c5b99f5c2c)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
Dictionary watchers on an object's attributes dictionary
(`object.__dict__`) were not triggered when the managed dictionary used
the object's inline values.
(cherry picked from commit 5989eb7446)
Co-authored-by: Sam Gross <colesbury@gmail.com>
GH-125789: fix `fut._callbacks` to always return a copy of callbacks (#125922)
Fix `asyncio.Future._callbacks` to always return a copy of the internal list of callbacks to avoid mutation from user code affecting the internal state.
(cherry picked from commit cae853e3b4)
gh-123930: Correct test of attribute failure to account for iOS (GH-125959)
Update a test of importing attributes from binary modules to account for iOS conditions.
(cherry picked from commit 75401febc9)
Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
gh-125942: Android: set stdout to `errors="backslashreplace"` (GH-125943)
Android stdout/err streams now use `backslashreplace` encoding to ensure readability of the Android log.
(cherry picked from commit b08570c90e)
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Miss Islington (bot)