Commit Graph

33245 Commits

Author SHA1 Message Date
Tian Gao 8207454bc0
gh-124703: Add extra checks for pdb quit test (#130286) 2025-02-18 16:55:39 -05:00
Sergey Miryanov bcc9a5dddb
gh-129515: Clarify syntax error messages for conditional expressions (#129880)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-02-18 21:43:19 +00:00
Tian Gao 076300d795
gh-126944: Show explicit errors when required arguments of pdb commands are missing (#130240) 2025-02-18 10:58:15 -05:00
donBarbos 427dd10250
gh-127260: Improve error consistency in both `fromisoformat` implementations (#130134)
In the Python implementation, "Z" was allowed where only "+" or "-" should be allowed in time zone specifiers. In the C implementation, ":" was allowed as a separator between the whole and fractional portion of times (seconds). These have both been forbidden and the error messages harmonized.
2025-02-18 15:49:28 +00:00
Jacob Walls dab456dcef
gh-130164: Fix inspect.Signature.bind() handling of positional-only args without defaults (GH-130192)
Follow-up to 9c15202.
2025-02-18 17:19:04 +02:00
Stan Ulbrych 01ba7df499
gh-44827: Improve error if BOM on first line of .po file (GH-130187) 2025-02-18 15:59:34 +02:00
Kumar Aditya 2e3e65380b
improve `test_tasks` to use correct idiom for starting task in asyncio (#130257)
The test should use the correct idiom for starting the task, `loop._run_once` is private API which should not be used directly, instead use `asyncio.sleep(0)` for 1 event loop cycle.
2025-02-18 12:08:25 +00:00
Serhiy Storchaka b93b7e566e
gh-130230: Fix crash in pow() with only Decimal third argument (GH-130237) 2025-02-18 09:46:48 +02:00
Barney Gale 6f07016bf0
GH-127381: pathlib ABCs: remove `ReadablePath.rglob()` (#130207)
Remove `ReadablePath.rglob()` from the private pathlib ABCs. This method is
a trivial wrapper around `glob()` and easily replaced.
2025-02-17 19:15:59 +00:00
Barney Gale 7fcace99bb
GH-125413: Add private metadata methods to `pathlib.Path.info` (#129897)
Add the following private methods to `pathlib.Path.info`:

- `_posix_permissions()`: the POSIX file permissions (`S_IMODE(st_mode)`)
- `_file_id()`: the file ID (`(st_dev, st_ino)`)
- `_access_time_ns()`: the access time in nanoseconds (`st_atime_ns`)
- `_mod_time_ns()`: the modify time in nanoseconds (`st_mtime_ns`)
- `_bsd_flags()`: the BSD file flags (`st_flags`)
- `_xattrs()`: the file extended attributes as a list of key, value pairs,
  or an empty list if `listxattr()` or `getxattr()` fail in an ignorable 
  way.

These methods replace `LocalCopyReader.read_metadata()`, and so we can
delete the `CopyReader` and `LocalCopyReader` classes. Rather than reading
metadata via `source._copy_reader.read_metadata()`, we instead call
`source.info._posix_permissions()`, `_access_time_ns()`, etc.

Preserving metadata is only supported for local-to-local copies at the
moment. To support copying metadata between arbitrary `ReadablePath` and
`WritablePath` objects, we'd need to make the new methods public and
documented.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2025-02-17 19:15:25 +00:00
Andrey Efremov ef8eeca9d8
gh-129678: ConfigParser: do not write an empty unnamed section (GH-129679)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2025-02-17 14:24:57 +01:00
Tomas R. aa845af9bb
gh-130057: Pygettext: Support translator comments (GH-130061) 2025-02-17 12:41:28 +02:00
Serhiy Storchaka 395335d0ff
gh-127750: Fix and optimize functools.singledispatchmethod() (GH-130008)
Remove broken singledispatchmethod caching introduced in gh-85160.
Achieve the same performance using different optimization.

* Add more tests.

* Fix issues with __module__ and __doc__ descriptors.
2025-02-17 11:11:20 +02:00
Tomas R. fb2d325725
gh-130193: Increase test coverage of gettext.c2py (GH-130208) 2025-02-17 10:57:34 +02:00
Serhiy Storchaka cfe41037eb
gh-86069: Add more PyNumber_InPlacePower() tests (GH-130111)
Test it with the third argument.
2025-02-17 00:07:10 +00:00
Tomas R. 73d03005b0
gh-130185: Fix unintentionally skipped tests in `test_functools` (#130186) 2025-02-16 16:34:54 +03:00
Irit Katriel c9b1bf302c
gh-130139: always check ast node type in ast.parse() with ast input (#130140) 2025-02-16 13:32:39 +00:00
Tomas R. 2e8044a4f7
gh-130177: Fix unintentionally skipped tests in test_gettext (GH-130178) 2025-02-16 14:14:37 +02:00
Tomas R. 361083b84b
gh-127488: Add tests for Tools/i18n/msgfmt.py (GH-127540) 2025-02-16 14:05:01 +02:00
AN Long 798f8d3ea9
Replace non-breaking spaces with normal spaces (#130116)
Using normal spaces in place of non-breaking spaces.
2025-02-16 09:33:14 +08:00
Barney Gale a7d41a8947
GH-128520: Subclass `abc.ABC` in `pathlib._abc` (#128745)
Convert `JoinablePath`, `ReadablePath` and `WritablePath` to real ABCs
derived from `abc.ABC`.

Make `JoinablePath.parser` abstract, rather than defaulting to `posixpath`.

Register `PurePath` and `Path` as virtual subclasses of the ABCs rather
than deriving. This avoids a hit to path object instantiation performance.

No change of behaviour in the public (non-abstract) classes.
2025-02-16 00:37:26 +00:00
Ken Jin 359c7dde3b
gh-129989: Properly disable tailcall interp in configure (GH-129991)
Co-authored-by: Zanie Blue <contact@zanie.dev>
2025-02-16 03:01:24 +08:00
Kumar Aditya a545749b0e
gh-130145: fix `loop.run_forever` when loop is already running (#130146) 2025-02-15 15:01:53 +05:30
Tomasz Pytel a05433f24a
gh-129107: make `bytearray` thread safe (#129108)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-02-15 07:19:42 +00:00
Sam Gross 39cd9728a6
gh-130095: Mark `_py_abc` tests as not thread-safe (gh-130131)
The `_py_abc` implementation is not currently thread-safe (even with the GIL).
Don't run these tests with `--parallel-threads=N` for now.
2025-02-14 16:21:45 -05:00
Stan Ulbrych 3402e133ef
gh-82045: Correct and deduplicate "isprintable" docs; add test. (GH-130118)
We had the definition of what makes a character "printable" documented in three places, giving two different definitions.
The definition in the comment on `_PyUnicode_IsPrintable` was inverted; correct that.

With that correction, the two definitions turn out to be equivalent -- but to confirm that, you have to go look up, or happen to know, that those are the only five "Other" categories and only three "Separator" categories in the Unicode character database.  That makes it hard for the reader to tell whether they really are the same, or if there's some subtle difference in the intended semantics.

Fix that by cutting the C API docs' and the C comment's copies of the subtle details, in favor of referring to the Python-level docs. That ensures it's explicit that these are all meant to agree, and also lets us concentrate improvements to the wording in one place.

Speaking of which, borrow some ideas from the C comment, along with other tweaks, to hopefully add a bit more clarity to that one newly-centralized copy in the docs.

Also add a thorough test that the implementation agrees with this definition.

Author:    Greg Price <gnprice@gmail.com>

Co-authored-by: Greg Price <gnprice@gmail.com>
2025-02-14 18:16:47 +01:00
Yan Yanchii 334589f619
gh-126835: Set location for noped out instructions after constant folding in CFG. (#130109) 2025-02-14 14:15:08 +00:00
Tomas R. 0f20281fa2
gh-107510: gettext: Remove unnecessary tests (GH-127965)
There are now separate pygettext tests.
2025-02-14 12:16:10 +00:00
Tomas R. 9d1e668e6f
gh-129911: pygettext: Fix the keyword entry in help output (GH-129914) 2025-02-14 11:34:09 +02:00
Pablo Galindo Salgado 3bd3e09588
gh-125331: Allow the parser to activate future imports on the fly (#125482) 2025-02-14 04:54:56 +00:00
Filipe Laíns 🇵🇸 1eb3ade6e5
GH-107956: install build-details.json (PEP 739) (#130069) 2025-02-13 13:58:00 +00:00
Yan Yanchii 140e69c4a8
gh-126835: Move const folding of lists & sets from ast_opt.c to flowgraph.c (#130032) 2025-02-13 12:11:07 +00:00
Ammar Askar f9a7d41bac
gh-96092: Fix traceback.walk_stack(None) skipping too many frames (#129330)
As it says in its documentation, walk_stack was meant to just
follow `f.f_back` like other functions in the traceback module.
Instead it was previously doing `f.f_back.f_back` and then this
changed to `f_back.f_back.f_back.f_back' in Python 3.11 breaking
its behavior for external users.

This happened because the walk_stack function never really had
any good direct tests and its only consumer in the traceback module was
`extract_stack` which passed the result into `StackSummary.extract`.
As a generator, it was previously capturing the state of the stack
when it was first iterated over, rather than the stack when `walk_stack`
was called. Meaning when called inside the two method deep
`extract` and `extract_stack` calls, two `f_back`s were needed.
When 3.11 modified the sequence of calls in `extract`, two more
`f_back`s were needed to make the tests happy.

This changes the generator to capture the stack when `walk_stack` is
called, rather than when it is first iterated over. Since this is
technically a breaking change in behavior, there is a versionchanged
to the documentation. In practice, this is unlikely to break anyone,
you would have been needing to store the result of `walk_stack` and
expecting it to change.
2025-02-13 01:43:09 +00:00
Pablo Galindo Salgado 6fb5138776
gh-88535: Improve syntax error for wrongly closed strings (#26633) 2025-02-13 01:30:20 +00:00
Pablo Galindo Salgado 56eda25633
gh-116042: Fix location for SyntaxErrors of invalid escapes in the tokenizer (#116049) 2025-02-13 01:07:37 +00:00
Tomas R. 2dd018848c
gh-129693: Suppress `SyntaxWarning` in test_fstring (#129830)
Suppress SyntaxWarning in test_fstring
2025-02-13 00:38:28 +00:00
Mark Shannon 72f56654d0
GH-128682: Account for escapes in `DECREF_INPUTS` (GH-129953)
* Handle escapes in DECREF_INPUTS

* Mark a few more functions as escaping

* Replace DECREF_INPUTS with PyStackRef_CLOSE where possible
2025-02-12 17:44:59 +00:00
donBarbos 3e222e3a15
gh-109798: Normalize `_datetime` and `datetime` error messages (#127345)
Updates error messages in datetime and makes them consistent between Python and C.

---------

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com>
2025-02-12 09:54:22 -05:00
Andrew Svetlov 469d2e416c
gh-129889: Support context manager protocol by contextvars.Token (#129888) 2025-02-12 12:32:58 +01:00
Sam Gross a7427f2db9
gh-129967: Fix race condition in `repr(set)` (gh-129978)
The call to `PySequence_List()` could temporarily unlock and relock the
set, allowing the items to be cleared and return the incorrect
notation `{}` for a empty set (it should be `set()`).

Co-authored-by: T. Wouters <thomas@python.org>
2025-02-11 17:29:27 -05:00
Sam Gross 1f233f56d6
gh-128657: Skip test_get_builtin_constructor when running with --parallel-threads (GH-130018)
The test modifies sys.modules and is not thread-safe.
2025-02-11 16:59:36 -05:00
Sam Gross 1a8082a4bf
gh-117657: Add test_thread_local_bytecode to TSAN tests (gh-129753)
Skip `test_no_copies_if_tlbc_disabled` when run under TSAN for now
due to a data race on the adaptive counter (see gh-129752).
2025-02-11 16:54:32 -05:00
Sam Gross f151d27159
gh-117657: Enable test_opcache under TSAN (GH-129831)
Fix a few thread-safety bugs to enable test_opcache when run with TSAN:

 * Use relaxed atomics when clearing `ht->_spec_cache.getitem`
   (gh-115999)
 * Add temporary suppression for type slot modifications (gh-127266)
 * Use atomic load when reading `*dictptr`
2025-02-11 16:53:08 -05:00
Sam Gross 3b548adc76
gh-129980: Include test name in TSAN filename in multiprocess test runner (GH-129981) 2025-02-11 12:09:43 -05:00
Tomas R. 374abded07
gh-104400: pygettext: use an AST parser instead of a tokenizer (GH-104402)
This greatly simplifies the code and fixes many corner cases.
2025-02-11 13:51:42 +02:00
Wulian233 12bd15f7b3
gh-129939: Add darkmode support for difflib's comparison pages (#129940) 2025-02-11 10:58:57 +02:00
Erlend E. Aasland 3a2e7aacf6
gh-129928: Raise more accurate exception for incorrect sqlite3 UDF creation (#129941)
Consistently raise ProgrammingError if the user tries to create an UDF
with an invalid number of parameters.
2025-02-11 08:26:01 +01:00
Kumar Aditya 94cd2e0dde
gh-129289: fix crash when task finalizer is not called in asyncio (#129840) 2025-02-10 17:03:59 +05:30
Peter Marko 978211c8a8
gh-126554: correct detection of `gcc` for `TestNullDlsym.test_null_dlsym` (GH-129872)
In case gcc is not available, the test will fail with FileNotFoundError.
So catch the exception to skip the test correctly.

Signed-off-by: Peter Marko <peter.marko@siemens.com>
2025-02-10 10:51:56 +01:00
Erlend E. Aasland 7e6ee50b6b
gh-129603: Don't segfault if sqlite3.Row description is None (#129604) 2025-02-10 00:27:28 +01:00
Erlend E. Aasland cda83cade0
gh-129870: Skip test_dump_virtual_tables if SQLite lacks FTS4 support (#129913) 2025-02-09 21:43:23 +01:00
Yan Yanchii 91d9544112
gh-126835: Make CFG optimizer skip over NOP's when looking for const sequence construction (#129703)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2025-02-09 18:00:41 +00:00
Kumar Aditya f7c7decc4c
gh-129874: improve test_events to use correct task implementation (#129891) 2025-02-09 18:36:16 +05:30
Kumar Aditya ce0cf7a73a
gh-129874: improve `test_tasks` in asyncio to use correct internal functions (#129890) 2025-02-09 13:02:11 +00:00
Kumar Aditya 09fe550ecc
gh-129874: improve asyncio tests to use correct internal functions (#129887) 2025-02-09 12:05:39 +00:00
Tian Gao 29f8a67ae0
Provide curframe_locals for backward compatibility but deprecate it (#125951) 2025-02-08 23:35:28 +00:00
Sam Gross c1f352bf08
gh-128657: Run test_hashlib with `--parallel-threads` (GH-129833)
* gh-128657: Run test_hashlib with `--parallel-threads`

This catches the race in `py_digest_by_name` that is fixed separately
in gh-128886.

* Adjust assertion order
2025-02-08 12:12:21 -08:00
Brian Ward 421ea1291d
gh-119349: Add ctypes.util.dllist -- list loaded shared libraries (GH-122946)
Add function to list the currently loaded libraries to ctypes.util

The dllist() function calls platform-specific APIs in order to
list the runtime libraries loaded by Python and any imported modules.
On unsupported platforms the function may be missing.


Co-authored-by: Eryk Sun <eryksun@gmail.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2025-02-08 14:02:36 +01:00
Barney Gale 707d066193
GH-129835: Yield path with trailing slash from `ReadablePath.glob('')` (#129836)
In the private pathlib ABCs, make `ReadablePath.glob('')` yield a path with
a trailing slash (if it yields anything at all). As a result, `glob()`
works similarly to `joinpath()` when given a non-magic pattern.

In the globbing implementation, we preemptively add trailing slashes to
intermediate paths if there are pattern parts remaining; this removes the
need to check for existing trailing slashes (in the removed `add_slash()`
method) at subsequent steps.
2025-02-08 06:47:09 +00:00
Barney Gale 718ab66299
GH-125413: Add `pathlib.Path.info` attribute (#127730)
Add `pathlib.Path.info` attribute, which stores an object implementing the `pathlib.types.PathInfo` protocol (also new). The object supports querying the file type and internally caching `os.stat()` results. Path objects generated by `Path.iterdir()` are initialised with status information from `os.DirEntry` objects, which is gleaned from scanning the parent directory.

The `PathInfo` protocol has four methods: `exists()`, `is_dir()`, `is_file()` and `is_symlink()`.
2025-02-08 01:16:45 +00:00
Irit Katriel a1417b211f
gh-100239: replace BINARY_SUBSCR & family by BINARY_OP with oparg NB_SUBSCR (#129700) 2025-02-07 22:39:54 +00:00
Sam Gross 2248a9c153
gh-129825: Skip test_faulthandler.test_register_chain under TSAN (gh-129827)
The test hangs when run under TSAN due to an interaction between TSAN's
signal interception and our attempt to call the previous signal handler.
2025-02-07 17:35:59 -05:00
Brandt Bucher 5fa7e1b7fd
GH-129715: Remove _DYNAMIC_EXIT (GH-129716) 2025-02-07 11:41:17 -08:00
Kumar Aditya 49bd47d5f1
improve `test_log_destroyed_pending_task` in asyncio (#129821) 2025-02-07 18:00:59 +00:00
Kumar Aditya f52a3a51eb
fix tests in `test_asyncio.test_tasks` to use correct `all_tasks` (#129815) 2025-02-07 21:46:26 +05:30
Ken Jin 175844713a
gh-128563: Move assignment of opcode into ifdef (GH-129803) 2025-02-07 21:11:57 +08:00
sobolevn ae132edc29
gh-129766: Fix crash on calling `warnings._release_lock` with no lock (#129771) 2025-02-07 12:59:52 +03:00
Forest 0fef47e5bb
gh-55454: Add IMAP4 IDLE support to imaplib (#122542)
* gh-55454: Add IMAP4 IDLE support to imaplib

This extends imaplib with support for the rfc2177 IMAP IDLE command,
as requested in #55454.  It allows events to be pushed to a client as
they occur, rather than having to continually poll for mailbox changes.

The interface is a new idle() method, which returns an iterable context
manager.  Entering the context starts IDLE mode, during which events
(untagged responses) can be retrieved using the iteration protocol.
Exiting the context sends DONE to the server, ending IDLE mode.

An optional time limit for the IDLE session is supported, for use with
servers that impose an inactivity timeout.

The context manager also offers a burst() method, designed for programs
wishing to process events in batch rather than one at a time.

Notable differences from other implementations:

- It's an extension to imaplib, rather than a replacement.
- It doesn't introduce additional threads.
- It doesn't impose new requirements on the use of imaplib's existing methods.
- It passes the unit tests in CPython's test/test_imaplib.py module
  (and adds new ones).
- It works on Windows, Linux, and other unix-like systems.
- It makes IDLE available on all of imaplib's client variants
  (including IMAP4_stream).
- The interface is pythonic and easy to use.

Caveats:

- Due to a Windows limitation, the special case of IMAP4_stream running
  on Windows lacks a duration/timeout feature. (This is the stdin/stdout
  pipe connection variant; timeouts work fine for socket-based
  connections, even on Windows.) I have documented it where appropriate.

- The file-like imaplib instance attributes are changed from buffered to
  unbuffered mode. This could potentially break any client code that
  uses those objects directly without expecting partial reads/writes.
  However, these attributes are undocumented. As such, I think (and
  PEP 8 confirms) that they are fair game for changes.
  https://peps.python.org/pep-0008/#public-and-internal-interfaces

Usage examples:

https://github.com/python/cpython/issues/55454#issuecomment-2227543041

Original discussion:

https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272

Earlier requests and suggestions:

https://github.com/python/cpython/issues/55454

https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/

* gh-55454: Clarify imaplib idle() docs

- Add example idle response tuples, to make the minor difference from other
  imaplib response tuples more obvious.
- Merge the idle context manager's burst() method docs with the IMAP
  object's idle() method docs, for easier understanding.
- Upgrade the Windows note regarding lack of pipe timeouts to a warning.
- Rephrase various things for clarity.

* docs: words instead of <=

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* docs: improve style in an example

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* docs: grammatical edit

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* docs consistency

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* comment -> docstring

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* docs: refer to imaplib as "this module"

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* imaplib: simplify & clarify idle debug message

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* imaplib: elaborate in idle context manager comment

* imaplib: re-raise BaseException instead of bare except

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* imaplib: convert private doc string to comment

* docs: correct mistake in imaplib example

This is a correction to 8077f2eab2, which
changed a variable name in only one place and broke the subsequent
reference to it, departed from the naming convention used in the rest of
the module, and shadowed the type() builtin along the way.

* imaplib: simplify example code in doc string

This is for consistency with the documentation change in 8077f2eab2
and subsequent correction in 013bbf18fc.

* imaplib: rename _Idler to Idler, update its docs

* imaplib: add comment in Idler._pop()

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* imaplib: remove unnecessary blank line

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* imaplib: comment on use of unbuffered pipes

* docs: imaplib: use the reStructuredText :class: role

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* Revert "docs: imaplib: use the reStructuredText :class: role"

This reverts commit f385e441df, because it
triggers CI failures in the docs by referencing a class that is
(deliberately) undocumented.

* docs: imaplib: use the reST :class: role, escaped

This is a different approach to f385e441df, which was reverted for
creating dangling link references.

By prefixing the reStructuredText role target with a ! we disable
conversion to a link, thereby passing continuous integration checks
even though the referenced class is deliberately absent from the
documentation.

* docs: refer to IMAP4 IDLE instead of just IDLE

This clarifies that we are referring to the email protocol, not the editor with the same name.

Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>

* imaplib: IDLE -> IMAP4 IDLE in exception message

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* docs: imaplib idle() phrasing and linking tweaks

* docs: imaplib: avoid linking to an invalid target

This reverts and rephrases part of a3f21cd75b
which created links to a method on a deliberately undocumented class.
The links didn't work consistently, and caused sphinx warnings that
broke cpython's continuous integration tests.

* imaplib: update test after recent exception change

This fixes a test that was broken by changing an exception in
b01de95171

* imaplib: rename idle() dur argument to duration

* imaplib: bytes.index() -> bytes.find()

This makes it more obvious which statement triggers the branch.

* imaplib: remove no-longer-necessary statement

Co-authored-by: Martin Panter <vadmium@users.noreply.github.com>

* docs: imaplib: concise & valid method links

The burst() method is a little tricky to link in restructuredText, due
to quirks of its parent class.  This syntax allows sphinx to generate
working links without generating warnings (which break continuous
integration) and without burdening the reader with unimportant namespace
qualifications.  It makes the reST source ugly, but few people read
the reST source, so it's a tolerable tradeoff.

* imaplib: note data types present in IDLE responses

* docs: imaplib: add comma to reST changes header

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

* imaplib: sync doc strings with reST docs

* docs: imaplib: minor Idler clarifications

* imaplib: idle: emit (type, [data, ...]) tuples

This allows our iterator to emit untagged responses that contain literal
strings in the same way that imaplib's existing methods do, while still
emitting exactly one whole response per iteration.

* imaplib: while/yield instead of yield from iter()

* imaplib: idle: use deadline idiom when iterating

This simplifies the code, and avoids idle duration drift from time spent
processing each iteration.

* docs: imaplib: state duration/interval arg types

* docs: imaplib: minor rephrasing of a sentence

* docs: imaplib: reposition a paragraph

This might improve readability, especially when encountering Idler.burst()
for the first time.

* docs: imaplib: wrap long lines in idle() section

* docs: imaplib: note: Idler objects require 'with'

* docs: imaplib: say that 29 minutes is 1740 seconds

* docs: imaplib: mark a paragraph as a 'tip'

* docs: imaplib: rephrase reference to MS Windows

* imaplib: end doc string titles with a period

* imaplib: idle: socket timeouts instead of select()

IDLE timeouts were originally implemented using select() after
checking for the presence of already-buffered data.
That allowed timeouts on pipe connetions like IMAP4_stream.
However, it seemed possible that SSL data arriving without any
IMAP data afterward could cause select() to indicate available
application data when there was none, leading to a read() call
that would block with no timeout. It was unclear under what
conditions this would happen in practice. This change switches
to socket timeouts instead of select(), just to be safe.

This also reverts IMAP4_stream changes that were made to support IDLE
timeouts, since our new implementation only supports socket connections.

* imaplib: Idler: rename private state attributes

* imaplib: rephrase a comment in example code

* docs: imaplib: idle: use Sphinx code-block:: pycon

* docs: whatsnew: imaplib: reformat IMAP4.idle entry

* imaplib: idle: make doc strings brief

Since we generally rely on the reST/html documentation for details, we
can keep these doc strings short. This matches the module's existing doc
string style and avoids having to sync small changes between two files.

* imaplib: Idler: split assert into two statements

* imaplib: Idler: move assignment out of try: block

* imaplib: Idler: move __exit__() for readability

* imaplib: Idler: move __next__() for readability

* imaplib: test: make IdleCmdHandler a global class

* docs: imaplib: idle: collapse double-spaces

* imaplib: warn on use of undocumented 'file' attr

* imaplib: revert import reformatting

Since we no longer import platform or selectors, the original import
statement style can be restored, reducing the footprint of PR #122542.

* imaplib: restore original exception msg formatting

This reduces the footprint of PR #122542.

* docs: imaplib: idle: versionadded:: next

* imaplib: move import statement to where it's used

This import is only needed if external code tries to use an attribute
that it shouldn't be using. Making it a local import reduces module
loading time in supported cases.

* imaplib test: RuntimeWarning on IMAP4.file access

* imaplib: use stacklevel=2 in warnings.warn()

* imaplib test: simplify IMAP4.file warning test

* imaplib test: pre-idle-continuation response

* imaplib test: post-done untagged response

* imaplib: downgrade idle-denied exception to error

This makes it easier for client code to distinguish a temporary
rejection of the IDLE command from a server responding incorrectly to
IDLE.

* imaplib: simplify check for socket object

* imaplib: narrow the scope of IDLE socket timeouts

If an IDLE duration or burst() was in use, and an unsolicited response
contained a literal string, and crossed a packet boundary, and the
subsequent packet was delayed beyond the IDLE feature's time limit, the
timeout would leave the incoming protocol stream in a bad state (with
the tail of that response appearing where the start of a response is
expected).

This change moves the IDLE socket timeout to cover only the start
of a response, so it can no longer cause that problem.

* imaplib: preserve partial reads on exception

This ensures that short IDLE durations / burst() intervals
won't risk corrupting response lines that span multiple packets.

* imaplib: read/readline: save multipart buffer tail

For resilience if read() or readline() ever complete with more than one
bytes object remaining in the buffer. This is not expected to happen,
but it seems wise to be prepared for a future change making it possible.

* imaplib: use TimeoutError subclass only if needed

* doc: imaplib: elaborate on IDLE response delivery

* doc: imaplib: elaborate in note re: IMAP4.response

* imaplib: comment on benefit of reading in chunks

Our read() implementation designed to support IDLE replaces the one from
PR #119514, fixing the same problem it was addressing. The tests that it
added are preserved.

* imaplib: readline(): treat ConnectionError as EOF

---------

Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
Co-authored-by: Martin Panter <vadmium@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-02-06 19:15:11 -08:00
Sam Gross a191d6f78e
gh-117657: Include all of test_free_threading in TSAN tests (#129749) 2025-02-07 00:37:05 +01:00
Sam Gross 4e3330f054
gh-117657: Skip tests that may cause stack overflows under TSan (#129751)
These tests crash under TSan due to stack overflows. Just skip them if
TSan is enabled.
2025-02-06 17:03:01 -05:00
Kumar Aditya 0d68b14a0d
gh-128002: use per threads tasks linked list in asyncio (#128869)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2025-02-06 19:51:07 +01:00
Ken Jin cb640b659e
gh-128563: A new tail-calling interpreter (GH-128718)
Co-authored-by: Garrett Gu <garrettgu777@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2025-02-06 23:21:57 +08:00
Sam Gross 555dc50c81
gh-129694: Add `--parallel-threads` TSAN job to CI (gh-129696)
For now, this just adds a single test suite to the TSAN CI to be run
with `--parallel-threads`.
2025-02-06 15:19:00 +00:00
Xuanteng Huang 55f17b77c3
gh-128714: Fix function object races in `__annotate__`, `__annotations__` and `__type_params__` in free-threading build (#129016) 2025-02-06 20:10:50 +05:30
Cody Maloney 5fb019fc29
gh-129559: Add `bytearray.resize()` (GH-129560)
Add bytearray.resize() which wraps PyByteArray_Resize.

Make negative size passed to resize exception/error rather than crash in optimized builds.
2025-02-05 11:33:17 -08:00
Victor Stinner e5c3b7e349
gh-129354: Cleanup test_coroutines (#129684)
Remove unused variables.
2025-02-05 11:43:46 +00:00
Victor Stinner a25042e6d2
gh-129354: Use PyErr_FormatUnraisable() function (#129523)
Replace PyErr_WriteUnraisable() with PyErr_FormatUnraisable().

Update tests:

* test_coroutines
* test_exceptions
* test_generators
* test_struct
2025-02-05 10:31:59 +00:00
Sam Gross e5f10a7414
gh-127933: Add option to run regression tests in parallel (gh-128003)
This adds a new command line argument, `--parallel-threads` to the
regression test runner to allow it to run individual tests in multiple
threads in parallel in order to find multithreading bugs.

Some tests pass when run with `--parallel-threads`, but there's still
more work before the entire suite passes.
2025-02-04 17:44:59 -05:00
Serhiy Storchaka 979d766209
gh-128772: Fix pydoc for methods with __module__ is None (GH-129177) 2025-02-04 16:25:49 +02:00
Serhiy Storchaka 8b5c8508c7
gh-129350: Make tests for glob with trailing slash more strict (GH-129376)
Test that the trailing pathname separator is preserved.

Multiple trailing pathname separators are only preserved if the pattern
does not contain metacharacters, otherwise only one trailing pathname
separator is preserved. This is rather an implementation detail.
2025-02-04 16:24:22 +02:00
Mark Shannon 2effea4dab
GH-128682: Spill the stack pointer in labels, as well as instructions (GH-129618) 2025-02-04 12:18:31 +00:00
Yan Yanchii 0664c1af9b
gh-126835: Move constant subscript folding to CFG (#129568)
Move folding of constant subscription from AST optimizer to CFG.

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2025-02-04 10:10:55 +02:00
Mark Shannon 808071b994
GH-128682: Make `PyStackRef_CLOSE` escaping. (GH-129404) 2025-02-03 12:41:32 +00:00
Victor Stinner 218f205f20
gh-129354: Use PyErr_FormatUnraisable() function (#129524)
Replace PyErr_WriteUnraisable() with PyErr_FormatUnraisable().

Update test_sqlite3 tests.

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2025-02-03 12:10:18 +00:00
Illia Volochii a33dcb9e43
gh-81340: Use copy_file_range in shutil.copyfile copy functions (GH-93152)
This allows the underlying file system an opportunity to optimise or avoid the actual copy.
2025-02-03 10:23:27 +00:00
Vojtěch Boček 4e38eeafe2
gh-115514: Fix incomplete writes after close while using ssl in asyncio(#128037)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-02-02 15:11:25 +00:00
Gregory P. Smith e1006ce1de
gh-119461: Restore the testSocket VSOCK skipUnless removed by PR #119465 (#129561)
Restore the skipUnless removed by #119465.

This test can only pass on virtual machines, not actual machines.

actual machines see:

```
    self.cli.connect((cid, VSOCKPORT))
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
OSError: [Errno 19] No such device
```

Reproduced on (Linux) Ubuntu 24.04.1 running 6.8.0-52-generic.
2025-02-01 20:46:56 -08:00
Victor Stinner 71ae93374d
gh-93649: Move _testcapi tests to specific files (#129544)
Move many functions from _testcapimodule.c into more specific files
in Modules/_testcapi/.

In moved code:

* Replace get_testerror() with PyExc_AssertionError.
* Replace raiseTestError() with
  PyErr_Format(PyExc_AssertionError, ...).
2025-02-01 13:39:16 +01:00
Kirill Podoprigora 7d0521d5fc
gh-126835: Move optimization of constant sequence creation from codegen to CFG (#129426)
Codegen phase has an optimization that transforms
```
LOAD_CONST x
LOAD_CONST y
LOAD_CONXT z
BUILD_LIST/BUILD_SET (3)
```
->
```
BUILD_LIST/BUILD_SET (0)
LOAD_CONST (x, y, z)
LIST_EXTEND/SET_UPDATE 1
```
This optimization has now been moved to CFG phase to make #128802 work.


Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Co-authored-by: Yan Yanchii <yyanchiy@gmail.com>
2025-02-01 11:39:44 +00:00
Seth Michael Larson d89a5f6a6e
gh-105704: Disallow square brackets (`[` and `]`) in domain names for parsed URLs (#129418)
* gh-105704: Disallow square brackets ( and ) in domain names for parsed URLs

* Use Sphinx references

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

* Add mismatched bracket test cases, fix news format

* Add more test coverage for ports

---------

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2025-01-31 09:41:34 -08:00
Victor Stinner 60a85415ae
gh-93649: Add Modules/_testcapi/function.c file (#129521)
* Move PyFunction C API tests to a new file.
* Add Lib/test/test_capi/test_function.py.
* Move tests from test_capi.test_misc to test_capi.test_function.
2025-01-31 16:02:50 +01:00
Victor Stinner 5424e3b034
gh-93649: Add Modules/_testcapi/type.c file (#129516)
Move PyType C API tests to a new file.

Move following tests from test_capi.test_misc to test_capi.test_type:

* BuiltinStaticTypesTests
* test_get_type_name()
* test_get_base_by_token()
2025-01-31 15:03:54 +01:00
Victor Stinner 8b70ff5872
gh-93649: Move PyFrame C API tests to test_capi (#129512)
* Add Lib/test/test_capi/test_frame.py file.
* Move C API tests from test_frame to test_capi.test_frame.
* Add Modules/_testcapi/frame.c file.
* Move C API tests from _testcapimodule.c to frame.c
2025-01-31 13:51:58 +01:00
qm2k 22b2d37f42
gh-71494: string.Formatter: support keys/attributes in unnumbered fields (GH-21767) 2025-01-31 13:16:24 +01:00
Petr Viktorin 31c82c28f9
GH-128469: Revert "warn when libpython was loaded from outside the build directory (#128645)" (#129506) 2025-01-31 11:17:37 +00:00
Brandt Bucher 674befbd7b
GH-129386: Add `test.support.reset_code` (GH-129486) 2025-01-31 11:50:54 +01:00
Shantanu 8df5193d37
gh-127975: Avoid reusing quote types in ast.unparse if not needed (#127980) 2025-01-31 08:49:06 +00:00
Victor Stinner 95504f429e
gh-129354: Fix grammar in PyErr_FormatUnraisable() (#129475)
Replace "on verb+ing" with "while verb+ing".
2025-01-31 09:45:35 +01:00
donBarbos 510fefdc62
gh-127349: Add check for correct resizing in REPL (#127387) 2025-01-30 19:34:09 +00:00
Victor Stinner 4ca9fc08f8
gh-111495: Add PyFile tests (#129449)
Add tests for the following functions in test_capi.test_file:

* PyFile_FromFd()
* PyFile_GetLine()
* PyFile_NewStdPrinter()
* PyFile_WriteObject()
* PyFile_WriteString()
* PyObject_AsFileDescriptor()

Add Modules/_testlimitedcapi/file.c file.

Remove test_embed.StdPrinterTests which became redundant.
2025-01-30 18:05:32 +01:00
Victor Stinner 3bebe46d34
gh-128911: Add PyImport_ImportModuleAttr() function (#128912)
Add PyImport_ImportModuleAttr() and
PyImport_ImportModuleAttrString() functions.

* Add unit tests.
* Replace _PyImport_GetModuleAttr()
  with PyImport_ImportModuleAttr().
* Replace _PyImport_GetModuleAttrString()
  with PyImport_ImportModuleAttrString().
* Remove "pycore_import.h" includes, no longer needed.
2025-01-30 11:17:29 +00:00
Filipe Laíns 🇵🇸 a4722449ca
tests: add test.support.venv.VirtualEnvironmentMixin (#129461) 2025-01-30 03:32:24 +00:00
Filipe Laíns 🇵🇸 c931d75831
GH-127178: improve compatibility in `_sysconfig_vars_(...).json` (#128558) 2025-01-29 22:47:20 +00:00
Filipe Laíns 🇵🇸 29b3ce8355
GH-128469: warn when libpython was loaded from outside the build directory (#128645)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2025-01-29 22:35:55 +00:00
Tomas R. a1a4e9f39a
gh-125522: Fix bare except in `test_uuid` (#129018) 2025-01-29 20:59:45 +00:00
Hod fdcedfd3cf
gh-126400: Add TCP socket timeout to SysLogHandler to prevent blocking (GH-126716)
Co-authored-by: Vinay Sajip <vinay_sajip@yahoo.co.uk>
2025-01-29 19:37:43 +00:00
Brandt Bucher 002c4e2982
GH-129386: Use symbolic constants for specialization tests (GH-129415) 2025-01-29 10:49:58 -08:00
Irit Katriel 4815131910
gh-100239: specialize bitwise logical binary ops on ints (#128927) 2025-01-29 09:28:21 +00:00
Brandt Bucher 828b27680f
GH-126599: Remove the PyOptimizer API (GH-129194) 2025-01-28 16:10:51 -08:00
T. Wouters 5c930a26fb
gh-115999: Enable free-threaded specialization of LOAD_CONST (#129365)
Enable free-threaded specialization of LOAD_CONST.
2025-01-29 01:07:56 +01:00
Hood Chatham 789390872b
gh-127146: Skip test_readinto_non_blocking on Emscripten (#129421)
Skips an additional test due to non_blocking not working reliably on Emscripten.
2025-01-29 07:30:42 +08:00
Pieter Eendebak 64c417dee5
gh-112075: Remove critical section in dict.get (gh-129336)
The `dict.get` implementation uses `_Py_dict_lookup_threadsafe`, which is
thread-safe, so we remove the critical section from the argument clinic.

Add a test for concurrent dict get and set operations.
2025-01-28 21:55:45 +00:00
Barney Gale a4459c34ea
GH-127381: pathlib ABCs: remove `JoinablePath.match()` (#129147)
Unlike `ReadablePath.[r]glob()` and `JoinablePath.full_match()`, the
`JoinablePath.match()` method doesn't support the recursive wildcard `**`,
and matches from the right when a fully relative pattern is given. These
quirks means its probably unsuitable for inclusion in the pathlib ABCs,
especially given `full_match()` handles the same use case.
2025-01-28 20:22:55 +00:00
Yury Manushkin 4d0d24f6e3
gh-112064: Fix incorrect handling of negative read sizes in `HTTPResponse.read()` (#128270)
The parameter `amt` of `HTTPResponse.read()`, which could be a negative integer,
has not been handled before and led to waiting for the connection to close
for `keep-alive connections`. Now, this has been fixed, and passing negative values
to `HTTPResponse().read()` works the same as passing `None` value.
2025-01-28 11:37:32 +00:00
Hood Chatham 8e57877e3f
gh-127146: Update test skips for Emscripten 4.0.1 (#129375)
Updates the Emscripten test skips to reflect recent changes in Emscripten capabilities.
2025-01-28 01:05:21 +00:00
Sam Gross a6a8c6f86e
gh-128954: Reorder _PyInterpreterFrame fields for reduced memory usage (#128958)
This reduces the size of _PyInterpreterFrame by 8 bytes on 64-bit
platforms using the free threading build due to alignment requirements.

This allows for slightly more recursive calls into the interpreter (from
C), but `test_call.test_super_deep` still crashes.
2025-01-27 17:14:51 +00:00
Victor Stinner 8a5a18a36e
gh-129205: Modernize test_eintr (#129316)
* Use f-string.
* Fix grammar: replace 'datas' with 'data' (and replace 'data' with
  'item').
* Remove unused variables: 'pid' and 'old_mask'.
* Factorize test_read() and test_readinto() code.

Co-authored-by: Cody Maloney <cmaloney@users.noreply.github.com>
2025-01-27 18:04:45 +01:00
Mark Shannon 75b4962157
GH-128914: Remove all but one conditional stack effects (GH-129226)
* Remove all 'if (0)' and 'if (1)' conditional stack effects

* Use array instead of conditional for BUILD_SLICE args

* Refactor LOAD_GLOBAL to use a common conditional uop

* Remove conditional stack effects from LOAD_ATTR specializations

* Replace conditional stack effects in LOAD_ATTR with a 0 or 1 sized array.

* Remove conditional stack effects from CALL_FUNCTION_EX
2025-01-27 16:24:48 +00:00
Nick Pope 8ec76d9034
gh-128427: Add `uuid.NIL` and `uuid.MAX` (#128429) 2025-01-27 18:17:17 +02:00
Hugo van Kemenade 9546fe2ef2
gh-129061: Fix `FORCE_COLOR` and `NO_COLOR` when empty strings (#129140) 2025-01-27 16:24:10 +02:00
Serhiy Storchaka 735f25c5e3
gh-119511: Fix a potential denial of service in imaplib (#119514)
The IMAP4 client could consume an arbitrary amount of memory when trying
to connect to a malicious server, because it read a "literal" data with a
single read(size) call, and BufferedReader.read() allocates the bytes
object of the specified size before reading. Now the IMAP4 client reads data
by chunks, therefore the amount of used memory is limited by the
amount of the data actually been sent by the server.

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2025-01-27 14:44:00 +01:00
Peter Bierma 3fb5f6eb9b
gh-128509: Add `PyUnstable_IsImmortal` for finding immortal objects (GH-129182)
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2025-01-27 14:36:33 +01:00
Ken Jin 87fb8b198c
gh-128563: Move labels in ceval.c to bytecodes.c (GH-129112) 2025-01-27 18:30:20 +08:00
Tian Gao 7d275611f6
gh-124703: Do not raise an exception when quitting pdb (#124704) 2025-01-26 22:29:16 -05:00
Alex Willmer a8dc6d6d44
gh-115911: Ignore PermissionError during import from cwd (#116131)
Ignore PermissionError when checking cwd during import

On macOS `getcwd(3)` can return EACCES if a path component isn't readable,
resulting in PermissionError. `PathFinder.find_spec()` now catches these and
ignores them - the same treatment as a missing/deleted cwd.

Introduces `test.support.os_helper.save_mode(path, ...)`, a context manager
that restores the mode of a path on exit.

This is allows finer control of exception handling and robust environment
restoration across platforms in `FinderTests.test_permission_error_cwd()`.

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Co-authored-by: Brett Cannon <brett@python.org>
2025-01-26 19:00:28 +00:00
Andrey Efremov 914c232e93
gh-127096: Do not recreate unnamed section on every read in ConfigParser (#127228)
* Do not recreate unnamed section on every read in ConfigParser

* Remove duplicate section creation code
2025-01-26 13:33:07 -05:00
Jason R. Coombs b543b32eff
gh-123987: Fix NotADirectoryError in NamespaceReader when sentinel present (#124018) 2025-01-26 16:23:54 +00:00
Thomas Grainger fccbfc40b5
gh-129195: use `future_add_to_awaited_by/future_discard_from_awaited_by` in `asyncio.staggered.staggered_race` (#129253)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-01-26 21:14:16 +05:30
Cody Maloney 1ed4487968
gh-129205: Add os.readinto() API for reading data into a caller provided buffer (#129211)
Add a new OS API which will read data directly into a caller provided
writeable buffer protocol object.

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2025-01-26 14:21:03 +01:00
Irit Katriel c39ae8922b
gh-128799: Add frame of except* to traceback when wrapping a naked exception (#128971) 2025-01-25 13:00:23 +00:00
Giles Copp 9abbb58e3f
gh-112713 : Add support for 'partitioned' attribute in http.cookies (GH-112714)
* Add support for 'partitioned' attribute in http.cookies

Co-authored-by: Giles Copp <gilesc@dropbox.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
2025-01-24 22:31:52 +00:00
Kumar Aditya e635bf2e49
gh-128002: fix `test_all_tasks_different_thread` in asyncio (#129267) 2025-01-24 23:10:24 +05:30
Serhiy Storchaka 7907203bc0
Improve tests for _colorize.can_colorize() (#129234)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2025-01-24 18:33:40 +02:00
Kumar Aditya 8e0b36006c
gh-128002: use `_PyObject_SetMaybeWeakref` when creating tasks in asyncio (#128885) 2025-01-24 21:12:56 +05:30
Thomas Grainger ec91e1c276
gh-128479: fix asyncio staggered race leaking tasks, and logging unhandled exception.append exception (#128475)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2025-01-23 16:53:53 +01:00
Bénédikt Tran cf0b2da1e6
gh-126004: Remove redundant safeguards for codecs handlers tests (#127680)
We remove the safeguards that were added in `Lib/test/test_capi/test_codecs.py`
since they are now redundant (see 32e07fd377
for additional context).

Indeed, the codecs handlers now correctly handle the `start` and `end` positions
of `UnicodeError` objects and thus should not crash.
2025-01-23 16:06:16 +01:00
Victor Stinner b23b27bc55
gh-128690: Update test_embed for getpath.py exec_prefix change (#129137) 2025-01-23 15:50:04 +01:00
Sergey B Kirpichev 75f59bb629
gh-101410: support custom messages for domain errors in the math module (#124299)
This adds basic support to override default messages for domain errors
in the math_1() helper.  The sqrt(), atanh(), log2(), log10() and log()
functions were modified as examples.  New macro supports gradual
changing of error messages in other 1-arg functions.

Co-authored-by: CharlieZhao <zhaoyu_hit@qq.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-01-23 13:55:25 +00:00
Bénédikt Tran 25a614a502
gh-126004: Fix positions handling in `codecs.backslashreplace_errors` (#127676)
This fixes how `PyCodec_BackslashReplaceErrors` handles the `start` and `end`
attributes of `UnicodeError` objects via the `_PyUnicodeError_GetParams` helper.
2025-01-23 14:28:33 +01:00
Victor Stinner 46c7e13c05
gh-129185: Fix PyTraceMalloc_Untrack() at Python exit (#129191)
Support calling PyTraceMalloc_Track() and PyTraceMalloc_Untrack()
during late Python finalization.

* Call _PyTraceMalloc_Fini() later in Python finalization.
* Test also PyTraceMalloc_Untrack() without the GIL
* PyTraceMalloc_Untrack() now gets the GIL.
* Test also PyTraceMalloc_Untrack() in test_tracemalloc_track_race().
2025-01-23 12:07:34 +01:00
Bénédikt Tran 225296cd5b
gh-126004: Fix positions handling in `codecs.replace_errors` (#127674)
This fixes how `PyCodec_ReplaceErrors` handles the `start` and `end` attributes
of `UnicodeError` objects via the `_PyUnicodeError_GetParams` helper.
2025-01-23 11:44:18 +01:00
Bénédikt Tran 70dcc847df
gh-126004: Fix positions handling in `codecs.xmlcharrefreplace_errors` (#127675)
This fixes how `PyCodec_XMLCharRefReplaceErrors` handles the `start` and `end`
attributes of `UnicodeError` objects via the `_PyUnicodeError_GetParams` helper.
2025-01-23 11:42:38 +01:00
Sam Gross a10f99375e
Revert "GH-128914: Remove conditional stack effects from `bytecodes.c` and the code generators (GH-128918)" (GH-129202)
The commit introduced a ~2.5-3% regression in the free threading build.

This reverts commit ab61d3f430.
2025-01-23 09:26:25 +00:00
Kirill Podoprigora fc6d4b71eb
gh-129192: Use `EnvironmentVarGuard` to restore environment variables (#129193)
Co-authored-by: Cody Maloney <cmaloney@theoreticalchaos.com>
2025-01-22 20:39:26 +00:00
Yury Selivanov 188598851d
GH-91048: Add utils for capturing async call stack for asyncio programs and enable profiling (#124640)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Savannah Ostrowski <savannahostrowski@gmail.com>
Co-authored-by: Jacob Coffee <jacob@z7x.org>
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2025-01-22 17:25:29 +01:00
Pablo Galindo Salgado 60a3a0dd6f
gh-124363: Treat debug expressions in f-string as raw strings (#128399)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2025-01-22 16:24:54 +00:00
Tomas R. ba9a4b6215
gh-128636: Fix crash in PyREPL when `os.environ` is overwritten with an invalid value (#128653) 2025-01-22 16:15:23 +00:00
Łukasz Langa 2ed5ee9a50
gh-129158: Fix unwanted trailing whitespace in test_asyncio.test_subprocess (#129181) 2025-01-22 13:52:45 +00:00
Mikhail Efimov 8e20e42cc6
gh-125723: Fix crash with f_locals when generator frame outlive their generator (#126956)
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
2025-01-22 15:50:01 +03:00
Thomas Grainger 24c84d816f
gh-128770: fix ResourceWarning in test_pyrepl (#128906) 2025-01-22 12:48:33 +00:00
sobolevn a16ded10ad
gh-128894: Fix `TracebackException._format_syntax_error` on custom `SyntaxError` metadata (#128946) 2025-01-22 12:47:03 +00:00
Pablo Galindo Salgado a9f5edbf5f
gh-129158: Ensure we restore unix_events.can_use_pidfd after SubprocessThreadedWatcherTests finishes (#129160) 2025-01-22 12:25:42 +00:00
Victor Stinner 9012fa741d
gh-128863: Deprecate private C API functions (#128864)
Deprecate private C API functions:

* _PyBytes_Join()
* _PyDict_GetItemStringWithError()
* _PyDict_Pop()
* _PyThreadState_UncheckedGet()
* _PyUnicode_AsString()
* _Py_HashPointer()
* _Py_fopen_obj()

Replace _Py_HashPointer() with Py_HashPointer().

Remove references to deprecated functions.
2025-01-22 11:04:19 +00:00
Ken Jin 86c1a60d5a
gh-128563: Move GO_TO_INSTRUCTION and PREDICT to cases generator (GH-129115) 2025-01-22 09:22:25 +08:00
Tomas R. 767cf70844
gh-129093: Fix f-string debug text sometimes getting cut off when expression contains `!` (#129159) 2025-01-22 00:26:37 +00:00
Daniel Hollas 29caec62ee
gh-118878: Pyrepl: show completions menu below the current line (#118939)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2025-01-21 21:06:13 +00:00
Lysandros Nikolaou 5a9afe2362
gh-123024: Correctly prepare/restore around help and show-history commands (#124485)
Co-authored-by: Emily Morehouse <emily@cuttlesoft.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2025-01-21 21:04:30 +00:00
Pieter Eendebak d147e5e52c
gh-126332: Fix pyrepl crash for double ctrl-z in line overflow (#126650)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2025-01-21 20:14:10 +00:00
Tomas R. 7ad793e5db
gh-125553: Fix backslash continuation in `untokenize` (#126010) 2025-01-21 19:58:44 +00:00
Barney Gale 01d91500ca
GH-128520: Make `pathlib._abc.WritablePath` a sibling of `ReadablePath` (#129014)
In the private pathlib ABCs, support write-only virtual filesystems by
making `WritablePath` inherit directly from `JoinablePath`, rather than
subclassing `ReadablePath`.

There are two complications:

- `ReadablePath.open()` applies to both reading and writing
- `ReadablePath.copy` is secretly an object that supports the *read* side
  of copying, whereas `WritablePath.copy` is a different kind of object
  supporting the *write* side

We untangle these as follow:

- A new `pathlib._abc.magic_open()` function replaces the `open()` method,
  which is dropped from the ABCs but remains in `pathlib.Path`. The
  function works like `io.open()`, but additionally accepts objects with
  `__open_rb__()` or `__open_wb__()` methods as appropriate for the mode.
  These new dunders are made abstract methods of `ReadablePath` and
  `WritablePath` respectively.  If the pathlib ABCs are made public, we
  could consider blessing an "openable" protocol and supporting it in
  `io.open()`, removing the need for `pathlib._abc.magic_open()`.
- `ReadablePath.copy` becomes a true method, whereas `WritablePath.copy` is
  deleted. A new `ReadablePath._copy_reader` property provides a
  `CopyReader` object, and similarly `WritablePath._copy_writer` is a
  `CopyWriter` object. Once GH-125413 is resolved, we'll be able to move
  the `CopyReader` functionality into `ReadablePath.info` and eliminate
  `ReadablePath._copy_reader`.
2025-01-21 18:35:37 +00:00
Hugo van Kemenade 05d12eecbd
gh-127873: Only check `sys.flags.ignore_environment` for `PYTHON*` env vars (#127877) 2025-01-21 16:10:08 +00:00
Petr Viktorin 13475e0a5a
gh-128889: Zero out memory ctypes for generated struct layout tests (GH-128944) 2025-01-21 16:29:02 +01:00
Serhiy Storchaka 4b37a6bda2
gh-71339: Fix an order-dependent failure in test_unittest (GH-129133)
It failed if it was preceded by test_builtin.
2025-01-21 16:45:20 +02:00
Petr Viktorin bf150f61ad
gh-126349: test_turtle: Add cleanup to avoid reference leaks (GH-129079) 2025-01-21 11:28:34 +01:00
Petr Viktorin d3b1bb228c
gh-128156: Guard use of `ffi_type_complex_double` on macOS system libffi (GH-128680)
* Determine ffi complex support at runtime
* Also, generate SIMPLE_TYPE_CHARS once at runtime
2025-01-21 10:59:18 +01:00
Serhiy Storchaka f7cc7d296c
gh-71339: Use new assertion methods in test_import and test_importlib (GH-129052) 2025-01-21 11:24:19 +02:00
Hugo van Kemenade 417f7a9ef7
gh-128595: Fix `test__colorize` unexpected keyword argument 'file' on buildbots (#129070) 2025-01-21 11:20:54 +02:00
Filipe Laíns 🇵🇸 e52ab564da
GH-92897: schedule the check_home deprecation to 3.15 (#129102) 2025-01-20 21:25:14 +00:00
Wulian 5d57959d7d
gh-91279: ZipFile.writestr now respect SOURCE_DATE_EPOCH (#124435) 2025-01-20 13:12:29 -05:00
5ec1cff dda02eb7be
GH-128131: Completely support random read access of uncompressed unencrypted files in ZipFile (#128143)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-01-20 13:04:43 -05:00
Thomas Grainger ed6934e71e
gh-128588: gh-128550: remove eager tasks optimization that missed and introduced incorrect cancellations (#129063)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-01-20 17:13:01 +00:00
Mark Shannon ab61d3f430
GH-128914: Remove conditional stack effects from `bytecodes.c` and the code generators (GH-128918) 2025-01-20 17:09:23 +00:00
Filipe Laíns 🇵🇸 0a6412f9cc
GH-129064: deprecate sysconfig.expand_makefile_vars (#129082) 2025-01-20 17:03:44 +00:00
Thomas Grainger 38a9956876
gh-128308: pass `**kwargs` to asyncio task_factory (#128768)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-01-20 22:23:55 +05:30
Mark Shannon f0f7b978be
GH-128939: Refactor JIT optimize structs (GH-128940) 2025-01-20 15:49:15 +00:00
Victor Stinner 8ceb6cb117
gh-129033: Remove _PyInterpreterState_SetConfig() function (#129048)
Remove _PyInterpreterState_GetConfigCopy() and
_PyInterpreterState_SetConfig() private functions. PEP 741 "Python
Configuration C API" added a better public C API: PyConfig_Get() and
PyConfig_Set().
2025-01-20 16:31:33 +01:00
Serhiy Storchaka 38c3cf6320
gh-71339: Use new assertion methods in test_ctypes (GH-129054) 2025-01-20 13:30:00 +00:00
Bénédikt Tran 59fcae793f
Remove duplicated dict keys in `test_{embed,long}.py` fixtures (#128727) 2025-01-20 13:50:10 +01:00
Serhiy Storchaka a30277a06a
gh-71339: Use new assertion methods in test_capi (GH-129053) 2025-01-20 14:32:27 +02:00
Erlend E. Aasland 537296cdcd
gh-111178: Generate correct signature for most self converters (#128447) 2025-01-20 12:40:18 +01:00
Serhiy Storchaka c6b570e5e3
gh-71339: Use new assertion methods in `test_asyncio` (#129051) 2025-01-20 17:02:39 +05:30
Hugo van Kemenade 6f167d7134
gh-128595: Default to stdout isatty for colour detection instead of stderr (#128498)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2025-01-20 12:52:42 +02:00
Victor Stinner 07c3518ffb
gh-129033: Remove _Py_InitializeMain() function (#129034)
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
2025-01-20 10:03:22 +00:00
Serhiy Storchaka c463270c73
gh-71339: Use new assertion methods in test_traceback (GH-128848) 2025-01-20 11:22:33 +02:00
Serhiy Storchaka 485d2e831e
gh-71339: Use assertIsSubclass() and assertNotIsSubclass() in test_decimal (GH-128827) 2025-01-20 11:20:49 +02:00
Serhiy Storchaka 36aaf4137d
gh-71339: Use new assertion methods in test_abc (GH-128826) 2025-01-20 11:19:43 +02:00
Serhiy Storchaka da122b5fac
gh-71339: Improve error report for types in assertHasAttr() and assertNotHasAttr() (GH-128818) 2025-01-20 11:17:49 +02:00
Serhiy Storchaka 89c401fb9a
gh-71339: Use new assertion methods in test_functools (GH-128829) 2025-01-20 11:17:04 +02:00
Serhiy Storchaka 887f2bcf48
gh-71339: Use assertIsSubclass() and assertNotIsSubclass() in test_collections (GH-128824) 2025-01-20 11:16:32 +02:00
Kirill Podoprigora 6c52ada551
gh-100239: Handle NaN and zero division in guards for `BINARY_OP_EXTEND` (#128963)
Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2025-01-19 11:02:49 +00:00
Mike Edmunds 5aaf416858
gh-80222: Fix email address header folding with long quoted-string (#122753)
Email generators using email.policy.default could incorrectly omit the
quote ('"') characters from a quoted-string during header refolding,
leading to invalid address headers and enabling header spoofing. This
change restores the quote characters on a bare-quoted-string as the
header is refolded, and escapes backslash and quote chars in the string.
2025-01-18 19:50:52 -05:00
Victor Stinner 9bc19643ed
gh-128679: Skip test_tracemalloc_track_race() on debug build (#128988)
There is a race condition between PyMem_SetAllocator() and
PyMem_RawMalloc()/PyMem_RawFree(). While PyMem_SetAllocator() write
is protected by a lock, PyMem_RawMalloc()/PyMem_RawFree() reads are
not protected by a lock. PyMem_RawMalloc()/PyMem_RawFree() can be
called with an old context and the new function pointer.

On a release build, it's not an issue since the context is not used.
On a debug build, the debug hooks use the context and so can crash.
2025-01-18 15:13:54 +00:00
Marie Roald d3adf02c90
gh-126349: Add 'fill', 'poly', and 'no_animation' context managers to turtle (#126350)
Co-authored-by: Marie Roald <roald.marie@gmail.com>
Co-authored-by: Yngve Mardal Moe <yngve.m.moe@gmail.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Daniel Hollas <danekhollas@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2025-01-18 11:27:22 +01:00
Tomasz Pytel 4dade055f4
gh-128961: Fix exhausted array iterator crash in __setstate__() (#128962) 2025-01-18 10:55:29 +01:00
Zanie Blue 9ed7bf2cd7
gh-128515: Add BOLT build to CI (gh-128845) 2025-01-18 07:32:23 +00:00
T. Wouters d4544cb232
gh-128923: fix test_pydoc for object subclasses without `__module__` (#128951)
Fix pydoc's docclass() for classes inheriting from object without the `__module__` attribute, like `_testcapi.HeapType`.
2025-01-17 23:43:17 +01:00
Victor Stinner d95ba9fa11
gh-128911: Add tests on the PyImport C API (#128915)
* Add Modules/_testlimitedcapi/import.c
* Add Lib/test/test_capi/test_import.py
* Remove _testcapi.check_pyimport_addmodule(): tests already covered
  by newly added tests.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2025-01-17 17:52:18 +00:00
Sam Gross d66c08aa75
gh-128923: Use zero to indicate unassigned unique id (#128925)
In the free threading build, the per thread reference counting uses a
unique id for some objects to index into the local reference count
table. Use 0 instead of -1 to indicate that the id is not assigned. This
avoids bugs where zero-initialized heap type objects look like they have
a unique id assigned.
2025-01-17 16:42:27 +01:00
Tian Gao 767c89ba7c
gh-58956: Fix a frame refleak in bdb (#128190) 2025-01-17 10:33:04 -05:00
Victor Stinner d7f703d54d
gh-59705: Implement _thread.set_name() on Windows (#128675)
Implement set_name() with SetThreadDescription() and _get_name() with
GetThreadDescription(). If SetThreadDescription() or
GetThreadDescription() is not available in kernelbase.dll, delete the
method when the _thread module is imported.

Truncate the thread name to 32766 characters.

Co-authored-by: Eryk Sun <eryksun@gmail.com>
2025-01-17 14:55:43 +01:00