Commit Graph

27203 Commits

Author SHA1 Message Date
Miss Islington (bot) 46347d3caf
[3.11] gh-110647: Fix signal test_stress_modifying_handlers() (GH-110650) (#110659)
gh-110647: Fix signal test_stress_modifying_handlers() (GH-110650)

* cycle_handlers() now waits until at least one signal is received.
* num_received_signals can be equal to num_sent_signals.
(cherry picked from commit e07c37cd52)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-10-11 00:21:51 +00:00
Erlend E. Aasland a009bb872c
[3.11] gh-109286: Update Windows installer to use SQLite 3.43.1 (#110403) (#110479) 2023-10-11 00:41:12 +02:00
Serhiy Storchaka 3788c48d12
[3.11] gh-110388: Add tests for tty (GH-110394) (GH-110634)
(cherry picked from commit 7f702b2)
2023-10-10 16:27:22 +02:00
Miss Islington (bot) e511f544ea
[3.11] gh-81002: Add tests for termios (GH-110386) (GH-110620)
(cherry picked from commit 92a9e98024)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-10-10 14:11:06 +02:00
Miss Islington (bot) 7fefed091a
[3.11] gh-110590: Fix a bug where _sre.compile would overwrite exceptions (GH-110591) (#110614)
TypeError would be overwritten by OverflowError
if 'code' param contained non-ints.
(cherry picked from commit 344d3a222a)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2023-10-10 10:48:07 +00:00
Miss Islington (bot) 2b3a418279
[3.11] gh-110378: Close invalid generators in contextmanager and asynccontextmanager (GH-110499) (#110589)
contextmanager and asynccontextmanager context managers now close an invalid
underlying generator object that yields more then one value.
(cherry picked from commit 96fed66a65)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-10-10 11:12:52 +02:00
Bo Anderson 9e9df93ffc
[3.11] gh-109191: Fix build with newer editline (gh-110239) (#110575)
(cherry picked from commit f4cb0d27cc)
2023-10-09 19:42:25 +00:00
Miss Islington (bot) e913c6f4d6
[3.11] gh-110519: Improve deprecation warning in the gettext module (GH-110520) (GH-110564)
Deprecation warning about non-integer numbers in gettext now always refers
to the line in the user code where gettext function or method is used.
Previously, it could refer to a line in gettext code.

Also, increase test coverage for NullTranslations and domain-aware functions
like dngettext().
(cherry picked from commit 326c6c4e07)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-10-09 16:10:34 +02:00
Miss Islington (bot) 26c3e700b1
[3.11] gh-110437: Allow overriding VCRuntimeDLL with a semicolon separated list of DLLs to bundle (GH-110470)
gh-110437: Allow overriding VCRuntimeDLL with a semicolon separated list of DLLs to bundle (GH-110470)
(cherry picked from commit 12cc6792d0)

Co-authored-by: Steve Dower <steve.dower@python.org>
2023-10-09 11:42:57 +00:00
Miss Islington (bot) b473d48505
[3.11] gh-109286: Update macOS installer to use SQLite 3.43.1 (GH-110482) (#110551)
(cherry picked from commit 48419a50b4)

Co-authored-by: jtranquilli <76231120+jtranquilli@users.noreply.github.com>
2023-10-09 12:31:27 +02:00
Miss Islington (bot) f21c09ca03
[3.11] gh-110237: Check `PyList_Append` for errors in `_PyEval_MatchClass` (GH-110238) (#110512)
gh-110237: Check `PyList_Append` for errors in `_PyEval_MatchClass` (GH-110238)
(cherry picked from commit dd9d781da3)

Co-authored-by: denballakh <47365157+denballakh@users.noreply.github.com>
2023-10-08 00:29:46 +00:00
Serhiy Storchaka 6a33529cf0
[3.11] gh-109521: Fix obscure cases handling in PyImport_GetImporter() (GH-109522) (GH-109781)
PyImport_GetImporter() now sets RuntimeError if it fails to get sys.path_hooks
or sys.path_importer_cache or they are not list and dict correspondingly.

Previously it could return NULL without setting error in obscure cases,
crash or raise SystemError if these attributes have wrong type.
(cherry picked from commit 62c7015e89)
2023-10-07 16:05:13 +03:00
Miss Islington (bot) 4499e6caff
[3.11] gh-103053: Fix test_tools.test_freeze on FreeBSD (GH-110451) (#110457)
gh-103053: Fix test_tools.test_freeze on FreeBSD (GH-110451)

Fix test_tools.test_freeze on FreeBSD: run "make distclean" instead
of "make clean" in the copied source directory to remove also the
"python" program.

Other test_freeze changes:

* Log executed commands and directories, and the current directory.
* No longer uses make -C option to change the directory, instead use
  subprocess cwd parameter.
(cherry picked from commit a4baa9e8ac)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-10-06 01:32:21 +00:00
Miss Islington (bot) f7a1d7d060
[3.11] gh-103053: Fix make check-clean-src: check "python" program (GH-110449) (#110454)
gh-103053: Fix make check-clean-src: check "python" program (GH-110449)

"make check-clean-src" now also checks if the "python" program is
found in the source directory: fail with an error if it does exist.
(cherry picked from commit a155f9f342)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-10-06 01:10:39 +00:00
Miss Islington (bot) 8da3367067
[3.11] gh-110167: Fix test_socket deadlock in doCleanups() (GH-110416) (#110424)
gh-110167: Fix test_socket deadlock in doCleanups() (GH-110416)

Fix a deadlock in test_socket when server fails with a timeout but
the client is still running in its thread. Don't hold a lock to call
cleanup functions in doCleanups(). One of the cleanup function waits
until the client completes, whereas the client could deadlock if it
called addCleanup() in such situation.

doCleanups() is called when the server completed, but the client can
still be running in its thread especially if the server failed with a
timeout. Don't put a lock on doCleanups() to prevent deadlock between
addCleanup() called in the client and doCleanups() waiting for
self.done.wait of ThreadableTest._setUp().
(cherry picked from commit 318f5df271)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-10-05 19:18:44 +00:00
Miss Islington (bot) 34c7793cd0
[3.11] gh-110365: Fix error overwrite in `termios.tcsetattr` (GH-110366) (#110390)
(cherry picked from commit 2bbbab212f)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-10-05 14:21:34 +02:00
Victor Stinner 8f22504d74
[3.11] gh-109974: Fix threading lock_tests race conditions (#110057) (#110355)
[3.12] gh-109974: Fix threading lock_tests race conditions (#110057) (#110346)

* gh-109974: Fix threading lock_tests race conditions (#110057)

Fix race conditions in test_threading lock tests. Wait until a
condition is met rather than using time.sleep() with a hardcoded
number of seconds.

* Replace sleeping loops with support.sleeping_retry() which raises
  an exception on timeout.
* Add wait_threads_blocked(nthread) which computes a sleep depending
  on the number of threads. Remove _wait() function.
* test_set_and_clear(): use a way longer Event.wait() timeout.
* BarrierTests.test_repr(): wait until the 2 threads are waiting for
  the barrier. Use a way longer timeout for Barrier.wait() timeout.
* test_thread_leak() no longer needs to count
  len(threading.enumerate()): Bunch uses
  threading_helper.wait_threads_exit() internally which does it in
  wait_for_finished().
* Add BaseLockTests.wait_phase() which implements a timeout.
  test_reacquire() and test_recursion_count() use wait_phase().

(cherry picked from commit 4e356ad183)

* gh-109974: Fix more threading lock_tests race conditions (#110089)

* Add context manager on Bunch class.
* Bunch now catchs exceptions on executed functions and re-raise them
  at __exit__() as an ExceptionGroup.
* Rewrite BarrierProxy.test_default_timeout(). Use a single thread.
  Only check that barrier.wait() blocks for at least default timeout
  seconds.
* test_with(): inline _with() function.

(cherry picked from commit 743e3572ee)
(cherry picked from commit 1d032ea3d6)
2023-10-04 12:07:57 +00:00
Miss Islington (bot) aa8d3db6f2
[3.11] [3.12] gh-109972: Enhance test_gdb (GH-110026) (GH-110351) (#110354)
[3.12] gh-109972: Enhance test_gdb (GH-110026) (GH-110351)

* gh-109972: Enhance test_gdb (GH-110026)

* Split test_pycfunction.py: add test_cfunction_full.py.
  Split the function into the following 6 functions. In verbose
  mode, these "pycfunction" tests now log each tested call.

  * test_pycfunction_noargs()
  * test_pycfunction_o()
  * test_pycfunction_varargs()
  * test_pycfunction_varargs_keywords()
  * test_pycfunction_fastcall()
  * test_pycfunction_fastcall_keywords()

* Move get_gdb_repr() to PrettyPrintTests.
* Replace DebuggerTests.get_sample_script() with SAMPLE_SCRIPT.
* Rename checkout_hook_path to CHECKOUT_HOOK_PATH.
* Rename gdb_version to GDB_VERSION_TEXT.
* Replace (gdb_major_version, gdb_minor_version) with GDB_VERSION.
* run_gdb() uses "backslashreplace" error handler instead of "replace".
* Add check_gdb() function to util.py.
* Enhance support.check_cflags_pgo(): check also for sysconfig
  PGO_PROF_USE_FLAG (if available) in compiler flags.
* Move some SkipTest checks to test_gdb/__init__.py.
* Elaborate why gdb cannot be tested on Windows: gdb doesn't support
  PDB debug symbol files.

(cherry picked from commit 757cbd4f29)

* gh-104736: Fix test_gdb tests on ppc64le with clang (GH-109360)

Fix test_gdb on Python built with LLVM clang 16 on Linux ppc64le (ex:
Fedora 38). Search patterns in gdb "bt" command output to detect
when gdb fails to retrieve the traceback. For example, skip a test if
"Backtrace stopped: frame did not save the PC" is found.

(cherry picked from commit 44d9a71ea2)

* gh-110166: Fix gdb CFunctionFullTests on ppc64le clang build (GH-110331)

CFunctionFullTests now also runs "bt" command before "py-bt-full",
similar to CFunctionTests which also runs "bt" command before
"py-bt". So test_gdb can skip the test if patterns like "?? ()" are
found in the gdb output.

(cherry picked from commit bbce8bd05d)

Co-authored-by: Victor Stinner <vstinner@python.org>
(cherry picked from commit 1de9406f91)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-10-04 12:03:28 +00:00
Victor Stinner 6c98c734c7
[3.11] gh-109972: Split test_gdb.py into test_gdb package (#109977) (… (#110343)
[3.12] gh-109972: Split test_gdb.py into test_gdb package (#109977) (#110339)

gh-109972: Split test_gdb.py into test_gdb package (#109977)

Split test_gdb.py file into a test_gdb package made of multiple
tests, so tests can now be run in parallel.

* Create Lib/test/test_gdb/ directory.
* Split test_gdb.py into multiple files in Lib/test/test_gdb/
  directory.
* Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory.
  Update get_sample_script(): use __file__ to locate gdb_sample.py.
* Move gdb_has_frame_select() and HAS_PYUP_PYDOWN to test_misc.py.
* Explicitly skip test_gdb on Windows. Previously, test_gdb was
  skipped even if gdb was available because of
  gdb_has_frame_select().

(cherry picked from commit 8f324b7ecd)
(cherry picked from commit e7a61d34b7)
2023-10-04 10:53:28 +00:00
Miss Islington (bot) 5039db7f9b
[3.11] gh-110267: Add tests for pickling and copying PyStructSequence objects (GH-110272) (GH-110284)
(cherry picked from commit 2d4865d775)

Co-authored-by: Xuehai Pan <XuehaiPan@pku.edu.cn>
2023-10-04 12:01:19 +03:00
Miss Islington (bot) 0c6ae623e5
[3.11] Remove unused Misc/requirements-test.txt (GH-110240) (#110254)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-10-02 23:36:35 -06:00
Pablo Galindo 8b6ee5ba3b Python 3.11.6 2023-10-02 14:29:10 +01:00
Miss Islington (bot) a673248d6c
[3.11] GH-101100: Fix reference warnings for ``namedtuple`` (GH-110113) (#110136)
GH-101100: Fix reference warnings for ``namedtuple`` (GH-110113)
(cherry picked from commit cbdacc738a)

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-09-30 06:42:37 +00:00
Zachary Ware d87217ff44
[3.11] gh-109991: Update Windows build to use OpenSSL 3.0.11 (GH-110059)
(cherry picked from commit cf4c297256)
2023-09-29 10:03:48 -05:00
Victor Stinner 8ac6890d5e
[3.11] gh-110031: Skip test_threading fork tests if ASAN (#110100) (#110104)
gh-110031: Skip test_threading fork tests if ASAN (#110100)

Skip test_threading tests using thread+fork if Python is built with
Address Sanitizer (ASAN).

(cherry picked from commit 86e76ab8af)
2023-09-29 15:19:37 +02:00
Victor Stinner 184ce1414b
[3.11] gh-110088, gh-109878: Fix test_asyncio timeouts (#110092) (#110099)
gh-110088, gh-109878: Fix test_asyncio timeouts (#110092)

Fix test_asyncio timeouts: don't measure the maximum duration, a test
should not measure a CI performance. Only measure the minimum
duration when a task has a timeout or delay. Add CLOCK_RES to
test_asyncio.utils.

(cherry picked from commit db0a258e79)
2023-09-29 14:16:15 +02:00
Adam Turner d81bcc2327
[3.11] gh-102304: Add links to the Stable ABI and Limited C API docs (#105345) (#105371) (#109901)
* Add "limited-c-api" and "stable-api" targets
* Rename the "stable-abi-list" target to "limited-api-list"
2023-09-29 11:17:44 +02:00
Serhiy Storchaka c534637106
[3.11] gh-109634: Use :samp: role (GH-109635) (GH-109778)
(cherry picked from commit 92af0cc580)

Co-authored-by: Jacob Coffee <jacob@z7x.org>
2023-09-29 09:39:31 +03:00
Miss Islington (bot) efe83ad276
[3.11] gh-110036: multiprocessing Popen.terminate() catches PermissionError (GH-110037) (#110065)
gh-110036: multiprocessing Popen.terminate() catches PermissionError (GH-110037)

On Windows, multiprocessing Popen.terminate() now catchs
PermissionError and get the process exit code. If the process is
still running, raise again the PermissionError. Otherwise, the
process terminated as expected: store its exit code.
(cherry picked from commit bd4518c60c)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-29 01:04:58 +00:00
Davide Rizzo d6e8001467
[3.11] gh-110038: KqueueSelector must count all read/write events (GH-110039) (#110044)
[3.11] gh-110038: KqueueSelector must count all read/write events (GH-110039).
(cherry picked from commit b14f0ab51c)
2023-09-28 20:58:24 +02:00
Miss Islington (bot) 973d549e72
[3.11] gh-110033: Fix signal test_interprocess_signal() (GH-110035) (#110041)
gh-110033: Fix signal test_interprocess_signal() (GH-110035)

Fix test_interprocess_signal() of test_signal. Make sure that the
subprocess.Popen object is deleted before the test raising an
exception in a signal handler. Otherwise, Popen.__del__() can get the
exception which is logged as "Exception ignored in: ...." and the
test fails.
(cherry picked from commit 7e0fbf5175)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-28 17:48:58 +00:00
Miss Islington (bot) ff3cadd2a1
[3.11] gh-109594: Fix concurrent.futures test_timeout() (GH-110018) (#110022)
gh-109594: Fix concurrent.futures test_timeout() (GH-110018)

Fix test_timeout() of test_concurrent_futures.test_wait. Remove the
future which may or may not complete depending if it takes longer
than the timeout ot not. Keep the second future which does not
complete before wait(). Make also the test faster: 0.5 second instead
of 6 seconds, so remove @support.requires_resource('walltime')
decorator.
(cherry picked from commit 9be283e5e1)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-28 13:49:56 +00:00
Miss Islington (bot) 82dea84dc4
[3.11] gh-109991: Update macOS installer to use OpenSSL 3.0.10. (GH-110010)
(cherry picked from commit 98c0c1de18)

Co-authored-by: Ned Deily <nad@python.org>
2023-09-28 06:34:24 +00:00
Ned Deily 1fd6a73bd8
[3.11] gh-109991: Update GitHub CI workflows to use OpenSSL 3.0.11 and multissltests to use 1.1.1w, 3.0.11, and 3.1.3. (#110006)
(cherry picked from commit c88037d137)
2023-09-28 01:31:17 -04:00
Miss Islington (bot) f764abb375
[3.11] gh-109593: Fix reentrancy issue in multiprocessing resource_tracker (GH-109629) (#109897)
gh-109593: Fix reentrancy issue in multiprocessing resource_tracker (GH-109629)

---------

(cherry picked from commit 0eb98837b6)

Co-authored-by: Antoine Pitrou <antoine@python.org>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-09-26 12:21:52 +00:00
Miss Islington (bot) 97f7e9d5d6
[3.11] gh-109631: Allow interruption of short repeated regex matches (GH-109867) (GH-109885)
Counting for signal checking now continues in new match from the point where
it ended in the previous match instead of starting from 0.
(cherry picked from commit 8ac2085b80)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-09-26 08:19:29 +00:00
Miss Islington (bot) 358282e965
[3.11] gh-109748: Fix venv test_zippath_from_non_installed_posix() (GH-109872) (#109874)
gh-109748: Fix venv test_zippath_from_non_installed_posix() (GH-109872)

Fix test_zippath_from_non_installed_posix() of test_venv: don't copy
__pycache__/ sub-directories, because they can be modified by other
Python tests running in parallel.
(cherry picked from commit 25bb266fc8)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-26 01:41:32 +02:00
elfstrom 97ea90194c
[3.11] gh-105829: Fix concurrent.futures.ProcessPoolExecutor deadlock (GH-108513) (#109783)
This fixes issue GH-105829, https://github.com/python/cpython/issues/105829

(cherry picked from commit 405b06375a)
2023-09-24 20:28:03 +01:00
Hugo van Kemenade cbe153edd6
[3.11] Docs: Update Donghee Na's name (GH-109743) (#109759)
* Update Donghee Na's name in Docs/

* Update Donghee Na's name in Misc/ACKS

* Update Donghee Na's name in Misc/NEWS.d/
2023-09-23 06:53:47 +03:00
Miss Islington (bot) 567c3e846e
[3.11] ACKS: Fix ordering; Correct Itamar Oren's surname; Add Adam Turner (GH-109737) (#109742)
ACKS: Fix ordering; Correct Itamar Oren's surname; Add Adam Turner  (GH-109737)
(cherry picked from commit 3e8fcb7df7)

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-09-22 15:05:04 +00:00
Miss Islington (bot) 43ff8fcce2
[3.11] gh-109375: Fix bug where pdb registers an alias without an associated command (GH-109376) (#109430)
gh-109375: Fix bug where pdb registers an alias without an associated command (GH-109376)
(cherry picked from commit 68a6f21f47)

Co-authored-by: buermarc <44375277+buermarc@users.noreply.github.com>
2023-09-22 03:45:11 +02:00
Victor Stinner 66a973a09e
[3.11] gh-108388: Convert test_concurrent_futures to package (#108401) (#109704)
* gh-108388: Convert test_concurrent_futures to package (#108401)

Convert test_concurrent_futures to a package of sub-tests.

(cherry picked from commit aa6f787faa)

Notes on backport to 3.11:

* AsCompletedTests: Revert test_future_times_out() => test_zero_timeout()
* Restore TODO comment
* ThreadPoolExecutorTest.test_hang_global_shutdown_lock():
  add @support.requires_resource('cpu').
2023-09-22 01:37:25 +00:00
Miss Islington (bot) b3af888342
[3.11] gh-108388: Split test_multiprocessing_spawn (GH-108396) (#109688)
gh-108388: Split test_multiprocessing_spawn (GH-108396)

Split test_multiprocessing_fork, test_multiprocessing_forkserver and
test_multiprocessing_spawn into test packages. Each package is made
of 4 sub-tests: processes, threads, manager and misc. It allows
running more tests in parallel and so reduce the total test duration.
(cherry picked from commit aa9a359ca2)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-21 20:44:21 +00:00
Victor Stinner 0c79353e11
[3.11] gh-109613: _pystat_fromstructstat() checks for exceptions (#109618) (#109668)
gh-109613: _pystat_fromstructstat() checks for exceptions (#109618)

Fix os.stat() and os.DirEntry.stat(): check for exceptions.
Previously, on Python built in debug mode, these functions could
trigger a fatal Python error (and abort the process) when a function
succeeded with an exception set.

_pystat_fromstructstat() now exits immediately if an exception is
raised, rather only checking for exceptions at the end. It fix
following fatal error in fill_time():

    Fatal Python error: _Py_CheckSlotResult:
    Slot * of type int succeeded with an exception set

(cherry picked from commit d4cea794a7)
2023-09-21 18:00:38 +00:00
Miss Islington (bot) e918b19f42
[3.11] GH-109209: Bump the minimum Sphinx version to 4.2 (GH-109210) (#109637)
GH-109209: Bump the minimum Sphinx version to 4.2 (GH-109210)
(cherry picked from commit 712cb173f8)

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-09-21 06:32:10 +00:00
Victor Stinner 8c7fadb9bf
[3.11] gh-103053: Skip test_freeze_simple_script() on PGO build (#109591) (#109616)
gh-103053: Skip test_freeze_simple_script() on PGO build (#109591)

Skip test_freeze_simple_script() of test_tools.test_freeze if Python
is built with "./configure --enable-optimizations", which means with
Profile Guided Optimization (PGO): it just makes the test too slow.
The freeze tool is tested by many other CIs with other (faster)
compiler flags.

test.pythoninfo now gets also get_build_info() of
test.libregrtests.utils.

(cherry picked from commit 81cd1bd713)
2023-09-20 15:45:23 +00:00
Shantanu 2184b76496
[3.11] gh-108843: fix ast.unparse for f-string with many quotes (#108980)
* [3.11] gh-108843: fix ast.unparse for f-string with many quotes

* 📜🤖 Added by blurb_it.

* simplify

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-09-18 15:11:56 +01:00
Miss Islington (bot) f7bfac4b3d
[3.11] gh-109396: Fix test_socket.test_hmac_sha1() in FIPS mode (GH-109423) (#109427)
gh-109396: Fix test_socket.test_hmac_sha1() in FIPS mode (GH-109423)

Use a longer key: FIPS mode requires at least of at least 112 bits.
The previous key was only 32 bits.
(cherry picked from commit e091b9f20f)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-15 00:52:40 +02:00
Miss Islington (bot) f1f85a42ea
[3.11] gh-109351: Fix crash when compiling AST with invalid NamedExpr (GH-109352) (#109380)
gh-109351: Fix crash when compiling AST with invalid NamedExpr (GH-109352)
(cherry picked from commit 79101edb03)

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-13 16:32:08 +00:00
Miss Islington (bot) 66c0d0ac8c
[3.11] gh-104736: Fix test_gdb tests on ppc64le with clang (GH-109360) (#109362)
gh-104736: Fix test_gdb tests on ppc64le with clang (GH-109360)

Fix test_gdb on Python built with LLVM clang 16 on Linux ppc64le (ex:
Fedora 38). Search patterns in gdb "bt" command output to detect
when gdb fails to retrieve the traceback. For example, skip a test if
"Backtrace stopped: frame did not save the PC" is found.
(cherry picked from commit 44d9a71ea2)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-13 04:48:44 +00:00
Miss Islington (bot) b4c1cf93c7
[3.11] gh-84867: Do not load tests from TestCase and FunctionTestCase (GH-100497) (GH-109328)
(cherry picked from commit 66d1d7eb06)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2023-09-12 14:05:58 +00:00
Irit Katriel 1e8696133c
[3.11] gh-109179: Fix traceback display for SyntaxErrors with notes (#109197) (#109283)
gh-109179: Fix traceback display for SyntaxErrors with notes (#109197)

(cherry picked from commit ecd21a629a)
2023-09-12 09:57:28 +00:00
Victor Stinner 82a18069a1
[3.11] gh-108987: Fix _thread.start_new_thread() race condition (#109135) (#109272)
gh-108987: Fix _thread.start_new_thread() race condition (#109135)

Fix _thread.start_new_thread() race condition. If a thread is created
during Python finalization, the newly spawned thread now exits
immediately instead of trying to access freed memory and lead to a
crash.

thread_run() calls PyEval_AcquireThread() which checks if the thread
must exit. The problem was that tstate was dereferenced earlier in
_PyThreadState_Bind() which leads to a crash most of the time.

Move _PyThreadState_CheckConsistency() from thread_run() to
_PyThreadState_Bind().

(cherry picked from commit 517cd82ea7)
2023-09-11 19:33:08 +02:00
Miss Islington (bot) 2f8c80f531
[3.11] gh-107219: Fix concurrent.futures terminate_broken() (GH-109244) (#109255)
gh-107219: Fix concurrent.futures terminate_broken() (GH-109244)

Fix a race condition in concurrent.futures. When a process in the
process pool was terminated abruptly (while the future was running or
pending), close the connection write end. If the call queue is
blocked on sending bytes to a worker process, closing the connection
write end interrupts the send, so the queue can be closed.

Changes:

* _ExecutorManagerThread.terminate_broken() now closes
  call_queue._writer.
* multiprocessing PipeConnection.close() now interrupts
  WaitForMultipleObjects() in _send_bytes() by cancelling the
  overlapped operation.
(cherry picked from commit a9b1f84790)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-11 08:52:17 +00:00
Miss Islington (bot) a7e80fb628
[3.11] gh-109230: test_pyexpat no longer depends on the current directory (GH-109233) (#109242)
gh-109230: test_pyexpat no longer depends on the current directory (GH-109233)

Fix test_pyexpat.test_exception(): it can now be run from a directory
different than Python source code directory. Before, the test failed
in this case.

Skip the test if Modules/pyexpat.c source is not available. Skip also
the test on Python implementations other than CPython.
(cherry picked from commit e55aab9578)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-10 22:46:47 +00:00
Miss Islington (bot) cef4951116
[3.11] gh-109237: Fix test_site for non-ASCII working directory (GH-109238) (#109240)
gh-109237: Fix test_site for non-ASCII working directory (GH-109238)

Fix test_site.test_underpth_basic() when the working directory
contains at least one non-ASCII character: encode the "._pth" file to
UTF-8 and enable the UTF-8 Mode to use UTF-8 for the child process
stdout.
(cherry picked from commit cbb3a6f8ad)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-10 22:39:49 +00:00
Miss Islington (bot) b070d73ff2
[3.11] gh-50644: Forbid pickling of codecs streams (GH-109180) (GH-109232)
Attempts to pickle or create a shallow or deep copy of codecs streams
now raise a TypeError.

Previously, copying failed with a RecursionError, while pickling
produced wrong results that eventually caused unpickling to fail with
a RecursionError.
(cherry picked from commit d6892c2b92)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-09-10 17:41:19 +00:00
Miss Islington (bot) c20658249d
[3.11] gh-109207: Fix SystemError when printing symtable entry object. (GH-109225) (GH-109228)
(cherry picked from commit 4297499696)

Co-authored-by: 云line <31395137+yunline@users.noreply.github.com>
2023-09-10 16:50:22 +03:00
Miss Islington (bot) c1a2ef5efc
[3.11] gh-106922: Fix error location for constructs with spaces and parentheses (GH-108959) (#109148)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2023-09-08 16:56:10 +00:00
Miss Islington (bot) 9dd28d2da9
[3.11] gh-108962: Skip test_tempfile.test_flags() if not supported (GH-108964) (#108968)
gh-108962: Skip test_tempfile.test_flags() if not supported (GH-108964)

Skip test_tempfile.test_flags() if chflags() fails with "OSError:
[Errno 45] Operation not supported" (ex: on FreeBSD 13).
(cherry picked from commit cd2ef21b07)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-08 07:07:54 +00:00
Ethan Furman effa2ecdcf
[3.11] gh-108682: [Enum] raise TypeError if super().__new__ called in custom __new__ (GH-108704) (GH-108739)
When overriding the `__new__` method of an enum, the underlying data type should be created directly; i.e. .

    member = object.__new__(cls)
    member = int.__new__(cls, value)
    member = str.__new__(cls, value)

Calling `super().__new__()` finds the lookup version of `Enum.__new__`, and will now raise an exception when detected.

(cherry picked from commit d48760b2f1)
2023-09-07 18:57:48 -07:00
Serhiy Storchaka dae62d456e
[3.11] gh-88943: Improve syntax error for non-ASCII character that follows a numerical literal (GH-109081) (GH-109091)
It now points on the invalid non-ASCII character, not on the valid numerical literal.
(cherry picked from commit b2729e93e9)
2023-09-07 14:54:07 +00:00
Miss Islington (bot) d9d64a4a13
[3.11] gh-102823: Document return type of floor division on floats (GH-102824) (#109093)
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-09-07 14:34:30 +00:00
Victor Stinner 1ebe014d62
[3.11] gh-109015: Add test.support.socket_helper.tcp_blackhole() (#109016) (#109042)
gh-109015: Add test.support.socket_helper.tcp_blackhole() (#109016)

Skip test_asyncio, test_imaplib and test_socket tests if FreeBSD TCP
blackhole is enabled (net.inet.tcp.blackhole=2).

(cherry picked from commit a52a350977)
2023-09-07 00:34:16 +00:00
Victor Stinner ed233ddc00
[3.11] gh-108740: Fix "make regen-all" race condition (#108741) (#109021)
gh-108740: Fix "make regen-all" race condition (#108741)

Fix a race condition in "make regen-all". The deepfreeze.c source and
files generated by Argument Clinic are now generated or updated
before generating "global objects". Previously, some identifiers may
miss depending on the order in which these files were generated.

* "make regen-global-objects": Make sure that deepfreeze.c is
  generated and up to date, and always run "make clinic".
* "make regen-deepfreeze" now only updates deepfreeze.c (C file).
  It doesn't build deepfreeze.o (object) anymore.
* Remove misleading messages in "make regen-global-objects" and
  "make clinic". They are now outdated, these commands are now
  safe to use.

Backport notes:

* Omit Doc/using/configure.rst changes.
* no need to change "make clinic", it didn't run
  generate_global_objects.py script before.

Co-authored-by: Erlend E. Aasland <erlend@python.org>
(cherry picked from commit db1ee6a19a)
2023-09-06 23:00:30 +02:00
Victor Stinner 95eb9849dd
[3.11] gh-108851: Fix tomllib recursion tests (#108853) (#109013)
gh-108851: Fix tomllib recursion tests (#108853)

* Add get_recursion_available() and get_recursion_depth() functions
  to the test.support module.
* Change infinite_recursion() default max_depth from 75 to 100.
* Fix test_tomllib recursion tests for WASI buildbots: reduce the
  recursion limit and compute the maximum nested array/dict depending
  on the current available recursion limit.
* test.pythoninfo logs sys.getrecursionlimit().
* Enhance test_sys tests on sys.getrecursionlimit()
  and sys.setrecursionlimit().

Backport notes:

* Set support.infinite_recursion() minimum to 4 frames.
* test_support.test_get_recursion_depth() uses limit-2, apparently
  f-string counts for 2 frames in Python 3.11.
* test_sys.test_setrecursionlimit_to_depth() tests depth+2 instead of
  depth+1.

(cherry picked from commit 8ff1142578)
2023-09-06 16:40:39 +00:00
Miss Islington (bot) d61b8f9b8b
[3.11] gh-91960: Skip test_gdb if gdb cannot retrive Python frames (GH-108999) (#109011)
gh-91960: Skip test_gdb if gdb cannot retrive Python frames (GH-108999)

Skip test_gdb if gdb is unable to retrieve Python frame objects: if a
frame is "<optimized out>". When Python is built with "clang -Og",
gdb can fail to retrive the 'frame' parameter of
_PyEval_EvalFrameDefault(). In this case, tests like py_bt() are
likely to fail. Without getting access to Python frames,
python-gdb.py is mostly clueless on retrieving the Python traceback.
Moreover, test_gdb is no longer skipped on macOS if Python is built
with Clang.
(cherry picked from commit fbce43a251)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-06 15:32:36 +00:00
Zachary Ware 53148af9d7
[3.11] gh-107565: Update Windows build to use OpenSSL 3.0.10 (GH-108932) 2023-09-05 16:54:23 +00:00
Serhiy Storchaka 492e1ffdce
[3.11] gh-89392: Remove support of test_main() in libregrtest (GH-108876) (GH-108898)
(cherry picked from commit 04a0830b00)
2023-09-05 06:54:12 +00:00
Victor Stinner 79f7a4c0a4
[3.11] gh-108822: Backport libregrtest changes from the main branch (#108820)
* Revert "[3.11] gh-101634: regrtest reports decoding error as failed test (#106169) (#106175)"

This reverts commit d5418e97fc.

* Revert "[3.11] bpo-46523: fix tests rerun when `setUp[Class|Module]` fails (GH-30895) (GH-103342)"

This reverts commit ecb09a8496.

* Revert "gh-95027: Fix regrtest stdout encoding on Windows (GH-98492)"

This reverts commit b2aa28eec5.

* Revert "[3.11] gh-94026: Buffer regrtest worker stdout in temporary file (GH-94253) (GH-94408)"

This reverts commit 0122ab235b.

* Revert "Run Tools/scripts/reindent.py (GH-94225)"

This reverts commit f0f3a424af.

* Revert "gh-94052: Don't re-run failed tests with --python option (GH-94054)"

This reverts commit 1347607db1.

* Revert "[3.11] gh-84461: Fix Emscripten umask and permission issues (GH-94002) (GH-94006)"

This reverts commit 1073184918.

* gh-93353: regrtest checks for leaked temporary files (#93776)

When running tests with -jN, create a temporary directory per process
and mark a test as "environment changed" if a test leaks a temporary
file or directory.

(cherry picked from commit e566ce5496)

* gh-93353: Fix regrtest for -jN with N >= 2 (GH-93813)

(cherry picked from commit 36934a16e8)

* gh-93353: regrtest supports checking tmp files with -j2 (#93909)

regrtest now also implements checking for leaked temporary files and
directories when using -jN for N >= 2. Use tempfile.mkdtemp() to
create the temporary directory. Skip this check on WASI.

(cherry picked from commit 4f85cec9e2)

* gh-84461: Fix Emscripten umask and permission issues (GH-94002)

- Emscripten's default umask is too strict, see
  https://github.com/emscripten-core/emscripten/issues/17269
- getuid/getgid and geteuid/getegid are stubs that always return 0
  (root). Disable effective uid/gid syscalls and fix tests that use
  chmod() current user.
- Cannot drop X bit from directory.

(cherry picked from commit 2702e408fd)

* gh-94052: Don't re-run failed tests with --python option (#94054)

(cherry picked from commit 0ff7b996f5)

* Run Tools/scripts/reindent.py (#94225)

Reindent files which were not properly formatted (PEP 8: 4 spaces).

Remove also some trailing spaces.

(cherry picked from commit e87ada48a9)

* gh-94026: Buffer regrtest worker stdout in temporary file (GH-94253)

Co-authored-by: Victor Stinner <vstinner@python.org>
(cherry picked from commit 199ba23324)

* gh-96465: Clear fractions hash lru_cache under refleak testing (GH-96689)

Automerge-Triggered-By: GH:zware
(cherry picked from commit 9c8f379433)

* gh-95027: Fix regrtest stdout encoding on Windows (#98492)

On Windows, when the Python test suite is run with the -jN option,
the ANSI code page is now used as the encoding for the stdout
temporary file, rather than using UTF-8 which can lead to decoding
errors.

(cherry picked from commit ec1f6f5f13)

* gh-98903: Test suite fails with exit code 4 if no tests ran (#98904)

The Python test suite now fails wit exit code 4 if no tests ran. It
should help detecting typos in test names and test methods.

* Add "EXITCODE_" constants to Lib/test/libregrtest/main.py.
* Fix a typo: "NO TEST RUN" becomes "NO TESTS RAN"

(cherry picked from commit c76db37c0d)

* gh-100086: Add build info to test.libregrtest (#100093)

The Python test runner (libregrtest) now logs Python build information like
"debug" vs "release" build, or LTO and PGO optimizations.

(cherry picked from commit 3c89202247)

* bpo-46523: fix tests rerun when `setUp[Class|Module]` fails (#30895)

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
(cherry picked from commit 995386071f)

* gh-82054: allow test runner to split test_asyncio to execute in parallel by sharding. (#103927)

This runs test_asyncio sub-tests in parallel using sharding from Cinder. This suite is typically the longest-pole in runs because it is a test package with a lot of further sub-tests otherwise run serially. By breaking out the sub-tests as independent modules we can run a lot more in parallel.

After porting we can see the direct impact on a multicore system.

Without this change:
  Running make test is 5 min 26 seconds
With this change:
  Running make test takes 3 min 39 seconds

That'll vary based on system and parallelism. On a `-j 4` run similar to what CI and buildbot systems often do, it reduced the overall test suite completion latency by 10%.

The drawbacks are that this implementation is hacky and due to the sorting of the tests it obscures when the asyncio tests occur and involves changing CPython test infrastructure but, the wall time saved it is worth it, especially in low-core count CI runs as it pulls a long tail. The win for productivity and reserved CI resource usage is significant.

Future tests that deserve to be refactored into split up suites to benefit from are test_concurrent_futures and the way the _test_multiprocessing suite gets run for all start methods. As exposed by passing the -o flag to python -m test to get a list of the 10 longest running tests.

---------

Co-authored-by: Carl Meyer <carl@oddbird.net>
Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google, LLC]
(cherry picked from commit 9e011e7c77)

* Display the sanitizer config in the regrtest header. (#105301)

Display the sanitizers present in libregrtest.

Having this in the CI output for tests with the relevant environment
variable displayed will help make it easier to do what we need to
create an equivalent local test run.

(cherry picked from commit 852348ab65)

* gh-101634: regrtest reports decoding error as failed test (#106169)

When running the Python test suite with -jN option, if a worker stdout
cannot be decoded from the locale encoding report a failed testn so the
exitcode is non-zero.

(cherry picked from commit 2ac3eec103)

* gh-108223: test.pythoninfo and libregrtest log Py_NOGIL (#108238)

Enable with --disable-gil --without-pydebug:

    $ make pythoninfo|grep NOGIL
    sysconfig[Py_NOGIL]: 1

    $ ./python -m test
    ...
    == Python build: nogil debug
    ...

(cherry picked from commit 5afe0c17ca)

* gh-90791: test.pythoninfo logs ASAN_OPTIONS env var (#108289)

* Cleanup libregrtest code logging ASAN_OPTIONS.
* Fix a typo on "ASAN_OPTIONS" vs "MSAN_OPTIONS".

(cherry picked from commit 3a1ac87f8f)

* gh-108388: regrtest splits test_asyncio package (#108393)

Currently, test_asyncio package is only splitted into sub-tests when
using command "./python -m test". With this change, it's also
splitted when passing it on the command line:
"./python -m test test_asyncio".

Remove the concept of "STDTESTS". Python is now mature enough to not
have to bother with that anymore. Removing STDTESTS simplify the
code.

(cherry picked from commit 174e9da083)

* regrtest computes statistics (#108793)

test_netrc, test_pep646_syntax and test_xml_etree now return results
in the test_main() function.

Changes:

* Rewrite TestResult as a dataclass with a new State class.
* Add test.support.TestStats class and Regrtest.stats_dict attribute.
* libregrtest.runtest functions now modify a TestResult instance
  in-place.
* libregrtest summary lists the number of run tests and skipped
  tests, and denied resources.
* Add TestResult.has_meaningful_duration() method.
* Compute TestResult duration in the upper function.
* Use time.perf_counter() instead of time.monotonic().
* Regrtest: rename 'resource_denieds' attribute to 'resource_denied'.
* Rename CHILD_ERROR to MULTIPROCESSING_ERROR.
* Use match/case syntadx to have different code depending on the
  test state.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
(cherry picked from commit d4e534cbb3)

* gh-108822: Add Changelog entry for regrtest statistics (#108821)

---------

Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Zachary Ware <zach@python.org>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Co-authored-by: Joshua Herman <zitterbewegung@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-09-03 19:21:53 +02:00
Miss Islington (bot) b30c83eb21
[3.11] [3.12] gh-63760: Don't declare gethostname() on Solaris (GH-108817) (GH-108824) (#108832)
[3.12] gh-63760: Don't declare gethostname() on Solaris (GH-108817) (GH-108824)

gh-63760: Don't declare gethostname() on Solaris (GH-108817)

Since 2005, Solaris defines gethostname(). socketmodule.c no longer
has to define gethostname() for Solaris.

Oracle Solaris and OpenSolaris have patches to remove the
gethostname() definition in Python:

* https://github.com/oracle/solaris-userland/blob/master/components/python/python37/patches/15-gethostname.patch
* https://github.com/OpenIndiana/oi-userland/blob/oi/hipster/components/python/python37/patches/15-gethostname.patch
* https://github.com/omniosorg/omnios-build/blob/master/build/python27/patches/24-gethostname.patch

(cherry picked from commit 7269916cd7)

Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Jakub Kulík <Kulikjak@gmail.com>
(cherry picked from commit 0e6d582b3b)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-09-03 08:53:02 +02:00
Gregory P. Smith 6ba1234c1c
[3.11] gh-104372: Drop the GIL around the vfork() call. (#104782) (#104958)
gh-104372: Drop the GIL around the vfork() call. (#104782)

On Linux where the `subprocess` module can use the `vfork` syscall for
faster spawning, prevent the parent process from blocking other threads
by dropping the GIL while it waits for the vfork'ed child process `exec`
outcome.  This prevents spawning a binary from a slow filesystem from
blocking the rest of the application.

Fixes #104372.

(cherry picked from commit d08679212d)
2023-09-01 08:53:06 +00:00
Miss Islington (bot) 385b1952f0
[3.11] gh-108520: Fix bad fork detection in nested multiprocessing use case (GH-108568) (#108692)
gh-107275 introduced a regression where a SemLock would fail being passed along nested child processes, as the `is_fork_ctx` attribute would be left missing after the first deserialization.

---------

(cherry picked from commit add8d45cbe)

Co-authored-by: albanD <desmaison.alban@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-08-30 22:36:26 +02:00
Erlend E. Aasland 6f24420cbf
[3.11] gh-108590: Revert gh-108657 (commit 400a1cebc) (#108686) (#108694)
(cherry picked from commit 2a3926fa51)

Reverted per Serhiy's request.
2023-08-30 20:12:10 +00:00
Corvin 5a6d1238bb
[3.11] gh-108590: Fix sqlite3.iterdump for invalid Unicode in TEXT columns (GH-108657) (#108674)
(cherry picked from commit 400a1cebc7)
2023-08-30 12:29:33 +02:00
Erlend E. Aasland 33eee4340e
[3.11] gh-64662: Fix virtual table support in sqlite3.Connection.iterdump (#108340) (#108564)
(cherry picked from commit d0160c7c22)

Co-authored-by: Aviv Palivoda <palaviv@gmail.com>
2023-08-28 13:09:33 +00:00
Erlend E. Aasland 73f89b1458
[3.11] gh-108542: Fix incorrect module name in NEWS entry for gh-105475 (#108543) (#108544)
(cherry picked from commit a429eafef2)
2023-08-27 20:07:56 +00:00
Serhiy Storchaka b9fc536399
[3.11] gh-107913: Fix possible losses of OSError error codes (GH-107930) (GH-108524)
Functions like PyErr_SetFromErrno() and SetFromWindowsErr() should be
called immediately after using the C API which sets errno or the Windows
error code.
(cherry picked from commit 2b15536fa9)
2023-08-27 12:18:58 +00:00
Miss Islington (bot) 8a275f7c01
[3.11] gh-105052:update timeit function's description (GH-105060) (#108535)
---------
Co-authored-by: Terry Jan Reedy tjreedy@udel.edu
Co-authored-by: R <cherrymelon@foxmail.com>
(cherry picked from commit 7096a2be33)
2023-08-27 03:57:32 -04:00
Pablo Galindo cce6ba91b3 Python 3.11.5 2023-08-24 13:09:18 +01:00
Łukasz Langa c3d129f07e
[3.11] Revert "gh-46376: Return existing pointer when possible in ctypes (GH-107131) (GH-107488)" (#108412)
This reverts commit 57f27e4441.

The fix caused gh-107940. Until we have a bulletproof fix for that, the 3.11 backport needs to be reverted to make way for 3.11.5.
2023-08-24 13:26:19 +02:00
Miss Islington (bot) 34ef75d3ef
[3.11] gh-77377: Ensure multiprocessing SemLock is valid for spawn-based Process before serializing it (GH-107275) (#108378)
gh-77377: Ensure multiprocessing SemLock is valid for spawn-based Process before serializing it (GH-107275)

Ensure multiprocessing SemLock is valid for spawn Process before serializing it.

Creating a multiprocessing SemLock with a fork context, and then trying to pass it to a spawn-created Process, would segfault if not detected early.

---------

(cherry picked from commit 1700d34d31)

Co-authored-by: albanD <desmaison.alban@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-08-23 21:11:20 +00:00
Victor Stinner 1aff195903
[3.11] gh-105776: Fix test_cppext when CC contains -std=c11 option (#108343) (#108347)
gh-105776: Fix test_cppext when CC contains -std=c11 option (#108343)

Fix test_cppext when the C compiler command has the "-std=c11" option.
Remove "-std=" options from the compiler command.

(cherry picked from commit 9173b2bbe1)
2023-08-23 03:47:41 +00:00
Serhiy Storchaka 3ca9264aba
[3.11] gh-107298: Fix numerous ref errors and typos in the C API docs (GH-108258) (#108290)
(cherry picked from commit d7202e4879)
2023-08-22 20:33:07 +02:00
Łukasz Langa 75a875e0df
[3.11] gh-108310: Fix CVE-2023-40217: Check for & avoid the ssl pre-close flaw (#108317)
gh-108310: Fix CVE-2023-40217: Check for & avoid the ssl pre-close flaw

Instances of `ssl.SSLSocket` were vulnerable to a bypass of the TLS handshake
and included protections (like certificate verification) and treating sent
unencrypted data as if it were post-handshake TLS encrypted data.

The vulnerability is caused when a socket is connected, data is sent by the
malicious peer and stored in a buffer, and then the malicious peer closes the
socket within a small timing window before the other peers’ TLS handshake can
begin. After this sequence of events the closed socket will not immediately
attempt a TLS handshake due to not being connected but will also allow the
buffered data to be read as if a successful TLS handshake had occurred.

Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
2023-08-22 19:53:19 +02:00
Miss Islington (bot) 8927cf0200
[3.11] gh-106242: Make ntpath.realpath errors consistent with abspath when there are embedded nulls (GH-108248)
gh-106242: Make ntpath.realpath errors consistent with abspath when there are embedded nulls (GH-108248)

---------

(cherry picked from commit de33b5c662)

Co-authored-by: Steve Dower <steve.dower@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-08-22 15:35:16 +01:00
Miss Islington (bot) 8e837373ed
[3.11] gh-107845: Fix symlink handling for tarfile.data_filter (GH-107846) (GH-108209)
gh-107845: Fix symlink handling for tarfile.data_filter (GH-107846)

(cherry picked from commit acbd3f9c5c)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>
2023-08-22 10:51:58 +02:00
Miss Islington (bot) 145d9252b7
[3.11] gh-107298: Fix references to deprecated and removed PyUnicode C API (GH-108077) (GH-108226)
(cherry picked from commit db55383829)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-21 20:35:36 +03:00
Miss Islington (bot) 75617ac3f0
[3.11] gh-107396: tarfiles: set self.exception before _init_read_gz() (GH-107485) (GH-108208)
gh-107396: tarfiles: set self.exception before _init_read_gz() (GH-107485)

In the stack call of: _init_read_gz()
```
_read, tarfile.py:548
read, tarfile.py:526
_init_read_gz, tarfile.py:491
```
a try;except exists that uses `self.exception`, so it needs to be set before
calling _init_read_gz().
(cherry picked from commit 37135d25e2)

Co-authored-by: balmeida-nokia <83089745+balmeida-nokia@users.noreply.github.com>
2023-08-21 17:16:59 +02:00
Miss Islington (bot) ed67e60f48
[3.11] gh-107916: Save the error code before decoding the filename in PyErr_SetFromErrnoWithFilename() etc (GH-107929) (GH-108206)
(cherry picked from commit 80bdebdd85)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-21 11:53:05 +00:00
Miss Islington (bot) a1d2e2c2ac
[3.11] gh-105736: Sync pure python version of OrderedDict with the C version (GH-108098) (GH-108201)
(cherry picked from commit 20cc90c0df)

Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>
2023-08-21 10:37:09 +00:00
Miss Islington (bot) f8b14fea33
[3.11] gh-107565: Update macOS installer to use OpenSSL 3.0.10. (GH-107897) (#108122) 2023-08-19 19:02:12 -04:00
Serhiy Storchaka 92a578409b
[3.11] gh-107915: Handle errors in C API functions PyErr_Set*() and PyErr_Format() (GH-107918) (GH-108135)
Such C API functions as PyErr_SetString(), PyErr_Format(),
PyErr_SetFromErrnoWithFilename() and many others no longer crash or
ignore errors if it failed to format the error message or decode the
filename. Instead, they keep a corresponding error.
(cherry picked from commit 633ea217a8)
2023-08-19 12:22:13 +00:00
Ned Deily 441797d4ff
[3.11] gh-107565: Update multissltests and GitHub CI workflows to use OpenSSL 1.1.1v, 3.0.10, and 3.1.2. (GH-108119) 2023-08-18 20:50:11 +00:00
Miss Islington (bot) c1c3f0bfec
[3.11] bpo-18319: gettext() can retrieve a message even if a plural form exists (GH-19869) (GH-107107)
(cherry picked from commit 54632528ee)

Co-authored-by: Gilles Bassière <gbassiere@gmail.com>
2023-08-16 09:15:01 +00:00
Serhiy Storchaka 26137e2cf7
[3.11] gh-100061: Proper fix of the bug in the matching of possessive quantifiers (GH-102612) (GH-108004)
Restore the global Input Stream pointer after trying to match a sub-pattern.

Co-authored-by: Ma Lin <animalize@users.noreply.github.com>

(cherry picked from commit abd9cc52d9)
    
Co-authored-by: SKO <41810398+uyw4687@users.noreply.github.com>
2023-08-16 08:36:36 +00:00
Miss Islington (bot) af08bcab8f
[3.11] gh-100814: Fix exception for invalid callable value of Tkinter image option (GH-107692) (GH-107723)
Passing a callable object as an option value to a Tkinter image now raises
the expected TclError instead of an AttributeError.
(cherry picked from commit 50e3cc9748)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-16 11:07:02 +03:00
Serhiy Storchaka 4cfbcffd3d
[3.11] gh-99203: shutil.make_archive(): restore select CPython <= 3.10.5 behavior (GH-99802) (GH-107999)
Restore following CPython <= 3.10.5 behavior of shutil.make_archive()
that went away as part of gh-93160:

Do not create an empty archive if root_dir is not a directory, and, in
that case, raise FileNotFoundError or NotADirectoryError regardless
of format choice. Beyond the brought-back behavior, the function may
now also raise these exceptions in dry_run mode.
(cherry picked from commit a86df298df)

Co-authored-by: 6t8k <58048945+6t8k@users.noreply.github.com>
2023-08-16 07:44:21 +00:00
Steve Dower ccf81e1088
[3.11] gh-106242: Fix path truncation in os.path.normpath (GH-106816) (#107982)
Co-authored-by: Finn Womack <flan313@gmail.com>
2023-08-15 19:07:52 +02:00
Miss Islington (bot) db4400b5b2
[3.11] gh-107963: Fix set_forkserver_preload to check the type of given list (GH-107965) (gh-107976)
gh-107963: Fix set_forkserver_preload to check the type of given list (GH-107965)
(cherry picked from commit 6515ec3d3d)


gh-107963: Fix set_forkserver_preload to check the type of given list

Co-authored-by: Dong-hee Na <donghee.na@python.org>
2023-08-15 23:53:25 +09:00
Miss Islington (bot) 5f36e5ca5f
[3.11] gh-107715: Escape class name in regular expression (GH-107716) (GH-107727)
This patch escapes the class name before embedding it in the regular expression
for `pat` in `doctest.DocTestFinder._find_lineno`. While class names do not
ordinarily contain special characters, it is possible to encounter these when a
class is created dynamically. Escaping the name will correctly return `None` in
this scenario, rather than potentially matching a different class or raising
`re.error` depending on the symbols used.
(cherry picked from commit 8579327879)

Co-authored-by: Gertjan van Zwieten <git@gjvz.nl>
2023-08-12 11:57:49 +00:00
Serhiy Storchaka ec254c5dfa
[3.11] gh-106844: Fix issues in _winapi.LCMapStringEx (GH-107832) (GH-107875)
* Strings with length from 2**31-1 to 2**32-2 always caused MemoryError,
   it doesn't matter how much memory is available.
* Strings with length exactly 2**32-1 caused OSError.
* Strings longer than 2**32-1 characters were truncated due to integer overflow bug.

Now strings longer than 2**31-1 characters caused OverflowError.
(cherry picked from commit 04cc01453d)
2023-08-12 14:06:56 +03:00
Miss Islington (bot) fb08b7905e
[3.11] [3.12] GH-106684: Close `asyncio.StreamWriter` when `asyncio.StreamWriter` is not closed by application (GH-107650) (GH-107656) (#107836)
[3.12] GH-106684:  Close `asyncio.StreamWriter` when `asyncio.StreamWriter` is not closed by application (GH-107650) (GH-107656)

GH-106684: raise `ResourceWarning` when `asyncio.StreamWriter` is not closed (GH-107650)
(cherry picked from commit 41178e4199)

(cherry picked from commit 7853c76906)

Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2023-08-10 16:14:31 +05:30
Miss Islington (bot) eff2042fac
[3.11] gh-107814: Avoid output from Nuget installation in find_python.bat (GH-107815)
gh-107814: Avoid output from Nuget installation in find_python.bat (GH-107815)
(cherry picked from commit 1e229e2c3d)

Co-authored-by: Max Bachmann <kontakt@maxbachmann.de>
2023-08-09 23:15:43 +00:00
Erlend E. Aasland b0b26af822
[3.11] gh-86457: Add docs for Argument Clinic @text_signature directive (#107747) (#107799)
(cherry picked from commit a9aeb99579)

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-09 09:25:25 +02:00
Serhiy Storchaka 5b76eaf02e
[3.11] gh-106052: Fix bug in the matching of possessive quantifiers (GH-106515) (GH-107795)
It did not work in the case of a subpattern containing backtracking.

Temporary implement possessive quantifiers as equivalent greedy qualifiers
in atomic groups.
(cherry picked from commit 7b6e34e5ba)
2023-08-09 06:15:27 +00:00
Serhiy Storchaka edaa0db93e
[3.11] gh-86457: Fix signature for code.replace() (GH-23199) (GH-107746)
Also add support of @text_signature in Argument Clinic.
(cherry picked from commit 0e6e32fb84)
2023-08-09 06:12:02 +00:00
Miss Islington (bot) 4f65f03f2d
[3.11] gh-107077: Raise SSLCertVerificationError even if the error is set via SSL_ERROR_SYSCALL (GH-107586) (#107588)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: T. Wouters <thomas@python.org>
2023-08-03 15:09:27 +01:00
Miss Islington (bot) 46cae02085
[3.11] gh-106092: Fix use-after-free crash in frame_dealloc (GH-106875) (#107533) 2023-08-01 11:21:08 +01:00
Charlie Zhao a15d06c230
[3.11] gh-106263: Fix segfault in `signaldict_repr` in `_decimal` module (#… (#107490)
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
(cherry picked from commit 3979150a0d)
2023-07-31 15:52:23 +02:00
Łukasz Langa 57f27e4441
[3.11] gh-46376: Return existing pointer when possible in ctypes (GH-107131) (#107488)
(cherry picked from commit 08447b5deb)

Co-authored-by: Konstantin <kpp.live+github@gmail.com>
2023-07-31 11:16:59 +02:00
Ned Deily b8beb07620
[3.11] gh-99079: Update macOS installer to use OpenSSL 3.0.9. (GH-107476) 2023-07-31 06:16:39 +00:00
Miss Islington (bot) ddccdbfc40
[3.11] gh-106634: Corrected minor asyncio doc issues (GH-106671) (#106711)
gh-106634: Corrected minor asyncio doc issues (GH-106671)
(cherry picked from commit 4b4a5b70aa)

Co-authored-by: Chris Brett <chrisbrett665@gmail.com>
2023-07-29 18:07:49 +00:00
Miss Islington (bot) 3c1bcae075
[3.11] gh-101100: Sphinx warnings: pick the low hanging fruits (GH-107386) (GH-107418)
(cherry picked from commit f2d07d3289)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-07-29 09:17:20 +03:00
Serhiy Storchaka 733a2a9046
[3.11] gh-107091: Fix some uses of :func: role (GH-107378) (GH-107417)
:c:func: or :c:macro: should be used instead.
(cherry picked from commit 413ba8943e)
2023-07-29 05:56:26 +00:00
justdan6 4049c5d6f8
[3.11] gh-106881: Check for linux/limits.h before including it (#107397) (#107415)
* [3.11] gh-106881: Check for linux/limits.h before including it (#107397)

* Check for linux/limits.h before including it

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
(cherry picked from commit 11c055f5ff)

* Fix sphinx-lint error in NEWS entry
2023-07-28 23:36:54 +00:00
Miss Islington (bot) c7d9976094
[3.11] gh-104432: Use `memcpy()` to avoid misaligned loads (GH-104433) (#107356)
gh-104432: Use `memcpy()` to avoid misaligned loads (GH-104433)

Fix potential unaligned memory access on C APIs involving returned sequences
of `char *` pointers within the :mod:`grp` and :mod:`socket` modules. These
were revealed using a ``-fsaniziter=alignment`` build on ARM macOS.
(cherry picked from commit f01e4cedba)

Co-authored-by: Christopher Chavez <chrischavez@gmx.us>
2023-07-28 06:36:45 +00:00
Miss Islington (bot) 8492aae9bd
[3.11] gh-106723: forward -Xfrozen_modules option to spawned process interpreters (GH-106724) (#107368)
gh-106723: forward -Xfrozen_modules option to spawned process interpreters (GH-106724)
(cherry picked from commit 3dcac78581)

Co-authored-by: Felipe A. Hernandez <ergoithz@gmail.com>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-07-28 11:20:56 +05:30
Miss Islington (bot) 603c5d50e8
[3.11] gh-107091: Fix some uses of :c:type: role (GH-107138) (GH-107313)
(cherry picked from commit 6d5b6e71c8)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-07-27 00:00:36 +03:00
Miss Islington (bot) 36208b5f6a
[3.11] gh-106350: Tkinter: do not ignore return value of `mp_init()` (GH-106351) (GH-107259)
(cherry picked from commit b5ae7c4984)

Co-authored-by: Christopher Chavez <chrischavez@gmx.us>
2023-07-26 10:58:07 +03:00
Miss Islington (bot) 2ce8e133d0
[3.11] gh-107226: PyModule_AddObjectRef() should only be in the limited API 3.10 (GH-107227) (GH-107261)
(cherry picked from commit 698b015135)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-07-25 23:02:06 +03:00
Miss Islington (bot) 41756e3960
[3.11] gh-62519: Make pgettext search plurals when translation is not found (GH-107118) (GH-107133)
(cherry picked from commit b3c34e55c0)

Co-authored-by: Tomas R <tomas.roun8@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-07-25 21:49:28 +03:00
Victor Stinner db03cb953a
[3.11] gh-107237: Fix test_udp_reconnection() of test_logging (#107238) (#107245)
gh-107237: Fix test_udp_reconnection() of test_logging (#107238)

test_logging: Fix test_udp_reconnection() by increasing the timeout
from 100 ms to 5 minutes (LONG_TIMEOUT).

Replace also blocking wait() with wait(LONG_TIMEOUT) in
test_output() to prevent the test to hang.

(cherry picked from commit ed08238327)
2023-07-25 14:49:32 +00:00
Serhiy Storchaka b8b3e6afc0
[3.11] gh-99612: Fix PyUnicode_DecodeUTF8Stateful() for ASCII-only data (GH-99613) (GH-107224)
Previously *consumed was not set in this case.
(cherry picked from commit f08e52ccb0)
2023-07-25 14:15:14 +03:00
Paul Moore 058741cc39
[3.11] gh-106774: Update bundled pip version to 23.2.1 (GH-106775) (gh-107223)
* Update bundled pip version to 23.2.1.
(cherry picked from commit f443b54a2f)
2023-07-25 11:38:19 +01:00
Erlend E. Aasland b3e600a632
[3.11] gh-106948: Docs: Disable links for C standard library functions, OS utility functions and system calls (#107062) (#107157)
(cherry picked from commit b447e19e72)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-07-23 23:49:48 +02:00
Miss Islington (bot) afa24d52b8
[3.11] gh-106186: Don't report MultipartInvariantViolationDefect for valid multipart emails when parsing header only (GH-107016) (#107112)
(cherry picked from commit c65592c4d6)

Co-authored-by: htsedebenham <31847376+htsedebenham@users.noreply.github.com>
2023-07-23 14:28:35 +02:00
Serhiy Storchaka d2cdf0888b
[3.11] gh-107091: Fix the use of some C domain roles (GH-107092) (GH-107121)
(cherry picked from commit 08a228da05)
2023-07-23 15:06:32 +03:00
Serhiy Storchaka ec8718dd7c
[3.11] gh-106948: Add standard external names to nitpick_ignore (GH-106949) (#107061)
It includes standard C types, macros and variables like "size_t",
"LONG_MAX" and "errno", and standard environment variables like "PATH"..
(cherry picked from commit f8b7fe2f26)
2023-07-23 11:24:37 +02:00
Miss Islington (bot) 65e40aaed0
[3.11] gh-54738: Add argparse i18n howto (GH-104562) (#107101)
(cherry picked from commit dcd7acb04a)

Co-authored-by: Tomas R <tomas.roun8@gmail.com>
2023-07-23 11:23:51 +02:00
Miss Islington (bot) 2895dbd0f6
[3.11] gh-83006: Document behavior of `shutil.disk_usage` for non-mounted filesystems on Unix (GH-107031) (#107048)
(cherry picked from commit 6e5f2235f3)

Co-authored-by: Matthieu Caneill <matthieucan@users.noreply.github.com>
2023-07-22 20:25:15 +02:00
Miss Islington (bot) e5d2a19ac5
[3.11] gh-107008: Document the curses module variables LINES and COLS (GH-107011) (GH-107058)
LINES and COLS referred in curses.update_lines_cols() documentations are
the module variables, not the environment variables.
(cherry picked from commit 26e08dfdd7)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-07-22 21:13:38 +03:00
Erlend E. Aasland 63c945a3f5
[3.11] gh-106970: Fix Argument Clinic 'destination <name> clear' command (#106972) (#107059)
Add test for the 'destination <name> clear' command,
and the 'destination' directive in general.

Fix two bugs in 'destination <name> clear' command:

1. The text attribute of the allocator is called 'text', not '_text'
2. Return after processing the 'clear' command,
   instead of proceeding directly to the fail().

(cherry picked from commit 3372bcba98)
2023-07-22 17:44:11 +00:00
Serhiy Storchaka b338ac7540
[3.11] gh-106909: Use role :const: for referencing module constants (GH-106910) (GH-106957)
(cherry picked from commit 4b9948617f)
2023-07-21 14:49:57 +03:00
Serhiy Storchaka cc76113cf8
[3.11] gh-106892: Use roles :data: and :const: for referencing module variables (GH-106894) (GH-106955)
(cherry picked from commit d036db728e)
2023-07-21 14:49:21 +03:00
Serhiy Storchaka 6bde1b9f04
[3.11] gh-106919: Use role :c:macro: for referencing the C "constants" (GH-106920) (GH-106952)
(cherry picked from commit fcc816dbff)
2023-07-21 14:48:30 +03:00
Serhiy Storchaka fb04874053
[3.11] gh-106719: Fix __annotations__ getter and setter in the type and module types (GH-106720) (GH-106850)
No longer suppress arbitrary errors. Simplify the code.
(cherry picked from commit e1c295e3da)
2023-07-18 10:19:45 +03:00
Miss Islington (bot) a782d51913
[3.11] gh-106831: Fix NULL check of d2i_SSL_SESSION() result in _ssl.c (GH-106832) (#106836)
gh-106831: Fix NULL check of d2i_SSL_SESSION() result in _ssl.c (GH-106832)
(cherry picked from commit ebf2c56b33)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2023-07-17 20:31:10 +00:00
Zachary Ware 263d8aa017
[3.11] gh-99079: Update Windows build to use OpenSSL 3.0.9 (GH-106649) (GH-106761)
Co-authored-by: Steve Dower <steve.dower@python.org>
2023-07-17 12:07:52 -05:00
Jason R. Coombs 465f5b09df
[3.11] gh-106752: Sync with zipp 3.16.2 (GH-106757) (#106778)
* gh-106752: Sync with zipp 3.16.2 (#106757)

* gh-106752: Sync with zipp 3.16.2

* Add blurb

(cherry picked from commit 22980dc7c9)

* [3.11] gh-106752: Sync with zipp 3.16.2 (GH-106757)

* gh-106752: Sync with zipp 3.16.2

* Add blurb.
(cherry picked from commit 22980dc7c9)

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>

* Remove Python 3.12 concerns from changelog.
2023-07-15 10:15:38 -04:00
Dennis Sweeney d488970ae6
[3.11] gh-105235: Prevent reading outside buffer during mmap.find() (… (#106710)
[3.11] gh-105235: Prevent reading outside buffer during mmap.find() (GH-105252)

* Add a special case for s[-m:] == p in _PyBytes_Find

* Add tests for _PyBytes_Find

* Make sure that start <= end in mmap.find.
(cherry picked from commit ab86426a34)
2023-07-14 22:17:09 -04:00
Miss Islington (bot) a276ce4505
[3.11] gh-106602: [Enum] Add __copy__ and __deepcopy__ (GH-106694)
gh-106602: [Enum] Add __copy__ and __deepcopy__ (GH-106666)
(cherry picked from commit 357e9e9da3)

Co-authored-by: Prince Roshan <princekrroshan01@gmail.com>
2023-07-12 15:48:16 -07:00
Miss Islington (bot) da15b12d8d
[3.11] gh-106498: Revert incorrect colorsys.rgb_to_hls change (GH-106627) (#106633)
gh-106498: Revert incorrect colorsys.rgb_to_hls change (GH-106627)

gh-86618 assumed a-b-c = a-(b+c) = a-d where d = b+d.
For floats 2.0, 1.0, and 0.9999999999999999, this assumption
is false.  The net change of 1.1102230246251565e-16 to 0.0
results in division by 0.  Revert the replacement.  Add test.
(cherry picked from commit a2d54d4e8a)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2023-07-11 15:48:00 +00:00
Miss Islington (bot) 318f6ae1cd
[3.11] gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist (GH-106468) (#106621)
gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist (GH-106468)

For example:

    class Flag(enum.Flag):
        A = 0x01
        B = 0x02
        MASK = 0xff

    ~Flag.MASK is Flag(0)
(cherry picked from commit 95b7426f45)

Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
2023-07-11 14:10:02 +02:00
Miss Islington (bot) d63953d0f5
[3.11] gh-94777: Fix deadlock in ProcessPoolExecutor (GH-94784) (#106607)
gh-94777: Fix deadlock in ProcessPoolExecutor (GH-94784)

Fixes a hang in multiprocessing process pool executor when a child process crashes and code could otherwise block on writing to the pipe.  See GH-94777 for more details.
(cherry picked from commit 6782fc0502)

Co-authored-by: Louis Paulot <55740424+lpaulot@users.noreply.github.com>
2023-07-10 16:21:04 -06:00
Miss Islington (bot) 2d037fb406
[3.11] gh-106510: Fix DEBUG output for atomic group (GH-106511) (GH-106549)
(cherry picked from commit 74ec02e949)
2023-07-08 15:15:22 +03:00
Miss Islington (bot) 1931c2a438
[3.11] gh-90876: Restore the ability to import multiprocessing when `sys.executable` is `None` (GH-106464) (#106495)
gh-90876: Restore the ability to import multiprocessing when `sys.executable` is `None` (GH-106464)

Prevent `multiprocessing.spawn` from failing to *import* in environments
where `sys.executable` is `None`.  This regressed in 3.11 with the addition
of support for path-like objects in multiprocessing.

Adds a test decorator to have tests only run when part of test_multiprocessing_spawn to `_test_multiprocessing.py` so we can start to avoid re-running the same not-global-state specific test in all 3 modes when there is no need.
(cherry picked from commit c60df361ce)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-07-06 16:25:20 -07:00
Miss Islington (bot) 3f244b2f6e
[3.11] gh-105497: [Enum] Fix Flag inversion when alias/mask members exist. (GH-105542) (#105571)
When inverting a Flag member (or boundary STRICT), only consider other canonical flags; when inverting an IntFlag member (or boundary KEEP), also consider aliases.
(cherry picked from commit 59f009e589)

Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
2023-07-05 13:59:18 +02:00
Miss Islington (bot) 23b731a10d
[3.11] GH-104554: Add RTSPS support to `urllib/parse.py` (GH-104605) (#105760)
RTSPS is the permanent scheme defined in
https://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml
alongside RTSP and RTSPU schemes.

---------

(cherry picked from commit f3266c05b6)

Co-authored-by: zentarim <33746047+zentarim@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-07-05 13:02:36 +02:00
Jeffery To 6c6a2a9143
[3.11] gh-104692: Include commoninstall as a prerequisite for bininstall (GH-104693) (#105429)
This ensures that `commoninstall` is completed before `bininstall` is
started when parallel builds are used (`make -j install`), and so the
`python3` symlink is only installed after all standard library modules
are installed.

.
(cherry picked from commit 990cb3676c)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-07-05 12:51:26 +02:00
Miss Islington (bot) 9e33586018
[3.11] gh-102541: Fix Helper.help("mod") for non-existent mod (GH-105934) (#106323)
gh-102541: Fix Helper.help("mod") for non-existent mod (GH-105934)

If the output arg to Helper() is a stream rather than the default None, which means 'page to stdout', the ImportError from pydoc.resolve is currently not caught in pydoc.doc. The same error is caught when output is None.
---------

(cherry picked from commit 0530f4f646)

Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2023-07-01 23:15:44 +00:00
Miss Islington (bot) b937ca0a5a
[3.11] gh-101006: Improve error handling when read marshal data (GH-101007) (GH-106227)
* EOFError no longer overrides other errors such as MemoryError or OSError at
  the start of the object.
* Raise more relevant error when the NULL object occurs as a code object
  component.
* Minimize an overhead of calling PyErr_Occurred().
(cherry picked from commit 8bf6904b22)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-06-29 13:47:30 +03:00
Miss Islington (bot) 966e0f6494
[3.11] gh-100340: Allows -Wno-int-conversion for wasm (GH-100341) (#106066)
(cherry picked from commit 75c8133efe)

Co-authored-by: Kushal Das <mail@kushaldas.in>
2023-06-28 09:27:59 +02:00
Victor Stinner d5418e97fc
[3.11] gh-101634: regrtest reports decoding error as failed test (#106169) (#106175)
gh-101634: regrtest reports decoding error as failed test (#106169)

When running the Python test suite with -jN option, if a worker stdout
cannot be decoded from the locale encoding report a failed testn so the
exitcode is non-zero.

(cherry picked from commit 2ac3eec103)
2023-06-28 02:58:34 +00:00
Kumar Aditya 307c59a0c1
[3.11] gh-105979: Fix exception handling in unmarshal_frozen_code (`Python/import.c`) (GH-105980). (#106100)
* [3.11] gh-105979: Fix exception handling in `unmarshal_frozen_code` (`Python/import.c`) (GH-105980).
(cherry picked from commit cd5280367a)

Co-authored-by: chgnrdv <52372310+chgnrdv@users.noreply.github.com>
2023-06-26 09:00:51 +00:00
Miss Islington (bot) 2944a6cc0a
[3.11] GH-105840: Fix assertion failures when specializing calls with too many __defaults__ (GH-105864)
GH-105840: Fix assertion failures when specializing calls with too many __defaults__ (GH-105847)
(cherry picked from commit 2beab5bdef)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
2023-06-16 18:30:22 +00:00
Miss Islington (bot) c0c4186858
[3.11] GH-105588: Add missing error checks to some obj2ast_* converters (GH-105839)
GH-105588: Add missing error checks to some obj2ast_* converters (GH-105589)
(cherry picked from commit a4056c8f9c)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
2023-06-15 23:13:51 +00:00
Dong-hee Na abeb589290
[3.11] gh-102541: Hide traceback in help prompt (gh-102614). (gh-105830)
(cherry picked from commit ba516e70c6)

Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
2023-06-15 15:29:00 +00:00
Erlend E. Aasland 2b1745f37c
[3.11] gh-105375: Improve error handling in _Unpickler_SetInputStream() (#105667) (#105721)
Prevent exceptions from possibly being overwritten in case of multiple
failures.

(cherry picked from commit 217589d4f3)
2023-06-13 09:07:20 +00:00
Miss Islington (bot) 261d0b4f0e
[3.11] gh-105436: The environment block should end with two null wchar_t values (GH-105495) (#105701)
gh-105436: The environment block should end with two null wchar_t values (GH-105495)
(cherry picked from commit 4f7d3b602d)

Co-authored-by: Dora203 <66343334+sku2000@users.noreply.github.com>
2023-06-12 16:49:57 +00:00
Erlend E. Aasland e5fe017143
[3.11] gh-105375: Harden pyexpat initialisation (#105606) (#105668)
(cherry picked from commit 20a56d8bec)

Add proper error handling to add_errors_module() to prevent exceptions
from possibly being overwritten.
2023-06-11 21:28:12 +00:00
Erlend E. Aasland f98d475ee3
[3.11] gh-105375: Improve error handling in the sys extension module (#105611) (#105666)
(cherry picked from commit 41cddc2e93)

In _PySys_AddXOptionWithError() and sys_add_xoption(),
bail on first error to prevent exceptions from possibly being
overwritten.
2023-06-11 21:08:40 +00:00
Miss Islington (bot) a03449374e
[3.11] gh-105375: Improve error handling in _ctypes (GH-105593) (#105664)
Prevent repeated PyLong_FromVoidPtr() from possibly overwriting the
current exception.
(cherry picked from commit e8998e46a7)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 20:30:47 +00:00
Miss Islington (bot) 91877478ed
[3.11] gh-105375: Improve error handling in PyUnicode_BuildEncodingMap() (GH-105491) (#105662)
Bail on first error to prevent exceptions from possibly being overwritten.
(cherry picked from commit 555be81026)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 20:01:52 +00:00
Miss Islington (bot) 05c73e1cd8
[3.11] gh-105375: Improve PyErr_WarnExplicit() error handling (GH-105610) (#105660)
Bail on first error to prevent exceptions from possibly being
overwritten.
(cherry picked from commit 567d6ae8e7)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 19:46:02 +00:00
Erlend E. Aasland 87e493b11a
[3.11] gh-105375: Harden _ssl initialisation (#105599) (#105651)
(cherry picked from commit 01f4230460)

Add proper error handling to prevent reference leaks and overwritten
exceptions.

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 21:25:36 +02:00
Miss Islington (bot) b3d95d4e61
[3.11] gh-105375: Improve error handling in the builtins extension module (GH-105585) (#105650)
(cherry picked from commit d4fa52934a)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 11:13:45 +00:00
Miss Islington (bot) cb26fafdcc
[3.11] gh-105375: Improve _decimal error handling (GH-105605) (#105648)
Fix a bug where an exception could end up being overwritten.
(cherry picked from commit c932f72849)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 10:58:56 +00:00
Miss Islington (bot) cfa0f7c59b
[3.11] gh-105375: Harden _datetime initialisation (GH-105604) (#105646)
Improve error handling so init bails on the first exception.
(cherry picked from commit 16d49680b5)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 10:41:37 +00:00
Miss Islington (bot) 3c08e54ccf
[3.11] gh-105375: Improve array.array exception handling (GH-105594) (#105643)
Fix a bug where 'tp_richcompare' could end up overwriting an exception.
(cherry picked from commit 35cff545db)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 10:22:31 +00:00
Miss Islington (bot) 4ceb5c4924
[3.11] gh-105375: Improve error handling in `zoneinfo` module (GH-105586) (#105613)
Fix bugs where exceptions could end up being overwritten
because of deferred error handling.

(cherry picked from commit 33c92c4f15)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-09 22:54:00 +00:00
Miss Islington (bot) e4748628e1
[3.11] gh-105375: Improve error handling in _elementtree (GH-105591) (#105601)
Fix bugs where exceptions could end up being overwritten.
(cherry picked from commit 00b599ab5a)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-09 21:19:59 +00:00
Miss Islington (bot) c1797f661d
[3.11] gh-105375: Improve posix error handling (GH-105592) (#105597)
Fix a bug where an IndexError could end up being overwritten.
(cherry picked from commit f668f73bc8)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-09 20:55:34 +00:00
Erlend E. Aasland 76682ba7af
[3.11] gh-105375: Improve errnomodule error handling (#105590) (#105595)
(cherry picked from commit eede1d2f48)

Bail immediately if an exception is set, to prevent exceptions from
being overwritten.
2023-06-09 20:35:30 +00:00
Erlend E. Aasland 4d4251d6ec
[3.11] gh-105375: Improve _pickle error handling (#105475) (#105583)
(cherry picked from commit 89aac6f6b7)

Error handling was deferred in some cases, which could potentially lead
to exceptions being overwritten.
2023-06-09 17:56:02 +00:00
Erlend E. Aasland 90e357bc8a
[3.11] gh-105375: Improve error handling in compiler_enter_scope() (#105494) (#105582)
(cherry picked from commit 6c832ddcf2)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-09 17:36:06 +00:00
Miss Islington (bot) fed1b5a198
[3.11] gh-105332: [Enum] Fix unpickling flags in edge-cases (GH-105348) (GH-105519)
* revert enum pickling from by-name to by-value

(cherry picked from commit 4ff5690e59)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
2023-06-08 18:02:12 -07:00
Miss Islington (bot) 571ec56b65
[3.11] gh-105375: Improve error handling in sqlite3 collation callback (GH-105412) (#105441)
Check for error after each call to PyUnicode_FromStringAndSize().
(cherry picked from commit a24a780d93)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-07 21:22:01 +00:00
Alex Waygood 18e9fd80d9
[3.11] gh-103171: Revert undocumented behaviour change for runtime-checkable protocols decorated with `@final` (#105445) 2023-06-07 21:18:21 +00:00
Pablo Galindo d2340ef257 Python 3.11.4 2023-06-06 23:00:27 +01:00
Miss Islington (bot) c644fe403a
[3.11] gh-104399: Use newer libtommath APIs when necessary (GH-104407) (#105344)
gh-104399: Use newer libtommath APIs when necessary (GH-104407)
(cherry picked from commit 00d73caf80)

Co-authored-by: Christopher Chavez <chrischavez@gmx.us>
2023-06-06 10:01:54 -04:00
Erlend E. Aasland b8d3bb72b0
[3.11] gh-90005: Don't link with libbsd if not needed (#105236) (#105359)
The regression was introduced with commit 5b946cada.
Restore pre gh-29696 behaviour.
2023-06-06 11:00:38 +00:00
Miss Islington (bot) faae548e0e
[3.11] gh-89412: Add missing attributes (added in 3.10) to traceback module docs (GH-105046) (#105328)
(cherry picked from commit a4f72fa39a)

Co-authored-by: Jakub Kuczys <me@jacken.men>
2023-06-05 19:02:23 +02:00
Jelle Zijlstra 48957888d2
[3.11] gh-105164: Detect annotations inside match blocks (GH-105177). (#105314)
(cherry picked from commit 69d1245685)
2023-06-05 13:46:12 +00:00
Tian Gao aca77b5530
[3.11] gh-105080: Fixed inconsistent signature on derived classes (GH… (#105274) 2023-06-04 15:26:01 -06:00
Gregory P. Smith a5d2b546c1
[3.11] gh-103142: Upgrade binary builds and CI to OpenSSL 1.1.1u (GH-105174) (#105200)
Upgrade builds to OpenSSL 1.1.1u.

This OpenSSL version addresses a pile if less-urgent CVEs since 1.1.1t.

The Mac/BuildScript/build-installer.py was already updated.

Also updates _ssl_data_111.h from OpenSSL 1.1.1u, _ssl_data_300.h from 3.0.9.

Manual edits to the _ssl_data_300.h file prevent it from removing any existing definitions in case those exist in some peoples builds and were important (avoiding regressions during backporting).

(cherry picked from commit ede89af)
2023-06-01 18:07:53 +00:00
Miss Islington (bot) 3b0747af90
gh-105146: Update links at end of Windows installer (uninstall/repair) (GH-105147)
(cherry picked from commit ed86e14b16)

Co-authored-by: Steve Dower <steve.dower@python.org>
2023-05-31 22:42:57 +01:00
Miss Islington (bot) 67a8469237
[3.11] gh-103142: Update macOS installer to use OpenSSL 1.1.1u. (GH-105131)
(cherry picked from commit f90d3f68db)

Co-authored-by: Ned Deily <nad@python.org>
2023-05-31 06:15:45 +00:00
Miss Islington (bot) c39500db52
[3.11] GH-89455: Add missing attributes (added in 3.11) to traceback module docs (GH-105044) (#105067)
GH-89455: Add missing attributes (added in 3.11) to traceback module docs (GH-105044)
(cherry picked from commit 39f6a0489f)

Co-authored-by: Jakub Kuczys <me@jacken.men>
2023-05-29 20:37:49 +01:00
Miss Islington (bot) 13424746ce
[3.11] gh-104943: Remove mentions of old Python versions (GH-104945) (#104962)
(cherry picked from commit 46857d0b2a)

Co-authored-by: Tomas R <tomas.roun8@gmail.com>
2023-05-26 07:13:45 +00:00
Miss Islington (bot) 3b3f2e107b
[3.11] gh-104874: Document NewType.__supertype__ (GH-104875) (#104907)
gh-104874: Document NewType.__supertype__ (GH-104875)
(cherry picked from commit 41768a2bd3)

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-25 00:35:58 +01:00
Miss Islington (bot) 22618a86f4
[3.11] gh-104719: IDLE - test existence of all tokenize references. (GH-104767) (#104845)
gh-104719: IDLE - test existence of all tokenize references. (GH-104767)

Class editor.IndentSearcher contains all editor references to tokenize module.
Module io tokenize reference cover those other modules.

(cherry picked from commit e561c09975)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-24 08:46:33 -04:00
Gregory P. Smith 6d00ae3c28
[3.11] gh-104372: Cleanup _posixsubprocess make_inheritable for async signal safety gh-104518 (#104785)
Move all of the Python C API calls into the parent process up front
instead of doing PyLong_AsLong and PyErr_Occurred and PyTuple_GET from
the post-fork/vfork child process.

Much of this was long overdue. We shouldn't have been using PyTuple and
PyLong APIs within all of these low level functions anyways.

This is a backport of c649df6 for #104518 and the tiny adjustment in d1732fe #104697.

Backporting this allows backporting of the real bug fix that requires it.

Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>
2023-05-23 20:42:25 -07:00
Miss Islington (bot) 6f94bbf77e
[3.11] gh-104536: Improve `multiprocessing.process._cleanup` logic (GH-104537) (#104737)
Fix a race condition in the internal `multiprocessing.process` cleanup
logic that could manifest as an unintended `AttributeError` when calling
`BaseProcess.close()`.

(cherry picked from commit ef5d00a592)

Co-authored-by: Luccccifer <lukezhang764@gmail.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-05-22 12:49:33 +02:00
Miss Islington (bot) e95ca78fab
[3.11] gh-102978: Fix mock.patch function signatures for class and staticmethod decorators (GH-103228) (#103499)
Fixes unittest.mock.patch not enforcing function signatures for methods
decorated with @classmethod or @staticmethod when patch is called with
autospec=True.

(cherry picked from commit 59e0de4903)

Co-authored-by: Tomas R <tomas.roun8@gmail.com>
2023-05-22 12:47:12 +02:00
Miss Islington (bot) 2e457bc202
[3.11] gh-101857: Allow xattr detection on musl libc (GH-101858) (#101894)
gh-101857: Allow xattr detection on musl libc (GH-101858)

Previously, we checked exclusively for `__GLIBC__` (AND'd with some other
conditions). Checking for `__linux__` instead should be fine.

This fixes using e.g. `os.listxattr()` on systems using musl libc.

Bug: https://bugs.gentoo.org/894130

(cherry picked from commit 8be8101bca)

Co-authored-by: Sam James <sam@gentoo.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-05-21 00:17:27 +00:00
Miss Islington (bot) dbb011afae
[3.11] gh-103987: fix several crashes in mmap module (GH-103990) (#104677)
gh-103987: fix several crashes in mmap module (GH-103990)
(cherry picked from commit ceaa4c3476)

Co-authored-by: Prince Roshan <princekrroshan01@gmail.com>
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-20 03:59:54 +00:00
Mark Shannon 7b3bc95067
[3.11] GH-102818: Do not call PyTraceBack_Here in sys.settrace trampoline (GH-104650)
Backport of GH-104579
2023-05-19 19:08:43 +01:00
Carl Meyer 667e4ece98
[3.11] gh-104615: don't make unsafe swaps in apply_static_swaps (GH-104620). (#104636)
(cherry picked from commit 0589c6a4d3)
2023-05-19 12:04:04 -06:00
Łukasz Langa 702ce87736
[3.11] gh-96522: Fix deadlock in pty.spawn (GH-96639) (#104655)
(cherry picked from commit 9c5aa8967b)

Co-authored-by: Youfu Zhang <1315097+zhangyoufu@users.noreply.github.com>
2023-05-19 16:54:50 +02:00
Miss Islington (bot) aaeaf01cc2
[3.11] gh-85984: Utilize new "winsize" functions from termios in pty tests. (GH-101831) (#104652)
Utilize new functions termios.tcgetwinsize() and termios.tcsetwinsize in test_pty.py.

(cherry picked from commit da2fb92643)

Co-authored-by: Soumendra Ganguly <67527439+8vasu@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-05-19 15:57:52 +02:00
Miss Islington (bot) d1bfefd8de
[3.11] gh-104623: Update macOS installer to SQLite 3.42.0 (GH-104643)
(cherry picked from commit fd04bfeaf7)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-19 07:16:43 +00:00
Erlend E. Aasland ba39ea48d5
[3.11] gh-104623: Update Windows installer to use SQLite 3.42.0 (#104625) (#104633) 2023-05-18 23:33:58 +00:00
Miss Islington (bot) e96fb399c1
[3.11] gh-104499: IDLE - fix completions for tk aqua 8.7 (GH-104591) (#104596)
gh-104499: IDLE - fix completions for tk aqua 8.7 (GH-104591)
(cherry picked from commit 678bf57ed0)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2023-05-18 02:02:53 +00:00
Miss Islington (bot) a8fc8b7e81
[3.11] gh-104340: Suppress warning about unawaited exception for closed pipe stdin (GH-104586) (#104594)
(cherry picked from commit 7fc8e2d462)

Co-authored-by: Guido van Rossum <guido@python.org>
2023-05-18 00:09:12 +00:00
Miss Islington (bot) 610cc0ab1b
[3.11] gh-102153: Start stripping C0 control and space chars in `urlsplit` (GH-102508) (#104575)
* gh-102153: Start stripping C0 control and space chars in `urlsplit` (GH-102508)

`urllib.parse.urlsplit` has already been respecting the WHATWG spec a bit GH-25595.

This adds more sanitizing to respect the "Remove any leading C0 control or space from input" [rule](https://url.spec.whatwg.org/GH-url-parsing:~:text=Remove%20any%20leading%20and%20trailing%20C0%20control%20or%20space%20from%20input.) in response to [CVE-2023-24329](https://nvd.nist.gov/vuln/detail/CVE-2023-24329).

---------

(cherry picked from commit 2f630e1ce1)

Co-authored-by: Illia Volochii <illia.volochii@gmail.com>
Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>
2023-05-17 21:41:25 +00:00
Miss Islington (bot) 0560fd3f98
[3.11] gh-104496: IDLE - fix About for mixed tcl/tk versions (GH-104585) (#104587)
gh-104496: IDLE - fix About for mixed tcl/tk versions (GH-104585)

Print both if they are different, as may happen in the future.
(cherry picked from commit aed643baa9)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2023-05-17 20:31:22 +00:00
Gregory P. Smith 133bf0927e
[3.11] gh-103861: Fix Zip64 extensions not being properly applied in some cases (GH-103863) (#104534)
Fix Zip64 extensions not being properly applied in some cases:

Fixes an issue where adding a small file to a `ZipFile`
object while forcing zip64 extensions causes an extra Zip64 record to be
added to the zip, but doesn't update the `min_version` or file sizes in
the primary central directory header.

Also fixed an edge case in checking if zip64 extensions are required:

This fixes an issue where if data requiring zip64 extensions was added
to an unseekable stream without specifying `force_zip64=True`, zip64
extensions would not be used and a RuntimeError would not be raised when
closing the file (even though the size would be known at that point).
This would result in successfully writing corrupt zip files.

Deciding if zip64 extensions are required outside of the `FileHeader`
function means that both `FileHeader` and `_ZipWriteFile` will always be
in sync. Previously, the `FileHeader` function could enable zip64
extensions without propagating that decision to the `_ZipWriteFile`
class, which would then not correctly write the data descriptor record
or check for errors on close.

If anyone is actually using `ZipInfo.FileHeader` as a public API without
explicitly passing True or False in for zip64, their own code may still be
susceptible to that kind of bug unless they make a similar change to
where the zip64 decision happens.

Fixes GH-103861

---------

.
(cherry picked from commit 798bcaa1eb)

Co-authored-by: Carey Metcalfe <carey@cmetcalfe.ca>
2023-05-17 01:07:10 -07:00
Miss Islington (bot) 3ce7d57c8a
[3.11] gh-87474: Fix file descriptor leaks in subprocess.Popen (GH-96351) (#104563)
gh-87474: Fix file descriptor leaks in subprocess.Popen (GH-96351)

This fixes several ways file descriptors could be leaked from `subprocess.Popen` constructor during error conditions by opening them later and using a context manager "fds to close" registration scheme to ensure they get closed before returning.

---------

(cherry picked from commit 3a4c44bb1e)

Co-authored-by: cptpcrd <31829097+cptpcrd@users.noreply.github.com>
Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>
2023-05-17 00:48:10 -07:00
Miss Islington (bot) dece9c06bb
[3.11] gh-104494: Update certain Tkinter pack/place tests for Tk 8.7 errors (GH-104495) (#104569)
gh-104494: Update certain Tkinter pack/place tests for Tk 8.7 errors (GH-104495)
(cherry picked from commit 3cba61f111)

Co-authored-by: Christopher Chavez <chrischavez@gmx.us>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-16 20:03:51 -04:00
Miss Islington (bot) f748fc9f44
[3.11] gh-75367: Fix data descriptor detection in inspect.getattr_static (GH-104517) (#104557)
gh-75367: Fix data descriptor detection in inspect.getattr_static (GH-104517)
(cherry picked from commit 5e9f471e7d)

Co-authored-by: Furkan Onder <furkanonder@protonmail.com>
Co-authored-by: Carl Meyer <carl@oddbird.net>
2023-05-16 17:57:34 +00:00
Irit Katriel 52a18feaaa
[3.11] gh-104482: Fix error handling bugs in ast.c (#104514) 2023-05-16 10:14:36 +01:00
Miss Islington (bot) 7779027084
[3.11] gh-104461: Run tkinter test_configure_screen on X11 only (GH-104526)
Co-authored-by: Christopher Chavez <chrischavez@gmx.us>
2023-05-16 02:47:51 +00:00
Miss Islington (bot) a712c5f42d
[3.11] gh-67056: document that registering/unregistering an atexit func from within an atexit func is undefined (GH-104473) (#104500) 2023-05-15 10:15:07 +01:00
Brandt Bucher fbb6def08a
[3.11] GH-104405: Add missing PEP 523 checks (GH-104441) 2023-05-12 23:03:47 +00:00
Miss Islington (bot) b4c1ca29cc
[3.11] gh-103204: `http.server` - Enforce that HTTP version numbers must consist only of digits (GH-103205) (#104438)
gh-103204: `http.server` - Enforce that HTTP version numbers must consist only of digits (GH-103205)

Reject HTTP requests with invalid http/x.y version numbers: x or y being non-digits or too-long.

---------

(cherry picked from commit cf720acfcb)

Co-authored-by: Ben Kallus <49924171+kenballus@users.noreply.github.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-05-12 20:54:12 +00:00
Miss Islington (bot) 846a23d0b8
[3.11] GH-87695: Fix OSError from `pathlib.Path.glob()` (GH-104292) (GH-104362)
Fix issue where `pathlib.Path.glob()` raised `OSError` when it encountered
a symlink to an overly long path.
(cherry picked from commit a33ce66dca)

Co-authored-by: Barney Gale <barney.gale@gmail.com>
2023-05-10 23:19:49 +00:00
Miss Islington (bot) b2171a2fd4
[3.11] gh-103848: Adds checks to ensure that bracketed hosts found by urlsplit are of IPv6 or IPvFuture format (GH-103849) (#104349)
gh-103848: Adds checks to ensure that bracketed hosts found by urlsplit are of IPv6 or IPvFuture format (GH-103849)

* Adds checks to ensure that bracketed hosts found by urlsplit are of IPv6 or IPvFuture format

---------

(cherry picked from commit 29f348e232)

Co-authored-by: JohnJamesUtley <81572567+JohnJamesUtley@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-05-10 06:35:24 +00:00
Miss Islington (bot) 2fa14d47af
[3.11] gh-48241: Clarify URL needs to be encoded when provided to urlopen and Request (GH-103855) (#103891)
(cherry picked from commit 44010d0f12)

Co-authored-by: Michael Blahay <mblahay@users.noreply.github.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-05-10 00:22:06 +02:00
Miss Islington (bot) 4ed59b1f33
[3.11] gh-99889: Fix directory traversal security flaw in uu.decode() (GH-104096) (#104329)
gh-99889: Fix directory traversal security flaw in uu.decode() (GH-104096)

* Fix directory traversal security flaw in uu.decode()
* also check absolute paths and os.altsep
* Add a regression test.

---------

(cherry picked from commit 0aeda29793)


 [Google]

Co-authored-by: Sam Carroll <70000253+samcarroll42@users.noreply.github.com>
2023-05-09 09:46:25 -07:00
Miss Islington (bot) b7622e7a05
[3.11] GH-104308: socket.getnameinfo should release the GIL (GH-104307) (#104313)
GH-104308: socket.getnameinfo should release the GIL (GH-104307)

* socket.getnameinfo should release the GIL

* 📜🤖 Added by blurb_it.

---------

(cherry picked from commit faf196213e)

Co-authored-by: Nathaniel J. Smith <njs@pobox.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-05-08 23:49:45 +00:00
Miss Islington (bot) 065e2ae99a
[3.11] gh-104265 Disallow instantiation of `_csv.Reader` and `_csv.Writer` (GH-104266) (#104278)
gh-104265 Disallow instantiation of `_csv.Reader` and `_csv.Writer` (GH-104266)
(cherry picked from commit 06c2a4858b)

Co-authored-by: chgnrdv <52372310+chgnrdv@users.noreply.github.com>
2023-05-08 17:34:43 +05:30
Erlend E. Aasland 681d5028bd
[3.11] gh-100370: fix OverflowError in sqlite3.Connection.blobopen for 32-bit builds (#103902) (#104285) 2023-05-08 08:41:34 +00:00
Tian Gao 19abf691fe
[3.11] gh-103225: Fixed zero lineno issue for pdb (#103265) (#104262)
gh-103225: Fixed zero lineno issue for pdb (#103265)

Co-authored-by: Artem Mukhin <ortem00@gmail.com>
2023-05-07 12:21:57 +01:00
Lysandros Nikolaou a09d3901a5
[3.11] gh-96670: Raise SyntaxError when parsing NULL bytes (GH-97594) (#104195) 2023-05-07 11:12:04 +01:00
Miss Islington (bot) cf1c25fd6e
[3.11] gh-101640: Make argparse _print_message catch any write error (GH-101802) (#104250)
gh-101640: Make argparse _print_message catch any write error (GH-101802)

* In particular, don't exit when trying to print to stderr = None.
* Add tests

(cherry picked from commit 42f54d1f92)

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2023-05-06 23:17:27 +00:00
Dong-hee Na 81902d3ed3
[3.11] gh-104106: Add gcc fallback of mkfifoat/mknodat for macOS (gh-104129) (gh-104187)
gh-104106: Add gcc fallback of mkfifoat/mknodat for macOS (gh-104129)
2023-05-05 04:32:28 +00:00
Miss Islington (bot) 4536b2ec18
[3.11] gh-104049: do not expose on-disk location from SimpleHTTPRequestHandler (GH-104067) (#104123)
gh-104049: do not expose on-disk location from SimpleHTTPRequestHandler (GH-104067)

Do not expose the local server's on-disk location from `SimpleHTTPRequestHandler` when generating a directory index. (unnecessary information disclosure)

---------

(cherry picked from commit c7c3a60c88)

Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-03 04:27:04 +00:00
Irit Katriel 5386730cdd
[3.11] gh-103590: do not wrap a single exception raised from a try-except* (#104094) 2023-05-02 19:24:02 +01:00
Rafael Fontenelle 5e76305236
[3.11] GH-103484: Fix redirected permanently URLs (GH-104001) (#104088)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Oleg Iarygin <dralife@yandex.ru>
Fix redirected permanently URLs (#104001)
2023-05-02 14:50:16 +03:00
Erlend E. Aasland 61f1fbd040
[3.11] gh-102997: Update Windows installer to SQLite 3.41.2. (#102999) (#104085)
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2023-05-02 08:15:06 +00:00
Miss Islington (bot) 0d40264325
[3.11] GH-103472: close response in HTTPConnection._tunnel (GH-103473) (#104077)
GH-103472: close response in HTTPConnection._tunnel (GH-103473)

Avoid a potential `ResourceWarning` in `http.client.HTTPConnection`
by closing the proxy / tunnel's CONNECT response explicitly.

---------

(cherry picked from commit 9de0cf20fa)

Co-authored-by: Thomas Grainger <tagrain@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-05-01 23:48:07 -07:00
Miss Islington (bot) 6805e49569
[3.11] gh-102997: Update macOS installer to SQLite 3.41.2. (GH-104080)
(cherry picked from commit f0ad456731)

Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2023-05-02 05:01:20 +00:00
Miss Islington (bot) 10db28bfcf
[3.11] gh-104018: disallow "z" format specifier in %-format of byte strings (GH-104033) (#104058)
gh-104018: disallow "z" format specifier in %-format of byte strings (GH-104033)

PEP-0682 specified that %-formatting would not support the "z" specifier,
but it was unintentionally allowed for bytes. This PR makes use of the "z"
flag an error for %-formatting in a bytestring.

Issue: GH-104018

---------

(cherry picked from commit 3ed8c88290)

Co-authored-by: John Belmonte <john@neggie.net>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-05-01 20:18:29 +00:00
Miss Islington (bot) 0df7c3a466
[3.11] gh-104035: Do not ignore user-defined `__{get,set}state__` in slotted frozen dataclasses (GH-104041) (#104044)
gh-104035: Do not ignore user-defined `__{get,set}state__` in slotted frozen dataclasses (GH-104041)
(cherry picked from commit 99aab61062)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2023-05-01 10:02:41 -06:00
Miss Islington (bot) 2e5f1a1259
[3.11] gh-88496: Fix IDLE test hang on macOS (GH-104025) (#104027)
gh-88496: Fix IDLE test hang on macOS (GH-104025)

Replace widget.update() with widget.update_idletasks in two places.
(cherry picked from commit 4b27972f5f)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2023-05-01 00:36:21 -04:00
Miss Islington (bot) ba16324b27
[3.11] Update name in acknowledgements and add mailmap (GH-103696) (#104002)
Update name in acknowledgements and add mailmap (GH-103696)

I changed my name last year, and would like to update my name in the
acknowledgements and git history accordingly.

git-mailmap reference: https://git-scm.com/docs/gitmailmap

(cherry picked from commit 4b10ecc29f)

Co-authored-by: Amethyst Reese <amethyst@n7.gg>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-04-30 03:51:37 +00:00
Tian Gao fee3c91a19
[3.11] GH-103971: Fix incorrect locations for code following case blocks 2023-04-28 20:08:25 +00:00
Petr Viktorin 241f2e54a6
[3.11] gh-102950: Implement PEP 706 – Filter for tarfile.extractall (GH-102953) (GH-103832)
See [Backporting & Forward Compatibility in PEP 706](https://peps.python.org/pep-0706/#backporting-forward-compatibility).

- Backport b52ad18a76
- Backport c8c3956d90
- Remove the DeprecationWarning
- Adjust docs
- Remove new `__all__` entries
2023-04-28 17:41:09 +02:00
Steve Dower 57d198ec83
gh-103935: Use `io.open_code()` when executing code in trace and profile modules (GH-103947)
Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
2023-04-27 21:23:46 +00:00
Miss Islington (bot) 2cd1b9c2ca
[3.11] gh-103607: Fix `pause_reading` to work when called from `connection_made` in `asyncio`. (GH-17425) (#103918)
gh-103607: Fix `pause_reading` to work when called from `connection_made` in `asyncio`. (GH-17425)
(cherry picked from commit 78942ecd9b)

Co-authored-by: Itayazolay <itayazolay@gmail.com>
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-04-27 10:29:54 +00:00
Paul Moore 57c96346dd
[3.11] gh-103872: Update bundled pip version to 23.1.2 (gh-103874)
closes: #103872
xref: https://github.com/pypa/pip/pull/12000

<!-- gh-issue-number: gh-103872 -->
* Issue: gh-103872 <!-- /gh-issue-number -->

---------

Co-authored-by: blurb-it[bot]
<43283697+blurb-it[bot]@users.noreply.github.com>
(cherry picked from commit 438b811761)

<!--
Thanks for your contribution!
Please read this comment in its entirety. It's quite important.

# Pull Request title

It should be in the following format:

```
gh-NNNNN: Summary of the changes made
```

Where: gh-NNNNN refers to the GitHub issue number.

Most PRs will require an issue number. Trivial changes, like fixing a
typo, do not need an issue.

# Backport Pull Request title

If this is a backport PR (PR made against branches other than `main`),
please ensure that the PR title is in the following format:

```
[X.Y] <title from the original PR> (GH-NNNN)
```

Where: [X.Y] is the branch name, e.g. [3.6].

GH-NNNN refers to the PR number from `main`.

-->
2023-04-26 10:50:34 +00:00
Miss Islington (bot) f11ba1c2de
[3.11] gh-103578: Fix pdb reading code with non-utf8 encoding (GH-103581) (#103867)
`pdb` should use `io.open_code` to open code to avoid encoding issue.
(cherry picked from commit 31acfd78a0)

Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
2023-04-26 05:28:29 +00:00
Irit Katriel 93a9535d54
[3.11] gh-101517: fix line number propagation in code generated for except* (#103550) (#103816)
Manual backport of https://github.com/python/cpython/pull/103550.



<!-- gh-issue-number: gh-101517 -->
* Issue: gh-101517
<!-- /gh-issue-number -->

---------

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-04-25 18:08:54 +01:00
Paul Moore a59dc1fb43
[3.11] gh-103559: Update bundled pip version to 23.1.1 (gh-103560) (#103752)
(cherry picked from commit b2862950dc)

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-04-25 02:58:19 +00:00
Miss Islington (bot) c8de883bcb
[3.11] gh-99184: Bypass instance attribute access in `repr` of `weakref.ref` (GH-99244) (#103789)
gh-99184: Bypass instance attribute access in `repr` of `weakref.ref` (GH-99244)
(cherry picked from commit 58b6be3791)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2023-04-24 19:58:55 +00:00
Miss Islington (bot) dc08c7a515
[3.11] gh-103685: Fix tkinter.Menu.index() for Tk 8.7 (GH-103686) (#103734)
gh-103685: Fix tkinter.Menu.index() for Tk 8.7 (GH-103686)

---------

(cherry picked from commit f0ed293f6a)

Co-authored-by: Christopher Chavez <chrischavez@gmx.us>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-04-23 22:29:58 -04:00
Miss Islington (bot) 7b2ac6cf3d
[3.11] gh-102310: Change error range for invalid bytes literals (GH-103663) (#103703) 2023-04-23 17:21:27 -06:00
Rafael Fontenelle 8642fdce8c
[3.11] GH-103484: Fix broken links reported by linkcheck (GH-103608) (#103683)
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Fix broken links reported by linkcheck (#103608)
2023-04-23 23:26:15 +03:00
Miss Islington (bot) ec29d0c091
[3.11] gh-103449: Fix a bug in dataclass docstring generation (GH-103454) (#103599) 2023-04-23 09:40:52 -06:00