Commit Graph

14578 Commits

Author SHA1 Message Date
Sebastian Pipping 8e48a6edc7
gh-126624: Expose error code ``XML_ERROR_NOT_STARTED`` of Expat >=2.6.4 (#126625)
Expose error code ``XML_ERROR_NOT_STARTED`` in `xml.parsers.expat.errors` which was
introduced in Expat 2.6.4.


Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-01-02 14:54:38 +02:00
Serhiy Storchaka a3711d1541
gh-124130: Fix a bug in matching regular expression \B in empty string (GH-127007) 2025-01-02 12:11:21 +00:00
Serhiy Storchaka 8d16919a06
gh-123925: Fix building curses on platforms without libncursesw (GH-128405) 2025-01-02 13:38:21 +02:00
Kumar Aditya bb9d955e16
gh-128277: remove unnecessary critical section from `socket.close` (#128305)
Remove unnecessary critical section from `socket.close` as it now uses relaxed atomics for `sock_fd`.
2025-01-01 18:00:47 +05:30
Kumar Aditya e389d6c650
gh-128277: make globals variables thread safe in socket module (#128286) 2024-12-31 19:10:06 +05:30
Kumar Aditya 7c72c1f0df
gh-128277: use relaxed atomics for `sock_fd` (#128304) 2024-12-31 11:50:35 +05:30
Furkan Onder 492b224b99
gh-128279: Enhance the NetBSD compatibility for thread naming (#128280)
Enhance NetBSD compatibility for thread naming in _threadmodule.c.
2024-12-28 19:49:45 +01:00
Bénédikt Tran fb0b94223d
gh-87138: convert blake2b/2s types to heap types (#127669) 2024-12-26 20:33:57 +05:30
Thomas Grainger 9ddc388527
gh-124761: add `socket.SO_REUSEPORT_LB` (#124961) 2024-12-26 20:20:20 +05:30
Yan Yanchii 5c814c83cd
gh-128198: Add missing error checks for usages of PyIter_Next() (GH-128199) 2024-12-25 19:42:04 +02:00
Kumar Aditya 76f1785657
gh-128002: use internal llist implementation for asyncio tasks (#128256) 2024-12-25 17:51:27 +05:30
Will Childs-Klein 418114c139
gh-128035: Add ssl.HAS_PHA to detect libssl PHA support (GH-128036)
* Add ssl.HAS_PHA to detect libssl Post-Handshake-Auth support

Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-12-24 18:29:27 +00:00
Sergey Muraviov 3ddd70ceaa
gh-128217: Validate the normalized_environment variable instead of the similarly named function (GH-128220) 2024-12-24 13:06:41 +00:00
Petr Viktorin 78ffba4221
gh-127295: ctypes: Switch field accessors to fixed-width integers (GH-127297)
This should be a pure refactoring, without user-visible behaviour changes.

Before this change, ctypes uses traditional native C types, usually identified 
by [`struct` format characters][struct-chars] when a short (and 
identifier-friendly) name is needed:
- `signed char` (`b`) / `unsigned char` (`B`)
- `short` (`h`) / `unsigned short` (`h`)
- `int` (`i`) / `unsigned int` (`i`)
- `long` (`l`) / `unsigned long` (`l`)
- `long long` (`q`) / `unsigned long long` (`q`)

These map to C99 fixed-width types, which this PR switches to: - 
- `int8_t`/`uint8_t`
- `int16_t`/`uint16_t`
- `int32_t`/`uint32_t`
- `int64_t`/`uint64_t`

The C standard doesn't guarantee that the “traditional” types must map to the 
fixints. But, [`ctypes` currently requires it][swapdefs], so the assumption won't 
break anything.

By “map” I mean that the *size* of the types matches. The *alignment* 
requirements might not. This needs to be kept in mind but is not an issue in 
`ctypes` accessors, which [explicitly handle unaligned memory][memcpy] for the 
integer types.

Note that there are 5 “traditional” C type sizes, but 4 fixed-width ones. Two of 
the former are functionally identical to one another; which ones they are is 
platform-specific (e.g. `int`==`long`==`int32_t`.) This means that one of the 
[current][current-impls-1] [implementations][current-impls-2] is redundant on 
any given platform.


The fixint types are parametrized by the number of bytes/bits, and one bit for 
signedness. This makes it easier to autogenerate code for them or to write 
generic macros (though generic API like 
[`PyLong_AsNativeBytes`][PyLong_AsNativeBytes] is problematic for performance 
reasons -- especially compared to a `memcpy` with compile-time-constant size).

When one has a *different* integer type, determining the corresponding fixint  
means a `sizeof` and signedness check. This is easier and more robust than the 
current implementations (see [`wchar_t`][sizeof-wchar_t] or 
[`_Bool`][sizeof-bool]).


[swapdefs]: https://github.com/python/cpython/blob/v3.13.0/Modules/_ctypes/cfield.c#L420-L444
[struct-chars]: https://docs.python.org/3/library/struct.html#format-characters
[current-impls-1]: https://github.com/python/cpython/blob/v3.13.0/Modules/_ctypes/cfield.c#L470-L653
[current-impls-2]: https://github.com/python/cpython/blob/v3.13.0/Modules/_ctypes/cfield.c#L703-L944
[memcpy]: https://github.com/python/cpython/blob/v3.13.0/Modules/_ctypes/cfield.c#L613
[PyLong_AsNativeBytes]: https://docs.python.org/3/c-api/long.html#c.PyLong_AsNativeBytes
[sizeof-wchar_t]: https://github.com/python/cpython/blob/v3.13.0/Modules/_ctypes/cfield.c#L1547-L1555
[sizeof-bool]: https://github.com/python/cpython/blob/v3.13.0/Modules/_ctypes/cfield.c#L1562-L1572


Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-12-20 14:28:18 +01:00
Peter Bierma ba45e5cdd4
gh-127946: Use a critical section for `CFuncPtr` attributes (GH-128109) 2024-12-20 14:02:46 +01:00
Neil Schemenauer 1b15c89a17
gh-115999: Specialize `STORE_ATTR` in free-threaded builds. (gh-127838)
* Add `_PyDictKeys_StringLookupSplit` which does locking on dict keys and
  use in place of `_PyDictKeys_StringLookup`.

* Change `_PyObject_TryGetInstanceAttribute` to use that function
  in the case of split keys.

* Add `unicodekeys_lookup_split` helper which allows code sharing
  between `_Py_dict_lookup` and `_PyDictKeys_StringLookupSplit`.

* Fix locking for `STORE_ATTR_INSTANCE_VALUE`.  Create
  `_GUARD_TYPE_VERSION_AND_LOCK` uop so that object stays locked and
  `tp_version_tag` cannot change.

* Pass `tp_version_tag` to `specialize_dict_access()`, ensuring
  the version we store on the cache is the correct one (in case of
  it changing during the specalize analysis).

* Split `analyze_descriptor` into `analyze_descriptor_load` and
  `analyze_descriptor_store` since those don't share much logic.
  Add `descriptor_is_class` helper function.

* In `specialize_dict_access`, double check `_PyObject_GetManagedDict()`
  in case we race and dict was materialized before the lock.

* Avoid borrowed references in `_Py_Specialize_StoreAttr()`.

* Use `specialize()` and `unspecialize()` helpers.

* Add unit tests to ensure specializing happens as expected in FT builds.

* Add unit tests to attempt to trigger data races (useful for running under TSAN).

* Add `has_split_table` function to `_testinternalcapi`.
2024-12-19 10:21:17 -08:00
Mark Shannon d2f1d917e8
GH-122548: Implement branch taken and not taken events for sys.monitoring (GH-122564) 2024-12-19 16:59:51 +00:00
Sam Gross 7b811d0562
gh-128008: Add `PyWeakref_IsDead()` (GH-128009)
The `PyWeakref_IsDead()` function tests if a weak reference is dead
without any side effects. Although you can also detect if a weak
reference is dead using `PyWeakref_GetRef()`, that function returns a
strong reference that must be `Py_DECREF()`'d, which can introduce side
effects if the last reference is concurrently dropped (at least in the
free threading build).
2024-12-19 16:17:15 +01:00
Peter Bierma b5d1e4552f
gh-128083: Fix macro redefinition warning in clinic. (GH-127950) 2024-12-19 15:00:30 +01:00
RUANG (James Roy) ea578fc6d3
gh-127688: Add `SCHED_DEADLINE` and `SCHED_NORMAL` constants to `os` module (GH-127689) 2024-12-19 14:51:21 +01:00
Kumar Aditya 8a433b683f
gh-121621: clear running loop early in asyncio (#128004) 2024-12-18 14:25:03 +00:00
Kumar Aditya dbd08fb60d
gh-127949: deprecate `asyncio.get_event_loop_policy` (#128053)
This deprecates `asyncio.get_event_loop_policy` and will be removed in Python 3.16.
2024-12-18 18:04:20 +05:30
aeiouaeiouaeiouaeiouaeiouaeiou 329165639f
gh-127897: fix HACL* build on macOS/Catalina (GH-127932)
gh-127897: Update HACL* module from upstream sources to get:

- Lib_Memzero0.c: don't use memset_s() on macOS <10.9
- Use _mm_malloc() for KRML_ALIGNED_MALLOC on macOS <10.15
- Add LEGACY_MACOS macros, use _mm_free() for KRML_ALIGNED_FREE on macOS <10.15
2024-12-17 22:14:16 -08:00
Bénédikt Tran 7303f06846
gh-126742: Add _PyErr_SetLocaleString, use it for gdbm & dlerror messages (GH-126746)
- Add a helper to set an error from locale-encoded `char*`
- Use the helper for gdbm & dlerror messages

Co-authored-by: Victor Stinner <vstinner@python.org>
2024-12-17 12:12:45 +01:00
Peter Bierma 3b766824fe
gh-126907: make `atexit` thread safe in free-threading (#127935)
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2024-12-16 19:31:44 +00:00
Bénédikt Tran 0d8e7106c2
gh-111178: fix UBSan failures in `_elementtree.c` (#127982) 2024-12-16 14:40:26 +01:00
Filipe Laíns 🇵🇸 b74c8f58e8
GH-126985: Don't override venv detection with PYTHONHOME (#127968) 2024-12-15 21:12:13 +00:00
Filipe Laíns 🇵🇸 3683b2f9e5
getpath: Add comments highlighing details of the pyvenv.cfg detection (#127966) 2024-12-15 15:40:19 +00:00
Victor Stinner 6446408d42
gh-102471, PEP 757: Add PyLong import and export API (#121339)
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
Co-authored-by: Steve Dower <steve.dower@microsoft.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-12-13 14:24:48 +01:00
Victor Stinner 6ff38fc4e2
gh-127870: Detect recursive calls in ctypes _as_parameter_ handling (#127872) 2024-12-13 13:53:47 +01:00
Mark Shannon bc262de06b
GH-125174: Mark objects as statically allocated. (#127797)
* Set a bit in the unused part of the refcount on 64 bit machines and the free-threaded build.

* Use the top of the refcount range on 32 bit machines
2024-12-11 17:37:38 +00:00
Pieter Eendebak b0f278ff05
gh-127065: Make methodcaller thread-safe and re-entrant (GH-127746)
The function `operator.methodcaller` was not thread-safe since the additional
of the vectorcall method in gh-89013. In the free threading build the issue
is easy to trigger, for the normal build harder.

This makes the `methodcaller` safe by:

* Replacing the lazy initialization with initialization in the constructor.
* Using a stack allocated space for the vectorcall arguments and falling back
  to `tp_call` for calls with more than 8 arguments.
2024-12-11 10:06:07 -05:00
Justin Applegate ce76b547f9
gh-126992: Change pickle code to base 10 for load_long and load_int (GH-127042)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-12-11 12:37:59 +00:00
Peter Bierma d5d84c3f13
gh-127791: Fix, document, and test `PyUnstable_AtExit` (#127793) 2024-12-11 12:14:04 +01:00
Victor Stinner c91ccbe4ac
gh-59705: Set OS thread name when Thread.name is changed (#127702)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-12-10 17:33:11 +01:00
Melissa0x1f992 cef0a90d8f
gh-126937: ctypes: fix TypeError when a field's size is >65535 bytes (GH-126938)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-12-10 13:13:11 +01:00
Petr Viktorin f4b31edf2d
gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361)
From the ERR_raise manpage:

    ERR_LIB_SYS

        This "library code" indicates that a system error is
        being reported.  In this case, the reason code given
        to `ERR_raise()` and `ERR_raise_data()` *must* be
        `errno(3)`.


This PR only handles ERR_LIB_SYS for the high-lever error types
SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where
OpenSSL indicates it has some more information about the issue.
2024-12-10 11:56:24 +01:00
Bénédikt Tran 3b78409878
gh-87138: convert SHA-3 object type to heap type (GH-127670) 2024-12-08 09:31:10 -08:00
Victor Stinner 67b18a18b6
gh-59705: Add _thread.set_name() function (#127338)
On Linux, threading.Thread now sets the thread name to the operating
system.

* configure now checks if pthread_getname_np()
  and pthread_setname_np() functions are available.
* Add PYTHREAD_NAME_MAXLEN macro.
* Add _thread._NAME_MAXLEN constant for test_threading.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-12-06 16:27:12 +00:00
Hood Chatham 25eee578c8
gh-127627: Add `posix._emscripten_debugger` function (#127628)
Add a posix._emscripten_debugger function to add an emscripten breakpoint.
2024-12-06 10:47:51 +08:00
Peter Bierma 208b0fb645
gh-122431: Disallow negative values in `readline.append_history_file` (#122469)
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-12-05 16:07:38 +00:00
Victor Stinner fcbe6ecdb6
gh-93312: Include <sys/pidfd.h> to get PIDFD_NONBLOCK (#127593) 2024-12-05 10:39:44 +01:00
RUANG (James Roy) 6bc3e830a5
gh-127481: Add `EPOLLWAKEUP` to the `select` module (GH-127482) 2024-12-04 14:30:38 +01:00
Bénédikt Tran bc0f2e9459
gh-123378: Ensure results of `PyUnicode*Error_Get{Start,End}` are clamped (GH-123380)
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
2024-12-04 14:13:52 +01:00
Sam Gross 6fc6436749
gh-127572: Fix `test_structmembers` initialization (GH-127577)
gh-127572: Fix `test_structmembers` initialization.

The 'C' format code expects an `int` as a destination (not a `char`).
This led to test failures on big-endian platforms like s390x. Use the
'c' format code, which expects a `char` as the destination (but requires
a Python byte objects instead of a str).
2024-12-04 09:58:22 +01:00
Daniele Parmeggiani 979bf2489d
gh-117657: TSAN Fix races in `PyMember_Get` and `PyMember_Set` for C extensions (GH-123211) 2024-12-03 09:41:53 -05:00
RUANG (James Roy) 84ff1313d0
gh-126585: Add EHWPOISON error code (#126586) 2024-12-03 12:45:50 +00:00
Giovanni Siragusa 31f16e427b
gh-109523: Raise a BlockingIOError if reading text from a non-blocking stream cannot immediately return bytes. (GH-122933) 2024-12-02 14:18:30 +01:00
Sergey B Kirpichev 930ba0ce60
gh-126618: fix repr(itertools.count(sys.maxsize)) (#127048)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-12-02 14:14:40 +01:00
Peter Bierma c112de1da2
gh-126890: Restore stripped `ssl` docstrings (GH-127281) 2024-12-02 13:50:34 +01:00