[3.11] gh-105235: Prevent reading outside buffer during mmap.find() (GH-105252)
* Add a special case for s[-m:] == p in _PyBytes_Find
* Add tests for _PyBytes_Find
* Make sure that start <= end in mmap.find.
(cherry picked from commit ab86426a34)
Fix possible refleak in CodeType.replace() (GH-106243)
A reference to c_code was leaked if PySys_Audit() failed.
(cherry picked from commit 3c70d467c1)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
[3.11] gh-106033: Get rid of new occurrences of PyDict_GetItem and PyObject_HasAttr (GH-106034)
These functions are broken by design because they discard any exceptions raised
inside, including MemoryError and KeyboardInterrupt. They should not be
used in new code..
(cherry picked from commit 1d33d53780)
Bail on first error to prevent exceptions from possibly being overwritten.
(cherry picked from commit 555be81026)
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
gh-104018: remove unused format "z" handling in string formatfloat() (GH-104107)
This is a cleanup overlooked in PR GH-104033.
(cherry picked from commit 69621d1b09)
Co-authored-by: John Belmonte <john@neggie.net>
gh-104018: disallow "z" format specifier in %-format of byte strings (GH-104033)
PEP-0682 specified that %-formatting would not support the "z" specifier,
but it was unintentionally allowed for bytes. This PR makes use of the "z"
flag an error for %-formatting in a bytestring.
Issue: GH-104018
---------
(cherry picked from commit 3ed8c88290)
Co-authored-by: John Belmonte <john@neggie.net>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
[3.11] GH-101696: invalidate type version tag in `_PyStaticType_Dealloc` (GH-101697).
(cherry picked from commit d9de079248)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
Fix the behaviour of the `__sizeof__` method (and hence the results returned
by `sys.getsizeof`) for subclasses of `int`. Previously, `int` subclasses gave
identical results to the `int` base class, ignoring the presence of the instance
dictionary.
(Manual backport of #101394 to the Python 3.11 branch.)
gh-101037: Fix potential memory underallocation for zeros of int subtypes (GH-101038)
This PR fixes object allocation in long_subtype_new to ensure that there's at least one digit in all cases, and makes sure that the value of that digit is copied over from the source long.
Needs backport to 3.11, but not any further: the change to require at least one digit was only introduced for Python 3.11.
Fixes GH-101037.
(cherry picked from commit 401fdf9c85)
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
gh-100637: Fix int and bool __sizeof__ calculation to include the 1 element ob_digit array for 0 and False (GH-100663)
Fixes behaviour where int (and subtypes like bool) __sizeof__ under-reports true size as it did not take into account the size 1 `ob_digit` array for the zero int.
(cherry picked from commit d7e7f79ca7)
Co-authored-by: Ionite <dev@ionite.io>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
Fix subscription of type aliases containing bare generic types or types
like TypeVar: for example tuple[A, T][int] and tuple[TypeVar, T][int],
where A is a generic type, and T is a type variable.
(cherry picked from commit 0e15c31c7e)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-98783: Fix crashes when `str` subclasses are used in `_PyUnicode_Equal` (GH-98806)
(cherry picked from commit 76f989dc3e)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Fix multiplying a list by an integer (list *= int): detect the
integer overflow when the new allocated length is close to the
maximum size. Issue reported by Jordan Limor.
list_resize() now checks for integer overflow before multiplying the
new allocated length by the list item size (sizeof(PyObject*)).
(cherry picked from commit a5f092f3c4)
Co-authored-by: Victor Stinner <vstinner@python.org>
When ValueError is raised if an integer is larger than the limit,
mention sys.set_int_max_str_digits() in the error message.
(cherry picked from commit e841ffc915)
Co-authored-by: Victor Stinner <vstinner@python.org>
- On WASI `ENOTCAPABLE` is now mapped to `PermissionError`.
- The `errno` modules exposes the new error number.
- `getpath.py` now ignores `PermissionError` when it cannot open landmark
files `pybuilddir.txt` and `pyenv.cfg`.
Converting a large enough `int` to a decimal string raises `ValueError` as expected. However, the raise comes _after_ the quadratic-time base-conversion algorithm has run to completion. For effective DOS prevention, we need some kind of check before entering the quadratic-time loop. Oops! =)
The quick fix: essentially we catch _most_ values that exceed the threshold up front. Those that slip through will still be on the small side (read: sufficiently fast), and will get caught by the existing check so that the limit remains exact.
The justification for the current check. The C code check is:
```c
max_str_digits / (3 * PyLong_SHIFT) <= (size_a - 11) / 10
```
In GitHub markdown math-speak, writing $M$ for `max_str_digits`, $L$ for `PyLong_SHIFT` and $s$ for `size_a`, that check is:
$$\left\lfloor\frac{M}{3L}\right\rfloor \le \left\lfloor\frac{s - 11}{10}\right\rfloor$$
From this it follows that
$$\frac{M}{3L} < \frac{s-1}{10}$$
hence that
$$\frac{L(s-1)}{M} > \frac{10}{3} > \log_2(10).$$
So
$$2^{L(s-1)} > 10^M.$$
But our input integer $a$ satisfies $|a| \ge 2^{L(s-1)}$, so $|a|$ is larger than $10^M$. This shows that we don't accidentally capture anything _below_ the intended limit in the check.
<!-- gh-issue-number: gh-95778 -->
* Issue: gh-95778
<!-- /gh-issue-number -->
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
(cherry picked from commit b126196838)
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds.
This PR comes fresh from a pile of work done in our private PSRT security response team repo.
This backports https://github.com/python/cpython/pull/96499 aka 511ca94520
Signed-off-by: Christian Heimes [Red Hat] <christian@python.org>
Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org>
Reviews via the private PSRT repo via many others (see the NEWS entry in the PR).
<!-- gh-issue-number: gh-95778 -->
* Issue: gh-95778
<!-- /gh-issue-number -->
I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#).
Serhiy Storchaka