Commit Graph

12195 Commits

Author SHA1 Message Date
Miss Islington (bot) a7aa105702
bpo-44252: Correctly implement gc support for SSLError objects (GH-26439) (GH-26441)
(cherry picked from commit 8b4312b909)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-05-29 00:47:36 +01:00
Ken Jin 0d39951632
[3.10] bpo-42972: Fully support GC for _winapi.Overlapped (GH-26381) (#26430)
* bpo-42972: Fully support GC for _winapi.Overlapped (GH-26381)

* untrack earlier
2021-05-28 18:47:15 +01:00
Miss Islington (bot) 1c0106ca8c
bpo-42972: Fully implement GC protocol for functools LRU cache (GH-26423)
(cherry picked from commit 3f8d332527)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-05-28 07:26:07 -07:00
Miss Islington (bot) eb8ab04dd7
bpo-42972: Fully implement GC protocol for functools keywrapper and partial types (GH-26363) (GH-26424)
(cherry picked from commit 8994e9c2cd)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-05-28 11:08:01 +02:00
Miss Islington (bot) 35be1f3602
bpo-44256: Do not expose _functools._list_elem_type (GH-26416)
It is internal use only type.
(cherry picked from commit 28be3191a9)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2021-05-28 16:10:45 +09:00
Miss Islington (bot) da9e0cb4de
bpo-42972: Fully implement GC protocol for re types (GH-26368) (GH-26414)
(cherry picked from commit fba42d1188)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-05-28 02:23:37 +02:00
Miss Islington (bot) d1c732912e
bpo-42972: Fully support GC protocol for _operator heap types (GH-26371) (GH-26413)
(cherry picked from commit f4b70c22c8)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-05-27 23:49:43 +02:00
Miss Islington (bot) da8097aaf5
bpo-42972: Fully support GC for mmap heap types (GH-26373)
(cherry picked from commit 318adeba78)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-05-27 09:43:56 -07:00
Miss Islington (bot) e73b3b1cd4
bpo-42972: Fully support GC protocol for _queue.SimpleQueue (GH-26372) (GH-26406)
(cherry picked from commit 4d7f8f9f7f)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-05-27 18:25:50 +02:00
Miss Islington (bot) ea47a8a71a
bpo-42972: Fully implement GC protocol for ssl heap types (GH-26370) (GH-26399)
(cherry picked from commit dcb8786a98)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-05-27 18:25:22 +02:00
Miss Islington (bot) 0bf0500baa
bpo-42972: Fully support GC for pyexpat, unicodedata, and dbm/gdbm heap types (GH-26376)
* bpo-42972: pyexpat
* bpo-42972: unicodedata
* bpo-42972: dbm/gdbm
(cherry picked from commit 59af59c2df)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-05-27 08:26:15 -07:00
Miss Islington (bot) 4431922f92
[3.10] bpo-42972: Fully support GC for hashlib heap types (GH-26374) (GH-26398)
(cherry picked from commit 6ef5ba391d)


Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>

Automerge-Triggered-By: GH:tiran
2021-05-27 01:10:39 -07:00
Miss Islington (bot) 4115996342
bpo-42161: mathmodule.c: move _PyLong_GetOne() loop invariant (GH-26391) (GH-26393)
Move _PyLong_GetZero() and _PyLong_GetOne() loop invariants outside
loops in functions:

* math.comb()
* math.gcd()
* math.lcm()
* math.perm()
(cherry picked from commit 3e7ee02327)

Co-authored-by: Victor Stinner <vstinner@python.org>
2021-05-27 01:13:17 +02:00
Miss Islington (bot) 534da740a2
bpo-42972: Fully implement GC protocol for arraymodule types (GH-26114)
(cherry picked from commit bd404ccac0)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-05-25 11:49:19 -07:00
Miss Islington (bot) e8d9df0089
bpo-42972: Fully implement GC protocol for sqlite3 heap types (GH-26104)
(cherry picked from commit d3c277a59c)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-05-25 11:08:39 -07:00
Miss Islington (bot) 46f96f00f7
Fix compiler warning for misleading guarding in the tkinter (GH-26244) (GH-26251)
The newest gcc emmits this warning:

```
/Modules/_tkinter.c:272:9: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation]
  272 |         if(tcl_lock)PyThread_acquire_lock(tcl_lock, 1); tcl_tstate = tstate; }
      |         ^~
/Modules/_tkinter.c:2869:5: note: in expansion of macro ‘LEAVE_PYTHON’
 2869 |     LEAVE_PYTHON
      |     ^~~~~~~~~~~~
/Modules/_tkinter.c:243:5: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘if’
  243 |     (*(PyThreadState**)Tcl_GetThreadData(&state_key, sizeof(PyThreadState*)))
      |     ^
/Modules/_tkinter.c:272:57: note: in expansion of macro ‘tcl_tstate’
  272 |         if(tcl_lock)PyThread_acquire_lock(tcl_lock, 1); tcl_tstate = tstate; }
      |                                                         ^~~~~~~~~~
/Modules/_tkinter.c:2869:5: note: in expansion of macro ‘LEAVE_PYTHON’
 2869 |     LEAVE_PYTHON

```

that's because the macro packs together two statements at the same level
as the "if". The warning is misleading but is very noisy so it makes
sense to fix it.
(cherry picked from commit 95d04710c5)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-05-19 19:46:45 +01:00
Miss Islington (bot) e87fd41d02
Fix compiler warning in the xml module (GH-26245) (GH-26249)
The newest version of gcc complains about passing un-initialized arrays
as constant pointers:

```
/Modules/expat/xmltok_ns.c: In function ‘findEncodingNS’:
/Modules/expat/xmltok.h:272:10: warning: ‘buf’ may be used uninitialized [-Wmaybe-uninitialized]
  272 |   (((enc)->utf8Convert)(enc, fromP, fromLim, toP, toLim))
      |   ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Modules/expat/xmltok_ns.c:95:3: note: in expansion of macro ‘XmlUtf8Convert’
   95 |   XmlUtf8Convert(enc, &ptr, end, &p, p + ENCODING_MAX - 1);
      |   ^~~~~~~~~~~~~~
/Modules/expat/xmltok.h:272:10: note: by argument 5 of type ‘const char *’ to ‘enum XML_Convert_Result(const ENCODING *, const char **, const char *, char **, const char *)’ {aka ‘enum XML_Convert_Result(const struct encoding *, const char **, const char *, char **, const char *)’}
  272 |   (((enc)->utf8Convert)(enc, fromP, fromLim, toP, toLim))
      |   ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Modules/expat/xmltok_ns.c:95:3: note: in expansion of macro ‘XmlUtf8Convert’
   95 |   XmlUtf8Convert(enc, &ptr, end, &p, p + ENCODING_MAX - 1);
      |   ^~~~~~~~~~~~~~
In file included from /Modules/expat/xmltok.c:1657:
/Modules/expat/xmltok_ns.c:92:8: note: ‘buf’ declared here
   92 |   char buf[ENCODING_MAX];

```
(cherry picked from commit be93f81e58)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-05-19 19:21:19 +01:00
Miss Islington (bot) 60fa8b32db
bpo-44145: Release the GIL around HMAC_Update. (GH-26157)
It was always meant to be released for parallelization.
This now matches the other similar code in the module.

Thanks michaelforney for noticing!
(cherry picked from commit c10392e7dd)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2021-05-17 01:03:57 -07:00
Miss Islington (bot) ba260acb22
bpo-44116: Add GC support to _csv heap types (GH-26074) (GH-26081)
(cherry picked from commit e5ba1fe995)
2021-05-12 19:56:19 +01:00
Miss Islington (bot) 1ee58f2524
bpo-40645: Fix ref leaks in _hashopenssl (GH-26079)
(cherry picked from commit 504ffdae4e)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-05-12 11:45:06 -07:00
Miss Islington (bot) 3e44e9af9e
bpo-44089: Allow subclassing of ``csv.Error`` (GH-26008) (GH-26066)
* fix subclass error

* Update 2021-05-09-22-52-34.bpo-44089.IoANsN.rst
(cherry picked from commit 2b458c1dba)

Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>

Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
2021-05-12 07:02:46 -07:00
Miss Islington (bot) 569ca81adf
bpo-40943: Fix skipitem() didn't raise SystemError (GH-25937)
`convertitem()` raises `SystemError` when 'GH-' is used without `PY_SSIZE_T_CLEAN`.
This commit makes `skipitem()` raise it too.
(cherry picked from commit 4ebf4a6bfa)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2021-05-06 20:18:42 -07:00
Miss Islington (bot) 3b2a45ff95
bpo-36515: Disable unaligned memory access in _sha3 on ARM (GH-25927)
Contributed-By: Matthias Klose

Automerge-Triggered-By: GH:tiran
(cherry picked from commit da5c808fb5)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2021-05-05 15:05:22 -07:00
Miss Islington (bot) b19ec7f9df
Eliminate duplicated assignment in _randommodule.c (GH-25904) (GH-25909) 2021-05-04 21:39:09 -07:00
Miss Islington (bot) 912ef3f248
Add C-API tests (GH-25886) (#25887)
(cherry picked from commit 2f5baa1750)

Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
2021-05-04 14:29:56 +02:00
Dennis Sweeney 80a2a4ed7d
bpo-38530: Refactor and improve AttributeError suggestions (GH-25776)
- Make case-swaps half the cost of any other edit
- Refactor Levenshtein code to not use memory allocator, and to bail early on no match.
- Add comments to Levenshtein distance code
- Add test cases for Levenshtein distance behind a debug macro
- Set threshold to `(name_size + item_size + 3) * MOVE_COST / 6`.
  - Reasoning: similar to `difflib.SequenceMatcher.ratio()` >= 2/3:
```
"Multiset Jaccard similarity" >= 2/3
matching letters / total letters >= 2/3
(name_size - distance + item_size - distance) / (name_size + item_size) >= 2/3
1 - (2*distance) / (name_size + item_size) >= 2/3
1/3 >= (2*distance) / (name_size + item_size)
(name_size + item_size) / 6 >= distance
With rounding:
(name_size + item_size + 3) // 6 >= distance
```

Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-05-03 16:47:27 +01:00
Pablo Galindo c2931d31f8
bpo-43916: Move the _PyStructSequence_InitType function to the internal API (GH-25854) 2021-05-03 15:50:24 +01:00
Batuhan Taskaya ad106c68eb
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583) 2021-05-03 10:43:00 +03:00
Mark Shannon 33ec88ac81
bpo-43977: Make sure that tp_flags for pattern matching are inherited correctly. (GH-25813) 2021-05-03 00:38:22 +01:00
Erlend Egeberg Aasland cbb7b9ed4a
bpo-43434: Clean up sqlite3.connect() after GH-25818 (GH-25823) 2021-05-02 23:10:51 +01:00
Erlend Egeberg Aasland c96cc089f6
bpo-43434: Move sqlite3.connect audit events to sqlite3.Connection.__init__ (GH-25818) 2021-05-02 22:25:17 +01:00
Zackery Spytz 73766b0341
bpo-32745: Fix a regression in the handling of ctypes' c_wchar_p type (#8721)
Embedded nulls would cause a ValueError to be raised. Thanks go to
Eryk Sun for their analysis.

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-05-02 12:40:01 +02:00
Christian Heimes 91554e4c5c
bpo-43908: Mark ssl, hash, and hmac types as immutable (GH-25792)
Signed-off-by: Christian Heimes <christian@python.org>
2021-05-02 09:47:45 +02:00
Christian Heimes e983252b51
bpo-43998: Default to TLS 1.2 and increase cipher suite security (GH-25778)
The ssl module now has more secure default settings. Ciphers without forward
secrecy or SHA-1 MAC are disabled by default. Security level 2 prohibits
weak RSA, DH, and ECC keys with less than 112 bits of security.
:class:`~ssl.SSLContext` defaults to minimum protocol version TLS 1.2.
Settings are based on Hynek Schlawack's research.

```
$ openssl version
OpenSSL 1.1.1k  FIPS 25 Mar 2021
$ openssl ciphers -v '@SECLEVEL=2:ECDH+AESGCM:ECDH+CHACHA20:ECDH+AES:DHE+AES:!aNULL:!eNULL:!aDSS:!SHA1:!AESCCM'
TLS_AES_256_GCM_SHA384  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(256) Mac=AEAD
TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 Kx=any      Au=any  Enc=CHACHA20/POLY1305(256) Mac=AEAD
TLS_AES_128_GCM_SHA256  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(128) Mac=AEAD
TLS_AES_128_CCM_SHA256  TLSv1.3 Kx=any      Au=any  Enc=AESCCM(128) Mac=AEAD
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=RSA  Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA384
ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA384
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA256
ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA256
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES256-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA256
DHE-RSA-AES128-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA256
```

Signed-off-by: Christian Heimes <christian@python.org>
2021-05-01 20:53:10 +02:00
Ma Lin 251ffa9d2b
bpo-41486: Fix initial buffer size can't > UINT32_MAX in zlib module (GH-25738)
* Fix initial buffer size can't > UINT32_MAX in zlib module

After commit f9bedb630e, in 64-bit build,
if the initial buffer size > UINT32_MAX, ValueError will be raised.

These two functions are affected:
1. zlib.decompress(data, /, wbits=MAX_WBITS, bufsize=DEF_BUF_SIZE)
2. zlib.Decompress.flush([length])

This commit re-allows the size > UINT32_MAX.

* adds curly braces per PEP 7.

* Renames `Buffer_*` to `OutputBuffer_*` for clarity
2021-04-30 16:32:49 -07:00
Victor Stinner 665c7746fc
bpo-43916: _md5.md5 uses Py_TPFLAGS_DISALLOW_INSTANTIATION (GH-25753)
The following types use Py_TPFLAGS_DISALLOW_INSTANTIATION flag:

* _md5.md5
* _sha1.sha1
* _sha256.sha224
* _sha256.sha256
* _sha512.sha384
* _sha512.sha512
2021-04-30 18:40:30 +02:00
Victor Stinner 7dcf0f6db3
bpo-43916: select.devpoll uses Py_TPFLAGS_DISALLOW_INSTANTIATION (GH-25751) 2021-04-30 18:19:57 +02:00
Erlend Egeberg Aasland 9746cda705
bpo-43916: Apply Py_TPFLAGS_DISALLOW_INSTANTIATION to selected types (GH-25748)
Apply Py_TPFLAGS_DISALLOW_INSTANTIATION to the following types:

* _dbm.dbm
* _gdbm.gdbm
* _multibytecodec.MultibyteCodec
* _sre..SRE_Scanner
* _thread._localdummy
* _thread.lock
* _winapi.Overlapped
* array.arrayiterator
* functools.KeyWrapper
* functools._lru_list_elem
* pyexpat.xmlparser
* re.Match
* re.Pattern
* unicodedata.UCD
* zlib.Compress
* zlib.Decompress
2021-04-30 16:04:57 +02:00
Erlend Egeberg Aasland 387397f8a4
bpo-43916: select.poll uses Py_TPFLAGS_DISALLOW_INSTANTIATION (GH-25750) 2021-04-30 15:49:17 +02:00
Victor Stinner 0cad068ec1
bpo-43916: Remove _disabled_new() function (GH-25745)
posix and _hashlib use the new Py_TPFLAGS_DISALLOW_INSTANTIATION
flag on their heap types, rather than using a custom tp_new function
(_disabled_new).
2021-04-30 14:06:49 +02:00
Victor Stinner 3bb09947ec
bpo-43916: Add Py_TPFLAGS_DISALLOW_INSTANTIATION type flag (GH-25721)
Add a new Py_TPFLAGS_DISALLOW_INSTANTIATION type flag to disallow
creating type instances: set tp_new to NULL and don't create the
"__new__" key in the type dictionary.

The flag is set automatically on static types if tp_base is NULL or
&PyBaseObject_Type and tp_new is NULL.

Use the flag on the following types:

* _curses.ncurses_version type
* _curses_panel.panel
* _tkinter.Tcl_Obj
* _tkinter.tkapp
* _tkinter.tktimertoken
* _xxsubinterpretersmodule.ChannelID
* sys.flags type
* sys.getwindowsversion() type
* sys.version_info type

Update MyStr example in the C API documentation to use
Py_TPFLAGS_DISALLOW_INSTANTIATION.

Add _PyStructSequence_InitType() function to create a structseq type
with the Py_TPFLAGS_DISALLOW_INSTANTIATION flag set.

type_new() calls _PyType_CheckConsistency() at exit.
2021-04-30 12:46:15 +02:00
Mark Shannon 069e81ab3d
bpo-43977: Use tp_flags for collection matching (GH-25723)
* Add Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING, add to all relevant standard builtin classes.

* Set relevant flags on collections.abc.Sequence and Mapping.

* Use flags in MATCH_SEQUENCE and MATCH_MAPPING opcodes.

* Inherit Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING.

* Add NEWS

* Remove interpreter-state map_abc and seq_abc fields.
2021-04-30 09:50:28 +01:00
Erlend Egeberg Aasland c6ad03fddf
bpo-43908: Make array.array type immutable (GH-25696)
Co-authored-by: Victor Stinner <vstinner@python.org>
2021-04-29 08:47:48 +02:00
Erlend Egeberg Aasland 5daf70b22e
bpo-43908: Make re types immutable (GH-25697)
Co-authored-by: Victor Stinner <vstinner@python.org>
2021-04-29 08:47:11 +02:00
Victor Stinner 103d5e420d
bpo-28254: _posixsubprocess uses PyGC_Enable/PyGC_Disable (GH-25693) 2021-04-28 19:09:29 +02:00
scoder 3cc481b9de
bpo-28254: Add a C-API for controlling the GC state (GH-25687)
Add new C-API functions to control the state of the garbage collector:
PyGC_Enable(), PyGC_Disable(), PyGC_IsEnabled(),
corresponding to the functions in the gc module.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2021-04-28 18:12:16 +02:00
Ma Lin f9bedb630e
bpo-41486: Faster bz2/lzma/zlib via new output buffering (GH-21740)
Faster bz2/lzma/zlib via new output buffering.
Also adds .readall() function to _compression.DecompressReader class
to take best advantage of this in the consume-all-output at once scenario.

Often a 5-20% speedup in common scenarios due to less data copying.

Contributed by Ma Lin.
2021-04-27 23:58:54 -07:00
Victor Stinner a5e64444e6
bpo-43963: Add _signal module state (GH-25676)
* Add signal_state_t structure and signal_global_state variable.
* Add a module state to the _signal module.
* Move and rename variables:

  * DefaultHandler becomes state->default_handler
  * IgnoreHandler becomes state->ignore_handler
  * sigint_event becomes state->sigint_event
  * ItimerError becomes modstate->itimer_error

* Rename SetHandler() to set_handler() to be consistent with
  get_handler().
2021-04-28 03:02:55 +02:00
Victor Stinner a09766deab
bpo-43963: Fix import _signal in subinterpreters (GH-25674)
Importing the _signal module in a subinterpreter has no longer side
effects.

signal_module_exec() no longer modifies Handlers and no longer attempts
to set SIGINT signal handler in subinterpreters.
2021-04-28 01:50:04 +02:00
Ma Lin 93f411838a
Fix thread locks in zlib module may go wrong in rare case. (#22126)
Setting `next_in` before acquiring the thread lock may mix up compress/decompress state in other threads.
2021-04-27 10:37:11 +02:00