* Add _zstd module for https://peps.python.org/pep-0784/
This commit introduces the `_zstd` module, with bindings to libzstd from
the pyzstd project. It also includes the unix build system configuration.
Windows build system support will be integrated independently as it
depends on integration with cpython-source-deps.
* Add _zstd to modules
* Fix path for compression.zstd module
* Ignore _zstd module like _io
* Expand module state macros to improve code quality
Also removes module state references from the classes in the _zstd
module and instead uses PyType_GetModuleState()
* Remove backticks suggested in review
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
* Use critical sections to lock object state
This should avoid races and deadlocks.
* Remove compress/decompress and mark module as not reliant on the GIL
The `compress`/`decompress` functions will be moved to Python code for simplicity.
C implementations can always be re-added in the future.
Also, mark _zstd as not requiring the GIL.
* Lift critical section to avoid clang warning
* Respond to comments by picnixz
* Call out pyzstd explicitly in license description
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
* Use a much more robust implementation...
... for `get_zstd_state_from_type`
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* Use PyList_GetItemRef for thread safety purposes
* Use a macro for the minimum supported version
* remove const from primivite types
* Use PyMem_New in another spot
* Simplify error handling in _get_frame_size
* Another simplification of error handling in get_frame_info
* Rename _module_state to mod_state
* Rewrite comment explaining the context of the code
* Add link to pyzstd
* Add TODO about refactoring dict training code
* Use PyModule_AddObjectRef over PyModule_AddObject
PyModule_AddObject is soft-deprecated, so we should use PyModule_AddObjectRef
* Check result of OutputBufferGrow
* Simplify return logic in `add_constant_to_type`
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* Ignore return value of _zstd_clear()
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* Remove redundant comments
* Remove __reduce__ from ZstdDict
We should instead document that to pickle a dictionary a user should use
the `.dict_content` attribute.
* Use PyUnicode_FromFormat instead of a buffer
* Don't use C constants/types in error messages
* Make error messages easier to understand for Python users
* Lower minimum required version 1.4.0
* Use casts and make slot function signatures correct
* Be consistent with CPython on const usage
* Make else clauses in line with PEP 7
* Fix over-indented blocks in argument clinic
* Add critical section around ZSTD_DCtx_setParameter
* Add a TODO about refactoring critical sections
* Use Py_UNREACHABLE
* Move bytes operations out of Py_BEGIN_ALLOW_THREADS
* Add TODO about ensuring a lock is held
* Remove asserts that may not be correct
* Add TODO to make ZstdDict and others GC objects
* Make objects GC tracked
* Remove unused include
* Fix some memory issues
* Fix refleaks on module and in ZstdDict
* Update configure to check for ZDICT_finalizeDictionary
* Properly check version in configure
* exit(1) if check fails
* Use AC_RUN_IFELSE
* Use a define() to re-use version check
* Actually properly set _zstd module status based on version
---------
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* Introduces `compression` package for https://peps.python.org/pep-0784/
This commit introduces the `compression` package, specified in PEP 784
to re-export the `lzma`, `bz2`, `gzip`, and `zlib` modules. Introduction
of `compression.zstd` will be completed in a future commit once the
`_zstd` module is merged.
This commit also moves the `_compression` private module to
`compression._common._streams`.
* Re-exports existing module docstrings.
Make `warnings.catch_warnings()` use a context variable for holding
the warning filtering state if the `sys.flags.context_aware_warnings`
flag is set to true. This makes using the context manager thread-safe in
multi-threaded programs.
Add the `sys.flags.thread_inherit_context` flag. If true, starting a new
thread with `threading.Thread` will use a copy of the context
from the caller of `Thread.start()`.
Both these flags are set to true by default for the free-threaded build
and false for the default build.
Move the Python implementation of warnings.py into _py_warnings.py.
Make _contextvars a builtin module.
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
A new extension module, `_hmac`, now exposes the HACL* HMAC (formally verified) implementation.
The HACL* implementation is used as a fallback implementation when the OpenSSL implementation of HMAC
is not available or disabled. For now, only named hash algorithms are recognized and SIMD support provided
by HACL* for the BLAKE2 hash functions is not yet used.
Adds a `use_system_log` config item to enable stdout/stderr redirection for
Apple platforms. This log streaming is then used by a new iOS test runner
script, allowing the display of test suite output at runtime. The iOS test
runner script can be used by any Python project, not just the CPython test
suite.
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Co-authored-by: Ned Deily <nad@python.org>
* Remove the Lib/test/imghdrdata/ directory.
* Copy 5 pictures (gif, png, ppm, pgm, xbm) from removed
Lib/test/imghdrdata/ to a new Lib/test/tkinterdata/ directory.
* Update Sphinx from 4.5 to 6.2 in Doc/requirements.txt.
* socket_helper.transient_internet() no longer imports nntplib to
catch nntplib.NNTPTemporaryError.
* ssltests.py no longer runs test_nntplib.
* "make quicktest" no longer runs test_nntplib.
* WASM: remove nntplib from OMIT_NETWORKING_FILES.
* Remove mentions to nntplib in the email documentation.
This breaks the tests, but we are keeping it as a separate commit so
that the move operation and editing of the moved files are separate, for
a cleaner history.
This merges their code. They're backed by the same single HACL* static library, having them be a single module simplifies maintenance.
This should unbreak the wasm enscripten builds that currently fail due to linking in --whole-archive mode and the HACL* library appearing twice.
Long unnoticed error fixed: _sha512.SHA384Type was doubly assigned and was actually SHA512Type. Nobody depends on those internal names.
Also rename LIBHACL_ make vars to LIBHACL_SHA2_ in preperation for other future HACL things.
Remove the distutils package. It was deprecated in Python 3.10 by PEP
632 "Deprecate distutils module". For projects still using distutils
and cannot be updated to something else, the setuptools project can
be installed: it still provides distutils.
* Remove Lib/distutils/ directory
* Remove test_distutils
* Remove references to distutils
* Skip test_check_c_globals and test_peg_generator since they use
distutils
Add Python implementations of certain longobject.c functions. These use
asymptotically faster algorithms that can be used for operations on
integers with many digits. In those cases, the performance overhead of
the Python implementation is not significant since the asymptotic
behavior is what dominates runtime. Functions provided by this module
should be considered private and not part of any public API.
Co-author: Tim Peters <tim.peters@gmail.com>
Co-author: Mark Dickinson <dickinsm@gmail.com>
Co-author: Bjorn Martinsson
gh-93243
This PR is required to reduce diffs of the following porting (no need to either maintain documentation and tests consistent with each porting step, or try to port everything and remove smtpd in a single PR).
Automerge-Triggered-By: GH:warsaw
This adds a new standard library module, `tomllib`, for parsing TOML.
The implementation is based on Tomli (https://github.com/hukkin/tomli).
## Steps taken (converting `tomli` to `tomllib`)
- Move everything in `tomli:src/tomli` to `Lib/tomllib`. Exclude `py.typed`.
- Remove `__version__ = ...` line from `Lib/tomllib/__init__.py`
- Move everything in `tomli:tests` to `Lib/test/test_tomllib`. Exclude the following test data dirs recursively:
- `tomli:tests/data/invalid/_external/`
- `tomli:tests/data/valid/_external/`
- Create `Lib/test/test_tomllib/__main__.py`:
```python
import unittest
from . import load_tests
unittest.main()
```
- Add the following to `Lib/test/test_tomllib/__init__.py`:
```python
import os
from test.support import load_package_tests
def load_tests(*args):
return load_package_tests(os.path.dirname(__file__), *args)
```
Also change `import tomli as tomllib` to `import tomllib`.
- In `cpython/Lib/tomllib/_parser.py` replace `__fp` with `fp` and `__s` with
`s`. Add the `/` to `load` and `loads` function signatures.
- Run `make regen-stdlib-module-names`
- Create `Doc/library/tomllib.rst` and reference it in `Doc/library/fileformats.rst`
Remove the asyncore and asynchat modules, deprecated in Python
3.6: use the asyncio module instead.
Remove the smtpd module, deprecated in Python 3.6: the aiosmtpd
module can be used instead, it is based on asyncio.
* Remove asyncore, asynchat and smtpd documentation
* Remove test_asyncore, test_asynchat and test_smtpd
* Rename Lib/asynchat.py to Lib/test/support/_asynchat.py
* Rename Lib/asyncore.py to Lib/test/support/_asyncore.py
* Rename Lib/smtpd.py to Lib/test/support/_smtpd.py
* Remove DeprecationWarning from private _asyncore, _asynchat and
_smtpd modules
* _smtpd: remove deprecated properties
Adam Turner