* bpo-39421: Fix posible crash in heapq with custom comparison operators
* fixup! bpo-39421: Fix posible crash in heapq with custom comparison operators
* fixup! fixup! bpo-39421: Fix posible crash in heapq with custom comparison operators
* Add three new bytecodes: LIST_TO_TUPLE, LIST_EXTEND, SET_UPDATE. Use them to implement star unpacking expressions.
* Remove four bytecodes BUILD_LIST_UNPACK, BUILD_TUPLE_UNPACK, BUILD_SET_UNPACK and BUILD_TUPLE_UNPACK_WITH_CALL opcodes as they are now unused.
* Update magic number and dis.rst for new bytecodes.
* bpo-39336: Allow setattr to fail on modules which aren't assignable
When attaching a child module to a package if the object in sys.modules raises an AttributeError (e.g. because it is immutable) it causes the whole import to fail. This now allows immutable packages to exist and an ImportWarning is reported and the AttributeError exception is ignored.
When communicate() is called in a loop, it crashes when the child process
has already closed any piped standard stream, but still continues to be running
Co-authored-by: Andriy Maletsky <andriy.maletsky@gmail.com>
Deprecate binhex4 and hexbin4 standards. Deprecate the binhex module
and the following binascii functions:
* b2a_hqx(), a2b_hqx()
* rlecode_hqx(), rledecode_hqx()
* crc_hqx()
As described in RFC 1952, section 2.3.1, the XFL (eXtra FLags) byte of a
gzip member header should indicate whether the DEFLATE algorithm was
tuned for speed or compression ratio. Prior to this patch, archives
emitted by the `gzip` module always indicated maximum compression.
Remove the buffering parameter of bz2.BZ2File. Since Python 3.0, it
was ignored and using it was emitting a DeprecationWarning. Pass an
open file object to control how the file is opened.
The compresslevel parameter becomes keyword-only.
Remove base64.encodestring() and base64.decodestring(), aliases
deprecated since Python 3.1: use base64.encodebytes() and
base64.decodebytes() instead.
pstats is really useful or profiling and printing the output of the execution of some block of code, but I've found on multiple occasions when I'd like to access this output directly in an easily usable dictionary on which I can further analyze or manipulate.
The proposal is to add a function called get_profile_dict inside of pstats that'll automatically return this data the data in an easily accessible dict.
The output of the following script:
```
import cProfile, pstats
import pprint
from pstats import func_std_string, f8
def fib(n):
if n == 0:
return 0
if n == 1:
return 1
return fib(n-1) + fib(n-2)
pr = cProfile.Profile()
pr.enable()
fib(5)
pr.create_stats()
ps = pstats.Stats(pr).sort_stats('tottime', 'cumtime')
def get_profile_dict(self, keys_filter=None):
"""
Returns a dict where the key is a function name and the value is a dict
with the following keys:
- ncalls
- tottime
- percall_tottime
- cumtime
- percall_cumtime
- file_name
- line_number
keys_filter can be optionally set to limit the key-value pairs in the
retrieved dict.
"""
pstats_dict = {}
func_list = self.fcn_list[:] if self.fcn_list else list(self.stats.keys())
if not func_list:
return pstats_dict
pstats_dict["total_tt"] = float(f8(self.total_tt))
for func in func_list:
cc, nc, tt, ct, callers = self.stats[func]
file, line, func_name = func
ncalls = str(nc) if nc == cc else (str(nc) + '/' + str(cc))
tottime = float(f8(tt))
percall_tottime = -1 if nc == 0 else float(f8(tt/nc))
cumtime = float(f8(ct))
percall_cumtime = -1 if cc == 0 else float(f8(ct/cc))
func_dict = {
"ncalls": ncalls,
"tottime": tottime, # time spent in this function alone
"percall_tottime": percall_tottime,
"cumtime": cumtime, # time spent in the function plus all functions that this function called,
"percall_cumtime": percall_cumtime,
"file_name": file,
"line_number": line
}
func_dict_filtered = func_dict if not keys_filter else { key: func_dict[key] for key in keys_filter }
pstats_dict[func_name] = func_dict_filtered
return pstats_dict
pp = pprint.PrettyPrinter(depth=6)
pp.pprint(get_profile_dict(ps))
```
will produce:
```
{"<method 'disable' of '_lsprof.Profiler' objects>": {'cumtime': 0.0,
'file_name': '~',
'line_number': 0,
'ncalls': '1',
'percall_cumtime': 0.0,
'percall_tottime': 0.0,
'tottime': 0.0},
'create_stats': {'cumtime': 0.0,
'file_name': '/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/cProfile.py',
'line_number': 50,
'ncalls': '1',
'percall_cumtime': 0.0,
'percall_tottime': 0.0,
'tottime': 0.0},
'fib': {'cumtime': 0.0,
'file_name': 'get_profile_dict.py',
'line_number': 5,
'ncalls': '15/1',
'percall_cumtime': 0.0,
'percall_tottime': 0.0,
'tottime': 0.0},
'total_tt': 0.0}
```
As an example, this can be used to generate a stacked column chart using various visualization tools which will assist in easily identifying program bottlenecks.
https://bugs.python.org/issue37958
Automerge-Triggered-By: @gpshead
On Unix, subprocess.Popen.send_signal() now polls the process status.
Polling reduces the risk of sending a signal to the wrong process if
the process completed, the Popen.returncode attribute is still None,
and the pid has been reassigned (recycled) to a new different
process.
* Reorder the __aenter__ and __aexit__ checks for async with
* Add assertions for async with body being skipped
* Swap __aexit__ and __aenter__ loading in the documentation
Break up COMPARE_OP into four logically distinct opcodes:
* COMPARE_OP for rich comparisons
* IS_OP for 'is' and 'is not' tests
* CONTAINS_OP for 'in' and 'is not' tests
* JUMP_IF_NOT_EXC_MATCH for checking exceptions in 'try-except' statements.
* Add tests for case insensitive check of types and extensions as fallback.
* Add tests for data url with no comma.
* Add tests for read_mime_types.
* Add tests for the mimetypes cli and refactor __main__ code to private function.
* Restore mimetypes.knownfiles value at the end of the test.
Copying property objects results in a TypeError. Steps to reproduce:
```
>>> import copy
>>> obj = property()
>>> copy.copy(obj)
````
This affects both shallow and deep copying.
My idea for a fix is to add property objects to the list of "atomic" objects in the copy module.
These already include types like functions and type objects.
I also added property objects to the unit tests test_copy_atomic and test_deepcopy_atomic. This is my first PR, and it's highly likely I've made some mistake, so please be kind :)
https://bugs.python.org/issue38293
nntplib.NNTP and nntplib.NNTP_SSL now raise a ValueError
if the given timeout for their constructor is zero to
prevent the creation of a non-blocking socket.
poplib.POP3 and poplib.POP3_SSL now raise a ValueError
if the given timeout for their constructor is zero to
prevent the creation of a non-blocking socket.
* Add test cases for dataclasses.
* Add test for repr output of field.
* Add test for ValueError to be raised when both default and default_factory are passed.
imaplib.IMAP4 and imaplib.IMAP4_SSL now have an
optional *timeout* parameter for their constructors.
Also, the imaplib.IMAP4.open() method now has an optional *timeout* parameter
with this change. The overridden methods of imaplib.IMAP4_SSL and
imaplib.IMAP4_stream were applied to this change.
Pablo Galindo