This PR deprecate explicit loop parameters in all public asyncio APIs
This issues is split to be easier to review.
Third step: locks.py
https://bugs.python.org/issue36373
When the test is ran with `PYTHONWARNINGS=error` the environment variable is passed to the python interpreter used in `assert_python_ok` where `DeprecationWarning` from `@asyncio.coroutine` is converted into an error. Ignore the `DeprecationWarning` in `assert_python_ok`.
https://bugs.python.org/issue37323
Replace asyncio.set_event_loop() with TestCase.set_event_loop() of
test_asyncio.utils: this method calls TestCase.close_loop() which
waits until the executor completes, to avoid leaking dangling
threads.
Inherit from test_asyncio.utils.TestCase rather than
unittest.TestCase.
This will address the common mistake many asyncio users make:
an "except Exception" clause breaking Tasks cancellation.
In addition to this change, we stop inheriting asyncio.TimeoutError
and asyncio.InvalidStateError from their concurrent.futures.*
counterparts. There's no point for these exceptions to share the
inheritance chain.
In 3.9 we'll focus on implementing supervisors and cancel scopes,
which should allow better handling of all exceptions, including
SystemExit and KeyboardInterrupt
When the future returned by shield is cancelled, its completion callback of the
inner future is not removed. This makes the callback list of inner inner future
grow each time a shield is created and cancelled.
This change unregisters the callback from the inner future when the outer
future is cancelled.
https://bugs.python.org/issue35125
inspect.isfunction() processes both inspect.isfunction(func) and
inspect.isfunction(partial(func, arg)) correctly but some other functions in the
inspect module (iscoroutinefunction, isgeneratorfunction and isasyncgenfunction)
lack this functionality. This commits adds a new check in the mentioned functions
in the inspect module so they can work correctly with arbitrarily nested partial
functions.
The C implementation of asyncio.Task currently fails to perform the
cancellation cleanup correctly in the following scenario.
async def task1():
async def task2():
await task3 # task3 is never cancelled
asyncio.current_task().cancel()
await asyncio.create_task(task2())
The actuall error is a hardcoded call to `future_cancel()` instead of
calling the `cancel()` method of a future-like object.
Thanks to Vladimir Matveev for noticing the code discrepancy and to
Yury Selivanov for coming up with a pathological scenario.
* Insert the warn in the asyncio.sleep when the loop argument is used
* Insert the warn in the asyncio.wait and asyncio.wait_for when the loop argument is used
* Better format of the code
* Add news file
* change calls for get_event_loop() to calls for get_running_loop()
* Change message to be more clear in News
* Improve the comments in test_tasks
Fix "<CoroWrapper ...> was never yielded from" warning in
PyTask_PyFuture_Tests.test_error_in_call_soon() of
test_asyncio.test_tasks.
Close manually the coroutine on error.
Currently, asyncio.wait_for(fut), upon reaching the timeout deadline,
cancels the future and returns immediately. This is problematic for
when *fut* is a Task, because it will be left running for an arbitrary
amount of time. This behavior is iself surprising and may lead to
related bugs such as the one described in bpo-33638:
condition = asyncio.Condition()
async with condition:
await asyncio.wait_for(condition.wait(), timeout=0.5)
Currently, instead of raising a TimeoutError, the above code will fail
with `RuntimeError: cannot wait on un-acquired lock`, because
`__aexit__` is reached _before_ `condition.wait()` finishes its
cancellation and re-acquires the condition lock.
To resolve this, make `wait_for` await for the task cancellation.
The tradeoff here is that the `timeout` promise may be broken if the
task decides to handle its cancellation in a slow way. This represents
a behavior change and should probably not be back-patched to 3.6 and
earlier.
Specifically, it's not possible to subclass Task/Future classes
and override the following methods:
* Future._schedule_callbacks
* Task._step
* Task._wakeup
* Convert asyncio/tasks.py to async/await
* Convert asyncio/queues.py to async/await
* Convert asyncio/test_utils.py to async/await
* Convert asyncio/base_subprocess.py to async/await
* Convert asyncio/subprocess.py to async/await
* Convert asyncio/streams.py to async/await
* Fix comments
* Convert asyncio/locks.py to async/await
* Convert asyncio.sleep to async def
* Add a comment
* Add missing news
* Convert stubs from AbstrctEventLoop to async functions
* Convert subprocess_shell/subprocess_exec
* Convert connect_read_pipe/connect_write_pip to async/await syntax
* Convert create_datagram_endpoint
* Convert create_unix_server/create_unix_connection
* Get rid of old style coroutines in unix_events.py
* Convert selector_events.py to async/await
* Convert wait_closed and create_connection
* Drop redundant line
* Convert base_events.py
* Code cleanup
* Drop redundant comments
* Fix indentation
* Add explicit tests for compatibility between old and new coroutines
* Convert windows event loop to use async/await
* Fix double awaiting of async function
* Convert asyncio/locks.py
* Improve docstring
* Convert tests to async/await
* Convert more tests
* Convert more tests
* Convert more tests
* Convert tests
* Improve test
when there are no more `await` or `yield (from)` before return in coroutine,
cancel was ignored.
example:
async def coro():
asyncio.Task.current_task().cancel()
return 42
...
res = await coro() # should raise CancelledError
This implementation provides additional 10-20% speed boost for
asyncio programs.
The patch also fixes _asynciomodule.c to use Arguments Clinic, and
makes '_schedule_callbacks' an overridable method (as it was in 3.5).
Issue #26777: Fix random failing of the test on the "AMD64 FreeBSD 9.x 3.5"
buildbot:
File ".../Lib/test/test_asyncio/test_tasks.py", line 2398, in go
self.assertTrue(0.09 < dt < 0.11, dt)
AssertionError: False is not true : 0.11902812402695417
Replace "< 0.11" with "< 0.15".
Don't raise a TimeoutError if we reached the timeout and the future completed
in the same iteration of the event loop. A side effect of the bug is that
Queue.get() looses items.
* _WaitHandleFuture.cancel() now notify IocpProactor through the overlapped
object that the wait was cancelled.
* Optimize IocpProactor.wait_for_handle() gets the result if the wait is
signaled immediatly.
* Enhance representation of Future and Future subclasses
- Add "created at filename:lineno" in the representation
- Add Future._repr_info() method which can be more easily overriden than
Future.__repr__(). It should now be more easy to enhance Future
representation without having to modify each subclass. For example,
_OverlappedFuture and _WaitHandleFuture get the new "created at" information.
- Use reprlib to format Future result, and function arguments when formatting a
callback, to limit the length of the representation.
* Fix repr(_WaitHandleFuture)
* _WaitHandleFuture and _OverlappedFuture: hide frames of internal calls in the
source traceback.
* Cleanup ProactorIocp._poll(): set the timeout to 0 after the first call to
GetQueuedCompletionStatus()
* test_locks: close the temporary event loop and check the condition lock
* Remove workaround in test_futures, no more needed
* Tulip issue #182: Improve logs of BaseEventLoop._run_once()
- Don't log non-blocking poll
- Only log polling with a timeout if it gets events or if it timed out after
more than 1 second.
* Fix some pyflakes warnings: remove unused imports
- CoroWrapper.__del__() now reuses repr(CoroWrapper) to log the "... was never
yielded from" warning
- Improve CoroWrapper: copy also the qualified name on Python 3.4, not only on
Python 3.5+
- repr(Task) and repr(CoroWrapper) now also includes where these objects were
created. If the coroutine is not a generator (don't use "yield from"), use
the location of the function, not the location of the coro() wrapper.
- Fix create_task(): truncate the traceback to hide the call to create_task().
- Tulip issue 185: Add a create_task() method to event loops. The create_task()
method can be overriden in custom event loop to implement their own task
class. For example, greenio and Pulsar projects use their own task class. The
create_task() method is now preferred over creating directly task using the
Task class.
- tests: fix a warning
- fix typo in the name of a test function
- Update AbstractEventLoop: add new event loop methods; update also the unit test
* _UnixSubprocessTransport: fix file mode of stdin. Open stdin in write mode,
not in read mode
* Examples: close the event loop at exit
* More reliable CoroWrapper.__del__. If the constructor is interrupted by
KeyboardInterrupt or the coroutine objet is destroyed lately, some the
_source_traceback attribute doesn't exist anymore.
* repr(Task): include also the future the task is waiting for
Yury Selivanov