bpo-42340: Document issues around KeyboardInterrupt (GH-23255)

Update documentation to note that in some circumstances,
KeyboardInterrupt may cause code to enter an inconsistent state. Also
document sample workaround to avoid KeyboardInterrupt, if needed.
(cherry picked from commit d0906c90fc)

Co-authored-by: benfogle <benfogle@gmail.com>

Doc/library/exceptions.rst

@@ -234,6 +234,15 @@ The following exceptions are the exceptions that are usually raised.
    accidentally caught by code that catches :exc:`Exception` and thus prevent
    the interpreter from exiting.
 
+   .. note::
+
+      Catching a :exc:`KeyboardInterrupt` requires special consideration.
+      Because it can be raised at unpredictable points, it may, in some
+      circumstances, leave the running program in an inconsistent state. It is
+      generally best to allow :exc:`KeyboardInterrupt` to end the program as
+      quickly as possible or avoid raising it entirely. (See
+      :ref:`handlers-and-exceptions`.)
+
 
 .. exception:: MemoryError
 

Doc/library/signal.rst

@@ -46,6 +46,9 @@ This has consequences:
   arbitrary amount of time, regardless of any signals received.  The Python
   signal handlers will be called when the calculation finishes.
 
+* If the handler raises an exception, it will be raised "out of thin air" in
+  the main thread. See the :ref:`note below <handlers-and-exceptions>` for a
+  discussion.
 
 .. _signals-and-threads:
 
@@ -677,3 +680,70 @@ Do not set :const:`SIGPIPE`'s disposition to :const:`SIG_DFL`
 in order to avoid :exc:`BrokenPipeError`.  Doing that would cause
 your program to exit unexpectedly also whenever any socket connection
 is interrupted while your program is still writing to it.
+
+.. _handlers-and-exceptions:
+
+Note on Signal Handlers and Exceptions
+--------------------------------------
+
+If a signal handler raises an exception, the exception will be propagated to
+the main thread and may be raised after any :term:`bytecode` instruction. Most
+notably, a :exc:`KeyboardInterrupt` may appear at any point during execution.
+Most Python code, including the standard library, cannot be made robust against
+this, and so a :exc:`KeyboardInterrupt` (or any other exception resulting from
+a signal handler) may on rare occasions put the program in an unexpected state.
+
+To illustrate this issue, consider the following code::
+
+    class SpamContext:
+        def __init__(self):
+            self.lock = threading.Lock()
+
+        def __enter__(self):
+            # If KeyboardInterrupt occurs here, everything is fine
+            self.lock.acquire()
+            # If KeyboardInterrupt occcurs here, __exit__ will not be called
+            ...
+            # KeyboardInterrupt could occur just before the function returns
+
+        def __exit__(self, exc_type, exc_val, exc_tb):
+            ...
+            self.lock.release()
+
+For many programs, especially those that merely want to exit on
+:exc:`KeyboardInterrupt`, this is not a problem, but applications that are
+complex or require high reliability should avoid raising exceptions from signal
+handlers. They should also avoid catching :exc:`KeyboardInterrupt` as a means
+of gracefully shutting down.  Instead, they should install their own
+:const:`SIGINT` handler. Below is an example of an HTTP server that avoids
+:exc:`KeyboardInterrupt`::
+
+    import signal
+    import socket
+    from selectors import DefaultSelector, EVENT_READ
+    from http.server import HTTPServer, SimpleHTTPRequestHandler
+
+    interrupt_read, interrupt_write = socket.socketpair()
+
+    def handler(signum, frame):
+        print('Signal handler called with signal', signum)
+        interrupt_write.send(b'\0')
+    signal.signal(signal.SIGINT, handler)
+
+    def serve_forever(httpd):
+        sel = DefaultSelector()
+        sel.register(interrupt_read, EVENT_READ)
+        sel.register(httpd, EVENT_READ)
+
+        while True:
+            for key, _ in sel.select():
+                if key.fileobj == interrupt_read:
+                    interrupt_read.recv(1)
+                    return
+                if key.fileobj == httpd:
+                    httpd.handle_request()
+
+    print("Serving on port 8000")
+    httpd = HTTPServer(('', 8000), SimpleHTTPRequestHandler)
+    serve_forever(httpd)
+    print("Shutdown...")

Misc/NEWS.d/next/Documentation/2020-11-12-21-26-31.bpo-42340.apumUL.rst

@@ -0,0 +1,3 @@
+Document that in some circumstances :exc:`KeyboardInterrupt` may cause the
+code to enter an inconsistent state. Provided a sample workaround to avoid
+it if needed.