Create basic structure of the asyncio tutorial

Doc/library/asyncio-tutorial/async-functions.rst

@@ -0,0 +1,60 @@
+Functions: Sync vs Async
+========================
+
+Regular Python functions are created with the keyword ``def``,
+and look like this:
+
+.. code-block:: python
+
+    def f(x, y):
+        print(x + y)
+
+    f(1, 2)
+
+The snippet also shows how the function is evaluated. Async functions are
+different in two respects:
+
+.. code-block:: python
+
+    import asyncio
+
+    async def f(x, y):
+        print(x + y)
+
+    asyncio.run(f(1, 2))
+
+The first difference is that the function declaration is spelled
+``async def``. The second difference is that async functions cannot be
+executed by simply evaluating them. Here, we use the ``run()`` function
+from the ``asyncio`` module.
+
+The ``run`` function is only good for executing an async function
+from "synchronous" code; and this is usually only used to execute
+a "main" async function, from which others can be called in a simpler
+way.
+
+That means the following:
+
+.. code-block:: python
+
+    import asyncio
+
+    async def f(x, y):
+        print(x + y)
+
+    async def main():
+        await f(1, 2)
+
+    asyncio.run(main())
+
+
+The execution of the first async function, ``main()``, is performed
+with ``run()``, but once you're inside an ``async def`` function, then
+all you need to execute another async function is the ``await`` keyword.
+
+TODO:
+- which kind of functions can be called from which other kind
+- use the "inspect" module to verify the formal names of functions,
+coroutine functions, coroutines, etc.
+
+

Doc/library/asyncio-tutorial/asyncio-cookbook.rst

@@ -0,0 +1,13 @@
+Asyncio Cookbook
+================
+
+Let's look at a few common situations that will come up in your
+``asyncio`` programs, and how best to tackle them.
+
+TODO
+
+Notes:
+
+- My thinking here was a Q&A style, and then each section has
+  a code snippet demonstrating the answer.
+

Doc/library/asyncio-tutorial/case-study-cli.rst

@@ -0,0 +1,12 @@
+Asyncio Case Study: Chat Application
+====================================
+
+TODO
+
+Notes:
+
+- using the streams API
+- first show the client.
+- will have to explain a message protocol. (But that's easy)
+- then show the server
+- spend some time on clean shutdown.

Doc/library/asyncio-tutorial/case-study-gui.rst

@@ -0,0 +1,20 @@
+Asyncio Case Study: Chat Application with GUI client
+====================================================
+
+TODO
+
+Notes:
+
+- server code remains identical to prior case study
+- focus is on making a nice client
+- The focus area here is: can you use asyncio if there is another
+  blocking "loop" in the main thread? (common with GUIs and games)
+  How do you do that?
+- Mention any special considerations
+- Show and discuss strategies for passing data between main thread
+  (GUI) and the asyncio thread (IO).
+- We can demonstrate the above with tkinter, allowing the
+  case study to depend only on the stdlib
+- Towards the end, mention how the design might change if
+  the client was a browser instead of a desktop client.
+  (can refer to the 3rd party websocket library, or aiohttp)

Doc/library/asyncio-tutorial/index.rst

@@ -0,0 +1,24 @@
+Asyncio Tutorial
+================
+
+Programming with ``async def`` functions is different to normal Python
+functions; enough so that it is useful to explain a bit more
+about what ``asyncio`` is for, and how to use it in typical
+programs.
+
+This tutorial will focus on what an end-user of ``asyncio`` should
+learn to get the most value out of it. Our focus is going to be
+primarily on the "high-level" API, as described in the
+:mod:`documentation <asyncio>`.
+
+
+.. toctree::
+   :maxdepth: 1
+
+   what-asyncio.rst
+   why-asyncio.rst
+   async-functions.rst
+   running-async-functions.rst
+   asyncio-cookbook.rst
+   case-study-cli.rst
+   case-study-gui.rst

Doc/library/asyncio-tutorial/running-async-functions.rst

@@ -0,0 +1,11 @@
+Executing async functions
+=========================
+
+TODO
+
+Notes:
+
+- can be called by other async functions
+- can NOT be called by sync functions
+- can be executed by ``asyncio.run()``
+- can be executed in task by ``asyncio.create_task()``

Doc/library/asyncio-tutorial/what-asyncio.rst

@@ -0,0 +1,123 @@
+What Does Async Mean?
+=====================
+
+Let's make a function that communicates over the network:
+
+.. code-block:: python
+
+    import socket
+
+    def greet(host, port):
+        with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+            s.connect((host, host))
+            s.sendall(b'Hello, world')
+            reply = s.recv(1024)
+
+        print('Reply:', repr(reply))
+
+This function will:
+
+#. make a socket connection to a host,
+#. send a greeting, and
+#. wait for a reply.
+
+The key point here is about the word *wait*: execution proceeds line-by-line
+until the line ``reply = s.recv(1024)``. We call this behaviour
+*synchronous*. At this point, execution pauses
+until we get a reply from the host. This is as we expect:
+Python only ever executes one line at a time.
+
+Now the question comes up: what if you need to send a greeting to
+*multiple* hosts? You could just call it twice, right?
+
+.. code-block:: python
+
+    import socket
+
+    def greet(host, port):
+        with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+            s.connect((host, host))
+            s.sendall(b'Hello, world')
+            reply = s.recv(1024)
+
+        print('Reply:', repr(reply))
+
+    greet(host1, port1)
+    greet(host2, port2)
+
+This works fine, but see that the first call to ``greet`` will completely
+finish before the second call (to ``host2``) can begin.
+
+Computers process things sequentially, which is why programming languages
+like Python also work sequentially; but what we see here is that our
+code is also going to **wait** sequentially. This is, quite literally,
+a waste of time. What we would really like to do here is wait for the
+all the replies *concurrently*, i.e., at the same time.
+
+Operating systems like Windows, Mac and Linux, and others, understand
+this problem deeply. If you're reading this on a computer or even your
+mobile, there will be tens or hundreds of processes running at the same
+time on your device. At least, they will appear to be running
+concurrently.  What is really happening is that the operating system
+is sharing little slices of processor (CPU) time among all the
+processes.  If we started two copies of our ``greet`` program at the
+same time, then they *would* run (and therefore wait) concurrently.
+
+However, there is a price for that: each new process consumes resources
+from the operating system.  But more than that, there is another tricky
+problem about *how* the operating system knows when to allocate
+execution time between each process. The answer: it doesn't! This means
+that the operating system can decide when to give processor time to each
+process.
+
+And this means that you will never be sure of when each of your processes
+is actually running, relative to each other. This is quite safe because
+processes are isolated from each other; however, **threads** are not
+isolated from each other. In fact, the primary feature of threads over
+processes is that multiple threads within a single process can
+access the same memory. And this is where all the problems appear.
+
+So: we can also run the ``greet()`` function in multiple threads, and then
+they will also wait for replies concurrently. However, now you have
+two threads that is allowed to access the same objects, with no control over
+how execution will be transferred between the two threads. This
+situation can result in *race conditions* in how objects are modified,
+and these bugs can be very difficult to debug.
+
+This is where "async" programming comes in. It provides a way to manage
+multiple socket connections all in a single thread; and the best part
+is that you get to control *when* execution is allowed to switch between
+these different contexts.
+
+We will explain more of the details later on in the tutorial,
+but briefly, our earlier example becomes something like the following
+pseudocode:
+
+.. code-block:: python
+
+    import asyncio
+
+    async def greet(host, port):
+        reader, writer = await asyncio.open_connection(host, port)
+        writer.write(b'Hello, world')
+        reply = await reader.recv(1024)
+        writer.close()
+
+        print('Reply:', repr(reply))
+
+    async def main():
+        await asyncio.gather(
+            greet(host1, port1),
+            greet(host2, port2)
+        )
+
+    asyncio.run(main())
+
+There are a couple of new things here, but I want you to focus
+on the new keyword ``await``. Unlike threads, execution is allowed to
+switch between the two ``greet()`` invocations **only** where the
+``await`` keyword appears. On all other lines, execution is exactly the
+same as normal Python.  These ``async def`` functions are called
+"asynchronous" because execution does not pass through the function
+top-down, but instead can suspend in the middle of a function at the
+``await`` keyword, and allow another function to execute.

Doc/library/asyncio-tutorial/why-asyncio.rst

@@ -0,0 +1,35 @@
+Why Asyncio?
+============
+
+There are two very specific reasons for using ``async def`` functions:
+
+#. Safety: easier reasoning about concurrency, and virtually
+   eliminate `memory races <https://en.wikipedia.org/wiki/Race_condition#Software>`_
+   in concurrent network code
+#. High concurrency: huge number of open socket connections
+
+Safety
+------
+
+- async/await makes all context switches visible; that makes it easy
+  to spot race conditions and
+  `reason about your code <https://glyph.twistedmatrix.com/2014/02/unyielding.html>`_
+
+- in general, all datastructures are safe for async (we cannot say same
+  for threads)
+
+- an async/await library means that it's safe to use it in concurrent
+  async/await code (you can never be sure if some library is thread-safe,
+  even if it claims that)
+
+- language constructs like 'async for' and 'async with' enable structured
+  concurrency
+
+High Concurrency
+----------------
+
+- high-throughput IO or 1000s of long-living connections are only
+  doable with asyncio
+
+- if you don't need to scale your code right now but might need
+  in near future investing in async/await is wise

Doc/library/asyncio.rst

@@ -88,6 +88,7 @@ Additionally, there are **low-level** APIs for
    :caption: Guides and Tutorials
    :maxdepth: 1
 
+   asyncio-tutorial/index.rst
    asyncio-api-index.rst
    asyncio-llapi-index.rst
    asyncio-dev.rst