How we can use
yield from to implement coroutines
Give my new understanding of generator functions and
yield from, we can now see how to use
from to implement coroutines and an event loop. Consider a three
level stack of functions, where on the top layer you have an event
loop, in the middle you have the processing code you write, and on
the bottom are event functions like
Let's start with an example processing function or two:
def countdown(n): while n: print("T-minus", n) n -= 1 yield from sleep(1) def launch(what, nsecs): print("counting down for", what) yield from countdown(nsecs) print("launching", what)
To start a launch, we call something like '
10))', which looks a bit peculiar since it sort of seems like
coro.start() should get control only after the launch. However,
we already know that calling a generator function doesn't do exactly
what it looks like. What
coro.start() gets when we do this is an
unstarted generator object (which handily encapsulates those
launch(), so we don't have to do it by hand).
When the coroutine scheduler starts the
launch() generator object,
we wind up with a chain of
yield froms that bottoms out at
sleep() yields is passed back up to the coroutine
scheduler and the entire call chain is suspended; this is no different
that what I did by calling
.send() by hand yesterday. What
sleep() returns to the
scheduler is an object (call it an event object) that tells the
coroutine scheduler under what conditions this coroutine should be
resumed. When the scheduler reaches the point that the coroutine
should be run again, the scheduler will once again call
which will resume execution in
sleep(), which will then return
countdown(), and so on. The scheduler may use this
to pass information back to
sleep(), such as how long it took
before the coroutine was restarted.
yield from are being used for two things. First,
they create a communication channel between the coroutine scheduler
and the low-level event functions like
countdown() functions are oblivious to this since they don't
touch either the value
sleep() yields up to the scheduler or the
value that the scheduler injects to
the chain of
yield from and the final
yield neatly suspend the
entire call stack.
In order for this to work reliably, there are two rules that our
user-written processing functions have to follow. First, they must
never accidentally attempt to do anything with the
function. It is okay but unclear for a non-generator function to
sleep() and return the result:
def sleep_minutes(n): return sleep(n * 60) def long_countdown(n): while n: print("T-minus", n, "minutes") yield from sleep_minutes(1) n -= 1
This is ultimately because '
yield from func()' is equivalent to
t = func(); yield from t'. We don't care just how the generator
object got to us so we can
yield from it, we just care that it
However, at no stage in our processing functions can we attempt to
look at the results of iterating
sleep()'s generator object, either
directly or indirectly by writing, say, '
for i in countdown(10):'.
This rules out certain patterns for writing processing functions, for
instance this one:
def label_each_sec(label, n): for _ in tick_once_per_sec(n): print(label)
This leads to the second rule, which is that we must have an
unbroken chain of
yield froms from the top to the bottom of our
processing functions, right down to where you use an event function
sleep(). Each function must 'call' the next using the
yield from func()' idiom. In effect we don't have calls from one
processing function to another; instead we're passing control from
one function to the next. In my example,
launch() passes control
countdown() until the countdown expires (and
passes control to
sleep()). If we actually call a processing
function normally or accidentally use '
yield' instead of '
from', the entire collection explodes into various sorts of errors
without getting off the launch pad and you will not go to space
As you might imagine, this is a little bit open to errors. Under
normal circumstances you'll catch the errors fairly fast (when
your main code doesn't work). However, since errors can only be
caught at runtime when a non-
yield from code path is reached,
you may have mistakes that lurk in rarely executed code paths.
Perhaps you have a rarely invoked last moment launch abort:
def launch(what, nsecs): print("counting down for", what) yield from countdown(nsecs) if launch_abort: print("Aborting launch! Clear the launch pad for", what) yield sleep(1) print("Flooding fire suppression ...") else: print("launching", what)
It might be a while before you discovered that mistake (I'm doing
a certain amount of hand-waving about early aborts in
(See also my somewhat related attempt at understanding this sort
help asynchronous programming.
Note that you can't use my particular approach from that entry in
Python with '
yield from' for reasons beyond the scope of this
Comments on this page:Written on 16 March 2017.