What does (yield) do in python?

I browsed through the documentation for the Python package m4us and found a "simple example". In it, there are yield statements.

@coroutine(lazy=False)
def lines_producer(file_):
    """Emit lines from a file as messages."""
    inbox, message = (yield)
    for line in file_:
        if is_shutdown(inbox, message):
            yield 'signal', message
            break
        inbox, message = (yield 'outbox', line)
    (yield 'signal', ProducerFinished())

Ok got that, kind of. Yield is followed by some arguments that it, well yields. But in the code I found this:

inbox, message = (yield)

Uhm, what is that? It yields something into the method we are in. That's like the other way around. But from where?

I've made a test script sheds some light:

def a_generator():
    message = yield('return value of first yield')
    message2 = yield(message)
    message3 = yield(message2)

g = a_generator()
print "first send:", g.send(None)
print "second send:", g.send('Second send')
print "third send:", g.send('third send')

First, a function is defined. It has the name "a_generator", but it could be anything. However the fact that it contains at least one yield statement means that python will treat the function as a generator.

Now, on the first send, the method is executed until it reaches the first yield. That yield then returns to the caller, in this case with the string "return value of first yield". But it is only half the yield that is executed by the first send! For the next send, the execution is resumed in such a way that the first yield is the input point for the argument of the second send. The string "Second send" is assigned to the variable "message" on the second send.

So the first send used the first yield to get something back. But there is more life left in that yield: That yield's ability to inject something into the method has not been used up yet, the second send will do that. The python documentation on yield talks about suspending and resuming execution, I did not at first realize that it suspends and resumes right in the the middle of the yield word, so to speak.

Since the first yield returns, the first call to send or next can feel a bit futile, if you intend to send stuff into generator. Your first send cannnot contain any useful parameters. One way of dealing with this is to wrap the function inside a decorator that takes care of calling next or send the first time. See David Beazley's script here.

Reading further on Beazley's site it turns out he has made an excellent presentation on the topic. It turns out that the usage of yield with send is not used so much with generators but rather with something called coroutines, and my guess is that the code at the top of this blog posting is coroutine-oriented, so my use of the term generator in this post may be a bit out of place.