Some notes on abusing the
pexpect Python module
What you are theoretically supposed to use pexpect for is to have your program automatically interact with interactive programs. When they produce certain sorts of output, you recognize it and take action; when you see prompts, you can automatically answer them. Pexpect is often used this way to automate things that expect to be operated manually by a real person. This is not what I'm using pexpect for. What I'm using it for is to start a program in what it thinks is an interactive environment, capture its output if all goes well, and if things go wrong allow a human operator to step in and interact with the program (all the while still capturing the output). This means that I'm ignoring almost all of pexpect's functionality and abusing parts of the rest in ways that it was probably not designed for.
Before I start, I need to throw in a disclaimer. There are multiple versions of pexpect out there; my impression is that development stalled for a while and then picked up recently. As I write this, the pexpect documentation talks about 4.0.1, but what I've used is no later than 3.1. Pexpect 4 may fix some of the issues I'm going to grumble about.
Supposing that my case is what you want to do, you start out by spawning a command:
child = pexpect.spawn(YOURCOMMAND, args=args, timeout=None)
It's important to set a timeout of
None as the starting timeout.
If you want to have a timeout at all, for example to detect that
the remote end has gone silent, you want to control it on a call
by call basis.
Now you want to collect output from the child command:
res =  while not child.closed and child.isalive(): try: r = child.read_nonblocking(size = 16*1024, timeout=YOURTIMEOUT) res.append(r) except pexpect.EOF: # expected, just stop break except pexpect.TIMEOUT: # do whatever you want to recover return recover_child(child, res)
You might as well set
size to large here. Although the documentation
doesn't tell you this, it is just the maximum amount of data your
read can ever return; it doesn't block until that much data is
available. My principle is 'if the command generates a lot of output,
let's read it in big blocks'.
We're not done once pexpect has raised an EOF. We need to do some cleanup to make sure that the child's exit status is available:
# Some of this is probably superstition if not child.closed and child.isalive(): child.wait() return (res, child.status)
Pexpect 3.1's documentation is not entirely clear on what you have
to check when in order to see if the child is alive or not. Note
.isalive() has the (useful) side effect of harvesting the
child's exit status if the child is not alive. It's helpfully not
valid to call
.wait() on a dead child, at least in 3.1, so you
have to check carefully first.
As pexpect documents, it splits the actual OS process exit status
child.signalstatus (and various things
return one or the other). The whole status is available as
child.status, but you may find one or the other variant more
useful (for example if you're really only interested in 'did the
command exit with status 0 or did something go boom').
Allowing the user to interact with the child is somewhat more
involved. Fundamentally we call
but there is a bunch of things that you need to do around this.
def talkto(child): # Set up to log interactive output res =  def save_output(data): if data: res.append(data) return data while not child.closed and child.isalive(): try: child.interact(output_filter=save_output) except OSError as e: # Usually an EOF from the command. # Complain somehow. break # If the child is alive here, the user has # typed a ^] to escape from interact(). # What happens next is up to you.
Yes, you read that right. Uniquely, pexpect's
does not raise
pexpect.EOF on EOF from the child; instead it
generally passes through an underlying
OSError that it got (my
notes don't say what that OSError usually is). In general, if you
OSError here you have to assume that the session is dead,
although pexpect doesn't necessarily know it yet.
child.interact() sets things up so that control characters
and so on that the user types are normally passed through directly
to the child process instead of affecting your Python program. This
means that under normal circumstances, if you type eg ^C your Python
code won't get hit with a
SIGINT; it'll go through to the child
program and the child program will do whatever it does in reaction.
What you do if the user chooses to use ^[ to exit from
is up to you. Note that you can allow them to resume the interaction;
just go back through your loop to call
If you allow the user to abandon the child and exit your
function (you probably want to), you need to do some more cleanup
of the child:
# after interact() returns, try to # read anything left over, then close the child. try: r = child.read_nonblocking(size=128*1024, timeout=0) res.append(r) except (pexpect.EOF, pexpect.TIMEOUT, OSError): pass child.close(force=True)
timeout=0 means what you think
it does; it's a non-blocking read of whatever (final) data is
available right now, with no waiting for anything more to come in
from the child.
At least in pexpect 3.1, you basically should call
force=True or you will get a pexpect error if the child stays
alive, which it may. Setting
force winds up hitting the child
SIGKILL if nothing else seems to work, which is relatively
(Although the documentation doesn't mention it, if the child is
alive it always gets sent
SIGHUP and then
SIGINT first. Well,
this happens in older versions of pexpect; the 4.0.1 code is a bit
different and I haven't dug through it.)
Possibly there is a better Python module for this sort of interaction in general. If so, it is too late for me; I've already written all of this code and I hope to not have to touch it again before we have to port it to Python 3 (if ever).
(My impression is that you should try to use pexpect 4 if you can, as the code has been overhauled and the documentation at least somewhat improved.)