Did you ever wanted to know how to call a process from Python and get the output instantly, instead of waiting for the execution to finish?

Try this:

import os
import sys
import subprocess

s = subprocess.Popen([“tail”, “/tmp/wee”, “-f”],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
follow = s.stdout
while s.poll() == None:
mess = follow.readline()
if mess:
print mess.strip()

ret = s.wait()
sys.exit(0)

Now go ahead and type in another terminal: echo PRINTME » /tmp/wee

And then go weeee!

Archived Comments

Comment from …

Yves Junqueira - Mar 1, 2007

Comment from Teste:

Testando.

Comment from Colin Walters:

I dug into this problem and it turns out that Python’s internal implementation of the readlines() iteration has an internal 8192 character buffer, even if you specify unbuffered I/O.

So yeah, the fix to do unbuffered line reading in Python is:

def readlines_unbuffered(stream):
line = stream.readline()
while line:
yield line
line = stream.readline()

Comment from Tal Einat:

It’s worth noting that for some programs, the output may be buffered internally, so the above won’t avoid all buffering.

For example, C’s stdio library doesn’t buffer output when it is started from a tty (a terminal), but does buffer otherwise. In this specific case, you can avoid buffering by fooling the process into thinking it was run in a terminal using pexpect (doesn’t work on windows).

This post has more details:
http://groups.google.com/group/comp.lang.python/tree/browse_frm/thread/8abcc616d0ce419d/4981ae9dd98f9166?rnum=1&_done=%2Fgroup%2Fcomp.lang.python%2Fbrowse_frm%2Fthread%2F8abcc616d0ce419d%2F4981ae9dd98f9166%3F#doc_82e72b4babce0c7a