10
我想執行一個進程，限制執行時間以秒爲單位的超時並獲取進程產生的輸出。我想在windows，linux和freebsd上執行此操作。蟒蛇子進程超時和大輸出（> 64K）
我曾嘗試三種不同的方式實現這一點：
CMD - 沒有超時和subprocess.PIPE輸出捕獲。
行爲：操作與預期，但不支持超時，我需要超時...
cmd_to - 隨着超時和subprocess.PIPE輸出捕獲。
行爲：輸出> = 2^16字節時阻止子進程執行。
cmd_totf - 用於輸出捕獲的超時和tempfile.NamedTemporaryfile。
行爲：按預期方式運行，但使用磁盤上的臨時文件。
這些可以在下面查看。
從下面的輸出中可以看出，當使用子處理時，超時碼會阻止子進程的執行.PIPE和子進程的輸出大於等於2^16字節。
子進程文檔指出，在調用process.wait（）和使用subprocessing.PIPE時，這是預期的，但是在使用process.poll（）時沒有給出警告，那麼這裏出了什麼問題？
我有一個在cmd_totf中使用tempfile模塊的解決方案，但是權衡是它將輸出寫入磁盤，這是我真的很想避免的。
所以我的問題是：
- 我是什麼在cmd_to做錯了嗎？
- 有沒有辦法做到我想要的，而不使用臨時文件/保持輸出內存。
腳本來生成一束輸出（ 'exp_gen.py'）的：包裝紙
#!/usr/bin/env python
import sys
output = "b"*int(sys.argv[1])
print output
三種不同的實施方式（CMD，cmd_to，cmd_totf）周圍subprocessing.Popen：
#!/usr/bin/env python
import subprocess, time, tempfile
bufsize = -1
def cmd(cmdline, timeout=60):
"""
Execute cmdline.
Uses subprocessing and subprocess.PIPE.
"""
p = subprocess.Popen(
cmdline,
bufsize = bufsize,
shell = False,
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
out, err = p.communicate()
returncode = p.returncode
return (returncode, err, out)
def cmd_to(cmdline, timeout=60):
"""
Execute cmdline, limit execution time to 'timeout' seconds.
Uses subprocessing and subprocess.PIPE.
"""
p = subprocess.Popen(
cmdline,
bufsize = bufsize,
shell = False,
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
t_begin = time.time() # Monitor execution time
seconds_passed = 0
while p.poll() is None and seconds_passed < timeout:
seconds_passed = time.time() - t_begin
time.sleep(0.1)
#if seconds_passed > timeout:
#
# try:
# p.stdout.close() # If they are not closed the fds will hang around until
# p.stderr.close() # os.fdlimit is exceeded and cause a nasty exception
# p.terminate() # Important to close the fds prior to terminating the process!
# # NOTE: Are there any other "non-freed" resources?
# except:
# pass
#
# raise TimeoutInterrupt
out, err = p.communicate()
returncode = p.returncode
return (returncode, err, out)
def cmd_totf(cmdline, timeout=60):
"""
Execute cmdline, limit execution time to 'timeout' seconds.
Uses subprocessing and tempfile instead of subprocessing.PIPE.
"""
output = tempfile.NamedTemporaryFile(delete=False)
error = tempfile.NamedTemporaryFile(delete=False)
p = subprocess.Popen(
cmdline,
bufsize = 0,
shell = False,
stdin = None,
stdout = output,
stderr = error
)
t_begin = time.time() # Monitor execution time
seconds_passed = 0
while p.poll() is None and seconds_passed < timeout:
seconds_passed = time.time() - t_begin
time.sleep(0.1)
#if seconds_passed > timeout:
#
# try:
# p.stdout.close() # If they are not closed the fds will hang around until
# p.stderr.close() # os.fdlimit is exceeded and cause a nasty exception
# p.terminate() # Important to close the fds prior to terminating the process!
# # NOTE: Are there any other "non-freed" resources?
# except:
# pass
#
# raise TimeoutInterrupt
p.wait()
returncode = p.returncode
fd = open(output.name)
out = fd.read()
fd.close()
fd = open(error.name)
err = fd.read()
fd.close()
error.close()
output.close()
return (returncode, err, out)
if __name__ == "__main__":
implementations = [cmd, cmd_to, cmd_totf]
bytes = ['65535', '65536', str(1024*1024)]
timeouts = [5]
for timeout in timeouts:
for size in bytes:
for i in implementations:
t_begin = time.time()
seconds_passed = 0
rc, err, output = i(['exp_gen.py', size], timeout)
seconds_passed = time.time() - t_begin
filler = ' '*(8-len(i.func_name))
print "[%s%s: timeout=%d, iosize=%s, seconds=%f]" % (repr(i.func_name), filler, timeout, size, seconds_passed)
輸出：
['cmd' : timeout=5, iosize=65535, seconds=0.016447]
['cmd_to' : timeout=5, iosize=65535, seconds=0.103022]
['cmd_totf': timeout=5, iosize=65535, seconds=0.107176]
['cmd' : timeout=5, iosize=65536, seconds=0.028105]
['cmd_to' : timeout=5, iosize=65536, seconds=5.116658]
['cmd_totf': timeout=5, iosize=65536, seconds=0.104905]
['cmd' : timeout=5, iosize=1048576, seconds=0.025964]
['cmd_to' : timeout=5, iosize=1048576, seconds=5.128062]
['cmd_totf': timeout=5, iosize=1048576, seconds=0.103183]
試試從http://stackoverflow.com/questions/874815/how-do-i-get-real-time-information-back-from-a-subprocess-popen-in-python-2-5回答。 – 2010-12-04 21:05:31
你應該提到哪個版本的python。由於AFAIK，有相當多的變化從2.6到2.7相關的'subprocess'模塊 – 2011-12-12 04:19:49
也參見http://stackoverflow.com/questions/1191374/subprocess-with-timeout/8507775#8507775 – bortzmeyer 2011-12-14 16:15:09