2017-11-10 183 views
1

我的Python軟件死鎖interminttently(可能是由於一些競爭條件)蟒蛇2.7多線程技術在印刷掛

內正在運行的線程中的一個,它運行

while True: 
    msg = printQ.get() 
    print msg 
    sys.stdout.flush() 

在調試時,我看到我的一個線程掛起在 sys.stdout.flush()。如果我刪除了沖洗線,線程會在打印時掛起。 這是打印到stdout

如果停用此線程,我的軟件犯規僵局

有沒有在多線程ENV沖洗標準輸出的任何已知問題的唯一線索?

這些是在軟件死鎖時正在運行線程的跟蹤。什麼東西都跳出來給你們?對我來說,它看起來像睡了幾次,等待隊列中的物品。只是很奇怪我的事情是sys.stdout.flush()

# ThreadID: 139736001795840 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 726, in worker 
    jobref = Host.r.blpop(Dispatch.rkey_completeQ, timeout=1) 
File: "/usr/software/rats/bedrock/lib/python2.7/site-packages/redis/client.py", line 1090, in blpop 
    return self.execute_command('BLPOP', *keys) 
File: "/usr/software/rats/bedrock/lib/python2.7/site-packages/redis/client.py", line 529, in execute_command 
    return self.parse_response(connection, command_name, **options) 
File: "/usr/software/rats/bedrock/lib/python2.7/site-packages/redis/client.py", line 541, in parse_response 
    response = connection.read_response() 
File: "/usr/software/rats/bedrock/lib/python2.7/site-packages/redis/connection.py", line 545, in read_response 
    response = self._parser.read_response() 
File: "/usr/software/rats/bedrock/lib/python2.7/site-packages/redis/connection.py", line 213, in read_response 
    response = self._buffer.readline() 
File: "/usr/software/rats/bedrock/lib/python2.7/site-packages/redis/connection.py", line 151, in readline 
    self._read_from_socket() 
File: "/usr/software/rats/bedrock/lib/python2.7/site-packages/redis/connection.py", line 109, in _read_from_socket 
    data = self._sock.recv(socket_read_size) 

# ThreadID: 139736012285696 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 605, in worker 
    job = Factory.dispQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139736022775552 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 605, in worker 
    job = Factory.dispQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139737333483264 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 236, in worker 
    (job, msg, to_debug) = Factory.logQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139736461068032 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 605, in worker 
    job = Factory.dispQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139736492537600 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 470, in worker 
    job = Factory.rdyQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139737322993408 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 313, in worker 
    sys.stdout.flush() 

# ThreadID: 139736909850368 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 470, in worker 
    job = Factory.rdyQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139736920340224 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 470, in worker 
    job = Factory.rdyQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139737354462976 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 350, in worker 
    item = Factory.streamQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139736930830080 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 470, in worker 
    job = Factory.rdyQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139736471557888 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 470, in worker 
    job = Factory.rdyQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139736951809792 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 470, in worker 
    job = Factory.rdyQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139736962299648 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 470, in worker 
    job = Factory.rdyQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139737474270976 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Status_thread.py", line 131, in run 
    request = self._queue.get(timeout=1) # blocking get with a timeout of 1 second 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 177, in get 
    self.not_empty.wait(remaining) 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 359, in wait 
    _sleep(delay) 

# ThreadID: 139736450578176 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 605, in worker 
    job = Factory.dispQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139737364952832 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Stack_Tracer.py", line 56, in run 
    self.stacktraces() 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Stack_Tracer.py", line 70, in stacktraces 
    fout.write(stacktraces()) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Stack_Tracer.py", line 22, in stacktraces 
    for filename, lineno, name, line in traceback.extract_stack(stack): 

# ThreadID: 139737343973120 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 416, in worker 
    time.sleep(1) 

# ThreadID: 139736440088320 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 605, in worker 
    job = Factory.dispQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139736482047744 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 470, in worker 
    job = Factory.rdyQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139737312503552 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 470, in worker 
    job = Factory.rdyQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

# ThreadID: 139737628997376 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Br.py", line 86, in <module> 
    Command.main() 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Command.py", line 149, in main 
    com.run(args) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Btool.py", line 181, in run 
    self.make_locked(args) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Btool.py", line 227, in make_locked 
    Dispatch.run_jobs() 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch.py", line 732, in run_jobs 
    # we got any hung jobs?? 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch.py", line 332, in check_for_hung_jobs 
    really_running = job.log_check() 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Job.py", line 391, in log_check 
    if Checkproc.check_remote_process(Options.options.rsh_cmd, lock.hostname, lock.pid, 20): 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Checkproc.py", line 92, in check_remote_process 
    time.sleep(1) 

# ThreadID: 139736941319936 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 783, in __bootstrap 
    self.__bootstrap_inner() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
File: "/u/eugenep/bedrock/source/br-new-release/lib/btool/Dispatch_Factory.py", line 470, in worker 
    job = Factory.rdyQ.get() 
File: "/usr/software/rats/bedrock/lib/python2.7/Queue.py", line 168, in get 
    self.not_empty.wait() 
File: "/usr/software/rats/bedrock/lib/python2.7/threading.py", line 340, in wait 
    waiter.acquire() 

回答

0

AFAIK sys.stdout的是不是真的線程安全的,但沒有任何代碼,它實際上是很難說。出於調試目的,您可以嘗試鎖定sys.stdout.flush,例如:

orig_flush = sys.stdout.flush 
flush_lock = threading.Lock() 

def myflush(): 
    with flush_lock: 
     orig_flush() 

sys.stdout.flush = myflush 

# ... eventually: 
sys.stdout.flush = orig_flush