python subprocess.Popen塊在futex_狀態

我試圖實現一些基本的調度，最多產生n個進程。一次打開並等待它們執行。爲此，我做了：python subprocess.Popen塊在futex_狀態

CHECKING_INTERVAL = 10 

class StandAloneClient(object): 
    """ 
    Instead of communicating with a backend cluster, fire locally a new thread. 
    """ 
    max_thread_nr = 4 
    thread_pool = [] 
    pending_operations = Queue.Queue(0) 

    class OperationExecutor(threading.Thread): 

     def run(self): 
      """ 
      Get the required data from the operation queue and launch the operation. 
      """ 
      while True: 
       launch_data = StandAloneClient.pending_operations.get() 
       if launch_data != None: 
        operation_id = launch_data[0] 
        user_name_label = launch_data[1] 
        LOGGER.debug("Launching operation " + str(operation_id) + " with name " + str(user_name_label)) 
        ## Create a new process for the new launched operation 
        oper = ['python', '-m', 'processRunner', str(operation_id), user_name_label] 
        launched_process = Popen(oper, stdout=PIPE, stdin=PIPE, stderr=PIPE) 
#     launched_process.wait() 
#     while launched_process.poll() is None: 
#      sleep(CHECKING_INTERVAL) 
#      LOGGER.debug("Operation id=%s is still running. Going to sleep for %s seconds."%(operation_id, 
#                           CHECKING_INTERVAL)) 
        LOGGER.debug("===========================================================") 
        LOGGER.debug("Finished operation %s succesfully."%(operation_id,)) 


    def __init__(self): 
     """ 
     If there are still empty spots create a new executor thread. 
     """ 
     for _ in xrange(self.max_thread_nr - len(self.thread_pool)): 
      new_executor = StandAloneClient.OperationExecutor() 
      self.thread_pool.append(new_executor) 
      new_executor.start() 

    @staticmethod 
    def execute(operation_id, user_name_label="Unknown"): 
     """Start asynchronous operation locally""" 
     StandAloneClient.pending_operations.put((operation_id, user_name_label))

我加入操作到隊列中：

StandAloneClient().execute(...)

現在，我已經作了評論我因子評分，可能出於某種原因堵塞螺紋部分。但即便如此，似乎沒有任何一個孩子會產生結果。執行完成，直到最後，我檢查了日誌和processRunner.py做它需要做的，直到最後的一切，但是如果我做了ps -el|grep python我仍然可以看到所有的進程產生了：

0 S 1000 755  1 5 80 0 - 548314 poll_s pts/0 00:00:13 python 
0 S 1000 1198 755 4 80 0 - 280172 futex_ pts/0 00:00:09 python 
0 S 1000 1201 755 4 80 0 - 280176 futex_ pts/0 00:00:09 python 
0 S 1000 1206 755 4 80 0 - 280230 futex_ pts/0 00:00:09 python 
0 S 1000 1215 755 4 80 0 - 280198 futex_ pts/0 00:00:09 python 
0 S 1000 1216 755 4 80 0 - 281669 futex_ pts/0 00:00:09 python 
0 S 1000 1221 755 4 80 0 - 280201 futex_ pts/0 00:00:09 python 
0 S 1000 1231 755 4 80 0 - 281668 futex_ pts/0 00:00:09 python 
0 S 1000 1240 755 4 80 0 - 280229 futex_ pts/0 00:00:09 python 
0 S 1000 1257 755 4 80 0 - 280201 futex_ pts/0 00:00:09 python

我在一臺裝有Python 2.7.2的fedora 16機器上試試這個。任何建議>

問候，波格丹

來源

2012-07-19 Bogdan

爲什麼您使用子進程而不是多進程在多進程中運行Python代碼？一個方面：你的代碼是這樣一個簡單的任務錯綜複雜 – jfs 2012-07-19 13:34:51

你的線程不是守護程序。主線程退出後，它們繼續生活。他們永遠被阻止在pending_operations.get()。

來源

2012-07-19 13:33:49 jfs

python subprocess.Popen塊在futex_狀態

回答

相關問題