2012-07-07 46 views
3
from random import randrange 
from time import sleep 
#import thread 
from threading import Thread 
from Queue import Queue 
'''The idea is that there is a Seeker method that would search a location 
for task, I have no idea how many task there will be, could be 1 could be 100. 
Each task needs to be put into a thread, does its thing and finishes. I have 
stripped down a lot of what this is really suppose to do just to focus on the 
correct queuing and threading aspect of the program. The locking was just 
me experimenting with locking''' 
class Runner(Thread): 
    current_queue_size = 0 
    def __init__(self, queue): 
     self.queue = queue 
     data = queue.get() 
     self.ID = data[0] 
     self.timer = data[1] 
     #self.lock = data[2] 
     Runner.current_queue_size += 1 
     Thread.__init__(self) 

    def run(self): 
     #self.lock.acquire() 
     print "running {ID}, will run for: {t} seconds.".format(ID = self.ID, 
                   t = self.timer) 
     print "Queue size: {s}".format(s = Runner.current_queue_size) 
     sleep(self.timer)   
     Runner.current_queue_size -= 1 
     print "{ID} done, terminating, ran for {t}".format(ID = self.ID, 
                   t = self.timer) 
     print "Queue size: {s}".format(s = Runner.current_queue_size) 
     #self.lock.release() 
     sleep(1) 
     self.queue.task_done() 

def seeker(): 
    '''Gathers data that would need to enter its own thread. 
    For now it just uses a count and random numbers to assign 
    both a task ID and a time for each task''' 
    queue = Queue() 
    queue_item = {} 
    count = 1 
    #lock = thread.allocate_lock() 
    while (count <= 40): 
     random_number = randrange(1,350) 
     queue_item[count] = random_number 
     print "{count} dict ID {key}: value {val}".format(count = count, key = random_number, 
                  val = random_number) 
     count += 1 

    for n in queue_item: 
     #queue.put((n,queue_item[n],lock)) 
     queue.put((n,queue_item[n])) 
     '''I assume it is OK to put a tulip in and pull it out later''' 
     worker = Runner(queue) 
     worker.setDaemon(True) 
     worker.start() 
    worker.join() 
    '''Which one of these is necessary and why? The queue object 
    joining or the thread object''' 

    #queue.join()  

if __name__ == '__main__': 
    seeker() 

我已經把我的大部分代碼本身的問題,而是要在主點(Python2.7):Python的多線程時間敏感的任務

  • 我想確保我稍後不會爲自己造成大量內存泄漏。
  • 我注意到,當我在putty或VNC上運行計數爲40時,在我的linuxbox上我並不總是得到所有的輸出,但是當我在窗口上使用IDLE和Aptana時,我會這樣做。
  • 是的,我明白,隊列的一點是要錯開了你 線程,這樣就不會充斥你的系統內存,但因此他們需要爲他們 而不管檢測,以儘快處理在 手頭的任務對時間敏感多少或多少;我有 發現,當我有隊列時,我可以清楚地指示何時有一個任務完成 反對讓垃圾收集器猜測。
  • 我仍然不知道爲什麼我能夠脫身使用線程或隊列對象上的 .join()。
  • 提示,技巧,一般幫助。
  • 感謝您的閱讀。

回答

0

如果我理解你正確,你需要一個線程來監視某些東西,看看是否有任務需要完成。如果找到任務,則希望與任務和其他當前正在運行的任務並行運行。

如果是這種情況,那麼我認爲你可能會出現這個錯誤。看看GIL如何在Python中工作。我認爲你可能真正想要的是多處理。

在此看一看從pydocs:

CPython的實現細節:在CPython的,由於全局解釋器鎖,只有一個線程可以同時執行Python代碼(即使某些業績爲導向的庫可能克服這個限制)。如果您希望您的應用程序更好地利用多核計算機的計算資源,建議您使用多處理。但是,如果要同時運行多個I/O限制任務,線程仍然是一個合適的模型。