修改多處理池的經理字典中的列表

Question

我有一個元素列表，我正在處理多處理apply_async任務中的元素，並使用經理字典中的一個關鍵字逐個處理元素，我想在其中映射整個列表。

我嘗試下面的代碼：

在最後乾脆在字典職位空單。 輸出：

{'task': {'processed_list': []}}

現在研究了一下後，我才知道該經理字典中的元素成爲不可變的，所以你必須以更新它重新初始化新的數據字典整體。所以我試着下面的代碼，它給出了一個奇怪的錯誤。

#!/usr/bin/python 

from multiprocessing import Pool, Manager 

def spammer_task(d, my_list): 
    #Initialize manager dict 
    d['task'] = { 
     'processed_list': [] 
    } 

    for ele in my_list: 
     #process here 
     old_list = d['task']['processed_list'] 
     new_list = old_list.append(ele) 
     #Have to do it this way since elements inside a manager dict become 
     #immutable so 
     d['task'] = { 
      'processed_list': new_list 
     } 

    return 

p = Pool() 
m = Manager() 
d = m.dict() 

my_list = ["one", "two", "three"] 

p.apply_async(spammer_task (d, my_list)) 
print d 

輸出：

Traceback (most recent call last): File "./a.py", line 29, in p.apply_async(spammer_task (d, my_list)) File "./a.py", line 14, in spammer_task new_list = old_list.append(ele) AttributeError: 'NoneType' object has no attribute 'append'

不知怎的，這似乎是追加None，而我無法弄清楚，爲什麼名單。

Answer 1

Accoridng到溶液中https://bugs.python.org/issue6766

下面的代碼修復它，通過複製整個任務字典，然後修改它並重新複製它

#!/usr/bin/python 

from multiprocessing import Pool, Manager 

def spammer_task(d, my_list): 
    #Initialize manager dict 
    d['task'] = { 
     'processed_list': [] 
    } 

    for ele in my_list: 
     #process here 
     foo = d['task'] 
     foo['processed_list'].append(ele) 
     d['task'] = foo 
    return 

p = Pool() 
m = Manager() 
d = m.dict() 

my_list = ["one", "two", "three"] 

p.apply_async(spammer_task (d, my_list)) 
print d 

輸出：從確保除了

{'task': {'processed_list': ['one', 'two', 'three']}}

Answer 2

該d實際上包含的東西，當打印時，結果仍然是{'task': {'processed_list': ['one', 'two', 'three']}}

#!/usr/bin/python 

from multiprocessing import Pool 

def spammer_task(my_list): 
    #Initialize manager dict 
    out= { 
     'processed_list': [] 
    } 

    for ele in my_list: 
     #process here 
     out['processed_list'].append(ele) 

    return 'task',out 



my_list = ["one", "two", "three"] 

if __name__=="__main__": 

    p = Pool() 
    d=dict(p.imap_unordered(spammer_task, [my_list])) #this line blocks until finished 
    print d