concurrent.futures.ThreadPoolExecutor是比列表理解

Question

我使用列表理解VS concurrent.futures測試平凡函數慢：

class Test: 

    @staticmethod 
    def something(times = 1): 
     return sum([1 for i in range(times)]) 

    @staticmethod 
    def simulate1(function, N): 
     l = [] 

     for i in range(N): 
      outcome = function() 
      l.append(outcome) 

     return sum(l)/N 

    @staticmethod 
    def simulate2(function, N): 
     import concurrent.futures 

     with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor: 
      l = [outcome for outcome in executor.map(lambda x: function(), range(N))] 

     return sum(l)/N 

    @staticmethod 
    def simulate3(function, N): 
     import concurrent.futures 

     l = 0 

     with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor: 
      futures = [executor.submit(function) for i in range(N)] 
      for future in concurrent.futures.as_completed(futures): 
       l += future.result() 

     return l/N 

def simulation(): 
    simulationRate = 100000 

    import datetime 

    s = datetime.datetime.now() 
    o = Test.simulate1(lambda : Test.something(10), simulationRate) 
    print((datetime.datetime.now() - s)) 

    s = datetime.datetime.now() 
    o = Test.simulate2(lambda : Test.something(10), simulationRate) 
    print((datetime.datetime.now() - s)) 

    s = datetime.datetime.now() 
    o = Test.simulate3(lambda : Test.something(10), simulationRate) 
    print((datetime.datetime.now() - s)) 

simulation() 

測量的時候，我得到：

0:00:00.258000 
0:00:10.348000 
0:00:10.556000 

我開始使用併發性，所以我不明白阻止線程運行速度的瓶頸是什麼。

Answer 1

如果你改變你的任務功能，這一點，你會看到其中的差別：

def something(n): 
    """ simulate doing some io based task. 
    """ 
    time.sleep(0.001) 
    return sum(1 for i in range(n)) 

在我的Mac Pro，這給：

0:00:13.774700 
0:00:01.591226 
0:00:01.489159 

的concurrent.future明顯快多了這時間。

原因是：你正在模擬一個基於cpu的任務，因爲python的GIL，concurrent.future使它變慢。

concurrent.future提供了一個用於異步執行可調用對象的高級接口，您正在將它用於錯誤場景。