異步獲取和存儲圖像在Python中

Question

以下代碼是非異步代碼的示例，有沒有什麼方法異步獲取圖像？

​​

我也看到了Grequests庫，但我想不出太多，如果這是可能的，或者如何從文檔做到這一點。

Answer 1

你不需要任何第三方庫。只需爲每個請求創建一個線程，啓動線程，然後等待所有這些線程在後臺完成，或者在下載圖像時繼續執行應用程序。

import threading 

results = [] 
def getter(url, dest): 
    results.append(urllib.urlretreave(url, dest)) 

threads = [] 
for x in range(0,10): 
    t = threading.Thread(target=getter, args=('http://test.com/file %s.png' % x, 
               'temp/file %s.png' % x)) 
    t.start() 
    threads.append(t) 
# wait for all threads to finish 
# You can continue doing whatever you want and 
# join the threads when you finally need the results. 
# They will fatch your urls in the background without 
# blocking your main application. 
map(lambda t: t.join(), threads) 

您還可以選擇創建一個線程池，將讓urls和dests從隊列中。

如果您使用Python 3，它已經在futures模塊中爲您實施。

Answer 2

像這樣的東西應該幫助你

import grequests 
urls = ['url1', 'url2', ....] # this should be the list of urls 

    requests = (grequests.get(u) for u in urls) 
    responses = grequests.map(requests) 
    for response in responses: 
     if 199 < response.status_code < 400: 
      name = generate_file_name() # generate some name for your image file with extension like example.jpg 
      with open(name, 'wb') as f: # or save to S3 or something like that 
        f.write(response.content) 

這裏只圖像的下載將是平行的，但每個圖像內容寫入文件將是連續的，所以你可以創建一個線程或做其他事，使之並行或異步