1
我有一個腳本,可以在我的舊Windows XP計算機上正常工作,但我需要在Windows 7上運行。所以我已經安裝了所有需要的模塊& Cie。但是我仍然遇到一些奇怪的錯誤,通過互聯網,我想這裏有一些好的靈魂可以幫助我。請求與異步超時。我應該如何解決這個錯誤?
因此,這裏是我的腳本有問題的功能:
from gevent import monkey
import gevent
import requests
from requests import async
import xlwt
from bs4 import BeautifulSoup as soup
from urllib2 import urlopen
import time
from twisted.web import client
from twisted.internet import reactor, defer
import re
import os
import urllib
import random
import sys
import re
import logging
from threading import RLock
logging.basicConfig(level=logging.INFO)
log = logging.getLogger()
from grab.spider import Spider, Task
def BULATS_TC():
''' FETCHING BULATS AGENTS '''
x = 0
START_BULATS_TC = time.time()
ws = wb.add_sheet("BULATS_AGENTS")
Page_List = ['0','1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19', '20']
to_send = []
def parse(response):
global x
html = soup(response.text)
tableau = html.find('table')
try:
rows = tableau.findAll('tr')
for tr in rows:
cols = tr.findAll('td')
y = 0
x = x + 1
for td in cols:
texte_bu = td.text
texte_bu = texte_bu.encode('utf-8')
texte_bu = texte_bu.strip()
ws.write(x,y,texte_bu)
y = y + 1
except (IndexError, AttributeError):
pass
for Page in Page_List:
r = requests.async.get('http://www.bulats.org/agents/find-an-agent?field_continent_tid=All&field_country_tid=All&page=%s' % Page, timeout=20)
to_send.append(r)
responses = requests.async.map(to_send)
parse_jobs = [gevent.spawn(parse, resp) for resp in responses]
gevent.joinall(parse_jobs)
ELAPSED_TIME_BULATS_AGENTS = time.time() - START_BULATS_TC
return ELAPSED_TIME_BULATS_AGENTS
,這裏是我在終端得到的錯誤:
exceptions.IOError: [Errno 0] No error
Restarting gevent.core.dispatch() after an error [1928]: [Errno 0] No error
Unhandled Error
Traceback (most recent call last):
File "C:\Users\Admin\Desktop\EAGLE_DATA_UPDATE.py", line 509, in <module>
BULATS_IA_TIME = BULATS_IA()
File "C:\Users\Admin\Desktop\EAGLE_DATA_UPDATE.py", line 430, in BULATS_IA
reactor.run()
File "C:\Python27\lib\site-packages\twisted\internet\base.py", line 1169, in r
un
self.mainLoop()
--- <exception caught here> ---
File "C:\Python27\lib\site-packages\twisted\internet\base.py", line 1181, in m
ainLoop
self.doIteration(t)
File "C:\Python27\lib\site-packages\twisted\internet\selectreactor.py", line 1
04, in doSelect
[], timeout)
File "C:\Python27\lib\site-packages\twisted\internet\selectreactor.py", line 3
6, in win32select
r, w, e = select.select(r, w, w, timeout)
File "C:\Python27\lib\site-packages\gevent\select.py", line 63, in select
result.event.wait(timeout=timeout)
File "C:\Python27\lib\site-packages\gevent\event.py", line 74, in wait
result = get_hub().switch()
File "C:\Python27\lib\site-packages\gevent\hub.py", line 164, in switch
return greenlet.switch(self)
File "C:\Python27\lib\site-packages\gevent\hub.py", line 179, in run
result = core.dispatch()
File "core.pyx", line 398, in gevent.core.dispatch (gevent/core.c:5404)
exceptions.IOError: [Errno 0] No error
Restarting gevent.core.dispatch() after an error [1929]: [Errno 0] No error
Unhandled error in Deferred:
Unhandled Error
Traceback (most recent call last):
Failure: twisted.internet.defer.FirstError: FirstError[#2, [Failure instance: Tr
aceback (failure with no frames): <class 'twisted.internet.error.TimeoutError'>:
User timeout caused connection failure.
]]
我認爲我的接頭是不是非常穩定(我在中國),所以也許我需要設置一個更大的超時時間,但有了請求和我的腳本,我不知道如何繼續。
我希望這一切都足夠清楚。
您是否在瀏覽器中嘗試過其中一個網址?只是爲了驗證你沒有連接問題。例如:http://www.bulats.org/agents/find-an-agent?field_continent_tid=All&field_country_tid=All&page=3 – snies
也簡化了您的示例,並使其可執行,例如,您的代碼不顯示對扭曲反應器。去除連接中不需要的所有東西,例如所有的湯料。嘗試用最少的代碼行和最小的庫導入重現錯誤。 – snies
它應該是一個連接問題...我想你是對的。我檢查過,我可以訪問該網站...任何人都知道如何改變超時時間? –