2012-05-02 30 views
1

我有一個腳本,可以在我的舊Windows XP計算機上正常工作,但我需要在Windows 7上運行。所以我已經安裝了所有需要的模塊& Cie。但是我仍然遇到一些奇怪的錯誤,通過互聯網,我想這裏有一些好的靈魂可以幫助我。請求與異步超時。我應該如何解決這個錯誤?

因此,這裏是我的腳本有問題的功能:

from gevent import monkey 
import gevent 
import requests 
from requests import async 
import xlwt 
from bs4 import BeautifulSoup as soup 
from urllib2 import urlopen 
import time 
from twisted.web import client 
from twisted.internet import reactor, defer 
import re 
import os 
import urllib 
import random 
import sys 
import re 
import logging 
from threading import RLock 

logging.basicConfig(level=logging.INFO) 
log = logging.getLogger() 

from grab.spider import Spider, Task 



def BULATS_TC(): 
    ''' FETCHING BULATS AGENTS ''' 
    x = 0 
    START_BULATS_TC = time.time() 
    ws = wb.add_sheet("BULATS_AGENTS") 
    Page_List = ['0','1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19', '20'] 
    to_send = [] 

    def parse(response): 
     global x 
     html = soup(response.text) 
     tableau = html.find('table') 
     try: 
      rows = tableau.findAll('tr') 
      for tr in rows: 
       cols = tr.findAll('td') 
       y = 0 
       x = x + 1 
       for td in cols: 
        texte_bu = td.text 
        texte_bu = texte_bu.encode('utf-8') 
        texte_bu = texte_bu.strip() 
        ws.write(x,y,texte_bu) 
        y = y + 1 
     except (IndexError, AttributeError): 
      pass 

    for Page in Page_List: 
     r = requests.async.get('http://www.bulats.org/agents/find-an-agent?field_continent_tid=All&field_country_tid=All&page=%s' % Page, timeout=20) 
     to_send.append(r) 

    responses = requests.async.map(to_send) 
    parse_jobs = [gevent.spawn(parse, resp) for resp in responses] 
    gevent.joinall(parse_jobs) 

    ELAPSED_TIME_BULATS_AGENTS = time.time() - START_BULATS_TC 
    return ELAPSED_TIME_BULATS_AGENTS 

,這裏是我在終端得到的錯誤:

exceptions.IOError: [Errno 0] No error 
Restarting gevent.core.dispatch() after an error [1928]: [Errno 0] No error 
Unhandled Error 
Traceback (most recent call last): 
    File "C:\Users\Admin\Desktop\EAGLE_DATA_UPDATE.py", line 509, in <module> 
    BULATS_IA_TIME = BULATS_IA() 
    File "C:\Users\Admin\Desktop\EAGLE_DATA_UPDATE.py", line 430, in BULATS_IA 
    reactor.run() 
    File "C:\Python27\lib\site-packages\twisted\internet\base.py", line 1169, in r 
un 
    self.mainLoop() 
--- <exception caught here> --- 
    File "C:\Python27\lib\site-packages\twisted\internet\base.py", line 1181, in m 
ainLoop 
    self.doIteration(t) 
    File "C:\Python27\lib\site-packages\twisted\internet\selectreactor.py", line 1 
04, in doSelect 
    [], timeout) 
    File "C:\Python27\lib\site-packages\twisted\internet\selectreactor.py", line 3 
6, in win32select 
    r, w, e = select.select(r, w, w, timeout) 
    File "C:\Python27\lib\site-packages\gevent\select.py", line 63, in select 
    result.event.wait(timeout=timeout) 
    File "C:\Python27\lib\site-packages\gevent\event.py", line 74, in wait 
    result = get_hub().switch() 
    File "C:\Python27\lib\site-packages\gevent\hub.py", line 164, in switch 
    return greenlet.switch(self) 
    File "C:\Python27\lib\site-packages\gevent\hub.py", line 179, in run 
    result = core.dispatch() 
    File "core.pyx", line 398, in gevent.core.dispatch (gevent/core.c:5404) 

exceptions.IOError: [Errno 0] No error 
Restarting gevent.core.dispatch() after an error [1929]: [Errno 0] No error 
Unhandled error in Deferred: 
Unhandled Error 
Traceback (most recent call last): 
Failure: twisted.internet.defer.FirstError: FirstError[#2, [Failure instance: Tr 
aceback (failure with no frames): <class 'twisted.internet.error.TimeoutError'>: 
User timeout caused connection failure. 
]] 

我認爲我的接頭是不是非常穩定(我在中國),所以也許我需要設置一個更大的超時時間,但有了請求和我的腳本,我不知道如何繼續。

我希望這一切都足夠清楚。

+0

您是否在瀏覽器中嘗試過其中一個網址?只是爲了驗證你沒有連接問題。例如:http://www.bulats.org/agents/find-an-agent?field_continent_tid=All&field_country_tid=All&page=3 – snies

+2

也簡化了您的示例,並使其可執行,例如,您的代碼不顯示對扭曲反應器。去除連接中不需要的所有東西,例如所有的湯料。嘗試用最少的代碼行和最小的庫導入重現錯誤。 – snies

+0

它應該是一個連接問題...我想你是對的。我檢查過,我可以訪問該網站...任何人都知道如何改變超時時間? –

回答

0

我終於通過重新安裝Twisted工作。