Proxybroker IndexError

我正在嘗試使用python軟件包proxybroker。
我試圖使用這裏提到的例子之一。我只是複製下面的例子在本地運行：當我嘗試運行下面的錯誤與一些廢棄警告一起拋出的代碼Proxybroker IndexError

import asyncio from proxybroker import Broker 


async def save(proxies, filename): 
    """Save proxies to a file.""" 
    with open(filename, 'w') as f: 
     while True: 
      proxy = await proxies.get() 
      if proxy is None: 
       break 
      proto = 'https' if 'HTTPS' in proxy.types else 'http' 
      row = '%s://%s:%d\n' % (proto, proxy.host, proxy.port) 
      f.write(row) 


def main(): 
    proxies = asyncio.Queue() 
    broker = Broker(proxies) 
    tasks = asyncio.gather(broker.find(types=['HTTP', 'HTTPS'], limit=10), 
          save(proxies, filename='proxies.txt')) 
    loop = asyncio.get_event_loop() 
    loop.run_until_complete(tasks) 


if __name__ == '__main__': 
    main()

：

/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/aiohttp/client.py:494: 
DeprecationWarning: Use async with instead warnings.warn("Use async 
with instead", DeprecationWarning) 
/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/aiohttp/client.py:494: 
DeprecationWarning: Use async with instead warnings.warn("Use async 
with instead", DeprecationWarning) 
/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/aiohttp/client.py:494: 
DeprecationWarning: Use async with instead warnings.warn("Use async 
with instead", DeprecationWarning) 
/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/aiohttp/client.py:494: 
DeprecationWarning: Use async with instead warnings.warn("Use async 
with instead", DeprecationWarning) 
/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/aiohttp/client.py:494: 
DeprecationWarning: Use async with instead warnings.warn("Use async 
with instead", DeprecationWarning) 
/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/aiohttp/client.py:494: 
DeprecationWarning: Use async with instead warnings.warn("Use async 
with instead", DeprecationWarning) 
/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/aiohttp/client.py:494: 
DeprecationWarning: Use async with instead warnings.warn("Use async 
with instead", DeprecationWarning) 
/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/aiohttp/client.py:494: 
DeprecationWarning: Use async with instead warnings.warn("Use async 
with instead", DeprecationWarning) 
/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/aiohttp/client.py:494: 
DeprecationWarning: Use async with instead warnings.warn("Use async 
with instead", DeprecationWarning) 
/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/aiohttp/client.py:494: 
DeprecationWarning: Use async with instead warnings.warn("Use async 
with instead", DeprecationWarning) https://getproxy.net/en/ is failed. 
Error: ClientOSError(101, 'Cannot connect to host getproxy.net:443 
ssl:True [Can not connect to getproxy.net:443 [Network is 
unreachable]]'); https://getproxy.net/en/ is failed. Error: 
ClientOSError(101, 'Cannot connect to host getproxy.net:443 ssl:True 
[Can not connect to getproxy.net:443 [Network is unreachable]]'); 
https://getproxy.net/en/ is failed. Error: ClientOSError(101, 'Cannot 
connect to host getproxy.net:443 ssl:True [Can not connect to 
getproxy.net:443 [Network is unreachable]]'); Traceback (most recent 
call last): File 
"/home/sebastian/PycharmProjects/testing/test/test_prox.py", line 27, 
in 
    main() File "/home/sebastian/PycharmProjects/testing/test/test_prox.py", line 23, 
in main 
    loop.run_until_complete(tasks) File "/usr/lib/python3.5/asyncio/base_events.py", line 387, in 
run_until_complete 
    return future.result() File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result 
    raise self._exception File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step 
    result = coro.throw(exc) File "/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/proxybroker/api.py", 
line 108, in find 
    await self._run(self._checker.check_judges(), action) File "/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/proxybroker/api.py", 
line 114, in _run 
    await tasks File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__ 
    yield self # This tells Task to wait for completion. File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup 
    future.result() File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result 
    raise self._exception File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step 
    result = coro.throw(exc) File "/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/proxybroker/checker.py", 
line 26, in check_judges 
    await asyncio.gather(*[j.check() for j in self._judges]) File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__ 
    yield self # This tells Task to wait for completion. File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup 
    future.result() File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result 
    raise self._exception File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step 
    result = coro.send(None) File "/home/sebastian/PycharmProjects/STW/venv/lib/python3.5/site-packages/proxybroker/judge.py", 
line 82, in check 
    j=self, code=resp.status, page=page[0], IndexError: string index out of range

我使用python3.5.2和最新版本的proxybroker（0.1.4）aiohttp（1.0.2）asyncio（3.4.3）。

我不確定是什麼導致了錯誤，因爲我沒有更改代碼示例，據我所知我已經安裝了所有依賴關係。任何人都可以幫助我，告訴我我做錯了什麼，甚至更好的如何做對嗎？

編輯
一種解決方法的問題是，當錯誤發生時改線。該行僅用於記錄錯誤，因此更改不應有任何傷害。對於這種解決方法 - 並非解決方案 - 我在79行的judge.py中添加了一個額外的檢查，其中異常是在之前提出的。本地我把它改爲：

 if isinstance(page, type(list())) or isinstance(page, type(dict())): 
      log.error(('{j} is failed. HTTP status code: {code}; ' 
         'Real IP on page: {ip}; Version: {word}; ' 
         'Response: {page}').format(
       j=self, code=resp.status, page=page[0], 
       ip=(get_my_ip() in page), word=(rv in page))) 
     else: 
      log.error(('{j} is failed. HTTP status code: {code}; ' 
         'Real IP on page: {ip}; Version: {word}; ' 
         'Response: {page}').format(
         j=self, code=resp.status, page=page, 
         ip=(get_my_ip() in page), word=(rv in page)))

這樣我可以再次使用proxybroker。這個問題提交到gihub上proxybroker。

來源

2016-09-30 SSchneid

棄用警告是無害的（至少除非我會刪除這種向後兼容性）。

錯誤只是說getproxy.net不可用 - 這是您的主要問題。

來源

2016-10-01 09:15:02

所以有什麼我可以做的，因爲如果我嘗試通過瀏覽器訪問'getproxy.net'網站是可用的。 – SSchneid

我不知道。嘗試在https://github.com/constverum/ProxyBroker/issues –

中提出問題感謝您的幫助。爲了能夠工作我目前正在實現一個使用另一個名爲[freeproxy]的python包的回退（https://github.com/yieldnull/freeproxy）。這似乎是一個非常好的選擇，它找到了相當一些代理（超過5000，儘管尚未測試狀態），最重要的是非常容易擴展。所以對於其他人來說，這是需要改變的地方。 – SSchneid

目前有一個與問題：

<Judge [HTTP] www.ingosander.net> is failed. HTTP status code: 302; Real IP on page: False; Version: False; Response:

，因爲它重定向到的響應內容是空的（狀態碼：302）。可能的解決方案： 1.更改http客戶端。使用請求pakage - 自動跟蹤重定向。用於重定向的網址2.更改錯誤記錄在judge.py加入

elif (resp.status == 302): 
    log.error(('{j} is failed. HTTP status code: {code}; ' 
       'Real IP on page: {ip}; Version: {word}; ' 
       'Response: {page}').format(
       j=self, code=resp.status, page=None, 
       ip=(get_my_ip() in page), word=(rv in page)))

在judge.py 底部

評論這個法官在judgeList

來源

2017-02-15 22:23:19

Proxybroker IndexError

回答

相關問題