無法使用urllib2從網站中提取數據

Question

我當時正在關注pythonforbeginners.com上的一個教程，並且我遇到了一個在我的OSX上沒有正確運行的代碼。

from bs4 import BeautifulSoup 
import urllib2 
url = "http://www.pythonforbeginners.com" 
content = urllib2.urlopen(url).read() 
soup = BeautifulSoup(content) 
print soup.prettify() 

這給我的錯誤：

Traceback (most recent call last): File "/Users/dhruvmullick/CS/Python/Extracting Data/test.py", line 8, in content = urllib2.urlopen(url).read() File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 127, in urlopen return _opener.open(url, data, timeout) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 410, in open response = meth(req, response) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 523, in http_response 'http', request, response, code, msg, hdrs) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 448, in error return self._call_chain(*args) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 382, in _call_chain result = func(*args) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 531, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 403: Forbidden

Answer 1

的403 error表示服務器阻止您的連接。

...a request from a client for a web page or resource to indicate that the server can be reached and understood the request, but refuses to take any further action.

嘗試一個不同的域，你會發現它按預期工作。

要做出變通，您可以添加一個custom user-agent。