我剛開始python爬行,並試圖抓取web文本一個月。 我用python 2.7.13試過這段代碼,它運行得很好。AttributeError:'NoneType'對象沒有'python 2.7'使用BeautifulSoup屬性'find'
class IEEECrawler:
def __init__(self):
self.baseUrl = "http://ieeexplore.ieee.org"
self.targetUrl = "http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?reload=true&filter%3DAND%28p_IS_Number%3A4359286%29&rowsPerPage=100&pageNumber=1&resultAction=REFINE&resultAction=ROWS_PER_PAGE&isnumber=4359286#1.html"
self.soup = BeautifulSoup(urllib.urlopen(self.targetUrl).read(), "lxml")
self.doc_list = self.soup.find_all('div', {'class': "txt"})
self.subUrl = []
def crawlOriginalPage(self):
file = open("./result.txt", "w")
for doc in self.doc_list:
head = doc.find("h3")
author_list = ''
for author in doc.find_all("div", {'class':"authors"}):
for tt in author.find_all('span', {'id':"preferredName"}):
author_list += tt['data-author-name'] + ";"
author_list = author_list[:-1]
file.write(head.find("span").text + ';')
file.write(author_list.strip() + ';')
file.write(self.baseUrl+head.find('a')['href']+ ';')
file.write(doc.find("div", {'class': "hide abstract RevealContent"}).find("p").text.replace('View full abstract'+'»'.decode('utf-8'),'').strip()+ '\n')
file.close()
print 'finish'
但是,今天我再次運行此代碼,我沒有與這個錯誤masseges工作。我無法弄清楚應該修復哪些代碼。
Traceback (most recent call last):
File "/Users/user/Downloads/ieee_fin/ieee.py", line 35, in <module>
crawler.crawlOriginalPage()
File "/Users/user/Downloads/ieee_fin/ieee.py", line 29, in crawlOriginalPage
file.write(doc.find("div", {'class': "hide abstract RevealContent"}).find("p").text.replace('View full abstract'+'»'.decode('utf-8'),'').strip()+ '\n')
AttributeError: 'NoneType' object has no attribute 'find'
'doc'爲空。調試以找出原因。 – Carcigenicate