Python 2.7。 BeautifulSoup不能打開HTML

我有一個BeautifulSoup奇怪的問題。我試圖閱讀這個網站：http://lol.esportswikis.com/wiki/2015_International_Wild_Card_Invitational/Match_History和我所得到的是空列表。但是，如果我嘗試不同的網站，如：http://lol.esportswikis.com/wiki/Season_1_World_Championship/Match_History它的作品就像一個魅力。任何想法是什麼問題？Python 2.7。 BeautifulSoup不能打開HTML

從鉻檢查器工具，我得到了兩個網站表的第一行相同的代碼，所以最新的問題是什麼？

#mw-content-text > table.wikitable > tbody > tr:nth-child(3) 
    mw-content-text > table.wikitable > tr #I use this

即使我嘗試打開剛剛wikitable

url = 'insert url here' 
    con = urllib2.urlopen(url) 
    HTML = con.read() 
    soup = BeautifulSoup(HTML, 'html.parser') 
    stuff = soup.select('#mw-content-text > table.wikitable') 
    print stuff

它只是打印出空單

來源

2016-11-14 tmi12

這可能是因爲您正在尋找的是由JavaScript生成的。 'urlopen'不執行JavaScript。你能確認HTML實際上是在「湯」嗎？ – sytech

我不太清楚你在問什麼，但命令「打印湯」正確打印頁面源代碼，我不認爲頁面中有任何JavaScript，至少不是在「wikitable」我試圖打開。 – tmi12

我只是試着自己測試一下。當使用'html.parser'時，我得到你描述的結果。但是，當使用'lxml'解析器時，它按預期工作。 – sytech

import requests 
from bs4 import BeautifulSoup 
r = requests.get('http://lol.esportswikis.com/wiki/2015_International_Wild_Card_Invitational/Match_History') 
soup = BeautifulSoup(r.text , 'lxml') 

row_text=[] 
for row in soup.select('.wikitable tr')[2:]: 
    print(row.get_text(' ',strip=True).replace(' • ','•'))

我沒有2.7 ENV，所以我用的請求+ BS4 ，但它是相同的我將每個單元格按行中的空格分開以防您想分割它

出：

2015-04-25 5.6 Thaldrin•Theokoles•Energy•Nardeus•Dumbledoge Yang•Revolta•Tockers•micaO•Jockster 36:54 k k - - - - - - SB - - 
2015-04-25 5.6 Yang•Revolta•Tockers•micaO•Jockster Thaldrin•Theokoles•Energy•Nardeus•Dumbledoge 53:47 k k - - - - - - SB - - 
2015-04-25 5.6 Thaldrin•Theokoles•Energy•Nardeus•Dumbledoge Yang•Revolta•Tockers•micaO•Jockster 53:40 k k - - - - - - SB - - 
2015-04-25 5.6 Yang•Revolta•Tockers•micaO•Jockster Thaldrin•Theokoles•Energy•Nardeus•Dumbledoge 32:30 k k - - - - - - SB - - 
2015-04-24 5.6 WarL0cK•007x•G4•Moss•Lloyd Thaldrin•Theokoles•Energy•Nardeus•Dumbledoge 45:53 k k - - - - - - SB - - 
2015-04-24 5.6 Thaldrin•Theokoles•Energy•Nardeus•Dumbledoge WarL0cK•007x•G4•Lloyd•Moss 34:16 k k - - - - - - SB - - 
2015-04-24 5.6 WarL0cK•007x•G4•Lloyd•Moss Thaldrin•Theokoles•Energy•Nardeus•Dumbledoge 39:39 k k - - - - - - SB - - 
2015-04-24 5.6 Yang•Revolta•Tockers•micaO•Jockster Smurf•Symphony•Kira•LeX•Dimonko 36:58 k k - - - - - - SB - - 
2015-04-24 5.6 Smurf•Symphony•Kira•LeX•Dimonko Yang•Revolta•Tockers•micaO•Jockster 32:03 k k - - - - - - SB - - 
2015-04-24 5.6 Yang•Revolta•Tockers•micaO•Jockster Smurf•Symphony•Kira•LeX•Dimonko 24:06 k k - - - - - - SB - - 
2015-04-24 5.6 Smurf•Symphony•Kira•LeX•Dimonko Yang•Revolta•Tockers•micaO•Jockster 38:01 k k - - - - - - SB - - 
2015-04-23 5.6 BonziN•Astarore•Ceros•Yutapongo•KazuXD Thaldrin•Theokoles•Energy•Nardeus•Dumbledoge 32:01 k k - - - - - - SB - - 
2015-04-23 5.6 WarL0cK•007x•G4•Lloyd•Moss Swip3rR•Spookz•Swiffer•Raydere•Rosey 35:05 k k - - - - - - SB - - 
2015-04-23 5.6 Yang•Revolta•Tockers•micaO•Jockster Helior•Juliostito•Regi•Zeicro•BearJew 18:40 k k - - - - - - SB - - 
2015-04-23 5.6 Swip3rR•Spookz•Swiffer•Raydere•Rosey BonziN•Astarore•Ceros•Yutapongo•KazuXD 36:26 k k - - - - - - SB - - 
2015-04-23 5.6 Helior•Juliostito•Regi•Zeicro•BearJew WarL0cK•007x•G4•Lloyd•Moss 28:26 k k - - - - - - SB - - 
2015-04-22 5.6 Smurf•Symphony•Kira•LeX•Dimonko Yang•Revolta•Tockers•micaO•Jockster 37:03 k k - - - - - - SB - - 
2015-04-22 5.6 BonziN•Astarore•Ceros•Yutapongo•KazuXD Helior•Juliostito•Regi•Zeicro•BearJew 34:25 k k - - - - - - SB - - 
2015-04-22 5.6 Thaldrin•Theokoles•Energy•Nardeus•Dumbledoge Smurf•Symphony•Kira•LeX•Dimonko 40:49 k k - - - - - - SB - - 
2015-04-22 5.6 Yang•Revolta•Tockers•micaO•Jockster WarL0cK•007x•G4•Lloyd•Moss 29:30 k k - - - - - - SB - - 
2015-04-22 5.6 Smurf•Symphony•Kira•LeX•Dimonko Swip3rR•Spookz•Swiffer•Raydere•Rosey 29:42 k k - - - - - - SB - - 
2015-04-22 5.6 Thaldrin•Theokoles•Energy•Nardeus•Dumbledoge Yang•Revolta•Tockers•micaO•Jockster 37:33 k k - - - - - - SB - - 
2015-04-22 5.6 BonziN•Astarore•Ceros•Yutapongo•KazuXD WarL0cK•007x•G4•Lloyd•Moss 27:38 k k - - - - - - SB - - 
2015-04-22 5.6 Helior•Juliostito•Regi•Zeicro•BearJew Smurf•Symphony•Kira•LeX•Dimonko 38:38 k k - - - - - - SB - - 
2015-04-21 5.6 Yang•Revolta•Tockers•micaO•Jockster BonziN•Astarore•Ceros•Yutapongo•KazuXD 35:52 k k - - - - - - SB - - 
2015-04-21 5.6 Swip3rR•Spookz•Swiffer•Raydere•Rosey Thaldrin•Theokoles•Energy•Nardeus•Dumbledoge 44:25 k k - - - - - - SB - - 
2015-04-21 5.6 WarL0cK•007x•G4•Lloyd•Moss Smurf•Symphony•Kira•LeX•Dimonko 45:10 k k - - - - - - SB - - 
2015-04-21 5.6 Thaldrin•Theokoles•Energy•Nardeus•Dumbledoge Helior•Juliostito•Regi•Zeicro•BearJew 35:51 k k - - - - - - SB - - 
2015-04-21 5.6 Swip3rR•Spookz•Swiffer•Raydere•Rosey Yang•Revolta•Tockers•micaO•Jockster 43:21 k k - - - - - - SB - - 
2015-04-21 5.6 WarL0cK•007x•G4•Lloyd•Moss Thaldrin•Theokoles•Energy•Nardeus•Dumbledoge 29:40 k k - - - - - - SB - - 
2015-04-21 5.6 Smurf•Symphony•Kira•LeX•Dimonko BonziN•Astarore•Ceros•Yutapongo•KazuXD 30:17 k k - - - - - - SB - - 
2015-04-21 5.6 Helior•Juliostito•Regi•Zeicro•BearJew Swip3rR•Spookz•Swiffer•Raydere•Rosey 43:52 63.3k 16 3 2 81.4k 22 11 3 -18.1k ▼ 18.1k -6 ▼ 6 -8 ▼ 8 -1 ▼ 1 - - SB - -

來源

2016-11-16 10:30:49

Python 2.7。 BeautifulSoup不能打開HTML

回答

相關問題