下特定的標籤Beautifulsoup解析數據

Question

現在我解析網頁使用此代碼：

boards = soup(itemprop="name") 
prices = soup("span", { "class" : "price-currency" }) 

for board, price in zip(boards, prices): 
    print(board.text.strip(), price.next_sibling) 

並印刷電路板和價格如下：

SURFBOARD RACK free delivery to your door 120.00 
Huge Beginner Surfboard Sale! Kids & Adult Softboards all 1/2 Price!! 90.00 
Mega Softboard Clearance Sale! Beginner Foam SurfBoards 1/2 Price! 90.00 
Surfboard 6'2" Simon Anderson Spudnick 360.00 
Surfboard Cover, Surfboard Bags, Cheap Single Surf Board Bags 50.00 

的網頁我解析的內容分爲3個部分：贊助商鏈接，頂級廣告和最近的廣告。 我從這些部分的所有3只從最近的廣告部分，它具有這個網站打印數據，而是希望數據：

<div class="module__body ad-listing"> 

如何指定，我只想從這個下面印刷板和價格部分？

頁碼：https://www.gumtree.com.au/s-surfing/mona-vale-sydney/surfboard/k0c18568l3003999r10?fromSearchBox=true

Answer 1

你可能會討厭這個答案。當我看到複雜的HTML時，我傾向於使用lxml模塊，因爲我可以使用xpath表達式。

在這種情況下，第一個xpath會在您想要的HTML中找到li元素的集合。該循環使用兩個xpath表達式，一個可在li元素內找到諸如「Quicksale 6'4 Dylan Surfboard RX5」之類的內容，另一個可在相同元素內查找價格信息的文本集合。第12項似乎有不同的編碼;我沒有調查過。

>>> import requests 
>>> from lxml import etree 
>>> page = requests.get('https://www.gumtree.com.au/s-surfing/mona-vale-sydney/surfboard/k0c18568l3003999r10?fromSearchBox=true').text 
>>> parser = etree.HTMLParser() 
>>> tree = etree.fromstring(page, parser=parser) 
>>> recents = tree.xpath('.//div[@class="module__body ad-listing"]/ul/li') 
>>> for i, recent in enumerate(recents): 
...  try: 
...   i, recent.xpath('.//span[@itemprop="name"]/text()')[0].strip() 
...  except: 
...   '-------------> item', i, 'failed' 
...   continue 
...  one_span = first_recent.xpath('.//span[@class="j-original-price"]')[0] 
...  ' '.join([_.strip() for _ in list(one_span.itertext()) if _.strip()]) 
... 
(0, "Quicksale 6'4 Dylan Surfboard RX5") 
'$ 450.00 Negotiable' 
(1, 'DHD 5\'9 "Switchblade" Surfboard') 
'$ 450.00 Negotiable' 
(2, '6ft Modern Surfboards - Highline') 
'$ 450.00 Negotiable' 
(3, "5'11 Channel Island T-Low surfboard") 
'$ 450.00 Negotiable' 
(4, 'Chill Rare Bird Surfboard 5"8') 
'$ 450.00 Negotiable' 
(5, 'Vintage surfboard') 
'$ 450.00 Negotiable' 
(6, "5'7 Annesley Blonde model") 
'$ 450.00 Negotiable' 
(7, 'McCoy single fin surfboard') 
'$ 450.00 Negotiable' 
(8, 'Sculpt surfboard') 
'$ 450.00 Negotiable' 
(9, '8\'1" longboard surfboard travel cover') 
'$ 450.00 Negotiable' 
(10, 'Longboard Surfboard') 
'$ 450.00 Negotiable' 
(11, "5'10 Custom Chaos Surfboard") 
'$ 450.00 Negotiable' 
('-------------> item', 12, 'failed') 
(13, "6'0 JS lowdown") 
'$ 450.00 Negotiable' 
(14, 'Mega Softboard Clearance Sale! Beginner Foam SurfBoards 1/2 Price!') 
'$ 450.00 Negotiable' 
(15, 'Surfboard') 
'$ 450.00 Negotiable' 
(16, 'Surfboard 5\'10" 30 lt') 
'$ 450.00 Negotiable' 
(17, 'Christenson Super Sport Surfboard') 
'$ 450.00 Negotiable' 
(18, 'TOMO Firewire V4 Surfboard') 
'$ 450.00 Negotiable' 
(19, "Surfboard 6'6 baked bean") 
'$ 450.00 Negotiable' 
(20, 'foam surfboards') 
'$ 450.00 Negotiable' 
(21, 'Channel Islands surfboard') 
'$ 450.00 Negotiable' 
(22, 'Channel Islands Surfboard') 
'$ 450.00 Negotiable' 
(23, 'JS surfboard') 
'$ 450.00 Negotiable' 
(24, 'CLASSIC RETRO SURF FACTORY MINI MAL') 
'$ 450.00 Negotiable' 
(25, 'Surfboard JS') 
'$ 450.00 Negotiable'