0
Scrapy是一個非常強大的工具,但有時它在XPath方面很令人沮喪。
從下面的html中,我想提取<b>January 2017</b>
和<b>February 2017</b>
之間的鏈接和鏈接文本(Title 1
,Title 2
等),並按「部分」對它們進行分組。 實際的html。Scrapy和XPath - 選擇部分之間的鏈接和鏈接文本
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Scrapy</title>
</head>
<body>
<hr size=1>
<h2 style="margin-top: 36px; margin-bottom: 24px">
Abcd efgh for 2017
</h2>
Part 1 |
Part 2 |
Part 3 |
Part 4 |
<a href="#">A very bold title</a>
<hr size="1" style="margin-top: 36px; margin-bottom: 24px">
<a name="part1"></a>
<h3>Part 1</h3>
<ul>
</ul>
<a name="part2"></a>
<h3>Part 2</h3>
<ul>
</ul>
<a name="part3"></a>
<h3>Part 3</h3>
<ul>
</ul>
<a name="part4"></a>
<h3>Part 4</h3>
<ul>
</ul>
<div style="margin-top: 36px; margin-bottom: 24px">
<a name="non_rep"></a>
<h3>Abcd efgh</h3>
</div>
<b>January 2017</b>
<ul>
<li>
<b>Part1 1</b>
</li>
<ul>
<li>
<a href="/cgi-bin/o.pl?file=/a/1.htm">Title 1</a>
</li>
<br>
<li>
<a href="/cgi-bin/o.pl?file=/a/11.htm">Title 2</a>
</li>
<br>
</ul>
<li>
<b>Part1 2</b>
</li>
<ul>
<li>
<a href="/cgi-bin/o.pl?file=/a/2.htm">Title A</a>
</li>
<br>
<li>
<a href="/cgi-bin/o.pl?file=/a/22.htm">Title B</a>
</li>
<br>
</ul>
<li>
<b>Part1 3</b>
</li>
<ul>
<li>
<a href="/cgi-bin/o.pl?file=/a/3.htm">Some text 1</a>
</li>
<br>
<li>
<a href="/cgi-bin/o.pl?file=/a/33.htm">Some Text 2</a>
</li>
</ul>
</ul>
<b>February 2017</b>
<ul>
<li>
<b>Part1 1</b>
</li>
<ul>
<li>
<a href="/cgi-bin/o.pl?file=/b/1.htm">Title 1</a>
</li>
<br>
<li>
<a href="/cgi-bin/o.pl?file=/b/11.htm">Title 2</a>
</li>
<br>
</ul>
<li>
<b>Part1 2</b>
</li>
<ul>
<li>
<a href="/cgi-bin/o.pl?file=/b/2.htm">Title A</a>
</li>
<br>
<li>
<a href="/cgi-bin/o.pl?file=/b/22.htm">Title B</a>
</li>
<br>
</ul>
<li>
<b>Part1 3</b>
</li>
<ul>
<li>
<a href="/cgi-bin/o.pl?file=/b/3.htm">Some text 1</a>
</li>
<br>
<li>
<a href="/cgi-bin/o.pl?file=/b/33.htm">Some Text 2</a>
</li>
</ul>
</ul>
<b>March 2017</b>
<ul>
<li>
<b>Part1 1</b>
</li>
<ul>
<li>
<a href="/cgi-bin/o.pl?file=/c/1.htm">Title 1</a>
</li>
<br>
<li>
<a href="/cgi-bin/o.pl?file=/c/11.htm">Title 2</a>
</li>
<br>
</ul>
<li>
<b>Part1 2</b>
</li>
<ul>
<li>
<a href="/cgi-bin/o.pl?file=/c/2.htm">Title A</a>
</li>
<br>
<li>
<a href="/cgi-bin/o.pl?file=/c/22.htm">Title B</a>
</li>
<br>
</ul>
<li>
<b>Part1 3</b>
</li>
<ul>
<li>
<a href="/cgi-bin/o.pl?file=/c/3.htm">Some text 1</a>
</li>
<br>
<li>
<a href="/cgi-bin/o.pl?file=/c/33.htm">Some Text 2</a>
</li>
</ul>
</ul>
<b>April 2017</b>
...
...
So on so forth
</body>
</html>
結果應該是:
January 2017
Part1 1
Title: Title 1, link: /cgi-bin/o.pl?file=/a/1.htm
Title: Title 1, link: /cgi-bin/o.pl?file=/a/1.htm
Part1 2
Title: Title 1, link: /cgi-bin/o.pl?file=/a/2.htm
Title: Title 1, link: /cgi-bin/o.pl?file=/a/22.htm
Part1 3
Title: Title 1, link: /cgi-bin/o.pl?file=/a/3.htm
Title: Title 1, link: /cgi-bin/o.pl?file=/a/33.htm
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
February 2017
Part1 1
Title: Title 1, link: /cgi-bin/o.pl?file=/b/1.htm
Title: Title 1, link: /cgi-bin/o.pl?file=/b/1.htm
Part1 2
Title: Title 1, link: /cgi-bin/o.pl?file=/b/2.htm
Title: Title 1, link: /cgi-bin/o.pl?file=/b/22.htm
Part1 3
Title: Title 1, link: /cgi-bin/o.pl?file=/b/3.htm
Title: Title 1, link: /cgi-bin/o.pl?file=/b/33.htm
我試圖//text()[following-sibling::b/text()='January 2017']/following::a[contains(@href, 'cgi-bin')]/text()
和類似的法術都無濟於事。
我應該怎麼辦?