Question

我有一個html代碼，顯示了谷歌趨勢的相關專題：

<iframe id="trends-widget-1" src='https://trends.google.com/trends/embed/explore/RELATED_TOPICS?req={"comparisonItem":[{"keyword":"stack","geo":"BR","time":"today 5-y"}],"category":0,"property":""}&amp;tz=180&amp;eq=geo=BR&q=stack' width="100%" frameborder="0" scrolling="0" style="border-radius: 2px; box-shadow: rgba(0, 0, 0, 0.12) 0px 0px 2px 0px, rgba(0, 0, 0, 0.24) 0px 2px 2px 0px; height: 384px;"></iframe> 

現在，我想找到一個方法來拯救這個網站（以備將來使用...）。爲此，我嘗試使用CURL：

$url = 'https://trends.google.com/trends/embed/explore/RELATED_TOPICS?req={"comparisonItem":[{"keyword":"stack","geo":"BR","time":"today 5-y"}],"category":0,"property":""}&tz=180&eq=geo=BR&q=stack'; 

     $ch = curl_init(); 
     $source = $url; 
     curl_setopt($ch, CURLOPT_URL, $source); 
     curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
     curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0); 
     curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0); 
     curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 1000); 
     curl_setopt($ch, CURLOPT_TIMEOUT, 100); 
     curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); 
     curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.A.B.C Safari/525.13"); 
     $html = curl_exec($ch); 
     curl_close($ch); 
     echo $html; 

問題？ curl示出了與此消息的谷歌頁面：

這是一個錯誤。您的客戶發佈了格式不正確或非法的請求。我們知道的就這些。

我怎樣才能避免這種類型的問題，並提取html？

Answer 1

來源url的查詢字符串部分是html實體和非url編碼文本的混合。

我認爲這樣做是爲了更加難以正確解碼抓取工具的URL。

無論如何，瀏覽器能夠正確解釋查詢字符串首先解碼html實體，然後識別每個查詢參數及其值。

瀏覽器用來完成上述解碼的algorythm並不是微不足道的，並且沒有專門的PHP函數來完成這項工作。如果你對這個主題感興趣，我認爲它應該得到一個專門的問題。

針對您的特殊情況下，您可以修正網址是這樣的：

// The base URL is ok 

$url = 'https://trends.google.com/trends/embed/explore/RELATED_TOPICS?'; 

// The `req` parameter's value must be url-encoded 

$url .= 'req=' . urlencode('{"comparisonItem":[{"keyword":"stack","geo":"BR","time":"today 5-y"}],"category":0,"property":""}'); 

// The last part of the query string contains html entities, specifically & 
// They have to be "translated" into ampersands to let the query make sense 
// (I did it manually) 
// 
// Note also the final part of the query string does not contain special 
// characters so I skiped the URL encoding 

$url .= '&tz=180&eq=geo=BR&q=stack'; 

你這個網址

https://trends.google.com/trends/embed/explore/RELATED_TOPICS?req=%7B%22comparisonItem%22%3A%5B%7B%22keyword%22%3A%22stack%22%2C%22geo%22%3A%22BR%22%2C%22time%22%3A%22today+5-y%22%7D%5D%2C%22category%22%3A0%2C%22property%22%3A%22%22%7D&tz=180&eq=geo=BR&q=stack 

這工作正常在瀏覽器欄，並與捲曲

都粘貼結束

---

底注：

我不確定從頁面的源中獲取了多少信息，因爲它大量使用JavaScript和Ajax調用來呈現內容。