2016-11-23 42 views
0

我想發送json請求來刮掉this link中的無限滾動元素。儘管如此,我知道一些參數不是必需的,但可以肯定的是,我定義了完全相同的參數發送到服務器:我的參數和代碼發送不同的json請求,但在Python中得到類似的結果

import requests 
import json 
parameters1 = {'ticker':'XOM', 'countryCode':'US', 
      'dateTime':'12%3A38+p.m.+Oct.+24%2C+2016', 'docId':'', 
      'docType':'806','sequence':'e5a00f51-8821-4fbc-8ac6-e5f64b5eb0f2', 
      'messageNumber':'8541','count':'10', 
      'channelName':'%2Fnews%2Flatest%2Fcompany%2Fus%2Fxom', 'topic':'', 
      '_':'1479888927416' } 



parameters2 = {'ticker':'XOM', 'countryCode':'US', 
      'dateTime':'12%3A38+p.m.+Oct.+24%2C+2016','docId':'', 
      'docType':'806' ,'sequence':'e5a00f51-8821-4fbc-8ac6-e5f64b5eb0f2', 
      'messageNumber':'8525','count':'10', 
      'channelName':'%2Fnews%2Flatest%2Fcompany%2Fus%2Fxom', 'topic':'', 
      '_':'1479888927417' } 


firstUrl = "http://www.marketwatch.com/news/headline/getheadlines" 
html1 = requests.get(firstUrl, params = parameters1) 
result1 = (json.loads(html1.text)) 

html2 = requests.get(firstUrl, params = parameters2) 
result2 = (json.loads(html2.text)) 

,我檢查,如果他們是相同的:

if(result2 == result1): 
    print(True) 

答案始終是真實的。我改變了許多參數,但沒有奏效。我的代碼或prodecure有什麼問題,我經歷了?

回答

1

你的問題是,你發送JSON,但使用編碼的字符串。而不是%2F您應該使用/,而不是+一個空格,而不是%3A一個:等。你可以解碼你的字符串,例如on this site

import requests 
import json 
parameters1 = {'ticker':'XOM', 'countryCode':'US', 
      'dateTime':'12:38 p.m. Oct., 2016', 'docId':'', 
      'docType':'806','sequence':'e5a00f51-8821-4fbc-8ac6-e5f64b5eb0f2', 
      'messageNumber':'8541','count':'10', 
      'channelName':'/news/latest/company/us/xom', 'topic':'', 
      '_':'1479888927416' } 

parameters2 = {'ticker':'XOM', 'countryCode':'US', 
      'dateTime':'12:38 p.m. Oct., 2016','docId':'', 
      'docType':'806' ,'sequence':'e5a00f51-8821-4fbc-8ac6-e5f64b5eb0f2', 
      'messageNumber':'8525','count':'10', 
      'channelName':'/news/latest/company/us/xom', 'topic':'', 
      '_':'1479888927417' }; 

firstUrl = "http://www.marketwatch.com/news/headline/getheadlines" 
html1 = requests.get(firstUrl, params = parameters1) 
result1 = (json.loads(html1.text)) 


html2 = requests.get(firstUrl, params = parameters2); 
result2 = (json.loads(html2.text)) 

然後result1==result2False