用cURL登錄到網頁與PHP

Question

我買了一本書在網上抓取php。其中作者登錄到https://www.packtpub.com/。這本書已經過時了，所以我無法真正測試出想法，因爲網頁自發布以來已經發生了變化。這是我正在使用的修改後的代碼，但登錄失敗，我從「帳戶選項」字符串中得出的結論不在$results變量中。我應該改變什麼？我相信錯誤來自錯誤地指定目的地。

<?php 
// Function to submit form using cURL POST method 
function curlPost($postUrl, $postFields, $successString) { 
    $useragent = 'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; 
     en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3'; // Setting useragent of a popular browser 
    $cookie = 'cookie.txt'; // Setting a cookie file to storecookie 
    $ch = curl_init(); // Initialising cURL session 
    // Setting cURL options 
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // PreventcURL from verifying SSL certificate 
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0); 
    curl_setopt($ch, CURLOPT_FAILONERROR, TRUE); // Script shouldfail silently on error 
    curl_setopt($ch, CURLOPT_COOKIESESSION, TRUE); // Use cookies 
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // FollowLocation: headers 
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Returningtransfer as a string 
    curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie); // Settingcookiefile 
    curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie); // Settingcookiejar 
    curl_setopt($ch, CURLOPT_USERAGENT, $useragent); // Settinguseragent 
    curl_setopt($ch, CURLOPT_URL, $postUrl); // Setting URL to POSTto 
    curl_setopt($ch, CURLOPT_POST, TRUE); // Setting method as POST 
    curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($postFields)); // Setting POST fields as array 
      $results = curl_exec($ch); // Executing cURL session 
      $httpcode = curl_getinfo($ch,CURLINFO_HTTP_CODE); 
       echo "$httpcode"; 
      curl_close($ch); // Closing cURL session 
      // Checking if login was successful by checking existence of string 
      if (strpos($results, $successString)) { 
       echo "I'm in."; 
       return $results; 
      } else { 
       echo "Nope, sth went wrong."; 
       return FALSE; 
      } 
} 

$userEmail = 'youremail@email.com'; // Setting your email address for site login 
$userPass = 'yourpass'; // Setting your password for sitelogin 
$postUrl = 'https://www.packtpub.com'; // Setting URL toPOST to 
// Setting form input fields as 'name' => 'value' 
$postFields = array(
     'email' => $userEmail, 
     'password' => $userPass, 
     'destination' => 'https://www.packtpub.com', 
     'form_id' => 'packt-user-login-form' 
); 
$successString = 'Account Options'; 
$loggedIn = curlPost($postUrl, $postFields, $successString); //Executing curlPost login and storing results page in $loggedIn 

編輯：POST請求：

我取代了線

'destination' => 'https://www.packtpub.com' 
with  

'op' => 'Login' 

，加入

'form_build_id' => '' 

和編輯

$postUrl = 'https://www.packtpub.com/register'; 

因爲這是我在選擇複製爲cURL並在編輯器中粘貼時所得到的URL。

我仍然在「沒有，出錯了信息」。我認爲這是因爲$successString首先不會被存儲在curl中。應該設置的form-b​​uild-id是什麼？它每次登錄時都在變化。

Answer 1

你正在使用的書是舊的，Packt Publishing已經改變了他們的網站。它現在包含一個CSRF令牌，如果不通過這個，您將永遠無法登錄。

我開發了一個工作解決方案。它使用pQuery來解析HTML。您可以使用Composer安裝它，或者下載該軟件包並將其包含到您的應用程序中。如果這樣做，請刪除require __DIR__ . '/vendor/autoload.php';，並將其替換爲系統上pquery軟件包的位置。

要通過命令行進行測試，只需運行：php packt_example.php。

您還會注意到許多頭文件甚至都不需要，比如useragent。我已經離開了這些。

<?php 

require __DIR__ . '/vendor/autoload.php'; 

$email = 'myemail@gmail.com'; 
$password = 'mypassword'; 

# Initialize a cURL session. 
$ch = curl_init('https://www.packtpub.com/register'); 

# Set the cURL options. 
$options = [ 
    CURLOPT_COOKIEFILE  => 'cookies.txt', 
    CURLOPT_COOKIEJAR  => 'cookies.txt', 
    CURLOPT_RETURNTRANSFER => 1 
]; 

# Set the options 
curl_setopt_array($ch, $options); 

# Execute 
$html = curl_exec($ch); 

# Grab the CSRF token from the HTML source 
$dom = pQuery::parseStr($html); 
$csrfToken = $dom->query('[name="form_build_id"]')->val(); 

# Now we have the form_build_id (aka the CSRF token) we can 
# proceed with making the POST request to login. First, 
# lets create an array of post data to send with the POST 
# request. 
$postData = [ 
    'email'   => $email, 
    'password'  => $password, 
    'op'   => 'Login', 
    'form_build_id' => $csrfToken, 
    'form_id'  => 'packt_user_login_form' 
]; 


# Convert the post data array to URL encoded string 
$postDataStr = http_build_query($postData); 

# Append some fields to the CURL options array to make a POST request. 
$options[CURLOPT_POST] = 1; 
$options[CURLOPT_POSTFIELDS] = $postDataStr; 
$options[CURLOPT_HEADER] = 1; 

curl_setopt_array($ch, $options); 

# Execute 
$response = curl_exec($ch); 

# Extract the headers from the response 
$headerSize = curl_getinfo($ch, CURLINFO_HEADER_SIZE); 
$headers = substr($response, 0, $headerSize); 

# Close cURL handle 
curl_close($ch); 

# If login is successful, the headers will contain a location header 
# to the url http://www.packtpub.com/index 
if(!strpos($headers, 'packtpub.com/index')) 
{ 
    print 'Login Failed'; 
    exit; 
} 

print 'Logged In'; 

Answer 2

我在發佈這個答案，因爲我認爲它可能會在將來遇到這樣的問題時幫助你。我在寫網絡刮板時會做很多事情。

1. 打開Firefox。按CTRL + SHIFT + Q
2. 新聞網絡標籤
3. 轉到網站。您將注意到正在監視HTTP請求
4. 成功登錄，同時監視HTTP請求
5. 登錄後，右鍵單擊用於登錄的HTTP請求，然後複製爲CURL。

現在您有CURL請求。使用PHP的cURL複製HTTP請求。並再次測試。

對於網頁抓取，您應該非常熟悉監視HTTP標頭。您可以使用：

• 網絡監控器（Chrome，火狐）

• 提琴手

• Wiresharp

• MITMProxy

• 查爾斯

等...