我想湊這個網站:https://www.99acres.com如何湊多頁有一個不變的URL - 的Python和BeautifulSoup
到目前爲止,我已經使用BeautifulSoup來執行代碼,並提取從網站上的數據;然而,我現在的代碼只能讓我成爲第一頁。我想知道是否有訪問其他頁面的方法,因爲當我點擊下一頁時,URL不會改變,所以我不能每次迭代不同的URL。
下面是到目前爲止我的代碼:
import io
import csv
import requests
from bs4 import BeautifulSoup
response = requests.get('https://www.99acres.com/search/property/buy/residential-all/hyderabad?search_type=QS&search_location=CP1&lstAcn=CP_R&lstAcnId=1&src=CLUSTER&preference=S&selected_tab=1&city=269&res_com=R&property_type=R&isvoicesearch=N&keyword_suggest=hyderabad%3B&bedroom_num=3&fullSelectedSuggestions=hyderabad&strEntityMap=W3sidHlwZSI6ImNpdHkifSx7IjEiOlsiaHlkZXJhYmFkIiwiQ0lUWV8yNjksIFBSRUZFUkVOQ0VfUywgUkVTQ09NX1IiXX1d&texttypedtillsuggestion=hy&refine_results=Y&Refine_Localities=Refine%20Localities&action=%2Fdo%2Fquicksearch%2Fsearch&suggestion=CITY_269%2C%20PREFERENCE_S%2C%20RESCOM_R&searchform=1&price_min=null&price_max=null')
html = response.text
soup = BeautifulSoup(html, 'html.parser')
list=[]
dealer = soup.findAll('div',{'class': 'srpWrap'})
for item in dealer:
try:
p = item.contents[1].find_all("div",{"class":"_srpttl srpttl fwn wdthFix480 lf"})[0].text
except:
p=''
try:
d = item.contents[1].find_all("div",{"class":"lf f13 hm10 mb5"})[0].text
except:
d=''
li=[p,d]
list.append(li)
with open('project.txt','w',encoding="utf-8") as file:
writer= csv.writer(file)
for row in list:
writer.writerows(row)
file.close()
抱歉打擾你,但我不明白你在告訴我什麼。你能修改我的代碼,以便我能夠從下一頁中提取數據。 –