Webscraping - 找不到元素

https://www.novanthealth.org/home/patients--visitors/locations/clinics.aspx?behavioral-health=yes

我想要得到的診所名稱和地址，這是我使用

的Python代碼

from selenium import webdriver 
import pd 
import time 

#driver = webdriver.Chrome() 
specialty = ["behavioral-health","dermatology","colon","ear-nose-and- throat","endocrine","express","family-practice","foot-and-ankle", 
      "gastroenterology","heart-%26-vascular","hepatobiliary-and-pancreas","infectious-disease","inpatient","internal-medicine", 
      "neurology","nutrition","ob%2Fgyn","occupational-medicine","oncology","orthopedics","osteoporosis","pain-management", 
      "pediatrics","plastic-surgery","pulmonary","rehabilitation","rheumatology","sleep","spine","sports-medicine","surgical","urgent-care", 
      "urology","weight-loss","wound-care","pharmacy"] 
name = [] 
address = [] 

for q in specialty: 
    driver = webdriver.Chrome() 
    driver.get("https://www.novanthealth.org/home/patients-- visitors/locations/clinics.aspx?"+q+"=yes") 
    x = driver.find_element_by_class_name("loc-link-right") 
    num_page = str(x.text).split(" ") 
    x.click() 

    for i in num_page: 
     btn = driver.find_element_by_xpath('//*[@id="searchResults"]/div[2]/div[2]/button['+i+']') 
     btn.click() 
     time.sleep(8) #instaed of this use waituntil #  
     temp = driver.find_element_by_class_name("gray-background").text 
     temp0 = temp.replace("Get directions Website View providers\n","") 

     x_temp = temp0.split("\n\n\n") 

     for j in range(0,len(x_temp)-1): 
      temp1 = x_temp[j].split("Phone:") 
      name.append(temp1[0].split("\n")[1]) 
      temp3 = temp1[1].split("Office hours:") 
      temp4 = temp3[0].split("\n") 
      temp5 = temp4[1:len(temp4)] 
      address.append(" ".join(temp5)) 
    driver.close()

此代碼工作正常，如果我用它只有一個特殊的時間，但是當我通過特色的循環如上，代碼與錯誤的第二次迭代失敗：

Traceback (most recent call last): 
File "<stdin>", line 10, in <module> 
File "C:\Anaconda2\lib\site- packages\selenium\webdriver\remote\webelement.py", line 77, in click self._execute(Command.CLICK_ELEMENT) 
File C:\Anaconda2\lib\sitepackages\selenium\webdriver\remote\webelement.py", line 493, in _execute return self._parent.execute(command, params) 
File "C:\Anaconda2\lib\site-packages\selenium\webdriver\remote\webdriver.py",  line 249, in execute self.error_handler.check_response(response) 
File "C:\Anaconda2\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 193, in check_response 
    raise exception_class(message, screen, stacktrace) 
selenium.common.exceptions.ElementNotVisibleException: Message: element not visible 
(Session info: chrome=46.0.2490.80) 
(Driver info: chromedriver=2.19.346078 (6f1f0cde889532d48ce8242342d0b84f94b114a1),platform=Windows NT 6.1 SP1 x86_64

我不使用Python太多的經驗，任何幫助將不勝感激

來源

2017-04-24 Vaibhav

您必須讓您的網絡驅動程序等待幾秒鐘，直到相應的elem出現在頁面上。看看webdriver_wait函數.. –

我已經在閱讀這個文檔，但正面臨一些實現它的問題，你能給它一個示例代碼嗎？謝謝！ – Vaibhav

這裏是http://stackoverflow.com/a/41832157/3297613 –

錯誤消息告訴您爲什麼它不能正常工作。

ElementNotVisibleException: Message: element not visible

The element is not visible if you do not scroll down to see it.

你必須根據你的瀏覽器的尺寸向下滾動列表，

單從源頁，哪一個更容易提取數據。

來源

2017-04-24 07:56:02

通常我會在Selenium Basic中做一個excel插件。你可以在Python中使用相同的邏輯。這在VBA中嘗試過，對我來說工作正常。

Private assert As New assert 
Private driver As New Selenium.ChromeDriver 

Sub sel_novanHealth() 
Set ObjWB = ThisWorkbook 
Set ObjExl_Sheet1 = ObjWB.Worksheets("Sheet1") 
Dim Name As Variant 

    'Open the website 
    driver.get "https://www.novanthealth.org/home/patients--visitors/locations.aspx" 

    driver.Window.Maximize 

    driver.Wait (1000) 

    'Find out the total number of pages to be scraped 
    lnth = driver.FindElementsByXPath("//button[@class='paginate_button']").Count 
    'Running the Loop for the Pages 
    For y = 2 To lnth 
      'Running the Loop for the Elements 
      For x = 1 To 10 
       Name = driver.FindElementsByXPath("//div[@class='span12 loc-heading']")(x).Text 
       ' Element 2 
       'Element 3 
      Next x 
       driver.FindElementsByXPath("//button[@class='paginate_button']")(y).Click 
    Next y 

     driver.Wait (1000) 


End Sub

來源

2017-06-09 07:59:20 Sanjoy

Webscraping - 找不到元素

回答

相關問題