Winterflags Winterflags - 5 months ago 26
Python Question

Selenium: Dynamically loaded page causes automatic scrolling to fail

I'm using

Selenium
for Python to navigate a dynamic webpage (link).

I would like to automatically scroll down to a y-point on the page. I have tried various ways to code this, which seem to work on other pages, but not on this particular webpage.

Some examples of my efforts that don't seem to work for Firefox with Selenium:

from selenium import webdriver

driver = webdriver.Firefox()
driver.get("http://en.boerse-frankfurt.de/etp/Deka-Oekom-Euro-Nachhaltigkeit-UCITS-ETF-DE000ETFL474")


By scrolling to a y-position:

driver.execute_script("window.scrollTo(0, 1400);")


Or by finding an element:

try:
find_scrollpoint = driver.find_element_by_xpath("//*[@id='main-wrapper']/div[10]/div/div[1]/div[1]")
except:
pass
else:
scrollpoint = find_scrollpoint.location["y"]
driver.execute_script("window.scrollTo(0, scrollpoint);")





Questions:

What could be some unusual circumstances on webpages such as this, where this very typical
scrollTo
code may fail?

What can possibly be done to mitigate it? Perhaps keypresses to scroll down – but it would still have to know when to stop pressing down.

Answer

It is just quite a dynamic page. I would wait for this specific element to be visible and then scroll into it's view. This works for me:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By


driver = webdriver.Firefox()
driver.get("http://en.boerse-frankfurt.de/etp/Deka-Oekom-Euro-Nachhaltigkeit-UCITS-ETF-DE000ETFL474")

wait = WebDriverWait(driver, 10)
elm = wait.until(EC.visibility_of_element_located((By.XPATH, "//*[@id='main-wrapper']/div[10]/div/div[1]/div[1]")))

driver.execute_script("arguments[0].scrollIntoView();", elm)
Comments