user6575792 user6575792 - 4 months ago 9
Python Question

Scrape heading of a video using selenium python3

I want to scrape video name of this link as it is " Insane Woman Goes Crazy On Guy Who Just Wants A Refund".

The code on the web is:

<span id="eow-title" class="watch-title" dir="ltr" title="Insane Woman Goes Crazy On Guy Who Just Wants A Refund">
Insane Woman Goes Crazy On Guy Who Just Wants A Refund




I am doing in this way:

browser = webdriver.Firefox()
browser.get("https://www.youtube.com/watch?v=POk-uOQSJVk")
head = browser.find_elements_by_class_name('watch-title')
print(head.text)


It is prompting as:


AttributeError: 'list' object has no attribute 'text'


Is there any wrong?

Answer

First of all, find_elements_by_class_name() method returns a list of WebElements, while you need a single one. Also, you need to let the page load until the desired element is present:

from selenium import webdriver

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC


browser = webdriver.Firefox()
browser.get("https://www.youtube.com/watch?v=POk-uOQSJVk")

# wait for the presence of the video title
element = WebDriverWait(browser, 10).until(
    EC.presence_of_element_located((By.ID, "eow-title"))
)
print(element.text)

browser.close()

Prints:

Insane Woman Goes Crazy On Guy Who Just Wants A Refund