Rojo Rojo - 22 days ago 10
Python Question

Python Webscraping - Not able to get the value attribute from an element

I am trying the web scrape the value and text from a particular element(Drop Down) from a webpage using a combination of Python with selenium and Beautiful Soup. I am able to get the text but I am not able to get the value through get_attribute command. When I print the element that I located on the Webpage it returns the following content print(price) and for the print statement that gets it gives the error as None Type object is not callable. Kindly help me with this issue. Thanks in advance

price=soup.find("select",{"id":"space-prices"})
print(price)
print(price.text)
print(price.get_attribute('value'))


The output for print(price) is

<select class="pricing-bar-select" id="space-prices" name="space-prices"><option selected="selected" value="£360">Per Day</option>
<option value="£1,260">Per Week</option>
<option value="£5,460">Per Month</option>
<option value="£16,380">Per Quarter</option>
<option value="£65,520">Per Year</option></select>


The Url of the webpage is https://www.appearhere.co.uk/spaces/north-kensington-upcycling-store-and-cafe

Answer

It's because get_attribute seems to be None. It's not a valid attribute of the prices object. So it's not a function that you can call - hence the error. If you took away the parentheses and just printed prices.get_attribute nothing would print, because the value is None.

Also, the <select> tag doesn't have a "value" attribute in the first place. What you've done is you've grabbed the <select> tag, and all of it's children. Each child in the <select> tag (the <option> tags) have a "value" attribute. If you are trying to get all of the values of all of the <option> tags in that <select>, then you should do the following:

price=soup.find("select",{"id":"space-prices"})

# get all <options> in a list
options = price.find_all("option")

# for each element in that list, pull out the "value" attribute
values = [o.get("value") for o in options]
print(values)
#[u'\xa3360', u'\xa31,260', u'\xa35,460', u'\xa316,380', u'\xa365,520']
Comments