Seung Seung - 2 years ago 98
Python Question

web scraping using python beautifulsoup getting values after equal sign

enter image description here

Using Python, I need to get the name of the colors.

For example, data-label="BLACK" - here I need to get BLACK as the output

So far, I have:


Answer Source

Iterate over the "thumbnail" elements and get the value of the data-label attribute:

colors = [elm.get("data-label", "No color specified") 
          for elm in soup.find_all("div", class_="sfa-pa-product-swatches-color")]

The complete code:

import requests
from bs4 import BeautifulSoup

url = ""
response = requests.get(url)

soup = BeautifulSoup(response.content, "html.parser")

for product in"#product-container [id^=product-]"):
    product_name = product.find(class_="product-designer-name").get_text()
    colors = [elm["data-label"]
              for elm in product.find_all("div", class_="sfa-pa-product-swatches-color")]
    print(product_name, colors)


(u'Bao Bao Issey Miyake', [u'SILVER'])
(u'Nancy Gonzalez', [u'BLACK', u'BLUSH'])
(u'Bao Bao Issey Miyake', [u'GUNMETAL'])
(u'Saint Laurent', [u'WINE'])
(u'Fendi', [u'BLUE', u'FUCHSIA', u'WATER GREEN'])
(u'Prada', [u'LAGO'])

And make sure you are not violating the website's Terms of Use - stay on the legal side.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download