Seung Seung - 7 months ago 11
Python Question

web scraping using python beautifulsoup getting values after equal sign

enter image description here

Using Python, I need to get the name of the colors.

For example, data-label="BLACK" - here I need to get BLACK as the output

So far, I have:

Color=section.find("div",{"class":"sfa-pa-product-swatches-thumbnails-container"})

Answer

Iterate over the "thumbnail" elements and get the value of the data-label attribute:

colors = [elm.get("data-label", "No color specified") 
          for elm in soup.find_all("div", class_="sfa-pa-product-swatches-color")]

The complete code:

import requests
from bs4 import BeautifulSoup

url = "http://www.saksfifthavenue.com/Handbags/shop/_/N-52jzot/Ne-6lvnb5?FOLDER%3C%3Efolder_id=2534374306622829"
response = requests.get(url)

soup = BeautifulSoup(response.content, "html.parser")

for product in soup.select("#product-container [id^=product-]"):
    product_name = product.find(class_="product-designer-name").get_text()
    colors = [elm["data-label"]
              for elm in product.find_all("div", class_="sfa-pa-product-swatches-color")]
    print(product_name, colors)

Prints:

(u'Bao Bao Issey Miyake', [u'SILVER'])
(u'Nancy Gonzalez', [u'BLACK', u'BLUSH'])
(u'Bao Bao Issey Miyake', [u'GUNMETAL'])
...
(u'Saint Laurent', [u'WINE'])
(u'Fendi', [u'BLUE', u'FUCHSIA', u'WATER GREEN'])
(u'Prada', [u'LAGO'])

And make sure you are not violating the website's Terms of Use - stay on the legal side.