Edward Edward - 3 years ago 426
Python Question

Scrapy returning scraped values into an array

Scrapy seems to be pulling the data out correctly, but is formatting the output in my JSON object as if it were an array:

[{"price": ["$34"], "link": ["/product/product..."], "name": ["productname"]},
{"price": ["$37"], "link": ["/product/product"]...

My spider class looks like this:

def parse(self, response):
sel = Selector(response)
items = sel.select('//div/ul[@class="product"]')
skateboards = []
for item in items:
skateboard = SkateboardItem()
skateboard['name'] = item.xpath('li[@class="desc"]//text()').extract()
skateboard['price'] = item.xpath('li[@class="price"]"]//text()[1]').extract()
skateboard['link'] = item.xpath('li[@class="image"]').extract()
return skateboards

How would I go about ensuring that Scrapy is only outputting a single value for each key, rather than the array it's currently producing?

Answer Source

always returns a list you can use


to get a string

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download