user3706469 user3706469 - 1 year ago 106
Python Question

How to rename images downloaded by scrapy

I am downloading multiple images from a single page and I want to rename them as

etc means in the numbers only. For this i have to various other questions on this topic. Everybody says to change in the file_path method also i am able to do so but now problem is its overriding the previous image because of same name.

I don't know any other way to increase the count.only way is static count but in this case its not able to scrap any image.

My code is:

count1 = '1' # change to request.url after deprecation
return 'full/%s.jpg' % (count1)

Answer Source

Because file names must be unique within a directory, I would recommend creating a new directory based on the URL from which you've scraped the images.

Then, numbering the images becomes as simple as keeping a counter of how many images you've downloaded so far:

counter = 1
img_sources = sel.xpath("//img/@src").extract()
for img_source in img_sources:
    download_image(img_source, os.path.join(url, "%d.jpg" % (counter)))
    counter += 1
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download