Kainesplain Kainesplain - 1 year ago 68
Python Question

How to save a scraped list in a CSV file?

I wrote this code below, which scrapes from the OED.com website words by subject and date and prints them out in a list.

import requests
import re
import urllib2
import os
import csv

year_search = 1550
subject_search = ['Law']

path = '/Applications/Python 3.5/Economic'
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())

user_agent = 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)'
header = {'User-Agent':user_agent}
request = urllib2.Request('http://www.oed.com/', None, header)
f = opener.open(request)
data = f.read()
print 'database first access was successful'

resultPath = os.path.join(path, 'OED_table.csv')
htmlPath = os.path.join(path, 'OED.html')
outputw = open(resultPath, 'w')
outputh = open(htmlPath, 'w')
request = urllib2.Request(
+ str(subject_search) + '&type=dictionarysearch', None, header)
page = opener.open(request)
urlpage = page.read()
new_word = re.findall(
r'<span class=\"hwSect\"><span class=\"hw\">(.*?)</span>', urlpage)
print str(new_word)

Now I want to print them to a CSV file but in such a way that every year I input will be placed as a row, and the words will all fall in the line of the row.

Sort of like:

1550| word1| word2| etc.|
1551| word1| word2| etc.|

Does anyone have any ideas?

Answer Source

I suggest using the csv.writer method. Here's the sample code:


with open('/Applications/Python 3.5/Economic/OED_table.csv', 'w') as csv_file:
    csv_writer = csv.writer(csv_file)
    year = ["1550"]
    new_word = ["apple", "banana"]
    complete_row = year + new_word
    # writes 1550, apple, banana to OED_table.csv


You can modify it with a for loop to insert multiple rows.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download