whoisearth whoisearth - 5 months ago 9
JSON Question

python - not printing additional rows from json

I have the below code but for some reason it's only writing the first row for car and ignoring every car after.

edit - I tried putting a

print field
in the row after
for field in required_fields:
and for some reason it only shows the field when parsing the first row. Every additional row it returns
[]
for each field.

What I see in the csv is the header and one row and then
,,,,


import csv
import json

car_list = [
{
"Year": 1997,
"Make": "Ford",
"Model": "E350",
"Description": "ac, abs, moon",
"Price": 3000
},
{
"Year": 1999,
"Make": "Chevy",
"Model": "Venture \"Extended Edition\"",
"Price": 4900
},
{
"Year": 1999,
"Model": "Venture \"Extended Edition, Very Large\"",
"Description": "",
"Price": 5000
},
{
"Year": 1996,
"Make": "Jeep",
"Model": "Grand Cherokee",
"Description": "MUST SELL!air, moon roof, loaded",
"Price": 4799
}
]


def _get_attr(dict_, attrs):
try:
src = attrs[:]
root = attrs.pop(0)
node = dict_[root]
null = object()
for i, attr in enumerate(attrs[:]):
try:
node = node.get(attr, null)
except AttributeError:
node = null
if node is null:
# i+2 pop and last element
raise ValueError("%s not present (level %s)" % (attr, '->'.join(src[: i+2])))
return node
except KeyError:
raise ValueError("%s not present" % root)

required_fields = [
['Year'],
['Make'],
['Model'],
['Description'],
['Price']
]

f = csv.writer(open('test.csv', 'wb+'))

f.writerow(['Year',
'Make',
'Model',
'Description',
'Price'])

for car in car_list:
row_values = []
for field in required_fields:
try:
row_values.append(_get_attr(car, field))
except:
row_values.append('')
f.writerow(row_values)

Answer

The reason this occurs is because of a combination of two things:

  • Lists (and more generally mutable objects) are passed by reference in Python. Mutating a list passed as an argument to a function mutates that list globally.

  • You're mutating attrs passed in to _get_attr by calling attrs.pop(0). This removes the field everywhere, including in required_fields.

A much better way to do what you want while using native Python:

required_fields = ['Year', 'Make', 'Model','Description','Price']

for car in car_list:
    row_values = [car.get(field, '') for field in required_fields]
    f.writerow(row_values)

This immediately takes care of possibly missing values by using the power of get() on dictionaries to supply a default value - '' in this case - if a key is missing, and requires no messy _get_attr function either.