jas jas - 5 months ago 26
Python Question

ValueError: too many values to unpack with too many tuples

I get an error while sorting about 470000 tuples.
I read data from csv file into list of tuples

fp = open(filename, 'Ur')
for line in fp:
listOfCitiesTuples.append(tuple(line.strip().split(',')))
fp.close()


Tuples are some cities names associated with a number.

[('Chiredzi', '4117'), ('Gaths', '4117'), ('Masvingo', '4117'), ('Chivhu', '4120'), ('Gweru', '4120'), ('Kwekwe', '4120'), ('Mvuma', '4120'), ('Redcliffe', '4120'), ('Shurugwi', '4120'), ('Zvishavane', '4120')]


I collect all the names associated to same number together in a list and map it to that number and form a dict of all those items, something like this

{'1': ['Bombuflat', 'Garacharma', 'Port Blair', 'Rangat'], '113': ['Hydra', 'Kouba'], '294': ['Vienna', 'Wien'], '1327': ['Lambarene', 'Ndjole']}


Using following method to achieve it

for k, v in listOfCitiesTuples: dictOfCitiesTuples.setdefault(v, []).append(k)


This works fine for a smaller number of tuples( I have tested 20,000 so far), but doesn't seem to be working on larger number like 470000.

Could large number of tuples be a problem or could it be due to corrupted file? If it is due to some corrupted data in a file, is there something I can do to find out what corrupted data it is like try, exception ?

I get this error

File "/../view.py", line 186, in getCities
for k, v in listOfCitiesTuples: dictOfCitiesTuples.setdefault(v, []).append(k)
ValueError: too many values to unpack (expected 2)

Answer

Based on your title ValueError: too many values to unpack you have some data in a different format than you are expecting.

Specifically, some lines have more than one comma which will make a tuple with more than 2 values, which then causes the error when you try to unpack them.

While you are iterating over the file, you can check if the tuple is the correct length. If you find bad data, you can make a note of it and fix it, or ignore it. Depends on your needs.

with open(filename, 'Ur') as infile:
    for line_num, line in enumerate(infile):
      vals = tuple(line.strip().split(','))
      if len(vals) == 2:
          listOfCitiesTuples.append(vals)
      else:
          bad_data.append((line_num, line))