Martin Thoma Martin Thoma - 1 month ago 11
Python Question

How can random.shuffle result in a KeyError?

I just got the error:

Traceback (most recent call last):
File "./download_documents.py", line 153, in <module>
paragraphs, used_pages = find_pages(lang, to_extract)
File "./download_documents.py", line 67, in find_pages
random.shuffle(page_titles_queue)
File "/usr/lib/python2.7/random.py", line 291, in shuffle
x[i], x[j] = x[j], x[i]
KeyError: 1


Which confuses me quite a bit.


  1. random.shuffle
    seems to work on zero-element lists and on one-element lists.

  2. page_titles_queue
    is a list of tuples.

  3. Two lines after the
    random.shuffle(page_titles_queue)
    , there is
    page_titles_queue.pop()
    , but that should not affect the shuffle. Right?



So what are possible reasons for the KeyError?

I use
Python 2.7.12
on Ubuntu 16.04.

Answer Source

random.shuffle just exchanges items, the line where the exception happened makes this perfectly clear:

x[i], x[j] = x[j], x[i]

Where x is the "sequence" that was passed in. In this case i and j will be values in the range range(0, len(x)) and if any of these i or j isn't present in the "sequence" it will throw an Exception. In your case it's very likely given that it throws a KeyError:

>>> import random
>>> d = {i: i for i in range(7, 10)}
>>> random.shuffle(d)
KeyError: 3

However it works by exchanging the values in case the dictionary contains exactly the keys that make up the range(0, len(x)):

>>> d = {i: i for i in range(10)}
>>> random.shuffle(d)
>>> d
{0: 7, 1: 9, 2: 3, 3: 4, 4: 0, 5: 2, 6: 1, 7: 6, 8: 8, 9: 5}

If one or multiple keys are missing it could work or it could throw an Exception. That depends on which random numbers will be drawn:

d = {i: i for i in range(1, 10)}
random.shuffle(d)   # works sometimes, but sometimes it throws the KeyError