user3226932 user3226932 - 5 months ago 23
Python Question

Python list comprehension: adding unique elements into list?

I'm trying to write simpler code for adding unique elements into a python list. I have a dataset that contains a list of dictionaries, and I'm trying to iterate through a list inside the dictionary

Why doesn't this work? It's adding all the items, including the duplicates, instead of adding unique items.

unique_items = []
unique_items = [item for d in data for item in d['items'] if item not in unique_items]

vs. the longer form which works:

unique_items = []
for d in data:
for item in d['items']:
if (item not in unique_items):

Is there a way of making this work using list comprehension, or am I stuck with using double for loops? I want to keep the ordering for this.

Here's the list of dictionaries:

[{"items":["apple", "banana"]}, {"items":["banana", "strawberry"]}, {"items":["blueberry", "kiwi", "apple"]}]

output should be ["apple", "banana", "strawberry", "blueberry", "kiwi"]

I noticed someone asking a similar question on another post: Python list comprehension, with unique items, but I was wondering if there's another way to do it without OrderedDict or if that's the best way


The easiest way is to use OrderedDict:

from collections import OrderedDict
from itertools import chain

l = [{"items":["apple", "banana"]}, {"items":["banana", "strawberry"]}, {"items":["blueberry", "kiwi", "apple"]}]
OrderedDict.fromkeys(chain.from_iterable(d['items'] for d in l)).keys() # ['apple', 'banana', 'strawberry', 'blueberry', 'kiwi']

If you want alternatives check OrderedSet recipe and package based on it.