Chris Chris - 5 months ago 9
Python Question

How to remove elements of lists in a dictionary of lists?

I am attempting to filter out pesky None values from a dictionary of lists. Here is an example dictionary:

parsed_data = {'DATA1': [None, None, None, None, None, None, None, None, None, None, None,
None, None, None, None, None, None, None, None, '0.17998362', '0.06388072',
'0.02091766', None, '0.00602364', '0.03171121', None, '1.39579976', '0.16731957',
'0.21564664', '0.03516583'], 'DATA2': [None, None, None, None, None, None, None,
None, None, None, None, None, None, None, None, None, None, None, None, None,
None, None, None, None, None, None, None, None, None, None, None, None], 'DATA3':
[None, None, None, None], 'DATA4': ['0.18406565', '0.06776296', '0.14278403',
'0.11962064', '0.00998447']}


I thought this dictionary comprehension would work, but it removes the entire key if it finds None anywhere in the list.

filtered = {x:y for x,y in parsed_data.items() if len(y)>0 and None not in y}


I also tried doing a dictionary comprehension with a nested list comprehension but it looked insane and made by brain melt a little just by looking at it.

Answer

You need to iterate over the elements in the list filtering the Nones and keeping everything else, if you want to change the original just update each list:

for v in parsed_data.values():
        v[:] = (ele for ele in v if ele is not None)

Or if you want a new dict:

 new = {k : [ele for ele in v if ele is not None] for k,v in parsed_data.items()}

Both will give you:

{'DATA1': ['0.17998362',
           '0.06388072',
           '0.02091766',
           '0.00602364',
           '0.03171121',
           '1.39579976',
           '0.16731957',
           '0.21564664',
           '0.03516583'],
 'DATA2': [],
 'DATA3': [],
 'DATA4': ['0.18406565',
           '0.06776296',
           '0.14278403',
           '0.11962064',
           '0.00998447']}

If you are using python 3, you can use combine filter into the first logic using None.ne:

for v in parsed_data.values():
        v[:] = filter(None.__ne__, v)