ok123jump ok123jump - 7 months ago 13
Python Question

Python Recursive Search of Dict with Nested Keys

I recently had to solve a problem in a real data system with a nested dict/list combination. I worked on this for quite a while and came up with a solution, but I am very unsatisfied. I had to resort to using

globals()
and a named temporary global parameter.

I do not like to use globals. That's just asking for an injection vulnerability. I feel that there must be a better way to perform this task without resorting to globals.

Problem Dataset:

d = {
"k":1,
"stuff":"s1",
"l":{"m":[
{
"k":2,
"stuff":"s2",
"l":None
},
{
"k":3,
"stuff":"s3",
"l":{"m":[
{
"k":4,
"stuff":"s4",
"l":None
},
{
"k":5,
"stuff":"s5",
"l":{"m":[
{
"k":6,
"stuff":"s6",
"l":None
},
]}
},
]}
},
]}
}


Desired Output:

[{'k': 1, 'stuff': 's1'},
{'k': 2, 'stuff': 's2'},
{'k': 3, 'stuff': 's3'},
{'k': 4, 'stuff': 's4'},
{'k': 5, 'stuff': 's5'},
{'k': 6, 'stuff': 's6'}]


My Solution:

def _get_recursive_results(d, iter_key, get_keys):
if not 'h' in globals():
global h
h = []
h.append({k:d.get(k) for k in get_keys})

d2 = d.copy()
for k in iter_key:
if not d2:
continue
d2 = d2.get(k)

for td in d2:
d3 = td.copy()
for k in iter_key:
if not d3:
continue
d3 = d3.get(k)

if d3:
return _get_recursive_results(td, iter_key, get_keys)
h.append({k:td.get(k) for k in get_keys})
else:
l = [k for k in h]
del globals()['h']
return l


Calling my function as follows returns the desired result:

_get_recursively(d, ['l','m'], ['k','stuff'])


How would I build a better solution?

Answer

This is a slightly modified version without using globals. Set h to None as default and create a new list for the first call to _get_recursive_results(). Later provide h as an argument in the recursive calls to _get_recursive_results():

def _get_recursive_results(d, iter_key, get_keys, h=None):
    if h is None:
        h = []
    h.append({k:d.get(k) for k in get_keys})
    d2 = d.copy()
    for k in iter_key:
        if not d2:
            continue
        d2 = d2.get(k)
    for td in d2:
        d3 = td.copy()
        for k in iter_key:
            if not d3:
                continue
            d3 = d3.get(k)
        if d3:
            return _get_recursive_results(td, iter_key, get_keys, h)
        h.append({k:td.get(k) for k in get_keys})
    else:
        l = [k for k in h]
        return l

Now:

>>> _get_recursive_results(d, ['l','m'], ['k','stuff'])
[{'k': 1, 'stuff': 's1'},
 {'k': 2, 'stuff': 's2'},
 {'k': 3, 'stuff': 's3'},
 {'k': 4, 'stuff': 's4'},
 {'k': 5, 'stuff': 's5'},
 {'k': 6, 'stuff': 's6'}]

There is no need for the copying of intermediate dicts. This is a further modified version without copying:

def _get_recursive_results(d, iter_key, get_keys, h=None):
    if h is None:
        h = []
    h.append({k: d.get(k) for k in get_keys})
    for k in iter_key:
        if not d:
            continue
        d = d.get(k)
    for td in d:
        d3 = td
        for k in iter_key:
            if not d3:
                continue
            d3 = d3.get(k)
        if d3:
            return _get_recursive_results(td, iter_key, get_keys, h)
        h.append({k: td.get(k) for k in get_keys})
    else:
        return h
Comments