Blabber Blabber - 5 months ago 13
Python Question

Union of two lists of dictionaries containing lists

I have two lists:

lst1 = [{u'Hours': [{u'HourOfDay': 15, u'TotalVisits': 1, u'TotalTimeSpent': 7223.0},
{u'HourOfDay': 12, u'TotalVisits': 3, u'TotalTimeSpent': 37550.0}],
u'DayOfWeek': 1},
{u'Hours': [{u'HourOfDay': 19, u'TotalVisits': 6, u'TotalTimeSpent': 23497.0}],
u'DayOfWeek': 2},
{u'Hours': [{u'HourOfDay': 12, u'TotalVisits': 10, u'TotalTimeSpent': 36453.0}],
u'DayOfWeek': 3},
{u'Hours': [{u'HourOfDay': 13, u'TotalVisits': 22, u'TotalTimeSpent': 0.0},
{u'HourOfDay': 13, u'TotalVisits': 3, u'TotalTimeSpent': 4800.0}],
u'DayOfWeek': 4},
{u'Hours': [{u'HourOfDay': 9, u'TotalVisits': 5, u'TotalTimeSpent': 11879.0},
{u'HourOfDay': 8, u'TotalVisits': 1, u'TotalTimeSpent': 924.0},
{u'HourOfDay': 19, u'TotalVisits': 1, u'TotalTimeSpent': 8075.0}],
u'DayOfWeek': 5},
{u'Hours': [{u'HourOfDay': 19, u'TotalVisits': 1, u'TotalTimeSpent': 426.0},
{u'HourOfDay': 14, u'TotalVisits': 1, u'TotalTimeSpent': 1528.0}],
u'DayOfWeek': 6},
{u'Hours': [{u'HourOfDay': 17, u'TotalVisits': 8, u'TotalTimeSpent': 0.0}],
u'DayOfWeek': 7}]
lst2 = [{u'Hours': [{u'HourOfDay': 3, u'TotalVisits': 2, u'TotalTimeSpent': 127.0},
{u'HourOfDay': 12, u'TotalVisits': 2, u'TotalTimeSpent': 107.0}],
u'DayOfWeek': 1},
{u'Hours': [{u'HourOfDay': 8, u'TotalVisits': 1, u'TotalTimeSpent': 3.0},
{u'HourOfDay': 7, u'TotalVisits': 1, u'TotalTimeSpent': 14.0}],
u'DayOfWeek': 2},
{u'Hours': [{u'HourOfDay': 8, u'TotalVisits': 1, u'TotalTimeSpent': 85.0},
{u'HourOfDay': 0, u'TotalVisits': 10, u'TotalTimeSpent': 1634.0}],
u'DayOfWeek': 3},
{u'Hours': [{u'HourOfDay': 8, u'TotalVisits': 10, u'TotalTimeSpent': 719.0},
{u'HourOfDay': 17, u'TotalVisits': 1, u'TotalTimeSpent': 2.0}],
u'DayOfWeek': 4},
{u'Hours': [{u'HourOfDay': 23, u'TotalVisits': 2, u'TotalTimeSpent': 32.0},
{u'HourOfDay': 20, u'TotalVisits': 3, u'TotalTimeSpent': 322.0},
{u'HourOfDay': 21, u'TotalVisits': 1, u'TotalTimeSpent': 0.0}],
u'DayOfWeek': 5},
{u'Hours': [{u'HourOfDay': 8, u'TotalVisits': 1, u'TotalTimeSpent': 548.0},
{u'HourOfDay': 17, u'TotalVisits': 2, u'TotalTimeSpent': 40.0}],
u'DayOfWeek': 6},
{u'Hours': [{u'HourOfDay': 3, u'TotalVisits': 3, u'TotalTimeSpent': 5324.0},
{u'HourOfDay': 20, u'TotalVisits': 2, u'TotalTimeSpent': 45.0}],
u'DayOfWeek': 7}]


They are separated by DayOfWeek and the Hours list contains dicts, each with different data for different hours of the day. Both lists are for separate data. I want to add the total time spent and visits in these two lists and create a new similar list out of it. I.e. if HourOfDay for the same DayOfWeek is the same then add the TotalTimeSpent and total visits, else append the info to the Hours list. I do not want to miss out on any data, just append the two as they might not have data for some hours of the day, but I still want them in the final one.

What I want is the union of the two lists for every day and if the HourOfDay field is the same for the two sets on the same DayOfWeek, then add the TotalVisits and TotalTimeSpent.

I've tried:

final = lst1[::]
i = j = 0
for item1, item2 in zip(lst1,lst2):
for hr1 in item1["Hours"]:
for hr2 in item2["Hours"]:
if hr1["HourOfDay"] == hr2["HourOfDay"]:
print "i: {0} ;; j: {1}".format(i,j)
final[i]["Hours"][j]["TotalTimeSpent"] = hr1["TotalTimeSpent"] + hr2["TotalTimeSpent"]
j += 1
i += 1
j = 0

print final


But it adds them up and leaves out values in the Hours list of every dict in lst2 that do not exist in lst1 because of obvious reasons. Please help!

Answer

Ok, here is a solution using your current data structure, which personally I think is silly because it combines the worst aspects of a list with the worst aspects of a dictionary:

final = [] 
for d1,d2 in zip(list1,list2):
    new = {'DayOfWeek':d1['DayOfWeek'], 'Hours':[]}
    d1copy = d1['Hours'][:]
    d2copy = d2['Hours'][:]
    for hr1 in d1['Hours']:
        for hr2 in d2['Hours']:
            if hr1["HourOfDay"] == hr2["HourOfDay"]:
                ttspent = hr1["TotalTimeSpent"] + hr2["TotalTimeSpent"]
                tvisits = hr1['TotalVisits'] + hr2['TotalVisits']
                combo = {'HourOfDay':hr1['HourOfDay'], 
                         'TotalTimeSpent':ttspent,
                         'TotalVisits':tvisits}
                new['Hours'].append(combo)
                d1copy.remove(hr1)
                d2copy.remove(hr2)
    new['Hours'].extend(d1copy)
    new['Hours'].extend(d2copy)
    final.append(new)

Results:

[{'DayOfWeek': 1,
  'Hours': [{'HourOfDay': 12, 'TotalTimeSpent': 37657.0, 'TotalVisits': 5},
   {'HourOfDay': 15, 'TotalTimeSpent': 7223.0, 'TotalVisits': 1},
   {'HourOfDay': 3, 'TotalTimeSpent': 127.0, 'TotalVisits': 2}]},
 {'DayOfWeek': 2,
  'Hours': [{'HourOfDay': 19, 'TotalTimeSpent': 23497.0, 'TotalVisits': 6},
   {'HourOfDay': 8, 'TotalTimeSpent': 3.0, 'TotalVisits': 1},
   {'HourOfDay': 7, 'TotalTimeSpent': 14.0, 'TotalVisits': 1}]},
 {'DayOfWeek': 3,
  'Hours': [{'HourOfDay': 12, 'TotalTimeSpent': 36453.0, 'TotalVisits': 10},
   {'HourOfDay': 8, 'TotalTimeSpent': 85.0, 'TotalVisits': 1},
   {'HourOfDay': 0, 'TotalTimeSpent': 1634.0, 'TotalVisits': 10}]},
 {'DayOfWeek': 4,
  'Hours': [{'HourOfDay': 13, 'TotalTimeSpent': 0.0, 'TotalVisits': 22},
   {'HourOfDay': 13, 'TotalTimeSpent': 4800.0, 'TotalVisits': 3},
   {'HourOfDay': 8, 'TotalTimeSpent': 719.0, 'TotalVisits': 10},
   {'HourOfDay': 17, 'TotalTimeSpent': 2.0, 'TotalVisits': 1}]},
 {'DayOfWeek': 5,
  'Hours': [{'HourOfDay': 9, 'TotalTimeSpent': 11879.0, 'TotalVisits': 5},
   {'HourOfDay': 8, 'TotalTimeSpent': 924.0, 'TotalVisits': 1},
   {'HourOfDay': 19, 'TotalTimeSpent': 8075.0, 'TotalVisits': 1},
   {'HourOfDay': 23, 'TotalTimeSpent': 32.0, 'TotalVisits': 2},
   {'HourOfDay': 20, 'TotalTimeSpent': 322.0, 'TotalVisits': 3},
   {'HourOfDay': 21, 'TotalTimeSpent': 0.0, 'TotalVisits': 1}]},
 {'DayOfWeek': 6,
  'Hours': [{'HourOfDay': 19, 'TotalTimeSpent': 426.0, 'TotalVisits': 1},
   {'HourOfDay': 14, 'TotalTimeSpent': 1528.0, 'TotalVisits': 1},
   {'HourOfDay': 8, 'TotalTimeSpent': 548.0, 'TotalVisits': 1},
   {'HourOfDay': 17, 'TotalTimeSpent': 40.0, 'TotalVisits': 2}]},
 {'DayOfWeek': 7,
  'Hours': [{'HourOfDay': 17, 'TotalTimeSpent': 0.0, 'TotalVisits': 8},
   {'HourOfDay': 3, 'TotalTimeSpent': 5324.0, 'TotalVisits': 3},
   {'HourOfDay': 20, 'TotalTimeSpent': 45.0, 'TotalVisits': 2}]}]