pproctor pproctor - 16 days ago 4
Python Question

Dict of lists to df

I have a dict of lists like so:

{291840: ['http://www.rollanet.org', 'http://www.rollanet.org],
291841: ['http://www.superpages.com', 'http://www.superpages.com],
291848: ['http://www.drscore.com/App/ScoreDr', 'http://www.drscore.com'],...etc }


I want to convert it to a two column dataframe, one for the subj_id and the other for the corresponding list. Each row will be a key of the dict and the column be the value(list) using from_dict with orient set to index. According to documentation: "orient: if the keys should be rows, pass ‘index’."

names = ['subj_id', 'URLs']

dfDict = pd.DataFrame(columns = names)
dfDict.from_dict(listDict, orient = 'index')


Instead I get a dataframe that has each element of the lists as a column. I only want two columns. One for the subj_ID and the other for the lists of URLs associated with subj_ID.

Answer

I think you need:

listDict = {291840: ['http://www.rollanet.org', 'http://www.rollanet.org'], 
     291841: ['http://www.superpages.com', 'http://www.superpages.com'], 
     291848: ['http://www.drscore.com/App/ScoreDr', 'http://www.drscore.com']}

names = ['subj_id', 'URLs']

df = pd.DataFrame(listDict).stack().reset_index(drop=True, level=0).reset_index()
df.columns = names
print (df)
   subj_id                                URLs
0   291840             http://www.rollanet.org
1   291841           http://www.superpages.com
2   291848  http://www.drscore.com/App/ScoreDr
3   291840             http://www.rollanet.org
4   291841           http://www.superpages.com
5   291848              http://www.drscore.com

Old answer:

df = pd.DataFrame.from_dict(listDict, orient='index').stack().reset_index(drop=True, level=1)

If need list in column URLs use list comprehensions:

df = pd.DataFrame({'subj_id': pd.Series([k for k,v in listDict.items()]),
                   'URLs': pd.Series([v for k,v in listDict.items()])}, columns = names)
print (df)
   subj_id                                               URLs
0   291840  [http://www.rollanet.org, http://www.rollanet....
1   291841  [http://www.superpages.com, http://www.superpa...
2   291848  [http://www.drscore.com/App/ScoreDr, http://ww...
Comments