user2338089 user2338089 - 25 days ago 12
Python Question

create a dictionary of unique nodes with unique properties from a csv in python to load in to neo4j

I'm having trouble wrapping my brain around the best way to do this; I can't figure out the dictionary structure to parse a csv and create various node dictionaries of unique nodes with unique properties to then load in to Neo4j.

Here is what I have so far:

import blah blah

driver = GraphDatabase.driver("bolt://localhost")
input_location = "/path/to/file/file.csv"
output_location = "/path/to/dir/"
def main():

origin = {"PORT": [], "NUMBER": [], "CALL_SIGN": []}
ship = {"BOAT_NAME": [], "BOAT_NUMBER": []}
shipper = {"NAME": [], "STREET": [], "CITY": [], "ZIPCODE": []}
destination = {"COUNTRY": [], "CITY": []}
node_type_list = [origin, ship, shipper, destination]

with open(input_location, "rb") as ship_data:
reader = csv.DictReader(ship_data, delimiter='|')
print "parsing & uploading data\n"
for row in reader:
for node_type in node_type_list:
for key in node_type:
dict_load(node_type,key,row[key])
send_nodes_to_graph(node_type_list)

def dict_load(node_type,key,value):
try:
if value not in node_type[key]:
node_type[key].append(value)
except Exception as e:
print e

def send_nodes_to_graph(node_type_list):
session = driver.session()
session.run('''something_like_this:http://stackoverflow.com/questions/40332698/can-a-python-dictionary-be-passed-as-neo4j-literal-maps/40333843#40333843''')
session.close()
if __name__ == '__main__':

main()


my csv looks like this:

COUNTRY NUMBER CALL_SIGN PORT BOAT_NAME BOAT_NUMBER NAME STREET CITY ST ZIPCODE
D REP 91487 S DOMINGO BALTIMORE PESCADO 1276394 JH FWEICH 9874 LOMBARDO WAY PORT ELIZABETH NJ 8348
D REP 91487 S DOMINGO VA BEACH TROPIC 9872347 JH FWEICH 9874 LOMBARDO WAY PORT ELIZABETH NJ 8348
D REP 91487 S DOMINGO VA BEACH TROPIC 9872347 JH FWEICH 9874 LOMBARDO WAY PORT ELIZABETH NJ 8348
D REP 91487 S DOMINGO VA BEACH CAPRICORN 8761231 JH FWEICH 9874 LOMBARDO WAY PORT ELIZABETH NJ 8348


my dict structure currently produces this:

origin {'NUMBER': ['91487'], 'CALL_SIGN': ['S DOMINGO'], 'PORT': ['BALTIMORE', 'VA BEACH']}


but I think it needs to look more like this, in order to load only unique nodes in to Neo4j:

origin {'91487': {'CALL_SIGN': 'S DOMINGO', 'PORT': 'BALTIMORE'}}
origin {'91487': {'CALL_SIGN': 'S DOMINGO', 'PORT': 'VA BEACH'}}
ship {'1276394': {'BOAT_NAME': 'PESCADO'}}
shipper {'JH FWEICH': {'STREET': '9874 LOMBARDO WAY', 'CITY':'PORT ELIZABETH'}}
etc....

Answer

Maybe try something like this?

origin = {"PORT", "NUMBER", "CALL_SIGN"}
ship = {"BOAT_NAME", "BOAT_NUMBER"}
shipper = {"NAME", "STREET", "CITY", "ZIPCODE"}
destination = {"COUNTRY", "CITY"}
node_type_list = [origin, ship, shipper, destination]

with open(input_location, "rb") as ship_data:
    reader = csv.DictReader(ship_data, delimiter='|')
    print "parsing & uploading data\n"
    for row in reader:
        dict_list = [{row["NUMBER"]: {key: row[key] for key in sublist}}for sublist in node_type_list]

This builds a list of dicts that map the NUMBERto a portion of the input dict. I'm not familiar with neo4j, but hopefully this is more what you wanted. Output should look like [{'91487': {'CALL_SIGN': 'S DOMINGO', 'PORT': 'BALTIMORE'}}, {'91487': {'BOAT_NUMBER': '1276394', 'BOAT_NAME': 'PESCADO'}}, ...]

Comments