Yoklan Yoklan - 2 months ago 19
Node.js Question

Parse CSV into SQL Database

I've been tasked with parsing a CSV file and storing its data into an SQL database using Node.js. I'm a complete beginner with Node.js but have done similar tasks before in Rails. The CSV file given isn't like the previous ones I've used however and is in a different format.

Format of csv: http://imgur.com/a/gsQkl

I'm looking for any pointers on how to handle this task. Thanks


This question has 2 aspects,

  1. How to do the task
  2. How to do it with Node.JS

Regarding the first aspect - if you know how to do it with Rails it means that you should already know that the CSV example that you've provided is not just a table - it includes hierarchy which can be treated in multiple ways - either add category indicator and date fields to every row in order to flatten the table - or, create separate tables and connect them with external keys. Anyway - this has nothing to do with Node.JS, and you'll most probably have to "massage" your data before you enter it into SQL the database.

Regarding the second questions - in Node.JS you'll find modules to handle almost every task that you can imagine (some things can be done natively with the core modules, Google would be a good start in most cases)

in your case you'd need modules to handle CSV parsing, and SQL server connection

For CSV parsing you can use: https://github.com/wdavidw/node-csv

For SQL - you didn't mention which server are you using (SQL is a language used by many different database servers), assuming that you use one of the populars - this are the relevant modules:

MySQL - https://github.com/mysqljs/mysql

Microsoft SQL Server - https://github.com/patriksimek/node-mssql

PostgreSQL - https://github.com/brianc/node-postgres

Each one has its own interface - read the docs for further information