Kastany Kastany - 1 year ago 73
R Question

read.csv to import more than 2105 columns?

Leaving question for archival purposes only.
(Read.csv did read in all columns, I just did not see them in the preview when opening the data.frame)

Related to this question:
Maximum number of columns that can be read using read.csv
I would like to import a csv file into R that contains about 3200 columns (100 rows). I am used to working with data.frames and read.csv, but my usual approach failed because

data <- read.csv("data.csv", header=TRUE)

only imported the first 2105 columns. It did not display an error message.

  • How can I readin a csv file with more than 2105 columns?

    • without specifying column classes

    • into a data frame

    • the file contains different data types (dates, strings, numbers, ..)

    • speed is not my biggest concern

I did not manage to apply the solutions in Quickly reading very large tables as dataframes in R to my situation. Tried this, but it does not seem to work without information on column classes:

df <- as.data.frame(scan("data.csv",sep=','))

There are already several questions about reading in large datafiles with millions of rows/columns and how to speed up the process, but my files are much smaller, so I am hoping that there is an easier solution that I overlooked.

Answer Source

Try using data.table.


data <- fread("data.csv")
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download