AppleGate0 AppleGate0 - 3 months ago 11
Python Question

Manipulating data frame in R

I'm trying to mungle my data from the following data frame to the one following it where the values in column B and C are combined to column names for the values in D grouped by the values in A.

Below is a reproducible example.

set.seed(10)

fooDF <- data.frame(A = sample(1:4, 10, replace=TRUE), B = sample(letters[1:4], 10, replace=TRUE), C= sample(letters[1:4], 10, replace=TRUE), D = sample(1:4, 10, replace=TRUE))
fooDF[!duplicated(fooDF),]

A B C D
1 4 c b 2
2 4 d a 2
3 2 a b 4
4 3 c a 1
5 4 a b 3
6 4 b a 2
7 1 b d 2
8 1 a d 4
9 2 b a 3
10 2 d c 2

newdata <- data.frame(A = 1:4)
for(i in 1:nrow(fooDF)){
col_name <- paste(fooDF$B[i], fooDF$C[i], sep="")
newdata[newdata$A == fooDF$A[i], col_name ] <- fooDF$D[i]


}


The format I am trying to get it in.

> newdata
A cb da ab ca ba bd ad dc
1 1 NA NA NA NA NA 2 4 NA
2 2 NA NA 4 NA 3 NA NA 2
3 3 NA NA NA 1 NA NA NA NA
4 4 2 2 3 NA 2 NA NA NA


Right now I am doing it line by line but that is unfeasible for a large csv containing 5 million + lines. Is there a way to do it faster in R or python?

Answer

In R, this can be done with tidyr

library(tidyr)
fooDF %>%
     unite(BC, B, C, sep="")  %>%
     spread(BC, D)
#  A ab ad ba bd ca cb da dc
#1 1 NA  4 NA  2 NA NA NA NA
#2 2  4 NA  3 NA NA NA NA  2
#3 3 NA NA NA NA  1 NA NA NA
#4 4  3 NA  2 NA NA  2  2 NA

Or we can do this with dcast

library(data.table)
dcast(setDT(fooDF), A~paste0(B,C), value.var = "D")
#    A ab ad ba bd ca cb da dc
#1: 1 NA  4 NA  2 NA NA NA NA
#2: 2  4 NA  3 NA NA NA NA  2
#3: 3 NA NA NA NA  1 NA NA NA
#4: 4  3 NA  2 NA NA  2  2 NA

data

fooDF <- structure(list(A = c(4L, 4L, 2L, 3L, 4L, 4L, 1L, 1L, 2L, 2L), 
B = c("c", "d", "a", "c", "a", "b", "b", "a", "b", "d"), 
C = c("b", "a", "b", "a", "b", "a", "d", "d", "a", "c"), 
D = c(2L, 2L, 4L, 1L, 3L, 2L, 2L, 4L, 3L, 2L)), .Names = c("A", 
"B", "C", "D"), class = "data.frame", row.names = c("1", "2", 
"3", "4", "5", "6", "7", "8", "9", "10"))