tuxdna tuxdna - 2 months ago 44
R Question

R DataFrame - One Hot Encoding of column containing multiple terms

I have a dataframe with a column having multiple values ( comma separated ):

mydf <- structure(list(Age = c(99L, 10L, 40L, 15L),
Info = c("good, bad, sad", "nice, happy, joy", "NULL", "okay, nice, fun, wild, go"),
Target = c("Boy", "Girl", "Boy", "Boy")),
.Names = c("Age", "Info", "Target"),
row.names = c(NA, 4L),
class = "data.frame")

> mydf
Age Info Target
1 99 good, bad, sad Boy
2 10 nice, happy, joy Girl
3 40 NULL Boy
4 15 okay, nice, fun, wild, go Boy


I want to split Info column into one-hot-encoded columns, and append the results besides Target column, for example:

Age Info Target good bad sad nice ... NULL ..
1 99 good, bad, sad Boy 1 1 1 0 0
2 10 nice, happy, joy Girl 0 0 0 1 0
3 40 NULL Boy 0 0 0 0 1
4 15 okay, nice, fun, wild, go Boy 0 0 0 0 0


In python I could do something like below, to obtain a dictionary and then use it to assign columns.

In [1]: import itertools

In [2]: values = ["good, bad, sad", "nice, happy, joy", "NULL", "okay, nice, fun, wild, go"]

In [3]: terms = list(itertools.chain(*[v.split(", ") for v in values]))

In [4]: dictionary = {v:k for k,v in enumerate(terms)}

In [6]: dictionary
Out[6]:
{'NULL': 6, 'bad': 1,
'fun': 9, 'go': 11, 'good': 0, 'happy': 4,
'joy': 5, 'nice': 8, 'okay': 7, 'sad': 2, 'wild': 10}


So far I can do this in R

> lapply(mydf["Info"], function(x) { strsplit(x, ", ") } )
$Info
$Info[[1]]
[1] "good" "bad" "sad"

$Info[[2]]
[1] "nice" "happy" "joy"

$Info[[3]]
[1] "NULL"

$Info[[4]]
[1] "okay" "nice" "fun" "wild" "go"


I am not getting how to convert it into a dictionary in R, and use it to convert to columns for One-Hot-Encoding.

How can I solve this?

Answer

One option is mtabulate from qdapTools after splitting the 'Info' column by ,

library(qdapTools)
cbind(mydf, mtabulate(strsplit(mydf$Info, ", ")))
#Age                      Info Target bad fun go good happy joy nice NULL okay sad wild
#1  99            good, bad, sad    Boy   1   0  0    1     0   0    0    0    0   1    0
#2  10          nice, happy, joy   Girl   0   0  0    0     1   1    1    0    0   0    0
#3  40                      NULL    Boy   0   0  0    0     0   0    0    1    0   0    0
#4  15 okay, nice, fun, wild, go    Boy   0   1  1    0     0   0    1    0    1   0    1
Comments