Fenil Suchak - 10 months ago 51

R Question

I have a data frame with no NA's(removed by na.omit()) with number of rows = 1,450,683 , while converting it into model.matrix to feed into glmnet the final matrix has some values that are Infinite.

`str(train_again)`

Classes 'tbl_df', 'tbl' and 'data.frame': 1450683 obs. of 24 variables:

$ vendor_id : Factor w/ 2 levels "1","2": 2 1 2 2 2 2 1 2 1 2 ...

$ passenger_count : int 1 1 1 1 1 6 4 1 1 1 ...

$ pickup_longitude : num -74 -74 -74 -74 -74 ...

$ pickup_latitude : num 40.8 40.7 40.8 40.7 40.8 ...

$ dropoff_longitude : num -74 -74 -74 -74 -74 ...

$ dropoff_latitude : num 40.8 40.7 40.7 40.7 40.8 ...

$ store_and_fwd_flag : Factor w/ 2 levels "N","Y": 1 1 1 1 1 1 1 1 1 1 ...

$ trip_duration : int 455 663 2124 429 435 443 341 1551 255 1274 ...

$ month : Factor w/ 6 levels "1","2","3","4",..: 3 6 1 4 3 1 6 5 5 5 ...

$ wday : Factor w/ 7 levels "Fri","Mon","Sat",..: 2 4 6 7 3 3 1 3 1 6 ...

$ hour : int 17 0 11 19 13 22 22 7 23 22 ...

$ work : Factor w/ 2 levels "FALSE","TRUE": 2 1 2 1 1 1 1 1 1 1 ...

$ jfk_trip : Factor w/ 2 levels "FALSE","TRUE": 1 1 1 1 1 1 1 1 1 1 ...

$ lg_trip : Factor w/ 2 levels "FALSE","TRUE": 1 1 1 1 1 1 1 1 1 1 ...

$ average.temperature: num 45.5 72.5 22 39 46.5 33.5 70.5 60 80 56.5 ...

$ rain : num 25 2 2 2 2 2 2 6 2 2 ...

$ s_fall : num 2 2 2 2 2 2 2 2 2 2 ...

$ s_depth : num 1 1 0.01 1 1 8 1 1 1 1 ...

$ total_distance : num 2009 2513 11061 1779 1615 ...

$ number_of_steps : int 5 6 16 4 5 5 5 17 2 6 ...

$ fastest_speed : num 43.9 27.3 51.9 27.2 41.5 ...

$ left_turns : int 1 2 5 2 2 1 1 4 0 2 ...

$ right_turns : int 1 2 7 1 2 3 3 9 1 2 ...

$ turns : int 1 2 9 1 3 3 2 6 0 3 ...

x = model.matrix(trip_duration~.,train_again) #here train_again is a data frame with no NA's

y = train_again$trip_duration

sum(is.infinite(x)) #gives output as 537

What could be the possible reason? Must there be something wrong with my original dataset?

Recommended for you: Get network issues from **WhatsUp Gold**. **Not end users.**

Answer Source

Given your comments that one of your variables has an `Inf`

(not `NA`

) value, I would recommend something like:

(Using tidyverse since you already show you have a tibble.) Drop response variable:

```
predvars <- dplyr::select(train_again,-trip_duration)
```

Find all-finite rows (no `NA`

, `NaN`

, or `Inf`

):

```
all_finite <- apply(is.finite(predvars),1,all)
```

You can use a one-sided formula in `model.matrix()`

:

```
x <- model.matrix(~.,predvars[all_finite,])
```

Recommended from our users: **Dynamic Network Monitoring from WhatsUp Gold from IPSwitch**. ** Free Download**