Alex P - 1 year ago 106

R Question

I would like to add additional metrics other than RMSE and Rsquared to the output of my linear model that I creating with the caret package. From what I understand, the code below will output the repeated cross-validated RMSE and Rsquared:

`library(caret)`

lm_reg1 <- train(log1p(mpg) ~ log1p(hp) + log1p(disp),

data = mtcars,

trControl = trainControl(method = "repeatedcv",

number = 10,

repeats = 10),

method = 'lm')

lm_reg

Output:

`Linear Regression`

32 samples

10 predictors

No pre-processing

Resampling: Cross-Validated (10 fold, repeated 10 times)

Summary of sample sizes: 30, 29, 28, 29, 29, 28, ...

Resampling results:

RMSE Rsquared

0.1134972 0.8808378

I know I can modify the output to a custom metric by modifying the summaryFunction in trainControl and referring to it's name in the metric parameter. Here's an example of one that I created that calculates the MAPE of a log-log model:

`mape <- function(actual, predicted){`

mean(abs((actual - predicted)/actual))

}

mapeexpSummary <- function (data,

lev = NULL,

model = NULL) {

out <- mape(expm1(data$obs), expm1(data$pred))

names(out) <- "MAPEEXP"

out

}

lm_reg2 <- train(log1p(mpg) ~ log1p(hp) + log1p(disp),

data = mtcars,

trControl = trainControl(method = "repeatedcv",

number = 10,

summaryFunction = mapeexpSummary,

repeats = 10),

metric = 'MAPEEXP',

method = 'lm')

lm_reg2

Output:

`Linear Regression`

32 samples

10 predictors

No pre-processing

Resampling: Cross-Validated (10 fold, repeated 10 times)

Summary of sample sizes: 28, 29, 29, 28, 28, 30, ...

Resampling results:

MAPEEXP

0.1022028

Is there any way to add them to a single output? I'm looking to save all of these values, but want to avoid creating two identical models to do so.

Answer Source

add in RMSE and Rsquared in your `mapeexpSummary`

?

```
mapeexpSummary <- function (data,
lev = NULL,
model = NULL) {
c(MAPEEXP=mape(expm1(data$obs), expm1(data$pred)),
RMSE=sqrt(mean((data$obs-data$pred)^2)),
Rsquared=summary(lm(pred ~ obs, data))$r.squared)
}
```