Rafa OR Rafa OR - 29 days ago 6
R Question

Change default arguments of ROC computation (pROC) in the RFE process (caret) - R

I am computing a SVM-RFE model using

"ROC"
as the metric, with the
rfe
function (
caret
package). To my knowledge, the
rfe
algorithm optimizes the AUC values using the
roc
function of the
pROC
package, with its predefined arguments. However, I would like to set the
direction
argument to
"<"
instead of
"auto"
because in some cases the resulting average AUC is computed in reverse (my data is not very good...). This problem is explained in the answer here: Difference in average AUC computation using ROCR and pROC (R)

How could I change this default argument value of the
roc
function in the
rfe
computation?

I have tried this simple option, but it does not work:

svmRFE_NG3 <- rfe(x = TAll[,2:50],
y = TAll[,1],
sizes = seq(1,42),
metric = "ROC",
levels = c("BREAST","LUNG"),
direction = "<",
rfeControl = FSctrl,
## Options to train()
method = "svmLinear",
tuneLength = 10,
preProc = c("center", "scale"),
## Inner resampling process
trControl = TRctrl)

Answer

I had a look at the source code of caret, and it doesn't seem to be readily possible at the moment. Arguments are not passed down to the call to the roc function.

I would suggest submitting a request for enhancement on the github repository of the package. Max Kuhn the maintainer is pretty responsive and you have a good chance to see this implemented in a future version.