sweeeeeet sweeeeeet - 1 month ago 10
R Question

Get optimal threshold with pROC in R

I have a dataframe with two columns :

score1
which is
numeric
and
truth1
which is
boolean
.
I want to predict
truth1
using
score1
. To do that, I want a simple linear model, and then ask for a good threshold, i.e., a threshold which gives me 75% of sensitivity in my ROC curve. Hence, I do :

roc_curve = roc(truth1 ~ score1 , data = my_data)
coords(roc=roc_curve, x = 0.75, input='sensitivity', ret='threshold')


My problem is that coords return 'NA', because the sensitivty of 0.75 does not appear in the ROC curve. So here is my question: how can I get the threshold which gives me a sensitivity of at least 0.75, with max specificity?

Answer

Option 1: you filter the results

my.coords <- coords(roc=roc_curve, x = "all")
my.coords[,my.coords["sensitivity",]>=.75]

Option 2: you can trick pROC by requesting a partial AUC between 75% and 100% of sensitivity:

roc_curve = roc(truth1 ~ score1 , data = my_data, partial.auc = c(1, .75), partial.auc.focus="sensitivity")

All other functions will follow and give you results only in this area of interest:

coords(roc=roc_curve, x = "local maximas", ret='threshold')