Euskalduna Euskalduna - 2 months ago 80
Python Question

Scikit-learn: How to obtain True Positive, True Negative, False Positive and False Negative

I am new in machine learning and in scikit-learn.

My problem:

(Please, correct any type of missconception)

I have a dataset which is a BIG JSON, I retrieve it and store it in a

trainList
variable.

I pre-process it in order to be able to work with it.

Once I have done that, I start the classification:


  1. I use kfold cross validation method in order to obtain the mean
    accuracy and I train a classifier.

  2. I make the predicctions and I obtain the accuracy and confusion matrix of that fold.

  3. After this, I would like to obtain the True Positive(TP), True Negative(TN), False Positive(FP) and False Negative(FN) values. I would use these paramters to obtain the Sensitivity and the specificity and I would them and the total of the TPs to a HTML in order to show a chart with the TPs of each label.



Code:

The variables I have for the moment:

trainList #It is a list with all the data of my dataset in JSON form
labelList #It is a list with all the labels of my data


Most part of the method:

#I transform the data from JSON form to a numerical one
X=vec.fit_transform(trainList)

#I scale the matrix (don't know why but without it, it makes an error)
X=preprocessing.scale(X.toarray())

#I generate a KFold in order to make cross validation
kf = KFold(len(X), n_folds=10, indices=True, shuffle=True, random_state=1)

#I start the cross validation
for train_indices, test_indices in kf:
X_train=[X[ii] for ii in train_indices]
X_test=[X[ii] for ii in test_indices]
y_train=[listaLabels[ii] for ii in train_indices]
y_test=[listaLabels[ii] for ii in test_indices]

#I train the classifier
trained=qda.fit(X_train,y_train)

#I make the predictions
predicted=qda.predict(X_test)

#I obtain the accuracy of this fold
ac=accuracy_score(predicted,y_test)

#I obtain the confusion matrix
cm=confusion_matrix(y_test, predicted)

#I should calculate the TP,TN, FP and FN
#I don't know how to continue

Answer

If you have two lists that have the predicted and actual values; as it appears you do you can pass them to a function that will calculate TP, FP, TN, FN with something like this:

def perf_measure(y_actual, y_hat):
    TP = 0
    FP = 0
    TN = 0
    FN = 0

    for i in range(len(y_hat)): 
        if y_actual[i]==y_hat[i]==1:
           TP += 1
    for i in range(len(y_hat)): 
        if y_actual[i]==1 and y_actual!=y_hat[i]:
           FP += 1
    for i in range(len(y_hat)): 
        if y_actual[i]==y_hat[i]==0:
           TN += 1
    for i in range(len(y_hat)): 
        if y_actual[i]==0 and y_actual!=y_hat[i]:
           FN += 1

return(TP, FP, TN, FN)

From here you I think you will be able to calculate rates of interest to you, and other performance measure like specificity and sensitivity.