- we worked with this confusion matrix, representing the resultsof a model used to differentiate between fraudulent and legitimatetransactions (where p = actually fraudulent, n = actuallylegitimate, T = labelled fraudulent, F = labelled legitimate):
Raw Counts (Model 1) |
True Class |
||
p |
n |
||
Predicted Class |
T |
30 |
58 |
F |
32 |
920 |
- (12 points) Convert this matrix to one of expected rates (i.e.,probabilities) and determine the class priors. (Please round yourresults to 3 digits.)
Expected Rates (Model 1) |
True Class |
||
p |
n |
||
Predicted Class |
T |
||
F |
Prob (p) = Prob (n) =
- (8 points) Calculate the following rates:
TP Rate = FP Rate =
FN Rate = TN Rate =
- (8 points) Develop a cost-benefit matrix, where any transactionclassified as fraudulent costs a standard unit (i.e., -1) tohandle, but a fraudulent transaction misclassified as legitimatecosts 100 times that amount. Correctly identifying a legitimatetransaction has neither a cost nor a benefit.
Costs/Benefits (Model 1) |
True Class |
||
p |
n |
||
Predicted Class |
T |
-1 |
|
F |
Calculate the expected cost of this model.
Expected cost =
- (6 points) Calculate the following metrics for this model:
Accuracy =
Sensitivity =
Specificity =
- (17 points) Now consider a second model that has managed to cutin half the false negatives from the original model, but at thesame time has also cut in half the true negatives. (Remember thatthere are 1040 instances in total.)
Perform the same calculations asabove.
Raw Counts (Model 2) |
True Class |
||
p |
n |
||
Predicted Class |
T |
||
F |
Expected Rates (Model 2) |
True Class |
||
p |
n |
||
Predicted Class |
T |
||
F |
Prob (p) = Prob (n) =
TP Rate = FP Rate =
FN Rate = TN Rate =
Expected cost (using the same costs/benefits) =
Accuracy =
Sensitivity =
Specificity =
- (4 points) Which model would you recommend to management, andwhy?
Expert Answer
Answer to we worked with this confusion matrix, representing the results of a model used to differentiate between fraudulent and …