Module 07: Classification - Participation Exercises

Open In Colab

Module 07: Classification - Participation Exercises#

Exercise 7.1: Discussion - Choosing Metrics#

Type: đź’¬ Discussion (5 min)

You’re building a classifier to detect faulty batches in a chemical plant. Only 2% of batches are faulty.

Discuss with a partner:

  1. If you predict “good” for every batch, what’s your accuracy?

  2. Why is accuracy a bad metric here?

  3. Which metric would you use instead: precision, recall, or F1? Why?

Discussion notes:

Exercise 7.2: Mini-Exercise - Confusion Matrix Interpretation#

Type: 🔧 Mini-Exercise (6 min)

Calculate metrics from a confusion matrix.

# A classifier for detecting equipment failures
# Confusion matrix:
#                 Predicted
#              | Normal | Failure |
# Actual Normal |   85   |   10    |
# Actual Failure|    3   |    2    |

# True Positives (TP) = correctly predicted failures = 2
# True Negatives (TN) = correctly predicted normal = 85
# False Positives (FP) = normal predicted as failure = 10
# False Negatives (FN) = failure predicted as normal = 3

TP, TN, FP, FN = 2, 85, 10, 3

# TASK: Calculate these metrics
# accuracy = ???
# precision = ???  (of predicted failures, how many were real?)
# recall = ???  (of actual failures, how many did we catch?)
# f1 = ???

# Which metric is most concerning? Why?

Your calculations and interpretation:

Exercise 7.3: Prediction - ROC Curves#

Type: đź”® Prediction (3 min)

You have two classifiers:

  • Model A: High precision (0.9), low recall (0.3)

  • Model B: Low precision (0.5), high recall (0.9)

Predict: Sketch (mentally or on paper) where each model’s operating point would be on an ROC curve. Which model has higher AUC?

Your prediction:

Contents