Precision vs Recall in ML: The Ultimate Guide to Catching the Bad Guys (Without Annoying the Good Guys)

The Problem 

Imagine you are running a free health camp in rural Bihar. If your diagnostic test misses a single Tuberculosis (TB) patient, they go back home, infect their family, and face life-threatening risks; but if your test falsely flags a healthy person as sick, they suffer immense mental trauma and undergo painful, expensive, and unnecessary treatment.

How do you tune your machine learning model to balance these two catastrophic mistakes?

What is precision vs recall in ml? (And Why Should You Care?)

Before we look at any math, let’s stand at the gates of a high-security government building in New Delhi.

Think of your machine learning model as a security guard at the gate:

• The Paranoid Guard (High Recall): This guard’s absolute priority is to make sure no unauthorized trespasser slips inside. To achieve this, they search everyone thoroughly, check multiple IDs, and flag almost anyone who looks even slightly suspicious. They catch 100% of the bad guys. But in doing so, they also hold up dozens of genuine, honest employees in long queues, falsely accusing them of being suspicious.

• The Easygoing Guard (High Precision): This guard’s absolute priority is to never falsely accuse an honest employee. They only stop someone if they are absolutely, 100% certain that the person is a trespasser. Honest employees breeze through the gate with zero hassle. But because the guard is so cautious about making a false accusation, a few clever trespassers easily slip past the gate.

In Machine Learning:

• Precision is your model’s trustworthiness. When it claims someone is a “trespasser” (positive), how often is it actually correct?

• Recall is your model’s thoroughness. Out of all the actual “trespassers” in the crowd, how many did the model successfully catch?

Why Simpler Models Fail: The Accuracy Paradox —

Why can’t we just use a simple metric like Accuracy (the percentage of correct predictions)?

Imagine a rare financial fraud scenario in an Indian cooperative bank. Out of 10,000 daily transactions, only 10 are actually fraudulent. If you build an incredibly lazy “dummy” model that simply predicts “No Fraud” for every single transaction, what is its accuracy?

Accuracy = (9,990 correct predictions) / (10,000 total transactions) = 99.9%

This is the Accuracy Paradox. A completely useless model that catches zero fraud shows a spectacular 99.9% accuracy! This is why we need Precision and Recall. They force us to look directly at how we handle the rare, critical events instead of hiding behind a high accuracy score.

Where is precision vs recall in ml Used in Real Life?

Example 1: Loan Default Prediction at a Public Sector Bank

• X (Inputs): Customer’s monthly salary, CIBIL score, existing debts, and employment history.

• Y (Output): Will they default on their loan? (1 = Yes, 0 = No).

• Why it fits: If the bank prioritizes Precision, they only reject loan applications when they are absolutely certain of a default. This keeps genuine customers happy but risks giving loans to bad borrowers, leading to massive Non-Performing Assets (NPAs). If they prioritize Recall, they catch every potential defaulter, but they end up rejecting many honest applicants, losing massive business to competitors.

Example 2: Cyclone Landfall Prediction in Coastal Odisha

• X (Inputs): Wind speed, atmospheric pressure, sea surface temperature, and satellite cloud imagery.

• Y (Output): Will a severe cyclone make landfall at a specific coastal village? (1 = Yes, 0 = No).

• Why it fits: Here, a False Negative (predicting no cyclone when one actually hits) is a human catastrophe—lives will be lost because people weren’t evacuated. A False Positive (evacuating a village when no cyclone hits) is expensive and causes panic, but lives are saved. In this life-and-death scenario, we must tune our model for near-100% Recall, even if it means accepting lower Precision.

Example 3: Assembly Line Defect Detection for EV Batteries in Tamil Nadu

• X (Inputs): Thermal imaging scans, battery cell voltage stability, and laser weld measurements.

• Y (Output): Is the battery cell defective? (1 = Yes, 0 = No).

• Why it fits: If a defective lithium-ion battery slips through the assembly line (False Negative), it could catch fire in a customer’s electric scooter, destroying the brand’s reputation. If a good battery is flagged as defective (False Positive), it is sent to the scrap heap, wasting expensive raw materials. The manufacturer must carefully balance Precision and Recall to protect both customer safety and profit margins.

The Math Behind precision vs recall in ml —

To understand how these metrics are calculated, we must first look at the Confusion Matrix—the scoreboard of our model’s predictions:

Let’s write down the exact mathematical equations for our metrics:

Precision = TP / (TP + FP)

Recall = TP / (TP + FN)

F1 Score = 2 × (Precision × Recall) / (Precision + Recall)

Let’s break down these symbols one by one:

• TP (True Positives): The number of times the model correctly predicted the positive class (e.g., correctly identifying a defective battery).

• FP (False Positives): The “False Alarms”. The model predicted positive, but the actual ground truth was negative (e.g., flagging a perfectly healthy battery as defective).

• FN (False Negatives): The “Misses”. The model predicted negative, but the actual ground truth was positive (e.g., letting a defective battery slip through as healthy).

• TN (True Negatives): The number of times the model correctly predicted the negative class (e.g., correctly identifying a healthy battery as healthy).

• F1 Score: This is the harmonic mean of Precision and Recall. Why do we use a harmonic mean instead of a simple average?

Imagine a model with a Precision of 1.0 (perfect) and a Recall of 0.0 (it missed everything). A simple average would give us:

(1.0 + 0.0) / 2 = 0.5 (Looks okay on paper, but the model is actually useless!)

The harmonic mean, however, severely penalizes extreme values:

2 × (1.0 × 0.0) / (1.0 + 0.0) = 0.0 (Accurately reflects that the model is useless!)

The Critical Method: Decision-Threshold Tuning and the Tradeoff Curves

How do we actually control the balance between Precision and Recall?

Under the hood, classification algorithms (like Logistic Regression) do not output a direct “1” or “0“. Instead, they output a raw probability score between 0.0 and 1.0 (e.g., “There is an 82% chance this EV battery is defective”). To make a final decision, we apply a Decision Threshold (t):

Predicted Class = 1 if Probability ≥ t, else 0

By default, most machine learning libraries set t = 0.5. But we can tune this threshold to shift our model’s behavior step-by-step:

Step 1: Raising the Threshold (t → 0.9)

We tell the model: “Only flag a battery as defective if you are 90% or more certain.”

The Ripple Effect: The model becomes highly conservative. The number of False Alarms (FP) drops to near zero, causing Precision to shoot up. However, because the bar is so high, the model misses many borderline defective batteries. The number of Misses (FN) climbs, causing Recall to drop.

Step 2: Lowering the Threshold (t → 0.1)

We tell the model: “If there is even a tiny 10% suspicion that a battery is defective, flag it immediately!”

The Ripple Effect: The model becomes highly paranoid. It catches almost every single defective battery, so Misses (FN) drop to near zero, causing Recall to shoot up. However, it now flags many perfectly healthy batteries as defective. False Alarms (FP) skyrocket, causing Precision to plunge.

Step 3: Plotting the Tradeoff Curves

To find the perfect sweet spot, practitioners plot two curves across all possible thresholds from 0.0 to 1.0:

• The Precision-Recall (PR) Curve: We plot Precision on the Y-axis and Recall on the X-axis. A perfect model would have a curve that hugs the top-right corner (Area Under Curve, or AUC-PR = 1.0). We choose the threshold point on this curve that matches our business cost tolerance.

• The ROC Curve (Receiver Operating Characteristic): We plot the True Positive Rate (Recall) on the Y-axis against the False Positive Rate (FPR = FP / (FP + TN)) on the X-axis. The Area Under the ROC Curve (ROC-AUC) tells us how incredibly good our model is at separating the two classes, regardless of the threshold.

The Cost Function: Minimizing Real-World Consequences

While we train models using mathematical loss functions like Cross-Entropy, we evaluate and tune our final threshold using a Business Cost Function:

Total Cost = (C_FP × FP) + (C_FN × FN)

Where:

• C_FP is the financial or human cost of a single False Positive (e.g., the cost of scrapping a good EV battery cell = ₹500).

• C_FN is the cost of a single False Negative (e.g., the cost of a battery fire lawsuit and recall = ₹5,00,000).

“Minimizing” this cost function in human terms means finding the exact decision threshold t that results in the lowest possible total rupee loss for the factory.

Parameter Sensitivity Analysis

How precision vs recall in ml Actually Works ?

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation

# Set random seed for reproducibility
np.random.seed(42)

# =====================================================================
# 1. DATA CREATION (EV Battery Defect Detection in Tamil Nadu)
# =====================================================================
# We simulate 80 healthy batteries and 80 defective batteries.
# The model outputs a probability score of a battery being defective.
n_samples_class = 80

# Healthy batteries: centered around 0.30 probability of defect
scores_healthy = np.random.normal(loc=0.30, scale=0.12, size=n_samples_class)
# Defective batteries: centered around 0.70 probability of defect
scores_defective = np.random.normal(loc=0.70, scale=0.12, size=n_samples_class)

# Clip scores to keep them strictly within valid probability bounds [0.01, 0.99]
scores_healthy = np.clip(scores_healthy, 0.01, 0.99)
scores_defective = np.clip(scores_defective, 0.01, 0.99)

# Combine into a single dataset
all_scores = np.concatenate([scores_healthy, scores_defective])
all_labels = np.concatenate([np.zeros(n_samples_class), np.ones(n_samples_class)])

# Add vertical jitter to scatter points so they don’t overlap on a single line
jitter = np.random.uniform(-0.15, 0.15, size=len(all_scores))

# =====================================================================
# 2. PRECOMPUTING THE PRECISION-RECALL CURVE
# =====================================================================
# We sweep the decision threshold from 1.0 down to 0.0
thresholds = np.linspace(1.0, 0.0, 100)
ref_precisions = []
ref_recalls = []
ref_f1s = []

for t in thresholds:
preds = (all_scores >= t).astype(int)
tp = np.sum((preds == 1) & (all_labels == 1))
fp = np.sum((preds == 1) & (all_labels == 0))
fn = np.sum((preds == 0) & (all_labels == 1))

prec = tp / (tp + fp) if (tp + fp) > 0 else 1.0
rec = tp / (tp + fn) if (tp + fn) > 0 else 0.0
f1 = 2 * (prec * rec) / (prec + rec) if (prec + rec) > 0 else 0.0

ref_precisions.append(prec)
ref_recalls.append(rec)
ref_f1s.append(f1)

# Find the optimal threshold that maximizes the F1-Score (Convergence/Sweet Spot)
opt_idx = np.argmax(ref_f1s)
opt_thresh = thresholds[opt_idx]
opt_rec = ref_recalls[opt_idx]
opt_prec = ref_precisions[opt_idx]

# =====================================================================
# 3. MATPLOTLIB SETUP
# =====================================================================
fig, (ax_left, ax_right) = plt.subplots(1, 2, figsize=(15, 6.5))
fig.suptitle(“EV Battery Defect Detection (Tamil Nadu Factory)\nPrecision vs. Recall Tradeoff Live”,
fontsize=14, fontweight=’bold’, color=’#2c3e50′)

# Left Plot: Live Classification on Data
ax_left.set_xlim(-0.05, 1.05)
ax_left.set_ylim(-0.25, 0.25)
ax_left.set_xlabel(“Model Predicted Probability of Defect”, fontsize=11, fontweight=’bold’)
ax_left.set_ylabel(“Jittered Samples”, fontsize=11, fontweight=’bold’)
ax_left.axhline(0, color=’gray’, linestyle=’–‘, alpha=0.3)
ax_left.get_yaxis().set_ticks([]) # Hide Y-axis ticks for clean look

# Right Plot: Precision-Recall Curve
ax_right.set_xlim(-0.05, 1.05)
ax_right.set_ylim(-0.05, 1.05)
ax_right.set_xlabel(“Recall (Thoroughness: How many defects caught?)”, fontsize=11, fontweight=’bold’)
ax_right.set_ylabel(“Precision (Trustworthiness: How many flagged are actually defective?)”, fontsize=11, fontweight=’bold’)
ax_right.grid(True, linestyle=’:’, alpha=0.6)

# Plot the background reference curve path
ax_right.plot(ref_recalls, ref_precisions, color=’#bdc3c7′, linestyle=’–‘, linewidth=1.5, label=’Full PR Curve Path’)

# Annotate the optimal F1-Score convergence point
ax_right.annotate(‘Optimal F1 Sweet Spot\n(Balanced Risk)’,
xy=(opt_rec, opt_prec),
xytext=(opt_rec – 0.35, opt_prec – 0.25),
arrowprops=dict(facecolor=’#27ae60′, shrink=0.08, width=1.5, headwidth=8),
fontsize=10, fontweight=’bold’, color=’#27ae60′,
bbox=dict(boxstyle=”round,pad=0.3″, fc=”#e8f8f5″, ec=”#27ae60″, lw=1))

# Dynamic elements to update frame-by-frame
threshold_line = ax_left.axvline(x=1.0, color=’#2c3e50′, linestyle=’-‘, linewidth=2.5, label=’Decision Threshold’)

# Scatter plots for TP, FP, TN, FN
scatter_tp = ax_left.scatter([], [], color=’#e74c3c’, marker=’o’, s=60, edgecolors=’black’, label=’True Positive (Correct Defect)’)
scatter_fp = ax_left.scatter([], [], color=’#f39c12′, marker=’^’, s=60, edgecolors=’black’, label=’False Positive (False Alarm)’)
scatter_tn = ax_left.scatter([], [], color=’#2ecc71′, marker=’o’, s=60, edgecolors=’black’, label=’True Negative (Correct Healthy)’)
scatter_fn = ax_left.scatter([], [], color=’#3498db’, marker=’v’, s=60, edgecolors=’black’, label=’False Negative (Missed Defect)’)
ax_left.legend(loc=’upper left’, fontsize=8.5, framealpha=0.9)

# Right plot dynamic elements
pr_line, = ax_right.plot([], [], color=’#c0392b’, linewidth=3, label=’Current Path’)
current_point, = ax_right.plot([], [], color=’#e74c3c’, marker=’*’, markersize=14, markeredgecolor=’black’, label=’Current Threshold’)
ax_right.legend(loc=’lower left’, fontsize=9.5)

# Text boxes for live metrics
stats_text = ax_left.text(0.02, -0.22, “”, fontsize=10, family=’monospace’,
bbox=dict(boxstyle=”round,pad=0.5″, facecolor=’#f8f9f9′, edgecolor=’#bdc3c7′, alpha=0.9))
metric_text = ax_right.text(0.05, 0.12, “”, fontsize=10, family=’monospace’,
bbox=dict(boxstyle=”round,pad=0.5″, facecolor=’#f8f9f9′, edgecolor=’#bdc3c7′, alpha=0.9))

# =====================================================================
# 4. ANIMATION UPDATE FUNCTION
# =====================================================================
def update(frame):
t = thresholds[frame]

# Predictions at current threshold
preds = (all_scores >= t).astype(int)

# Identify indices for each confusion matrix quadrant
tp_mask = (preds == 1) & (all_labels == 1)
fp_mask = (preds == 1) & (all_labels == 0)
tn_mask = (preds == 0) & (all_labels == 0)
fn_mask = (preds == 0) & (all_labels == 1)

# Update scatter plots safely (handling empty masks)
if np.any(tp_mask):
scatter_tp.set_offsets(np.column_stack((all_scores[tp_mask], jitter[tp_mask])))
else:
scatter_tp.set_offsets(np.empty((0, 2)))

if np.any(fp_mask):
scatter_fp.set_offsets(np.column_stack((all_scores[fp_mask], jitter[fp_mask])))
else:
scatter_fp.set_offsets(np.empty((0, 2)))

if np.any(tn_mask):
scatter_tn.set_offsets(np.column_stack((all_scores[tn_mask], jitter[tn_mask])))
else:
scatter_tn.set_offsets(np.empty((0, 2)))

if np.any(fn_mask):
scatter_fn.set_offsets(np.column_stack((all_scores[fn_mask], jitter[fn_mask])))
else:
scatter_fn.set_offsets(np.empty((0, 2)))

# Update threshold line position
threshold_line.set_xdata([t, t])

# Calculate metrics
tp = np.sum(tp_mask)
fp = np.sum(fp_mask)
tn = np.sum(tn_mask)
fn = np.sum(fn_mask)

precision = ref_precisions[frame]
recall = ref_recalls[frame]
f1 = ref_f1s[frame]

# Update left plot text box
stats_text.set_text(f”CONFUSION MATRIX:\n—————–\nTP: {tp:2d} (Defects Caught)\nFP: {fp:2d} (False Alarms)\nFN: {fn:2d} (Missed Defects)\nTN: {tn:2d} (Healthy Saved)”)

# Update right plot path and current point
pr_line.set_data(ref_recalls[:frame+1], ref_precisions[:frame+1])
current_point.set_data([recall], [precision])

# Update right plot text box
metric_text.set_text(f”Threshold: {t:.2f}\nPrecision: {precision:.2f}\nRecall: {recall:.2f}\nF1-Score: {f1:.2f}”)

# Dynamic title narration based on the threshold phase
if t > 0.75:
ax_left.set_title(f”Threshold = {t:.2f} — Easygoing Guard (High Precision, Low Recall). F1 = {f1:.2f}”,
color=’#d35400′, fontsize=10.5, fontweight=’bold’)
elif 0.35 <= t <= 0.75:
if abs(t – opt_thresh) < 0.03:
ax_left.set_title(f”Threshold = {t:.2f} — Converged! Optimal Balance. F1 = {f1:.2f}”,
color=’#27ae60′, fontsize=10.5, fontweight=’bold’)
else:
ax_left.set_title(f”Threshold = {t:.2f} — Balancing Act… F1 = {f1:.2f}”,
color=’#2980b9′, fontsize=10.5, fontweight=’bold’)
else:
ax_left.set_title(f”Threshold = {t:.2f} — Paranoid Guard (High Recall, Low Precision). F1 = {f1:.2f}”,
color=’#c0392b’, fontsize=10.5, fontweight=’bold’)

return scatter_tp, scatter_fp, scatter_tn, scatter_fn, threshold_line, pr_line, current_point, stats_text, metric_text

# Create the animation
ani = FuncAnimation(fig, update, frames=len(thresholds), interval=120, blit=True)
plt.tight_layout()

# Save the animation as a high-quality GIF
ani.save(‘algo_working_precision_vs_recall_in_ml.gif’, writer=’pillow’, fps=10)

============================================================
EV BATTERY DEFECT DETECTION - TAMIL NADU FACTORY
============================================================

[1] Generating synthetic battery data...
   Total samples: 1000
   Healthy (0): 850
   Defective (1): 150

[2] Splitting data into Train (80%) and Test (20%)...
   Train size: 800
   Test size: 200

[3] Scaling features...
   Scaling complete! (mean=0, variance=1)

[4] Training models...
   • Logistic Regression...
   • Random Forest (100 trees)...
   • SVM (RBF Kernel)...
   All models trained successfully!

[5] Applying safety-first threshold (t = 0.25)...

[6] Calculating performance metrics...

======================================================================
MODEL PERFORMANCE COMPARISON (Threshold = 0.25)
======================================================================
              Model Accuracy Precision Recall F1-Score
Logistic Regression   0.9900    0.9375 1.0000   0.9677
      Random Forest   0.9950    0.9677 1.0000   0.9836
                SVM   1.0000    1.0000 1.0000   1.0000
======================================================================

🏆 Best Recall: Logistic Regression (1.0000)
🏆 Best F1-Score: SVM (1.0000)

📊 Confusion Matrix for Logistic Regression:
                 Predicted
              Healthy  Defective
Actual Healthy   168         2
       Defective   0        30

[7] Generating comparison plot...
   ✅ Plot saved as 'model_comparison_precision_vs_recall_in_ml.png'

[8] Generating Precision-Recall curve...
   ✅ Plot saved as 'precision_recall_curves_comparison.png'

======================================================================
📋 FINAL SUMMARY
======================================================================

🔍 KEY FINDINGS:
• Most Accurate Model: SVM (1.0000)
• Highest Precision: SVM (1.0000)
• Highest Recall: Logistic Regression (1.0000)
• Best Balanced (F1): SVM (1.0000)

💡 RECOMMENDATION:
For EV battery defect detection, we prioritize RECALL (catching all defects)
over Precision (avoiding false alarms). A missed defective battery can 
cause fires and brand damage costing crores of rupees.

✅ Recommended Model: Logistic Regression
   (Threshold = 0.25)

======================================================================
✅ COMPLETE! All analyses and plots generated successfully.
======================================================================