Session 16: Fraud Detection Using ML

1. The Class Imbalance Problem

Class imbalance is the defining technical challenge of fraud detection in insurance. When only 2–5% of claims are fraudulent, a model that achieves 95% accuracy by simply predicting "not fraud" for every claim is useless — yet its accuracy metric suggests it is excellent. Understanding why this happens, how to measure model performance correctly, and what to do about it is the foundation of ML-based fraud detection.

1.1 Why Accuracy Fails

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier, IsolationForest
from sklearn.metrics import (accuracy_score, precision_score, recall_score, f1_score,
                             roc_auc_score, precision_recall_curve, average_precision_score,
                             confusion_matrix, classification_report)
from sklearn.preprocessing import StandardScaler
from imblearn.over_sampling import SMOTE
import xgboost as xgb
import matplotlib.pyplot as plt

# Load data
df = pd.read_csv('data/insurance_cleaned.csv')
print(f"Dataset shape: {df.shape}")
print(f"Fraud rate: {df['fraud_flag'].mean()*100:.2f}%")

# Create a naive model that predicts "not fraud" for everything
naive_pred = np.zeros(len(df))
naive_accuracy = accuracy_score(df['fraud_flag'], naive_pred)

print(f"\nNaive model (predicts 'not fraud' for ALL claims):")
print(f"  Accuracy: {naive_accuracy*100:.2f}%")
print(f"  But recall on fraud: 0.0% — it catches ZERO fraudulent claims.")
print(f"\n{'=' * 60}")
print(f"KEY INSIGHT: Accuracy is a misleading metric for fraud detection.")
print(f"When fraud rate is {df['fraud_flag'].mean()*100:.1f}%, a do-nothing model")
print(f"achieves {naive_accuracy*100:.1f}% accuracy. Always evaluate fraud models")
print(f"using precision, recall, F1-score, and precision-recall AUC.")
print(f"{'=' * 60}")

1.2 The Right Metrics for Imbalanced Fraud Data

2. Handling Imbalanced Data

Three main strategies exist for handling class imbalance in fraud detection: data-level (resampling — SMOTE), algorithm-level (class weighting), and cost-sensitive (threshold tuning). The best fraud detection systems use all three in combination.

2.1 Preparing the Fraud Dataset

# Create fraud detection dataset — one row per claim with features and target
fraud_features = ['premium', 'claim_amount', 'age', 'income', 'credit_score',
                  'sum_assured', 'policy_type', 'claim_type',
                  'days_to_settle' if 'days_to_settle' in df.columns else None]
fraud_features = [c for c in fraud_features if c is not None and c in df.columns]

# Clean data
fraud_data = df[fraud_features + ['fraud_flag']].dropna().copy()

X_fraud = fraud_data[fraud_features]
y_fraud = fraud_data['fraud_flag']

print(f"Fraud dataset: {len(fraud_data):,} claims, {X_fraud.shape[1]} features")
print(f"Class distribution:")
print(f"  Legitimate (0): {(y_fraud == 0).sum():>8,d} ({(y_fraud == 0).mean()*100:.2f}%)")
print(f"  Fraud (1):      {(y_fraud == 1).sum():>8,d} ({(y_fraud == 1).mean()*100:.2f}%)")
print(f"  Imbalance ratio: {(y_fraud == 0).sum() / (y_fraud == 1).sum():.1f}:1")

# Train/test split (stratified — preserves class proportions)
X_train_f, X_test_f, y_train_f, y_test_f = train_test_split(
    X_fraud, y_fraud, test_size=0.3, random_state=42, stratify=y_fraud
)

cat_cols = X_fraud.select_dtypes(include=['object', 'category']).columns.tolist()
num_cols = X_fraud.select_dtypes(include=[np.number]).columns.tolist()

2.2 SMOTE — Synthetic Minority Oversampling

SMOTE generates synthetic fraud examples by interpolating between existing fraud cases in feature space. For each fraud case, SMOTE selects its K nearest neighbours (also fraud cases) and creates a new synthetic case at a random point along the line connecting them. This is more effective than simple duplication (oversampling without replacement leads to overfitting) because it creates new examples that the model has not seen.

from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.compose import ColumnTransformer

# Preprocessing
preprocessor = ColumnTransformer(
    transformers=[
        ('num', StandardScaler(), num_cols),
        ('cat', OneHotEncoder(drop='first', handle_unknown='ignore', sparse_output=False), cat_cols)
    ])

# Transform the data
X_train_transformed = preprocessor.fit_transform(X_train_f)
X_test_transformed = preprocessor.transform(X_test_f)

# Feature names for later interpretation
cat_feature_names = []
if len(cat_cols) > 0:
    cat_encoder = preprocessor.named_transformers_['cat']
    cat_feature_names = cat_encoder.get_feature_names_out(cat_cols).tolist()
all_feature_names = num_cols + cat_feature_names

print(f"Before SMOTE — Training set shape: {X_train_transformed.shape}")
print(f"  Class 0 (legitimate): {(y_train_f == 0).sum():,}")
print(f"  Class 1 (fraud):      {(y_train_f == 1).sum():,}")

# Apply SMOTE
smote = SMOTE(random_state=42, k_neighbors=5)
X_train_smote, y_train_smote = smote.fit_resample(X_train_transformed, y_train_f)

print(f"\nAfter SMOTE — Training set shape: {X_train_smote.shape}")
print(f"  Class 0 (legitimate): {(y_train_smote == 0).sum():,}")
print(f"  Class 1 (fraud):      {(y_train_smote == 1).sum():,}")
print(f"Perfectly balanced training set — SMOTE has generated synthetic fraud examples.")

2.3 Alternative: Class Weights

Instead of resampling, we can adjust the algorithm to penalise misclassifications of the minority class more heavily. Most Scikit-learn and XGBoost models support a `class_weight` parameter:

# Random Forest with class_weight='balanced' — automatically adjusts weights
rf_weighted = RandomForestClassifier(
    n_estimators=200, max_depth=10, min_samples_leaf=20,
    class_weight='balanced',  # Adjusts weights inversely proportional to class frequencies
    random_state=42, n_jobs=-1
)

# XGBoost with scale_pos_weight — adjusts weight for positive class
fraud_ratio = (y_train_f == 0).sum() / (y_train_f == 1).sum()
xgb_weighted = xgb.XGBClassifier(
    n_estimators=200, max_depth=6, learning_rate=0.1,
    scale_pos_weight=fraud_ratio,  # scale_pos_weight = negative/positive
    subsample=0.8, colsample_bytree=0.8,
    random_state=42, n_jobs=-1, eval_metric='logloss'
)

print(f"XGBoost scale_pos_weight set to {fraud_ratio:.1f}:1")
print(f"This tells XGBoost: 'misclassifying a fraud case is {fraud_ratio:.0f}×")
print(f"more costly than misclassifying a legitimate case.'")

3. Anomaly Detection — Isolation Forest

Isolation Forest is an unsupervised anomaly detection algorithm that is particularly effective for fraud detection because: (a) it does not need labelled fraud data (most insurers have very few confirmed fraud labels), (b) it identifies claims that are "different" from the majority, which may be novel fraud patterns that have not been seen before, and (c) it is fast and scales to millions of claims.

3.1 How Isolation Forest Works

The algorithm works by randomly selecting a feature and a split value, and recursively partitioning the data into smaller regions. Anomalous claims — those that are rare and have unusual feature combinations — are isolated in fewer splits than normal claims. In the visual metaphor: isolating a legitimate claim is like finding a needle in a haystack; isolating a fraudulent claim is like finding a needle that is already sitting on top of the haystack. The anomaly score is derived from the average number of splits required to isolate each observation across many random trees.

3.2 Training Isolation Forest

# Isolation Forest for unsupervised fraud detection
# Uses ONLY legitimate claims for training (or the full dataset — it's unsupervised)
from sklearn.ensemble import IsolationForest

# Set contamination = expected fraud rate (or a conservative estimate)
isolation_forest = IsolationForest(
    n_estimators=200,
    max_samples=0.8,        # Use 80% of data per tree
    contamination=0.05,     # Expected fraud rate — affects threshold
    random_state=42,
    n_jobs=-1
)

# Train on the full training data (unsupervised — does not use labels)
isolation_forest.fit(X_train_transformed)

# Predict anomaly scores and labels
# predict(): 1 = normal, -1 = anomaly
if_pred = isolation_forest.predict(X_test_transformed)
if_scores = isolation_forest.score_samples(X_test_transformed)  # Higher = more normal

# Convert to fraud-like format: higher score = more anomalous
if_anomaly_score = -if_scores  # Flip so higher = more suspicious

# Normalise to 0-100
if_fraud_score = (if_anomaly_score - if_anomaly_score.min()) / (if_anomaly_score.max() - if_anomaly_score.min()) * 100

# Convert predictions: -1 (anomaly) → 1 (fraud), 1 (normal) → 0 (legit)
if_pred_binary = (if_pred == -1).astype(int)

# Evaluate
if_precision = precision_score(y_test_f, if_pred_binary, zero_division=0)
if_recall = recall_score(y_test_f, if_pred_binary, zero_division=0)
if_f1 = f1_score(y_test_f, if_pred_binary, zero_division=0)
if_pr_auc = average_precision_score(y_test_f, if_scores)  # Note: IF scores are inverted

print("=" * 60)
print("ISOLATION FOREST — UNSUPERVISED FRAUD DETECTION")
print("=" * 60)
print(f"Precision: {if_precision:.3f}")
print(f"Recall:    {if_recall:.3f}")
print(f"F1 Score:  {if_f1:.3f}")
print(f"\nInterpretation:")
print(f"  Precision = {if_precision:.1%} — {'Good. Most flagged claims are legitimate only if this is below fraud rate.' if if_precision > 0.3 else 'Low — many false positives.'}")
print(f"  Recall = {if_recall:.1%} — {'Catches a significant portion of fraud.' if if_recall > 0.5 else 'Misses most fraud cases. May need tighter contamination parameter.'}")

# Decile analysis for IF
if_score_df = pd.DataFrame({'score': if_fraud_score, 'actual': y_test_f.values})
if_score_df['decile'] = pd.qcut(if_score_df['score'], q=10, labels=False, duplicates='drop')
if_decile = if_score_df.groupby('decile').agg(
    count=('score', 'count'),
    fraud=('actual', 'sum'),
    fraud_rate=('actual', 'mean'),
    avg_score=('score', 'mean')
)

total_fraud_if = if_decile['fraud'].sum()
print(f"\nDecile Analysis (top decile = highest anomaly score):")
print(f"{'Decile':8s} {'Claims':>10s} {'Fraud':>8s} {'Rate':>8s} {'Cum. Fraud':>15s}")
cum_fraud = 0
for idx in reversed(range(len(if_decile))):
    r = if_decile.loc[idx]
    cum_fraud += r['fraud']
    cum_pct = cum_fraud / total_fraud_if * 100
    print(f"Top {10-idx*10 if idx < 9 else idx*10}-{10-idx*10-10 if idx < 9 else 100}%  {r['count']:>6,.0f}  {r['fraud']:>4,.0f}  {r['fraud_rate']*100:>4.1f}%  {cum_fraud:>4,.0f} ({cum_pct:.0f}%)")

4. Supervised Fraud Classification — XGBoost

XGBoost is the most widely used supervised algorithm for fraud detection in insurance. Its combination of handling missing values, class weighting, regularisation, and scalability makes it the default choice when labelled fraud data is available.

4.1 Training XGBoost for Fraud Detection

# XGBoost with SMOTE-balanced training data
xgb_model = xgb.XGBClassifier(
    n_estimators=300,
    max_depth=5,
    learning_rate=0.05,
    subsample=0.8,
    colsample_bytree=0.8,
    reg_alpha=0.1,
    reg_lambda=1.0,
    random_state=42,
    n_jobs=-1,
    eval_metric='logloss'
)

# Train on SMOTE-balanced data
xgb_model.fit(
    X_train_smote, y_train_smote,
    eval_set=[(X_test_transformed, y_test_f)],
    verbose=False
)

# Predict
xgb_prob = xgb_model.predict_proba(X_test_transformed)[:, 1]
xgb_pred = (xgb_prob >= 0.5).astype(int)

# Evaluate
xgb_precision = precision_score(y_test_f, xgb_pred)
xgb_recall = recall_score(y_test_f, xgb_pred)
xgb_f1 = f1_score(y_test_f, xgb_pred)
xgb_roc_auc = roc_auc_score(y_test_f, xgb_prob)
xgb_pr_auc = average_precision_score(y_test_f, xgb_prob)

print("=" * 60)
print("XGBoost — SUPERVISED FRAUD DETECTION (SMOTE + Default Threshold)")
print("=" * 60)
print(f"Precision: {xgb_precision:.3f}")
print(f"Recall:    {xgb_recall:.3f}")
print(f"F1 Score:  {xgb_f1:.3f}")
print(f"ROC-AUC:   {xgb_roc_auc:.3f}")
print(f"PR-AUC:    {xgb_pr_auc:.3f}")

# Confusion matrix
cm = confusion_matrix(y_test_f, xgb_pred)
print(f"\nConfusion Matrix (threshold = 0.5):")
print(f"{'':25s} {'Pred Legit':12s} {'Pred Fraud':12s}")
print(f"{'Actual Legit':25s} {cm[0,0]:>6,d}        {cm[0,1]:>6,d}")
print(f"{'Actual Fraud':25s} {cm[1,0]:>6,d}        {cm[1,1]:>6,d}")
print(f"\nAt threshold 0.5:")
print(f"  False positives: {cm[0,1]:,} legitimate claims flagged for investigation")
print(f"  Missed fraud:    {cm[1,0]:,} fraudulent claims paid (no investigation)")

4.2 Threshold Optimisation for Fraud

The default threshold of 0.5 is almost never optimal for fraud detection. We want to find the threshold that maximises business value — which depends on the relative cost of a false positive (investigation cost) vs. a false negative (fraud paid).

# Find the optimal threshold using precision-recall tradeoff
precisions, recalls, thresholds = precision_recall_curve(y_test_f, xgb_prob)

# Cost-based threshold optimisation
fp_cost = 500    # Cost of investigating a false positive (₹)
fn_cost = 75000  # Average cost of a paid fraudulent claim (₹)
# Average fraud claim is typically higher than legitimate

# Calculate total cost at each threshold
total_costs = []
valid_thresholds = []

for i in range(len(thresholds)):
    pred = (xgb_prob >= thresholds[i]).astype(int)
    cm = confusion_matrix(y_test_f, pred)
    fp = cm[0, 1]  # False positives
    fn = cm[1, 0]  # False negatives
    total_cost = fp * fp_cost + fn * fn_cost
    total_costs.append(total_cost)
    valid_thresholds.append(thresholds[i])

optimal_idx = np.argmin(total_costs)
optimal_threshold = valid_thresholds[optimal_idx]
optimal_cost = total_costs[optimal_idx]

# Calculate cost at default threshold
pred_default = (xgb_prob >= 0.5).astype(int)
cm_default = confusion_matrix(y_test_f, pred_default)
cost_default = cm_default[0, 1] * fp_cost + cm_default[1, 0] * fn_cost

print("=" * 65)
print("THRESHOLD OPTIMISATION — COST-BASED ANALYSIS")
print("=" * 65)
print(f"False positive cost (investigation): ₹{fp_cost:,}")
print(f"False negative cost (fraud paid):    ₹{fn_cost:,}")
print(f""
      f"")
print(f"Default threshold (0.5):")
print(f"  FP = {cm_default[0,1]:,} @ ₹{fp_cost:,} = ₹{cm_default[0,1]*fp_cost:,.0f}")
print(f"  FN = {cm_default[1,0]:,} @ ₹{fn_cost:,} = ₹{cm_default[1,0]*fn_cost:,.0f}")
print(f"  Total cost: ₹{cost_default:,.0f}")
print(f"")
print(f"Optimal threshold ({optimal_threshold:.3f}):")
pred_opt = (xgb_prob >= optimal_threshold).astype(int)
cm_opt = confusion_matrix(y_test_f, pred_opt)
fp_opt = cm_opt[0, 1]
fn_opt = cm_opt[1, 0]
print(f"  FP = {fp_opt:,} @ ₹{fp_cost:,} = ₹{fp_opt*fp_cost:,.0f}")
print(f"  FN = {fn_opt:,} @ ₹{fn_cost:,} = ₹{fn_opt*fn_cost:,.0f}")
print(f"  Total cost: ₹{optimal_cost:,.0f}")
print(f"  Savings vs. default: ₹{cost_default - optimal_cost:,.0f}")

4.3 Feature Importance for Fraud

importances = xgb_model.feature_importances_
fi_df = pd.DataFrame({
    'feature': all_feature_names[:len(importances)],
    'importance': importances
}).sort_values('importance', ascending=True)

fig, ax = plt.subplots(figsize=(10, max(5, len(fi_df.tail(10))*0.35)))
ax.barh(fi_df.tail(10)['feature'], fi_df.tail(10)['importance'],
        color='#e17055', edgecolor='white')
ax.set_xlabel('Feature Importance (XGBoost)')
ax.set_title('Top 10 Fraud Detection Features', fontweight='bold')
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
plt.tight_layout()
plt.show()

print("\nTop fraud indicators from ML model:")
for _, row in fi_df.sort_values('importance', ascending=False).head(5).iterrows():
    print(f"  {row['feature']:30s}: {row['importance']:.4f}")

5. Model Comparison & Ensemble

No single model is best for all fraud detection scenarios. Supervised models (XGBoost, Random Forest) are excellent at catching known fraud patterns but cannot detect novel patterns they have not been trained on. Unsupervised models (Isolation Forest) can detect novel anomalies but have higher false positive rates. The best approach is to combine them into an ensemble that leverages the strengths of both.

5.1 Direct Comparison

# Train Random Forest for comparison
rf_fraud = RandomForestClassifier(
    n_estimators=200, max_depth=10, min_samples_leaf=20,
    class_weight='balanced', random_state=42, n_jobs=-1
)
rf_fraud.fit(X_train_smote, y_train_smote)
rf_prob = rf_fraud.predict_proba(X_test_transformed)[:, 1]

rf_pr_auc = average_precision_score(y_test_f, rf_prob)
rf_roc_auc = roc_auc_score(y_test_f, rf_prob)

print("=" * 60)
print("MODEL COMPARISON — FRAUD DETECTION")
print("=" * 60)
print(f"{'Model':25s} {'PR-AUC':>10s} {'ROC-AUC':>10s} {'Precision':>10s} {'Recall':>8s} {'F1':>8s}")
print("-" * 71)

models = {
    'Isolation Forest (Unsupervised)': if_pr_auc,
    'Random Forest + SMOTE': rf_pr_auc,
    'XGBoost + SMOTE (Optimal)': xgb_pr_auc
}

# Get metrics for each
for name, metric in models.items():
    if 'Isolation' in name:
        p, r, f = if_precision, if_recall, if_f1
        roc = 0
    elif 'Random' in name:
        p = precision_score(y_test_f, (rf_prob >= 0.5).astype(int))
        r = recall_score(y_test_f, (rf_prob >= 0.5).astype(int))
        f = f1_score(y_test_f, (rf_prob >= 0.5).astype(int))
        roc = rf_roc_auc
    else:
        p = precision_score(y_test_f, (xgb_prob >= optimal_threshold).astype(int))
        r = recall_score(y_test_f, (xgb_prob >= optimal_threshold).astype(int))
        f = f1_score(y_test_f, (xgb_prob >= optimal_threshold).astype(int))
        roc = xgb_roc_auc

    print(f"{name:25s} {metric:>8.3f}   {roc:>7.3f}   {p:>7.3f}   {r:>7.3f}  {f:>7.3f}")

print(f"\n{'─' * 71}")
print(f"Best model: {'XGBoost' if xgb_pr_auc >= rf_pr_auc else 'Random Forest'} based on PR-AUC.")
print(f"Isolation Forest is {'competitive' if if_pr_auc > 0.15 else 'useful as a complement but not standalone'}.")
print(f"\nRecommendation: {'Use XGBoost as primary + IF as secondary for novel fraud detection.' if xgb_pr_auc > 0.20 else 'Need more labelled data for supervised models to be effective.'}")

5.2 Building the Ensemble — Combined Fraud Score

# Combine XGBoost and Isolation Forest into an ensemble score
# Weight: 70% XGBoost, 30% Isolation Forest (weights can be tuned)

# Normalise both scores to 0-100
xgb_score_norm = xgb_prob * 100
if_score_norm = if_fraud_score  # Already 0-100

# Weighted ensemble
ensemble_weight_xgb = 0.7
ensemble_weight_if = 0.3

ensemble_score = (xgb_score_norm * ensemble_weight_xgb +
                  if_score_norm * ensemble_weight_if)

# Cap at 100
ensemble_score = np.clip(ensemble_score, 0, 100)

# Evaluate ensemble
ensemble_pr_auc = average_precision_score(y_test_f, ensemble_score / 100)

# Decile analysis
ens_df = pd.DataFrame({'score': ensemble_score, 'actual': y_test_f.values})
ens_df['decile'] = pd.qcut(ens_df['score'], q=10, labels=False, duplicates='drop')

ens_decile = ens_df.groupby('decile').agg(
    count=('score', 'count'),
    fraud=('actual', 'sum'),
    fraud_rate=('actual', 'mean')
)

total_fraud_ens = ens_decile['fraud'].sum()
print(f"ENSEMBLE MODEL (XGBoost {ensemble_weight_xgb*100:.0f}% + Isolation Forest {ensemble_weight_if*100:.0f}%)")
print(f"Ensemble PR-AUC: {ensemble_pr_auc:.3f} {'(Improved)' if ensemble_pr_auc > max(xgb_pr_auc, if_pr_auc) else '(Similar to best individual model)'}")
print(f"\nDecile Analysis:")
print(f"{'Decile':8s} {'Claims':>10s} {'Fraud':>8s} {'Rate':>8s} {'Cum. Fraud Catch':>15s}")
cum = 0
for idx in reversed(range(len(ens_decile))):
    r = ens_decile.loc[idx]
    cum += r['fraud']
    cum_pct = cum / total_fraud_ens * 100
    print(f"Top {10 - idx*10 if idx < 9 else idx*10}%   {r['count']:>5,.0f}  {r['fraud']:>4,.0f}  {r['fraud_rate']*100:>4.1f}%  {cum:>4,.0f} ({cum_pct:.0f}%)")

print(f"\nThe top 20% of claims by ensemble score capture approximately")
print(f"{ens_decile.iloc[9]['fraud'] + ens_decile.iloc[8]['fraud']} of {total_fraud_ens:.0f} total fraud cases.")
print(f"This means investigating 20% of claims catches ~{(ens_decile.iloc[9]['fraud'] + ens_decile.iloc[8]['fraud'])/total_fraud_ens*100:.0f}% of fraud.")

5.3 Precision-Recall Curves — Visual Comparison

from sklearn.metrics import precision_recall_curve

# Get PR curves for all models
rf_prec, rf_rec, _ = precision_recall_curve(y_test_f, rf_prob)
xgb_prec, xgb_rec, _ = precision_recall_curve(y_test_f, xgb_prob)
if_prec_values, if_rec_values, _ = precision_recall_curve(y_test_f, -if_scores)
ens_prec, ens_rec, _ = precision_recall_curve(y_test_f, ensemble_score / 100)

fig, ax = plt.subplots(figsize=(10, 7))
ax.plot(rf_rec, rf_prec, linewidth=2, label=f'Random Forest (PR-AUC: {rf_pr_auc:.3f})')
ax.plot(xgb_rec, xgb_prec, linewidth=2, label=f'XGBoost (PR-AUC: {xgb_pr_auc:.3f})')
ax.plot(if_rec_values, if_prec_values, linewidth=2, label=f'Isolation Forest')
ax.plot(ens_rec, ens_prec, linewidth=2, linestyle='--', label=f'Ensemble (PR-AUC: {ensemble_pr_auc:.3f})')
ax.set_xlabel('Recall (True Positive Rate)')
ax.set_ylabel('Precision')
ax.set_title('Precision-Recall Curves — Fraud Detection Models', fontweight='bold')
ax.legend(loc='lower left')
ax.grid(True, alpha=0.3)
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
plt.tight_layout()
plt.show()

print("Interpretation:")
print("- Upper-right curves are better (high precision AND high recall)")
print("- The Ensemble curve should dominate (be above) individual model curves")
print("- The area under each curve is the PR-AUC — the single best metric for fraud")

6. KNIME for Fraud Analytics

KNIME Analytics Platform is a visual, no-code/low-code environment for data science that is widely used in insurance fraud analytics. In many Indian insurers, the fraud analytics team may include analysts who are proficient in KNIME but not Python. Building a fraud detection workflow in KNIME serves two purposes: (a) it makes fraud analytics accessible to a broader team, and (b) it provides a visual audit trail that is valuable for compliance — every step from data loading to model scoring is visible and documented.

6.1 The KNIME Fraud Detection Workflow

KNIME WORKFLOW: FRAUD_DETECTION_V1

┌────────────────────────────┐
│  CSV Reader                │
│  (insurance_cleaned.csv)   │
└─────────────┬──────────────┘
              │
              ▼
┌────────────────────────────┐
│  Column Filter             │
│  (Select features for      │
│   fraud detection)         │
└─────────────┬──────────────┘
              │
              ▼
┌────────────────────────────┐
│  Partitioning              │
│  (70% Train / 30% Test)   │
│  Stratified by fraud_flag  │
└─────────────┬──────────────┘
              │
    ┌─────────┴─────────┐
    │                   │
    ▼                   ▼
┌──────────────┐  ┌──────────────┐
│ SMOTE        │  │ (Test data   │
│ (Balance     │  │  — no SMOTE) │
│  training)   │  │              │
└──────┬───────┘  └──────┬───────┘
       │                  │
       ▼                  │
┌──────────────┐          │
│ XGBoost      │          │
│ Learner      │          │
│ (scale_pos = │          │
│  weight)     │          │
└──────┬───────┘          │
       │                  │
       ▼                  ▼
┌────────────────────────────┐
│  XGBoost Predictor         │
│  (Score test data)         │
└─────────────┬──────────────┘
              │
              ▼
┌────────────────────────────┐
│  Scorer                    │
│  (Accuracy, Precision,    │
│   Recall, F1, Confusion   │
│   Matrix, Cohen's Kappa)  │
└─────────────┬──────────────┘
              │
              ▼
┌────────────────────────────┐
│  CSV Writer                │
│  (Scored claims with      │
│   fraud probability)       │
└────────────────────────────┘

ADDITIONAL BRANCH (parallel):
┌────────────────────────────┐
│  Isolation Forest          │
│  (on same partitioned     │
│   training data)           │
└─────────────┬──────────────┘
              │
              ▼
┌────────────────────────────┐
│  Predictor                 │
│  (Anomaly scores)          │
└─────────────┬──────────────┘
              │
              ▼
┌────────────────────────────┐
│  Joiner                    │
│  (Merge XGBoost + IF      │
│   scores)                  │
└─────────────┬──────────────┘
              │
              ▼
┌────────────────────────────┐
│  Math Formula              │
│  Ensemble = 0.7×XGB +     │
│  0.3×IF                    │
└─────────────┬──────────────┘
              │
              ▼
┌────────────────────────────┐
│  Interactive Table         │
│  (View scored claims,     │
│   filter by threshold)    │
└────────────────────────────┘

6.2 KNIME vs. Python for Fraud — When to Use Which