Session 15: Insurance Fraud Analytics

1. The Fraud Problem in Insurance

Insurance fraud is estimated to cost the global insurance industry $80–$120 billion annually. In India, conservative estimates place annual fraud losses at ₹45,000 crore or more — approximately 8–12% of total claims spend. Frauds are not a tax that must be accepted; they are a cost that can be aggressively reduced through analytics, and every rupee saved on fraud flows directly to the bottom line.

1.1 Why Insurance is Vulnerable to Fraud

Insurance has structural characteristics that make it uniquely vulnerable to fraud — vulnerabilities that fraudsters exploit systematically:

• Information asymmetry: The policyholder knows more about their risk, their behaviour, and the circumstances of a loss than the insurer does. Fraud exploits this information gap — the insurer cannot independently verify every detail of every claim.
• Volume pressure: A typical Indian general insurer processes thousands of claims per day. Fraud detection must work at scale — most fraud is hidden in high-volume, low-value claims where manual review is impractical.
• Payment timing: Insurers are expected to pay legitimate claims quickly. Fraudsters exploit the pressure to settle fast — especially in lines where customer satisfaction is measured by settlement speed (motor, health, travel).
• Low detection, low consequence: Studies suggest that only 10–20% of insurance fraud is detected. Even when detected, prosecution rates are low — insurers often settle rather than litigate because the cost of prosecution exceeds the claim value. This creates a low-risk, high-reward environment for fraudsters.

2. Fraud Typology & Red Flags

Insurance fraud falls into three broad categories. Each has different perpetrators, detection methods, and economic characteristics. An effective anti-fraud programme must address all three.

2.1 The Three Types of Insurance Fraud

2.2 The 25+ Red Flags of Insurance Fraud

The following red flags, when present individually, are not conclusive evidence of fraud — but when multiple flags appear on the same claim or policy, the probability of fraud increases significantly.

3. The Fraud Analytics Framework

Fraud analytics operates on a continuous three-stage cycle: Detection → Investigation → Prevention. Each stage feeds into the next — detection identifies suspicious claims, investigation confirms or clears them, and the lessons from investigation are fed back into improved detection. The analytics team that masters this cycle is more effective than the one that builds the most sophisticated model in isolation.

3.1 The Detection → Investigation → Prevention Cycle

┌─────────────────────────────────────────────────────────────────────┐
│                    FRAUD ANALYTICS CYCLE                            │
│                                                                     │
│                          ┌──────────────┐                           │
│                          │  DETECTION   │                           │
│                          │  • Rules     │                           │
│                          │  • ML Models │                           │
│                          │  • Networks  │                           │
│                          │  • Anomalies │                           │
│                          └──────┬───────┘                           │
│                                 │  Suspicious claims flagged       │
│                                 ▼                                   │
│                          ┌──────────────┐                           │
│                     ┌───│ INVESTIGATION│───┐                       │
│                     │   │ • Triage     │   │                       │
│                     │   │ • Field inv. │   │                       │
│                     │   │ • Data check │   │                       │
│                     │   │ • Resolution │   │                       │
│                     │   └──────┬───────┘   │                       │
│                     │          │           │                       │
│                     ▼          ▼           ▼                       │
│               ┌─────────┐ ┌─────────┐ ┌─────────┐                 │
│               │ Fraud   │ │ Not     │ │ Unable  │                 │
│               │ Confirmed│ │ Fraud   │ │ to Det. │                 │
│               └────┬────┘ └────┬────┘ └────┬────┘                 │
│                    │           │           │                       │
│                    ▼           ▼           ▼                       │
│                 ┌──────────────────────────────┐                   │
│                 │        PREVENTION            │                   │
│                 │  • Update rules/models        │                   │
│                 │  • Update red flag library   │                   │
│                 │  • File FIR / share data     │                   │
│                 │  • Train claims team          │                   │
│                 │  • Add new network links     │                   │
│                 └──────────────┬───────────────┘                   │
│                                │  Loop back to Detection           │
│                                ▼                                   │
│                          ┌──────────────┐                           │
│                          │  DETECTION   │ (Improved)               │
│                          └──────────────┘                           │
└─────────────────────────────────────────────────────────────────────┘

3.2 The Analytics Tools at Each Stage

4. Building Fraud Indicator Dashboards in Power BI

A fraud monitoring dashboard serves three audiences: operations (claims managers who need to see today's flagged claims), analytics (fraud data scientists who need to monitor model performance), and management (CFO/COO who need to see fraud trends and ROI of anti-fraud investments). The dashboard design must serve all three without overwhelming any.

4.1 Dashboard Layout

┌─────────────────────────────────────────────────────────────────────┐
│ FRAUD MONITORING DASHBOARD  [Slicers: Product ▼] [Time Period ▼]   │
├──────────────────┬──────────────────┬──────────────────┬─────────────┤
│ Fraud Rate       │ Fraud Amount     │ Flags Raised     │ Investig'n  │
│ X.X%             │ ₹X.X Cr         │ X,XXX            │ Rate XX%    │
│ vs target: ±X.X% │ Recovery rate X% │ Flag hit rate X% │ Avg days XX │
├──────────────────┼──────────────────┴──────────────────┴─────────────┤
│ Total Claims vs  │ Fraud Amount Over Time (Line Chart)              │
│ Fraud Rate (Bar  │ • Monthly fraud amount                            │
│ + Line combo)    │ • Rolling 3-month average                        │
│                  │ • Target line                                     │
├──────────────────┬──────────────────────────────────────────────────┤
│ Fraud by Claim   │ Top Fraud Indicators (Horizontal Bar)            │
│ Type (Donut/Pie) │ • Indicator name vs. Confirm rate                │
│ [Accident, Theft,│ • Sorted by confirm rate descending              │
│  Fire, Health]   │ • Color-coded by effectiveness                   │
├──────────────────┴──────────────────────────────────────────────────┤
│ High-Risk Claims Table — Claims with Fraud Score > 70              │
│ [Claim ID, Customer, Amount, Status, Score, Indicators, Days Open] │
│ Conditional formatting: Red if >90 days, Yellow if score > 80      │
│ Drill-through to claim detail page                                  │
└─────────────────────────────────────────────────────────────────────┘

4.2 Power BI Measures for Fraud Monitoring

-- Load the fraud_indicators table alongside insurance_cleaned
-- Create relationships between Claims and Fraud Indicators tables

-- Core Fraud KPIs
Fraud Claim Count = CALCULATE(
    [Claim Count],
    insurance_cleaned[fraud_flag] = 1
)

Fraud Rate = DIVIDE([Fraud Claim Count], [Claim Count], 0)

Fraud Amount = CALCULATE(
    [Total Claims],
    insurance_cleaned[fraud_flag] = 1
)

Recovery Amount = SUM(fraud_indicators[recovery_amount])

Recovery Rate = DIVIDE([Recovery Amount], [Fraud Amount], 0)

-- Average fraud score from indicator table
Avg Fraud Score = AVERAGE(fraud_indicators[score])

-- Claims with multiple indicators (high suspicion)
High Suspicion Claims = CALCULATE(
    [Claim Count],
    fraud_indicators[score] > 70
)

-- Indicator effectiveness
Indicator Hit Rate = DIVIDE(
    COUNTROWS(FILTER(fraud_indicators, fraud_indicators[review_status] = "Confirmed")),
    COUNTROWS(fraud_indicators),
    0
)

-- Fraud rate by product
Fraud Rate by Product = DIVIDE(
    CALCULATE([Fraud Claim Count]),
    CALCULATE([Claim Count]),
    0
)

-- Year-over-year fraud rate change
Fraud Rate PY = CALCULATE(
    [Fraud Rate],
    SAMEPERIODLASTYEAR('DateTable'[Date])
)

Fraud Rate YoY Change = [Fraud Rate] - [Fraud Rate PY]

-- Fraud detection ROI (simplified)
Fraud Detection ROI = DIVIDE(
    [Fraud Amount] - [Fraud Amount] * DIVIDE([Fraud Rate PY], [Fraud Rate]),
    [Fraud Amount] * 0.05  -- Assume 5% of fraud spend on detection infrastructure
)

4.3 Configuring the Dashboard in Power BI

1. Import data: Load both the `insurance_cleaned.csv` and `fraud_indicators.csv` tables. Create a 1-to-many relationship from Claims.claim_id to Fraud Indicators.claim_id.
2. Create measures: All the DAX measures from Section 4.2. Set appropriate formats (Fraud Rate as %, Fraud Amount as ₹ Crores, Score as decimal).
3. Top KPI row: Add 4 KPI cards — Fraud Rate, Fraud Amount, Flags Raised, Investigation Rate. Use the KPI visual type for Fraud Rate with a goal of [Fraud Rate PY] (year-on-year improvement).
4. Fraud trend chart: Line and stacked column chart. Columns: Monthly claim count. Line: Fraud Rate (%). Dula axis formatting. Add a constant line at the industry average fraud rate (4–5% for general insurance).
5. Fraud by type donut: Donut chart with claim_type as legend and Fraud Claim Count as value. Add category labels showing fraud rate per type.
6. Top indicators bar chart: Horizontal bar chart with indicator_type as axis and Confirm Rate (confirmed ÷ total flagged for each indicator type) as value. Sorted descending by confirm rate.
7. High-risk claims table: Table with claim_id, customer_id, claim_amount, fraud_score, status. Conditionally format: background = red if fraud_score > 80, amber if > 60, green if ≤ 60. Add a drill-through to the claim detail page.
8. Slicers: Add policy_type (dropdown), claim_date (relative date: "last 12 months"), and fraud_score_range (pre-defined range). Sync across all pages.

5. Fraud Data Analysis in Python

Python is used for the behind-the-scenes analysis that feeds the Power BI dashboard — calculating fraud rates by segment, identifying the most predictive fraud indicators, and building statistical profiles of fraudulent vs. legitimate claims.

5.1 Loading and Exploring Fraud Data

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Load merged dataset with fraud flags
df = pd.read_csv('data/insurance_cleaned.csv')

# Load fraud indicators (if available as separate table)
try:
    fraud_indicators = pd.read_csv('data/fraud_indicators.csv')
    print(f"Fraud indicators loaded: {len(fraud_indicators):,} records")
except:
    print("Fraud indicators file not found. Using fraud_flag from main dataset.")
    fraud_indicators = None

# Basic fraud statistics
fraud_rate = df['fraud_flag'].mean() * 100
fraud_count = df['fraud_flag'].sum()
total_claims = len(df)
fraud_amount = df[df['fraud_flag'] == 1]['claim_amount'].sum()
total_amount = df['claim_amount'].sum()

print("=" * 55)
print("FRAUD OVERVIEW")
print("=" * 55)
print(f"Total claims:          {total_claims:>8,.0f}")
print(f"Fraud claims:          {fraud_count:>8,.0f} ({fraud_rate:.2f}%)")
print(f"Fraud amount:          ₹{fraud_amount/1e7:>8.1f} Cr")
print(f"Total claims amount:   ₹{total_amount/1e7:>8.1f} Cr")
print(f"Fraud % of amount:     {fraud_amount/total_amount*100:>8.2f}%")
print(f"Avg fraud claim:      ₹{fraud_amount/fraud_count:>8,.0f}")
print(f"Avg legitimate claim: ₹{(total_amount-fraud_amount)/(total_claims-fraud_count):>8,.0f}")

5.2 Fraud Rate by Segment

# Fraud rate by policy type
fraud_by_product = df.groupby('policy_type').agg(
    total_claims=('claim_id', 'count'),
    fraud_claims=('fraud_flag', 'sum'),
    total_amount=('claim_amount', 'sum'),
    fraud_amount=('claim_amount', lambda x: x[df.loc[x.index, 'fraud_flag'] == 1].sum())
).reset_index()

fraud_by_product['fraud_rate'] = fraud_by_product['fraud_claims'] / fraud_by_product['total_claims'] * 100
fraud_by_product = fraud_by_product.sort_values('fraud_rate', ascending=False)

print("=" * 75)
print(f"{'Policy Type':15s} {'Total Claims':>15s} {'Fraud Claims':>15s} {'Fraud Rate':>12s} {'Fraud Amount':>20s}")
print("-" * 75)
for _, row in fraud_by_product.iterrows():
    print(f"{row['policy_type']:15s} {row['total_claims']:>10,.0f}      {row['fraud_claims']:>8,.0f}      {row['fraud_rate']:>5.2f}%   ₹{row['fraud_amount']/1e7:>8.2f} Cr")

# Fraud rate by customer age group
df['age_group'] = pd.cut(df['age'], bins=[18, 30, 40, 50, 60, 100],
                          labels=['18-30', '31-40', '41-50', '51-60', '60+'])
fraud_by_age = df.groupby('age_group', observed=False).agg(
    total_claims=('fraud_flag', 'count'),
    fraud_claims=('fraud_flag', 'sum')
).reset_index()
fraud_by_age['fraud_rate'] = fraud_by_age['fraud_claims'] / fraud_by_age['total_claims'] * 100

print(f"\n{'Age Group':12s} {'Fraud Rate':12s} {'Claims':10s}")
print("-" * 34)
for _, row in fraud_by_age.iterrows():
    bar = '█' * int(row['fraud_rate'] * 2)
    print(f"{str(row['age_group']):12s} {row['fraud_rate']:>5.2f}%     {row['total_claims']:>6,.0f}  {bar}")

5.3 Identifying the Top Fraud Indicators

# Create fraud indicators from available data
# Each indicator is a binary variable: 1 = red flag present, 0 = not present

if 'days_to_settle' in df.columns:
    df['ind_rapid_settlement'] = (df['days_to_settle'] < 3).astype(int)
if 'claim_amount' in df.columns:
    threshold = df['claim_amount'].quantile(0.95)
    df['ind_amount_high'] = (df['claim_amount'] > threshold).astype(int)
if 'premium' in df.columns and 'claim_amount' in df.columns:
    df['claim_premium_ratio'] = df['claim_amount'] / (df['premium'] + 1)
    df['ind_high_ratio'] = (df['claim_premium_ratio'] > df['claim_premium_ratio'].quantile(0.90)).astype(int)
if 'credit_score' in df.columns:
    df['ind_low_credit'] = (df['credit_score'] < 600).astype(int)
if 'age' in df.columns:
    df['ind_young'] = (df['age'] < 25).astype(int)
if 'income' in df.columns:
    df['ind_low_income'] = (df['income'] < df['income'].quantile(0.25)).astype(int)

# Collect all indicator columns
indicator_cols = [c for c in df.columns if c.startswith('ind_')]

if len(indicator_cols) > 0:
    # Calculate the fraud rate for each indicator
    indicator_analysis = []
    for col in indicator_cols:
        flagged = df[df[col] == 1]
        not_flagged = df[df[col] == 0]
        if len(flagged) > 0 and len(not_flagged) > 0:
            fraud_rate_flagged = flagged['fraud_flag'].mean() * 100
            fraud_rate_not = not_flagged['fraud_flag'].mean() * 100
            lift = fraud_rate_flagged / fraud_rate_not if fraud_rate_not > 0 else 0
            indicator_analysis.append({
                'indicator': col.replace('ind_', '').replace('_', ' ').title(),
                'claims_flagged': len(flagged),
                'fraud_rate_flagged': fraud_rate_flagged,
                'fraud_rate_not': fraud_rate_not,
                'lift': lift,
                'total_fraud_caught': flagged['fraud_flag'].sum()
            })

    indicator_df = pd.DataFrame(indicator_analysis).sort_values('lift', ascending=False)

    print("=" * 85)
    print("TOP FRAUD INDICATORS (by lift — how much more fraud is found when this flag is present)")
    print("=" * 85)
    print(f"{'Indicator':25s} {'Flagged':>10s} {'Fraud Rate':>15s} {'Base Rate':>12s} {'Lift':>8s} {'Fraud Caught':>15s}")
    print("-" * 85)
    for _, row in indicator_df.iterrows():
        print(f"{row['indicator']:25s} {row['claims_flagged']:>8,.0f}     {row['fraud_rate_flagged']:>5.2f}%      {row['fraud_rate_not']:>5.2f}%    {row['lift']:>4.1f}x   {row['total_fraud_caught']:>6,.0f}")

    # Plot
    fig, ax = plt.subplots(figsize=(10, 5))
    top_n = indicator_df.head(8)
    bars = ax.barh(top_n['indicator'], top_n['lift'], color='#e17055', edgecolor='white')
    ax.axvline(x=1, color='gray', linestyle='--', alpha=0.7, label='No Predictive Power (Lift=1)')
    for bar, val in zip(bars, top_n['lift']):
        ax.text(val + 0.1, bar.get_y() + bar.get_height()/2, f'{val:.1f}x',
                va='center', fontsize=9, fontweight='bold')
    ax.set_xlabel('Lift (Fraud rate when flagged ÷ baseline fraud rate)')
    ax.set_title('Top Fraud Indicators by Predictive Lift', fontweight='bold')
    ax.legend()
    ax.spines['top'].set_visible(False)
    ax.spines['right'].set_visible(False)
    plt.tight_layout()
    plt.show()

5.4 Building a Weighted Fraud Risk Score

# Combine indicators into a composite fraud score
# Each indicator adds points proportional to its lift

if len(indicator_cols) > 0:
    # Normalise lifts to create weights
    max_lift = indicator_df['lift'].max()
    indicator_df['weight'] = indicator_df['lift'] / max_lift * 10  # scale to 0-10

    # Create score dictionary
    weight_dict = dict(zip(['ind_' + row['indicator'].lower().replace(' ', '_') for _, row in indicator_df.iterrows()],
                           indicator_df['weight']))

    # Calculate composite fraud score for each claim
    df['fraud_score'] = 0
    for col, weight in weight_dict.items():
        if col in df.columns:
            df['fraud_score'] += df[col] * weight

    # Scale to 0-100
    df['fraud_score'] = (df['fraud_score'] / df['fraud_score'].max() * 100)

    # Evaluate the score
    score_bins = pd.qcut(df['fraud_score'], q=10, labels=False, duplicates='drop')
    df['score_decile'] = score_bins

    score_eval = df.groupby('score_decile').agg(
        count=('fraud_flag', 'count'),
        actual_fraud=('fraud_flag', 'sum'),
        fraud_rate=('fraud_flag', 'mean'),
        avg_score=('fraud_score', 'mean'),
        total_amount=('claim_amount', 'sum')
    ).reset_index()

    print(f"\nFraud Score Performance (Decile Analysis)")
    print(f"{'Decile':8s} {'Claims':>10s} {'Fraud':>8s} {'Fraud Rate':>12s} {'Avg Score':>12s} {'Cum. Fraud':>15s}")
    print("-" * 65)
    cum_fraud = 0
    total_fraud = score_eval['actual_fraud'].sum()
    for _, row in score_eval.iterrows():
        cum_fraud += row['actual_fraud']
        cum_pct = cum_fraud / total_fraud * 100
        print(f"  {row['score_decile']+1:>1.0f}     {row['count']:>8,.0f}  {row['actual_fraud']:>6,.0f}   {row['fraud_rate']*100:>5.2f}%     {row['avg_score']:>6.1f}      {cum_fraud:>6,.0f} ({cum_pct:.0f}%)")

    print(f"\nInterpretation: The top 2 deciles capture approximately {cum_fraud / total_fraud * 100:.0f}%")
    print(f"of all fraud while requiring investigation of only 20% of claims.")
    print(f"This is the power of a scored fraud detection system vs. random review.")