Medical AI & Machine Learning Case Study

A production healthcare AI system requires much more than training a machine learning model.

Clinical Workflow Question

At or near discharge, can historical and currently available patient information help identify patients who may have an elevated probability of readmission within the next 30 days?

Prediction Unit

• One prediction per eligible discharge event.
• Prediction timestamp must be clearly defined.
• Only information available at prediction time may be used.
• The outcome window is defined before model development.

Target Variable

readmission_30d = 0

# 0 = no readmission within the defined window
# 1 = readmission within the defined window

A realistic system may combine multiple hospital information sources.

import pandas as pd

df = pd.read_csv("hospital_readmission.csv")

print(df.head())
print(df.shape)
print(df.info())

Basic Data Quality

print("Missing values:")
print(df.isnull().sum())

print("\nDuplicate rows:")
print(df.duplicated().sum())

print("\nTarget distribution:")
print(df["readmission_30d"].value_counts())

Healthcare data requires careful validation because incorrect or inconsistent values can affect downstream decisions.

• Check duplicate encounter records.
• Validate age ranges.
• Validate clinically plausible numerical ranges.
• Check missing values.
• Check inconsistent categorical values.
• Validate timestamps.
• Check whether events occurred before the prediction timestamp.
• Check patient and encounter relationships.

Data leakage is especially important in medical prediction because healthcare records contain events that may occur after the model's prediction point.

Incorrect Example

Suppose the model predicts risk at discharge but the training data includes information created after discharge. The model may appear highly accurate while being unusable in real operation.

Correct Principle

Raw medical records can be transformed into structured features representing patient history and recent healthcare utilization.

Example Features

df["utilization_score"] = (
    df["previous_admissions"]
    + df["emergency_visits"]
)

df["medication_complexity"] = (
    df["medication_count"] >= 6
).astype(int)

df["long_stay"] = (
    df["length_of_stay"] >= 7
).astype(int)

df["followup_delay"] = (
    df["followup_days"] > 14
).astype(int)

Medical data often contains repeated encounters for the same patient. Splitting data randomly without considering patient identity or time can produce overly optimistic results.

Recommended Concept

• Keep patients appropriately separated between development and evaluation datasets.
• Consider temporal validation when future deployment performance is the goal.
• Keep the final test set untouched until final evaluation.

from sklearn.model_selection import train_test_split

X = df.drop(columns=[
    "patient_id",
    "readmission_30d"
])

y = df["readmission_30d"]

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.20,
    random_state=42,
    stratify=y
)

A simple interpretable baseline provides a reference point before testing more complex models.

from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression

baseline_model = Pipeline([
    ("preprocessor", preprocessor),
    ("model", LogisticRegression(
        max_iter=1000,
        class_weight="balanced"
    ))
])

baseline_model.fit(X_train, y_train)

A tree-based ensemble can capture nonlinear relationships between patient characteristics and the target outcome.

from sklearn.ensemble import RandomForestClassifier

rf_model = Pipeline([
    ("preprocessor", preprocessor),

    ("model", RandomForestClassifier(
        n_estimators=300,
        random_state=42,
        class_weight="balanced"
    ))
])

rf_model.fit(X_train, y_train)

If the model outputs a probability, calibration becomes important. A predicted probability should be interpreted carefully and evaluated against observed outcomes.

Calibration should be evaluated on appropriate validation data and monitored after deployment.

A probability can be transformed into operational risk groups. Thresholds should be validated for the intended clinical workflow rather than assumed to be universally appropriate.

def risk_category(probability):

    if probability >= 0.80:
        return "High"

    elif probability >= 0.50:
        return "Moderate"

    return "Low"

Healthcare users may need to understand which inputs contributed to a model prediction.

Possible Explanation Signals

• Multiple previous admissions.
• Recent emergency utilization.
• Long hospital stay.
• High medication complexity.
• Long follow-up interval.

A production system may use an explainability library appropriate for the selected model.

# Conceptual example

explanation = {
    "previous_admissions": "increased risk",
    "emergency_visits": "increased risk",
    "medication_count": "moderate contribution",
    "followup_days": "increased risk"
}

print(explanation)

import joblib

joblib.dump(
    rf_model,
    "readmission_model.joblib"
)

In production, the model artifact should be versioned together with its training data definition, preprocessing logic, dependencies, evaluation results, and approval metadata.

A prediction API can expose the validated model to an authorized clinical application.

from fastapi import FastAPI
from pydantic import BaseModel
import joblib

app = FastAPI(
    title="Medical Readmission Risk API"
)

model = joblib.load(
    "readmission_model.joblib"
)


class PatientData(BaseModel):

    age: int
    length_of_stay: float
    previous_admissions: int
    emergency_visits: int
    medication_count: int
    followup_days: int


@app.get("/health")
def health():

    return {
        "status": "healthy"
    }


@app.post("/predict")
def predict(patient: PatientData):

    data = [[
        patient.age,
        patient.length_of_stay,
        patient.previous_admissions,
        patient.emergency_visits,
        patient.medication_count,
        patient.followup_days
    ]]

    probability = model.predict_proba(data)[0][1]

    return {
        "risk_probability": float(
            probability
        )
    }

{
    "age": 67,
    "length_of_stay": 6,
    "previous_admissions": 3,
    "emergency_visits": 2,
    "medication_count": 8,
    "followup_days": 14
}

{
    "risk_probability": 0.78
}

The consuming application can combine this risk signal with appropriate clinical context and human review.

A production application may maintain separate records for predictions, model versions, and eventual outcomes.

CREATE TABLE model_predictions (
    prediction_id BIGINT PRIMARY KEY,
    patient_key VARCHAR(100),
    prediction_time TIMESTAMP,
    model_version VARCHAR(50),
    risk_probability DECIMAL(6,5),
    risk_category VARCHAR(20),
    created_at TIMESTAMP
);

Access to patient-linked information should be restricted according to the organization's security and governance requirements.

Technical Monitoring

• API availability.
• Prediction latency.
• Error rates.
• Model loading failures.
• Infrastructure utilization.

Data Monitoring

• Missing-value rates.
• Unexpected categories.
• Feature distribution changes.
• Input volume changes.

Model Monitoring

• Prediction distribution.
• Calibration.
• Recall and precision when labels become available.
• Performance by relevant patient subgroups.

Healthcare practices, patient populations, documentation patterns, and operational processes can change.

Healthcare AI systems should be evaluated for differences in performance across relevant patient groups.

Example Evaluation Dimensions

• Recall across groups.
• False-positive rates.
• False-negative rates.
• Calibration across groups.
• Data representation.
• Missing-data patterns.

Security Controls

• Authentication.
• Role-based authorization.
• Encryption in transit.
• Encryption at rest.
• Audit logging.
• Secret management.
• Network segmentation.
• Secure API validation.

Privacy Controls

• Data minimization.
• De-identification where appropriate.
• Controlled access.
• Purpose limitation.
• Retention policies.
• Governance of model training datasets.

Data Tests

• Required fields are present.
• Data types are valid.
• Missing values remain within defined limits.
• Clinical ranges are validated where appropriate.

Model Tests

• Model loads successfully.
• Prediction output is valid.
• Probability remains between 0 and 1.
• Expected performance thresholds are checked.

API Tests

def test_prediction_response():

    response = client.post(
        "/predict",
        json={
            "age": 65,
            "length_of_stay": 5,
            "previous_admissions": 2,
            "emergency_visits": 1,
            "medication_count": 5,
            "followup_days": 10
        }
    )

    assert response.status_code == 200

    result = response.json()

    assert "risk_probability" in result

    assert 0 <= result["risk_probability"] <= 1

The project should not be evaluated only by model metrics. The deployed workflow should also be evaluated to determine whether it provides useful operational or clinical value.

• Treating an ML prediction as a diagnosis.
• Training on information unavailable at prediction time.
• Ignoring patient-level data leakage.
• Reporting only accuracy.
• Ignoring calibration.
• Deploying without clinical validation.
• Ignoring subgroup performance.
• Failing to monitor model drift.
• Allowing unauthorized access to sensitive data.
• Assuming model explanations prove causation.

• ☐ Clinical problem clearly defined
• ☐ Prediction timestamp documented
• ☐ Outcome definition documented
• ☐ Data sources documented
• ☐ Privacy requirements addressed
• ☐ Data leakage reviewed
• ☐ Patient-level splitting considered
• ☐ Feature engineering documented
• ☐ Baseline model created
• ☐ Multiple models evaluated
• ☐ Appropriate metrics selected
• ☐ Calibration evaluated
• ☐ Explainability considered
• ☐ Subgroup performance evaluated
• ☐ Human review incorporated
• ☐ API secured and tested
• ☐ Model versioned
• ☐ Monitoring implemented
• ☐ Drift detection implemented
• ☐ Retraining process defined
• ☐ Clinical validation completed before real deployment
• ☐ Governance and regulatory requirements reviewed

A professional Medical AI solution is not simply a machine learning algorithm.

The complete solution combines:

Medical AI & Machine Learning Case Study

Healthcare Data → ML → Explainability → API → Clinical Workflow → Monitoring → MLOps

Educational case study — real-world clinical deployment requires appropriate validation, governance, security, and regulatory review.