Classification in Data Analysis - Python Real-Time Examples

What is Classification?

Classification is a supervised machine learning technique used to predict which category or class an observation belongs to.

Input Features

The variables used by the model to make a prediction.

Target Class

The category that we want the model to predict.

Training Data

Historical observations containing features and known class labels.

Prediction

The class predicted by the trained classification model.

1. Classification Workflow

A typical classification project follows a sequence of data preparation, model training, prediction, and evaluation.

Collect Data

→

Clean Data

→

Select Features

→

Train Model

→

Predict Class

→

Evaluate

2. Classification vs Regression

Classification	Regression
Predicts categories	Predicts numerical values
Spam / Not Spam	House Price
Fraud / Not Fraud	Monthly Sales
Pass / Fail	Employee Salary
Churn / Stay	Electricity Consumption

Simple rule: Classification predicts a class or category, while regression predicts a numerical value.

3. Types of Classification

Binary Classification

There are two possible classes.

Example: Fraud / Not Fraud

Multiclass Classification

There are more than two possible classes.

Example: Low / Medium / High

Multilabel Classification

One observation can belong to multiple labels.

Example: Article topics

4. Installing Required Libraries

Scikit-learn provides many practical classification algorithms.

Terminal

pip install pandas numpy matplotlib scikit-learn

5. Importing Classification Libraries

Python

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
    accuracy_score,
    confusion_matrix,
    classification_report
)

6. Creating a Real-Time Customer Dataset

Suppose an online company wants to predict whether a customer will purchase a product after visiting its website.

Python

data = {
    "age": [22, 25, 28, 35, 40, 45, 50, 52, 30, 38],
    "visits": [2, 3, 4, 6, 7, 8, 9, 10, 5, 6],
    "time_minutes": [5, 8, 10, 18, 20, 25, 30, 35, 15, 17],
    "purchased": [0, 0, 0, 1, 1, 1, 1, 1, 0, 1]
}

df = pd.DataFrame(data)

print(df)

age visits time_minutes purchased 0 22 2 5 0 1 25 3 8 0 2 28 4 10 0 3 35 6 18 1 4 40 7 20 1 5 45 8 25 1 6 50 9 30 1 7 52 10 35 1 8 30 5 15 0 9 38 6 17 1

7. Understanding Features and Target

In this example:

age → customer age
visits → number of website visits
time_minutes → time spent on the website
purchased → target class

The target has two classes:

Did Not Purchase

Purchased

Features

Target Class

Python

X = df[["age", "visits", "time_minutes"]]
y = df["purchased"]

8. Splitting Training and Testing Data

Python

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

print("Training rows:", len(X_train))
print("Testing rows:", len(X_test))

The training set is used to learn patterns. The test set is reserved for evaluating predictions on unseen observations.

9. Logistic Regression

Logistic regression is a commonly used classification algorithm. Despite its name, it is generally used to estimate the probability of class membership and then assign classes using a decision rule.

P(Y = 1) = 1 / (1 + e^-z)

The output probability can be converted into a class prediction.

Python

model = LogisticRegression()

model.fit(X_train, y_train)

print("Model trained successfully")

10. Making Classification Predictions

Python

predictions = model.predict(X_test)

print(predictions)

Each prediction represents the class selected by the trained model.

11. Predicting a New Customer

Suppose a new customer is 42 years old, has visited the website 8 times, and spent 24 minutes browsing.

Python

new_customer = pd.DataFrame({
    "age": [42],
    "visits": [8],
    "time_minutes": [24]
})

prediction = model.predict(new_customer)

if prediction[0] == 1:
    print("Customer is predicted to purchase")
else:
    print("Customer is predicted not to purchase")

12. Getting Prediction Probability

Classification models can often provide probabilities for the possible classes.

Python

probability = model.predict_proba(new_customer)

print(probability)

[[probability_for_class_0, probability_for_class_1]]

Probabilities are useful when a business needs more information than only the final class label.

13. Classification Threshold

A binary classifier may convert a predicted probability into a class using a threshold. A commonly used default is 0.5, but the appropriate threshold depends on the application.

Probability ≥ Threshold → Class 1

Python

probability = model.predict_proba(new_customer)[0][1]

threshold = 0.5

if probability >= threshold:
    print("Predicted class: 1")
else:
    print("Predicted class: 0")

14. Accuracy Score

Accuracy measures the proportion of predictions that match the actual classes.

Accuracy = Correct Predictions / Total Predictions

Python

accuracy = accuracy_score(y_test, predictions)

print("Accuracy:", accuracy)

Accuracy can be misleading when one class is much more common than another. In such cases, other metrics should also be examined.

15. Confusion Matrix

A confusion matrix summarizes classification predictions by comparing actual classes with predicted classes.

	Predicted Negative	Predicted Positive
Actual Negative	True Negative	False Positive
Actual Positive	False Negative	True Positive

Python

cm = confusion_matrix(y_test, predictions)

print(cm)

16. Precision

Precision measures how many observations predicted as positive were actually positive.

Precision = TP / (TP + FP)

Precision can be particularly important when false positive predictions have significant consequences.

17. Recall

Recall measures how many of the actual positive observations were correctly identified.

Recall = TP / (TP + FN)

Recall can be particularly important when missing a positive case has a significant consequence.

18. F1 Score

F1 score combines precision and recall using their harmonic mean.

F1 = 2 × (Precision × Recall) / (Precision + Recall)

It can be useful when both false positives and false negatives matter.

19. Classification Report

Python

print(
    classification_report(
        y_test,
        predictions
    )
)

The classification report provides precision, recall, F1 score, and support for each class.

20. Common Classification Algorithms

Algorithm	Typical Use
Logistic Regression	Binary and multiclass classification
Decision Tree	Rule-based classification
Random Forest	Robust classification on structured data
K-Nearest Neighbors	Similarity-based classification
Support Vector Machine	Classification with separating boundaries
Naive Bayes	Text and probabilistic classification

21. Decision Tree Classification

A decision tree makes predictions by applying a sequence of decision rules to the features.

Python

from sklearn.tree import DecisionTreeClassifier

model = DecisionTreeClassifier(
    random_state=42
)

model.fit(X_train, y_train)

predictions = model.predict(X_test)

print("Accuracy:", accuracy_score(y_test, predictions))

22. Random Forest Classification

Random Forest combines predictions from multiple decision trees. It is commonly used for structured classification datasets.

Python

from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier(
    n_estimators=100,
    random_state=42
)

model.fit(X_train, y_train)

predictions = model.predict(X_test)

print("Accuracy:", accuracy_score(y_test, predictions))

23. K-Nearest Neighbors Classification

KNN predicts a class using the labels of nearby observations.

Python

from sklearn.neighbors import KNeighborsClassifier

model = KNeighborsClassifier(n_neighbors=3)

model.fit(X_train, y_train)

predictions = model.predict(X_test)

print("Accuracy:", accuracy_score(y_test, predictions))

KNN is distance-based, so feature scaling is often important when features are measured on different scales.

24. Support Vector Machine

Support Vector Machines attempt to find a decision boundary that separates classes while maximizing the margin under the chosen formulation.

Python

from sklearn.svm import SVC

model = SVC()

model.fit(X_train, y_train)

predictions = model.predict(X_test)

print("Accuracy:", accuracy_score(y_test, predictions))

25. Feature Scaling

Some algorithms perform better when numerical features are placed on comparable scales.

Python

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

model = LogisticRegression()

model.fit(X_train_scaled, y_train)

predictions = model.predict(X_test_scaled)

print("Accuracy:", accuracy_score(y_test, predictions))

Fit the scaler using training data and then transform both training and test data. This helps avoid leaking information from the test set.

26. Real-Time Example: Customer Churn Prediction

A telecom company wants to predict whether a customer is likely to leave the service.

Python

data = {
    "months": [3, 8, 12, 24, 36, 48, 60, 72],
    "support_calls": [5, 4, 3, 2, 2, 1, 1, 0],
    "monthly_bill": [900, 850, 800, 750, 700, 650, 600, 550],
    "churn": [1, 1, 1, 0, 0, 0, 0, 0]
}

df = pd.DataFrame(data)

X = df[["months", "support_calls", "monthly_bill"]]
y = df["churn"]

model = LogisticRegression()

model.fit(X, y)

customer = pd.DataFrame({
    "months": [10],
    "support_calls": [4],
    "monthly_bill": [820]
})

prediction = model.predict(customer)

print("Churn prediction:", prediction[0])

In a production churn system, the model should be trained and evaluated using a substantially larger dataset with appropriate time-based validation and carefully selected customer features.

27. Real-Time Example: Spam Email Detection

An email system can classify messages as spam or legitimate.

Email Feature	Possible Signal
Number of links	Large numbers may provide useful predictive information
Text content	Words and phrases can be transformed into numerical features
Sender information	Historical sender patterns may provide signals
Message length	Can be included as a numerical feature

Python

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB

emails = [
    "meeting at 10 am",
    "project report attached",
    "win free money now",
    "claim your prize today",
    "team meeting tomorrow",
    "free offer click now"
]

labels = [0, 0, 1, 1, 0, 1]

vectorizer = TfidfVectorizer()

X = vectorizer.fit_transform(emails)

model = MultinomialNB()

model.fit(X, labels)

new_email = ["claim your free prize"]

new_X = vectorizer.transform(new_email)

prediction = model.predict(new_X)

print("Spam prediction:", prediction[0])

28. Real-Time Example: Loan Approval Classification

A financial organization could classify applications into categories such as approved or not approved using historical application data, subject to applicable policies and validation requirements.

Python

data = {
    "income": [30000, 45000, 50000, 70000, 80000, 90000],
    "credit_score": [580, 620, 650, 720, 750, 780],
    "loan_amount": [25000, 30000, 35000, 30000, 40000, 45000],
    "approved": [0, 0, 1, 1, 1, 1]
}

df = pd.DataFrame(data)

X = df[["income", "credit_score", "loan_amount"]]
y = df["approved"]

model = LogisticRegression()

model.fit(X, y)

application = pd.DataFrame({
    "income": [65000],
    "credit_score": [700],
    "loan_amount": [30000]
})

prediction = model.predict(application)

print("Predicted class:", prediction[0])

Automated classification in lending and other high-impact domains requires appropriate governance, validation, fairness assessment, and compliance review.

29. Real-Time Example: Fraud Detection

A payment company can classify transactions as potentially fraudulent or legitimate using historical transaction data.

Python

data = {
    "amount": [500, 800, 1200, 200, 15000, 18000, 300, 700],
    "transactions_today": [2, 3, 4, 1, 15, 18, 2, 3],
    "international": [0, 0, 0, 0, 1, 1, 0, 0],
    "fraud": [0, 0, 0, 0, 1, 1, 0, 0]
}

df = pd.DataFrame(data)

X = df[["amount", "transactions_today", "international"]]
y = df["fraud"]

model = LogisticRegression()

model.fit(X, y)

transaction = pd.DataFrame({
    "amount": [12000],
    "transactions_today": [10],
    "international": [1]
})

prediction = model.predict(transaction)

print("Fraud prediction:", prediction[0])

Fraud detection is often highly imbalanced, so accuracy alone may not adequately describe model performance.

30. Class Imbalance

Class imbalance occurs when one class has substantially more observations than another.

Example	Class Distribution
Normal transactions	99%
Fraudulent transactions	1%

A model could achieve high accuracy by mostly predicting the majority class, while still performing poorly at identifying the minority class.

Precision, recall, F1 score, confusion matrices, and suitable threshold analysis can provide additional insight.

31. Handling Class Weights

Some scikit-learn classifiers support class weighting to give more importance to underrepresented classes during training.

Python

model = LogisticRegression(
    class_weight="balanced"
)

model.fit(X_train, y_train)

predictions = model.predict(X_test)

print(classification_report(y_test, predictions))

32. Multiclass Classification

Multiclass classification involves more than two possible classes. For example, a company might classify customers into low, medium, or high-value groups.

Python

data = {
    "visits": [2, 3, 5, 6, 8, 9, 12, 15, 18],
    "spending": [500, 700, 1200, 1500, 2500, 3000, 5000, 7000, 9000],
    "customer_type": [
        "Low", "Low", "Low",
        "Medium", "Medium", "Medium",
        "High", "High", "High"
    ]
}

df = pd.DataFrame(data)

X = df[["visits", "spending"]]
y = df["customer_type"]

model = LogisticRegression(max_iter=1000)

model.fit(X, y)

customer = pd.DataFrame({
    "visits": [10],
    "spending": [3500]
})

prediction = model.predict(customer)

print("Customer type:", prediction[0])

33. Feature Importance with Random Forest

Tree-based models can provide feature importance estimates that help analysts explore which input variables contributed to the model's decisions.

Python

from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier(
    n_estimators=100,
    random_state=42
)

model.fit(X_train, y_train)

importance = pd.Series(
    model.feature_importances_,
    index=X_train.columns
)

print(importance.sort_values(ascending=False))

Feature importance is model-dependent and should be interpreted carefully; it does not automatically establish causality.

34. Cross-Validation for Classification

Cross-validation can provide a more stable estimate of classification performance than relying on one train-test split alone.

Python

from sklearn.model_selection import cross_val_score

model = LogisticRegression()

scores = cross_val_score(
    model,
    X,
    y,
    cv=5,
    scoring="accuracy"
)

print("Scores:", scores)
print("Average accuracy:", scores.mean())

35. Real-Time Example: Employee Attrition

An organization can analyze historical employee records to study whether an employee is likely to leave, using an appropriately designed and validated classification model.

Python

data = {
    "experience": [1, 2, 3, 5, 7, 9, 12, 15],
    "satisfaction": [2, 3, 3, 4, 4, 5, 5, 5],
    "overtime_hours": [20, 18, 15, 10, 8, 5, 4, 2],
    "left_company": [1, 1, 1, 0, 0, 0, 0, 0]
}

df = pd.DataFrame(data)

X = df[["experience", "satisfaction", "overtime_hours"]]
y = df["left_company"]

model = LogisticRegression()

model.fit(X, y)

employee = pd.DataFrame({
    "experience": [3],
    "satisfaction": [3],
    "overtime_hours": [17]
})

prediction = model.predict(employee)

print("Predicted class:", prediction[0])

Employee-related classification should not be used as the sole basis for consequential employment decisions. Human review, appropriate governance, and validation are important.

36. Classification Using a CSV File

In real data analysis projects, classification data commonly comes from CSV files, databases, APIs, or data warehouses.

Python

df = pd.read_csv("customers.csv")

print(df.head())

X = df[
    [
        "age",
        "visits",
        "time_minutes"
    ]
]

y = df["purchased"]

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

model = LogisticRegression()

model.fit(X_train, y_train)

predictions = model.predict(X_test)

print(
    classification_report(
        y_test,
        predictions
    )
)

37. Classification with a Pipeline

A pipeline combines preprocessing and modeling steps into one workflow.

Python

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

pipeline = Pipeline([
    ("scaler", StandardScaler()),
    ("model", LogisticRegression())
])

pipeline.fit(X_train, y_train)

predictions = pipeline.predict(X_test)

print("Accuracy:", accuracy_score(y_test, predictions))

Pipelines help keep preprocessing and model training consistent, especially when cross-validation is used.

38. ROC Curve and AUC

For binary classification, the ROC curve examines the trade-off between true positive rate and false positive rate across classification thresholds.

Python

from sklearn.metrics import roc_auc_score

probabilities = model.predict_proba(X_test)[:, 1]

auc = roc_auc_score(
    y_test,
    probabilities
)

print("ROC AUC:", auc)

AUC summarizes ranking performance across thresholds, but the appropriate evaluation method depends on the business objective and class distribution.

39. Real-Time Example: Customer Purchase Prediction

An e-commerce company wants to identify visitors who may purchase during their current session.

Website Data

→

Customer Features

→

Classifier

→

Purchase Class

Feature	Example
Pages viewed	12
Time on website	25 minutes
Previous purchases	4
Cart items	3
Target	Purchased / Not Purchased

40. Real-Time Example: Product Category Classification

An e-commerce platform can classify products into categories such as electronics, clothing, books, or home products using product attributes and appropriately engineered text or numerical features.

Python

data = {
    "weight": [1.2, 0.8, 2.5, 0.3, 1.0, 2.8],
    "price": [15000, 2500, 1200, 500, 18000, 900],
    "rating": [4.5, 4.2, 4.0, 4.6, 4.4, 3.9],
    "category": [
        "Electronics",
        "Clothing",
        "Books",
        "Clothing",
        "Electronics",
        "Books"
    ]
}

df = pd.DataFrame(data)

X = df[["weight", "price", "rating"]]
y = df["category"]

model = LogisticRegression(max_iter=1000)

model.fit(X, y)

product = pd.DataFrame({
    "weight": [1.5],
    "price": [14000],
    "rating": [4.3]
})

prediction = model.predict(product)

print("Predicted category:", prediction[0])

41. Real-Time Example: Student Result Classification

A school analytics system can classify historical student records into categories such as Pass or Fail based on appropriate academic features.

Python

data = {
    "study_hours": [1, 2, 3, 4, 5, 6, 7, 8],
    "attendance": [60, 65, 70, 75, 80, 85, 90, 95],
    "assignments": [2, 3, 4, 5, 6, 7, 8, 9],
    "result": [0, 0, 0, 1, 1, 1, 1, 1]
}

df = pd.DataFrame(data)

X = df[["study_hours", "attendance", "assignments"]]
y = df["result"]

model = LogisticRegression()

model.fit(X, y)

student = pd.DataFrame({
    "study_hours": [5],
    "attendance": [82],
    "assignments": [6]
})

prediction = model.predict(student)

print("Predicted result:", prediction[0])

42. Real-Time Example: Support Ticket Classification

A company can automatically classify customer support tickets into categories such as Billing, Technical Support, Delivery, or Account.

Python

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression

tickets = [
    "payment failed on my card",
    "invoice amount is incorrect",
    "my package is late",
    "delivery has not arrived",
    "app is not opening",
    "website shows an error",
    "please update my account",
    "I cannot change my password"
]

labels = [
    "Billing",
    "Billing",
    "Delivery",
    "Delivery",
    "Technical",
    "Technical",
    "Account",
    "Account"
]

vectorizer = TfidfVectorizer()

X = vectorizer.fit_transform(tickets)

model = LogisticRegression()

model.fit(X, labels)

new_ticket = [
    "my payment was declined"
]

new_X = vectorizer.transform(new_ticket)

prediction = model.predict(new_X)

print("Ticket category:", prediction[0])

43. Comparing Classification Models

Different classification algorithms may behave differently on the same dataset. A practical workflow can compare several suitable models.

Python

from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression

models = {
    "Logistic Regression": LogisticRegression(),
    "Decision Tree": DecisionTreeClassifier(random_state=42),
    "Random Forest": RandomForestClassifier(
        n_estimators=100,
        random_state=42
    )
}

for name, model in models.items():

    model.fit(X_train, y_train)

    predictions = model.predict(X_test)

    score = accuracy_score(
        y_test,
        predictions
    )

    print(name, ":", score)

Model comparison should use evaluation procedures appropriate to the dataset and business objective rather than relying on a single metric.

44. Complete Real-Time Classification Project

The following example combines the major steps of a customer purchase classification workflow.

Complete Python Example

import pandas as pd

from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
    accuracy_score,
    classification_report,
    confusion_matrix
)

# 1. Create customer dataset
data = {
    "age": [22, 25, 28, 35, 40, 45, 50, 52, 30, 38,
            27, 33, 41, 48, 55],
    "visits": [2, 3, 4, 6, 7, 8, 9, 10, 5, 6,
               3, 5, 7, 9, 11],
    "time_minutes": [5, 8, 10, 18, 20, 25, 30, 35, 15, 17,
                     9, 14, 22, 29, 40],
    "purchased": [0, 0, 0, 1, 1, 1, 1, 1, 0, 1,
                  0, 0, 1, 1, 1]
}

df = pd.DataFrame(data)

# 2. Select features and target
X = df[
    [
        "age",
        "visits",
        "time_minutes"
    ]
]

y = df["purchased"]

# 3. Split data
X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

# 4. Create pipeline
model = Pipeline([
    ("scaler", StandardScaler()),
    ("classifier", LogisticRegression())
])

# 5. Train
model.fit(X_train, y_train)

# 6. Predict
predictions = model.predict(X_test)

# 7. Evaluate
print(
    "Accuracy:",
    accuracy_score(y_test, predictions)
)

print("\nConfusion Matrix:")
print(
    confusion_matrix(
        y_test,
        predictions
    )
)

print("\nClassification Report:")
print(
    classification_report(
        y_test,
        predictions
    )
)

# 8. Predict a new customer
new_customer = pd.DataFrame({
    "age": [42],
    "visits": [8],
    "time_minutes": [24]
})

prediction = model.predict(new_customer)
probability = model.predict_proba(new_customer)

print("\nPredicted class:", prediction[0])
print("Class probabilities:", probability)

This workflow can be adapted to larger real-world datasets after appropriate cleaning, feature engineering, validation, and monitoring.

45. Classification Evaluation Metrics

Accuracy

Overall correct predictions

Precision

Correct positive predictions

Recall

Actual positives identified

Balance of precision and recall

Metric	Question It Helps Answer
Accuracy	How many predictions were correct overall?
Precision	When the model predicts positive, how often is it correct?
Recall	How many actual positive cases did the model identify?
F1 Score	How well are precision and recall balanced?

46. Common Classification Problems

Class imbalance: One class has many more observations than another.
Overfitting: The model learns training-specific noise.
Underfitting: The model is too simple to capture useful patterns.
Data leakage: Information unavailable at prediction time enters the model.
Incorrect metric: Accuracy may not be sufficient for every problem.
Data quality: Incorrect labels or missing values can affect performance.
Distribution shift: Future data may differ from historical training data.

A classification model should be evaluated using metrics and validation procedures that reflect how errors affect the real application.

47. Classification Project Checklist

Define the classification problem.
Identify the target classes.
Collect representative historical data.
Inspect missing values and incorrect labels.
Explore class distribution.
Select useful features.
Encode categorical variables when necessary.
Scale features when appropriate for the chosen algorithm.
Split data into training and evaluation sets.
Train one or more suitable classification models.
Generate predictions and probabilities where appropriate.
Evaluate using suitable metrics.
Inspect the confusion matrix.
Check for overfitting and data leakage.
Validate performance on representative unseen data.
Monitor the model after deployment.

48. Important Classification Rules

Classification predicts categories rather than continuous numerical values.
Clearly define the target classes before training a model.
Use historical labeled data for supervised classification.
Keep training and evaluation data separate.
Use pipelines to keep preprocessing and modeling consistent.
Accuracy is not always the most informative metric.
Use precision and recall when the costs of false positives and false negatives differ.
Use confusion matrices to understand the types of errors being made.
Check class imbalance before interpreting accuracy.
Watch for data leakage and overfitting.
Choose classification algorithms based on the dataset and problem.
Validate the model on data representative of its future use.
For high-impact applications, include appropriate human oversight and governance.

Key idea: Classification converts historical labeled data into a model that can assign new observations to meaningful categories. Good classification analysis combines data quality, suitable algorithms, appropriate evaluation metrics, and real-world context.

Python Data Analysis - Classification

Learn classification concepts through practical Python and real-world examples.