Supervised Learning Using Python

What is Supervised Learning?

Supervised Learning is a type of Machine Learning where a model learns from historical data containing both input features and known target values. After learning the relationship between inputs and targets, the model can make predictions for new data.

Labeled Data

→

Features + Target

→

Train / Test

→

Train Model

→

Predict

→

Evaluate

Input Features

Variables supplied to the model, such as area, experience, income, study hours, or number of visits.

Target

The known output that the model learns to predict, such as price, salary, grade, or customer churn.

Labeled Data

Training examples where both input variables and expected outcomes are available.

Prediction

The output produced by a trained model for previously unseen input data.

1. How Supervised Learning Works

A supervised learning system learns a relationship between input variables and a known target variable.

Supervised Learning Flow

Historical Data
      ↓
Features + Known Target
      ↓
Training Algorithm
      ↓
Learned Model
      ↓
New Data
      ↓
Prediction

Example: A real-estate company can use historical house data containing area, bedrooms, age, and known selling prices to train a model that estimates the price of a new house.

2. Regression vs Classification

The two major supervised-learning problem types are regression and classification.

Type	Target	Example
Regression	Numerical value	Predict house price
Regression	Numerical value	Predict monthly sales
Classification	Category	Spam or not spam
Classification	Category	Customer churn or no churn

3. Installing Required Libraries

Scikit-learn provides many supervised-learning algorithms and evaluation utilities.

Terminal

pip install pandas numpy scikit-learn matplotlib seaborn

4. Importing Python Libraries

Python

import pandas as pd
import numpy as np

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

5. Creating a Labeled Dataset

Consider a simple business dataset containing advertising spending and corresponding sales.

Python

data = {
    "Advertising": [
        10, 15, 20, 25, 30,
        35, 40, 45, 50, 55
    ],

    "WebsiteVisits": [
        100, 130, 160, 190, 220,
        250, 290, 320, 350, 390
    ],

    "Sales": [
        20, 27, 33, 39, 45,
        52, 60, 66, 73, 81
    ]
}

df = pd.DataFrame(data)

print(df)

Target: Sales is the target variable because it is the value we want the model to predict.

6. Selecting Features and Target

Separate the input variables from the target variable.

Python

X = df[
    ["Advertising", "WebsiteVisits"]
]

y = df["Sales"]

print("Features:")
print(X)

print("\nTarget:")
print(y)

X: Input features.
y: Target value.

7. Splitting the Dataset

A common approach is to divide the data into training and testing portions.

Python

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

print("Training data:")
print(X_train)

print("\nTesting data:")
print(X_test)

Important: The test data should remain separate from the model's training process so that it can provide an estimate of performance on unseen examples.

8. Linear Regression

Linear Regression is a supervised-learning algorithm used to predict a continuous numerical target.

Python

from sklearn.linear_model import LinearRegression

model = LinearRegression()

model.fit(
    X_train,
    y_train
)

9. Making Predictions

Python

predictions = model.predict(
    X_test
)

print(predictions)

Result: The trained model generates estimated sales values for the test records.

10. Evaluating Regression Predictions

Mean Absolute Error

Python

mae = mean_absolute_error(
    y_test,
    predictions
)

print("MAE:", mae)

Mean Squared Error

Python

mse = mean_squared_error(
    y_test,
    predictions
)

print("MSE:", mse)

R² Score

Python

r2 = r2_score(
    y_test,
    predictions
)

print("R²:", r2)

11. Actual vs Predicted Values

Python

comparison = pd.DataFrame({
    "Actual": y_test.values,
    "Predicted": predictions
})

print(comparison)

12. Predicting New Sales

After training, the model can be used to estimate sales for a future business scenario.

Python

new_data = pd.DataFrame({
    "Advertising": [60],
    "WebsiteVisits": [420]
})

predicted_sales = model.predict(
    new_data
)

print(
    "Predicted Sales:",
    predicted_sales[0]
)

Real-Time Example: A retail company could use this type of model to estimate expected sales from planned advertising and website traffic.

13. Real-Time Example: House Price Prediction

A real-estate company has historical property information and wants to estimate the price of a new property.

Python

houses = pd.DataFrame({
    "Area": [
        800, 1000, 1200, 1500, 1800,
        2000, 2200, 2500, 2800, 3000
    ],

    "Bedrooms": [
        2, 2, 3, 3, 3,
        4, 4, 4, 5, 5
    ],

    "Age": [
        15, 10, 8, 12, 7,
        5, 4, 6, 3, 2
    ],

    "Price": [
        45, 55, 68, 82, 95,
        110, 125, 140, 158, 175
    ]
})

X = houses[
    ["Area", "Bedrooms", "Age"]
]

y = houses["Price"]

house_model = LinearRegression()

house_model.fit(X, y)

new_house = pd.DataFrame({
    "Area": [1700],
    "Bedrooms": [3],
    "Age": [5]
})

predicted_price = house_model.predict(
    new_house
)

print(
    "Predicted Price:",
    predicted_price[0],
    "lakhs"
)

Features: Area, Bedrooms, and Age.
Target: House Price.

14. Real-Time Example: Employee Salary Prediction

A company can use historical employee records to estimate salary from experience and other relevant variables.

Python

employees = pd.DataFrame({
    "Experience": [
        1, 2, 3, 5, 7,
        9, 11, 13
    ],

    "Projects": [
        1, 2, 3, 5, 7,
        8, 10, 12
    ],

    "Salary": [
        30000, 35000, 42000, 55000,
        68000, 80000, 95000, 110000
    ]
})

X = employees[
    ["Experience", "Projects"]
]

y = employees["Salary"]

salary_model = LinearRegression()

salary_model.fit(X, y)

new_employee = pd.DataFrame({
    "Experience": [6],
    "Projects": [6]
})

predicted_salary = salary_model.predict(
    new_employee
)

print(
    "Predicted Salary:",
    predicted_salary[0]
)

15. Classification in Supervised Learning

Classification predicts a category instead of a continuous numerical value.

Classification Concept

Customer Information
        ↓
Features
        ↓
Classification Model
        ↓
Predicted Category

Example:

Age + Visits + Spending
        ↓
Purchase Model
        ↓
Purchase / No Purchase

16. Logistic Regression

Logistic Regression is a common supervised-learning algorithm for classification tasks.

Python

from sklearn.linear_model import LogisticRegression

classifier = LogisticRegression()

classifier.fit(
    X_train,
    y_train
)

predictions = classifier.predict(
    X_test
)

print(predictions)

17. Real-Time Example: Customer Purchase Prediction

An online store wants to predict whether a customer will purchase a product based on age, website visits, and previous spending.

Python

customers = pd.DataFrame({
    "Age": [
        20, 22, 25, 28,
        32, 35, 40, 45,
        50, 55
    ],

    "Visits": [
        1, 2, 3, 5,
        6, 7, 8, 10,
        12, 14
    ],

    "Spending": [
        100, 150, 200, 350,
        450, 500, 650, 800,
        1000, 1200
    ],

    "Purchased": [
        0, 0, 0, 1,
        1, 1, 1, 1,
        1, 1
    ]
})

X = customers[
    ["Age", "Visits", "Spending"]
]

y = customers["Purchased"]

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

classifier = LogisticRegression()

classifier.fit(
    X_train,
    y_train
)

new_customer = pd.DataFrame({
    "Age": [30],
    "Visits": [6],
    "Spending": [400]
})

prediction = classifier.predict(
    new_customer
)

print(
    "Predicted Purchase:",
    prediction[0]
)

Output Meaning: 0 can represent no purchase and 1 can represent purchase, based on how the target was defined.

18. Classification Accuracy

Python

from sklearn.metrics import accuracy_score

accuracy = accuracy_score(
    y_test,
    predictions
)

print(
    "Accuracy:",
    accuracy
)

Important: Accuracy alone may not adequately describe a classifier, especially when one class is much more common than another.

19. Confusion Matrix

A confusion matrix summarizes classification results by comparing actual classes with predicted classes.

Python

from sklearn.metrics import confusion_matrix

cm = confusion_matrix(
    y_test,
    predictions
)

print(cm)

Term	Meaning
True Positive	Positive case correctly predicted as positive
True Negative	Negative case correctly predicted as negative
False Positive	Negative case incorrectly predicted as positive
False Negative	Positive case incorrectly predicted as negative

20. Precision, Recall, and F1 Score

Python

from sklearn.metrics import (
    precision_score,
    recall_score,
    f1_score
)

precision = precision_score(
    y_test,
    predictions,
    zero_division=0
)

recall = recall_score(
    y_test,
    predictions,
    zero_division=0
)

f1 = f1_score(
    y_test,
    predictions,
    zero_division=0
)

print("Precision:", precision)
print("Recall:", recall)
print("F1 Score:", f1)

21. Real-Time Example: Student Result Prediction

A school can use historical study behavior to classify whether a student is likely to pass or fail.

Python

students = pd.DataFrame({
    "StudyHours": [
        1, 2, 2, 3, 4,
        5, 6, 7, 8, 9
    ],

    "Attendance": [
        55, 60, 65, 68, 72,
        78, 82, 88, 92, 95
    ],

    "Passed": [
        0, 0, 0, 0, 1,
        1, 1, 1, 1, 1
    ]
})

X = students[
    ["StudyHours", "Attendance"]
]

y = students["Passed"]

student_model = LogisticRegression()

student_model.fit(X, y)

new_student = pd.DataFrame({
    "StudyHours": [5],
    "Attendance": [80]
})

prediction = student_model.predict(
    new_student
)

print(
    "Predicted Result:",
    prediction[0]
)

22. Decision Tree Classifier

A Decision Tree learns a sequence of rules that can be used to make predictions.

Python

from sklearn.tree import DecisionTreeClassifier

tree_model = DecisionTreeClassifier(
    random_state=42,
    max_depth=3
)

tree_model.fit(
    X_train,
    y_train
)

predictions = tree_model.predict(
    X_test
)

print(predictions)

23. Random Forest Classifier

Random Forest combines predictions from multiple decision trees to build an ensemble model.

Python

from sklearn.ensemble import RandomForestClassifier

forest_model = RandomForestClassifier(
    n_estimators=100,
    random_state=42
)

forest_model.fit(
    X_train,
    y_train
)

predictions = forest_model.predict(
    X_test
)

print(predictions)

24. K-Nearest Neighbors

K-Nearest Neighbors makes predictions using nearby examples in the feature space.

Python

from sklearn.neighbors import KNeighborsClassifier

knn_model = KNeighborsClassifier(
    n_neighbors=3
)

knn_model.fit(
    X_train,
    y_train
)

predictions = knn_model.predict(
    X_test
)

print(predictions)

Note: Distance-based algorithms such as KNN often benefit from appropriate feature scaling.

25. Feature Scaling

Features can have very different numerical ranges. Standardization can make them comparable for algorithms that are sensitive to scale.

Python

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(
    X_train
)

X_test_scaled = scaler.transform(
    X_test
)

Best Practice: Fit the scaler using training data only, then transform validation, test, and future data using that fitted scaler.

26. Pipeline for Supervised Learning

A Pipeline keeps preprocessing and model training together and helps reduce accidental differences between training and prediction workflows.

Python

from sklearn.pipeline import Pipeline

pipeline = Pipeline([
    ("scaler", StandardScaler()),
    ("model", LogisticRegression())
])

pipeline.fit(
    X_train,
    y_train
)

predictions = pipeline.predict(
    X_test
)

print(predictions)

27. Real-Time Example: Loan Approval Classification

A financial application could classify historical loan applications using selected features and an approval target.

Python

loans = pd.DataFrame({
    "Income": [
        25000, 30000, 35000, 40000,
        45000, 50000, 60000, 70000
    ],

    "CreditScore": [
        550, 580, 610, 630,
        660, 700, 730, 760
    ],

    "LoanAmount": [
        200000, 180000, 160000, 170000,
        150000, 140000, 160000, 180000
    ],

    "Approved": [
        0, 0, 0, 1,
        1, 1, 1, 1
    ]
})

X = loans[
    ["Income", "CreditScore", "LoanAmount"]
]

y = loans["Approved"]

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.25,
    random_state=42
)

loan_model = LogisticRegression(
    max_iter=1000
)

loan_model.fit(
    X_train,
    y_train
)

new_application = pd.DataFrame({
    "Income": [48000],
    "CreditScore": [690],
    "LoanAmount": [150000]
})

prediction = loan_model.predict(
    new_application
)

print(
    "Predicted Approval:",
    prediction[0]
)

Real-World Consideration: A production lending system requires careful validation, appropriate features, monitoring, regulatory compliance, fairness analysis, and human oversight. A small teaching dataset is not suitable for real lending decisions.

28. Cross-Validation

Cross-validation evaluates a model across multiple training and validation splits instead of relying on one split.

Python

from sklearn.model_selection import cross_val_score
from sklearn.linear_model import LinearRegression

model = LinearRegression()

scores = cross_val_score(
    model,
    X,
    y,
    cv=5,
    scoring="r2"
)

print("Scores:", scores)
print("Average Score:", scores.mean())

29. Hyperparameters

Hyperparameters are settings selected before or during model development rather than learned directly as model parameters.

Algorithm	Example Hyperparameter
Decision Tree	max_depth
Random Forest	n_estimators
KNN	n_neighbors
Logistic Regression	C

30. Grid Search

Grid Search can evaluate combinations of selected hyperparameter values using cross-validation.

Python

from sklearn.model_selection import GridSearchCV
from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier(
    random_state=42
)

parameters = {
    "n_estimators": [50, 100],
    "max_depth": [2, 4, 6]
}

grid = GridSearchCV(
    model,
    parameters,
    cv=5,
    scoring="accuracy"
)

grid.fit(
    X_train,
    y_train
)

print(
    "Best Parameters:",
    grid.best_params_
)

31. Overfitting in Supervised Learning

Overfitting occurs when a model captures training-specific patterns too closely and does not generalize well to new data.

Concept

Training Performance
        ↓
Very High

Unseen Data Performance
        ↓
Much Lower

Possible Problem
        ↓
Overfitting

Possible approaches include simplifying the model, regularization, cross-validation, more representative data, and careful feature selection.

32. Underfitting

Underfitting occurs when a model is too simple to capture important patterns in the data.

Goal: Build a model that captures useful patterns while maintaining good performance on relevant unseen data.

33. Bias and Variance

High Bias

The model may be too simple and can lead to underfitting.

High Variance

The model may be too sensitive to training data and can lead to overfitting.

Generalization

The objective is to perform well on relevant unseen examples.

34. Real-Time Example: Customer Churn Prediction

A subscription company wants to classify whether a customer is likely to churn using usage and support information.

Python

customers = pd.DataFrame({
    "MonthlyUsage": [
        5, 8, 10, 12,
        18, 22, 28, 32
    ],

    "SupportTickets": [
        5, 4, 4, 3,
        3, 2, 1, 1
    ],

    "Churn": [
        1, 1, 1, 0,
        0, 0, 0, 0
    ]
})

X = customers[
    ["MonthlyUsage", "SupportTickets"]
]

y = customers["Churn"]

model = LogisticRegression()

model.fit(X, y)

new_customer = pd.DataFrame({
    "MonthlyUsage": [15],
    "SupportTickets": [3]
})

prediction = model.predict(
    new_customer
)

print(
    "Predicted Churn:",
    prediction[0]
)

Business Use: Such a model can support customer-retention analysis, but its predictions should be evaluated carefully before being used operationally.

35. Real-Time Example: Email Spam Classification

Spam filtering is a classification problem where messages are assigned to categories such as spam or not spam.

Workflow

Email
  ↓
Text Cleaning
  ↓
Feature Extraction
  ↓
Labeled Training Data
  ↓
Classification Model
  ↓
Spam / Not Spam

Text classification can use techniques such as TF-IDF features together with a suitable classifier.

36. Text Classification with TF-IDF

Python

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline

messages = [
    "Win a free prize today",
    "Meeting scheduled for tomorrow",
    "Claim your free reward now",
    "Project report is attached",
    "You have won a cash prize",
    "Please review the project report"
]

labels = [
    1,
    0,
    1,
    0,
    1,
    0
]

text_model = Pipeline([
    ("tfidf", TfidfVectorizer()),
    ("classifier", LogisticRegression())
])

text_model.fit(
    messages,
    labels
)

new_message = [
    "You won a free prize"
]

prediction = text_model.predict(
    new_message
)

print(
    "Predicted Class:",
    prediction[0]
)

Example Labels: 1 represents spam and 0 represents non-spam in this teaching example.

37. Real-Time Example: Medical Classification Concept

Supervised learning can also be used in healthcare-related research and decision-support systems when suitable labeled data is available.

Concept

Patient Features
      ↓
Training Data
      ↓
Supervised Classification Model
      ↓
Predicted Risk Category
      ↓
Professional Review

Important: A teaching model should not be treated as a medical diagnostic system. Real clinical applications require extensive validation, safety controls, and professional oversight.

38. Comparing Common Supervised Algorithms

Algorithm	Common Use	Example
Linear Regression	Regression	Sales prediction
Logistic Regression	Classification	Purchase prediction
Decision Tree	Regression / Classification	Customer classification
Random Forest	Regression / Classification	Risk classification
KNN	Classification / Regression	Similarity-based prediction

39. Real-Time Example: Complete Sales Prediction Project

The following example combines the main steps of a supervised regression project.

Complete Python Example

import pandas as pd

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import (
    mean_absolute_error,
    r2_score
)

# --------------------------------
# 1. Create historical data
# --------------------------------

data = {
    "Advertising": [
        10, 15, 20, 25, 30,
        35, 40, 45, 50, 55,
        60, 65
    ],

    "WebsiteVisits": [
        100, 130, 160, 190,
        220, 250, 290, 320,
        350, 390, 420, 450
    ],

    "Sales": [
        20, 27, 33, 39,
        45, 52, 60, 66,
        73, 81, 88, 95
    ]
}

df = pd.DataFrame(data)

# --------------------------------
# 2. Define features and target
# --------------------------------

X = df[
    ["Advertising", "WebsiteVisits"]
]

y = df["Sales"]

# --------------------------------
# 3. Split the data
# --------------------------------

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.25,
    random_state=42
)

# --------------------------------
# 4. Create model
# --------------------------------

model = LinearRegression()

# --------------------------------
# 5. Train model
# --------------------------------

model.fit(
    X_train,
    y_train
)

# --------------------------------
# 6. Predict test data
# --------------------------------

predictions = model.predict(
    X_test
)

# --------------------------------
# 7. Evaluate model
# --------------------------------

mae = mean_absolute_error(
    y_test,
    predictions
)

r2 = r2_score(
    y_test,
    predictions
)

print("Mean Absolute Error:", mae)
print("R² Score:", r2)

# --------------------------------
# 8. Predict future sales
# --------------------------------

future = pd.DataFrame({
    "Advertising": [70],
    "WebsiteVisits": [500]
})

future_sales = model.predict(
    future
)

print(
    "Predicted Future Sales:",
    future_sales[0]
)

40. Supervised Learning Workflow

Collect Data

→

Prepare Features

→

Split Data

→

Train

→

Predict

→

Evaluate

Define the problem.
Collect labeled historical data.
Explore and clean the dataset.
Select useful features and the target.
Split data into appropriate development sets.
Train one or more candidate models.
Evaluate using suitable metrics.
Tune the workflow when necessary.
Test on appropriate unseen data.
Deploy and monitor when the model is ready.

41. Important Supervised Learning Metrics

MAE

Average absolute error

RMSE

Penalizes larger errors

Accuracy

Classification correctness

Precision and recall balance

Choose metrics based on the problem. There is no single metric that is appropriate for every supervised-learning task.

42. Common Supervised Learning Problems

House Price

Regression problem.

Sales Forecasting

Regression problem.

Salary Prediction

Regression problem.

Spam Detection

Classification problem.

Customer Churn

Classification problem.

Purchase Prediction

Classification problem.

43. Common Mistakes in Supervised Learning

Training on data that should have been held out for evaluation.
Using features that contain information from the future target.
Ignoring missing or inconsistent data.
Using an unsuitable metric for the business problem.
Relying only on training performance.
Ignoring class imbalance in classification problems.
Applying preprocessing before the train/test split in a way that leaks information.
Using a very small or unrepresentative dataset to make strong conclusions.
Ignoring changes in real-world data after deployment.
Assuming that a model's prediction is automatically a causal explanation.

44. Supervised Learning Best Practices

Clearly define the prediction target.
Understand where the labels came from.
Use representative and relevant data.
Perform exploratory data analysis.
Separate training and evaluation data correctly.
Prevent data leakage.
Build a simple baseline before complex models.
Use cross-validation when appropriate.
Choose metrics that reflect the actual objective.
Inspect errors instead of looking only at one score.
Document preprocessing and model assumptions.
Monitor performance after deployment.

45. Supervised Learning Project Checklist

Question	Check
What are we predicting?	Target is clearly defined
What data is available?	Features and labels are understood
Is the data suitable?	Quality and representativeness are checked
How is the data split?	Evaluation data is protected from training
Which model?	Algorithm matches the problem
Which metric?	Metric reflects the real objective
Does it generalize?	Performance is tested on relevant unseen data
What happens after deployment?	Performance and data drift are monitored

46. Important Supervised Learning Rules

Supervised learning requires labeled examples for training.
Features are inputs and the target is the value being predicted.
Regression predicts numerical values.
Classification predicts categories.
Training and testing data should be kept appropriately separate.
Preprocessing must be designed to avoid data leakage.
Model performance should be evaluated on data not used for fitting.
Use metrics that match the problem and its costs.
Watch for overfitting and underfitting.
Start with a simple baseline before increasing model complexity.
Use cross-validation when it is appropriate for the dataset and task.
Real-world models require monitoring after deployment.

Key Idea: Supervised Learning learns from labeled historical examples and uses those learned patterns to make predictions on new data. The quality of the data, target definition, evaluation design, and business context are just as important as the algorithm itself.

Python Supervised Learning • Regression • Classification • Real-Time Examples