Clustering in Data Analysis

A typical clustering project starts with raw data, prepares the numerical features, applies a clustering algorithm, and then interprets the resulting groups.

Clustering and classification both group data, but they solve different problems.

For practical clustering analysis in Python, Pandas, NumPy, Matplotlib, and Scikit-learn are commonly used.

pip install pandas numpy matplotlib scikit-learn

Suppose an online store wants to divide customers into groups based on their annual income and spending score.

data = {
    "Customer": ["A", "B", "C", "D", "E", "F",
                 "G", "H", "I", "J"],
    "Annual_Income": [25, 28, 32, 70, 75, 80, 120, 125, 130, 135],
    "Spending_Score": [85, 90, 80, 45, 50, 42, 20, 25, 18, 22]
}

df = pd.DataFrame(data)

print(df)

Clustering algorithms work with numerical features. We select the columns that describe customer behavior.

X = df[["Annual_Income", "Spending_Score"]]

print(X)

Clustering methods such as K-Means use distances between data points. Features with very different scales can therefore influence the result unevenly.

scaler = StandardScaler()

X_scaled = scaler.fit_transform(X)

print(X_scaled)

K-Means requires us to specify how many clusters we want. Suppose the business wants four customer segments.

model = KMeans(
    n_clusters=4,
    random_state=42,
    n_init=10
)

model.fit(X_scaled)

df["Cluster"] = model.labels_

print(df)

Cluster numbers themselves do not have business meaning. For example, Cluster 0 is not automatically "better" than Cluster 1.

We need to calculate summary statistics for each cluster and interpret the customer behavior.

cluster_summary = df.groupby("Cluster")[
    ["Annual_Income", "Spending_Score"]
].mean()

print(cluster_summary)

Choosing the number of clusters is an important part of K-Means. The Elbow Method is commonly used to explore suitable values of K.

inertia = []

for k in range(1, 10):
    model = KMeans(
        n_clusters=k,
        random_state=42,
        n_init=10
    )

    model.fit(X_scaled)
    inertia.append(model.inertia_)

print(inertia)

We can plot these values and look for a point where the reduction in within-cluster variation starts to slow down.

plt.scatter(
    df["Annual_Income"],
    df["Spending_Score"],
    c=df["Cluster"]
)

plt.xlabel("Annual Income")
plt.ylabel("Spending Score")
plt.title("Customer Clusters")

plt.show()

A scatter plot makes it easier to see how customers are distributed across the discovered groups.

K-Means creates a centroid for each cluster. A centroid represents the center of the corresponding group.

centers = model.cluster_centers_

print(centers)

The centers can help analysts understand the typical characteristics of each segment.

An e-commerce company has thousands of customers. Instead of sending the same marketing campaign to everyone, the company can use clustering to identify customer groups.

The marketing team can then study each group and design different campaigns based on observed behavior.

A retailer can group products using features such as price, sales volume, customer rating, and return rate.

products = pd.DataFrame({
    "Product": ["A", "B", "C", "D", "E"],
    "Price": [20, 25, 200, 220, 210],
    "Sales": [500, 450, 80, 70, 90],
    "Rating": [4.2, 4.4, 4.7, 4.8, 4.6]
})

features = products[["Price", "Sales", "Rating"]]

scaler = StandardScaler()
scaled_features = scaler.fit_transform(features)

model = KMeans(
    n_clusters=2,
    random_state=42,
    n_init=10
)

products["Cluster"] = model.fit_predict(scaled_features)

print(products)

This can help analysts discover groups such as lower-priced high-volume products and premium products.

A company operating multiple stores can group locations based on sales, customer count, average transaction value, and operating cost.

stores = pd.DataFrame({
    "Store": ["S1", "S2", "S3", "S4", "S5"],
    "Sales": [120000, 135000, 45000, 52000, 125000],
    "Customers": [5000, 5500, 1800, 2100, 5100],
    "Avg_Bill": [24, 25, 25, 24, 26]
})

features = stores[["Sales", "Customers", "Avg_Bill"]]

scaled = StandardScaler().fit_transform(features)

model = KMeans(
    n_clusters=2,
    random_state=42,
    n_init=10
)

stores["Cluster"] = model.fit_predict(scaled)

print(stores)

Hierarchical clustering builds a tree-like structure of related observations. It is useful when analysts want to inspect relationships between groups at multiple levels.

from sklearn.cluster import AgglomerativeClustering

model = AgglomerativeClustering(
    n_clusters=3
)

labels = model.fit_predict(X_scaled)

df["Hierarchical_Cluster"] = labels

print(df)

DBSCAN groups points based on density. Unlike K-Means, it can identify some observations as noise.

from sklearn.cluster import DBSCAN

model = DBSCAN(
    eps=0.5,
    min_samples=3
)

labels = model.fit_predict(X_scaled)

df["DBSCAN_Cluster"] = labels

print(df)

Clustering can be used as part of exploratory fraud analysis. Transactions with unusual combinations of amount, frequency, location, or timing may form small or isolated groups.

transactions = pd.DataFrame({
    "Amount": [100, 120, 90, 110, 105, 5000, 5200],
    "Frequency": [2, 3, 2, 4, 3, 1, 1]
})

features = transactions[["Amount", "Frequency"]]

scaled = StandardScaler().fit_transform(features)

model = DBSCAN(
    eps=0.8,
    min_samples=2
)

transactions["Cluster"] = model.fit_predict(scaled)

print(transactions)

The silhouette score measures how similar an observation is to its own cluster compared with other clusters.

Higher values generally indicate that observations are relatively well separated from other clusters, while values around zero can indicate overlapping groups.

from sklearn.metrics import silhouette_score

score = silhouette_score(
    X_scaled,
    df["Cluster"]
)

print("Silhouette Score:", score)

for k in range(2, 7):

    model = KMeans(
        n_clusters=k,
        random_state=42,
        n_init=10
    )

    labels = model.fit_predict(X_scaled)

    score = silhouette_score(
        X_scaled,
        labels
    )

    print(k, score)

This lets an analyst compare different cluster counts using a quantitative validation measure.

After clustering, Pandas can be used to summarize the characteristics of each group.

summary = df.groupby("Cluster").agg({
    "Annual_Income": "mean",
    "Spending_Score": "mean",
    "Customer": "count"
})

summary = summary.rename(
    columns={"Customer": "Customer_Count"}
)

print(summary)

An analytics team can cluster website visitors using:

• Pages viewed
• Session duration
• Number of sessions
• Products viewed
• Purchases

visitors = pd.DataFrame({
    "Pages": [3, 4, 20, 18, 5, 6],
    "Minutes": [2, 3, 25, 30, 5, 6],
    "Purchases": [0, 0, 4, 5, 1, 1]
})

features = visitors[
    ["Pages", "Minutes", "Purchases"]
]

scaled = StandardScaler().fit_transform(features)

model = KMeans(
    n_clusters=3,
    random_state=42,
    n_init=10
)

visitors["Cluster"] = model.fit_predict(scaled)

print(visitors)

The resulting groups may reveal browsing-only visitors, highly engaged visitors, and customers with purchase behavior.

HR analytics teams can explore employee groups using variables such as performance score, training hours, experience, and project count.

employees = pd.DataFrame({
    "Employee": ["E1", "E2", "E3", "E4", "E5"],
    "Performance": [72, 75, 92, 95, 68],
    "Training_Hours": [10, 12, 30, 35, 8],
    "Projects": [2, 3, 6, 7, 2]
})

features = employees[
    ["Performance", "Training_Hours", "Projects"]
]

scaled = StandardScaler().fit_transform(features)

model = KMeans(
    n_clusters=2,
    random_state=42,
    n_init=10
)

employees["Cluster"] = model.fit_predict(scaled)

print(employees)

Real-world data is often stored in CSV files.

import pandas as pd

df = pd.read_csv("customers.csv")

print(df.head())

After loading the data, select appropriate numerical features, clean missing values, scale the data, and then apply the clustering algorithm.

Most clustering algorithms require usable numerical input. Missing values should therefore be handled before training.

features = df[
    ["Annual_Income", "Spending_Score"]
].copy()

features["Annual_Income"] = features[
    "Annual_Income"
].fillna(
    features["Annual_Income"].median()
)

features["Spending_Score"] = features[
    "Spending_Score"
].fillna(
    features["Spending_Score"].median()
)

Clustering is not limited to two dimensions. For example, customers can be grouped using income, spending score, and purchase frequency.

features = df[
    [
        "Annual_Income",
        "Spending_Score",
        "Purchase_Frequency"
    ]
]

scaled = StandardScaler().fit_transform(features)

model = KMeans(
    n_clusters=4,
    random_state=42,
    n_init=10
)

df["Cluster"] = model.fit_predict(scaled)

print(df)

The following example combines data preparation, scaling, clustering, and cluster analysis into one workflow.

import pandas as pd

from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score


# 1. Create customer data
df = pd.DataFrame({
    "Customer": [
        "C1", "C2", "C3", "C4", "C5",
        "C6", "C7", "C8", "C9", "C10"
    ],
    "Income": [
        25, 28, 30, 45, 48,
        50, 80, 85, 90, 95
    ],
    "Spending": [
        85, 90, 88, 60, 65,
        62, 25, 20, 18, 22
    ]
})


# 2. Select features
features = df[
    ["Income", "Spending"]
]


# 3. Scale features
scaler = StandardScaler()

scaled_features = scaler.fit_transform(features)


# 4. Train clustering model
model = KMeans(
    n_clusters=3,
    random_state=42,
    n_init=10
)


# 5. Assign clusters
df["Cluster"] = model.fit_predict(
    scaled_features
)


# 6. Calculate silhouette score
score = silhouette_score(
    scaled_features,
    df["Cluster"]
)


# 7. Display customers
print(df)


# 8. Display cluster summary
summary = df.groupby("Cluster").agg({
    "Income": "mean",
    "Spending": "mean",
    "Customer": "count"
})

print(summary)

print("Silhouette Score:", score)

These descriptions are business interpretations rather than labels produced automatically by K-Means. The analyst must examine the cluster statistics before assigning meaningful names.

Marketing teams can use clustering to understand groups of customers with similar behavior.

• Identify groups with similar purchasing behavior.
• Analyze customer engagement patterns.
• Create different campaign strategies for different groups.
• Study high-frequency and low-frequency customers.
• Discover previously unknown customer patterns.

E-commerce platforms can cluster customers using features such as order frequency, average order value, product categories, and website activity.

ecommerce = pd.DataFrame({
    "Orders": [2, 3, 10, 12, 1, 2],
    "Average_Order": [500, 600, 2200, 2500, 300, 350],
    "Website_Visits": [5, 7, 30, 35, 2, 4]
})

features = ecommerce[
    ["Orders", "Average_Order", "Website_Visits"]
]

scaled = StandardScaler().fit_transform(features)

model = KMeans(
    n_clusters=3,
    random_state=42,
    n_init=10
)

ecommerce["Cluster"] = model.fit_predict(scaled)

print(ecommerce)

Clustering can be used as one component of a recommendation system.

Users with similar browsing or purchasing behavior can be placed into the same group. Products that are popular within a user's group can then become candidates for recommendations.

Products can be grouped according to demand, revenue, inventory turnover, and stock levels.

The resulting clusters can help analysts investigate different inventory behavior patterns.

The appropriate algorithm depends on the structure of the data, the business objective, and the assumptions of the chosen method.

• Choosing an unsuitable number of clusters.
• Using irrelevant features.
• Ignoring feature scaling.
• Allowing extreme outliers to distort distance-based groups.
• Using categorical data without appropriate encoding.
• Creating clusters without business interpretation.
• Assuming cluster labels have an inherent order.
• Ignoring the quality and completeness of the source data.

• Clean the dataset before clustering.
• Understand the business problem first.
• Select meaningful features.
• Scale features when the algorithm is distance-sensitive.
• Test more than one clustering configuration.
• Use metrics such as silhouette score where appropriate.
• Visualize the resulting clusters.
• Summarize each cluster using descriptive statistics.
• Give clusters meaningful business descriptions only after analysis.
• Validate whether the discovered groups are useful in practice.

This workflow can be applied to customer data, products, transactions, employees, website visitors, stores, and many other datasets.

Imagine an e-commerce company wants to understand its customers using three behavioral measures:

• Annual spending
• Number of orders
• Website visits

import pandas as pd
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans

customers = pd.DataFrame({
    "Customer": ["C1", "C2", "C3", "C4", "C5", "C6"],
    "Annual_Spending": [12000, 14000, 3000, 3500, 25000, 27000],
    "Orders": [18, 20, 4, 5, 30, 32],
    "Website_Visits": [60, 70, 15, 18, 100, 110]
})

features = customers[
    ["Annual_Spending", "Orders", "Website_Visits"]
]

scaler = StandardScaler()

scaled = scaler.fit_transform(features)

model = KMeans(
    n_clusters=3,
    random_state=42,
    n_init=10
)

customers["Cluster"] = model.fit_predict(scaled)

summary = customers.groupby("Cluster").agg({
    "Annual_Spending": "mean",
    "Orders": "mean",
    "Website_Visits": "mean",
    "Customer": "count"
})

print(customers)
print(summary)

• Understand the business question.
• Collect relevant data.
• Inspect missing values and duplicates.
• Select meaningful numerical features.
• Handle categorical variables appropriately.
• Scale features when required.
• Test appropriate clustering algorithms.
• Explore different cluster counts.
• Validate the clusters.
• Profile each cluster.
• Visualize the results.
• Connect clusters to business decisions carefully.

Clustering in Data Analysis

Python Data Analysis • Machine Learning • Real-Time Examples