Regression in Data Analysis

A typical regression analysis follows these steps:

For example, an online real-estate company can use regression to estimate the price of a house from its area, number of bedrooms, age, and location.

Regression and classification are both supervised learning techniques, but they solve different types of problems.

• Simple Linear Regression: One input variable predicts one numerical output.
• Multiple Linear Regression: Multiple input variables predict one numerical output.
• Polynomial Regression: Models curved relationships between variables.
• Ridge Regression: Linear regression with L2 regularization.
• Lasso Regression: Linear regression with L1 regularization.
• Elastic Net: Combines L1 and L2 regularization.

Pandas and NumPy are useful for data preparation, while Matplotlib and scikit-learn are commonly used for regression analysis.

pip install pandas numpy matplotlib scikit-learn

Suppose a real-estate company wants to estimate house prices based on the size of the property.

data = {
    "area_sqft": [800, 1000, 1200, 1400, 1600, 1800, 2000, 2200],
    "price_lakh": [42, 50, 58, 67, 76, 85, 94, 103]
}

df = pd.DataFrame(data)

print(df)

In this example:

• X = area_sqft → input variable
• Y = price_lakh → target variable

X = df[["area_sqft"]]
y = df["price_lakh"]

print(X)
print(y)

Before building a regression model, it is useful to understand whether the input and target appear to have a relationship.

plt.scatter(df["area_sqft"], df["price_lakh"])

plt.xlabel("Area (sq ft)")
plt.ylabel("Price (Lakh)")
plt.title("House Area vs House Price")

plt.show()

Simple linear regression models the relationship between one input variable and one numerical output.

• ŷ = predicted value
• b₀ = intercept
• b₁ = slope
• x = input variable

The model attempts to find the line that best represents the observed data.

A model should normally be evaluated on data that was not used to train it.

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.25,
    random_state=42
)

print("Training rows:", len(X_train))
print("Testing rows:", len(X_test))

print("Intercept:", model.intercept_)
print("Coefficient:", model.coef_[0])

The coefficient represents the estimated change in the target value for a one-unit increase in the input variable, under the model.

Imagine that a customer asks for an estimated price for a 2,500 square-foot house.

new_house = pd.DataFrame({
    "area_sqft": [2500]
})

predicted_price = model.predict(new_house)

print("Predicted price:", predicted_price[0], "lakh")

The exact prediction depends on the coefficients learned from the training data.

comparison = pd.DataFrame({
    "Actual": y_test.values,
    "Predicted": predictions
})

print(comparison)

Comparing actual and predicted values helps analysts understand where the model is accurate and where it makes larger errors.

Mean Absolute Error measures the average absolute difference between actual and predicted values.

mae = mean_absolute_error(y_test, predictions)

print("MAE:", mae)

MSE calculates the average squared prediction error. Squaring makes larger errors contribute more strongly.

mse = mean_squared_error(y_test, predictions)

print("MSE:", mse)

RMSE is the square root of MSE and expresses the error in the same units as the target variable.

rmse = np.sqrt(mean_squared_error(y_test, predictions))

print("RMSE:", rmse)

R², or the coefficient of determination, describes how much of the variation in the target is explained by the regression model.

r2 = r2_score(y_test, predictions)

print("R² Score:", r2)

plt.scatter(X, y, label="Actual Data")

plt.plot(
    X,
    model.predict(X),
    label="Regression Line"
)

plt.xlabel("Area (sq ft)")
plt.ylabel("Price (Lakh)")
plt.title("Linear Regression")
plt.legend()

plt.show()

The scatter points represent observed data, while the line represents the values predicted by the regression model.

In real-world data analysis, one variable is often not enough. House price may depend on area, bedrooms, house age, and parking.

data = {
    "area": [800, 1000, 1200, 1400, 1600, 1800, 2000, 2200],
    "bedrooms": [2, 2, 3, 3, 3, 4, 4, 4],
    "age": [20, 15, 12, 10, 8, 6, 4, 2],
    "price": [42, 50, 58, 67, 76, 85, 94, 103]
}

df = pd.DataFrame(data)

X = df[["area", "bedrooms", "age"]]
y = df["price"]

model = LinearRegression()
model.fit(X, y)

print("Coefficients:", model.coef_)
print("Intercept:", model.intercept_)

A new property can be described using several characteristics.

new_house = pd.DataFrame({
    "area": [1800],
    "bedrooms": [3],
    "age": [5]
})

predicted_price = model.predict(new_house)

print("Estimated price:", predicted_price[0], "lakh")

An e-commerce company wants to estimate monthly sales based on advertising spending.

data = {
    "advertising": [10, 15, 20, 25, 30, 35, 40],
    "sales": [110, 140, 170, 205, 235, 270, 300]
}

df = pd.DataFrame(data)

X = df[["advertising"]]
y = df["sales"]

model = LinearRegression()
model.fit(X, y)

new_campaign = pd.DataFrame({
    "advertising": [45]
})

predicted_sales = model.predict(new_campaign)

print("Predicted sales:", predicted_sales[0])

This type of model can help analysts study the relationship between advertising expenditure and sales.

A company can analyze whether years of experience are associated with employee salary.

data = {
    "experience": [1, 2, 3, 4, 5, 7, 9, 11],
    "salary": [3.2, 3.8, 4.5, 5.2, 6.1, 7.8, 9.5, 11.2]
}

df = pd.DataFrame(data)

X = df[["experience"]]
y = df["salary"]

model = LinearRegression()
model.fit(X, y)

new_employee = pd.DataFrame({
    "experience": [6]
})

salary = model.predict(new_employee)

print("Estimated salary:", salary[0], "LPA")

A digital marketing team wants to estimate website visits based on advertising expenditure.

data = {
    "ad_spend": [5, 10, 15, 20, 25, 30],
    "visits": [1200, 2100, 3000, 3900, 4850, 5700]
}

df = pd.DataFrame(data)

X = df[["ad_spend"]]
y = df["visits"]

model = LinearRegression()
model.fit(X, y)

next_month = pd.DataFrame({
    "ad_spend": [35]
})

predicted_visits = model.predict(next_month)

print("Predicted visits:", predicted_visits[0])

A utility company may analyze electricity consumption based on temperature, household size, or historical usage.

data = {
    "temperature": [20, 22, 24, 26, 28, 30, 32],
    "usage_kwh": [180, 190, 205, 225, 250, 280, 315]
}

df = pd.DataFrame(data)

X = df[["temperature"]]
y = df["usage_kwh"]

model = LinearRegression()
model.fit(X, y)

tomorrow = pd.DataFrame({
    "temperature": [31]
})

usage = model.predict(tomorrow)

print("Estimated electricity usage:", usage[0], "kWh")

A residual is the difference between an observed value and its predicted value.

residuals = y_test - predictions

print(residuals)

Analysts can inspect residuals to identify patterns that may indicate that a simple regression model does not adequately capture the data.

Not every relationship is a straight line. Polynomial regression can represent certain curved relationships.

from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import make_pipeline

model = make_pipeline(
    PolynomialFeatures(degree=2),
    LinearRegression()
)

model.fit(X, y)

predictions = model.predict(X)

print(predictions)

In a real project, data will usually come from files or databases rather than being manually entered.

df = pd.read_csv("house_prices.csv")

X = df[["area", "bedrooms", "age"]]
y = df["price"]

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

model = LinearRegression()
model.fit(X_train, y_train)

predictions = model.predict(X_test)

print("R²:", r2_score(y_test, predictions))

Correlation can help analysts explore whether numerical variables move together, although correlation alone does not establish causation.

print(df.corr(numeric_only=True))

Real datasets may contain many columns. Analysts should select variables that are relevant to the question being studied.

Real datasets often contain categories such as city, property type, or product category. These values need appropriate encoding before many regression algorithms can use them.

df = pd.get_dummies(
    df,
    columns=["city"],
    drop_first=True
)

print(df.head())

One-hot encoding converts categorical values into numerical indicator columns.

A retail company can build a regression model to estimate product demand using historical sales, discount percentage, advertising expenditure, and product price.

data = {
    "price": [100, 95, 90, 85, 80, 75],
    "discount": [0, 5, 10, 15, 20, 25],
    "ad_spend": [5, 8, 10, 12, 15, 18],
    "demand": [80, 95, 110, 130, 155, 180]
}

df = pd.DataFrame(data)

X = df[["price", "discount", "ad_spend"]]
y = df["demand"]

model = LinearRegression()
model.fit(X, y)

new_product = pd.DataFrame({
    "price": [78],
    "discount": [20],
    "ad_spend": [16]
})

prediction = model.predict(new_product)

print("Estimated demand:", prediction[0])

Ridge regression adds L2 regularization to the linear regression objective. It can be useful when predictors are highly correlated or when controlling coefficient magnitude is desirable.

from sklearn.linear_model import Ridge

model = Ridge(alpha=1.0)

model.fit(X_train, y_train)

predictions = model.predict(X_test)

print("R²:", r2_score(y_test, predictions))

Lasso regression uses L1 regularization. Under suitable conditions, it can shrink some coefficients to zero, which can be useful for feature selection.

from sklearn.linear_model import Lasso

model = Lasso(alpha=0.1)

model.fit(X_train, y_train)

predictions = model.predict(X_test)

print("R²:", r2_score(y_test, predictions))

A regression model should be evaluated on data that represents the situation in which the model will actually be used.

Cross-validation repeatedly divides available training data into training and validation portions to provide a more robust estimate of model performance.

from sklearn.model_selection import cross_val_score

scores = cross_val_score(
    LinearRegression(),
    X,
    y,
    cv=5,
    scoring="r2"
)

print("R² scores:", scores)
print("Average R²:", scores.mean())

An online store wants to estimate annual customer spending from the number of purchases, average order value, and website visits.

data = {
    "purchases": [5, 8, 10, 12, 15, 18, 20],
    "avg_order": [800, 750, 900, 850, 950, 1000, 1050],
    "visits": [20, 30, 35, 42, 50, 58, 65],
    "annual_spending": [4000, 6000, 9000, 10500, 14500, 18500, 22000]
}

df = pd.DataFrame(data)

X = df[["purchases", "avg_order", "visits"]]
y = df["annual_spending"]

model = LinearRegression()
model.fit(X, y)

customer = pd.DataFrame({
    "purchases": [16],
    "avg_order": [980],
    "visits": [55]
})

spending = model.predict(customer)

print("Estimated annual spending:", spending[0])

The following example combines data preparation, model training, prediction, and evaluation into one workflow.

import pandas as pd
import numpy as np

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

# 1. Create business data
data = {
    "ad_spend": [10, 15, 20, 25, 30, 35, 40, 45, 50, 55],
    "discount": [2, 3, 5, 5, 7, 8, 10, 10, 12, 15],
    "website_visits": [1000, 1200, 1450, 1600, 1850,
                       2100, 2350, 2600, 2850, 3100],
    "sales": [100, 125, 150, 170, 195, 220, 250, 275, 305, 330]
}

df = pd.DataFrame(data)

# 2. Select features and target
X = df[["ad_spend", "discount", "website_visits"]]
y = df["sales"]

# 3. Split the data
X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

# 4. Create the model
model = LinearRegression()

# 5. Train the model
model.fit(X_train, y_train)

# 6. Make predictions
predictions = model.predict(X_test)

# 7. Evaluate
mae = mean_absolute_error(y_test, predictions)
mse = mean_squared_error(y_test, predictions)
rmse = np.sqrt(mse)
r2 = r2_score(y_test, predictions)

print("MAE :", mae)
print("MSE :", mse)
print("RMSE:", rmse)
print("R²  :", r2)

# 8. Predict future sales
future_data = pd.DataFrame({
    "ad_spend": [60],
    "discount": [15],
    "website_visits": [3400]
})

future_sales = model.predict(future_data)

print("Predicted future sales:", future_sales[0])

1. Understand the business question.
2. Identify the numerical target variable.
3. Collect relevant historical data.
4. Inspect and clean missing or invalid values.
5. Explore relationships between variables.
6. Select appropriate features.
7. Split data into training and testing sets.
8. Train the regression model.
9. Generate predictions.
10. Measure MAE, MSE, RMSE, or R² as appropriate.
11. Inspect residuals and model assumptions.
12. Validate the model on suitable unseen data.
13. Interpret results in the context of the business problem.

• Overfitting: The model captures noise instead of general patterns.
• Underfitting: The model is too simple to capture important patterns.
• Outliers: Extreme observations can strongly affect some regression models.
• Multicollinearity: Predictors may contain highly overlapping information.
• Data leakage: Information unavailable at prediction time may accidentally enter the model.
• Non-linearity: A straight-line model may not represent the actual relationship.
• Changing variance: Prediction errors may behave differently across the range of predictions.

• Understand the target variable before selecting a model.
• Clean missing, duplicate, and invalid records.
• Inspect distributions and relationships visually.
• Keep training and evaluation data separate.
• Check for data leakage.
• Use cross-validation when appropriate.
• Compare suitable evaluation metrics.
• Inspect residuals and model diagnostics.
• Consider regularization when appropriate.
• Do not interpret correlation as proof of causation.
• Use domain knowledge when interpreting coefficients and predictions.
• Monitor model performance when deployed on changing data.

Example Project

Project: Predict monthly product sales for an e-commerce company.

Features: advertising spend, product price, discount, website traffic, number of promotions, and previous sales.

Target: Monthly sales quantity or revenue.

Business Use: Support inventory planning, campaign analysis, and sales forecasting.

• Regression is primarily used for predicting or explaining numerical outcomes.
• Always identify the target variable clearly.
• Separate training data from evaluation data.
• Use multiple evaluation metrics when appropriate.
• Inspect residuals and unusual observations.
• Do not assume that a high R² proves a model is useful.
• Correlation does not by itself prove causation.
• Choose features based on the prediction task and domain context.
• Watch for data leakage and overfitting.
• Validate the model using data representative of future use.
• Interpret predictions together with business and domain knowledge.

Python Data Analysis - Regression

Learn regression concepts through practical Python and real-world examples.