Linear regression (the math intuition)

The idea

Linear regression predicts a number as a weighted sum of the features plus a constant:

units = w₁·temperature + w₂·weekend + w₃·promo + w₄·ad_spend + b

It's the X @ w + b from the NumPy linear-algebra module, and the "model + loss" from two modules ago, with one special property: the weights with the lowest MSE can be computed exactly, with a formula, no searching needed. It's the first model to try for any regression problem, and often the one that ships, because its coefficients explain themselves.

One feature: a line

Start with the simplest version of the question: does spending more on ads sell more units? With one feature, a linear model is just a straight line through the points, and NumPy can find the best-fitting line directly:

import numpy as np
import pandas as pd

demand = pd.read_csv("daily_demand.csv")
x = demand["ad_spend"].to_numpy()
y = demand["units"].to_numpy()

w, b = np.polyfit(x, y, 1)
print(round(w, 2), round(b, 2))

7.51 44.56

The slope w means: each extra million Rupiah of ads goes with about that many more units sold. The intercept b is the predicted units with zero ad spend.

How is it computed? Least squares has a closed-form answer. For one feature it's short enough to write out:

w_formula = np.sum((x - x.mean()) * (y - y.mean())) / np.sum((x - x.mean()) ** 2)
b_formula = y.mean() - w_formula * x.mean()
print(round(w_formula, 2), round(b_formula, 2))

7.51 44.56

Same answer. The slope is "how much x and y move together" divided by "how much x moves", and the line always passes through the point (mean x, mean y).

Several features: a plane

With several features, NumPy solves the least-squares problem with np.linalg.lstsq. Add a column of ones so the model can learn the intercept:

features = ["temperature", "weekend", "promo", "ad_spend"]
X = demand[features].to_numpy()
X1 = np.column_stack([X, np.ones(len(X))])          # last column: the intercept
coefs, *_ = np.linalg.lstsq(X1, y, rcond=None)
for name, c in zip(features + ["intercept"], coefs):
    print(f"{name:12} {c:8.2f}")

temperature     -1.67
weekend          5.75
promo           18.92
ad_spend         7.62
intercept       85.75

Each coefficient is an effect holding the other features fixed:

• promo ≈ +19: a promotion day sells about 19 more units than an otherwise identical day.
• weekend ≈ +6 units.
• ad_spend ≈ +7.6 units per million Rupiah.
• temperature ≈ −1.7: every extra degree, slightly fewer hot coffees.

That's the attraction of linear regression: a manager can read it. "Promotions are worth about 19 units a day" is a sentence you can act on (compare with what a promotion costs).

The same thing with scikit-learn

In practice you use scikit-learn, which has the same interface for every model: fit, then predict:

from sklearn.linear_model import LinearRegression

model = LinearRegression()
model.fit(demand[features], y)
print(np.round(model.coef_, 2), round(model.intercept_, 2))

tomorrow = pd.DataFrame({"temperature": [29.0], "weekend": [1], "promo": [1], "ad_spend": [2.0]})
print(model.predict(tomorrow).round(1))

[-1.67  5.75 18.92  7.62] 85.75
[77.2]

Identical coefficients to lstsq. predict turns a new day's features into a forecast: a hot Saturday with a promotion and Rp 2M of ads.

R²: how much is explained?

model.score(X, y) returns R²: the share of the variation in y that the model explains (1.0 = perfect, 0 = no better than predicting the average).

print(round(model.score(demand[features], y), 3))

0.811

The regression metrics module explains R² properly. Here it says these four features explain most of the day-to-day variation in sales.

Check the residuals

A model can have a decent score and still be systematically wrong. Look at the residuals (actual − predicted):

residuals = y - model.predict(demand[features])
print("average is ~0:", bool(np.isclose(residuals.mean(), 0)), "| typical size:", round(residuals.std(), 2))
by_weekend = pd.Series(residuals).groupby(demand["weekend"]).mean()
print("weekday and weekend averages ~0:", bool(np.allclose(by_weekend, 0)))

average is ~0: True | typical size: 4.98
weekday and weekend averages ~0: True

Good signs: residuals average zero overall and within groups (no group is consistently over- or under-predicted), and their spread (about ±5 units) matches the noise you'd expect. A pattern in the residuals (for example, always under-predicting on hot days) means the model is missing something: a curved relationship, an interaction, or a feature.

Limitations

• It assumes effects add up and are straight lines. Curves and interactions need extra features (e.g. temperature², promo × weekend) or other models (trees).
• Coefficients describe association, not proof of cause: maybe ads run on days that were going to be busy anyway.
• Outliers pull the line strongly, because errors are squared.
• Correlated features (two measures of the same thing) make individual coefficients unstable; regularisation helps (later module).

Why this matters for clean code

Linear regression is the baseline every other regression model must beat, and the most explainable one you'll ever deploy. Keep the feature list explicit, print coefficients with their names, and always check residuals. A complex model that beats it by 1% often isn't worth losing the explanation.

Recap

• Linear regression: prediction = weighted sum of features + intercept; weights minimise MSE.
• One feature: np.polyfit(x, y, 1); many: np.linalg.lstsq with a column of ones, or LinearRegression().fit(X, y).
• Coefficients = effect of a feature with the others held fixed, in its units.
• model.predict(new_X) forecasts; model.score gives R².
• Check residuals: centred on zero, no patterns by group. Mind curves, causality, outliers and correlated features.