Lesson 19: Programming for Automation

Two tasks from Lesson 18. Think through your answers first, then reveal.

The NESA syllabus says to build ML models "using an OOP." Before you write any model code, notice that you've already been doing OOP every time you use scikit-learn — you just hadn't thought about it that way.

Every scikit-learn model follows the same OOP pattern. You instantiate an object, call its methods, and read its attributes:

from sklearn.linear_model import LinearRegression

# Instantiate — creates a LinearRegression object with chosen hyperparameters
model = LinearRegression()

# fit() — trains the model (modifies the object's internal state)
model.fit(X_train, y_train)

# Attributes set by fit() — the model's learned knowledge
print(model.coef_)        # slope(s)
print(model.intercept_)   # y-intercept

# predict() — uses learned knowledge to answer new questions
predictions = model.predict(X_test)

# score() — evaluates accuracy on unseen data
r2 = model.score(X_test, y_test)

OOP concepts you already know — applied to ML

Everything scikit-learn does is OOP by design. Spot the principles:

Train/test split — the rule you can't skip

Testing a model on the same data it trained on is like marking your own exam paper. You'd give yourself 100% — but that tells you nothing about whether you actually learned. The test set is data the model has never seen during training. Its score is the only honest measure of performance.

from sklearn.model_selection import train_test_split

# 80% for training, 20% for testing
# random_state=42 makes the split the same every run (reproducible)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

You'll use this before training every model today.

NESA Linear regression predicts a continuous numeric value by fitting a straight line through the data.

The line is: y = mx + b — where m (slope) and b (intercept) are what the model learns from your training data.

Classic question: "If I study for X hours, what exam score should I expect?"

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

# ── Data ──────────────────────────────────────────────────────────────
study_hours = np.array([1, 2, 2.5, 3, 4, 4.5, 5, 6, 6.5, 7, 8, 8.5])
exam_scores = np.array([42, 50, 53, 58, 64, 68, 73, 79, 82, 85, 90, 93])

# reshape(-1, 1): scikit-learn expects features as a 2D array (rows × columns)
X = study_hours.reshape(-1, 1)
y = exam_scores

# ── Split ─────────────────────────────────────────────────────────────
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

# ── Train ─────────────────────────────────────────────────────────────
model = LinearRegression()
model.fit(X_train, y_train)

# ── Evaluate ──────────────────────────────────────────────────────────
print(f"R² score:      {model.score(X_test, y_test):.3f}")

# ── Predict ───────────────────────────────────────────────────────────
print(f"Predicted score for 5.5 hours: {model.predict([[5.5]])[0]:.1f}")

# ── Inspect what the model learned ────────────────────────────────────
print(f"Slope (m):     {model.coef_[0]:.2f}  — score gain per hour studied")
print(f"Intercept (b): {model.intercept_:.2f} — predicted score at 0 hours")

Visualise the result

Add this block after the code above. Training points are shown in blue, test points in orange, and the fitted line in red:

import matplotlib.pyplot as plt
import numpy as np

# ── Visualise ─────────────────────────────────────────────────────────────
x_line = np.linspace(X.min(), X.max(), 100).reshape(-1, 1)
y_line = model.predict(x_line)

plt.figure(figsize=(8, 5))
plt.scatter(X_train, y_train, color='steelblue', label='Training data', zorder=3)
plt.scatter(X_test,  y_test,  color='orange', marker='D', label='Test data', zorder=3)
plt.plot(x_line, y_line, color='crimson', linewidth=2,
         label=f'Fitted line  (R²={model.score(X_test, y_test):.3f})')
plt.xlabel('Study hours')
plt.ylabel('Exam score')
plt.title('Linear Regression — Study Hours vs Exam Score')
plt.legend()
plt.tight_layout()
plt.show()

Orange diamonds are test points — they were never seen during training. The closer they sit to the line, the higher the R².

What is R²?

R² (R-squared) answers: "How much of the variation in exam scores does my model explain?"

NESA Linear regression only fits straight lines. But real relationships are often curved.

Consider: how many hours of sleep gives you the best exam performance? Too few and you're exhausted. Too many and you're groggy. The relationship peaks somewhere in the middle — that's a curve, not a line.

How it works: the PolynomialFeatures trick

There's no separate "polynomial regression" class. Instead, you use PolynomialFeatures to transform your input first — then pass the transformed features into an ordinary LinearRegression.

For degree 2: it takes your feature [x] and expands it to [1, x, x²]. Now a linear model fitting those three columns is actually fitting a curve.

import numpy as np
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

# ── Data — sleep hours vs exam performance ────────────────────────────
sleep_hours  = np.array([3, 4, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 10, 11])
performance  = np.array([42, 50, 60, 65, 72, 78, 85, 87, 84, 80, 74, 62, 50])

X = sleep_hours.reshape(-1, 1)
y = performance

# ── Split ─────────────────────────────────────────────────────────────
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

# ── Transform features — expand [x] to [1, x, x²] ────────────────────
poly = PolynomialFeatures(degree=2)          # degree=2 fits a parabola
X_train_poly = poly.fit_transform(X_train)  # fit_transform on training data
X_test_poly  = poly.transform(X_test)       # transform (NOT fit_transform) on test data

# ── Train — ordinary LinearRegression on the expanded features ────────
model = LinearRegression()
model.fit(X_train_poly, y_train)

# ── Evaluate ──────────────────────────────────────────────────────────
print(f"R² score: {model.score(X_test_poly, y_test):.3f}")

# ── Predict — must transform new input with the same poly object ──────
new_hours = poly.transform([[7.5]])
print(f"Predicted performance for 7.5h sleep: {model.predict(new_hours)[0]:.1f}")

Visualise the result — curve vs straight line

Add this block after the code above. It plots both a straight line and the polynomial curve on the same chart so you can see exactly why degree=2 wins here:

import matplotlib.pyplot as plt
import numpy as np
from sklearn.linear_model import LinearRegression

# ── Visualise — linear vs polynomial fit ─────────────────────────────────
x_range = np.linspace(X.min(), X.max(), 200).reshape(-1, 1)

# Polynomial curve (uses the poly and model objects from above)
y_curve = model.predict(poly.transform(x_range))

# Straight line for comparison — fit a plain linear model
lin = LinearRegression().fit(X_train, y_train)
y_line = lin.predict(x_range)

plt.figure(figsize=(8, 5))
plt.scatter(X_train, y_train, color='steelblue', label='Training data', zorder=3)
plt.scatter(X_test,  y_test,  color='orange', marker='D', label='Test data', zorder=3)
plt.plot(x_range, y_line,  color='gray',   linestyle='--', linewidth=1.5,
         label=f'Linear fit  (degree=1)')
plt.plot(x_range, y_curve, color='crimson', linewidth=2,
         label=f'Polynomial fit  (degree=2, R²={model.score(X_test_poly, y_test):.3f})')
plt.xlabel('Sleep hours')
plt.ylabel('Performance')
plt.title('Polynomial Regression — Sleep Hours vs Performance')
plt.legend()
plt.tight_layout()
plt.show()

The dashed grey line is what degree=1 would give you — flat and wrong. The red curve captures the peak around 7–8 hours then the drop-off.

NESA Despite the name, logistic regression is a classification algorithm — not regression. It predicts the probability that something belongs to a category.

Instead of predicting "score = 74," it predicts "probability of passing = 0.87" — and then classifies based on that probability (usually: above 0.5 → class 1, below → class 0).

The key difference from linear regression: the output is always squashed between 0 and 1 using a function called the sigmoid.

Example — predicting pass/fail

We have two features: hours studied and attendance percentage. The label is pass (1) or fail (0).

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# ── Data — [hours_studied, attendance_%] → pass/fail ─────────────────
X = np.array([
    [1,  40], [1,  55], [2,  50], [2,  70], [3,  60],
    [3,  75], [4,  65], [4,  85], [5,  70], [5,  90],
    [6,  75], [6,  95], [7,  80], [8,  85], [8,  95],
])
y = np.array([0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1])

# ── Split ─────────────────────────────────────────────────────────────
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# ── Scale — logistic regression performs better with scaled features ──
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled  = scaler.transform(X_test)

# ── Train ─────────────────────────────────────────────────────────────
model = LogisticRegression()
model.fit(X_train_scaled, y_train)

# ── Evaluate ──────────────────────────────────────────────────────────
print(f"Accuracy: {model.score(X_test_scaled, y_test):.1%}")

# ── Predict — hard label and probability ─────────────────────────────
student = scaler.transform([[5, 72]])   # 5 hours study, 72% attendance
label = model.predict(student)[0]
proba = model.predict_proba(student)[0]

print(f"Prediction: {'Pass' if label == 1 else 'Fail'}")
print(f"P(Fail) = {proba[0]:.3f}   P(Pass) = {proba[1]:.3f}")

Visualise the result — decision boundary

Logistic regression has two features, so the chart is a 2D scatter plot. The shaded region shows the model's probability gradient — red = likely fail, blue = likely pass. The dashed black line is the decision boundary where P(pass) = 0.5:

import matplotlib.pyplot as plt
import numpy as np

# ── Visualise — scatter + decision boundary ───────────────────────────────
fails  = X[y == 0]
passes = X[y == 1]

# Build a grid covering the feature space
h_range = np.linspace(X[:, 0].min() - 0.5, X[:, 0].max() + 0.5, 300)
a_range = np.linspace(X[:, 1].min() - 5,   X[:, 1].max() + 5,   300)
hh, aa = np.meshgrid(h_range, a_range)
grid_scaled = scaler.transform(np.c_[hh.ravel(), aa.ravel()])
Z = model.predict_proba(grid_scaled)[:, 1].reshape(hh.shape)

plt.figure(figsize=(8, 6))
plt.contourf(hh, aa, Z, levels=20, cmap='RdBu', alpha=0.35)
plt.contour( hh, aa, Z, levels=[0.5], colors='black', linewidths=1.5, linestyles='--')
plt.colorbar(label='P(Pass)')
plt.scatter(fails[:,  0], fails[:,  1], color='crimson',   label='Fail (0)', s=80, zorder=3)
plt.scatter(passes[:, 0], passes[:, 1], color='steelblue', label='Pass (1)', s=80, zorder=3)
plt.xlabel('Hours studied')
plt.ylabel('Attendance %')
plt.title('Logistic Regression — Decision Boundary')
plt.legend()
plt.tight_layout()
plt.show()

Anything above and to the right of the dashed line → the model predicts Pass. Everything below → Fail. Students close to the line are the uncertain cases where predict_proba returns a value near 0.5.

Linear vs Logistic — what's the same, what's different

NESA In Lesson 18 you learned that a neural network transforms data through layers of connected nodes. The MLPClassifier (Multi-Layer Perceptron) is scikit-learn's implementation of this — and it uses the exact same OOP interface you've been using all lesson.

Architecture — what hidden_layer_sizes means

The key hyperparameter is hidden_layer_sizes — a tuple where each number is the count of nodes in that hidden layer:

• (4,) → one hidden layer with 4 nodes
• (8, 4) → two hidden layers: 8 nodes, then 4
• (16, 8, 4) → three hidden layers: gets deeper and more complex

More layers = more capacity to find complex patterns, but also more risk of overfitting on small datasets.

Applying a neural network to the same pass/fail problem

Using the same data as Section 4 — notice how similar the code is. This is the point of the OOP pattern.

import numpy as np
from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# Same data as Section 4
X = np.array([
    [1,40],[1,55],[2,50],[2,70],[3,60],[3,75],[4,65],[4,85],
    [5,70],[5,90],[6,75],[6,95],[7,80],[8,85],[8,95],
])
y = np.array([0,0,0,0,0,1,0,1,1,1,1,1,1,1,1])

# ── Split ─────────────────────────────────────────────────────────────
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# ── Scale — neural networks require scaled features ───────────────────
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s  = scaler.transform(X_test)

# ── Train ─────────────────────────────────────────────────────────────
mlp = MLPClassifier(
    hidden_layer_sizes=(8, 4),   # two hidden layers
    max_iter=1000,               # maximum training passes
    random_state=42
)
mlp.fit(X_train_s, y_train)

# ── Evaluate ──────────────────────────────────────────────────────────
print(f"Accuracy: {mlp.score(X_test_s, y_test):.1%}")

# ── Predict with confidence ────────────────────────────────────────────
student = scaler.transform([[5, 72]])
label   = mlp.predict(student)[0]
proba   = mlp.predict_proba(student)[0]

print(f"Prediction: {'Pass' if label == 1 else 'Fail'}")
print(f"P(Fail)={proba[0]:.3f}  P(Pass)={proba[1]:.3f}")

NESA The syllabus asks you to build ML models "using an OOP." Using scikit-learn objects satisfies this — but designing your own wrapper class goes further and demonstrates that you understand OOP, not just the library.

The problem without a wrapper: every time you use a neural network you have to remember to scale the data, use transform() (not fit_transform()) on test data, and repeat those three steps every time. Forget one step and the model gives wrong answers silently.

Solution: wrap the model + scaler inside one class. The caller never thinks about scaling — they just call train() and predict().

The MLPipeline class

Each part maps directly to an OOP concept:

from sklearn.preprocessing import StandardScaler
from sklearn.neural_network import MLPClassifier
from sklearn.linear_model import LogisticRegression

class MLPipeline:

    def __init__(self, model):
        # Encapsulation — the object owns its scaler and model.
        # The caller doesn't touch these directly.
        self._model  = model
        self._scaler = StandardScaler()
        self._trained = False

    def train(self, X_train, y_train):
        # Abstraction — caller just says "train".
        # Scaling + fitting happens inside; the caller doesn't know or care.
        X_scaled = self._scaler.fit_transform(X_train)
        self._model.fit(X_scaled, y_train)
        self._trained = True

    def predict(self, X):
        # Message passing — pipeline.predict(X) sends a message to the object.
        # The object acts on its own internal state (_scaler, _model).
        X_scaled = self._scaler.transform(X)   # transform only, NOT fit_transform
        return self._model.predict(X_scaled)

    def evaluate(self, X_test, y_test):
        X_scaled = self._scaler.transform(X_test)
        return self._model.score(X_scaled, y_test)

    def is_trained(self):
        return self._trained

Using the class — one interface, any model

This is generalisation in action. The MLPipeline accepts any scikit-learn model because they all share the same interface. You swap the model; none of the surrounding code changes:

import numpy as np
from sklearn.model_selection import train_test_split

X = np.array([
    [1,40],[1,55],[2,50],[2,70],[3,60],[3,75],[4,65],[4,85],
    [5,70],[5,90],[6,75],[6,95],[7,80],[8,85],[8,95],
])
y = np.array([0,0,0,0,0,1,0,1,1,1,1,1,1,1,1])

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# ── Try a neural network ──────────────────────────────────────────────────
nn_pipeline = MLPipeline(MLPClassifier(hidden_layer_sizes=(8, 4), max_iter=1000, random_state=42))
nn_pipeline.train(X_train, y_train)
print(f"Neural network accuracy : {nn_pipeline.evaluate(X_test, y_test):.1%}")

# ── Swap in logistic regression — same code, different model ─────────────
lr_pipeline = MLPipeline(LogisticRegression())
lr_pipeline.train(X_train, y_train)
print(f"Logistic regression acc : {lr_pipeline.evaluate(X_test, y_test):.1%}")

# ── Predict with any pipeline — same call ────────────────────────────────
new_student = np.array([[5, 72]])
print(f"NN prediction      : {'Pass' if nn_pipeline.predict(new_student)[0] == 1 else 'Fail'}")
print(f"LogReg prediction  : {'Pass' if lr_pipeline.predict(new_student)[0] == 1 else 'Fail'}")

The key insight: nn_pipeline.train() and lr_pipeline.train() are identical calls. The pipeline doesn't care which model is inside. That's because both MLPClassifier and LogisticRegression respond to the same messages — fit(), predict(), score(). This is generalisation.