Supervised Learning Explained

The label is the entire difference

Supervised learning is machine learning where every training row comes with the correct answer attached. You show the algorithm a thousand dental appointments along with whether each patient turned up, and it works out how to predict attendance for the appointment booked this morning. That attached answer — the label — is the only thing separating this from every other kind of learning.

Everything else follows from it. Because you have the right answers, you can measure exactly how wrong the model is on every row, which means you can optimise against that error directly, which is why supervised methods are more accurate and more predictable than the alternatives. It is also why they are more expensive: somebody, somewhere, had to produce those labels.

This is the branch that pays most people’s salaries. Fraud scoring, churn prediction, credit decisions, demand forecasting, medical triage, defect detection on a production line — all supervised. If you are moving into a data role, this is the family you will be interviewed on.

Where the statistical framing of predictive modelling concentrates on assumptions and interpreting coefficients, the concern here is mechanical: what shape your data must take, which estimator to reach for, and where the approach quietly breaks.

Two shapes: classification and regression

The label’s data type decides which of two problems you have.

Classification predicts a category. Will this patient attend? Which of four fault codes does this vibration pattern indicate? Is this transaction fraud?

Regression predicts a number on a continuous scale. How many tonnes will this vineyard block yield? How many minutes until the parcel arrives?

The distinction sounds trivial and is not. It changes the estimator, the loss function, the metrics, and what a “good” model even means.

Watch for labels that look numeric but are not. A five-star rating is stored as an integer, but treating it as regression assumes the gap between one and two stars equals the gap between four and five. Sometimes it does. Often it does not, and you should be classifying.

The fit and predict contract

The labels are where the logic comes from, which is the whole inversion behind machine learning versus traditional programming. Every supervised estimator in scikit-learn obeys the same two-method contract: .fit(X, y) learns, .predict(X) applies. X is a table of features with one row per example. y is the labels, one per row, in the same order. Get the alignment wrong and nothing errors — you simply train a model on nonsense.

Here is a dental clinic predicting no-shows.

import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

rng = np.random.default_rng(7)
n = 600
appointments = pd.DataFrame({
    "days_booked_ahead": rng.integers(1, 60, n),
    "previous_no_shows": rng.poisson(0.6, n),
    "sms_reminder_sent": rng.integers(0, 2, n),
})
risk = (0.02 * appointments["days_booked_ahead"]
        + 0.5 * appointments["previous_no_shows"]
        - 0.8 * appointments["sms_reminder_sent"])
appointments["no_show"] = (risk + rng.normal(0, 0.5, n) > 1.0).astype(int)

X = appointments.drop(columns="no_show")   # features
y = appointments["no_show"]                # the label

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=7)

clf = RandomForestClassifier(n_estimators=200, random_state=7)
clf.fit(X_train, y_train)

print(round(accuracy_score(y_test, clf.predict(X_test)), 3))  # 0.753
print(round(y.mean(), 3))                                     # 0.28

Read those two numbers together. The model is right 75.3% of the time, and 28% of appointments are no-shows — so predicting “everyone attends” would score 72% without any model at all. A headline accuracy means nothing until you compare it to the base rate. That gap is the first thing an interviewer will probe.

Swap the estimator and the same contract handles a numeric label:

from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error

rainfall = np.array([[210.], [180.], [240.], [300.],
                     [275.], [190.], [260.], [320.]])   # mm over the season
tonnes = np.array([6.1, 5.4, 6.8, 8.0, 7.6, 5.6, 7.2, 8.4])

reg = LinearRegression().fit(rainfall, tonnes)
print(round(reg.coef_[0], 4), round(reg.intercept_, 3))  # 0.0217 1.527
print(round(mean_absolute_error(tonnes, reg.predict(rainfall)), 3))  # 0.051

Roughly 0.022 tonnes per extra millimetre of rain. The mechanics of how .fit() searches for those coefficients are covered in how machines learn from data.

Where labels come from, and what they cost

The textbook version hands you a labelled dataset. Real projects begin by asking where the labels will come from, and the answer determines whether the project is feasible.

Labels that already exist. The luckiest case. The clinic’s booking system already records attendance; the bank already knows which loans defaulted. The label is a by-product of running the business, and you get years of it free.

Labels that arrive late. A twelve-month loan tells you nothing about default for twelve months. You can build a model, but every retraining cycle is a year behind reality.

Labels someone must create. Radiologists marking scans, moderators tagging posts. This is where budgets die. Ten thousand labelled images at two minutes each is over three hundred hours of expert time, and expert time is the whole point of the exercise.

Labels you infer, and shouldn’t. Treating “no complaint” as “satisfied” is inventing a label. The model will learn your assumption rather than reality, and it will look fine in testing because the test labels carry the same flaw.

Ask about labels before you ask about algorithms. A weaker model on honest labels beats gradient boosting on labels somebody guessed at.

Where supervised learning stops being the right tool

Reach elsewhere when the label is missing, misleading, or the wrong idea entirely.

If nobody knows what the categories should be — you want to discover customer groupings rather than sort customers into groups you already defined — you want unsupervised methods, which get their own chapter next.

If the label exists but is rare to the point of absurdity, say four fraud cases in two million transactions, a standard classifier will predict “not fraud” every time and score 99.9998%. Anomaly detection is usually the better framing.

If the correct answer depends on what your system did earlier, supervised learning cannot express it. Recommending an item changes what the user sees next, so there is no fixed right answer per row.

There is also the honest case where the label is simply unstable. If three analysts disagree about whether the same support ticket is “urgent”, the ceiling on your model’s accuracy is set by that disagreement, and no estimator will climb past it. Fix the labelling guidelines before touching the model. Which family of system you are building at all is the subject of the three axes that classify machine learning systems.

The estimators used above are documented in the scikit-learn references for RandomForestClassifier and LinearRegression.

Frequently Asked Questions

What is the difference between supervised and unsupervised learning?

Supervised learning trains on rows that carry the correct answer, so it can be scored directly against known outcomes. Unsupervised learning gets no answers and searches for structure instead — groupings, outliers, compressed representations. The practical split is whether you can measure right and wrong, or only inspect what came out and judge it.

Do I need labelled data for supervised learning?

Yes, and that requirement is usually the binding constraint on a project. Without labels there is nothing to fit against. If you have a small labelled set and a large unlabelled one, semi-supervised methods can stretch what you have, but they never remove the need for some ground truth to anchor the model.

Is supervised learning the same as classification?

No. Classification is one of its two shapes, alongside regression. Classification predicts a category such as fraud or not fraud; regression predicts a number such as expected revenue. Both use labelled training data and the same fit and predict interface, so the terms get confused often.

Key Takeaways

• Establish where your labels will come from before choosing an estimator, because label availability and quality decide the project more often than algorithm choice does.
• Check the data type of your target to pick classification or regression, and question numeric-looking labels such as star ratings where the gaps between values are not equal.
• Compare every accuracy figure against the base rate of the majority class, or you will ship a model that performs worse than a constant guess.
• Keep X and y aligned row for row, since misalignment produces no error message and trains a perfectly confident model on noise.
• Move to anomaly detection when the positive class is vanishingly rare, and fix inconsistent labelling guidelines before blaming the model for a low ceiling.