Grouping 600 bike docks with nobody to mark your work
A city transport team hands you six months of usage records for 600 docking stations and asks how many kinds of station the network has. Nobody knows the answer. There is no column called station_type, no historical label, no expert willing to commit. That is the situation clustering in machine learning is built for: you have features, you have no target, and you want the rows to fall into groups that behave alike.
The mechanics are simpler than the reputation suggests. You define a notion of distance between two rows, then run an algorithm that puts rows with small distances in the same group. Everything interesting follows from those two choices. Change the distance and the groups change. Change the algorithm and the groups change again. Because there is no target, nothing stops you, and no error metric tells you that you got it wrong.
The wider family this sits in is covered in what unsupervised learning is and where it pays off, so this chapter stays on the clustering task itself: how distance is computed, which algorithm families exist, and how you judge a result with no answer key.
Here is the bike network. Three kinds of station were planted in the data, deliberately overlapping, because real station behaviour overlaps.
import numpy as np, pandas as pd
rng = np.random.default_rng(91)
def station_block(n, am_out, am_in, wknd, trip, spread):
return pd.DataFrame({
"am_outflow": rng.normal(am_out, am_out * spread, n).clip(1).round(1),
"am_inflow": rng.normal(am_in, am_in * spread, n).clip(1).round(1),
"weekend_share": rng.normal(wknd, 0.07, n).clip(0.02, 0.9).round(3),
"median_trip_min": rng.normal(trip, trip * 0.2, n).clip(3).round(1),
})
stations = pd.concat([
station_block(240, 38, 19, 0.22, 13.0, 0.34), # residential feeders
station_block(190, 17, 37, 0.20, 12.0, 0.34), # city centre sinks
station_block(170, 24, 24, 0.37, 19.0, 0.30), # riverside and park stations
], ignore_index=True)
print(stations.shape)
print(stations.agg(["mean", "std", "min", "max"]).round(2).to_string())
# (600, 4)
# am_outflow am_inflow weekend_share median_trip_min
# mean 27.70 26.61 0.26 14.39
# std 13.66 11.67 0.10 4.12
# min 1.00 1.20 0.02 3.60
# max 71.00 69.50 0.54 26.70
Because the planted types are known here, the quality of a clustering can be scored honestly with the adjusted Rand index. On real data you never have that luxury, which is the whole problem of the last section.
Distance is the algorithm
Look at those ranges. am_outflow spans 1 to 71. weekend_share spans 0.02 to 0.54. Euclidean distance adds squared differences across columns, so a station that differs by 20 morning trips is thirty times further away than a station whose weekend share differs by 0.4, even though the second difference is the one a planner cares about. Feed raw columns to a clustering algorithm and you have silently decided that weekend behaviour is irrelevant.
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans, AgglomerativeClustering, DBSCAN
from sklearn.metrics import (adjusted_rand_score, silhouette_score,
davies_bouldin_score, calinski_harabasz_score)
truth = np.repeat([0, 1, 2], [240, 190, 170])
X_raw = stations.to_numpy()
X = StandardScaler().fit_transform(X_raw)
print("ARI unscaled:", round(adjusted_rand_score(
truth, KMeans(n_clusters=3, random_state=0).fit_predict(X_raw)), 3))
print("ARI scaled: ", round(adjusted_rand_score(
truth, KMeans(n_clusters=3, random_state=0).fit_predict(X)), 3))
# ARI unscaled: 0.387
# ARI scaled: 0.738
Standardising nearly doubled the agreement with the planted types. This is not a tuning trick, it is the single highest-value line in most clustering scripts, and the reasoning behind it is set out in how feature scaling and normalisation change a model.
Scaling is not the only way to bias a distance. Suppose the data team helpfully adds two more columns, both of which are morning flow restated.
wide = stations.copy()
wide["am_outflow_per_dock"] = (stations["am_outflow"] / 24).round(3)
wide["net_am_flow"] = (stations["am_outflow"] - stations["am_inflow"]).round(1)
Xw = StandardScaler().fit_transform(wide.to_numpy())
lab_wide = KMeans(n_clusters=3, random_state=0).fit_predict(Xw)
print("columns:", Xw.shape[1], "ARI:", round(adjusted_rand_score(truth, lab_wide), 3))
# columns: 6 ARI: 0.694
Morning flow now gets three votes out of six and weekend behaviour gets one. The result degrades, from 0.738 to 0.694. Smaller than I expected, to be fair: the redundant columns were correlated with a genuinely informative axis, so the damage was limited. The lesson holds anyway, because the same trick with three restatements of a useless column would be fatal. Correlated features are a weighting decision, whether or not you meant to make one, which is one reason a proper exploratory pass over the data comes before the clustering and not after.
Distance choice matters just as much. Euclidean distance is the default in scikit-learn and in the neighbour-counting you met in how k-nearest neighbours classifies a point. For text or basket data where magnitude is an artefact of document length, cosine distance is usually right. For mixed numeric and categorical rows, Gower distance is the honest option and none of the common libraries implement it well.
Why clustering in machine learning has no right answer
Four algorithms, three families, one dataset. Centroid methods assign each row to the nearest of k centres. Hierarchy methods merge the closest pair repeatedly. Density methods grow clusters through dense neighbourhoods and leave sparse rows unassigned.
runs = {
"k-means": KMeans(n_clusters=3, random_state=0).fit_predict(X),
"ward": AgglomerativeClustering(n_clusters=3).fit_predict(X),
"average": AgglomerativeClustering(n_clusters=3, linkage="average").fit_predict(X),
"dbscan": DBSCAN(eps=0.55, min_samples=8).fit_predict(X),
}
for name, lab in runs.items():
print(f"{name:8s} clusters={len(set(lab) - {-1})} "
f"noise={int((lab == -1).sum()):3d} "
f"ARI={adjusted_rand_score(truth, lab):.3f}")
# k-means clusters=3 noise= 0 ARI=0.738
# ward clusters=3 noise= 0 ARI=0.550
# average clusters=3 noise= 0 ARI=0.642
# dbscan clusters=4 noise=407 ARI=0.097
print("k-means vs ward agreement:",
round(adjusted_rand_score(runs["k-means"], runs["ward"]), 3))
# k-means vs ward agreement: 0.604
Two reasonable methods asked the same question of the same 600 rows and agreed with each other only 0.604 of the way. Neither is buggy. K-means won because the planted groups are roughly spherical blobs of similar size, which is exactly the shape its objective rewards. DBSCAN scored 0.097 and called 407 of 600 stations noise, which is the correct behaviour for a density method handed overlapping Gaussian blobs with no empty space between them. A method is not better or worse in general, only better or worse matched to the geometry in front of it.
| Family | Representative | You must choose | Breaks when |
|---|---|---|---|
| Centroid | KMeans |
number of clusters | clusters are elongated or very unequal |
| Hierarchy | AgglomerativeClustering |
linkage and where to cut | n is large, memory is quadratic |
| Density | DBSCAN |
neighbourhood radius | cluster densities differ |
| Distribution | GaussianMixture |
number of components | features are far from Gaussian |
Judging a clustering you cannot check
Internal validation indices score a partition using only the data and the labels. Silhouette compares each row’s distance to its own cluster against the nearest other cluster, on a scale from minus one to one. Davies-Bouldin measures the ratio of within-cluster spread to between-cluster separation, so lower is better. Calinski-Harabasz is a variance ratio, so higher is better.
for k in range(2, 7):
lab = KMeans(n_clusters=k, random_state=0).fit_predict(X)
print(f"k={k} silhouette={silhouette_score(X, lab):.3f} "
f"davies_bouldin={davies_bouldin_score(X, lab):.3f} "
f"calinski_harabasz={calinski_harabasz_score(X, lab):.0f}")
# k=2 silhouette=0.280 davies_bouldin=1.304 calinski_harabasz=244
# k=3 silhouette=0.344 davies_bouldin=1.065 calinski_harabasz=336
# k=4 silhouette=0.279 davies_bouldin=1.199 calinski_harabasz=299
# k=5 silhouette=0.270 davies_bouldin=1.189 calinski_harabasz=278
# k=6 silhouette=0.239 davies_bouldin=1.301 calinski_harabasz=245
All three indices point at k equals three, which is the truth. Note the actual value though. A silhouette of 0.344 is not a good score by any usual rule of thumb, and it is only a little above the 0.349 that the same scoring gave to pure random noise in the worked unsupervised example on retail basket data. The correct reading is that absolute silhouette thresholds are close to meaningless. Compare a candidate partition against a permuted or resampled version of your own data, not against a number you read somewhere.
My own habit is to trust an index for ranking k within one algorithm, and not to trust it at all for choosing between algorithms, because each index encodes a cluster shape and will reward the method that shares its assumptions. The scikit-learn user guide sets out the same caution across all of its clustering estimators and evaluation metrics.
Frequently Asked Questions
How do I know if my data has clusters at all?
Run your pipeline on a version of the data with each column independently shuffled, which destroys the joint structure but keeps every marginal distribution. Score both. If your real silhouette is not clearly above the shuffled one, you have a continuum rather than clusters, and forcing groups onto it will not survive contact with new data.
Should I always standardise before clustering?
Almost always, when columns are in different units. Skip it when every column is already on one comparable scale, such as a basket of percentage shares, since standardising those inflates the influence of near-constant columns. Decide by asking what a one-unit change means in each column.
How many clusters should I ask for?
Start from what the result is for. Four station types is a plan a transport team can staff; seventeen is a spreadsheet nobody will act on. Use the indices to rank candidates inside that practical range rather than to search the whole space for a global optimum that does not exist.
Key Takeaways
- Clustering has no target and therefore no error, so every result is a consequence of the distance you chose and the algorithm family you picked, not a fact about the data.
- Standardise your features before any distance-based method unless you have a specific reason not to; on the bike data it lifted agreement with the planted types from 0.387 to 0.738.
- Treat correlated or restated columns as a weighting decision, because three versions of one measurement give that measurement three votes in the distance.
- Expect reasonable algorithms to disagree, and read that disagreement as information about the geometry rather than as a bug to be tuned away.
- Judge a partition against a shuffled version of your own data instead of against an absolute silhouette threshold, since a modest 0.344 can still be the right answer.