Nine hundred smart meters and one number to make small
An energy supplier wants tariff bands. It has daily electricity profiles for 900 households and a product team that can support three or four bands, not thirty. This is the job k-means clustering was designed for: carve the rows into a fixed number of groups, each represented by its average member, so that every household sits as close as possible to the profile of its own band.
The algorithm minimises one quantity, the within-cluster sum of squared distances to the assigned centre. Scikit-learn calls it inertia_. Every design decision in k-means follows from that objective, including the awkward ones. Centres are means, so the method assumes a cluster is a cloud around a point. Distances are squared, so an outlier pulls hard. The number of groups is fixed in advance, because the objective falls monotonically as you add centres and would otherwise be minimised by giving every household its own cluster.
Four features per meter: overnight base load, daily peak, the share of consumption after six in the evening, and the weekend uplift.
import numpy as np, pandas as pd
rng = np.random.default_rng(92)
def profile(n, base, peak, evening, weekend):
return pd.DataFrame({
"base_kw": rng.normal(base, base * 0.22, n).clip(0.05).round(3),
"peak_kw": rng.normal(peak, peak * 0.22, n).clip(0.2).round(3),
"evening_share": rng.normal(evening, 0.06, n).clip(0.05, 0.95).round(3),
"weekend_uplift": rng.normal(weekend, 0.09, n).round(3),
})
meters = pd.concat([
profile(330, 0.22, 1.9, 0.52, 0.06), # out all day, big evening peak
profile(300, 0.41, 1.3, 0.34, 0.03), # someone home all day, flat
profile(270, 0.26, 3.4, 0.46, 0.21), # electric heating plus weekend load
], ignore_index=True)
truth = np.repeat([0, 1, 2], [330, 300, 270])
print(meters.shape)
# (900, 4)
What k-means clustering is actually minimising
Lloyd’s algorithm alternates two steps until nothing moves. Assign every row to its nearest centre. Recompute each centre as the mean of the rows assigned to it. Both steps can only reduce the objective, which is why the loop always terminates, and why it terminates at a local minimum rather than the best possible one. Writing it out is twelve lines and worth the twelve lines.
from sklearn.preprocessing import StandardScaler
X = StandardScaler().fit_transform(meters.to_numpy())
def lloyd(X, k, seed, iters=30):
rs = np.random.default_rng(seed)
centres = X[rs.choice(len(X), k, replace=False)]
for step in range(iters):
d = ((X[:, None, :] - centres[None, :, :]) ** 2).sum(axis=2)
labels = d.argmin(axis=1)
inertia = d[np.arange(len(X)), labels].sum()
new = np.array([X[labels == j].mean(axis=0) for j in range(k)])
if np.allclose(new, centres):
print(f"converged after {step} updates, inertia {inertia:.1f}")
break
centres = new
return labels, inertia
labels_mine, inertia_mine = lloyd(X, 3, seed=7)
# converged after 16 updates, inertia 1384.6
from sklearn.cluster import KMeans
km = KMeans(n_clusters=3, random_state=0).fit(X)
print("scikit-learn inertia:", round(km.inertia_, 1))
# scikit-learn inertia: 1384.6
Identical to the first decimal place. The library version is faster and better initialised, but there is no hidden cleverness in the objective itself. Note the StandardScaler call before any of this: the four columns run from hundredths of a kilowatt to several kilowatts, and without standardising, peak_kw would be the only feature that counted, for the reasons set out in how clustering distances are built and in scaling and normalisation.
The local-minimum problem is the same one that shows up in how gradient descent finds a minimum, with one difference that matters: k-means takes discrete steps over assignments, so it cannot be rescued by a smaller learning rate. It can only be restarted.
Initialisation, k-means++ and why n_init exists
Pick starting centres badly and the loop converges to a poor partition and stays there. The k-means++ scheme spreads the initial centres by choosing each new one with probability proportional to its squared distance from the nearest existing centre. Twenty-five single-start runs of each at k equals six:
for init in ["random", "k-means++"]:
vals = [KMeans(n_clusters=6, init=init, n_init=1, random_state=s).fit(X).inertia_
for s in range(25)]
print(f"{init:10s} best={min(vals):8.1f} worst={max(vals):8.1f} mean={np.mean(vals):8.1f}")
# random best= 1019.6 worst= 1096.6 mean= 1044.5
# k-means++ best= 1019.5 worst= 1075.5 mean= 1037.2
print("k-means++ with n_init=10:", round(KMeans(
n_clusters=6, n_init=10, random_state=0).fit(X).inertia_, 1))
# k-means++ with n_init=10: 1019.5
Be honest about the size of that effect. The mean gap between random and k-means++ is 0.7 per cent, which is real but small on well-separated data like this. The spread within each scheme is the bigger story: the worst random start landed 7.5 per cent above the best. What actually protects you is n_init, the number of independent restarts from which scikit-learn keeps the lowest-inertia result. Ten restarts recovered the best single run exactly. Leave n_init at its default or raise it; do not set it to one to save a second.
Choosing k: the elbow and the silhouette
The elbow method plots inertia against k and looks for where the curve stops falling steeply. It is a judgement call dressed as a method, so read the drops rather than squinting at a chart.
from sklearn.metrics import silhouette_score, adjusted_rand_score
prev = None
for k in range(2, 9):
m = KMeans(n_clusters=k, n_init=10, random_state=0).fit(X)
drop = "" if prev is None else f" drop={prev - m.inertia_:7.1f}"
print(f"k={k} inertia={m.inertia_:8.1f} "
f"silhouette={silhouette_score(X, m.labels_):.3f}{drop}")
prev = m.inertia_
# k=2 inertia= 2128.1 silhouette=0.386
# k=3 inertia= 1384.6 silhouette=0.394 drop= 743.5
# k=4 inertia= 1245.7 silhouette=0.303 drop= 138.9
# k=5 inertia= 1123.3 silhouette=0.240 drop= 122.4
# k=6 inertia= 1019.5 silhouette=0.226 drop= 103.8
# k=7 inertia= 935.9 silhouette=0.227 drop= 83.6
# k=8 inertia= 885.7 silhouette=0.216 drop= 50.2
print("ARI at k=3:", round(adjusted_rand_score(truth, KMeans(
n_clusters=3, n_init=10, random_state=0).fit_predict(X)), 3))
# ARI at k=3: 0.812
Going from two groups to three buys 743.5 of inertia. The next step buys 138.9. That is an elbow you can defend to a product manager. The silhouette score is far less decisive here: 0.394 at k equals three against 0.386 at k equals two, a gap of 0.008 that would flip under a different seed. On this dataset the elbow is the stronger signal, which is the opposite of the usual advice, and worth knowing before you stake a tariff design on a silhouette curve.
Where k-means is the wrong choice
The objective assumes round clusters of broadly similar spread. Shear three blobs into diagonal streaks and it fails, while a Gaussian mixture, which fits a covariance per component instead of a single point, recovers them.
from sklearn.datasets import make_blobs
from sklearn.mixture import GaussianMixture
B, y = make_blobs(n_samples=600, centers=3, cluster_std=0.7, random_state=0)
B = B @ np.array([[0.6, -0.65], [-0.4, 0.85]]) # stretch into long streaks
print("k-means ARI:", round(adjusted_rand_score(y, KMeans(
n_clusters=3, n_init=10, random_state=0).fit_predict(B)), 3))
print("gaussian mixture ARI:", round(adjusted_rand_score(y, GaussianMixture(
n_components=3, random_state=0).fit_predict(B)), 3))
# k-means ARI: 0.579
# gaussian mixture ARI: 0.975
U, yu = make_blobs(n_samples=[400, 400, 30], centers=[[0, 0], [6, 0], [3, 7]],
cluster_std=[1.6, 1.6, 0.4], random_state=0)
print("unequal sizes, k-means ARI:", round(adjusted_rand_score(yu, KMeans(
n_clusters=3, n_init=10, random_state=0).fit_predict(U)), 3))
# unequal sizes, k-means ARI: 0.837
| Data shape | k-means result | Better choice |
|---|---|---|
| Round, similar size | works well | k-means |
| Elongated or sheared | 0.579 on the demo above | GaussianMixture |
| Non-convex, rings or arcs | fails outright | density methods |
| Mixed numeric and categorical | meaningless centres | k-prototypes or Gower distance |
One textbook failure proved milder than advertised. With clusters of 400, 400 and 30 points and unequal spreads, k-means still reached 0.837, so the equal-size assumption bends further than it is usually given credit for. The shape assumption does not bend. For the shapes k-means cannot touch, see the official notes on the comparative behaviour of k-means and the KMeans estimator parameters, and the broader framing in what unsupervised learning is for.
Frequently Asked Questions
Why do I get different clusters every time I run k-means?
Because the starting centres are random and the algorithm stops at a local minimum. Fix random_state for reproducibility and keep n_init at ten or more so scikit-learn picks the best of several restarts. If results still swing wildly across seeds, your k is probably too high for the structure present.
Is a lower inertia always a better clustering?
No. Inertia falls every time you add a centre, so comparing inertia across different k tells you nothing on its own. It is only comparable between runs at the same k, which is exactly how n_init uses it.
Can k-means handle categorical features?
Not sensibly. The mean of one-hot columns is a vector of proportions that corresponds to no real household, so the centres stop being interpretable. Use k-prototypes, or cluster on a distance matrix built for mixed types, rather than one-hot encoding and hoping.
Key Takeaways
- K-means minimises the within-cluster sum of squared distances, and every assumption it makes about round, similar-sized clusters comes from that one objective.
- Standardise first, then leave
n_initat its default; the restart count protected the result far more than the choice between random and k-means++ initialisation, which differed by only 0.7 per cent on average. - Read the elbow as the size of successive inertia drops, which gave a clear answer at k equals three, rather than trusting a silhouette curve that separated k equals two from k equals three by 0.008.
- Reach for
GaussianMixturethe moment clusters look elongated, since it recovered sheared blobs at 0.975 where k-means managed 0.579. - Never compare inertia across different values of k, because it falls monotonically and will always favour more clusters.