A factory sensor has been recording ordinary operation for weeks, and you want an alert when its readings begin to look unusual. You may have plenty of normal data but few examples of a failure, making it hard to train a classifier for every fault. Anomaly detection starts from the patterns in the data you do have and looks for observations that depart from them. The next challenge is deciding which departures deserve attention.
We test alerts on rare credit-card fraud. Isolation Forest scores ROC-AUC 0.951, but average precision is only 0.148. That ROC score can coexist with many false alerts. We compare isolation, boundary fitting and reconstruction error, checking their definitions of unusual and their disagreements.
This post builds all three from first principles: an Isolation Forest in pure NumPy, a One-Class SVM boundary, and a PyTorch autoencoder scored by reconstruction error. We run them on real credit-card fraud, compare them with one ROC and precision-recall panel, and confront the metric trap that makes every one of them look near-perfect when it is not. By the end you will have implemented isolation from scratch and you will know which number to trust.
Let's Build It
First, meet the haystack. The classic ULB credit-card fraud dataset holds 284,807 European card transactions from September 2013, of which 492 are fraud. That is 0.172 percent, one fraud in every 579 transactions. The features are 28 anonymised PCA components plus the transaction amount.

Open the notebook and run the whole pipeline in Colab:
Here is the fastest possible detector. No labels go into training, only the raw features:
import numpy as np
from sklearn.datasets import fetch_openml
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import IsolationForest
from sklearn.metrics import roc_auc_score, average_precision_score
df = fetch_openml(data_id=1597, as_frame=True).frame # ULB credit-card fraud
y = df.pop("Class").astype(int).values
X = df.select_dtypes("number").values
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.5, stratify=y, random_state=42)
Xtr, Xte = (s := StandardScaler().fit(Xtr)).transform(Xtr), s.transform(Xte)
model = IsolationForest(n_estimators=100, max_samples=256, random_state=0).fit(Xtr)
scores = -model.score_samples(Xte) # higher = more anomalous
print(f"ROC-AUC = {roc_auc_score(yte, scores):.3f}") # 0.951
print(f"PR-AUC = {average_precision_score(yte, scores):.3f}") # 0.148
A ROC-AUC of 0.951 out of the box, with zero labels and zero tuning. That number looks like a solved problem. It is not. The precision-recall score sitting right next to it is 0.148, and the gap between those two numbers is the whole story of this post. Hold that thought. First we need to understand what the forest did, and to do that we build it ourselves.
What Just Happened?
Novelty versus outlier detection
Two setups hide under the word "anomaly", and mixing them up causes most of the confusion in practice.
Outlier detection assumes your training data is already contaminated with a few anomalies, and you want to find them in that same batch. The Isolation Forest above works this way: it saw all 142,403 training transactions, fraud included, and still ranked the strange ones highly.
Novelty detection assumes you can train on a clean, anomaly-free sample of normal data, then flag anything that deviates at test time. The One-Class SVM and the autoencoder we build later work best this way: fit them on legitimate transactions only, and fraud shows up as a stranger.
The knob that connects the two is contamination, your assumed fraction of anomalies. It sets where the score threshold falls, not the ranking itself. Get the ranking right first; choose the threshold second.
Anomalies are easy to isolate
The Isolation Forest rests on one idea you can see in a picture. Pick a random feature, pick a random split point, and cut the data in two. Repeat inside each half. A point sitting far from the crowd gets fenced off into its own region after only a few cuts, because almost any random split separates it from the pack. A point buried in a dense cluster survives many cuts before it is finally alone. The number of splits needed to isolate a point is therefore a measure of how anomalous it is: short is strange, long is normal.

The whole algorithm is a handful of NumPy. An isolation tree recursively splits a small random subsample until every point is alone or a height limit is hit:
class Node:
__slots__ = ("feature", "thresh", "left", "right", "size", "depth", "leaf")
def build_tree(X, depth, height_limit, rng):
node = Node(); n = len(X)
if depth >= height_limit or n <= 1:
node.leaf, node.size, node.depth = True, n, depth
return node
f = rng.integers(0, X.shape[1]) # random feature
lo, hi = X[:, f].min(), X[:, f].max()
if lo == hi:
node.leaf, node.size, node.depth = True, n, depth
return node
t = rng.uniform(lo, hi) # random split point
mask = X[:, f] < t
node.leaf, node.feature, node.thresh, node.depth = False, f, t, depth
node.left = build_tree(X[mask], depth + 1, height_limit, rng)
node.right = build_tree(X[~mask], depth + 1, height_limit, rng)
return node
Each tree sees only a subsample of points, so trees are cheap and diverse. A forest is just a list of them:
def fit_iforest(X, n_trees=100, psi=256, seed=0):
rng = np.random.default_rng(seed)
height_limit = int(np.ceil(np.log2(psi))) # average tree height
return [build_tree(X[rng.choice(len(X), min(psi, len(X)), replace=False)],
0, height_limit, rng) for _ in range(n_trees)]
To score a point, drop it through every tree, record the depth at which it lands, and average. Anomalies land shallow, normal points land deep.

Run the from-scratch forest against sklearn's and they agree closely: our path-length scores correlate with sklearn's at a Spearman of 0.94, and the ROC-AUC lands at 0.956 versus sklearn's 0.951. The 60 lines above are the real algorithm, not a toy.
Going Deeper
Expected path length and the anomaly score
Averaging raw depths has a bug: a point in a big subsample naturally sits deeper than one in a small subsample, so depth is not comparable across trees. Liu, Ting and Zhou fix this by normalising against the depth you would expect by chance. An isolation tree is a binary search tree, and the average depth of a node in a random BST of points is known in closed form:
is the yardstick: the typical number of splits to isolate one of points if nothing were unusual. The anomaly score for a point compares its observed mean depth against that yardstick:
Read the exponent. If a point takes far fewer splits than expected, , the ratio is small, and the score pushes toward 1 (anomaly). If it takes about as many as expected, the ratio is near 1 and the score sits near 0.5. If it takes many more, the score falls toward 0 (firmly normal). That is the right panel of the histogram above: the score is nothing more than the path length, folded through this exponential so that "short path" becomes "high score".
EULER = 0.5772156649
def c_factor(n):
return 2.0 * (np.log(n - 1) + EULER) - 2.0 * (n - 1) / n if n > 1 else 0.0
def anomaly_score(paths, psi):
return 2.0 ** (-paths / c_factor(psi)) # paths = mean depth per point
The forest has two hyperparameters and both come straight from the paper: 100 trees and a subsample size . The subsample is the important one. A small is a feature, not a limitation: it keeps each tree's "normal" region tight, so a cluster of near-duplicate anomalies (a fraud ring reusing one pattern) cannot hide by making itself look dense.
A boundary around the normal region
The Isolation Forest asks how easy a point is to cut off. A One-Class SVM asks a geometric question instead: draw the smallest region in feature space that contains almost all the normal data, then flag anything outside it. Schölkopf's formulation separates the data from the origin with a maximum-margin hyperplane in a kernel feature space; the equivalent SVDD of Tax and Duin wraps it in the smallest enclosing hypersphere. With an RBF kernel both bend into a flexible contour that hugs the shape of the normal cloud, even when that cloud has several separate lumps.

It is the same margin machinery from Support Vector Machines, retargeted from "separate two classes" to "enclose one class". You get it in three lines, fitting on legitimate transactions only so it is a true novelty detector:
from sklearn.svm import OneClassSVM
normals = Xtr[ytr == 0] # train on legitimate only
oc = OneClassSVM(kernel="rbf", nu=0.05, gamma="scale").fit(normals[:6000])
oc_scores = -oc.decision_function(Xte) # outside the boundary = high
The parameter (here 0.05) does double duty: it upper-bounds the fraction of training points allowed outside the boundary and lower-bounds the fraction of support vectors. It is your contamination assumption in disguise. The RBF kernel scales poorly with sample count, which is why we fit on a 6,000-point subsample. On the fraud test set this boundary scores a ROC-AUC of 0.950, level with the forest.
Reconstruction error as a score
The third lens is the most flexible and the one that carries into deep learning. Train an autoencoder, a neural network that compresses its input to a small bottleneck and rebuilds it, on normal data alone. It learns to reconstruct legitimate transactions with tiny error because it has seen thousands of them. Feed it fraud and it stumbles: the patterns it learned do not fit, so the reconstruction error spikes. The error is the anomaly score.
import torch, torch.nn as nn
Xn = torch.tensor(Xtr[ytr == 0], dtype=torch.float32) # legitimate only
ae = nn.Sequential(nn.Linear(29, 16), nn.ReLU(), nn.Linear(16, 8), nn.ReLU(),
nn.Linear(8, 16), nn.ReLU(), nn.Linear(16, 29))
opt, loss_fn = torch.optim.Adam(ae.parameters(), lr=1e-3), nn.MSELoss()
for epoch in range(30):
for i in range(0, len(Xn), 512):
batch = Xn[torch.randperm(len(Xn))[i:i + 512]]
opt.zero_grad(); loss = loss_fn(ae(batch), batch); loss.backward(); opt.step()
with torch.no_grad():
Xt = torch.tensor(Xte, dtype=torch.float32)
ae_scores = ((ae(Xt) - Xt) ** 2).mean(1).numpy() # reconstruction MSE
The training loss falls smoothly from 0.87 to 0.46 and flattens, the sign of a fit that has learned the normal manifold without memorising noise.

Scored on the test set, legitimate transactions reconstruct with near-zero error while fraud spreads across a long high-error tail. A threshold at the 95th percentile of all errors catches most of the fraud at a cost of some false alarms.

The compress-then-reconstruct idea, and the discovery that reconstruction error carries a signal, is the same instinct behind JEPA and modern self-supervised learning, which push it further by predicting in latent space instead of input space.
The visually similar mimic
Reconstruction error shines on the problem that seeded this post: telling apart two things that look almost identical. Train the autoencoder on one class only, then show it a lookalike. Here the network learns handwritten 3s and never sees an 8. An 8 is close to a 3 with its left side closed, so a human glances and hesitates. The autoencoder does not: it tries to rebuild the 8 using only the vocabulary of 3s, erases the left loops, and returns something that looks like a 3. That failure is exactly the signal.

The reconstruction of the mimic is visibly wrong, and the error separates the two digits at a ROC-AUC of 0.96, without the network ever training on a single 8. This is the shape of the medical use case, a detector that learns healthy tissue and flags the lookalike that is not, and it is why reconstruction methods matter even where a boundary or an isolation tree would struggle.
Accuracy is the wrong metric
Now the reckoning. All three detectors land near a ROC-AUC of 0.95. Plot the ROC curves and they hug the top-left corner, the picture of success.

The precision-recall panel on the right tells a different story. The same three models score an average precision of 0.12 to 0.25. Why the disagreement? ROC-AUC measures the true-positive rate against the false-positive rate, and with 284,315 legitimate transactions the false-positive rate barely moves even when you wrongly flag thousands of them. It is measured against a denominator so large that mistakes vanish. Precision measures how many of your alerts are real, and with 492 needles in 284,807 pieces of hay, most alerts are false by construction. Accuracy is worse still: a model that predicts "legitimate" for everything scores 99.83 percent accuracy and catches zero fraud.
The metric that survives contact with this imbalance is precision@k: of the top transactions your model flags as most anomalous, how many are real fraud? Set to the 246 true anomalies in the test set and the three detectors catch 52, 70, and 75 of them respectively. The autoencoder, top of the class, still buries two false alarms for every real fraud it surfaces.

This is not a failure of the detectors, it is the reality of a 0.17 percent base rate, and it is the exact failure mode we dissected in SMOTE vs class weights. Report ROC-AUC on a problem like this and you will convince yourself a mediocre model is excellent. Report PR-AUC and precision@k and you see the real operating point: a triage tool that surfaces a manageable review queue in which a useful minority are true, not an oracle.
Choosing the threshold
A ranking is not yet a decision. Turning scores into alerts means picking a cut, and the honest way to pick it is by capacity, not by a magic number. If your review team can investigate 200 cases a day, threshold at the 200 highest scores and measure the precision you get there. If a missed fraud costs far more than a wasted review, push the threshold down and accept more false alarms to lift recall. The contamination argument in scikit-learn and the in the One-Class SVM are just conveniences that pick this cut for you from an assumed anomaly rate; when you know your real review budget, set it directly. For anomalies that arrive as a shift in a stream over time rather than as points in a fixed batch, a sequential method fits better: see Bayesian Online Changepoint Detection.
Where This Comes From
Anomaly detection is not one algorithm but a family, and the three lenses here trace to distinct lineages two decades deep.

Isolation Forest. Liu, Ting and Zhou introduced it in "Isolation Forest" (ICDM 2008). Their move was to break with every method that came before it. Earlier detectors first build a profile of normal points (a density, a distance, a boundary) and call anything far from it an anomaly, which is expensive and struggles in high dimensions. Liu and colleagues instead isolate anomalies directly, and they prove the key claim: because anomalies are "few and different", they are more susceptible to isolation, so their expected path length is shorter. The score and the normaliser we implemented are equations 1 and 2 of the paper, verbatim.
"Anomalies are 'few and different', which make them more susceptible to isolation than normal points." (Liu, Ting & Zhou, 2008)
One-Class SVM and SVDD. Schölkopf, Platt, Shawe-Taylor, Smola and Williamson posed one-class classification as estimating the support of a high-dimensional distribution (Neural Computation, 2001): find the maximum-margin hyperplane separating the data from the origin in a kernel space. Tax and Duin's Support Vector Data Description (Machine Learning, 2004) recast the same idea as the smallest enclosing sphere, which is equivalent for the RBF kernel and often more intuitive.
Autoencoder reconstruction. Sakurada and Yairi's "Anomaly Detection Using Autoencoders with Nonlinear Dimensionality Reduction" (MLSDA 2014) showed that reconstruction error from an autoencoder detects anomalies where linear PCA fails, because the nonlinear bottleneck captures curved manifolds a linear subspace cannot.
The synthesis. Ruff and colleagues' "A Unifying Review of Deep and Shallow Anomaly Detection" (Proceedings of the IEEE, 2021) is the map of the whole territory. It places isolation, boundary, and reconstruction methods in one framework and is the single best next read once the three from-scratch builds here have made the ideas concrete.
Further Reading
- The Isolation Forest paper: Liu, Ting & Zhou (2008), for equations 1 and 2 and the path-length argument.
- One-Class SVM: Schölkopf et al. (2001) and Tax & Duin (2004) for the boundary view.
- The modern map: Ruff et al. (2021), a unifying review of deep and shallow methods.
- Density as another route: a Gaussian mixture flags low-likelihood points as outliers; see From K-Means to GMM.
Interactive Tools
- Classification Metrics Calculator: Feed in a confusion matrix and watch precision, recall, and F1 diverge from accuracy exactly the way they do under the fraud imbalance in this post.
- Medical Statistics Calculator: Sensitivity, specificity, and PPV under a low base rate: the clinical mirror of precision@k, where a "99% accurate" test still yields mostly false positives when the disease is rare.
Related Posts
- SMOTE vs Class Weights: Imbalanced Classification: Why accuracy misleads under extreme imbalance, and the metrics that do not.
- Support Vector Machines: A Sparse Kernel Machine Guide: The margin-and-kernel machinery the One-Class SVM retargets to enclose a single class.
- JEPA from Scratch: Predicting in Latent Space: Where reconstruction gives way to prediction in latent space, the next step past autoencoders.
- Bayesian Online Changepoint Detection in Python: Detecting anomalies as shifts in a stream over time rather than points in a batch.
- From K-Means to GMM: Hard vs Soft Clustering: The density-based route to outliers: low likelihood under a fitted mixture.
- Variational Autoencoders in NumPy: Three Digits, No KL Term: A probabilistic version of the autoencoder used here, which can score a new point by a bound on its log-likelihood.






