You have a small labelled table and want predictions for a few new rows. A model such as gradient-boosted trees learns from your training rows before it can make those predictions. TabPFN takes another approach, reading the labelled and unlabelled rows together as context for a model trained in advance. The labelled rows supply examples at prediction time, so there is no optimiser run on your own dataset.
We test that prediction route against models fitted to each table. The 2025 Nature paper reports TabPFN v2 beating tuned baselines; ours are untuned. Prediction takes under a second on small datasets but nearly two minutes on digits using a CPU. We compare accuracy and elapsed time.
This post puts that claim under load. We run TabPFN against logistic regression, XGBoost, and histogram gradient boosting on six small datasets, measure accuracy and wall-clock, and watch its decision boundary form as we feed it more in-context examples. By the end you will understand what "prior-data fitted network" means, why a single forward pass can substitute for training, and exactly where this approach still falls down. Averaged over our six datasets, TabPFN scored 0.955 against XGBoost's 0.921.
Let's Build It
Click the badge to open the interactive notebook:
Watch the decision boundary form. Each frame fits TabPFN on a few more in-context examples and re-plots its predicted boundary. There are no training epochs between frames; every boundary is a single forward pass over the points shown so far.

Using it is anticlimactic, which is the point. There is no training loop to write:
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from tabpfn import TabPFNClassifier
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)
clf = TabPFNClassifier(device="cpu")
clf.fit(X_train, y_train) # no optimisation happens here -- it just stores the data
print("accuracy:", clf.score(X_test, y_test))
accuracy: 0.9766
That fit call does not train anything. It stores your training rows so they can be fed into the transformer alongside the test rows at prediction time. The pre-trained weights never change. Compare that with XGBoost, which runs hundreds of boosting rounds on every new dataset, and the shift in mental model is the whole story.
What Just Happened?
In-context learning, moved to tables
If you have used a large language model, you have already seen this trick. Give an LLM a few examples in the prompt ("translate hello to French: bonjour; translate cat to French: chat; translate dog to French:") and it completes the pattern without any weight updates. That is in-context learning: the model learns the task from the examples in its context window, at inference time, using only a forward pass.
TabPFN does exactly this for tables. Your training rows are the examples in the context; your test rows are the queries to complete. The transformer attends across all the rows at once, the labelled ones informing the unlabelled ones, and emits a predicted class distribution for each test row. The "learning" is the forward pass attending from test rows to training rows, not an optimiser grinding on a loss.
This is why the linear regression mental model (fit coefficients, store them, discard the data) does not apply. TabPFN keeps the data and throws away the idea of fitted parameters per dataset entirely.
The decision boundary tells the story
Different model families carve up feature space in characteristically different shapes. Plotting their decision boundaries on the same two-dimensional data makes the difference obvious.

Logistic regression can only draw a straight line, so the interleaved crescents defeat it. Gradient boosting stacks axis-aligned splits, giving the blocky staircase boundary that trees always produce. TabPFN draws a smooth, almost hand-drawn curve that follows the gap between the classes. That smoothness is not an accident: TabPFN was trained to approximate the Bayesian posterior predictive distribution, which we will unpack shortly. It behaves less like a fitted classifier and more like a calibrated probabilistic model.
Going Deeper
The benchmark: no training, still competitive
We ran four methods on six small classification datasets (four real, two synthetic), three random splits each, all out of the box with no hyperparameter tuning. The baselines (logistic regression, XGBoost, histogram gradient boosting) train normally on each split. TabPFN does its single forward pass.

| Dataset | LogReg | XGBoost | HistGBDT | TabPFN |
|---|---|---|---|---|
| breast cancer | 0.961 | 0.959 | 0.953 | 0.969 |
| wine | 0.988 | 0.981 | 1.000 | 1.000 |
| iris | 0.985 | 0.970 | 0.970 | 0.993 |
| digits | 0.965 | 0.963 | 0.972 | 0.990 |
| synth (overlap) | 0.638 | 0.808 | 0.806 | 0.903 |
| synth (wide) | 0.733 | 0.842 | 0.809 | 0.878 |
| mean | 0.878 | 0.921 | 0.918 | 0.955 |
TabPFN wins or ties on every dataset, and the margin is widest on the harder synthetic problems where the classes overlap. This matches what the TabPFN authors report on larger benchmark suites: on small tabular data, an untuned forward pass competes with carefully tuned gradient boosting. Remember the baselines here are also untuned; a heavily tuned XGBoost would close some of this gap. The remarkable part is that TabPFN needs no tuning at all to land where it does.
The catch: it is a small-data model that wants a GPU
Nothing about this is free. The same forward-pass design that removes training also caps what TabPFN can handle.

On the small datasets TabPFN predicts in well under a second. On digits (1,797 rows) it took nearly two minutes per split on a CPU, because attention cost grows with the number of in-context rows and the library deliberately discourages large CPU runs. The original TabPFN was capped around 1,000 training rows and 100 features; TabPFN v2 pushes that to roughly 10,000 rows, and a GPU makes the forward pass tractable. Past those limits, gradient-boosted trees remain the right tool. TabPFN is a precision instrument for the small-data regime, not a general replacement.
When to reach for it, and when not to
Use TabPFN when you have a small labelled dataset (up to a few thousand rows), a modest number of features, and you want a strong baseline in seconds without tuning. It shines exactly where deep learning usually struggles and where practitioners reach for trees by reflex. Its probabilities are well calibrated out of the box, which matters when you act on confidence rather than the top class, a theme we explored in handling class imbalance.
Avoid it when you have hundreds of thousands of rows, very high-dimensional inputs, or hard latency budgets on CPU. And like any dimensionality-sensitive method, it benefits from sensible feature handling; it is robust, not magic.
Where This Comes From
TabPFN is the tabular instance of a broader idea called a Prior-data Fitted Network (PFN), introduced by Müller et al. (2022) in "Transformers Can Do Bayesian Inference". The insight is worth stating carefully because it reframes what "training a model" means.
A Bayesian predictor answers: given a prior over possible data-generating functions and some observed data, what is the posterior predictive distribution over a new point's label? That integral is usually intractable. PFNs sidestep it by amortisation. You define a prior, sample millions of synthetic datasets from it, and train a transformer once to predict held-out labels given the rest of each synthetic dataset. In doing so the transformer is forced to learn the mapping from "observed rows" to "posterior prediction" directly. After this one-time pre-training, a forward pass on a real dataset approximates Bayesian inference under that prior, with no integration and no per-dataset fitting.
"We train a network to do Bayesian prediction for arbitrary priors... at inference time, the trained network can do Bayesian prediction in a single forward pass." (Müller et al., 2022)

Hollmann et al. (2023), "TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second", applied this to tables. The prior is the crucial design choice. TabPFN's prior is built to mimic the kinds of structures real tabular data exhibits: causal relationships between columns, mixtures of feature types, varying noise levels. The transformer pre-trained on this prior generalises to genuinely unseen real datasets because they look, statistically, like draws from a similar prior. The connection to Bayesian model comparison and information criteria is direct: TabPFN is implicitly averaging over models the way a fully Bayesian treatment would, rather than selecting one and fitting it.
Hollmann et al. (2025), the Nature paper "Accurate predictions on small data with a tabular foundation model", is the TabPFN v2 release. It scales the row and feature limits, adds regression, and demonstrates the approach beating tuned baselines on standard benchmark suites while running in seconds. Subsequent releases continue to push the size ceiling. The throughline from Müller's PFNs to TabPFN v2 is the same bet: replace per-dataset optimisation with one-time amortised pre-training over a well-chosen prior.
Further Reading
- Müller et al. (2022), "Transformers Can Do Bayesian Inference": the PFN foundation
- Hollmann et al. (2023), "TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second": the original tabular PFN
- Hollmann et al. (2025), "Accurate predictions on small data with a tabular foundation model": TabPFN v2 in Nature
- TabPFN GitHub: the implementation used in this post
Interactive Tools
- Classification Metrics Calculator: Turn the accuracy numbers from the benchmark into precision, recall, and F1; TabPFN's calibrated probabilities make threshold choice especially worthwhile
- Overfitting Explorer: See the bias-variance trade-off that gradient boosting tunes against and TabPFN sidesteps by amortising over a prior
Related Posts
- Linear Regression from Scratch: Five Solvers, One Answer: The fit-coefficients-and-discard-the-data paradigm that TabPFN abandons; a useful contrast for what "no training" really means
- PCR vs PLS: When Fewer Features Beat More: Dimensionality handling for tabular regression, the regime where TabPFN's feature limits matter most
- SMOTE vs Class Weights: Imbalanced Classification: Where calibrated probabilities like TabPFN's pay off, and how to handle skewed tabular classes
- Bayesian Model Comparison: The Evidence Framework: The Bayesian averaging TabPFN approximates in a forward pass, done explicitly
- AIC and BIC: Choosing the Right Model Without Overfitting: Classical model selection, the per-dataset tuning loop that TabPFN replaces with amortised pre-training
- Random Forests from Scratch: Bagging and OOB Error: The classic tabular baseline TabPFN is benchmarked against






