Please verify you are human to use the playground.
A GP starts with a prior distribution over functions (defined by the kernel) and conditions on observed data to produce a posterior distribution. The posterior mean is the best prediction, and the posterior variance quantifies uncertainty.
The RBF kernel has two key parameters: the length scale controls smoothness (how far apart points can be and still correlate), and the signal variance controls the amplitude of the function.
Unlike point-estimate models, GPs naturally quantify uncertainty. Predictions far from training data have wide confidence bands, while regions near observations are tight. This is invaluable for active learning and decision-making.
The noise variance parameter models observation noise. With zero noise the GP interpolates exactly; with higher noise it produces a smoother fit that does not pass through every point, better capturing the underlying trend.
Gaussian Process regression is a powerful non-parametric Bayesian approach to function approximation. Unlike parametric models that commit to a fixed functional form, a GP defines a distribution over all possible functions and uses observed data to narrow this distribution down. The result is not just a single prediction, but a full probability distribution at every input point.
For a complete implementation walkthrough, see our tutorial on Gaussian Process Regression from Scratch, which covers the math, code, and intuition behind GP regression step by step. GPs are also a core component of Bayesian optimisation, which we explore in our guide on Hyperparameter Optimisation: Grid, Random & Bayesian Search.
The key computational step is inverting the kernel matrix K, which scales as O(n³) for n data points. For small to moderate datasets (up to a few hundred points), this is fast enough for real-time exploration. For larger datasets, approximation methods like sparse GPs or random Fourier features are used instead.
Interactive linear and polynomial regression calculator. Enter data points, fit models, and visualise residuals with live R² and coefficient estimates.
Try itEdit a 2x2 matrix and read its eigenvectors and eigenvalues off the plot, then switch to SVD to see the unit circle stretched into an ellipse.
Try itInteractive walkthrough of polynomial overfitting from Bishop PRML Ch. 1. Explore degree selection, train/test error, data size effects, and ridge regularisation.
Try itInteractive probability distribution calculators. Visualise Normal, Binomial, Poisson, Beta, and 12 more distributions with live PDF/CDF charts.
Try itGet notified when new tutorials and deep-dives are published. No spam, unsubscribe anytime.
Common questions about Gaussian Processes and kernel methods