Please verify you are human to use the calculator.
The ordinary least squares (OLS) method finds the line or curve that minimises the sum of squared residuals — the vertical distances between each data point and the fitted model.
R² and adjusted R² quantify how well the model explains variation in the data. The residual standard error measures the typical size of prediction errors.
When the relationship is nonlinear, polynomial regression fits curves of degree 2, 3, or 4. The normal equations (X'X)-1X'y solve for the optimal coefficients.
The residual plot reveals whether the model captures the underlying pattern. Random scatter means a good fit; systematic patterns suggest a better model exists.
Regression analysis is one of the most widely used statistical techniques in data science and machine learning. At its core, it answers the question: given some input values, what output should we expect? The simplest form — linear regression — fits a straight line y = a + bx through your data using the normal equations, finding the slope b and intercept a that minimise prediction error.
For a deeper dive into different regression implementations, see our tutorial on Linear Regression Five Ways, which walks through OLS, gradient descent, scikit-learn, PyTorch, and Bayesian approaches. If you are working with high-dimensional data, our guide on PCR vs PLS for Dimensionality Reduction in Regression covers how to handle multicollinearity and reduce feature spaces.
When choosing a model, the bias-variance tradeoff is key. A linear model may underfit curvy data (high bias), while a high-degree polynomial may overfit noise (high variance). Techniques like cross-validation and regularisation help find the right balance. For systematic approaches to model tuning, see our post on Hyperparameter Optimisation: Grid, Random & Bayesian Search.
Interactive walkthrough of polynomial overfitting from Bishop PRML Ch. 1. Explore degree selection, train/test error, data size effects, and ridge regularisation.
Try itInteractive Gaussian process regression with RBF kernel. Add data points, adjust hyperparameters, and see posterior mean with uncertainty bands.
Try itEdit a 2x2 matrix and read its eigenvectors and eigenvalues off the plot, then switch to SVD to see the unit circle stretched into an ellipse.
Try itFree calculators for t-tests, chi-square, ANOVA, Mann-Whitney U, confidence intervals, sample size, power analysis, and effect size.
Try itGet notified when new tutorials and deep-dives are published. No spam, unsubscribe anytime.
Common questions about linear and polynomial regression