Fourier Transform for Time Series Forecasting

Decompose signals with the FFT, filter noise, visualise time-frequency with the Gabor transform, and forecast with one neural network per frequency band.

Cover Image for Fourier Transform for Time Series Forecasting

A sensor trace mixes slow changes with quick fluctuations, and the pattern you care about can be hard to see in the raw signal. Looking at its frequencies gives you another view of those changes. The Fourier transform separates contributions at different frequencies, helping us examine repeating structure and the effects of filtering before trying to forecast the series.

Separating frequencies helps fitting, but forecast errors accumulate. Per-band networks exceed R² = 0.999 in sample, yet drift over 80 steps. We recover a pulse with a Gaussian filter, build a Gabor spectrogram and train the networks to check that short-horizon limit.

Decomposing a Signal

Take a composite signal made of three sine waves at 5, 23, and 50 Hz. In the time domain, the waveform looks complex. But the FFT instantly reveals the three constituent frequencies as sharp spikes in the frequency spectrum:

Composite signal of three sine waves (5, 23, 50 Hz) in the time domain (top), and its FFT frequency spectrum showing three clean spikes at the correct frequencies (bottom).

Let's Build It

Click the badge to run this yourself:

Open In Colab

import numpy as np
import matplotlib.pyplot as plt

# Composite signal: 3 sinusoids
n = 512
t = np.linspace(0, 1, n, endpoint=False)
fs = n  # sampling frequency (512 Hz)

signal = (2*np.sin(2*np.pi*5*t) +
          1.5*np.sin(2*np.pi*23*t) +
          0.8*np.sin(2*np.pi*50*t))

# FFT: time domain → frequency domain
F = np.fft.fft(signal)
freqs = np.fft.fftfreq(n, d=1/fs)

# One-sided magnitude spectrum
magnitude = 2 * np.abs(F[:n//2]) / n
Three peaks at exactly 5, 23, and 50 Hz
with magnitudes 2.0, 1.5, and 0.8 — matching the input amplitudes.

The FFT takes nn time-domain samples and returns nn complex coefficients. Each coefficient encodes the amplitude and phase of a sine wave at a specific frequency. The magnitude tells you "how much" of that frequency is present; the phase tells you "where" the sine wave starts.

What Just Happened?

The DFT Formula

The Discrete Fourier Transform converts a sequence x[0],x[1],…,x[N−1]x[0], x[1], \ldots, x[N-1] into frequency coefficients:

X[k]=∑n=0N−1x[n] e−i2πkn/NX[k] = \sum_{n=0}^{N-1} x[n] \, e^{-i 2\pi k n / N}

Each X[k]X[k] is a complex number. Its magnitude ∣X[k]∣|X[k]| gives the amplitude of the frequency component at k⋅fs/Nk \cdot f_s / N Hz. The inverse transform reconstructs the original signal exactly:

x[n]=1N∑k=0N−1X[k] ei2πkn/Nx[n] = \frac{1}{N} \sum_{k=0}^{N-1} X[k] \, e^{i 2\pi k n / N}

This perfect reconstruction is the key property we exploit throughout this post: anything we do in the frequency domain can be undone by the inverse FFT.

Band-Pass Filtering: Splitting the Signal

The Fourier transform is linear. If a signal is the sum of several components, its FFT is the sum of their individual FFTs. This means we can decompose a signal into frequency bands, process each band independently, and sum the results to recover the original.

def bandpass(F, freqs, f_low, f_high):
    """Keep only frequencies in [f_low, f_high)."""
    mask = (np.abs(freqs) >= f_low) & (np.abs(freqs) < f_high)
    F_band = np.zeros_like(F)
    F_band[mask] = F[mask]
    return np.real(np.fft.ifft(F_band))

# Decompose into three bands
low  = bandpass(F, freqs, 0, 15)    # catches the 5 Hz component
mid  = bandpass(F, freqs, 15, 35)   # catches the 23 Hz component
high = bandpass(F, freqs, 35, 70)   # catches the 50 Hz component

# Verify: sum of bands = original (to machine precision)
error = np.max(np.abs(signal - (low + mid + high)))
# error ≈ 3.3e-14

Band-pass decomposition: original composite signal (top), then the low, mid, and high frequency bands isolated. Each band contains a clean sine wave at the expected frequency.

The low band isolates the slow 5 Hz wave, the mid band extracts the 23 Hz oscillation, and the high band captures the fast 50 Hz component. Summing them recovers the original to machine precision (∼10−14\sim 10^{-14}). This is the same linearity property the original R code exploits: "summed up, the sub-signals make up the original signal since FFT is a linear operation."

Going Deeper: Filtering Noise

Gaussian Filter in the Frequency Domain

Suppose a radar transmits a pulse, u(t)=sech(t)u(t) = \text{sech}(t), but the received signal is corrupted by white noise. The noise affects all frequencies equally, so the time-domain signal looks like random static. Can we recover the original pulse?

The Gaussian filter multiplies the frequency-domain signal by a bell curve centred at the known signal frequency k0k_0:

Ffiltered(k)=F(k)⋅e−b(k−k0)2F_{\text{filtered}}(k) = F(k) \cdot e^{-b(k - k_0)^2}

Frequencies near k0k_0 pass through; distant frequencies (mostly noise) are suppressed.

def gaussian_filter(F, k, k0, bandwidth):
    return F * np.exp(-bandwidth * (k - k0)**2)

# Filter the noisy signal (k0=0 is where sech(t) lives)
F_filtered = gaussian_filter(F_noisy, k, k0=0, bandwidth=0.2)
recovered = np.fft.ifft(F_filtered)

Three panels: clean sech(t) pulse (top), signal buried in heavy white noise (middle), recovered signal after Gaussian filtering at k₀=0 closely matching the original (bottom).

The filter succeeds because we know where in the frequency domain the signal lives. If we centred the filter at k0=15k_0 = 15 instead (where there is no signal), the filter would find only noise. This is the fundamental trade-off: a frequency-domain filter requires prior knowledge about which frequencies matter.

Signal Averaging: When You Have Repetitions

What if you do not know the signal frequency, but you can observe the signal multiple times? White noise has zero mean, so averaging multiple noisy observations causes the noise to cancel while the signal accumulates:

Four panels showing 1, 5, 10, and 100 averaged repetitions of a noisy sech(t) signal. At 1 repetition, the signal is invisible. By 100 repetitions, the clean pulse emerges clearly.

After 100 repetitions, the sech(t)\text{sech}(t) pulse emerges cleanly from the noise. The noise standard deviation decreases as 1/N1/\sqrt{N}, so 100 averages reduce it tenfold. This works because the signal is stationary: it stays at the same position in every measurement. For non-stationary signals (a moving target, a chirp, a time series with evolving frequency content), simple averaging fails. That is where the Gabor transform comes in.

The Gabor Transform: Time Meets Frequency

The standard FFT tells you which frequencies are present, but not when they occur. For a non-stationary signal whose frequency content changes over time, we need both. The Gabor transform (also called the Short-Time Fourier Transform) slides a Gaussian window across the signal and takes the FFT at each position:

G(τ,ω)=∫−∞∞x(t) e−α(t−τ)2 e−iωt dtG(\tau, \omega) = \int_{-\infty}^{\infty} x(t) \, e^{-\alpha(t-\tau)^2} \, e^{-i\omega t} \, dt

The window e−α(t−τ)2e^{-\alpha(t-\tau)^2} localises the signal around time τ\tau. At each position, the FFT reveals the local frequency content.

Watch the Gabor window slide across a complex non-stationary signal. At each position, the frequency spectrum changes:

Animated GIF: Gaussian window sliding across a multi-component signal. Three panels show: the signal with window overlay, the windowed segment, and the local frequency spectrum changing at each position.

The resulting spectrogram shows how frequency content evolves over time:

# Gabor spectrogram
spectrogram = np.zeros((n//2, len(window_centers)))
for j, tau in enumerate(window_centers):
    window = np.exp(-bandwidth * (t - tau)**2)
    windowed = signal * window
    F_windowed = np.fft.fft(windowed)
    spectrogram[:, j] = np.abs(F_windowed[:n//2])

Non-stationary signal (top) and its Gabor spectrogram (bottom). The chirp component forms a V-shaped ridge that sweeps from high frequency down to zero and back up, while two stationary sinusoids appear as horizontal bands.

The spectrogram reveals structure that is invisible in both the time domain (too complex) and the standard frequency domain (loses all temporal information). The chirp component cos⁡(3(t−6)2)\cos(3(t-6)^2) appears as a V-shaped ridge: its instantaneous frequency decreases from ~6 Hz at t=0t=0 down to zero at t=6t=6, then increases again. The two stationary sinusoidal components (sin⁡(2t)\sin(2t) at ~0.3 Hz and sin⁡(5t)\sin(5t) at ~0.8 Hz) appear as horizontal bands.

There is a fundamental limit to this approach. A narrow window gives good time resolution but poor frequency resolution; a wide window gives the opposite. This is the Heisenberg uncertainty principle for time-frequency analysis: you cannot simultaneously know exactly when and at what frequency a signal component occurs. The Gabor transform with a Gaussian window achieves the theoretical optimum for this trade-off.

Forecasting with FFT + Neural Networks

The original R code's most ambitious idea combines everything above into a forecasting pipeline:

  1. Decompose the signal into frequency bands via band-pass filtering
  2. Train one neural network per band (each band is simpler than the original)
  3. Forecast each band auto-regressively (predict one step, feed back, repeat)
  4. Sum the forecasts (exploiting linearity)
from sklearn.neural_network import MLPRegressor

# Train one MLP per frequency band
models = []
for sub in sub_signals:
    X_train, y_train = make_sliding_windows(sub, window=4)
    mlp = MLPRegressor(hidden_layer_sizes=(4,), max_iter=3000,
                       activation='tanh', solver='lbfgs')
    mlp.fit(X_train, y_train)
    models.append(mlp)

# Forecast: predict one step, feed back, repeat
for step in range(n_forecast):
    pred = model.predict(current_window)
    current_window = np.roll(current_window, 1)
    current_window[0, 0] = pred

Per-band decomposition showing four frequency bands (DC trend, 0.5 Hz, 2.0 Hz, 5.0 Hz). In-sample MLP fits (solid colours) match the sub-signals perfectly. Out-of-sample forecasts (dashed) diverge over the long horizon.

The in-sample fits are excellent (R^2 > 0.999 for all bands). But the auto-regressive forecasts diverge because each prediction error feeds into the next step, compounding over time:

Full forecast: in-sample fit (blue dashed) closely tracks the true signal (grey). Out-of-sample forecast (red) tracks well for the first few steps but diverges over the 80-step horizon.

The R code forecast only 14 steps ahead and used a window of 4 observations, matching the signal's periodicity. The key insight is that this approach works best for short-horizon forecasts of highly periodic signals. The more periodic the signal (fewer dominant frequencies, higher SNR), the better the per-band MLPs can extrapolate. For financial time series, where the signal-to-noise ratio is low and non-stationarity is the norm, this approach faces the same challenges as any frequency-domain method: the frequencies you discover in historical data may not persist into the future.

Where This Comes From

The Fourier transform traces back to Joseph Fourier's 1807 memoir on heat conduction, where he showed that any periodic function can be represented as a sum of sines and cosines. The discrete version became computationally practical with the Cooley-Tukey algorithm in 1965, which reduced the complexity from O(N2)O(N^2) to O(Nlog⁡N)O(N \log N).

"An arbitrary function, continuous or with discontinuities, defined in a finite interval by an arbitrarily capricious graph can always be expressed as a sum of sinusoids."

Fourier, J. (1807). Mémoire sur la propagation de la chaleur.

The Cooley-Tukey FFT is the algorithm that numpy.fft.fft actually runs. It works by recursively splitting the DFT of size NN into two DFTs of size N/2N/2, exploiting the symmetry of the complex exponentials. This is why FFT performance is best when NN is a power of 2.

The Gabor transform was introduced by Dennis Gabor in his 1946 paper "Theory of communication," where he proposed using Gaussian-windowed sinusoids as the elementary signals for communication. He showed that these "logons" (Gaussian-enveloped sinusoids) achieve the minimum time-frequency uncertainty, analogous to the Heisenberg uncertainty principle in quantum mechanics. Gabor received the 1971 Nobel Prize in Physics for the invention of holography, a related application of wave interference.

The idea of training separate neural networks per frequency band is not from any single paper but reflects a practical engineering insight: simpler signals are easier to learn. The original R code (2015) applied this to order volume forecasting, using R's nnet package (single-layer MLPs) on band-pass filtered components of weekly business data. The linearity of the FFT guarantees that summing the per-band predictions exactly reconstructs the full signal prediction.

Further Reading

Try It Yourself

  1. Add noise to the 3-component signal. Add Gaussian noise and see how the FFT peaks broaden. At what noise level do the peaks become undetectable?
  2. Build a spectrogram for audio. Load a .wav file and compute its Gabor spectrogram. Can you identify the vowels in a spoken word from their frequency signatures?
  3. Try a financial time series. Apply band-pass decomposition to SPY daily returns. Do any frequency bands have predictive power, or does the efficient market hypothesis hold?
  4. Compare window functions. Replace the Gaussian window in the Gabor transform with a rectangular window and a Hann window. How does the spectrogram change?

Stay in the loop

Get notified when new tutorials and deep-dives are published. No spam, unsubscribe anytime.

Related Posts

Frequently Asked Questions