Signal processing, from first principles to working practice

Sixty-two modules covering the whole roadmap. Every figure on this page is computed live in your browser from the controls beneath it — nothing here is a picture of a result, it is the result. Move a slider and the mathematics runs again.

How the modules are built

Each module follows the same shape: a short explanation of what the idea is and why anyone bothered inventing it, one or more interactive panels where you can break it, the formulas with every symbol defined, and a few questions to check that the idea landed rather than just the words.

The bench at the top of the screen

Two parameters follow you everywhere: the sampling rate fs and the record length N. Nearly every quantity in digital signal processing is derived from those two numbers, and the strip at the top shows the derivations updating as you work: the Nyquist frequency fs/2, the bin spacing Δf = fs/N, and the record duration T = N/fs. Change the sampling rate in the sampling module and the FFT module inherits it.

One signal, threaded through the course

Rather than a new toy example every module, most panels analyse the same synthetic measurement: an accelerometer on a gearbox test rig. It is deliberately awkward. It contains a shaft rotation and its harmonics, a gear mesh tone with sidebands, a bearing fault that rings after each impact, mains pickup, a slow thermal drift, and broadband noise. It is the signal the roadmap's closing section asks you to be able to analyse, and by the end of these modules you will have taken it apart with every tool here.

Suggested order

The stages in the left rail run in dependency order, and the roadmap's own recommended progression is built into them. If you are short of time, the Core path button in the rail hides everything except the modules that later ones actually depend on. That leaves twenty-six modules which take you from "what is a signal" to a defensible spectral analysis with a designed filter.

What a signal is

A signal is a measurable quantity that carries information about how something is behaving. The mathematics does not care whether the quantity is a voltage, a pressure, a temperature or a share price.

Formally a signal is a function: it maps an independent variable onto a value. Almost everything else in this course follows from asking two questions about that function — is the independent variable continuous or discrete, and is the value continuous or discrete?

The independent variable need not be time

Time is the usual one, written x(t). But a signal can equally be a function of space (the intensity along a line of a photograph), of two spatial coordinates (an image, I(x,y)), of frequency (a spectrum), of angle, or of sample index. Every technique here transfers: a spatial low-pass filter blurs an image for exactly the reason a temporal low-pass filter smooths a noisy measurement, and spatial aliasing produces moiré patterns for exactly the reason temporal aliasing produces a wagon wheel that turns backwards.

The notation matters and is worth being pedantic about early: round brackets mean a continuous argument, square brackets mean an integer index. x(0.5) is a legitimate question; x[0.5] is not, because there is no sample at index one half. Interpolation is the process of inventing an answer to that illegitimate question, and module 45 is about how honest that invention is.

Discrete-time is not the same as digital

A discrete-time signal is sampled in time but its values may still be real numbers of unlimited precision. A digital signal is discrete in both axes: sampled in time and quantised in amplitude. The distinction is not pedantry — sampling and quantisation cause different problems, are analysed with different mathematics, and are fixed by different means. Sampling too slowly causes aliasing, which is irreversible and is treated in module 15. Quantising too coarsely adds noise, which is tolerable and is treated in module 16.

Classifying signals

The classifications are not taxonomy for its own sake. Each one tells you which tools are legal to use, and several classic mistakes are simply the use of a tool on a signal of the wrong class.

Every signal splits into an even and an odd part

This decomposition is not a curiosity — it is why the Fourier transform of a real signal has a real part that comes entirely from the even content and an imaginary part that comes entirely from the odd content.

Signal operations

Five operations account for most of what is ever done to a signal before it reaches a system: scale the amplitude, add signals, shift in time, scale time, and multiply signals together.

The order of operations is where mistakes happen. In x(a(t − t0)) the shift is applied to t before scaling, so the feature that was at t = 0 in the original appears at t = t0 in the result regardless of a. Written the other way, x(at − t0), the same feature appears at t0/a. The panel below applies them in the bracketed order and draws the reference feature so you can watch where it lands.

Why multiplication deserves its own module

Adding two signals puts both spectra side by side. Multiplying two signals does something far more interesting: it moves them. Multiplying by a sinusoid shifts a spectrum bodily up and down the frequency axis, which is what modulation, mixing and heterodyning all are (module 39). Multiplying by a finite-length window is what every practical FFT does whether you asked for it or not, and it convolves the true spectrum with the window's own spectrum — the origin of spectral leakage (module 18).

Elementary signals

A dozen standard signals appear again and again, both as building blocks and as test inputs. Recognising them on sight, and knowing their transforms, removes a surprising amount of work later.

The impulse, and the property that makes it useful

The continuous-time impulse δ(t) is not a function in the ordinary sense. It is defined by what it does inside an integral: it has zero width, infinite height, unit area, and it picks out the value of whatever it multiplies. That is the sifting property, and it is the single most useful identity in the subject.

The discrete version is much better behaved: δ[n] is an ordinary sequence, equal to one at n = 0. The sum above says any sequence is a sum of shifted, scaled impulses — which is precisely the observation that leads to convolution in module 08.

Energy, power and the numbers that describe amplitude

"How big is this signal" has at least five defensible answers, and choosing the wrong one is a common source of disagreement between two people looking at the same data.

An energy signal has finite energy and therefore zero average power — every transient, impact and pulse. A power signal has infinite energy but finite average power — every sinusoid, every noise process, every steady machine vibration. No signal is both, and the classification decides whether you should be plotting an energy spectral density or a power spectral density.

Crest factor: the number that catches clipping

The ratio of peak to RMS is the single most useful diagnostic in the list. A pure sine has a crest factor of exactly √2 (3.01 dB). Gaussian noise sits around 3 to 4 depending on record length. A bearing fault, which is a train of sharp impacts sitting on top of a smooth vibration, pushes it well above that — which is why crest factor is a standard condition-monitoring feature. And a signal that has been clipped by the ADC has its crest factor squashed towards 1, which is how you detect clipping without ever looking at the raw trace.

What a system does, and the six questions to ask about it

A system maps an input signal to an output signal. Six properties decide which analytical tools are available to you — and if two of them hold, an enormous amount follows for free.

The six properties

Linearity is two conditions at once. Additivity: the response to a sum is the sum of the responses. Homogeneity: scaling the input scales the output by the same factor. Together they give superposition, which is what lets you decompose an input into simple pieces, process them separately, and add the answers.

Time invariance means the system does not care when you apply the input: delay the input and the output is delayed by the same amount, otherwise unchanged. A filter with fixed coefficients is time invariant; an adaptive filter, by design, is not. Causality means the output at any instant depends only on present and past inputs — mandatory for anything running in real time, optional for offline processing (module 48). BIBO stability means every bounded input produces a bounded output. Memory: a memoryless system's output depends only on the input at the same instant; anything with memory holds state. Invertibility: distinct inputs give distinct outputs, so the input can in principle be recovered.

Linear time-invariant systems

If a system is both linear and time invariant, one measurement characterises it completely and for ever: its response to a single impulse.

The argument is three lines long and is worth following, because everything else in the course rests on it. Any input can be written as a sum of shifted, scaled impulses (module 04). Time invariance says the response to a shifted impulse is a shifted impulse response. Linearity says the response to the sum is the sum of the responses. Therefore:

That is convolution, and module 08 is devoted to it. The point here is what it means: h completely determines the system. Measure it once and you can predict the response to any input whatsoever, without ever knowing what is inside the box.

Three faces of the same information

The impulse response, the step response and the frequency response are three views of one object. The step response is the running integral of the impulse response (because the step is the running integral of the impulse, and the system is linear). The frequency response is the Fourier transform of the impulse response. Measuring whichever is most convenient gives you the other two.

Why sinusoids, and not some other family

Feed a complex exponential into an LTI system and the output is the same complex exponential, scaled by a complex constant. The frequency is untouched. No other family of signals has this property for all LTI systems — sinusoids are the eigenfunctions of LTI systems, and the complex constant is the eigenvalue.

This is the entire justification for frequency-domain thinking. An LTI system cannot create a frequency that was not in its input — it can only change the amplitude and phase of what was already there. If you see a new frequency at the output of a supposedly linear system, the system is not linear, and that observation is a standard diagnostic (module 44 on harmonic distortion).

Convolution

Flip one signal, slide it across the other, and at every position multiply and add. That is the whole operation — and it is what every linear filter physically does.

Doing it graphically

The four steps never change: reflect h about the origin, shift it to position n, multiply point by point with x, and sum the products. The slider below moves n; the shaded area is the product being summed, and its total is the single output value plotted at that n. Work through it once by hand and you will never need to again.

The properties, and why they are useful

The result that changes how you work

Filtering is therefore a choice of viewpoint. In the time domain the filter slides a kernel along the data; in the frequency domain it multiplies the spectrum by a gain curve. They are the same operation. The practical consequence is computational: convolving two long signals directly costs order N·M operations, while transforming, multiplying and transforming back costs order N log N. Module 51 covers where the crossover sits and how overlap-add handles continuous streams.

Frequency-domain thinking

The same signal, described two ways. Neither description is more true than the other, but for any given question one of them is usually much easier.

The time domain answers questions about when: when did the impact occur, how long did the transient last, what is the peak value. The frequency domain answers questions about what: what is rotating, at what rate, and which resonance is being excited. A signal that looks like meaningless hash in time can be three clean lines in frequency, and a spectrum that looks featureless can hide an obvious event in time.

The vocabulary

Fourier series

Any reasonable periodic signal is a sum of sinusoids at integer multiples of its fundamental frequency. The coefficients tell you how much of each.

The complex form is not a different theory, it is the same theory written more compactly — one complex coefficient per harmonic instead of two real ones, and negative frequencies carrying the conjugate half. Every result later in the course is stated in the complex form, so it is worth becoming comfortable with it now.

Building waveforms from harmonics

The classic three shapes each have a characteristic harmonic signature, and recognising them in a real spectrum is a genuinely useful skill: odd harmonics only with 1/n amplitudes means a square wave somewhere in the chain; odd harmonics with 1/n² means a triangle; all harmonics with 1/n means a sawtooth.

Gibbs phenomenon

Add more harmonics to a square wave and the approximation improves everywhere except at the discontinuity, where an overshoot of about 9 % of the step height stubbornly refuses to shrink. It gets narrower as harmonics are added but never shorter. This is not a numerical artefact — it is what a truncated series does at a jump, and it is the same effect that produces ringing at the edges of a sharply filtered signal, and overshoot around impulsive events in filtered data.

The continuous Fourier transform

Let the period of a Fourier series go to infinity and the discrete harmonics merge into a continuum. The result applies to any signal, periodic or not.

The pairs worth memorising

Six transform pairs cover most of what you will meet. The panel lets you vary the width of each and watch the other side respond — always in the opposite direction, which is the reciprocal spreading principle: narrow in time is wide in frequency.

The properties, and what each one is used for

The Laplace transform

Generalise the Fourier transform by letting frequency become complex. You gain the ability to describe growth and decay, and with it a clean account of stability.

That last proviso is the whole reason the Laplace transform exists. A growing exponential has no Fourier transform — the integral does not converge. It does have a Laplace transform, valid in the region of the s-plane where e−σt decays fast enough to win. The region of convergence is part of the answer, not a footnote: the same algebraic expression can describe a stable causal system or an unstable anticausal one, and only the ROC distinguishes them.

Transfer functions, poles and zeros

The geometry is the intuition. To find the frequency response at some frequency ω, stand at the point s = jω on the imaginary axis and measure the distances to every pole and every zero. The gain is the product of the zero distances divided by the product of the pole distances. Move close to a pole and the gain rises — that is a resonance. Move close to a zero and it falls — that is a notch. The panel below draws those distances as you drag the poles.

Sampling theory

Sampling replaces a continuous signal with a list of numbers. The remarkable result is that under one condition, nothing is lost — the continuous signal can be reconstructed exactly.

What sampling does in the frequency domain

Sampling multiplies the signal by an impulse train. Multiplication in time is convolution in frequency, and the transform of an impulse train is another impulse train — so the spectrum is replicated at every multiple of f_s. If the replicas do not overlap, a low-pass filter can isolate the original and the signal is recovered exactly. If they do overlap, the overlapping parts add together and are permanently indistinguishable. That overlap is aliasing.

Aliasing

A frequency above the Nyquist limit does not disappear. It reappears somewhere else, indistinguishable from a genuine signal, and no amount of later processing can remove it.

Two sinusoids whose frequencies differ by a multiple of f_s produce identical samples. Not similar — identical. The information needed to tell them apart was never recorded. This is why an anti-aliasing filter must be analog and must sit before the ADC: once the samples exist, it is too late.

The roadmap's sampling experiment

Roadmap project 2 asks you to sample a 100 Hz sinusoid at 1000, 300, 210, 150 and 90 Hz and observe what happens. The table below runs it. Note 210 Hz: comfortably above the Nyquist rate of 200 Hz, and the result is correct but visually unconvincing — only 2.1 samples per cycle. Correct sampling and useful-looking sampling are different standards.

Aliasing in other independent variables

Sample space too coarsely and you get spatial aliasing: moiré patterns in a photograph of a striped shirt, jagged edges in an undersampled render, and — in sensor arrays — an ambiguity about which direction a wave arrived from, because the array elements sample the wavefront in space. Sample time too coarsely with a camera and a rotating propeller appears stationary or turns backwards. It is one phenomenon.

Analog-to-digital conversion

Four stages, each with its own failure mode: filter, sample, hold, quantise.

Quantisation

An N-bit converter has 2^N levels spread across its input range. The step size and the resulting noise follow immediately:

The 6.02 dB per bit is worth internalising: each extra bit halves the step and buys 6 dB. But the 1.76 dB assumes a sine filling the whole range. A signal using only a tenth of the range loses 20 dB of that, which is why input gain staging matters as much as converter resolution. A 16-bit converter used across 5 % of its range delivers roughly the performance of a 12-bit one.

Saturation and clipping

Exceed the input range and the converter simply stops following. The flat tops that result are not a gentle distortion — they are a discontinuity in the slope, and they generate harmonics across the entire spectrum. Clipped data cannot be repaired, and analysis of it produces confident, wrong answers. Detect it by checking for repeated extreme values, or by watching the crest factor collapse (module 05).

Digital-to-analog conversion and reconstruction

Getting back out is not simply the reverse of getting in. The staircase a DAC produces is not the original signal, and two corrections are needed.

A practical DAC holds each sample value until the next one arrives — a zero-order hold. That staircase contains the wanted signal plus images: copies of the spectrum centred on every multiple of f_s, which were always there in the sampled representation and are now radiated into the analog world unless removed. It also imposes a gentle low-pass tilt, because holding a value for T_s is convolution with a rectangle of width T_s, and that is a sinc in frequency.

So a complete reconstruction path needs a reconstruction filter to suppress the images, and often a sinc-compensation stage — a mild high-frequency boost — to undo the droop. Oversampling makes both jobs much easier by pushing the first image far away from the wanted band, which is why audio converters run at many times the nominal rate internally.

The discrete Fourier transform

N samples in, N complex numbers out. Each output is the correlation of your data with one complex exponential.

Negative frequencies and the two-sided spectrum

The second half of the output, bins N/2+1 to N−1, represents negative frequencies. For real input these are the complex conjugates of the first half and carry no new information, which is why instruments show a one-sided spectrum with the amplitudes doubled. For complex input — IQ data from a radio or radar receiver — the two halves are genuinely different, and discarding one destroys the ability to tell an upward Doppler shift from a downward one (module 38).

DFT and FFT are not the same kind of thing

The DFT is a mathematical transform: a definition. The FFT is an algorithm for computing it quickly, by exploiting the symmetry of the exponentials to reduce the cost from N² to N log₂N operations. The answers are identical. For N = 4096 the saving is a factor of about 340; for N = 1048576 it is nearly 90 000, which is the difference between practical and impossible.

Getting amplitudes right

Raw FFT output is not in engineering units. To read a sinusoid's amplitude off the plot you must divide by N, double for the one-sided spectrum, and divide by the window's coherent gain. Miss any of those and you will be out by a factor of two, N, or 2.3 — module 19 covers the window correction and module 61 the full estimation procedure.

Spectral leakage

The DFT assumes your record repeats for ever. When the ends do not match, it sees a discontinuity that was never in the signal — and reports the frequencies needed to make one.

The transform of an N-sample record is exactly the transform of an infinite signal formed by joining copies of that record end to end. If the record happens to contain a whole number of cycles, the joins are seamless and the tone lands exactly in one bin. If it does not, each join is a step change, and the spectrum acquires the broadband content of a step: energy smeared across every bin.

What leakage costs you

Two things. First, amplitude error: a tone falling between bins is split between neighbours and its peak reads low — up to 3.92 dB low with no window, which is the worst-case scalloping loss. Second, masking: the skirts of a strong component bury weak ones nearby. A −13 dB first side lobe means a component 20 dB down and a few bins away is simply invisible. Window functions trade the first problem against the second.

Window functions

Taper the ends of the record to zero and the periodic extension becomes continuous. The discontinuity goes away, and with it most of the leakage — at a price.

The price is resolution. Every window widens the main lobe in exchange for lowering the side lobes, and there is no window that does both. The choice is therefore a question about your measurement: are you trying to separate two close components of similar size, or find a small one next to a large one, or measure an amplitude accurately?

The three figures of merit

All three are computed live in the panel above from the actual window samples, not quoted from a table. They match the published values in Harris (1978), which is the paper every window table in every textbook derives from.

Choosing one

Zero padding, and what resolution actually means

Appending zeros to a record gives a smoother-looking spectrum with more points. It does not let you see anything that was not already resolvable.

Zero padding interpolates. The underlying spectrum — the transform of the windowed record — is a continuous function, and the DFT samples it at N points. Padding to 4N samples it at 4N points, revealing the shape between the original bins. That is genuinely useful: it makes peak interpolation easier, improves the readability of a plot, and reduces apparent scalloping. What it cannot do is separate two components that the record duration was too short to distinguish, because the width of the main lobe is set by the duration of the actual data and padding does not change it.

Resolution versus sampling rate

These are independent and constantly conflated. The sampling rate sets the highest frequency you can see: f_s/2. The record duration sets the finest frequency difference you can resolve: roughly 1/T. Sampling ten times faster for the same number of samples gives you a wider frequency axis and worse resolution, because T has fallen by ten.

Power spectral density

For random signals the spectrum of a single record is a poor estimate — as noisy as the data itself, and it does not improve with more data. The PSD is the estimate that does.

A single periodogram has a standard deviation roughly equal to its own mean, whatever N you use. Doubling the record length gives you twice as many bins, each just as noisy. The fix is to average: split the record into segments, transform each, and average the resulting power spectra. That is Welch's method, and the variance falls in proportion to the number of segments.

The defining property is that a PSD does not depend on how long you recorded or which window you used. A raw spectrum's noise floor drops as you lengthen the record, which is confusing; a PSD's noise floor stays put, and its level is a genuine property of the process. That is what makes PSDs the right currency for noise specifications, vibration standards and sensor datasheets.

Decibels

A logarithmic ratio. The only two things to get right are whether you are dealing with power or amplitude, and what the reference is.

The consequence is that a factor of two in power is +3.01 dB but a factor of two in amplitude is +6.02 dB. Mixing them up produces errors of exactly a factor of two in the answer, which is common enough to be worth a deliberate check whenever a result seems out by 3 or 6 dB.

Suffixes mean references

Noise

Noise is not one thing. Its colour, its distribution and its origin each tell you something different about where it came from and what will remove it.

Two independent descriptions are needed. Colour is how power is distributed across frequency. Distribution is how the amplitude values are spread. They are unrelated: white Gaussian and white uniform noise have the same spectrum and different histograms, and a signal can be Gaussian at every instant while being strongly coloured.

Colours

Signal-to-noise ratio

A single number that decides whether a measurement is possible at all. Almost every processing technique in this trainer is, at bottom, a way of improving it.

Three ways to buy SNR

All three work the same way: the signal adds coherently while the noise adds in power. Averaging 100 triggered repetitions gives 20 dB. Narrowing a 10 kHz measurement to the 100 Hz band the signal actually occupies gives 20 dB. Both require knowing something about the signal — when it occurs, or where it lives in frequency. SNR improvement is always paid for with prior knowledge.

Random signals and stationarity

A random signal has no formula. What it has is statistics — and those only mean something if they hold still.

A stationary process has statistics that do not change with time. Weak (wide-sense) stationarity, which is what practical work relies on, requires only that the mean is constant and the autocorrelation depends on the lag rather than on absolute time. An ergodic process is one where averaging over time gives the same answer as averaging over many realisations — which matters enormously, because you almost always have one long record rather than a thousand short ones.

The distributions worth recognising

Gaussian arises whenever many small independent contributions add — the central limit theorem — which covers thermal noise and most measurement error. Uniform describes quantisation error. Rayleigh describes the magnitude of a complex Gaussian, so it is the distribution of an FFT bin's magnitude in noise, and of fading in radio. Poisson counts rare independent events. Knowing which one applies determines the correct detection threshold (module 41) and whether a large excursion is remarkable or routine.

Correlation and autocorrelation

Slide two signals past each other and measure the overlap at every shift. It finds hidden periodicities, measures delays, and connects directly to the spectrum.

The autocorrelation of white noise is a single spike at zero lag: no sample tells you anything about any other. The autocorrelation of a periodic signal keeps returning to a peak at every period, even when the signal is invisible in the raw time record. That property makes it the classic tool for pulling periodicity out of noise, and it is the basis of pitch detection in speech.

Wiener–Khinchin

This is why a PSD can be computed either by averaging periodograms or by transforming the autocorrelation, and why a narrow spectral peak corresponds to a slowly decaying autocorrelation. It also explains the shape of coloured noise: filtering shapes the spectrum, which necessarily shapes the correlation between neighbouring samples.

Cross-correlation for time delay

If y is a delayed, scaled, noisy copy of x, the peak of R_xy sits at the delay. This is how sonar ranges, how a two-microphone array finds a direction, and how ultrasonic flow meters work. Normalising gives a coefficient between −1 and +1 that also reports how confident the match is, which matters when several peaks compete (module 60).

Filter fundamentals

A filter is an LTI system chosen for the shape of its frequency response. Everything else — the design method, the implementation — follows from a specification written in five numbers.

These five numbers determine the required order. A narrow transition band is expensive: halving it roughly doubles the order of an IIR filter and doubles the length of an FIR one. Before designing anything, ask whether the transition band can be widened — often the signal and the interference are further apart than the first specification assumed.

The four responses

Low-pass keeps frequencies below a cutoff: anti-aliasing, smoothing, envelope extraction. High-pass removes drift and DC offsets. Band-pass isolates a band of interest, such as a single gear-mesh order or a radio channel. Band-stop or notch removes a narrow interferer, typically mains hum. A notch is a band-stop with a very high Q and requires care: too narrow and it drifts off the interference, too wide and it removes signal.

The classical filter families

Four approximations to the impossible ideal, each sacrificing something different. Choosing between them is choosing which imperfection you can live with.

The ideal brick wall is unrealisable — it is non-causal and its impulse response extends infinitely in both directions. Every real filter approximates it, and the classical families differ in where they put the approximation error.

The z-transform, poles and zeros

The discrete-time counterpart of Laplace. It turns difference equations into algebra, and it turns a filter's behaviour into a picture you can read at a glance.

Evaluating H(z) on the unit circle, z = e^jω, gives the frequency response — the unit circle is the frequency axis, running from DC at z = 1 round to the Nyquist frequency at z = −1. A zero near the circle pulls the response down at that angle; a pole near it pushes the response up. The whole frequency response can be read off the pole–zero diagram as a ratio of distances.

Stability

This is a hard rule with practical consequences. A high-order IIR filter implemented as a single long difference equation can have poles that are stable in exact arithmetic but drift outside the circle when the coefficients are rounded to finite precision. Factoring into second-order sections avoids it, which is why every serious implementation uses them (module 32).

FIR filters

A finite impulse response: the output is a weighted sum of the last N inputs and nothing else. Always stable, and capable of exactly linear phase.

Linear phase

If the coefficients are symmetric, b_k = b_N−1−k, every frequency is delayed by exactly the same amount: (N−1)/2 samples. No frequency-dependent distortion of the waveform, and the delay is a known constant you can simply subtract. This is the reason FIR filters dominate anywhere waveform shape or relative timing matters — audio crossovers, radar pulse processing, biomedical signals, anything where a peak's arrival time is the measurement.

Design methods

IIR filters

Feedback makes them enormously more efficient than FIR filters, and introduces every problem that FIR filters do not have.

Designing by transformation

The standard route borrows a century of analog filter theory: design an analog prototype with the classical families, then map it to discrete time. The bilinear transform is the usual mapping. It substitutes s = (2/T)(z−1)/(z+1), which maps the entire left half plane inside the unit circle — so a stable analog filter always becomes a stable digital one.

Implementation structures and finite precision

The same transfer function can be computed in several arrangements. They are equivalent in exact arithmetic and very different in practice.

Direct form I keeps separate delay lines for input and output — more memory, but the intermediate sum has full precision. Direct form II shares one delay line and uses half the storage, at the cost of an internal node whose range can be much larger than either input or output, which is a real overflow risk. The transposed direct form II is usually the best choice in floating point: same low storage, better numerical behaviour.

Second-order sections

Any high-order filter should be factored into cascaded biquads. The reason is coefficient sensitivity: the roots of a high-degree polynomial move violently when its coefficients are perturbed slightly. A tenth-order filter written as a single difference equation can lose stability from rounding alone, while the same filter as five biquads is entirely robust, because each section's two poles depend on only two coefficients.

Fixed point

Fixed-point implementation adds three distinct effects. Coefficient quantisation moves the poles and zeros, changing the response from the one you designed. Round-off noise is injected at every multiply and shaped by the remaining filter stages. Limit cycles are small self-sustaining oscillations that persist with zero input, caused by rounding in the feedback path. Scaling — deliberately attenuating at each stage to prevent overflow — trades headroom against SNR, and getting that balance right is most of the work in a fixed-point design.

Phase, group delay and zero-phase filtering

A filter's magnitude response tells you what survives. Its phase response tells you where it ends up in time — and that is often the measurement.

If group delay is constant, every frequency is held up by the same amount and the waveform emerges intact, merely late. If it varies, different components arrive at different times and the shape is destroyed — a sharp pulse smears out, a step acquires overshoot and ringing, and the apparent arrival time of an event becomes a function of its frequency content.

Zero-phase filtering

Offline, you can escape the problem entirely. Filter the record forwards, reverse it, filter again, and reverse back. The two passes apply the same phase distortion in opposite senses, so it cancels exactly. The magnitude response is squared, so the effective attenuation doubles in dB and the effective cutoff moves.

Smoothing, and differentiating without disaster

The moving average is the most-used and least-understood filter in engineering. And differentiation, done naively, will amplify your noise by a factor of a thousand.

The moving average is a bad low-pass filter

It is optimal for one specific job — minimising white noise while preserving a step edge in the time domain — and it is genuinely the right choice for that. As a frequency-selective filter it is poor: the response is a sinc, with side lobes that leak badly, and a passband that already droops well before the nominal cutoff. If you want a low-pass filter, design one.

When the noise is spikes, not hiss

A single large outlier drags a moving average by (spike ÷ M) across the whole window. A median filter ignores it completely, because a median is insensitive to values that are merely extreme. Medians preserve step edges perfectly too, where a linear filter must always round them off. The cost is nonlinearity: no transfer function, no superposition, and the behaviour on real signals must be checked empirically. The Hampel filter refines this by replacing only points that lie more than a few robust standard deviations from the local median, leaving everything else untouched.

Savitzky–Golay

Fit a low-order polynomial to a sliding window by least squares and take the value of the fit at the centre. Because the fit is linear in the data, the whole operation collapses to a fixed FIR kernel — the famous [−3, 12, 17, 12, −3]/35 for a quadratic over five points. It preserves peak heights and widths far better than a moving average, which is why it dominates in spectroscopy and chromatography, and the same fit gives you smoothed derivatives for free.

Decimation, interpolation and resampling

Changing the sample rate is easy to do and easy to do wrong. Both directions need a filter, and each needs it on a different side of the operation.

Decimation. Keeping every M-th sample is sampling all over again, at f_s/M, so everything above the new Nyquist frequency aliases. The anti-alias filter must come first, cut off below f_s/(2M), and then the discarding is safe. Skipping it is one of the most common bugs in signal-processing code, and it produces plausible-looking spectra with entirely fictitious peaks.

Interpolation. Inserting L−1 zeros between samples leaves the spectrum unchanged but now, at the higher rate, the old replicas sit inside the new band as images. A low-pass filter at the old Nyquist removes them, and its gain must be L to restore the amplitude lost by the zeros.

The short-time Fourier transform

Real signals change. Chop the record into overlapping windows, transform each, and you can watch the spectrum evolve — subject to one unavoidable trade-off.

The trade-off is not a limitation of the method

So the window length is a statement about what you are looking for. Distinguishing two close tones needs a long window. Locating a click needs a short one. If you need both at once, you need more than one spectrogram — or a method that varies its resolution with frequency, which is what wavelets do.

Wavelets

The STFT uses one window length for every frequency. A wavelet transform scales the window with the frequency — short windows for high frequencies, long ones for low.

This matches how signals usually behave. A transient click is brief and broadband; a slow oscillation is long and narrowband. Analysing both with the same window means compromising on both. The wavelet's constant-Q tiling gives fine time resolution where events are fast and fine frequency resolution where they are slow.

Where it earns its place

Transient detection in vibration monitoring — a bearing fault is a train of short impacts that a spectrogram either locates badly or resolves badly. Denoising, by thresholding small wavelet coefficients and reconstructing, which removes noise while keeping edges sharp in a way linear filtering cannot. Compression, where JPEG 2000 uses a wavelet transform. And multi-resolution analysis generally, whenever a signal has structure at several scales at once.

The Hilbert transform, envelopes and IQ

Discard the negative frequencies and the signal becomes complex. Its magnitude is the envelope, its phase derivative is the instantaneous frequency, and it is the foundation of every modern receiver.

Envelope detection

The magnitude of the analytic signal is the envelope, computed without rectifying and low-pass filtering and without the distortion that introduces. This is the standard first step in bearing fault diagnosis: a fault produces impacts that ring the structure at a high resonant frequency, and the useful information is the rate of the impacts, not the resonance. Band-pass around the resonance, take the envelope, then take the spectrum of the envelope — and the fault frequency appears clearly where the raw spectrum showed nothing (module 54).

IQ representation

Splitting the analytic signal into real and imaginary parts gives in-phase and quadrature components. Radio, radar and sonar receivers work in IQ because it halves the required sample rate for a given bandwidth, and because it preserves the sign of a frequency offset — with a real signal you cannot tell an upward Doppler shift from a downward one, and with IQ you can.

Modulation, mixing and heterodyning

Multiplying by a sinusoid shifts a spectrum. That single fact underlies radio, every superheterodyne receiver, and the sidebands that appear around a gear mesh frequency.

Amplitude modulation, and why it matters mechanically

AM produces a carrier plus sidebands at f_c ± f_m. In communications that is a design choice. In machinery it is a diagnosis: if a load varies once per shaft revolution, it amplitude-modulates the gear mesh tone, and the spectrum shows the mesh frequency flanked by sidebands spaced at the shaft rate. The spacing identifies which shaft, and the sideband amplitude tracks the severity. The rig example threaded through this trainer has exactly this structure — 632.5 Hz mesh with ±27.5 Hz sidebands.

Frequency modulation

FM encodes information in the instantaneous frequency instead. Its spectrum is much less obvious — Bessel functions of the modulation index — and its bandwidth is approximated by Carson's rule, B ≈ 2(Δf + f_m). Mechanically, FM sidebands appear when shaft speed fluctuates within a revolution, so an AM/FM mixture in gear sidebands distinguishes a load problem from a torsional one.

Chirps and instantaneous frequency

When frequency changes during the record, a single spectrum describes a state the signal never occupied. You need to follow it instead.

A chirp sweeps frequency with time. Linear chirps are the workhorse of radar and sonar because they can be compressed to a sharp peak after transmission (module 41), and they appear naturally whenever a machine runs up or coasts down. Their whole-record spectrum is a broad, flat plateau spanning the swept range — technically correct and diagnostically useless.

Detection and the matched filter

If you know the shape of what you are looking for, correlating with that shape is provably the best thing you can do in white noise. There is no cleverer filter.

The output SNR depends only on the signal's energy, not on its shape or duration. A long quiet pulse and a short loud one with the same energy detect equally well — which is exactly why radar transmits long coded pulses and compresses them on receive, getting the range resolution of a short pulse with the energy of a long one.

Thresholds and the two ways to be wrong

The ROC curve plots P_D against P_FA as the threshold sweeps. Its shape is fixed by the SNR; the threshold only chooses your position along it. Improving detection therefore means improving SNR, not tuning the threshold — a distinction worth being firm about when someone asks you to "just make it more sensitive".

Preprocessing: the steps before the analysis

Most bad results come from skipping these, not from choosing the wrong transform.

Remove the mean and the trend

A DC offset puts all its energy in bin 0, and leakage from that huge bin swamps the low-frequency region you probably care about. A linear trend is worse: it is a ramp, its spectrum falls as 1/f², and it contaminates a wide band. Subtracting the mean is one line; subtracting a least-squares straight line is barely more, and both should be reflexive before any spectral estimate.

Normalisation

The checklist

What real data actually looks like

Textbook signals are clean, complete and correctly scaled. Real records have dropouts, spikes, saturation, and a sample clock that is not quite what the datasheet says.

Features, harmonics and distortion

Reducing a record to a handful of numbers that mean something — and knowing which of those numbers are actually sensitive to the thing you care about.

Time-domain features

Kurtosis illustrates the general point about features nicely. It is highly sensitive to early impulsive faults, and then it falls back towards 3 as the fault worsens and the impacts become continuous — so a single reading is ambiguous and the trend is what matters. No feature is meaningful without a baseline.

Harmonics and THD

Harmonics mean nonlinearity. A perfectly linear system fed a sine returns a sine; anything else — clipping, backlash, a loose joint, an amplifier at its limit — folds energy into integer multiples. The pattern is diagnostic: symmetric distortion such as clipping produces odd harmonics, while asymmetric distortion produces even ones too.

Interpolation, in time and in frequency

Estimating values between the samples you have. The right method depends entirely on whether you are reconstructing a band-limited signal or just drawing a smooth line.

Interpolating a spectral peak

A different problem with a neat solution. A peak in a magnitude spectrum rarely falls exactly on a bin, so the bin index underestimates the true frequency by up to half a bin. Fitting a parabola through the peak bin and its two neighbours recovers the fractional offset and typically improves the estimate by an order of magnitude — for essentially no cost.

Cross-spectra, coherence and frequency response

With two channels you can measure a system rather than a signal — and coherence tells you, frequency by frequency, whether to believe the measurement.

Coherence answers the question "how much of the output at this frequency is linearly caused by the input?" A value near 1 means the FRF estimate there is trustworthy. A value near 0 means it is not, and the reasons are worth distinguishing: noise on either channel, a nonlinearity, an additional unmeasured input, or a leakage or resolution problem in the estimate itself.

H₁ and H₂

H₁ = S_xy/S_xx is biased low when the input measurement is noisy; H₂ = S_yy/S_yx is biased high in the same situation but is unbiased when only the input is noisy. In practice, H₁ near resonances and H₂ near anti-resonances are the more accurate choices, and the ratio H₁/H₂ equals the coherence — which gives you a quick check that the two estimators agree.

Parametric spectral estimation

Instead of transforming the data, fit a model to it and derive the spectrum from the model. Much better resolution on short records — and much easier to fool yourself.

Because the model is all-pole, it represents peaks very efficiently and needs no window, so there is no leakage and no 1/T resolution limit. On a 64-sample record containing two tones 3 bins apart, an AR model will separate them cleanly where a periodogram shows one lump. This is why it dominates in speech coding, where the vocal tract genuinely is an all-pole resonant system.

Adaptive filters

A filter that tunes its own coefficients from the data, continuously, without being told what the interference looks like.

Choosing μ

Convergence requires 0 < μ < 2/(L·P_x), where L is the filter length and P_x the input power. Too small and it never catches up with a changing environment; too large and it becomes unstable, often after working fine for hours. Normalised LMS divides by the running input power, which makes the effective step independent of signal level and is almost always the better choice in practice. RLS converges far faster — in about 2L samples rather than 20L — at roughly L× the computation, and with a genuine risk of numerical divergence in finite precision.

Where they are used

Noise cancellation needs a reference sensor measuring the interference alone: the classic case is a microphone near the engine in an active-noise-control system, or a second electrode measuring mains pickup. Echo cancellation in telephony identifies the echo path and subtracts its prediction. Channel equalisation inverts a distorting medium. All share one requirement: something must define what "desired" means.

Multichannel data, PCA and beamforming

Several sensors give you something one sensor cannot: the ability to separate sources by where they are, or by how they co-vary.

Principal component analysis

Take the covariance matrix across channels, find its eigenvectors, and you have a new set of axes ordered by how much variance each explains. In practice the first few components usually capture nearly everything, so PCA is a compression and a denoiser at once — and the components often correspond to physically meaningful modes, though nothing guarantees that.

Beamforming

An array of sensors can be steered without moving. Delaying each channel by the propagation time from a chosen direction and summing makes signals from that direction add coherently while everything else partly cancels. The array's resolution is set by its aperture in wavelengths, and its ability to reject other directions by the number and spacing of elements — with element spacing above half a wavelength producing grating lobes, which is spatial aliasing and behaves exactly like the temporal kind.

State estimation and sensor fusion

When you have a model of how something moves and noisy measurements of where it is, the Kalman filter combines them optimally — and tells you how much to trust the result.

The gain is the whole idea. When measurements are noisy relative to the model, K is small and the filter coasts on prediction; when the model is uncertain, K is large and it follows the data. It does this optimally and automatically, and it is recursive — it needs only the previous state, not the history, which is why it fits real-time systems and why it flew on Apollo.

Real-time processing

Offline you have the whole record and all the time you want. In real time you have a fragment, a deadline, and no ability to look ahead.

Block processing and the overlap methods

Fast convolution via the FFT is far cheaper than direct convolution for long filters, but the FFT assumes circular convolution, which wraps the end of each block onto its start. Overlap-add and overlap-save are the two standard ways to stitch blocks together so the result matches true linear convolution exactly.

Latency

Total latency is the sum of the acquisition block, the algorithmic delay, and the output buffer — and it is usually dominated by the block size rather than the arithmetic. A 1024-sample block at 48 kHz costs 21 ms of buffering before a single instruction executes, plus the same again on output. Halving the block halves the latency and roughly doubles the per-sample overhead. Linear-phase FIR filters add (N−1)/2 samples on top; minimum-phase or IIR designs give less delay for the same magnitude response, which is exactly why control loops use them despite the phase distortion.

Getting the data in correctly

Everything in this trainer assumes the numbers in your array faithfully represent the physical quantity. That assumption is where most real errors live.

The signal chain

Calibration

Things that quietly corrupt a measurement

Ground loops — two earth connections at different potentials drive current through the signal return, appearing as mains hum. AC coupling — a high-pass corner you forgot about, which removes real low-frequency content. Aperture jitter — uncertainty in when the sample was taken, which limits SNR at high frequencies to roughly −20log₁₀(2πf·t_jitter) regardless of how many bits the ADC has. Channel skew — a multiplexed ADC does not sample its channels simultaneously, so phase and correlation between channels are wrong unless corrected.

Bandpass sampling

The Nyquist rule you learned is a special case. What actually matters is the signal's bandwidth, not its highest frequency — and exploiting that lets you sample a 100 MHz signal at 25 MHz on purpose.

Aliasing folds the band down to baseband, and provided the folded copies do not overlap each other, no information is lost — the process is reversible. This is undersampling or direct IF sampling, and it is how software-defined radios digitise high-frequency signals without impossibly fast converters.

Rotating machinery: order analysis and envelopes

The rig used throughout this trainer, treated properly. Two techniques carry most of the diagnostic weight, and both exist because machines do not run at a constant speed.

Order analysis

When shaft speed varies, every speed-related component smears across the spectrum while fixed-frequency components such as mains hum stay sharp. Order tracking resamples the signal at equal increments of shaft angle rather than of time, using a tachometer or an estimated speed. In the resampled domain every rotational component is at a fixed order — a multiple of shaft rotation — and becomes sharp again, while fixed-frequency components smear instead. That inversion is itself diagnostic.

Envelope analysis for bearings

Bearing defect frequencies are low — the rig's is 118.4 Hz — but a defect excites brief impacts that ring the structure at a high resonance, typically several kilohertz. The energy is at the resonance; the information is in the repetition rate. Band-pass around the resonance, take the analytic envelope, then take the spectrum of that envelope: the defect frequency appears clearly where the raw spectrum showed only a broad hump.

Transients, edge effects and causality

Every filter has a startup transient, every window has ends, and the first and last portions of any processed record are wrong. How wrong, and what to do about it.

How much to discard

An FIR filter's transient is finite and exactly known: until N samples have entered, the output is a partial sum. An IIR filter's decays asymptotically, so "settled" is a tolerance you choose. Either way, the fix is the same — record more than you need, and trim.

Reducing rather than discarding

Audio and speech

The domain where perceptual limits matter as much as mathematical ones — and where the ear's own properties decide what processing is worth doing.

What the ear imposes

Hearing spans roughly 20 Hz to 20 kHz, so 44.1 kHz sampling is a modest margin above Nyquist. Loudness perception is approximately logarithmic, which is why audio works in decibels throughout. Frequency perception is also approximately logarithmic — an octave sounds like a constant interval whether it spans 100–200 Hz or 4–8 kHz — which is why audio analysis uses constant-Q, mel or third-octave scales rather than the FFT's linear bins.

Speech

Because the vocal tract is genuinely an all-pole resonant cavity, linear prediction (module 47) models it almost exactly — which is why LPC underpins nearly all speech codecs. Pitch is estimated from the autocorrelation or cepstrum of a 20–40 ms frame, short enough that the signal is quasi-stationary and long enough to contain several pitch periods.

Radar, sonar and communications

Where detection theory, matched filtering and modulation come together — and where range and velocity resolution turn out to be two faces of one trade-off.

Those last two are the key insight. A short pulse has wide bandwidth and gives fine range resolution but little energy and poor velocity discrimination. A long tone has fine velocity resolution and hopeless range resolution. A chirp has both a long duration and a wide bandwidth, so after pulse compression it delivers both — which is why coded waveforms replaced simple pulses everywhere.

Doppler

A moving target returns a frequency-shifted echo. The sign of the shift gives the direction of motion, which is why receivers work in IQ (module 38) — a real-valued sampler cannot distinguish approach from recession. In sonar the shift is proportionally much larger because the propagation speed is so much lower, which means a chirp's compression can be significantly degraded by target motion: the Doppler tolerance of the waveform becomes a design parameter in its own right.

Two-dimensional signal processing

An image is a signal sampled on a grid. Nearly every concept so far carries over unchanged — convolution, filtering, the Fourier transform, aliasing — with one extra dimension and one extra idea.

What changes in two dimensions

Frequency becomes a vector: spatial frequency has a direction as well as a magnitude, so an image of vertical stripes has energy along the horizontal frequency axis. Aliasing becomes visible as moiré patterns — the same folding, now spatial, which is why cameras have optical low-pass filters over the sensor and why resizing an image without filtering first produces shimmering artefacts.

Edge detection is differentiation, with all the noise amplification that implies (module 34), which is why Sobel and Canny smooth before differentiating. Compression exploits the fact that natural images have most of their energy at low spatial frequencies — JPEG transforms 8×8 blocks with a DCT and quantises the high-frequency coefficients heavily, since the eye barely notices.

Feeding signals to machine learning

Most of the value is in the signal processing, not the model. Good features from a modest classifier beat raw samples into a large one, almost every time, on the data volumes engineering problems actually have.

The pipeline

Metrics

Accuracy is misleading whenever classes are imbalanced, which in condition monitoring they always are — a detector that never predicts a fault scores 99 % on a machine that fails 1 % of the time. Precision, recall and the confusion matrix say what actually happened, and the ROC and precision–recall curves show the whole threshold trade-off rather than one point on it. The choice between them is an engineering judgement about the relative cost of a missed fault and a false alarm — exactly the question module 41 posed for detection.

Estimating frequency, amplitude and delay well

Three measurements you will make constantly, each with a naive method that is needlessly poor and a better one that costs almost nothing.

Frequency

Taking the peak bin quantises the estimate to f_s/N. Parabolic interpolation over three bins (module 45) improves it by an order of magnitude. Better still, when the tone is stable across the record, is to unwrap the phase at the peak bin across successive frames — phase advances linearly at a rate set by the frequency offset, so the slope gives a very precise correction.

Phase unwrapping

Computed phase is confined to (−π, π] and jumps by 2π whenever it crosses. Unwrapping adds the multiples of 2π back, and works reliably only when the true phase changes by less than π between samples. Noise near a zero of the envelope, or genuine steps larger than π, produce errors that propagate forwards through the entire remainder of the record — a single bad sample can corrupt everything after it.

Time delay

Cross-correlation with parabolic interpolation is the baseline. Generalised cross-correlation with phase transform weighting (GCC-PHAT) whitens both spectra before correlating, which sharpens the peak dramatically in reverberant or coloured-noise conditions — at the cost of poor behaviour at low SNR, since whitening amplifies the bands where there is no signal.

Knowing whether your answer is right

The single most valuable habit in this field: test every processing chain on a signal whose answer you already know, before you point it at data you do not understand.

Synthetic signals are the test harness

If your pipeline cannot recover a 100 Hz sine of amplitude 2.0 to within a fraction of a percent, it will not tell you anything true about a gearbox. Build a synthetic signal with known amplitudes, frequencies, phases and noise level; run it through the exact chain you intend to use; and check the numbers that come out against the ones you put in. Nearly every scaling error, windowing error, and off-by-one is caught this way in minutes.

The errors that recur

Uncertainty

Projects, and what mid-level competence looks like

Reading about signal processing produces recognition. Building something produces the judgement that this trainer has been aiming at throughout.

The capstone: analyse the rig properly

Everything needed is in the modules above. Take the accelerometer record — 2 kHz sampling, a 27.5 Hz shaft, a 23-tooth pinion, a bearing defect at 118.4 Hz, mains at 50 Hz, thermal drift and broadband noise — and produce a defensible condition assessment.

Other projects, roughly in order of difficulty

What mid-level actually means

Not knowing more transforms. It means: choosing the window and record length from the measurement requirement rather than from habit; knowing what your filter does to phase and to the first samples of the record; recognising leakage, aliasing and non-stationarity from the shape of a plot; quantifying the uncertainty on the number you report; and testing every chain on a synthetic signal before trusting it. Those habits are worth more than any individual technique in this trainer.

Formula reference

Every equation used in this trainer, with every symbol defined. Type to filter across titles, equations and descriptions.

Glossary

The vocabulary of signal processing, defined in the sense used here. Filtering searches the definitions as well as the terms, so you can look up a concept you cannot yet name.