The Nyquist–Shannon Sampling Theorem: Exceeding the Nyquist Rate

In the first article of this series, we explored this concept by thinking in the time domain, and in the second article, we approached it from a frequency-domain perspective.

Now, we need to consider this theorem’s role in guiding the decisions of electrical engineers whose goal is to design functional circuits and systems. 

Shannon’s sampling theorem states the following:

If a system uniformly samples an analog signal at a rate that exceeds the signal’s highest frequency by at least a factor of two, the original analog signal can be perfectly recovered from the discrete values produced by sampling.

Theory informs practice but does not specify it. In other words, Shannon’s theorem doesn’t tell us how to design a sampled system; rather, it helps us to understand sampled systems and provides a framework that orients and supports the work of the engineer. Thus, it’s important to know where theory and practice diverge, and in the case of sampling theory, perhaps the most important divergence is that of the required sampling rate.

Sampling and Aliases

In the previous article, we saw that aliasing occurs when the sampling frequency (f_S) is less than twice the maximum signal frequency (f_MAX), such that the subspectra overlap. 

I think that most of us naturally interpret the term “aliasing” as inherently negative, i.e., as a potential problem that must be avoided. However, aliasing in the broader sense is an integral part of converting a signal from a continuous waveform into a sequence of discrete values.

I’ve been using the word “subspectra” to refer to the spectral replicas created by sampling, but a more official name is simply aliases.

We create aliases—i.e., the original signal frequencies “disguised” as different frequencies—every time we perform analog-to-digital conversion, regardless of the sampling rate. When sampled data are converted from digital back to analog, these aliases become part of the analog signal, and consequently, D/A conversion results in an analog signal that is not identical to the sampled signal. Thus, if we wish to perfectly reconstruct the original analog signal, we must eliminate the effect of the aliases.

As we know, to prevent alias-induced signal corruption, we need to sample at or above the Nyquist rate. If we don’t comply with this fundamental requirement, we don’t stand a chance against the aliases—by the time we even so much as glance at our sampled data, the aliases have already permanently mingled with the original spectrum. There’s nothing we can do to separate the authentic frequencies from the impostors.

But signal reconstruction only begins with adequate sampling frequency. A second fundamental requirement is low-pass filtering.

Reconstruction via Filtering

If we sample above the Nyquist rate, we still have aliases, but now there is a gap between the authentic spectrum and the aliased spectra:

This allows us to recover the original signal by converting the digitized waveform into an analog waveform and then applying a low-pass filter:

The low-pass filter that is intended to eliminate (or more realistically, mitigate) the effect of aliases in the reconstructed analog signal is called a reconstruction filter. This is an analog filter that is applied after D/A conversion. I’m using italics here to emphasize two things.

First, we can’t remove aliases by means of digital filtering. Aliases are inherent to the nature of sampled, quantized data and therefore can’t be eliminated in the digital realm (though oversampling and interpolation can make the analog filtering requirements less severe).

Second, a reconstruction filter is designed to remove aliases, but it is not an anti-aliasing filter! The term “anti-aliasing filter” refers to a low-pass filter that is applied before A/D conversion.

Ideal Filter vs. Real Filter

If you ponder the previous diagram for a moment, you may start to understand why Shannon’s theorem is not a “how-to” guide for designing mixed-signal electronic systems. If we bring the sampling rate down to the theoretical limit, the Fourier transform looks like this:

In the idealized mathematical realm, we can still separate the authentic spectrum from the aliases. However, physical components cannot create the “brick wall” type of frequency response that would be needed to slice straight down and thereby perfectly filter out the unwanted frequency content:

Furthermore, we typically prefer to avoid the cost, complexity, and board space required for filters that come anywhere near the brick-wall response. Instead, we use oversampling.

By sampling a signal at a rate that is much higher than the Nyquist rate, we ensure that there will be a large frequency gap between the authentic spectrum and the nearest alias. This large gap makes it much easier for us to build an effective reconstruction filter because the magnitude response can roll off slowly and still produce significant attenuation at the alias frequencies. With generous oversampling, even a first-order RC low-pass filter can provide adequate alias suppression.

There’s no fixed rule for how much oversampling is needed in a given application, but I like to have a sampling rate that is at least five times higher than the highest signal frequency of interest. If your signal frequencies are close to the maximum sampling rate of your ADC, you may have to sample closer to the Nyquist rate and then devote more time and money to your reconstruction filter.

Conclusion

We’ve seen that Shannon’s sampling theorem needs to be adapted to the constraints of real-life circuit design. Though perfect reconstruction is mathematically possible when the sampling rate is equal to twice the highest signal frequency, this approach requires an idealized low-pass filter and is, therefore, not directly applicable to engineered systems.

Another important issue that I mentioned is the difference between a reconstruction filter and an anti-aliasing filter. We’ll discuss anti-aliasing filters in the next article.