|Binary||2's Complement Binary||Hexadecimal||Why ADCs Use Integer Math||The Nyquist Sampling Theorem|
Digitization is not a continuous process. Just as the amplitude representations of data are discrete integers, so the values are digitized at specific times. One may snatch a single value from a data stream (sampling), one may take data at regular intervals (periodic sampling), or one may digitize in response to a specific, triggering event. In any case, given that discrete numbers are stored, there are a finite number of measurements taken over an interval of time. How much fidelity between the sampled signal and the real, analog, continuously-varying temporal waveform can be expected?
For demonstrations and explanations, we'll look at three closely related waveforms:
y = sin(2 π t), a 1 cycle per second or 1 Hertz sine wave.
y = sin(2 π t * 0.9), a 0.9 Hertz sine wave.
y = sin(2 π t * (1+ 0.02t)), a "chirped pulse," where the frequency continuously increases with time. If you want to see the original Excel 2007 file, click here.
We can write these down as continous functions, but any digital device will measure the signal only at discrete, specified times (typically, a signal is sampled, a digital number corresponding to the signal computed with an Analog to Digital Converter, and then another sample is taken. Each conversion takes a measureable amount of time). Here we show two of the three functions plotted as if they were continuous (they're actually computed in Microsoft Excel at discrete, 0.01 s intervals), and the third function showing each point discretely. Click on the picture to get a larger image.
The same waveforms, viewed over 20 s instead of 2 s look like this:
Notice how the waveforms drift in and out of phase with each other. At 10 s and 20 s, all the waveforms pass through zero.
Exercise: How many full cycles of each waveform occur between t = 0 and t = 10? How about between
t = 0 and t = 20? .
As is easily seen, when we take numerous data points during a cycle of the waveform, we can accurately trace out its shape. In the examples above, we are sampling slow waveforms (<20 Hz) at a frequency (100 Hz) greater than any of the contributing frequencies. What happens if the sampling occurs at a frequency similar to or less than the frequency of the observed waveform?
Aliasing (see also here)
"A stopped clock is right twice a day." If we sample the reading on the clock at 12 hour intervals, we will always see either that the time is correct or that the time is offset by a fixed amount. We have no knowledge of its behavior between readings. This is an example of aliasing, seeing a periodic event of one frequency occurring at a different frequency because of the ratio between sampling frequency and the actual behavior. For the stopped clock, sampling once per day or twice per day gives us no indication of a problem (other than, perhaps, a fixed time offset). Sampling at ANY higher frequency will reveal that the reading does not change as a function of sample time. Sampling at a lower frequency (once every 2 days? once every 20 days?) shows the same behavior as sampling at 12 hour intervals, provided of course that the sampling is always done at some multiple of 12.000 hour intervals.
For the waveforms from the previous section, here's sampled data with the sampling interval 0.5 s. In the first plot, the sampling occurs at 0, 0.5, 1.0 etc. seconds. In the second plot, the sampling occurs at 0.25, 0.75, 1.25 etc. seconds. Note particularly the light blue data points (those for the 1.0 Hz waveform:
These plots are undersampled. The true shape of the waveform is not evident because the sampling speed is too slow. Most clearly, we see that if we sample at 1/2 the frequency of the monitored waveform, we may get alternation between the highest and lowest values of the waveform or a pure zero (DC) result. We see an additional, unexpected behavior: while we know the Time Varying Frequency waveform is INCREASING in frequency, the sampled waveform looks like it is DECREASING in frequency. What in the world is happening?
We can get an idea by looking at an example of sampling the 0.9 Hz sine wave at 1 Hz i.e. sampling a point once per second. We will choose the specific phasing where the 0.9 Hz waveform goes through zero (with a positive derivative) just at the moment we take the first sample. The 11 the sampled point will occur at 10 s, and we know from the above plots that the 0.9 Hz waveform will have gone through exactly 9 oscillations in 10 s and be back at a postive-going zero crossing then. Here's the plot:
We see that the sample points go through 1 oscillation cycle, with the sample amplitude as the 0.9 Hz waveform, but "flipped upside down" or phase-reversed. One cycle in 10 s is 0.1 Hz. Is it a coincidence that the difference between the raw waveform (0.9 Hz) and the sampling (1 Hz) is 0.1 Hz? No -- in fact, that's exactly the point. We have aliased the 0.9 Hz waveform to 0.1 Hz by sampling it at 1 Hz: 0.9 Hz (raw data) - 1 Hz (sampling) = - 0.1 Hz (result appears to be at 0.1 Hz, and the minus sign says the waveform is phase-reversed).
We now know enough to appreciate the fundamental rule concerning sampled monitoring of a periodic waveform:
|To digitize a waveform without aliasing, sampling must be at least TWICE the frequency of the waveform.|
As we already saw, even at twice the waveform frequency, there is a significant chance that we will underestimate the amplitude of the waveform.
Dealing with Finite Sampling Frequency
- Tracing the shape of a waveform without aliasing
From what we saw above, we need to sample the waveform at a frequency higher than twice the waveform's frequency. If we have no knowledge of the phase of the waveform, we may need to sample at at least four times the waveform's frequency; if we can synchronize with the waveform so we can capture maximum and minimum points, twice the waveform frequency is a hard lower limit. Even then, we have to assume the waveform is a sine wave, square wave, or some other fixed form. If the waveform has some arbitrary shape, then we can only elucidate that shape up to components with a characteristic frequency of the Nyquist frequency. Any periodic waveform can be decomposed into a sum of harmonics, that is, multiples of the lowest periodic frequency in the waveform. See separate tutorial.
It will not escape the notice of the careful observer that the sampled points in the first two plots in the Aliasing section, particularly for the time-varying waveform, appears to be quite noisy. Any time one sees sampled data jumping noisily about, one should be highly suspicious that one is NOT seeing real noise and NOT seeing a high frequency waveform, but rather that one is seeing aliasing of something whose nature can not be directly inferred from the observed time series.
- How does an aliased waveform appear?
Let's look at some numerical examples, then generalize. Suppose the real waveform (observed by high frequency sampling) is known to be at 1000 Hz. At what frequency would it appear if sampled at 1 Hz? First of all, if there is jitter (variation in frequency over the course of an experiment), we may just see a blur. But if everything is stable, We will see what looks like a single value -- all zeros (if the sampling is synchronized to the zero crossing at each half-millisecond interval) or all some other value. 1000 Hz is the 1000 multiple or 1000th harmonic of 1 Hz. What if we sample at 1.1 Hz? If the first point is taken at a positive-going zero crossing of the 1000 Hz waveform, we will take 11 points during the ensuing 10 s before hitting the next mutual point of positive zero crossing. The apparent frequency of the sampled waveform will be 0.1 Hz. Sample at 202 Hz, and we see one sampled cycle in 0.1 s.
Is there a pattern emerging? An integer times the sampling rate differs from the actual signal frequency by the observed, aliased frequency. We'll put the integer in green to make it obvious. 1000 - 5*202 = 1000 - 1010 = -10 Hz (period = 0.1 s, as seen in the above figure). 0.9 Hz - 1*1 Hz = - 0.1 Hz (several previous examples). 1000 - 909*1.1 = 0.1 Hz.
The missing mathematical concept here is modulus. The modulus is the remainder from a division problem. 7 modulus 4 = 3. 19 modulus 6 = 2. So in determining the effect of aliasing, the apparent frequency is determined by finding which harmonic is closest to the actual frequency, then subtracting the harmonic number times the sampling frequency from the actual frequency. Thus, at the end of the previous paragraph, we subtracted the 909th harmonic of 1.1 Hz from 1000 Hz. What would have happened if we'd used the 908th or 910th harmonic? 1000 - 908*1.1 = 1000 - 998.8 = 1.2 Hz, and 1.2 is bigger (in absolute value) than the Nyquist frequency (1.1 Hz/2 = 0.55 Hz). 1000 - 910*1.1 = 1000 - 1001 = -1 Hz, which is also (in absolute value) bigger than the Nyquist freqency. So choose the harmonic such that, when the harmonic frequency is subtracted from the true frequency, the result lies between 1 Nyquist frequency (in absolute value) of 0.
Exercise: A 1.275 MHz signal is sampled at 50.000 KHz. At what frequency does the aliased signal appear?
- Preventing high frequency noise from being aliased to lower frequency measurements
Aliasing can only be prevented by suppressing high frequency information. If a waveform is a sum of a 1 KHz and a 12 KHz component, sampling at 7 KHz will give the 1 KHz component directly and alias the 12 KHz component to 1.5 KHz (Nyquist frequency 3.5 KHz; 3rd harmonic of the Nyquist frequency is at 10.5 KHz, so the aliasing is at 12 KHz - 10.5 KHz = 1.5 KHz). As we have said several times on this page, there is no way after sampling has occurred to tell if the 1.5 KHz component (or, for that matter, the 1 KHz component!) is real or aliased. Thus, one requires an anti-aliasing filter or an electronic device to limit the range of frequencies reaching the digitizer to suppress signals outside the unaliased range one wishes to observe. Alternately, one might suppress frequencies outside a narrow range (including suppressing low frequencies) so that one can INTENTIONALLY alias a high frequency signal into the range of a low frequency digitizer. One can then reconstruct the true waveform from knowledge of the sampled waveform and the anti-aliasing filter's throughput properties. By far the most common approach is to use a low-pass filter (a filter that lets through DC and slowly changing signals) to block high frequency noise and interfering signals. The design of such filters is outside the scope of this module. That such filters MUST be used should be evident, based on the discussion and exercises above.
- Using a low frequency digitizer to digitize a high frequency waveform
Sometimes we can use aliasing to our advantage. Suppose we use an anti-aliasing filter that passes signals from 11 to 13 KHz, and we sample the filtered signal with a 7 KHz digitizer. Over what range of frequencies would the signals appear? Use the discussion of modulus and aliasing above to compute the range of observed frequencies.