Today’s amazing oscilloscopes are instruments of choice in research, product development and troubleshooting. But to be effective, their signal integrity must be evaluated and maintained, starting with the probes and working downstream. The user must be fully aware of bandwidth and sampling rate issues. These metrics become increasingly critical as the frequency of the signal under investigation rises.

To begin, the probes must be compensated. This is a simple process, taking only a minute or so. The reason probes must be compensated in the first place arises from the fact that common 10:1 high-impedance passive probes use a resistor-capacitor, RC voltage divider to get the 10:1 voltage attenuation ratio. So probe compensation is the process of adjusting the RC divider (usually a variable capacitor) on the probe so the probe maintains its attenuation
ratio over the probe’s rated bandwidth.

Probe compensation can actually be defined in the form of an equation:

R_probe × C_probe = R_scope × C_scope

Most scope and probe vendors specify the scope’s input and the probe variable compensation range in the data sheet or manual. For example, the Agilent 8000 Series scope has a 1 MΩ input and 13 pF of typical input capacitance. The probe used with this scope should have at least 6-15 pF of compensation range to properly compensate for the scope input channel.

Some manufacturers, such as Tektronix, have automated the process so it is completed merely by pressing a button, whereas others require a manual operation involving the use of a non-metallic screwdriver to make an adjustment on the probe, still quite simple.

In the Tektronix MDO 3000 series oscilloscopes, the TPP0250, TPP0500B and TPP1000 passive voltage probes can be quickly and accurately compensated. Each probe and channel combination must be individually compensated. Using included hardware, you can color code each probe for a dedicated channel, or if desired you can compensate all probes for all channels.

To compensate a probe to a channel, first power up the instrument. Then, connect the probe connector to the desired channel. Also connect the probe tip (using a hook tip) and the ground return to the probe compensation terminals on the front panel, taking care to observe correct polarity.

Press a front panel button for the input channel connected to the probe that is to be compensated. The menu that appears below the display indicates that the type of probe (such as the widely used TPP1000) has been detected.

Press the far right menu item, More, repeatedly to select Probe Setup from the pop-up menu that appears. The compensation status begins as Default. Push Compensate Probe and follow the simple on-screen instructions. Each channel generates compensation values for each probe-channel combination. Each channel is capable of storing in memory values for ten individual probes. If you try to compensate an eleventh probe on any given channel, the instrument will delete the information for the least recently used probe and add the values for the new probe.

In all cases, use the shortest possible ground return lead and signal path. This will minimize ringing in the signal as displayed.

It is often the case that when first accessing a signal, even with a properly compensated probe, a meaningful trace will not appear in the display. If this is so, press Autoset. This adjusts triggering and scaling parameters so a good display appears. If problems persist, check the probe connections. A poor connection can result in instability where repeated autosets do not help.

Where possible, the hook tip should be used. In accessing signals from a dense circuit board, sometimes only the point tip is capable of contacting the desired termination, and this can be problematic because if the connection is to any degree intermittent, the display will degrade.

So far we have been considering factors that impact signal integrity at the probe interface. Inside the oscilloscope the signal is further modified as it moves through digitization, memory and toward display. Here the most important issues are noise and aliasing, either of which is capable of altering the waveform so that it does not display in a way that is true to the signal as formed in the circuitry under investigation.

In regard to aliasing, Harry Nyquist articulated the valuable sampling theorem in 1928. This theorem is applicable in the digitization of an analog signal. It is based on the observation that if a continuous function is reduced to a sequence of discrete samples that are subsequently interpolated so as to reform into analog information, the samples must be frequent enough so as to permit faithful representation without ambiguity.

The Nyquist sampling theorem relates the minimum number of samples to the frequency of the original analog signal. Obviously, if only one sample were taken, a vast number of interpretations would be possible. Specifically, the Nyquist sampling theorem asserts that if the function x(t) contains no frequencies higher than a frequency expressed as B, it is completely determined when the sampling rate consists of a series of points spaced no more than 1/(2B) seconds apart.

Practically speaking, the sampling rate per second must be twice the frequency in Hertz. If this sampling rate is not maintained, aliasing may arise, wherein false waveforms are displayed because they meet the conditions implicit in the inadequate sampling rate.

Modern oscilloscopes usually have the bandwidth and sampling rate displayed side-by-side on the front panel. The sampling rate should be twice the bandwidth, but this is not quite the whole story.

One would assume that a higher sampling rate would ensure an improved signal fidelity, but this is not always the case. Sometimes the oscilloscope with a higher sampling rate, given identical bandwidth, can exhibit reduced signal fidelity. This is because of a poorly aligned analog-to-digital converter (ADC). To make a valid comparison, we need to look at ADC sampling fidelity by viewing time domain and frequency domain properties.

The Nyquist sampling theorem states that the highest frequency component sampled must not exceed half the sampling frequency if the display is to be free of aliasing. That being said, there is an additional requirement, often neglected, which is that the samples must be equally spaced. It is necessary for signal fidelity that both of these requirements be met.

There is a further complication. It would be OK if an oscilloscope’s bandwidth were specified exactly at the Nyquist frequency, V_MAX, provided that the frequency response would abruptly fall off at that point, but this is not generally the case. In the real world, oscilloscopes typically follow a Gaussian frequency response curve, which means that approaching the bandwidth limit, amplitude drops gradually as opposed to abruptly. To prevent aliasing, the sample rate must be higher than twice the instrument’s nominal bandwidth. This becomes increasingly important in the digital regime, where fast rising edges mean that the signals are prone to aliasing even if it seems that the instrument happens to be Nyquist compliant. Unfortunate results include pre-shoot, over-shoot and untrue edge speeds.

For a reasonable degree of reliability, an oscilloscope with a bandwidth under 2 GHz should have a sampling rate as much as four times the bandwidth. At over 2 GHz, however, there is a more abrupt frequency roll-off, and 2.5 as a Nyquist ratio is usually adequate.

Oscilloscope manufacturers have been able to nit higher sampling rates by specifying faster ADCs, but this expedient is incapable of meeting the Nyquist requirements of higher bandwidths. An alternative arrangement known as interleaving provides a workable solution. Each channel consists of a signal path that passes through the analog amplifier and divides, forming the input of two separate ADCs and associated memories. A sample clock is connected to each ADC. In the line from the sample clock to one of the two ADCs, a one-half phase clock delay is introduced. (This is not the same as interleaving samples from successive acquisitions, which is known as equivalent-time sampling.)

The technique is called real-time sampling. It has the effect of doubling the sample density and sample rate. If the interleaving is to be free of distortion, the two ADCs must have closely matched frequency response and vertical gain, and there must be accurate offset. The clock signals, one of which incorporates one-half phase delay, must be synchronized with a high degree of precision so the samples, in accordance with the complete Nyquist theorem, are equally spaced. Poor phase-delayed clocking produces errors in the signal as displayed. To ascertain whether signal fidelity is sufficient, there are certain characteristics in the display that should be observed.

If the two clock outputs do not have an exact one-half sample period interval with respect to one another, the digitized points will not be evenly spaced and severe distortion will be apparent in the trace.

To detect interleaving distortion, the user can analyze the effective number of bits. One method of doing this involves exporting digitized data to a computer, where RMS error of a digitized sinewave is compared to a theoretical standard. An easier way to perform this test is to input a sinewave from a signal generator, the frequency set close to the oscilloscope’s bandwidth specification. Then a visual inspection of the displayed waveform will serve to assess signal fidelity.

Another easy test is to compare the displays of two oscilloscopes having similar bandwidths. Check rise and fall times and peak-to-peak amplitudes.
To perform visual tests of a sine wave, do a single acquisition. Poor signal fidelity due to interleaving distortion will appear as noise. Further information can be obtained by doing an FFT operation. This will ascertain the root problem – harmonic distortion, random noise or interleaved sampling distortion. Looking at the FFT representation of a known pure sine wave, there should be no visible harmonics, all power being concentrated in the fundamental.