We understand from the previous article that the process of digitizing an optical image with a photographic sensor can be thought of as two subsequent operations:

1. filtering (convolution) of the optical image on the sensing plane by the pixel’s finite effective active area (aka pixel aperture);
2. point sampling the convolved image at a given fixed rate and position, often corresponding to the center of each pixel.

Both affect resolution in different ways: the former can be thought of as modifying continuously the analog optical image, as seen below right; the latter as possibly introducing interference (aliasing) into the result.

In this page I will explore how the act of digitizing that image – the process of sampling – fundamentally alters what we can resolve.   In the next one we will discuss the impact on resolution of  pixel-shift modes available in current mirrorless cameras.

For simplicity and clarity I will assume a monochromatic sensor with 100% fill factor pixels and no noise, but the general principles can easily be adapted to practical Bayer sensors as well.

Measuring Resolution

As discussed in the dedicated article, we define resolution as the ability to tell things apart in an image. We quantify it by measuring the contrast between two closely spaced objects, like the simulated, highly magnified, noiseless stars shown above.   Notice the dip in intensity between the two peaks. This dip is a physical measure of resolution. 

With near perfect optics and a circular aperture, Rayleigh’s criterion for resolving two stars of presumably equal magnitude is a  separation of 1.22 on the sensing plane, with the wavelength of light and the effective f-number of the lens.   The simulated point sources in this page are spaced accordingly.

To more easily measure the dip, the image of Figure1 is projected to the horizontal axis in Figure 2 below, we’ll improperly call it a Line Spread Function.  This measure of intensity variation is more relevant for our purposes than a simple horizontal slice through the center of the image.^[*] 

In this case the dip corresponds to about a 32.4% drop in intensity from the LSF peaks, yielding a Michelson contrast of 19.3% (dotted curve corresponding to the optics only below). If that dip reduces, resolution is lost and by Rayleigh’s definition the two objects are no longer resolved.

Resolution Degradation by Pixel Aperture

In digital imaging, however, before a single value is recorded in the raw file the optical image undergoes convolution. As photons land on the sensor, they are integrated over the physical effective area of every pixel, also known as pixel aperture. This light collection acts as an inherent, image-wide, continuous low-pass filter, as previously discussed.^[1]

In the figure above, the dotted curve curve represents the LSF of the sharper optical image Rayleigh would have seen. The solid gray curve shows that same image after being smoothed by pixel aperture, in this case assumed to be a perfect square 0.61λN on the side. Notice how the dip between the stars has become shallower.

This is the first blow to resolution resulting from digitization: the physical size of our pixels has smoothed the very image detail we are trying to resolve.  Now Michelson contrast is only 12.2% and, according to Rayleigh’s criterion, the two stars are no longer resolved.

Resolution Degradation by Point Sampling

After convolution, the sensor effectively performs Dirac delta sampling, picking infinitesimal point samples of the continuous convolved image near the center of each pixel.  This is where the sampling rate – often improperly called sensor resolution – comes into play.

Below I assume that pitch is half of the distance between the two stars and pixel aperture is a perfect square with 100% Fill Factor.

Because every photographic image is band-limited by diffraction – thanks to the finite, often circular aperture of the lens – there is a maximum spatial frequency we can accurately record, known as the extinction frequency (, in cycles per the same units as ).  Recall that frequency is one over the period:

• At or Above Nyquist: If we sample at twice the extinction frequency or faster – i.e. at 2/, corresponding to a period of no more than 0.5 – we satisfy the Nyquist-Shannon theorem. In this case, optical resolution is only decreased by the convolution with pixel aperture. Using an ideal sinc kernel, we could perfectly reconstruct the convolved gray smoothed curve, as shown by the blue dots in the figure below, which overlay it completely.  This is true no matter the phase of the sampling train.

• Below Nyquist (Aliasing): If the sampling rate is too low – i.e. a frequency less than 2/, corresponding to a period longer than 0.5 – aliasing occurs. High frequencies “fold back” and interfere with lower ones, see Figure 7 for the frequency domain perspective. This results in the faulty reconstruction below, which fails to track the convolved solid gray curve.  In this case aliasing degrades resolution further than pixel aperture alone, the dip is gone and so is any clear indication that there were two stars:

Contrary to the properly sampled case shown in Figure 4, a particularly tricky aspect of aliasing is that the perceived resolution changes based on the phase of the sampling train. It matters exactly where the grid of pixels lands relative to the detail.  Notice for instance how the introduction of a quarter phase shift to the sampling train in Figure 5 skews the resulting reconstructed image:

Reducing aliasing through a higher sampling rate means a more accurate reconstruction, allowing for the resolution of smaller details more reliably.

Aliasing and the Reconstruction Problem

At its simplest, reconstruction can be considered part of demosaicing during raw conversion.  But it also affects the final image any time it is re-sampled or resized.  The best we can hope for, without resorting to advanced post processing like deconvolution, is to reconstruct the continuous optical image convolved with effective pixel aperture, overlaying the solid gray line in the plots.

It’s useful to distinguish where aliasing errors can come from.
Hardware or pre-aliasing occurs during capture when the image is sampled below the Nyquist rate (what this article is about).   This is what it looks like in the frequency domain for our two stars at the Rayleigh limit, sampled with perfect 1 square pixels as shown in Figures 5 and 6:

The perfect lens circular aperture band-limits the signal to , so if we wanted unaliased data we would need to sample the image at a rate of at least twice that or , which would put Nyquist at diffraction extinction.  With no energy above Nyquist there would be no aliasing and no foldback so we could aim for perfect reconstruction.

Alas, in Figures 5 and 6 we sampled it only every , so the Nyquist frequency is at 0.5 cycles/ and the energy above that does alias indeed, folding back and corrupting the lower spatial frequencies as can be seen in Figure 7.

However, we also face post-aliasing errors during the rendering stage.  While the sinc filter is the theoretical ideal for reconstruction, it has infinite support and is impractical for real-world processing. Instead, we are forced to use finite kernels like nearest neighbor, bilinear, cubic, Lanczos, etc. which introduce their own post-sampling aliasing artifacts.  Not to mention the effect of post processing sharpening or other algorithms, which can induce artificial peaks and valleys known as halos.

How Much Does Digitization Degrade Resolution?

All else equal it depends on the subject.  One may wonder: if we wanted to capture the same Michelson contrast that Rayleigh saw through his optical telescope, how much farther apart would the two stars need to be for a given setup?

Hand waving a bit, we could achieve a contrast of 19% with our example – pixels of pitch and 100% FF – with star separation of 1.51 instead of 1.22 once projected on the sensor, compare the gray curves to Figure 2.

So now we have:

• Optical Rayleigh Limit: 1.22λN separation for the given Michelson contrast.
• Digital Limit (with λN pitch): 1.51λN separation to achieve the same contrast.

For a practical example, assume that λN  is 3.7 microns, implying a common pitch in sensors of enthusiast digital cameras today, and a reasonable landscape working f-number around f/8:

Optical resolution = 1/(1.22λN) 220 lp/mm
Digital resolution = 1/(1.51λN) 180 lp/mm

In other words, digitizing the optical image imposed about a 20% penalty in resolution in this ideal case, just as a result of the inevitable convolution with pixel aperture.

Incidentally the hardware – the lens and the sensor – has not changed compared to the original Rayleigh separation capture, only the subject did, so the MTF plot in Figure 7 still applies as-is and the resulting image is just as aliased.  We can see that the reconstructed image in Figure 8 is not able to follow the convolved optical original (solid gray).

Because of aliasing, phase becomes relevant, so though the two stars may appear resolved with that alignment of the sampling train, they may not with a different one, see below what about a micron shift of the sensor to the right does to the reconstructed image:

Even with the same scene, camera and lens, aliasing can produce two different results with just a tiny shift of the sensor.   It becomes a game of chance depending on how detail is layed out relative to the sampling grid.

Could we use such phase changes to our advantage?   An analysis of the effect of pixel-shift modes on resolution is next.

 

Appendix – Aliasing in Digital Photography Redux

In photography, aliasing is a form of digital interference that occurs when spatial frequencies in an optical image exceed the capabilities of the sensor’s sampling grid (so-called sensor resolution).

It is fundamentally characterized by high-frequency energy “sneaking back” below the Nyquist frequency and masquerading as lower frequencies.

Every digital sensor has a limit known as the Nyquist frequency (typically 0.5 cycles per pixel pitch), which is half the sampling rate.  If the optical image contains detail at frequencies higher than this limit, those frequencies overlap with their neighbors in the frequency domain during the sampling process.

Because the sensor cannot accurately record these high-speed transitions, the energy “folds” back around the 0.5 cycles/pixel mark.  This causes fine, high-frequency detail to appear as much larger, low-frequency patterns that were not present in the original scene.

In final photographs, aliasing manifests as undesirable artifacts such as stair-stepping (jaggies) on sharp edges, false color, and moiré patterns.  These artifacts are particularly visible when photographing man-made objects with regular, repeating patterns like fabrics or architecture.  Less so in landscapes.

In typical sensors with a square pixel grid, aliasing is direction-dependent.  Detail aligned horizontally or vertically with the pixel rows is most prone to aliasing, while detail oriented at a 45-degree diagonal has a higher Nyquist limit and is therefore less likely to alias.

For cameras using a Bayer Color Filter Array, aliasing can be more severe because the individual color planes (red, green, and blue) are sub-sampled at a lower rate than the overall pixel count. This can lead to chrominance aliasing, where high-frequency luma detail corrupts the color signal, effectively halving the unaliased range of the sensor for non-neutral subjects.

To control aliasing, manufacturers often use an Anti-Aliasing (AA) filter (also called an Optical Low Pass Filter) to blur or attenuate high-frequency energy before it reaches the sensor. Alternatively, increasing the sampling rate by using denser pixels or techniques like pixel shifting can push the Nyquist limit further out, allowing the system to accurately record higher frequencies without corruption.

Notes and References

^[1]
_{1. It makes no difference whether the filtering of the optical image is performed by the effective pixel aperture of each pixel, or whether we assume a single image wide filter covering the sensor.  The result of the convolution is effectively continuous and the same.
2.  The code used to produce the above plots can be downloaded by clicking here.}