Tag Archives: sensor

Wavefront to PSF to MTF: Physical Units

In the last article we saw that the Point Spread Function and the Modulation Transfer Function of a lens could be easily obtained numerically by applying Discrete Fourier Transforms to its generalized exit pupil function P twice in sequence.[1]

Obtaining the 2D DFTs is easy: simply feed MxN numbers representing the two dimensional complex image of the pupil function in its uv space to a fast fourier transform routine and, presto, it produces MxN numbers that represent the amplitude of the PSF on the xy sensing plane, as shown below for the pupil function of a perfect lens with a circular aperture and MxN = 1024×1024.

Figure 1. 1a Left: Array of numbers representing a circular aperture (zeros for black and ones for white).  1b Right: Array of numbers representing the PSF of image 1a (contrast slightly boosted).

Simple and fast.  Wonderful.  Below is a slice through the center, the 513th row, zoomed in.  Hmm….  What are the physical units on the axes of displayed data produced by the DFT?

Figure 2. A slice through the center of the PSFshown in figure 1b.

Less easy – and the subject of this article as seen from a photographic perspective.

Continue reading Wavefront to PSF to MTF: Physical Units

Aberrated Wave to Image Intensity to MTF

Goodman, in his excellent Introduction to Fourier Optics[1], describes how an image is formed on a camera sensing plane starting from first principles, that is electromagnetic propagation according to Maxwell’s wave equation.  If you want the play by play account I highly recommend his math intensive book.  But for the budding photographer it is sufficient to know what happens at the exit pupil of the lens because after that the transformations to Point Spread and Modulation Transfer Functions are straightforward, as we will show in this article.

The following diagram exemplifies the last few millimeters of the journey that light from the scene has to travel in order to smash itself against our camera’s sensing medium.  Light from the scene in the form of  field  U arrives at the front of the lens.  It goes through the lens being partly blocked and distorted by it (we’ll call this blocking/distorting function P) and finally arrives at its back end, the exit pupil.   The complex light field at the exit pupil’s two dimensional uv plane is now  U\cdot P as shown below:

Figure 1. Simplified schematic diagram of the space between the exit of a camera lens and its sensing plane. The space is filled with air.

Continue reading Aberrated Wave to Image Intensity to MTF

Linear Color: Applying the Forward Matrix

Now that we know how to create a 3×3 linear matrix to convert white balanced and demosaiced raw data into XYZ_{D50}  connection space – and where to obtain the 3×3 linear matrix to then convert it to a standard output color space like sRGB – we can take a closer look at the matrices and apply them to a real world capture chosen for its wide range of chromaticities.

Figure 1. Image with color converted using the forward linear matrix discussed in the article.

Continue reading Linear Color: Applying the Forward Matrix

Color: Determining a Forward Matrix for Your Camera

We understand from the previous article that rendering color during raw conversion essentially means mapping raw data represented by RGB triplets into a standard color space via a Profile Connection Space in a two step process

    \[ RGB_{raw} \rightarrow  XYZ_{D50} \rightarrow RGB_{standard} \]

The process I will use first white balances and demosaics the raw data, which at that stage we will refer to as RGB_{rwd}, followed by converting it to XYZ_{D50} Profile Connection Space through linear transformation by an unknown ‘Forward Matrix’ (as DNG calls it) of the form

(1)   \begin{equation*} \left[ \begin{array}{c} X_{D50} \\ Y_{D50} \\ Z_{D50} \end{array} \right] = \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{bmatrix} \times \left[ \begin{array}{c} R_{rwd} \\ G_{rwd} \\ B_{rwd} \end{array} \right] \end{equation*}

Determining the nine a coefficients of this matrix is the main subject of this article[1]. Continue reading Color: Determining a Forward Matrix for Your Camera

Color: From Capture to Eye

How do we translate captured image information into a stimulus that will produce the appropriate perception of color?  It’s actually not that complicated[1].

Recall from the introductory article that a photon absorbed by a cone type (\rho, \gamma or \beta) in the fovea produces the same stimulus to the brain regardless of its wavelength[2].  Take the example of the eye of an observer which focuses  on the retina the image of a uniform object with a spectral photon distribution of 1000 photons/nm in the 400 to 720nm wavelength range and no photons outside of it.

Because the system is linear, cones in the foveola will weigh the incoming photons by their relative sensitivity (probability) functions and add the result up to produce a stimulus proportional to the area under the curves.  For instance a \gamma cone will see about 321,000 photons arrive and produce a relative stimulus of about 94,700, the weighted area under the curve:

Figure 1. Light made up of 321k photons of broad spectrum and constant Spectral Photon Distribution between 400 and 720nm  is weighted by cone sensitivity to produce a relative stimulus equivalent to 94,700 photons, proportional to the area under the curve

Continue reading Color: From Capture to Eye

An Introduction to Color in Digital Cameras

This article will set the stage for a discussion on how pleasing color is produced during raw conversion.  The easiest way to understand how a camera captures and processes ‘color’ is to start with an example of how the human visual system does it.

An Example: Green

Light from the sun strikes leaves on a tree.   The foliage of the tree absorbs some of the light and reflects the rest diffusely  towards the eye of a human observer.  The eye focuses the image of the foliage onto the retina at its back.  Near the center of the retina there is a small circular area called the foveola which is dense with light receptors of well defined spectral sensitivities called cones. Information from the cones is pre-processed by neurons and carried by nerve fibers via the optic nerve to the brain where, after some additional psychovisual processing, we recognize the color of the foliage as green[1].

Figure 1. The human eye absorbs light from an illuminant reflected diffusely by the object it is looking at.

Continue reading An Introduction to Color in Digital Cameras

A Simple Model for Sharpness in Digital Cameras – Polychromatic Light

We now know how to calculate the two dimensional Modulation Transfer Function of a perfect lens affected by diffraction, defocus and third order Spherical Aberration  – under monochromatic light at the given wavelength and f-number.  In digital photography however we almost never deal with light of a single wavelength.  So what effect does an illuminant with a wide spectral power distribution, going through the color filter of a typical digital camera CFA  before the sensor have on the spatial frequency responses discussed thus far?

Monochrome vs Polychromatic Light

Not much, it turns out. Continue reading A Simple Model for Sharpness in Digital Cameras – Polychromatic Light

A Simple Model for Sharpness in Digital Cameras – II

Now that we know from the introductory article that the spatial frequency response of a typical perfect digital camera and lens can be modeled simply as the product of the Modulation Transfer Function of the lens and pixel area, convolved with a Dirac delta grid at cycles-per-pixel spacing

(1)   \begin{equation*} MTF_{Sys2D} = \left|(\widehat{ PSF_{lens} }\cdot \widehat{PIX_{ap} })\ast\ast\: \delta\widehat{\delta_{pitch}}\right|_{pu} \end{equation*}

we can take a closer look at each of those components (pu here indicating normalization).   I used Matlab to generate the examples below but you can easily do the same in a spreadsheet.  Here is the code if you wish to follow along. Continue reading A Simple Model for Sharpness in Digital Cameras – II

A Simple Model for Sharpness in Digital Cameras – I

The next few posts will describe a linear spatial resolution model that can help a photographer better understand the main variables involved in evaluating the ‘sharpness’ of photographic equipment and related captures. I will show numerically that the combined spectral frequency response (MTF) of a perfect AAless monochrome digital camera and lens in two dimensions can be described as the normalized multiplication of the Fourier Transform (FT) of the lens Point Spread Function by the FT of the (square) pixel footprint, convolved with the FT of a rectangular grid of Dirac delta functions centered at each  pixel, as better described in the article

    \[ MTF_{2D} = \left|(\widehat{ PSF_{lens} }\cdot \widehat{PIX_{ap} })\ast\ast\: \delta\widehat{\delta_{pitch}}\right|_{pu} \]

With a few simplifying assumptions we will see that the effect of the lens and sensor on the spatial resolution of the continuous image on the sensing plane can be broken down into these simple components.  The overall ‘sharpness’ of the captured digital image can then be estimated by combining the ‘sharpness’ of each of them. Continue reading A Simple Model for Sharpness in Digital Cameras – I

A Longitudinal CA Metric for Photographers

While perusing Jim Kasson’s excellent Longitudinal Chromatic Aberration tests[1] I was impressed by the quantity and quality of the information the resulting data provides.  Longitudinal, or Axial, CA is a form of defocus and as such it cannot be effectively corrected during raw conversion, so having a lens well compensated for it will provide a real and tangible improvement in the sharpness of final images.  How much of an improvement?

In this article I suggest one such metric for the Longitudinal Chromatic Aberrations (LoCA) of a photographic imaging system: Continue reading A Longitudinal CA Metric for Photographers

Photographic Sensor Simulation

Physicists and mathematicians over the last few centuries have spent a lot of their time studying light and electrons, the key ingredients of digital photography.  In so doing they have left us with a wealth of theories to explain their behavior in nature and in our equipment.  In this article I will describe how to simulate the information generated by a uniformly illuminated imaging system using open source Octave (or equivalently Matlab) utilizing some of these theories.  Since as you will see the simulations are incredibly (to me) accurate, understanding how the simulator works goes a long way in explaining the inner workings of a digital sensor at its lowest levels; and simulated data can be used to further our understanding of photographic science without having to run down the shutter count of our favorite SLRs.  This approach is usually referred to as Monte Carlo simulation.

Continue reading Photographic Sensor Simulation

Dynamic Range and Bit Depth

My camera has an engineering Dynamic Range of 14 stops, how many bits do I need to encode that DR?  Well, to encode the whole Dynamic Range 1 bit will suffice.  The reason is simple, dynamic range is only concerned with the extremes, not with tones in between:

    \[ DR = \frac{Maximum Signal}{Minimum Signal} \]

So in theory we only need 1 bit to encode it: zero for minimum signal and one for maximum signal, like so

Continue reading Dynamic Range and Bit Depth

Determining Sensor IQ Metrics: RN, FWC, PRNU, DR, gain – 2

There are several ways to extract Sensor IQ metrics like read noise, Full Well Count, PRNU, Dynamic Range and others from mean and standard deviation statistics obtained from a uniform patch in a camera’s raw file.  In the last post we saw how to do it by using such parameters to make observed data match the measured SNR curve.  In this one we will achieve the same objective by fitting standard deviation data.  Since the measured data is identical, if the fit is good so should be the results.

Sensor Metrics from Measured Mean and Standard Deviation in DN

Continue reading Determining Sensor IQ Metrics: RN, FWC, PRNU, DR, gain – 2

Sensor IQ’s Simple Model

Imperfections in an imaging system’s capture process manifest themselves in the form of deviations from the expected signal.  We call these imperfections ‘noise’.   The fewer the imperfections, the lower the noise, the higher the image quality.  However, because the Human Visual System is adaptive within its working range, it’s not the absolute amount of noise that matters to perceived IQ as much as the amount of noise relative to the signal. That’s why to characterize the performance of a sensor in addition to noise we also need to determine its sensitivity and the maximum signal it can detect.

In this series of articles I will describe how to use the Photon Transfer method and a spreadsheet to determine basic IQ performance metrics of a digital camera sensor.  It is pretty easy if we keep in mind the simple model of how light information is converted into raw data by digital cameras:

Sensor photons to DN A


Continue reading Sensor IQ’s Simple Model

Olympus E-M5 II High-Res 64MP Shot Mode

Olympus just announced the E-M5 Mark II, an updated version of its popular micro Four Thirds E-M5 model, with an interesting new feature: its 16MegaPixel sensor, presumably similar to the one in other E-Mx bodies, has a high resolution mode where it gets shifted around by the image stabilization servos during exposure to capture, as they say in their press release

‘resolution that goes beyond full-frame DSLR cameras.  8 images are captured with 16-megapixel image information while moving the sensor by 0.5 pixel steps between each shot. The data from the 8 shots are then combined to produce a single, super-high resolution image, equivalent to the one captured with a 40-megapixel image sensor.’

A great idea that could give a welcome boost to the ‘sharpness’ of this handy system.  This preliminary test shows that the E-M5 mk II 64MP High-Res mode gives in this case a 10-12% advantage in MTF50 linear spatial resolution compared to the Standard Shot 16MP mode.  Plus it apparently virtually eliminates the possibility of  aliasing and moiré.  Great stuff, Olympus.

Continue reading Olympus E-M5 II High-Res 64MP Shot Mode

How to Measure the SNR Performance of Your Digital Camera

Determining the Signal to Noise Ratio (SNR) curves of your digital camera at various ISOs and extracting from them the underlying IQ metrics of its sensor can help answer a number of questions useful to photography.  For instance whether/when to raise ISO;  what its dynamic range is;  how noisy its output could be in various conditions; or how well it is likely to perform compared to other Digital Still Cameras.  As it turns out obtaining the relative data is a little  time consuming but not that hard.  All you need is your camera, a suitable target, a neutral density filter, dcraw and free ImageJ, Octave or (pay) Matlab.

Continue reading How to Measure the SNR Performance of Your Digital Camera