Imperfections in an imaging system’s capture process manifest themselves in the form of deviations from the expected signal. We call these imperfections ‘noise’. The fewer the imperfections, the lower the noise, the higher the image quality. However, because the Human Visual System is adaptive within its working range, it’s not the absolute amount of noise that matters to perceived IQ as much as the amount of noise relative to the signal. That’s why to characterize the performance of a sensor in addition to noise we also need to determine its sensitivity and the maximum signal it can detect.
In this series of articles I will describe how to use the Photon Transfer method and a spreadsheet to determine basic IQ performance metrics of a digital camera sensor. It is pretty easy if we keep in mind the simple model of how light information is converted into raw data by digital cameras:
Light in units of photons, typically reflected by the scene, travels through the lens and strikes pixels on the sensor proportionately to the given, uniform Exposure. In current Bayer implementations only a sensor-specific 15-30% of incident photons get converted to photoelectrons thanks to the photoelectric effect. We call this proportion Effective Quantum Efficiency (eQE).
The mean number of photoelectrons generated by such uniformly illuminated pixels is the Signal for the given exposure. In the diagram above we denote the Signal out of the sensor with the lowercase letter ‘s’, in units of photoelectrons (e-).
Noise: Deviation from the Expected Signal
For this simple sensor IQ model we will ignore minor sources of noise and concentrate on the main ones, which can mostly be considered to be random. Random quantities are characterized by their distribution and standard deviation: the higher the deviation, the higher the noise.
We will aggregate noise sources into three main categories: inherent to the Signal (shot noise); related to the electronics (read noise); and Signal related non-uniformities in the sensor (PRNU). If you are interested in a more in-depth description of these noise sources I suggest Emil Martinec’s pages.
The photoelectric effect performs its conversion of photons to photoelectrons according to Poisson statistics; if we were to count the number of photoelectrons generated by pixels in an area of a sensor containing 200×200 of them, all with the same number of photons striking silicon*, we would find that the count would have a standard deviation equal to the square root of the mean. This is ‘shot’ noise inherent in the signal, related to Signal (s) in e-. Therefore the standard deviation of Shot Noise is:
Read noise introduced by the electronics will have its own Gaussian standard deviation, in e-
with r a constant, function of gain.
PRNU is modeled proportional to the Signal in e- and has standard deviation
with p the PRNU factor usually expressed as a percentage of the signal. Ignoring Pattern and other types of noise, if the three noise sources are uncorrelated (they are) they add in quadrature so that the Total Random Noise standard deviation is
For convenience, in this phase of the simple model description we are assuming that the Signal and all noise sources are present at the output of the sensor before amplification and expressed in the same units as the Signal, photoelectrons. Therefore written using our definitions above, this total input-referred noise (n) is equal to
all variables in units of e-.
How to Measure RN and PRNU – without Gain
Moving along our model, the Signal and noise output by the sensor in units of photoelectrons are then amplified according to the gain (g, in units of DN/e-)** controlled by the in-camera ISO setting, converted by the Analog to Digital Converter (ADC) to digital units – sometimes referred to as Analog to Digital Units (ADU) or Data Numbers (DN) or simply ‘the count’ – before finally being stored in the camera’s Raw File. The value for g in the figure is typical of some current full frame sensors at base ISO. The mean Signal written to the raw file is (capital letters represent units in DNs)
and total noise standard deviation as measured in the raw file is
Therefore the signal to noise ratio, the IQ, of the information recorded in the camera’s raw file is
with S and N in units of DN, g in DN/e- and all other variables in units of e-.
Equation (2) is extremely powerful because it ties the Signal (S) and total noise (N) that we can measure in DN as the mean and standard deviation of data in the raw file to the internal figures of merit of the sensor – read noise (r) and PRNU (p) in physical units of photoelectrons – that we do not have direct access to.
How to Measure RN and PRNU – with Gain
Another way to determine sensor metrics from statistics collected from data in the raw file is to rewrite the total input-referred standard deviation noise equation (1) in output-referred DN by using as follows
With signal in DN and read noise in DN , the total standard deviation measured in the raw data is
Contrary to Equation (2) the formula above contains an additional variable, gain (g), its significance outlined below.
The Full Simple Model of Sensor IQ
Our full simplified model including random noise (but excluding pattern noise) then becomes
Note that shot noise is easily calculated as a function of signal in e- in the example above: = 5 e- (1 DN); we would not be able to calculate it from the signal in DN, though, as it would first need to be divided by gain: is not equal to 1 DN.
In the next posts I will show how to use measured statistics from the raw data and equations (2) or (3) to obtain the sensor’s noise figures of merit r (read noise), f (saturation) and p (PRNU constant) as a function of gain (g)**.
* Just for clarity. In practice photons are not emitted uniformly by natural light sources therefore they would not actually arrive in equal numbers on the pixel patch. They would instead arrive according to Poisson statistics, their standard deviation referred to as ‘photon’ noise.
** Some authors use sensitivity k in e-/DN instead of gain g, in DN/e-. It makes no difference as long as you remember that g = 1/k.