We understand from the previous article that rendering color during raw conversion essentially means mapping raw data represented by RGB triplets into a standard color space via a Profile Connection Space in a two step process
The process I will use first white balances and demosaics the raw data, which at that stage we will refer to as , followed by converting it to Profile Connection Space through linear transformation by an unknown ‘Forward Matrix’ (as DNG calls it) of the form
Determining the nine coefficients of this matrix is the main subject of this article.
The second step maps the resulting image information from to the ‘output’ colorimetric color space chosen by the photographer, say sRGB or Adobe RGB. The relative linear matrices are standardized and readily available.
9 Equations and 9 Unknowns
So how do we determine the nine coefficients of the Forward Matrix in Equation 1?
The 9 unknown coefficients operate on white balanced and demosaiced RGB data to transform it linearly into data. It follows that if we had 3 sets of values and their corresponding triplets we could solve for the coefficients. The results would be valid for the given hardware and illumination.
For instance we could capture with the camera whose matrix we want to determine 3 targets of uniform diffuse reflectance illuminated by a known light source; measure with a spectrophotometer or similar instrument the reflectance of the targets; calculate the values that the reflectance and illuminance imply; all that would be left to do then is assemble the 3×3 pairs of and corresponding data in the form of 9 equations and 9 unknowns to solve for the coefficients of the relative Forward Matrix. Below the ‘s are the unknowns, the X,Y,Z’s and R,G,B’s would be known:
Looks easy? In fact it is much easier than that.
1 Capture, 72 Equations
X-Rite produces a number of ColorChecker 24 patch standard targets whose reflectance information is published and well thumbed. For this example I will use their handy Passport Photo version.
The 24 patches in the ColorChecker target carry reflectances that are supposed to be representative of everyday photographic subjects, as found in skin, foliage and sky colors. The color scientists at BabelColor.com have measured a sample of 30 ColorChecker targets over the years and compared them to published specifications (pre-November 2014 formulations shown, as my unit is older than that):
Note that 1 is supposed to represent a just noticeable color difference, so you can see that with the exception of purple and white the patches do seem to provide a reasonably stable reference. This information, as well as average patch reflectance from 380 to 730nm at 10nm increments, is available in a spreadsheet from the linked page above.
Bingo! the L*a*b* color space is a simple transformation away from . With the data above and a single raw capture of a ColorChecker Passport Photo target in clear noonish sunlight (call it D50) we’ve got 24 as opposed to just 3 sets of raw and reference data triplets needed to solve for the coefficients of the Forward Matrix in Equation 1. And we can use the larger data set to make sure that the coefficients fit a wider number of potential photographic subjects. Shutter Release.
Computing the Coefficients
Ok, so now we have the cc24 target illuminated by a roughly D50 source captured in the raw file of a Nikon D610+24-120mm/4. Next we read the mean values of each of the 24 patches with a tool like RawDigger . Then we white balance each patch triplet based on the second gray patch from the left in the bottom row. Then we demosaic it. This is now the white balanced and demosaiced raw data set that we need, .
We could then compute the corresponding reference ColorChecker values published by X-Rite or BabelColor, brush up on linear algebra skills and solve the 72 equations for the coefficients that best fit the available data. However, linear algebra was never my favorite subject so why go through all that hard work when computers can do it much faster and painlessly for us?
We can set up a spreadsheet with the 24 triplets and 9 cells representing the coefficients in the forward matrix as arrays. Seed the coefficients to, say, 1/3 each. Matrix multiply the triplets by the seed matrix, convert to L*a*b*, compute differences to the reference values in Figure 1 above and let Excel solver figure out what values of the 9 coefficients minimize the sum of the differences. The resulting matrix will be the best compromise found for the 24 patches under that illuminant, giving each patch equal importance.
The Compromise Forward Matrix
I effectively followed that last procedure but using Matlab/Octave instead of Excel. Excellent toolkit optprop and its built-in ColorChecker reference data lent a helping hand. This is the forward matrix obtained for my setup:
Cool. These are the differences for each patch based on optrop’s ColorChecher reference data.
The average is 1.5 and maximum is 3.8 in the light skin patch. Recall that 1 is a just noticeable difference. I repeated the exercise this time using the BabelColor 30 database as reference and this is the resulting forward matrix then:
Looks like the differences to reference data from BabelColor’s database are more evenly distributed: average is still 1.5 but Maximum is a lower 3.1 in the ‘green’ patch.
To do it properly I should have measured the Spectral Power Distribution of the illuminant and the reflectance of the patches with a spectrophotometer around the time that the capture was taken (didn’t have one, ColorMunki Photo on the way). Using published reference reflectances pollute the data and contribute to degrade the results – but you get the idea.
Correcting Matrix ‘Errors’: Profiles
Note however that the compromise matrix is just that, a compromise, and even if the procedure had been perfect it could result in quite large errors. For better overall color rendering performance the preferred method is to correct such errors through optional nonlinear color profile adjustments that are introduced while in via HSV lookup tables. The tables and application methods are usually referred to as camera ‘profiles’ (ICC and DCP for instance), a subject beyond the scope of this article.
Calculating Sensitivity Metamerism Index (SMI)
While the data is out we can calculate the Sensitivity Metamerism Index, which is equal to 100 minus 5.5 times the average (not ) of just the 18 color patches. The shown values (misnamed CRI above) are not necessarily maximized for the given setup because the matrix finding routine is built upon minimizing . Still my D610 with its 24-120mm/4 around D50 shows SMIs of 80 in the first case and 83 in the second. They jump to 82 and 86 respectively if the routine is setup to minimize instead. Not bad, although I know most people do not give much credence to this metric – and I can see why.
Step 2: Matrix to Output Color Space
Now that we have done the hard work of determining the forward matrix to convert white balanced raw data to the PCS with this illuminant, we need a standard matrix to move it on to the colorimetric color space chosen by the photographer for output. In this case we will map it to by multiplication with the following transform matrix obtained from Bruce Lindbloom’s site
The product of this and the forward matrices calculated earlier produces the following combined linear transforms for a D50ish illuminant. My results are shown top and bottom, with DXOmark’s for the D610 at D50 in the center for reference:
Pretty close, which confirms the illuminant of my captures was indeed near D50.
So that’s where color matrices come from and why we need them. Next, a closer look and putting them to work.
PS: I got myself an Xrite ColorMunki Photo spectrophotometer, it’s a blast. It comes with its own ColorChecker 24 mini, I assume with the new formulation. I measured it with the Munki and captured it in the raw data in somewhat similar conditions to Figure 2. Correlated Color Temperature was about 5050K. These are the matrices that are produced from it (white balanced off my WhiBal card):
Pretty similar and SMI is now 85. Here are the differences, measured vs captured:
Now average is 1.17, there are only three patches above 2 and the rest seem to be better controlled overall. I guess pushing down the outliers is the reason why linear matrices are not enough and profiles are necessary.
Notes and References
1. Lots of provisos and simplifications for clarity as always. I am not a color scientist, so if you spot any mistakes please let me know.
2. The ColorChecker pages at BabelColor.com can be found here. Careful that there was a change in formulations in November 2014.
3. RawDigger can be found here and dcraw here.
4. optprop can be found here.
5. For an example of profile implementation see for instance Adobe’s Digital Negative Specification Version 18.104.22.168.
6. See here for a description of the Sensitivity Metamerism Index and here for DXO’s take on it.
7. See this page at Bruce Lindbloom’s site for precise matrices from XYZD50 to many colorimetric color spaces.
8. DXOmark.com D610 color measurement can be found here by clicking on the ‘color response’ tab.