What Colorcheck does

Colorcheck analyzes images of the widely-used GretagMacbethTM ColorChecker® for tonal response (using the six grayscale patches, which have an optical densities from 0.05 to 1.50; a range of 1.45 = 4.8 f-stops), noise (for the six grayscale patches), and color quality. Results for tonal response are similar to Imatest Q-13 program.

The GretagMacbeth ColorChecker is an 8x11" inch chart consisting of 24 patches with 18 familiar colors and six grayscale levels, white to black. None of the colors is completely saturated. The quality of the chart is very high each patch is printed separately using carefully controlled pigments. Patches have a smooth matte surface. It is available from Adorama and other dealers. A 2.25"x3.25" mini ColorChecker is also available.

An ideal simulated ColorChecker is shown on the right for sRGB color space (the Windows/Internet standard). The optical densities (-log10(reflectivity)) of the grayscale patches on the bottom row are shown in parentheses. One color, 18. Cyan, is out of the sRGB gamut; hence it can't be reproduced perfectly on most monitors.

 
1.
dark
skin 
2.
light
skin
3.
blue
sky
4.
foliage
5.
blue
flower
6.
bluish
green
7.
 orange 
8.
purplish
blue
9.
moderate
red
10.
purple
11.
yellow
green
12.
 orange 
 yellow 
13.
blue
14.
green
15.
red
16.
 yellow 
17.
magenta
18.
cyan
19.
white
(.05)
20.
neutral
8 (.23)
21.
neutral
6.5 (.44)
22.
neutral
5 (.70)
23.
neutral
3.5 (1.05)
24.
black
(1.50)
 
What colors are the ColorChecker?

The information supplied with the ColorChecker is lacking. Grayscale densities (for the bottom row) are printed on the back of the chart. CIE xyY values (used in the familiar xy chromaticity chart) are printed on the package and in a sheet that comes inside the package. The sheet also contains a set of RGB values that corresponds to no known color space used by photographers. It can be ignored.

xyY values are cannot be used directly by Imatest. RGB values are needed to display the ColorChecker (in sRGB color space for Website display). Also, xy chromaticity values are far from perceptually uniform, i.e., distances between points on the xy-plane are not proportional to perceptible differences between colors. (Green is greatly exaggerated.) Color differences are represented much better in the CIELAB color space, where a distance of 1 between L*a*b* values represents the minimum perceptible difference between colors (for relatively unsaturated colors). For colors on the a*b* plane, this distance is expressed by the equation,

Delta-E = sqrt( (a1* - a2*)2+ (b1* - b2*)2 )

Delta-E is widely used to quantify color differences. It is the length of the lines that connect the ideal and measured a*b* color values in the third figure, described below.

There are three sources of ColorChecker values. Each is slightly different.

  1. The GretagMacbeth website. Two downloads are available. (1) a ColorChecker image, evidently in sRGB color space (though GretagMacbeth neglects to mention this useful tidbit). (2) an Excel table of XYZ, L*a*b* and RGB values, measured with Illuminant C (an old light source standard similar to D65 = 6500K and daylight). Values are closer to to Danny Pascale's sRGB values (~2) than to any other space.
  2. BruceLindbloom.com. Contains equations and tables for converting between various RGB color spaces as well as XYZ, xyY, and CIELAB. Click on Math. His ColorChecker values are based on acurate spectrophotometer measurements of a single sample. He has a useful ColorChecker calculator. A rich resource.
  3. Danny Pascale/BabelColor. RGB coordinates of the Macbeth ColorChecker presents ColorChecker values in xyY, L*a*b*, and several color spaces, based on the same xyY values used by Gretag. He also presents procedures and equations for data conversion. The L*a*b* values are for D50 illumination. I prefer Gretag's values because Illuminant C is closer to the D65 white point of the sRGB color space.
  4. Bruce Fraser, Creativepro.com. Calibrating Camera Raw in Photoshop CS contains Bruce's ColorChecker L*a*b* values from his book, Real World Color Management. I not sure what reference color temperature he uses. Allen Pacheco has published a similar calibration technique.

For now, Imatest uses the GretagMacbeth L*a*b* values and the Pascale RGB values (close to GretagMacbeth). A future version may allow the source to be selected, but differences are minor.

Photographing the target and running colorcheck

Output

The example was photographed with the Canon EOS-10D at ISO 400, and RAW converted with Canon FVU using automatic color balance.

The first figure: Gray scale tonal response and noise

shows graphic results derived from the grayscale patches in the bottom row. The results include tonal response and noise.

The upper left plot shows the average density of the grayscale patches (black curve) and first and second order density fits (dashed blue and green curves). The horizontal axis is the distance along the target. A portion of the patches themselves are shown just above the plot. The average density curve is useful for diagnosing uneven lighting: the tops of this curve will be flat if the lighting is even. The equations for the first and second order density fits are given in the Algorithm section, below. The second order fit accurately matches the patches; it is used to calculate the corrected ideal ColorChecker values in the third figure, below. The upper right plot shows the density response of the colorchecker (gray squares) as well as the first and second order fits (dashed blue and green lines). The horizontal axis is Log Exposure (minus the target density), printed on the back of the ColorChecker.
The lower left plot shows the RMS noise for each patch: R, G, B, and Y (luminance). The average R, G, B, and Y noise for the gray zones (2-5) is also reported. RMS noise is expressed in percentage of the difference between the white and black patches. The lower right plot shows the noise spectrum of patch 4 in the bottom row (middle gray). It contains hidden information about signal processing, for example, an unusually rapid rolloff of noise energy may indicate excessive noise reduction. The ISO speed and Exposure time from the EXIF data are displayed, if available.

Two precautions when working with figures
Too many open Figures
Figures can proliferate if you do a number of runs, especially SFR runs with multiple regions, and system performance suffers if too many Figures are open. You will need to manage them. Figures can be closed individually by clicking X on the upper right of the Figure or by any of the usual Windows techniques. You can close them all by clicking Close figures in the Imatest main window.
Clicking on Fgures during calculations
can confuse Matlab. Plots can appear on the wrong figure (usually distorted) or disappear altogether. Wait until all calculations are complete until the Save or Imatest main window appears before clicking on any Figures.

The second figure: noise detail

shows the density response, noise in f-stops (a relative measurement that corresponds to the workings of the eye), noise for the third Colorchecker row, which contains primary colors, and selected EXIF data.

The upper left plot  is the density response of the colorchecker (gray squares). It includes the first and second order fits (dashed blue and green lines). The horizontal axis is Log Exposure (minus the target density), printed on the back of the ColorChecker. Q-13 Stepchart provides a more detailed density response curve. The upper right plot shows the noise in the third colorchecker row, which contains the most strongly colored patches: Blue, Green, Red, Yellow, Magenta, and Cyan. In certain cameras noise may vary with the color. Problems may be apparent that aren't visible in the gray patches.
The lower left plot shows the R, G, B, and Y (luminance) RMS noise for as a function of Log Exposure each patch. RMS noise is expressed in f-stops, a relative measure that corresponds closely to the workings of the human eye. This measurement is described in detail in the Q-13 Stepchart tour. It is largest in the dark areas because the pixel spacing between f-stops is smallest. The lower right region displays EXIF data, if available.

The third figure: color error

illustrates the color error in the device-independent CIELAB color space, where a* is the horizontal axis and b* is the vertical axis. The small squares are the ideal (a*, b*) ColorChecker values, calculated by Danny Pascale of BabelColor. The large circles are the (a*, b*) values of the actual measured ColorChecker. The numbers near the squares or circles correspond to the numbers of the ColorChecker patches: 1-6 are in the top row, 7-12 are in the second row, and 13-18 are in the third row. The numbers for patches 19-24 (the bottom row) are omitted because their (a*, b*) values cluster around (0, 0).

The background of the chart shows expected colors (in monitor sRGB color space) for L* = 0.95. It presents a reasonably accurate picture of the hues associated with a* and b*. There is significant scatter in the results because the luminance (L) of the color patches (not displayed) varies considerably.

CIELAB (often shortened to LAB) was designed to be perceptually uniform, meaning that the perceived difference between colors is proportional to the distance between them, which is called delta-E. ( Delta-E = sqrt( (a2*-a1*)2 + (b2*-b1*)2 ) ). (The "-" in Delta-E is a hyphen, not a minus.) The smallest perceptible difference corresponds roughly to delta-E = 1. CIELAB isn't perfect in this regard. Delta-E for the minimum perceptible color difference increases with distance from the origin (a* = b* = 0). But it is much more uniform than the popular CIE xy chromaticity diagram.

Four values are reported on the upper right.

It will be difficult  to assess these numbers until a large database of test results is available. I would characterize the mean value of 5.66 and sigma of 6.73 for the EOS-10D (above) as very good— perhaps excellent.

See the Links, below, for details on CIELAB color space.

The fourth figure: color analysis

is an image of the ColorChecker that allows you to compare the actual and ideal values. It also displays White Balance error using the bottom row (the grayscale patches).

The outer portion (1) of each patch contains the ColorChecker image as photographed, The inner portion contains the ideal ColorChecker values corrected for exposure (2) and uncorrected (3).

1. the outer region, is the patch as photographed. This corresponds to the circles in the L*a*b* color error plot, above.

2. the square in the center, is the ideal value of the patch, corrected for the luminance of the photographed chart. The correction is derived from the second order fit to the gray areas, described above. The average luminance of zones 1 and 2 shoud be close— zone 2 may be darker in some patches and lighter in others. Zones 2 and 3 correspond to the small squares in the L*a*b* color error plot, above.

3. the small rectangle to the right of the central square, is the ideal value of the patch, with no luminance correction. The luminance of zone 2 will be consistently lighter or darker than zone 3 for all patches, depending on the exposure.

In the above example, the ColorChecker image 1 is slightly underexposed; it is darker than the ideal uncorrected color 3. It is also darker than the corrected ideal color 2. This is a characteristic of this camera: it tends to darken oranges and lighten blues and cyans. This can be seen in the above right ColorChecker image, which is slightly overexposed. This image of the orange-yellow patch (row 2, column 6) has about the same luminance as the uncorrected ideal color 3, but is darker than the corrected ideal color. The uncorrected ideal value 3 in the two orange-yellow images above (the example on the left and the full chart on the right) are the same, but appear to be quite different because of the interaction of colors in the eye— an effect that should be familiar to artists and photographers.

This image provides a clear visual indication of color accuracy. The L*a*b* color error plot provides a quantitative indication.

The bottom of the third figure illustrates the White Balance error, which is quantified in three ways. (1) Saturation S in the HSV color model (burgundy). S can take on values between 0 for a perfect neutral gray and 1 for a totally saturated color. (2) Correlated color temperature error in Degrees Kelvin (blue), calculated from Color Science: Concepts and Methods, Quantitative Data and Formulae, by Günther Wyszecki and W. S. Stiles, pp. 2224-228. (2) Color temperature in Mireds (micro-reciproacal degrees) (blue), where Mireds = 106/(Degrees Kelvin) is a more perceptually uniform measure.

White Balance error tends to be most visible in the gray patches (2-5 in the bottom row). It is barely visible for S < 0.02. It is quite serious for S > 0.10, particularly for the lighter gray patches.
 
The image at the bottom shows  the bottom row (the grayscale patches) with exaggerated white balance error, calculated by boosting saturation S using the curve on the left, keeping H and V constant. Low values of saturation are boosted by a factor of 4; the boost decreases as saturation increases. This image shows the White Balance error much more clearly than the unexaggerated chart. Keep it in mind that this image is worse than reality; small White Balance errors may not be obvious in actual photographs, especially in the vicinity of strongly saturated colors.

Saving the results

When the Colorcheck calculations are complete, the Save results? dialog box appears. It allows you to choose which results to save and where to save them. The default is subdirectory Results in the data file directory. You can change to another existing directory, but new results directories must be created outside of Imatest— using a utility such as Windows Explorer. (This is a limitation of this version of Matlab.) The selections are saved between runs.

You can examine the output figures before you check or uncheck the boxes. After you click on Yes or No, the Imatest main window reappears.

The .CSV summary file contrains Excel .CSV output for the tone level, noise, and color calculations. It also contains some additional data, including EXIF data for JPEG files.


Algorithm
  • Locate the regions of interest (ROIs) for the 24 ColorChecker zones.
  • Calculate statistics for the six grayscale patches in the bottom row, including the average pixel levels and a second order polynomial fit to the pixel levels in the ROIs this fit is subtracted from the pixel levels for calculating noise. It removes the effects of nonuniform illumination. Calculate the noise for each patch.
  • Using the average pixel values of grayscale zones 2-5 in the bottom (omitting the extremes: white and black), the average pixel response is fit to a mathematical function (actually, two functions). This requires some explanation.
A simplified equation for a capture device (camera or scanner) response is,
    normalized pixel level = (pixel level/255) = k1 exposuregamc
Gamc is the gamma of the capture device. Monitors also have gamma = gamm defined by
    monitor luminance = (pixel level/255)gamm
Both gammas affect the final image contrast,
    System gamma = gamc * gamm
Gamc is typically around 0.5 = 1/2 for digital cameras. Gamm is 1.8 for Macintosh systems; gamm is 2.2 for Windows systems and several well known color spaces (sRGB, Adobe RGB 1998, etc.). Images tend to look best when system gamma is somewhat larger than 1.0, though this doesn't always hold certainly not for contrasty scenes. For more on gamma, see Glossary, Using Imatest SFR, and Monitor calibration.
Using the equation, density = - log10(exposure) + k,
    log10(normalized pixel level) = log10( k1 exposuregamc ) = k2 - gamc * density
This is a nice first order equation with slope gamc, represented by the blue dashed curves in the figure. But it's not very accurate. A second order equation works much better:
    log10(normalized pixel level) = k3 + k4 * density + k5 * density2
k3, k4, and k5 are found using second order regression and plotted in the green dashed curves. The second order fit works extremely well.
  • Saturation S in HSV color representation is defined as S(HSV) = (max(R,G,B)-min(R,G,B))/max(R,G,B). S correlates more closely with perceptual White Balance error in HSV representation than it does in HSL.
The the equation for saturation boost in the lower image of the third figure is S' = (1-e-4S )/(1-e-4), where e  = 2.71828...

Links

RGB coordinates of the Macbeth ColorChecker by Danny Pascale, The BabelColor Company, 2003 (PDF). Excellent survey of color spaces and techniques for calculating the RGB values of the ColorChecker.
Bruce Lindbloom  Outstanding site with equations for converting between color spaces and models. Slightly different L*a*b* values from Pascale.
Earl F. Glynn (EFG) has an excellent description of the ColorChecker. His Computer Lab and Reference Library are a goldmine of information about color, image processing, and related topics.