Using Stepchart

Stepchart analyzes the tonal response, noise, and dynamic range of digital cameras and scanners using

• reflection step charts such as the Kodak Q-13 and Q-14 Gray Scales, the Jessops Colour & Mono Separation Guide (UK), several from Danes-Picta (Czech republic), charts created by Test Charts (which can be printed any size), or
• transmission step charts from Stouffer, Applied Image, Kodak, DSC Labs, and Danes-Picta. Transmission charts are required for measuring dynamic range.
• Charts from Applied Image and additional ISO charts, shown below. (Imatest Pro only)

Stepchart can also measure veiling glare (lens flare).

The Kodak Q-13 Gray Scale is an 8 inch long chart consisting of 20 zones, labeled 0-19, which have optical densities from 0.05 to 1.95 in steps of 0.1 (reflectances from 0.891 to 0.011). The Kodak Q-14 is identical, except that it's 14 inches long. The chart is printed on a (semigloss) surface. The Jessops chart, from the UK, is similar to the Q-13, except that it has only 18 zones.

This image of the Q-13 Gray Scale was photographed slightly out of focus to minimize noise.

The Q-13 has finer steps and a higher maximum density than the GretagMacbeth ColorChecker (Dmax = 1.95 vs. 1.5), mostly because of its glossier surface. It considerably less expensive. The longer Q-14 is well suited for photographing near ColorCheckers or slanted-edge SFR test targets.

Iliah Borg has analyzed the Q-13 Gray Scale. He says, "I'm pretty sure it's screen-printed, most likely with automotive enamel. The spectral response is flat from 420 to 730nm, similar to titanium dioxide mixed with carbon in different proportions to achieve different reflectivities. The layer is pretty thick, to isolate from the substrate."

Applied Image and ISO targets: Imatest Pro Five monochrome targets, shown below, have been added to Imatest Pro with Imatest 1.7. To run them, open the image file, then select the approximate region of interest for the key features in the target, following the instructions in Stepchart with Applied Image/ISO targets. Additional dialog boxes allow you to specify the target type and refine the crop. QA-61 ISO-16067-1 Scanner QA-62 ST-51 EIA Grayscale ST-52 ISO-14524 12-patch OECF target ISO-15739 Noise target   20-patch OECF targets ITE Grayscale

Stepchart also analyzes transmission (i.e., film) charts, which have higher maximum densities than reflective charts, making them suitable for measuring the dynamic range of digital cameras and scanners, as discussed below, in the section on Dynamic range.

Photographing the chart and running Stepchart

• Light. Be sure the light is even and free of glare. As little as possible should come from behind the camera— it can cause glare and flatten image tones. At least two lamps with an incident angle of about 30° are recommended. A recommended lighting setup is described in The Imatest Test Lab. Be especially careful to avoid glare in the dark zones on the right of the Q-13/Q-14 charts; it can hard to avoid with wide angle lenses. The light should not emphasize the texture of the chart, which would cause erroreous noise measurements. If possible, the surroundings of the Step chart should be dark gray or black to minimize flare light (unless you are measuring flare light).
• Focus. The chart may be photographed slightly out of focus to minimize noise measurement errors due to texture in the patches. I emphasize slightly— the boundaries between the patches must remain distinct.
• Distance. The distance at which the target is photographed is not critical. The noise analysis will be most accurate if the total cropped chart length is at least 1000 pixels (about 50 pixels per zone), about 1/3 the width of the image in a 6 megapixel digital SLR. Increasing the size improves the accuracy of the noise measurement up to a point: there is little improvement over 2000 pixels wide. Useful (though less accurate)  noise analysis results may be obtained with total cropped length as small as 400 pixels. The tonal response can be measured with very small images, as long as the zones are distinctly visible.
• Photograph the chart.
• Save the image as a RAW file, TIFF, or maximum quality JPEG, then load it on your computer. If you are using a RAW converter, convert to JPEG (maximum quality), TIFF (without LZW compression), or PNG. If you are using film, develop and scan it.
• Run Imatest.
• Click on the Stepchart button in the Imatest main window.
• Open the input file using the dialog box. Imatest remembers the directory name of the last input file opened (for each module, individually).
  
  

  Batch mode  This module can operate in batch mode in Imatest Pro, i.e., it can read multiple input files. All you need to do is select several files using the standard Windows techniques of shift-click or control-click. There are three requirements for the files. They should (1) be in the same folder, (2) have the same pixel size, and (3) be framed identically. The input dialogs (cropping (if applicable), settings, and save) are the same for the first run as for standard non-batch runs. Additional runs use the same settings as the first run. Since no user input is required they can run extremely fast. If the order of the files in a batch runs is different from the selection order, click Settings, Options and Settings... (in the Imatest main window) and change the setting in Batch run order. One caution: Imatest can slow dramatically on most computers when more than about twenty figures are open. For this reason we recommend checking the Close figures after save checkbox, and saving the results. This allows a large number of image files to be run in batch mode without danger of bogging down the computer.

  
  
• Crop the image to minimize edge effects. Typical crops (for automatic zone detection) are shown below, indicated by red rectangles. Click outside the image for no crop. The orientation does not need to be correct; Imatest will rotate the image to the correct orientation. To display the exposure error correctly (for reflective targets only), select the entire chart, including any clipped highlight patches that may be present. If the aspect ratio (length:width) of the crop is less than 4:1, the selection will be rejected (Imatest Light) or a dialog box for one of the Applied Image/ISO charts will appear (Imatest Pro). If you plan to use manual Zone detection (selected in the Stepchart input dialog box, below), the selection can be approximate: a fine selection window will enable you to refine the crop.


Crop of a Kodak Q-13 chart .

Crop of a Stouffer T4110 chart (better without the tilt).

This is an image processed by Stepchart, after cropping.


• Make any needed changes to the Stepchart input dialog box.

Chart type   A reflective target with density steps of 0.10 (the Kodak Q-13/Q-14) is selected by default. If you are using a transmission target, be sure to choose the correct target type. In Imatest Pro you can enter a table of densities from an ASCII file (one density per line). Automatic (default) must be unchecked to enable Veiling glare measurements. (See Dynamic range, below.) Click OK to continue.

Zone detection  You can toggle between Automatic (the default) and Manual Zone detection with the Automatic (default) checkbox. If it is unchecked, a fixed number of patches (6-41) can be selected with the slider for Stepchart analysis, and a fine ROI adjustment box appears after OK is pressed. This option is useful when high noise in dark regions confuses the automatic ROI detection, or when veiling glare is to be calculated. (Zone detection can also be set from Options & Settings in the Imatest main window.)


Fine ROI selection (dialog box detail): Details here.

Scaling  You can select the minimum value of the x-axis for figures 2 and 3. This setting defaults to Auto. Setting it manually allows several successive runs to be scaled identically, which can clarify comparisons between different cameras.

Results (Fig. 1), Lower plot:  Pixel noise  Three noise displays are available for the lower-left plot in first figure.

Noise (%) normalized to image density range = 1.5	Values 0-1. Useful because it references the noise to the scene: noise performance is not affected by camera contrast.
Noise (%) normalized to 255 pixels	Values 0-1. Noise will be worse for higher contrast cameras (affected by the gamma encoding)
Noise in pixels (maximum of 255)	Values 0-255.

Noise Detail (Fig. 2), Middle plot:  f-stop noise or SNR   Two types of display are available for the middle-left plot of the second figure. The effects of gradual illumination nonuniformities have been removed from the results.

f-stop noise	Scene-referenced noise, based on f-stops (factors of 2 in exposure or illumination)
SNR (based on f-stops)	Scene-references Signal-to-Noise Ratio, based on f-stops. Note that there are many ways of defining SNR. Most are image (or pixel)-referenced. Results are different from S/N in the lower plot (which is more standard).

The scaling (minimum and maximum values) can be set in the dropdown menus to the right of the radio buttons. Most of the time these should be left at Auto. The values are inverted when the display is toggled between F-stop noise and SNR. For example, the minimum value of the x-axis in both Figures 2 and 3 can be set between -4 and -1.5.

Noise Detail (Fig. 2), Lower plot:  Pixel noise or SNR   Five types of display are available for the lower-left plot of the second figure. The first three are the same as the lower plot of Fig. 1. The effects of gradual illumination nonuniformities have been removed from the noise results. All displays are derived from Noise in pixels (the third selection below). In the notation below N_i is RMS noise and S_i is signal (pixel) level for patch i.

 

Noise (%) normalized to image density range = 1.5	100% * (Noise in pixels) divided by (the difference in pixel levels between light and dark patches that have a density difference of 1.5). This difference is close to the 1.45 White-Black difference in the ColorChecker. Useful because it references the noise to the scene: noise performance is not affected by camera contrast.
Noise (%) normalized to 255 pixels (max of 100)	100% * (Noise in pixels) / 255 = Ni/2.55. Values 0-100. This noise measurement will be worse for higher contrast cameras. (It is affected by the gamma encoding.)
Noise in pixels (maximum of 255)	Noise in pixels ( Ni ). Values 0-255.
Pixel S/N (Signal in patch/RMS noise) dimensionless	Noise in pixels ( Ni ). Values 0-255.
Pixel SNR (dB) (20*log10(S/N))	20 Log10(signal/noise) for each patch (where signal = pixel level). Units of dB (decibels). Doubling S/N increased dB measurement by 6.02.
	For this selection, SNR_BW is also displayed. SNR_BW is an average SNR based on White-Black patches (density difference = 1.5, close to the W-B difference in the ColorChecker). SNR_BW = 20 log10((SWHITE-SBLACK )/Nmid ), where Nmid is the noise in a middle gray patch (closest to nominal chart density = 0.7).

The Plot box on the right allows any of the figures to be turned off or on.

Output

The example was photographed with the Canon EOS-10D at ISO 100 and converted from RAW format using Capture One with default settings (no curves applied).

The results include tonal response and noise. Colorcheck produces a similar result, but with less tonal detail. Three figures are produced for color images; two for B&W.

First Figure The upper plot shows the average density of the grayscale patches (black curve) and first and second order density fits (dashed blue and green curves). The horizontal axis is the distance along the target. A portion of the patches themselves are shown just above the plot. The equations for the density fits are given in the Algorithm section, below. The second order fit closely matches the patches. The light cyan spikes are the differentiated and smoothed steps used to find the boundaries between zones. The lower plot shows the RMS noise for each patch: R, G, B, and Y (luminance). In this plot it is expressed as the percentage of the pixel level difference corresponding to a target density range of 1.5: the same as the white - black patches on the GretagMacbeth ColorChecker. For this camera, the pixel difference is 197.5. For this display, noise measured in pixels can be calculated by multiplying the percentage noise by 197.5. Noise can also be normalized to 255 pixels or expressed in pixels (maximum of 255). Options are described above. The high levels of Red and Blue noise in zones 3-6 may be due to imperfections in the target which are completely swamped by the noise at ISO 1600, shown below. Noise is largest in the dark areas because of gamma encoding: In the conversion from the sensor's linear output to the color space (sRGB, here) intended for viewing at gamma = 2.2, the dark areas are amplified more than the light areas and the noise is boosted as well.

See Noise in photographic images for a detailed explanation of noise: its appearance and measurement.

Second figure

The second figure (Noise detail) contains some of the most important results:

• the tonal response curve (displayed on a log scale, similar to film response curves),
• noise expressed as either a fraction of an f-stop (or EV or zone), or f-stop Signal-to-Noise Ratio (SNR = S/N = 1/f-stop noise),
• noise or SNR (signal-to-noise ratio), based on pixel levels, expressed in one of several ways, and
• dynamic range (for transmission step charts).

The horizontal axis for the three plots on the left is Log Exposure, which equals (minus) the nominal target density (0.05 - 1.95 for the Q-13 chart). This axis is reversed from Figure 1.

Second figure The x-axis (log exposure) is reversed in direction from Figure 1, which shows the chart zone (from light to dark) The f-stop Signal-to-Noise Ratio (SNR_f = 1/f-stop noise) can be displayed in the middle-left figure in place of RMS noise (in f-stops).

The middle left plot shows noise in f-stops (or EV), i.e., noise scaled to (divided by) the difference in pixel levels between f-stops, which decreases as brightness decreases. Noise measured this way is the inverse of the signal-to-noise ratio (N/S), which we will call SNR_f (to distinguish it from other SNR measurements). The darkest levels have the highest noise. This measurement corresponds to the workings of the eye and has important consequences for the calculation of practical dynamic range. The vertical axis is logarithmic to display low noise levels clearly. Dynamic range information is displayed when the range for a specific quality level (defined by maximum f-stop noise or minimum S/N) is within the range of the plot. It is omitted if noise or SNR_f is better than the specified level for all patches: it is not reported for quality levels lower than H in the above plot because the noise level never reaches 0.25 (the M-H level). The bottom left plot shows pixel noise or SNR scaled in one of several different ways. The above illustration shows pixel noise normalized to the difference in levels between lightest patch and the patch corresponding to a density of 1.5— close to the 1.45 density range as the GretagMacbeth Colorchecker. (The normalization is discussed in Fig. 1, above.)

EXIF data is shown in the middle right region. The lower right plot shows the noise spectrum. Digital camera images with excessive noise reduction will have an unusually rapid falloff of the noise spectrum.

The bottom left plot has several display options, listed above. Pixel SNR (dB) is shown on the left. SNR (dB) = 20 log10(Si/Ni), where Si is the signal (mean pixel level) of patch i and Ni is the noise (standard deviation of the pixel level, with slow variations removed) of patch i. SNR_BW is an average SNR based on white-black patches (density difference = 1.5) SNR_BW = 20 log10((SWHITE-SBLACK )/Nmid ), where Nmid is the noise in a middle level patch (closest to nominal chart density = 0.7).

Here are the results for ISO 1600. Tonal response is similar to ISO 100, but the noise is greatly increased— enough to swamp out any imperfections in the target. The noise is highly visible. It wouldn't be suitable for portraits and other high quality work, but it would be acceptable when a grainy "Tri-X" or "available light" look is desired. Neat image can do an excellent job of reducing it. The middle-left plot displays dynamic range for several quality levels, specified by the maximum noise within the range. The dynamic range for low quality (L; blue) has a maximum noise of 1 f-stop; the dynamic range for high quality (H; yellow) has a maximum noise of 0.1 f-stop.

Third Figure

The upper plot shows the density response (small circles) of the luminance (Y), red, green, and blue channels, as well as the first and second order fits (dotted and dashed gray lines). It resembles a set of traditional film density response curves. The R, G, and B curves are displayed in Imatest Pro only.

For reflection step charts only, the nominal exposure is shown as pink dots (•), and the exposure error in f-stops is displayed. Exposure error is less critical than for ColorCheck, where it strongly affects the a*b* color values: that's why it isn't displayed in the other figures. The equation for nominal exposure is similar to equation that calculates the grayscale pixel levels of the ColorChecker:

      pixel level = 255 * (10^–density/1.01)^(1/2.2)

The lower plot shows the slope of the density curve, which can also be regarded as the local contrast or gamma. This curve is the derivative of the density, d(Density) / d(Log Exposure).

Lens flare (stray light that bounces between lens elements and off the barrel) can be measured by photographing a reflective step chart (a Q-13 or Q-14) against dark and white backgrounds. When flare light is present (white background) it will reduce the slope of the density response in dark regions of the step chart (on the left of the plot). This curve makes it easy to measure the decrease in the slope.

Saving the results

When the Stepchart calculations are complete, the Save Stepchart Results? dialog box appears. It allows you to select figures to save and choose where to save them. The default is subdirectory Results in the data file directory. You can change to another existing directory, but new results directories must be created outside of Imatest— using a utility such as Windows Explorer. (This is a limitation of this version of Matlab.) The selections are saved between runs. You can examine the output figures before you check or uncheck the boxes. Figures, CSV, and XML data are saved in files whose names consist of a root file name with a suffix and extension. The root file name defaults to the image file name, but can be changed using the Results root file name box. Be sure to press enter. Checking Close figures after save is recommended for preventing a buildup of figures (which slows down most systems) in batch runs. After you click on Yes or No, the Imatest main window reappears.

Dynamic range of cameras and scanners

Dynamic range (DR) is the range of tones over which a camera responds. It is usually measured in f-stops, or equivalently, zones or EV. (It can also be measured in density units, where one density unit = 3.322 f-stops.) It can be specified in two ways:

• The total range. Stepchart is extremely sensitive at detecting a camera's total dynamic range, even when dark areas are extremely noisy or boundaries between chart zones become indistinct.
• A range of tones over which the RMS noise, measured in f-stops (the inverse of the signal-to-noise ratio, SNR), remains under a specified maximum value. The lower this value (the higher the minimum SNR), the better the image quality but the smaller the dynamic range. Noise tends to be worst in the darkest regions. Imatest calculates the dynamic range for several maximum RMS noise levels, from 0.1 f-stop (high image quality; SNR = 10) to 1 f-stop (low quality; SNR = 1).
  
  The dynamic range corresponding SNR = 1 (1 f-stop of noise) corresponds to the intent of the definition of ISO Dynamic range in section 6.3 of the ISO noise measurement standard: ISO 15739: Photography — Electronic still-picture imaging — Noise measurements. The Imatest measurement differs in several details from ISO 15739; hence the results cannot be expected to be identical. Imatest may well produce more accurate results because it measures DR directly from a transmission chart, rather than extrapolating results for a reflective chart with maximum density = 2.0.
  
  

  Change in dynamic range definition (Imatest 1.5.5, November 24, 2005) The definition of total dynamic range now includes indistinct zones (dark zones that the original Stepchart algorithm had difficulty detecting). This may cause some short-term confusion because Figure 2 will change: total DR will sometimes increase. But it better represents true camera performance.

A camera's (or scanner's) dynamic range can be accurately measured using a transmission step chart illuinated from behind by a lightbox. Reflection step charts such as the Kodak Q-13 or Q-14 are inadequate because their density range of around 1.9 is equivalent to 1.9 * 3.32 = 6.3 f-stops, well below that of digital cameras. Transmission step chart The table below lists several transmission step charts, all of which have a density range of at least 3 (10 f-stops). Kodak Photographic Step Tablets can be purchased calibrated or uncalibrated. Uncalibrated is usually sufficient. The Stouffer charts are attractively priced.

The Stouffer T4110 (13.3 f-stops range), Danes-Picta TS28D (13.6 f-stops range), or TE 241 (13.6 f-stops range) are recommended for digital SLRs, which can have dynamic ranges over 10 f-stops. The charts with Dmax = 3.05 = 10 f-stops are probably adequate for compact digital cameras with small pixel sizes (under 3 microns).

Lightbox

You'll need a lightbox that can evenly illuminate the transmission step chart. 8x10 inches is large enough. Avoid thin or "mini" models, which may not have even enough illumination. The Logan Tru-View 810/920 is inexpensive and quite adequate. Light boxes are widely available, though they may be hard to find on dealers' websites. On Adorama, click on Filing/Storage | Mounting & Viewing Equipment | Lightboxes & Loupes. On B&H, click on Projection & Viewing | General Presentation Equipment | Lightboxes, Loupes & Slide Viewers | Lightboxes & Slide Viewers | Lightboxes & Light Tables. T he Kyoritsu calibrated light sources, especially the pattern light boxes, available in the North America from C.R.I.S,, are worth checking out. Image Engineering/E sser Test Charts makes an excellent integrating sphere, ideal for use with the TE 241.

To measure dynamic range,

• Prepare a fixture for mounting the the step chart. It should be large enough to keep stray light out of the camera. Stray light can reduce the measured dynamic range; it should be avoided at all costs. I made a fixture out of scrap mat board held together with Scotch magic tape and Elmer's glue. (That old mainstay, duct tape, wasn't quite right for the job.)
• Place the fixture and step chart on top of the lightbox— or any other source of uniform diffuse light.
• Photograph the chart in a darkened room. No stray light should reach the front of the target; it will distort the results. The sourroundings of the chart should be kept as dark as possible to minimize flare light. An example is shown above. The density difference between the darker zones is not very visible, but it shows up clearly in the measurements. I used a Kodak step wedge (density steps of 0.15) that I purchased in 1969!
• Use your camera's histogram to determine the minimum exposure that saturates the lightest region of the chart. Overexposure (or underexposure) reduces the number of useful zones. The lightest region should have a relative pixel level of at least 0.98 (pixel level 250 or 255); otherwise the full dynamic range of the camera will not be detected. If the lightest zone is below this level, and a transmission chart is selected, a Dynamic range warning is issued.
• 
  For flatbed scanners with transparency units (TPUs, i.e., light sources for transparencies), you can simply lay the step chart down on the glass. Stray light shouldn't be an issue, though there is no harm in keeping it to a minimum. 35mm film scanners may be difficult to test since most can only scan 36mm segments. (Most transmission targets are longer.) For scanners specified as having Dmax greater than 3, the charts of choice are the Stouffer Transmission Step Wedge T4110 or Danes-Picta TS28D, which are too large to be tested easily with 35mm film scanners.
• Follow the remainder of instructions in Photographing the chart and running Stepchart, above. Be sure to select the correct chart type from the Stepchart input dialog box (right).

The Imatest algorithm for finding dynamic range is remarkably accurate. Imatest detects chart zones using the smallest density step that results in uniformly spaced detected zones (see Algorithm). For smaller steps, noise can be mistaken for zone boundaries. For larger steps, fewer zones are detected.

The dynamic range is the difference in density between the zone where the pixel level is 98% of its maximum value (250 for 24-bit color, where the maximum is 255), estimated by interpolation, and the darkest zone that meets the measurement criterion. The repeatability of this measurement is better than 1/4 f-stop.

Here is a result for the Canon EOS-10D at ISO 400, converted from RAW format with Capture One LE.

The total dynamic range of the Canon EOS-10D is 8.6 f-stops. Total dynamic range improves slightly for 48-bit TIFF conversion but very little for ISO 100. But the lower noise in 48-bit TIFF conversion and ISO 100 results in improved dynamic ranges for given quality levels. Cautionary note: This dynamic range is limited by the test chart (an old Kodak Step Tablet with a maximum range of 10 f-stops). A chart like the Stouffer T4110 might result in a higher number: possibly over 10 f-stops). The shape of the response curve is a strong function of the conversion software settings. The plot on the right is for Canon Zoom Browser with Contrast set to Low: the transfer curve is very different from Capture One LE, but the dynamic range is quite close— the result of slightly different noise reduction processing.

To convert dynamic range from f-stops into decibels (dB), the measurement normally given on sensor data sheets, multiply the dynamic range in f-stops by 6.02 (20 log₁₀(2)). The dynamic range for low quality (f-stop noise = 1; SNR = 1) corresponds most closely to the number on the data sheets. Measured dynamic range is normally somewhat lower than specified dynamic range because of lens flare and other factors.