Imatest™ Stepchart analyzes the tonal response, noiseand dynamic range of digital cameras and scanners using

  • reflection step charts such as the Kodak Q-13 and Q-14 Gray Scales (below, right), the ISO 15739 and ISO-14524 charts, and more.
  • transmission step charts from Imatest (the 36-patch Dynamic Chart shown below), Stouffer, Kodak, Danes-Picta, and Esser

Stepchart can also measure veiling glare (lens flare) using reflection step charts. Results are more detailed than those provided by Colorcheck. Transmission step charts are recommended for measuring dynamic range. Full instructions can be found on Using Stepchart. Noise is explained in Noise in photographic images.

Applied Image and ISO charts: Imatest Master only
QA-61
QA-61 ISO-
16067-1 Scanner
QA-62 target
QA-62
ST-51 target
ST-51
EIA Grayscale
ST-52 target
ST-52 ISO-14524
12-patch OECF target
ISO-15739 Noise target
ISO-15739
Noise target

 
20-patch OECF target crop
20-patch OECF
targets
ITE Grayscale target
ITE Grayscale

 dynamic_35_E_200W
36-Patch Dynamic
Range chart

 
To run Stepchart, photograph the chart, taking care to avoid glare (shiny reflections). Scale the image for 50 pixels per zone for greatest noise analysis accuracy; 20 pixels per zone is adequate for density analysis. Fewer are OK for tonal response curves. Load the image file, crop it (if needed), then specify the target density step and type (reflective or tramsmission). The default is a reflective target with density step = 0.1.

The two Figures below illustrate the results of analyzing a Q-13 image photographed with the Canon EOS-10D at ISO 100. A third figure (not shown here) displays density response for all channels (Y, R, G, and B) in color images.

The first figure

contains basic tonal response and noise measurements.

The horizontal axis for both plots is the chart zone— proportional to the distance along the chart. Density (–Log Exposure) increases by a fixed step (0.10 or 0.15, depending on the target) for each zone.

 

 

The upper plot
shows the normalized pixel level of the grayscale patches (black curve) and first and second order density fits (dashed blue and green curves). Gamma is derived from the first order fit. The acvite areas used for the analysis is shown as think pale pink bars.

The lower plot shows the RMS noise for each patch: R, G, B, and Y (luminance), expressed as the percentage of the pixel level difference corresponding to a target density range of 1.5: the same as the white – black patches on the GretagMacbeth ColorChecker. For this camera, the pixel difference is 197.5. Noise measured in pixels can be calculated by multiplying the percentage noise by 197.5. Noise can also be normalized to the maximum pixel level (255). The average noise for each channel (excluding the lightest and darkest zones) is displayed. See Noise in photographic images for a detailed explanation of noise.

 

 

The second Figure

contains the most important results:

  • the response curve displayed on a log scale, similar to film response curves,
  • noise expressed units of f-stops (the same as EV), explained below, or SNR (Signal-to-Noise Ratio), explained here,
  • noise in normalized pixel levels
  • dynamic range (for transmission step charts),
  • the noise spectrum.

The horizontal axis for the three plots is Log Exposure, which equals (–) the nominal target density (0.05 – 1.95 for the Q-13/Q-14). This axis is reversed from Figure 1.

The upper left plot shows the density response (gray squares), as well as the first and second order fits (dashed blue and green lines). It resembles a traditional film density response curve. Dynamic range is grayed out because the reflective Q-13 target has too small a dynamic range to measure a camera’s total dynamic range. See Dynamic range, below. This curve is sometimes called the Opto-Electronic Conversion Function (OECF).
The upper right box contains dynamic range results: total dynamic range and range for several quality levels, based on luminance (Y) noise. It is shown in gray when a reflective target is selected.

The middle left plot shows
noise in f-stops or EV, i.e., noise scaled to (divided by) the difference in pixel levels between f-stops, which decreases as brightness decreases. The darkest levels have the highest f-stop noise. This measurement corresponds to the response of the eye and has important consequences for the calculation of practical dynamic range. The vertical axis is logarithmic for clear display of low noise values.This plot can also be displayed as SNR (Signal-to-Noise Ratio; shown below), which is the inverse of f-stop noise.
EXIF data is shown in the
middle right region (JPEG files only).
The bottom left plot
shows the noise scaled to the difference in pixel levels between the maximum density level and the patch corresponding to a density of 1.5— the same density range as the GretagMacbeth Colorchecker. Several additional options are available for displaying noise or SNR (Signal-to-Noise Ratio).
The lower right plot
shows the noise spectrum. Digital camera images with excessive noise reduction will have an unusually rapid falloff of the noise spectrum. When combined with the eye’s contrast sensitivity function (CSF), the noise spectrum can be used to calculate the visiblity of noise.
SNR (S/N = 1/f-stop noise), in middle-left plot
in place of f-stop noise.
SNR plot (in place of f-stop noise; middle left)
 

Why measure noise in f-stops? Because the human eye responds to relative luminance differences. That’s why we think of exposure in terms of zones, f-stops, or EV (exposure value),
where a change of one unit corresponds to halving or doubling
the exposure.

The eye’s relative sensitivity is expressed by the Weber-Fechner
law,

ΔL ≈ 0.01 L   –or–   ΔL/L ≈ 0.01

where ΔL is the smallest luminance difference the eye can distinguish. (This equation is approximate; effective ΔL tends to be larger in dark areas of scenes and prints due to visual interference from bright areas.)

f-stop noise

Expressing noise in relative luminance units, such as f-stops, corresponds more closely to the eye’s response than standard pixel or voltage units. Noise in f-stops is obtained by dividing the noise in pixels by the number of pixels per f-stop. (I use “f-stop” rather than “zone” or “EV” out of habit; any of them are OK.)

noise in f-stops = noise in pixels / (d(pixel)/d(f-stop)) = 1/SNR
where 
d(pixel)/d(f-stop) is the derivative of the pixel level with respect to luminance measured in f-stops (~log2(luminance) ).  SNR is the Signal-to-Noise Ratio.

The above-right image illustrates how the pixel spacing between f-stops (and hence d(pixel)/d(f-stop)) decreases with decreasing brightness. This causes f-stop noise to increase with decreasing brightness, visible in the middle-left plot of the second figure, above.

Since luminance noise (measured in f-stops) is referenced to relative scene luminance,
independently of electronic processing or pixel levels, it is a universal measurement
that can be used to compare digital sensor quality when sensor RAW data is unavailable.

Dynamic range

The new Dynamic Range module calculates dynamic range from several reflective stepchart images, which are easier to work with than transmission step charts.

Dynamic
range
is the range of brightness over which a camera responds. It is
usually measured in f-stops, or equivalently, zones or EV. It can
be specified in two ways:

  • The total range. Stepchart measures a camera’s total dynamic range, including noisy dark areas.
  • A range of tones over which the RMS noise, measured in f-stops, is under a maximum specified value. The lower the maximum noise value, the better the image quality, but the smaller the dynamic range. Noise tends to be worst in
    the darkest regions. Imatest calculates the dynamic range for several maximum noise levels, from RMS noise = 0.1 f-stop (high image quality) to 1 f-stop (relatively low quality).

Dynamic range is measured using transmission step charts, which have density ranges of at least 3.0: a 1000:1 ratio (10 f-stops). Reflective charts such as the Q-13 have a density range of only around 1.90; an 80:1 ratio (6.3 f-stops), insufficient for digital cameras. These charts are recommended for digital SLRs, which can have over 10 f-stops of total dynamic range.

The dynamic range is the difference in density between the zone where the pixel level is 98% of its maximum value (250 our of 255 for 24-bit color), estimated by interpolation, and the darkest zone that meets the measurement criterion. The repeatability of this measurement is around 1/3 f-stop.

The figure below illustrates results for the Canon EOS-10D, taken from a JPEG image acquired at ISO 400 and converted with Canon Zoom Browser set for low contrast. A Kodak step tablet (density from 0.05 to 3.05 in steps of 0.15) was used. The total dynamic range is 8.58 f-stops (probably limited by the target; the Stouffer T4110 or Danes-Picta TS28D would have done a better job). It is displayed at the bottom of the upper-left plot (Density response). Total dynamic range changes little for 48-bit TIFF conversion or ISO 100. But 48-bit TIFF conversion has lower noise, hence higher dynamic range at any given quality level.

A medium quality image can ge achieved over a range of 8.21 f-stops out of a total dynamic range of 8.58 f-stops. When a high quality image is required (maximum noise = 0.1 f-stops), the dynamic range is reduced to 5.97 f-stops (indicated by the yellow line on the middle left plot). The best slide films have a total dynamic range of 5 to 6 f-stops.

Dynamic range detail for transmission stepchart
The second plot contains key Stepchart results.

The shape of the response curve depends strongly on the raw conversion software and settings.

Compact digital cameras have much higher noise levels, hence lower useful dynamic range, even though their total dynamic
range may be quite large. 

Scanner results

Here are the results of scanning the Kodak step
tablet with the Epson 3200 scanner (with transparency unit) set for negatives.

A few observations on the scanner results:

  • All 20 steps of the Kodak step tablet were detected. The total dynamic range is greater than 3 density units (the 10 f-stop range of the tablet). To determine the true total dynamic range we would need a chart with a higher density range, such as the Imatest 36-patch Dynamic Range chart or the Stouffer T4110 step wedge, which has a maximum density of 4.
  • The response closely follows an exponential curve with gamma = 0.506. No “S” curve has been superposed.
  • The practical dynamic range is limited by noise. It is 9.62 f-stops (2.9 density units) for a medium quality image.
  • The flat noise spectrum indicates that no software noise reduction has been applied. Most digital cameras have rolloffs in their noise spectra due to Bayer interpolation and noise reduction.

Page updated: 2013-03-28