 |
| |
Imatest™ Stepchart
analyzes the tonal response, noise,
and dynamic range of digital
cameras and scanners using
Stepchart can also measure veiling glare (lens flare) using reflection step charts. Results are more detailed than those provided by Colorcheck. Transmission step charts are recommended for measuring dynamic range. Full instructions can be found on Using Stepchart. Noise is explained in Noise in photographic images.
| To run Stepchart, photograph the chart, taking care to avoid glare (shiny reflections). Scale the image for 50 pixels per zone for greatest noise analysis accuracy; 20 pixels per zone is adequate in most cases. Fewer are OK for tonal response curves. Load the image file, crop it (if needed), then specify the target density step and type (reflective or tramsmission). The default is a reflective target with density step = 0.1. |
 |
The two Figures below illustrate the results of analyzing a Q-13 image photographed with the Canon EOS-10D at ISO 100. A third figure (not shown here) displays density response for all channels (Y, R, G, and B) in color images.
The first figure
contains basic tonal response and noise measurements.
The horizontal axis for both plots is the chart zone— proportional to the distance along the chart. Density (–Log Exposure) increases by a fixed step (0.10 or 0.15, depending on the target) for each zone.
The upper plot
shows the normalized pixel level of the
grayscale patches (black curve) and first and second order
density fits (dashed blue and green
curves). Gamma is derived from the first order fit. The acvite areas used for the analysis is shown as think pale pink bars.
The lower plot shows the RMS noise for
each patch:
R, G, B, and Y (luminance), expressed as the percentage of the pixel level difference corresponding to a target density range of 1.5: the same
as
the white - black patches on the GretagMacbeth ColorChecker. For this camera, the pixel difference is 197.5. Noise measured in pixels can be calculated by multiplying the percentage noise by 197.5. Noise can also be normalized to the maximum pixel level (255). The average noise for each channel (excluding the lightest and darkest zones) is displayed. See Noise in photographic images for a detailed explanation of noise.
|
 |
The second
Figure
contains the
most important results:
- the response curve displayed on a log
scale, similar to film response curves,
- noise expressed units of f-stops (the same as EV), explained below, or SNR (Signal-to-Noise Ratio), explained here,
- noise in normalized pixel levels
- dynamic range (for
transmission step charts),
- the noise spectrum.
The horizontal
axis for the three plots is Log Exposure, which equals
(–) the nominal target
density
(0.05 - 1.95 for the Q-13/Q-14). This axis is reversed from Figure 1.
The upper left plot shows the density response (gray squares), as well as the first and second order fits (dashed blue and green lines). It resembles a traditional film density response curve. Dynamic range is grayed out because the reflective Q-13 target has too small a dynamic range to measure a camera's total dynamic range. See Dynamic range, below. This curve is sometimes called the Opto-Electronic Conversion Function (OECF). |
The upper right box contains dynamic range results: total dynamic range and range for several quality levels, based on luminance (Y) noise. It is shown in gray when a reflective target is selected. |
The middle left plot shows
noise in f-stops or EV, i.e., noise scaled to (divided by) the
difference in pixel levels between f-stops, which decreases as
brightness decreases. The darkest levels have the highest f-stop noise.
This
measurement corresponds to the response of the eye and has
important consequences for the calculation of practical dynamic range.
The vertical axis is logarithmic for clear display of low noise values.
This plot can also be displayed as SNR (Signal-to-Noise Ratio; shown below), which is the inverse of f-stop noise.
|
EXIF data is shown in the
middle right region (JPEG files only).
|
The bottom left plot
shows the noise scaled to the difference in pixel levels between the
maximum density level and the patch corresponding to a density of 1.5—
the same density range as the GretagMacbeth Colorchecker. Several additional options are available for displaying noise or SNR (Signal-to-Noise Ratio).
|
The lower right plot
shows the noise
spectrum. Digital
camera images with excessive noise reduction will have an unusually
rapid
falloff of the noise spectrum. When combined with the eye's contrast sensitivity function (CSF), the noise spectrum can be used to calculate the visiblity of noise. |
SNR (S/N = 1/f-stop noise),
in middle-left plot
in
place of f-stop noise.
|
 |
Why measure noise in f-stops?
Because the human eye responds to relative luminance differences. That's why we think of exposure in terms of zones, f-stops, or EV (exposure value),
where a change of one unit corresponds to halving or doubling
the exposure.
The eye's relative sensitivity is expressed by the Weber-Fechner
law,
ΔL ≈ 0.01 L –or– ΔL/L ≈ 0.01
where ΔL is the smallest luminance difference the eye can distinguish. (This equation is approximate; effective ΔL tends to be larger in dark areas of scenes and prints due to visual interference from bright areas.)
|
 |
Expressing noise in relative luminance units, such as f-stops, corresponds more closely to the eye's response than standard pixel or voltage units. Noise in f-stops is obtained by dividing the noise in pixels
by the number of pixels per
f-stop. (I use "f-stop" rather than "zone" or "EV" out of habit; any of them are OK.)
noise in f-stops = noise in pixels / (d(pixel)/d(f-stop)) = 1/SNR
where d(pixel)/d(f-stop) is the derivative of the pixel level with respect to luminance measured in f-stops (~log2(luminance) ).
SNR is the Signal-to-Noise Ratio. |
The above-right image illustrates how the pixel spacing between f-stops (and hence d(pixel)/d(f-stop)) decreases with decreasing brightness. This causes f-stop noise to increase with decreasing brightness, visible in the middle-left plot of the second figure, above.
Since
luminance noise (measured in f-stops) is referenced to relative scene luminance,
independently of electronic processing or pixel levels, it is a universal measurement
that can be used to compare digital sensor quality when sensor RAW data is unavailable. |
Dynamic
range
| The new Dynamic Range module calculates dynamic range from several reflective stepchart images, which are easier to work with than transmission step charts. |
|
Dynamic
range is the range of brightness over which a camera responds. It is
usually measured in f-stops, or equivalently, zones or EV. It can
be specified in two ways:
- The total range.
Stepchart measures a camera's total dynamic range, including noisy dark areas.
- A range of tones over which the RMS noise, measured in f-stops, is under a maximum
specified value. The lower the maximum noise value, the better the
image
quality, but the smaller the dynamic range. Noise tends to be worst in
the darkest regions. Imatest calculates the
dynamic range for several maximum
noise levels, from RMS noise = 0.1 f-stop (high image
quality) to 1 f-stop (relatively low quality).
Change in dynamic range definition (Imatest 1.5.5, November 24, 2005)
The definition of total dynamic range now includes indistinct zones (dark zones that the original Stepchart algorithm had difficulty detecting). This may cause some short-term confusion because Figure 2 will change: total DR will sometimes increase. But it better represents true camera performance. |
|
Dynamic
range is measured using transmission
step charts, which have density
ranges of at least 3.0: a 1000:1 ratio (10 f-stops). Reflective
charts such as
the
Q-13 have a density range of only around 1.90; an 80:1 ratio (6.3
f-stops), insufficient for digital cameras. Two charts are recommended for digital SLRs, which can have over 10 f-stops of total dynamic range.
- The Stouffer T4110 transmission step wedge, which has a maximum density of 4.05 (41 steps, 0.1 density increment, 13.3 f-stops dynamic range)
- The Danes-Picta TS28D (on their Digital Imaging page), which has a maximum density of 4.2 ( 28 steps, 0.15 density increment, 13.6 f-stops dynamic range)
The dynamic range is the difference in density between the zone where the pixel level is 98% of its maximum value (250 our of 255 for 24-bit color), estimated by interpolation, and the darkest zone that meets the measurement criterion. The repeatability of this measurement is around 1/3 f-stop.
The
figure below illustrates results
for the Canon EOS-10D,
taken from a JPEG image acquired
at ISO 400 and converted with Canon Zoom Browser set for low contrast.
A Kodak step tablet (density from 0.05 to 3.05 in steps of 0.15) was
used.
The total dynamic
range is 8.58 f-stops (probably limited by the target; the Stouffer T4110 or Danes-Picta TS28D would have done a better job). It is displayed at the bottom of the upper-left plot (Density response). Total dynamic
range changes little for 48-bit TIFF conversion or ISO 100. But 48-bit
TIFF conversion has lower noise, hence higher dynamic range at any given
quality level.
A medium quality image can ge achieved over a range of 8.21 f-stops out of a total dynamic range of 8.58 f-stops. When a high quality image is required (maximum noise = 0.1 f-stops),
the dynamic range is reduced to 5.97 f-stops (indicated by the yellow line on the middle left plot). The best slide
films have a total dynamic range of 5 to 6 f-stops.
The
second plot contains all important Stepchart results.
The shape of the response curve
depends
strongly on the conversion software and settings.
Compact digital
cameras have much higher noise levels, hence lower useful dynamic
range, even though their total dynamic
range may be quite
large. Roger N. Clark, a space scientist and avid photographer has done a study of dynamic range. He finds digital (SLRs) to be superior to film.
Scanner results
Here are the results of scanning the Kodak step
tablet with the Epson 3200 scanner (with transparency unit) set for negatives.
A few
observations on the scanner results:
- All 20 steps of the Kodak step tablet were detected. The total
dynamic range is greater than 3 density units (the 10 f-stop range
of the tablet). To determine the true total dynamic range we would need the Stouffer T4110 step wedge,
which has a maximum density of 4.
- The response closely follows an exponential curve with gamma =
0.506. No "S" curve has been superposed.
- The practical dynamic range is limited by noise. It is 9.62
f-stops (2.9 density units) for a medium quality image.
- The flat noise spectrum indicates that no software noise
reduction has been applied. Most digital cameras have rolloffs in their noise spectra due to Bayer interpolation and noise reduction.
|
| |
