Running Imatest SFR

Imatest is opened by double-clicking the Imatest icon  on

After several seconds, the Imatest main window, shown below, opens. Then click on one of the SFR buttons on the upper left. The SFR: Current file button is grayed out (inoperative) for the first run because no file has yet been read. It can be used in succeeding runs to save time when analyzing different regions, though multiple ROI (region of interest) runs are more efficient.

Imatest main window

Several cropped slanted-edge JPEG images are included in the images subdirectory of the Imatest directory. Complete ISO12233 charts are not included with Imatest because they are large (usually over 2 megabytes). They can be downloaded from dpreview.com reviews, typically in the last page titled "Compared to...," and from Imaging-resource.com reviews, typically on the page labeled "sample images."

Selecting the image file

When you click SFR: New file, the window on the right appears, requesting the image file name. The folder saved from the previous run appears in the Look in: box on the top. You are free to change it. An image file for the Canon EOS-10D is highlighted. Double-click (or click and press Open ) to read the file name. Large files can take several seconds to load. Imatest remembers if you change the folder (for each module, individually).

Imagest SFR: Open slanted-edge input file dialog box

Selecting the ROI (Region of Interest)

If the image has the same pixel dimensions as the last image in the previous run, a dialog box shown below asks you if you want to repeat the same ROIs (regions of interest) as the previous image.

If you answer No or if the image has a different size, the image shown on the right (reduced) is displayed. The instructions are, Select ROI by clicking and dragging, or clicking outside image. Click on one corner of the intended region, drag the mouse to the other corner, then release the mouse button. Click outside the image to select the entire image. If the selected area is under 2500 pixels, over 200,000 pixels, or appears to be inappropriate, you'll be asked to repeat the selection. If it is seems OK, a red outline will appear around the selection, an enlarged image of the selection will be displayed, and you'll be asked, Is the selected region OK? (below). You have five choices.

ISO 12233 chart with crop indicated by red rectangle

Note: the Zoom feature of Matlab Figures does not work here. Imatest mistakely assumes you are trying to select the entire image. To enlarge the image (to view the crop area more clearly), maximize it by clicking on the square next to the upper right corner of the Window.

Yes, Continue The selected ROI is correct; no more ROIs are to be selected. Continue with SFR calculations in normal mode: You will be asked for additional input data, etc.
Yes, Continue in Express mode The selected ROI is correct; no more ROIs are to be selected. Continue with SFR calculations in Express mode: You will not be asked for additional input data or for Save options. Saved or default settings will be used.
Yes, select another region The selected ROI is correct. Select another ROI. For multiple ROIs additional Figures will display performance as a function of distance from the image center.
No, try again The selected ROI is not correct. Try again.
Cancel Cancel the SFR run. Return to the Imatest main window.

Cropping recommendations

Length should be between 80 and 500 pixels. Little is gained for length over 300 pixels. Width should be at least 60 pixels.

My typical crops are between around 120x80 and 240x140 pixels.

Widths for light and dark zones (shown on the right) should be at least 20 pixels. 30 is preferred. Little is gained for zone widths over 80 pixels.

Additional SFR input

Imatest SFR input dialog box The Imatest SFR data window, shown on the right, appears after the Region of Interest (ROI) has been selected, unless Express mode has been chosen. It appears only once during multiple ROI runs. Succeeding runs have the same entries except for Crop location. All input fields are optional. Most of the time you can simply click OK (the button on the lower right) to continue.

Title defaults to the input file name. You may leave it unchanged, replace it, or add descriptive information— as you please.

Plot controls which results are plotted. Plot settings are saved between runs. Only the Cycles per (pixel) and Chromatic Aberration plots are checked by default. The LW/PH plot is unchecked (by default) because it is largely redundant.

The boxes to the right of "Cycles per" control  the x-axis display. You can choose between Cycles per pixel (the default), Cycles per inch, or Cycles per mm. If you choose Cycles per inch or mm, you must enter a number for the pixel size— either in pixels per inch, pixels per mm, or microns per pixel. An example is shown below for a plot in Cycles per mm for the Canon EOS-10D, which has a pixel spacing of 7.4 microns. If you omit this, the x-axis will be displayed in Cycles per pixel.

If both Chromatic Aberration and Noise spectrum and Shannon capacity are checked, the two plots share the same figure: CA on top and Noise/Shannon capacity at the bottom. Noise spectrum and Shannon capacity is unchecked by default because there results are difficult for most users to interpret.

Picture Width and Height   defaults to the width and height of the input image in pixels, assuming landscape format, where height < width. If the input image has been cropped, or if it doesn't represent the entire camera image, Picture Height (and Width) should be entered manually.

Line Width per Picture Height (LW/PH) results are correct only if the Picture Height represents the entire uncropped frame. For example, Picture Height should be set to 2048 for the Canon EOS-10D and 1944 for the Canon G5 (the cameras in the cropped sample files), etc.

For film and scanner users
More film/scanner users than I expected are using Imatest. And film users prefer results in Line Pairs per mm (LP/mm). You can select cycles (line pairs) per inch or millimeter in the Plot box. Scanned images tend to be cropped by the time they are entered into Imatest because complete images are huge. (Try running the numbers for a 4,000 dpi scanner.) In this case, you must manually enter the picture height in pixels to get LW/PH. The best way is to use the equation,
     Picture height (pixels) = picture height (mm) * pixels/mm.
For example, a 4,000 dpi scanner has 4000/25.4 = 157.5 pixels/mm. For 35mm film, the picture height is 24 mm = 3780 pixels. With medium format cameras, the picture height is somewhat smaller than the nominal size. It is 41.5 mm for the 645 format and 56mm for the 6x6 and 6x7 (cm) formats. Picture Width is optional, but it's a good idea to enter it. It is calculated in the same way.

Channel is normally left at it's default value of Y for the luminance channel, where Y = 0.3*R + 0.59*G + 0.11*B. In rare instances the R, G, and B color channels might be of interest.

Gamma is used to linearize the input data (explained below). It defaults to 0.5 = 1/2, which is typical of digital cameras, but is affected by camera (or RAW converter) contrast settings. It may be quite different for scanned film images.

Since SFR sharpness measurements are moderately sensitive to the Gamma setting (a 10% error in gamma results in a 2.5% error in MTF50), it's a good idea to run Colorcheck or Q-13 stepchart to determine the correct value of Gamma.

Gamma
Gamma is the exponent of the equation that relates image pixel level to luminance. For a monitor or print,
Output luminance = (pixel level)gamma
When the raw output of the image sensor, which is linear, is converted to image file pixels with a standard color space, the approximate inverse of the above operation is applied.
pixel level = (RAW pixel level)gamma_camera ~= exposuregamma_camera
The total system gamma is gamma * gamma_camera.  Standard values of (output) gamma are 1.8 for color spaces used in the Macintosh and 2.2 for color spaces used in Windows, such as sRGB (the default when color space is unspecified) and Adobe RGB (1998).

In characteristic curves for film and paper, which use logarithmic scales (e.g., density (log10(absorbed light) vs. log10(exposure)), gamma is the average slope (excluding the "toe" and "shoulder" regions near the ends of the curve), i.e.,

gamma is contrast.

See Kodak's definition in Sensitometric and Image-Structure Data.


To obtain the correct MTF, Imatest must linearize the pixel levels; the camera's gamma must be removed. That is the purpose of Gamma in the SFR input dialog box, which defaults to 1/2, typical of a wide range of digital cameras. It can, however, vary considerably with camera and RAW converter settings, most notably contrast.

Characteristic curves for the Canon EOS-10D with three RAW converter settings are shown on the right. Gamma deviates considerably from 0.5. Gamma = 0.679 could result in a 9% MTF50 error. For best accuracy I recommend measuring gamma using Colorcheck or Q-13 Stepchart, which provides slightly more detailed results.

Confusion factor: Digital cameras rarely apply an exact gamma curve: A "tone reproduction curve" (an "S" curve) is frequently superposed on the gamma curve to extend dynamic range while maintaining visual contrast. This reduces contrast in highlights and (sometimes)deep shadows while maintaining it in middle tones. You can see it in curves 1 and 3, on the right. For this reason, "Linear response" (where no S-curves is applied on top of the gamma curve) is recommended for SFR measurements.

The transfer function may also be
adaptive: camera gamma may be higher for low contrast scenes than for contrasty scenes. This can cause headaches with SFR measurements. But it's not a bad idea generally; it's quite similar to the development adjustments (N-1, N, N+1, etc.) in Ansel Adams' zone system. For this reason it's not a bad idea to place a Q-13 or Q-14 chart near the slanted edges.

To learn more about gamma, read Tonal quality and dynamic range in digital cameras and Monitor calibration.
In characteristic curves, which use Log scales, gamma is defined as the average slope of the transfer function, excluding the extreme highlights and shadows.

The three curves below, produced by Q-13 Stepchart for the Canon EOS-10D, show how Gamma varies with RAW converter settings.
Capture One LE with Film Curve
1. Capture One LE set to Film standard (the default). Gamma = 0.679.
Capture One LE with Linear Curve
2. Capture One LE set to Linear response. Gamma = 0.508. Recommended for SFR runs.
Canon FVU with Standard contrast
3. Canon FVU set to Standard contrast. Gamma = 0.642.

Standardized sharpening radius controls the radius of the sharpening used to compensate for different amounts of sharpening in different digital cameras. Its default value of 2 (pixels) is a typical sharpening radius for compact digital cameras, which have tiny pixels. The sharpening radius is sometimes smaller for digital SLRs, which have larger pixels, and can be different for different RAW converters.  Although a value of 2 is recommended for comparing cameras, you may occasionally want to experiment with the sharpening radius. Results with standardized sharpening should not be used for comparing different lenses on the same camera. For more detail, see What is standardized sharpening, and why is it needed for comparing cameras?

Crop location is the position of the center of the selected ROI (Region of interest), expressed as the percentage the distance from the image center to the corner. The orientation of the ROI with respect to the image center is included. It is blank if the input image has not been cropped. Crop location can be entered manually. This is only recommended when that image has been cropped outside of Imatest SFR.

Additional parameters (all optional) for Excel .CSV output  EXIF data is entered, if available. These settings are optional but useful when several tests are run for different lenses, focal lengths, apertures, or other settings.

Save settings  If checked, most parameters are retained for additional runs. This is useful when the same camera and lens is tested at a number of apertures and focal lengths. Saved settings are overridden by EXIF data, if it is present. Some settings can change between runs: Description is always reset to the input file name and Picture Height and Width are calculated from the input file. Location is calculated for each Region of Interest (ROI).


When entries are complete, click OK. A Calculating... box appears to let you know that calculations are proceeding. Results appear in individual windows called Figures (Matlab's standard method of displaying plots). Figures can be examined, resized, maximized, and closed at will. Their contents are described in the pages on Imatest SFR results: MTF (Sharpness) plot, Chromatic Aberration, Noise, and Shannon Capacity plot, and Multiple ROI (Region of Interest) plot.
Two precautions when working with figures
Too many open Figures
Figures can proliferate if you do a number of runs, especially SFR runs with multiple regions, and system performance suffers if too many Figures are open. You will need to manage them. Figures can be closed individually by clicking X on the upper right of the Figure or by any of the usual Windows techniques. You can close them all by clicking Close figures in the Imatest main window.
Clicking on Figures during calculations
can confuse Matlab. Plots can appear on the wrong figure (usually distorted) or disappear altogether. Wait until all calculations are complete until the Save or Imatest main window appears before clicking on any Figures.

Saving the results

At the completion of the SFR calculations the Save results? dialog box appears, unless Express mode has been selected. It allows you to choose which results to save and where to save them. The default is subdirectory Results of the data file directory. You can change to another existing directory, but new results directories must first be created outside of Imatest using a utility such as Windows Explorer. (This is a limitation of this version of Matlab.)

When multiple ROIs are selected, the Save results? dialog box appears only after the first set of calculations. The remaining calculations use the same Save settings. Save results? is omitted entirely in an Express run for repeated images.

The first three check boxes are for the figures. The last three are for Excel .CSV results. You can examine the output figures before you check or uncheck the boxes. After you click on Yes or No, the Imatest main window reappears. These selections are saved between runs.

Result file names—   The roots of the file names are the same as the image file name. The channel (Y, R, G, or B) is included in the file name. If a Region of Interest has been selected from a complete digital camera image, information about the location of the ROI is included in the file name following the channel. For example, if the center of the ROI is above-right of the image, 20% of the distance from the center to the corner, the characters AR20 are included in the file name.

Output files for  canon_eos10d.jpg  (Y-channel)
(in subdirectory Results)
Figures (.PNG image files)
canon_eos10d_YA17_cpp.png Plot with x-axis in cycles/pixel (c/p), Y-channel,17% of the way to the corner above the center of the uncropped image.
canon_eos10d_YA17_lwph.png Plot with x-axis in Line Widths per Picture Height (LW/PH).
canon_eos10d_YA17_ca.png Plot of Chromatic Aberration, with noise statistics and Shannon information capacity.
Excel .CSV (ASCII text files)
SFR_cypx.csv (Database file for appending results: name does not change). Displays 10-90% rise in pixels and MTF in cycles/pixel (C/P). Described below.
SFR_LWPH.csv (Database file for appending results: name does not change). Displays 10-90% rise in number/Picture Height (/PH) and MTF in Line Widths per Picture Height (LW/PH). Described below.
canon_eos10d_YA17_MTF.csv Excel .CSV file of MTF results for this run. The first row has the headers: cy/pxl, LW/PH, MTF, MTF(corr). The remaining lines contain the data. Can easily be plotted or combined with data from other files.

The third file is identical to the Imatest Figure (below), except that the background is white instead of gray.

These results are explained in Sharpness: What is it and how is it measured?

Repeated runs

If you have already run SFR and you click on either of the Run SFR buttons, and the new (or repeated) file has the same pixel dimensions as the previous file, an image that includes the selected ROIs appears on the left of the screen and you'll be asked, "Do you want the same ROIs as the previous image?" A portion of the previous image and the ROI repeat box, which contains the four answers, are shown in the image on the right.

The meaning of the answers is self-evident except for "Yes, Express mode."This button is the same as "Yes," except that the Input dialog box and the two Save dialog boxes (for individual Figures and for the multiple ROI figures at the end) are omitted. Default settings or settings saved from previous runs are used. The clipping warning boxes are suppressed. This speeds up repeated runs; for example, it's very handy when you are testing a lens at several apertures (f/2.8, f/4, f/5.6, etc.).

Excel .CSV (Comma Separated Values) output

Imatest SFR creates or updates output files for use with Microsoft Excel. The files are in CSV (comma-separated values) format, and are written to the results directory. CSV files are ASCII text files that look pretty ugly when viewed in a text editor:

File        ,Date/time        ,PH,Ch,H/V,10-90U,10-90C,Over-,Over-,MTF50U,MTF50C,MTF,Camera,Lens,FL,f-stop,Loc,Misc.
,,,,,/PH,/PH,shoot%,sharp%,LW/PH,LW/PH,Nyq,,,(mm),,,settings
canon_eos10d_sfr.jpg,2004-03-19 22:21:34, 2048,Y,H, 1422, 1447, 19.5, -0.7, 1334, 1340,0.154,,,,,
canon_g5_sfr.jpg,2004-03-19 22:24:30, 1955,Y,H, 1973, 1301, 48.0, 21.3, 1488, 1359,0.268,Canon G5,,14,5.6,ctr,
sigma_sd9_sfr.jpg,2004-03-19 22:27:55, 1504,Y,H, 1432, 1676,  2.4, -7.7, 1479, 1479,0.494,,,,,
sigma_sd10_sfr.jpg,2004-03-19 22:28:32, 1504,Y,H, 1563, 1628, 11.9, -2.0, 1586, 1587,0.554,,,,,

But they look pretty decent when entered into Excel.

CSV files can be edited with any of the usual text editors: Notepad, Write (WordPad), etc, but it makes far more sense to edit them in Excel, where columns as well as rows can be moved and/or deleted. Some fields are truncated in the above display, and Date/time is displayed as a sequence of pound signs (#####...).

The format can be changed by dragging the boundaries between cells on the header row (A, B, C, ...) and by selecting the first two rows and setting the text to Bold. This makes the output look better. The modified file can be saved with formatting as an Excel Worksheet (XLS) file. This, of course, is just the beginning.

It's easy to customize the Excel spreadsheet to your liking. For example, suppose you want to make a concise chart. You can delete Date/time (Row B; useful when you're testing but not so interesting later) and Channel (all Y = luminance). You can add a blank line under the title, then you can select the data (rows A4 through J7 in the image below) and sort on any value you choose. Corrected MTF50 (column I) has been sorted in descending order. Modified worksheets should be saved in XLS format, which maintains formatting.

There are no limits. But I've said enough. ( I'm not an Excel expert! )

Database files

Two .CSV files are used as a database for storing data from SFR runs. If possible, data from the current run is appended to these files. A third file for storing MTF data is described below.

When SFR is run, it looks for the two files in the same folder as the slanted-edge input image. If it doesn't find them, it creates them, writes the header lines, then writes a line of data for the run. If it finds them, it appends data for the run to the files. You can rapidly build a spreadsheet by doing repeated runs from files in the same folder. The following table contains the entries. Camera and the entries that follow are all optional. With the exception of Loc, which is calculated when the ROI is manually selected, they are only added if you answer y to the question, Additional data for Excel file?, and enter them manually.

File File name. Should be concise and descriptive.
Date/time Date and time in sortable format. Displays differently in Excel.
PH Picture Height (in SFR_lwph.csv only).
Ch Channel. Y (luminance) [default], R, G, or B.
H/V Horizontal or Vertical measurements.
(A vertical chart gives you horizontal rise and MTF, etc.)
10-90U 10-90% rise distance, uncorrected.
10-90C 10-90% rise distance, corrected with standard sharpening.
Overshoot% Overshoot of the edge.
Oversharp% Oversharpening: the amount of sharpening relative to the standard sharpening.
If negative, the image is undersharpened.
MTF50U MTF50 (frequency where MTF = 50%), uncorrected.
MTF50C MTF50 (frequency where MTF = 50%), corrected with standard sharpening.
MTF Nyq MTF at Nyquist frequency. Indicates likelihood of aliasing problems.
Camera Camera name. This entry and those that follow are manually-entered and optional.
Lens Lens name. Only for DSLRs with interchangeable lenses.
FL (mm) Focal length in mm.
f-stop Aperture
Loc Location of image (center, edge, corner, etc.)
Misc. settings Anything else: RAW converter, Sharpening setting, etc.

Tips

Other .CSV files
 

Imatest writes a file that contains .CSV data for MTF. The file name is [root name]_MTF.csv. The format is shown on the right. The columns are

  1. Spatial frequency in the units chosen for the "Cycles per" plot (cycles/pixel, cycles/mm, or cycles/in), 
  2. Spatial frequency in LW/PH, 
  3. MTF, and 
  4. MTF (corrected, i.e., with standardized sharpening).
 
cy/pxl, LW/PH,    MTF, MTF(corr)
0.0000, 0.00, 1.0000, 1.0000
0.0059, 28.97, 0.9934, 0.9931
0.0118, 57.94, 0.9931, 0.9922