Running SFRplus
Imatest SFRplus performs highly automated measurements of several key image quality factors using a specially-designed test chart. This document shows how to run SFRplus in Rescharts and how to save settings for automated runs. Part 1 introduced SFRplus and explained how to obtain and photograph the chart. Part 3 illustrates the results.
New in Imatest 3.6 The Lens-style MTF plot is similar to MTF displays on the Canon, Nikon, and Zeiss websites. New in Imatest 3.5.1 Several geometrical alignment results are displayed in the Image & geometry plot. New in Imatest 3.4.1 Calculations are now much faster. A Speedup checkbox has been added to the SFRplus setup window to further increase speed by eliminating some calculations that most users don't need (SQF, noise histograms, clipping check, etc.). |
Open Imatest by double-clicking the Imatest icon 
on
- the Desktop,
- the Windows Start menu,
- the Imatest folder (typically C:\Program files\Imatest in English language installations).
After several seconds, the Imatest main window opens. Then click on or on the upper left.
SFRplus operates in two modes: interactive/setup and automatic.
Use to initiate an interactive/setup SFRplus run. This allows you to examine detailed results interactively to to save settings for the highly automated runs (or the even more automated EXE or DLL versions included in Imatest IT). SFRplus should be run at least once in prior to the first run.Rescharts SFRplus
Selecting file(s)
The portion of the Rescharts window used for opening files is shown on the right. You can open a file by clicking on if the correct chart type is displayed, or by selecting a Chart type. One or more files may be selected, as shown below. If you select multiple files, they will be combined (averaged), and you'll be given the option of saving the combined file.
If the folder contains meaningless camera-generated file names such as IMG_3734.jpg, IMG_3735.jpg, etc., you can change them to meaningful names that include focal length, aperture, etc., with the View/Rename Files utility, which takes advantage of EXIF data stored in each file.
The folder saved from the previous run appears in the Look in: box on the top. You are free to change it. You can open a single file by simply double-clicking on it. You can select multiple files for combined runs (in Imatest Master) by the usual Windows techniques: control-click to add a file; shift-click to select a block of files. Then click . Three image files for the Canon 17-40mm L lens are highlighted. Large files can take several seconds to load.
File selection
|
| RAW files Imatest can analyze Bayer raw files: standard files (TIFF, etc.) that contain undemosaiced data. RAW files are not very useful for measuring MTF because the pixel spacing in each image planes is twice that of the image as a whole; hence MTF is lower than for demosaiced files. But Chromatic aberration can be severely distorted by demosaicing, and is best measured in Bayer RAW files (and corrected during RAW conversion). Details of RAW files can be found here. |
SFRplus settings windows
SFRplus setup window
When the file (or files) have been opened, the SFRplus parameters & setup window, shown below, appears. This window allows you to select groups of regions (ROIs) for analysis, shown as violet rectangles. It also lets you select the size of the regions, whether to analyze vertical or horizontal edges, and change several additional settings. Pressing the button on the left opens the SFRplus Settings & options window, which allows you to select additional settings that affect the calculations, display, and output (for automated SFRplus runs). The light yellow-orange rectangles are for calculating the Color/lightness uniformity profiles.
SFRplus parameters & setup window; 9 regions selected for analysis
| SFRplus setup window controls Settings | |
| Settings area | |
| Gamma | Assumed Gamma (contrast) of the chart. Has a small effect on the MTF results. Default is 0.5. |
| Channel | Select channel to analyze: R, G, B, Y, R-only, G-only, B-only, Y-only. (Y is Luminance channel). Use one channel only to speed up calculations or where other channels are dark and may not contain valid data. |
| Chart contrast (for gamma calc.) |
Chart contrast-- for the contrasty squares (i.e., most of them). Used to estimate gamma from the image. |
| Use for MTF | (Checkbox, normally unchecked) When checked, use the gamma derived from the chart for the MTF calculation. This may result in a small improvement in accuracy. |
| Open the SFRplus settings & options window, shown below. | |
| ROI selection area | |
| Region (center, etc) (Selects which regions to locate. Actual ROIs are located automatically) |
Select the regions (ROIs) to analyze. Choices below. The number of regions is in parentheses. This
is a particularly important setting. We encourage users to become
familiar with the settings below. Note that the squares above and
below the middle square are never selected for analysis; they are
reserved for other purposes (stepchart, color chart, and/or focusing
pattern). At least 13 regions are required for 3D plots. These
selections are indicated by , below.
1. Center (1) 2. Center & corners (5) 3. Center, corners, part-way (9) (Part-way on diagonal between center & corners). This setting is often a good compromise between speed and detail. 4. Center, L, R, T, B (5) 5. Center, corners, L, R, T, B (13) 6. Center, corners, part-way, L, R, T, B (13) This is the smallest selection that can be used with 3D displays. 7. 5 rows, 5 columns (except step & color ROIs) (23) (edges on a 5x5 grid, omitting the squares above and below the middle). Highly detailed results, well-suited for 3D displays. 8. Inner squares (max 3x5), outer edges A 3x5 grid (omitting squares above and below the middle) representing the central region of the image. 9. All inner squares, outer edges. A tight grid of squares covering most of the image, except near the edges. 10. All squares, inner & boundary edges (best 3D map). A highly detailed grid, recommended for detailed 3D displays. The edges tend to be non-overlapping. 11. All squares, edges (many; overlapping; slow) All available edges; not generally recommended. 12. Center, part-way to corners (for mfg test) (5) This is a minimal pattern for manufacturing testing, where high speed is required. 13. All squares, inner & bdry except low contrast (good 3D map). A highly detailed grid, recommended for detailed 3D displays for charts that have two contrast levels (mostly high and a few low). The edges tend to be non-overlapping. Other selections may be added on user request. |
| Vertical, Horizontal edges (or both) | Chooses between Vertical and Horizontal edges (or both). Usually Vertical, but Horizontal is useful on occasion. Use both for Lens-style MTF plots. |
| Step chart (checkbox) | Perform step chart analysis. Should be unchecked for charts that do not include the step chart (such as transmissive chrome-on-glass charts). Checked by default. |
| Color chart (checkbox) | Perform color analysis. Should only be checked for charts that include the optional color pattern. Be sure the correct Color reference (file name) and Color space has been entered in the Settings region of the Settings window. Unchecked by default. |
| Rows (Auto or number) |
The number of rows of squares in the chart (between the top and bottom bars) or Auto for automatic row detection (the default). Since Automatic row detection may fail if the apect ratio of the squares is not 1:1, setting it to the actual number of rows (4, 5, or 7) is recommended. |
| Negative | Check this box for negative images (light squares on dark background). |
| Speedup (checkbox) | Speed up calculatons by eliminating some calculations that many users don't require, including SQF, noise statistics and historgrams, and edge roughness. |
| ROI size | Slider that determines the size of the ROI. Use the largest value that keeps a save distance from edges of squares and top and bottom bars. May need to be reduced where distortion is severe. |
| Allows borders to be cropped to remove interfering patterns that might otherwise be included in the image. This button is tinted pink whenever the image is cropped. | |
| ROI width (below ROI size slider) |
Width of ROI selection. Normal width for the standard rectangular ROI. Choose Wider or Widest for very fuzzy edges or for enhanced noise analysis. |
| Other controls | |
| Title | Title. Defaults to file name. You can add a description. |
| Open this web page in a web browser. | |
| Image setting | Selects image channel for display: Original (RGB) image, Red, Green, or Blue channels |
| Save settings (for use in auto SFRplus), but do not continue with run. | |
| Save settings and continue with run: Calculate results for all selected region. You will be able to view results interactively. | |
| Cancel run; do not save settings | |
After you've finished making settings, click to save settings and continue with the run. You can Click to save the settings without continuing.
SFRplus settings window
The SFRplus settings window, shown below, opens when in the setup window is pressed. The settings are read from the rescharts.ini file, and saved to both imatest.ini and rescharts.ini when is pressed. Settings are similar to the settings in the SFR input dialog box.
SFRplus settings window
This window is divided into sections: Title and on top, then Plot and save, Display options, Settings, Optional parameters, and finally, or .
Title defaults to the input file name. You may leave it unchanged, replace it, or add descriptive information for the camera, lens, converter settings, etc.— as you please.
opens a browser window containing a web page describing the module. The browser window sometimes opens behind other windows; you may need to check if it doesn't pop right up.
Plot and save (for SFRplus Master auto and IT; NOT for Rescharts). This area selects figures to plot and save as well as a number of data save settings. It only applies to the automatic version of SFRplus (also EXE and DLLs) — it is not for Rescharts. The leftmost checkboxes in this section select figures to plot and save. Note that all plotted figures are saved. Saved figures, CSV, and XML files are given names that consist of a root file name (which defaults to the image file name) with a suffix added. Examples:
Close figures after save should be checked if a large number of figures is to be displayed. It prevents a buildup of figures, which can slow processing. A CSV summary file is saved for all runs. An XML file is saved if Save XML results is checked. You can select either Save CSV files for individual ROIs or Save summary CSV file only (the summary file is always saved). Save figures as PNG or FIG files. PNG files (a losslessly-compressed image file format) are the default—.they require the least storage. Matlab FIG files allow the data to be manipulated-- Figures can be resized, zoomed, or rotated (3D figures-only). FIG files should be used sparingly because they can be quite large. PNG files are preferred if no additional manipulation is required. Save folder determines where results are stored. It can be set either to subfolder Results of the image folder or to a folder of your choice. Subfolder Results is recommended because it is easy to find if the image folder is known. When 3D plot is checked, SFRplus auto plots the last 3D plot displayed in Rescharts unless the button has been pressed and one or more plots has been selected. will be displayed in pink in this case. This allows several 3D plots to be displayed and saved by SFRplus auto. |
Display options (to the right of Plot) contains settings that affect the display (units, appearance, etc.). MTF plots (individual and summary) selects the spatial frequency scale for MTF plots for for the summary plot. Cycles/pixel (C/P), Cycles/mm (lp/mm), Cycles/inch (lp/in), Line Widths per Picture Height (LW/PH), and Line Pairs per Picture Height (LP/PH) are the choices. (Note that one cycle is the same as one line pair or two line widths.) If you select Cycles per inch or Cycles/mm, you must enter a number for the pixel size— either in pixels per inch, pixels per mm, or microns per pixel. For more detail on pixel size, see the box below. Maximum MTF plot frequency selects the maximum display frequency for MTF plots. The default is 2x Nyquist (1 cycle/pixel). This works well for high quality digital cameras, not for imaging systems where the edge is spread over several pixels. In such cases, a lower maximum frequency produces a more readable plot. 1x Nyquist (0.5 cycle/pixel), 0.5x Nyquist (0.25 cycle/pixel), and 0.2x Nyquist (0.1 cycle/pixel) are available.  Secondary readout controls the secondary readout display in MTF plots. The primary readout is MTF50 (the half-contrast spatial frequency). Two secondary readouts are available with several options. The first defaults to MTF30 (the spatial frequency where MTF is 30%). The third is used only for SFRplus Lens-style MTF plots. Clicking Change opens the window shown on the right. Secondary readout settings are saved between runs. Choices:
Edge plot selects the contents of the upper (edge) plot. The edge can be cropped (default) or the entire edge can be displayed. Three displays are available.
|
Settings affect the results as well as the display. Color reference and Color space are used when a color analysis is to be performed (if the image contains the color pattern and the Color chart box is checked in the SFRplus setup window). The Color reference dropdown menu allows you to browse for the reference file, typically a LAB data file in CSV format, supplied with color SFRplus charts. It should be stored in a convenient location. The last selected file is saved. Gamma is used to linearize the input data, i.e., to remove the gamma encoding applied in the camera or RAW converter. Gamma (input) defaults to 0.5 = 1/2, which is typical of digital cameras, but is affected by camera or RAW converter contrast settings. It should be set to 0.45 when RAW images are read into Imatest (to be converted by dcraw), but there is little loss in accuracy if it is left at 0.5. If is is set to less than 0.3 or greater than 0.8, the background will be changed to pink to indicate an unusual (possibly erroneous) selection. Chart contrast For a medium or low contrast charts (contrast <= 40; not recommended with the old ISO 12233 chart), you can enter the chart contrast (or Off). If the ROI is large enough, the actual (measured) gamma will be calculated and displayed along with the contrast factor (the chart contrast multiplier = measured gamma/nominal gamma, where nominal gamma is entered in the Settings area, described below). If the Use for MTF box just to the right is checked, this value will be used in the MTF calculation, which may result in a modest improvement in accuracy. SFR measurements are moderately sensitive to the Gamma setting (a 10% error in gamma results in roughly 2.5% error in MTF50; the higher the chart contrast the greater the error). For charts with contrast of 20 or less, a slightly more accurate value of gamma can be obtained by entering Chart contrast (for gamma calc.) and checking the Use for MTF box. (This works poortly for contrasty charts, like the old ISO 12233 chart.) If you do this, be sure the region selection includes edges of only one contrast. Gamma can also be obtained from the step chart in the SFRplus chart or (less conveniently) by running Colorcheck or Stepchart. A nominal value of gamma should be entered, even if the value of gamma derived from the chart (described above) is used to calculate MTF.
Channel is normally left at it's default value of Y for the luminance channel,
where Y = 0.3*R + 0.59*G + 0.11*B.
In rare instances the R, G, and B color channels might be of interest. Reset restores the settings in Options and Settings to their default values. |
|||||
Additional parameters (all optional) for Excel .CSV output contains a detailed description of the camera, lens, and test conditions. EXIF data is entered, if available, but can be overridden by manual settings. Description & settings is particularly useful for annotating the test system (it is displayed in MTF Compare).These settings are optional but can be useful when several tests are run for different lenses, focal lengths, apertures, or other settings. The settings are displayed next to the MTF plots. They are saved and reused in subsequent runs for files with the same pixel dimensions. If EXIF data is available (currently, only in JPEG files) it overrides the saved settings. The Reset button clears all entries. |
ISO standard SFR If this checkbox is checked, SFR calculations are performed according to the ISO 12233 standard, and the y-axis is labeled SFR (MTF) (ISO standard). This method is slightly less accurate than the normal Imatest calculation, which incorporates a number of refinements, including a better edge detection algorithm and a second-order polynomial fit to the average edge for a more accurate estimate of SFR in the presence of lens distortion. This box is normally left unchecked; it should only be used for comparing normal Imatest calculations to the ISO standard. The difference is typically very small.
When entries are complete, click to return control to the SFRplus settings & options window. When all entries are complete, click either , , or . saves the settings for use in automated SFRplus runs, which can be initiated from the button in the main Imatest window. saves the settings then calculates results for interactive viewing. A sequence of Calculating... boxes appear to let you know how calculations are proceeding. When calculations are complete, results are displayed interactively in the Rescharts window, as shown below.
Warnings
A Clipping warning is issued if more than 0.5% of the pixels are clipped
(saturated), i.e., if dark pixels reach level 0 or light pixels reach
the maximum level (255 for bit depth = 8). This warning is emphasized
if more than 5% of the pixels are clipped. Clipping reduces the
accuracy of SFR results. It makes measured sharpness better than
reality.
Clipping warnings
The percentage of clipped pixels is not a reliable index of the severity of clipping or of the measurement error. For example, it is possible to
just barely clip a large portion of the image with little loss of
accuracy. The plot on the right illustrates relatively severe clipping,
indicated by the sharp "shoulder" on the black line (the edge
without standardized sharpening). The sharp corner makes the MTF look
better than reality. The absence of a sharp corner may indicate that
there is little MTF error.
low contrast target is recommended for reducing the likelihood of clipping: it increases exposure latitude and reduces the sensitivity of the MTF results to errors in estimating gamma.
SFRplus summary
- SFRplus analyzes images of the SFRplus test chart, framed so that there is white space above and below the horizontal bars in the chart, i.e., so neither bar runs off the image. The bars may, however, run off the sides of the image. The white space should be between 0.5% and 25% of the image height. There should be few or no interfering patterns (bars, etc.) outside the image of the chart itself.
- Lighting should be even and glare-free. Lighting and alignment recommendations are given in The Imatest test lab.
- The first time SFRplus is run, it should be run through Rescharts. This allows
- parameters to be adjusted and saved for later use in the automatic version of SFRplus, which is opened with the button in the Imatest main window.
- results (listed above) to be examined interactively in the Rescharts window.
- The button on the main Imatest window runs SFRplus in full automatic mode using settings saved from the most Rescharts run.
Next: Using SFRplus Part 3: Results
Pixel sizePixel size has an important relationship to image quality. For very small pixels, noise, dynamic range and low light performance suffer. Pixel size is rarely given in spec sheets: it usually takes some math to find it. If the sensor type and the number of horizontal and vertical pixels (H and V) are available, you can find pixel size from the table on the right and the following equations.
pixel size in microns = 1000 (diagonal in mm) / sqrt( H2 + V2 ) |
|
