Using SFR Part 1

The slanted-edge test image

Imatest SFR measures the spatial frequency response, also known as the Modulation Transfer Function (MTF), of digital cameras and digitized film images.

ISO 12233 test chart with Imatest SFR cropping indicated by the red rectangle.

It uses a simple slanted-edge (black-to-white) target described in Sharpness: What is it and how is it measured? Although it doesn't provide the as strong a visual indication of MTF as the Log Frequency test chart, it provides a more accurate quantitative measurement.

Imatest SFR also measures Chromatic aberration, noise and Shannon information capacity— a useful but unfamiliar indicator of image quality.

Imatest SFR uses a standardized sharpening algorithm to compensate for the different amounts of sharpening in different cameras, allowing them to be fairly compared.

These instructions also apply to the Slanted-edge SFR module of Rescharts, which is performs identical calculations to Imatest SFR, but has a more interactive interface.

Origins of Imatest SFR  The algorithms for calculating MTF/SFR were adapted from a Matlab program, sfrmat, written by Peter Burns () to implement the ISO 12233 standard. Imatest SFR incorporates numerous improvements, including improved edge detection, better handling of lens distortion, and far more detailed output. A description can be found here. The original Matlab code is available on the I3A ISO tools download page by clicking on ISO 12233 Slant Edge Analysis Tool sfrmat 2.0. In comparing sfrmat 2.0 results with Imatest, keep the following in mind: If an OECF (tonal response curve) file is not entered into sfrmat, it assumes that there is no tonal response curve, i.e., gamma = 1. In Imatest, the default gamma is 0.5, which is typical of digital cameras. To obtain good agreement with sfrmat, you must set gamma to 1.

A horizontal slanted-edge for measuring vertical MTF performance is shown below; a vertical slanted-edge for measuring horizontal MTF performance is shown on the right. These edges were cropped from an image of the PIMA/ISO 12233 test chart taken by the Canon EOS-10D digital SLR, shown above. A crop area is indicated by a red rectangle.
Imatest Master can analyze edges of nearly any angle. For best results, exact vertical, horizontal, and 45° edges should be avoided.

Printing, assembling, and photographing the target

Summary
• Obtain a test chart
• Print the chart or Display a pattern on an LCD screen.
• Assemble the printed charts into a target.
• Photograph the target.
• Tips: Lighting | Distance | Exposure | Photographing the target
• Test chart print quality

You can run Imatest SFR with any clean, sharp, straight black-to-white or dark gray-to-white edge. The solid areas need to be smooth and uniform: white on one side and black or dark gray on the other. Since Imatest does not yet sell a printed chart, the quickest way to obtain a chart is to print one on a high quality inkjet printer.

Obtain a test chart.

Several alternatives are available for obtaining test charts.

• Download one of the test images below and print it on a high quality inkjet printer (details below) or send it to a lab for printing, for example, one of the labs listed by Dry Creek Photo.
• Create and print a test image with the Imatest Test Charts module. A variety of contrast ratios and highlight colors are available. Several Structured vector graphics (SVG) charts are available in Imatest Master. Some of the slanted square on an m x n grid patterns (example on right) are exceptionally well suited for printing large and for automated testing, and have numerous advantages over the ISO 12233 chart.
• Purchase a chart such as the ISO 12233 (QA-72)  or QA-77 (available from Applied Image) or as a less expensive alternative from Danes-Picta in the Czech Republic (the DCR3 chart on their Digital Imaging page). The Applied Image charts are printed with much higher resolution than an inkjet printer can achieve, hence they can be use at much closer distances. Their disadvantage is that they have very high contrasts (>=40:1), which often results in reduced accuracy, and they have a great deal of wasted real estate: usable edges may not be found in regions of greatest interest such as the four corners.
• A printable vector-graphics version of the ISO 12233 chart is available courtesy of Stephen H. Westin of the Cornell University Computer Graphics Department. It should be printed as large as possible (17x22 inches (A2) minimum; 24x26 inches (A1) if possible for 8+ megapixel cameras) so measured sharpness is limited by the camera and lens, not the printer. Because of the limitations of the ISO chart (described above), the Imatest SVG charts are generally preferable.

You can download printable test charts by right-clicking on the thumbnails below. The bitmap chart, which is included in the samples folder of the Imatest installation, has pixel levels 47 and 255, has a contrast ratio of  (255/47)^2.2 = 41 when printed at gamma = 2.2 (the normal setting). This is close to the minimum recommended by the ISO 12233 standard (but, as we've learned recently, higher than optimum). The two SVG charts have contrast ratios of 20 and 2. Instructions for their use can be found in Test Charts. The recommended print size is letter or A4, but these charts print at maximum quality for any size.

SVG charts — can be viewed, edited, and printed from Inkscape. Right-click on the links to save the file. Download size: 304 KB (SVG); 500 KB (PDF).
	Contrast = 20, light, gamma = 2.2, white highlights (SVG) | Contrast = 20 (PDF)  High enough contrast to excite maximum sharpening in most cameras but low enough to avoid clipping in most cases.		Contrast = 2, middle, gamma = 2.2, white highlights (SVG) | Contrast = 2 (PDF)  Relatively low contrast: little, if any, sharpening would be expected. The two charts should have the same MTF if no nonlinear processing is applied, for example, with RAW files with dcraw conversion.
Bitmap chart
	Edge_chart_low.png (reduced contrast; pixel levels 47, 255)

Test images with a varety of contrast ratios and highlight colors can be created with the Imatest Test Charts module.

Nonlinear signal processing and chart contrast Although Imatest SFR is relatively insensitive to chart contrast (MTF is normalized to 100% at low spatial frequencies), measured SFR is often affected by chart contrast due to nonlinear signal processing in cameras, i.e., processing that depends on the contents of neighboring pixels, and hence may vary throughout an image. Nonlinear processing is almost universal in digital cameras (though you can avoid it by using RAW images with dcraw). It improves pictorial quality but complicates measurements. It takes two primary forms. Sharpening, applied in the proximity of contrasty features like edges. Boosts response at high spatial frequencies. Noise reduction, applied in the absence of contrasty features. Attenuates response at high spatial frequencies, i.e., removes fine, low contrast detail (texture), which is interpreted as noise. Many cameras increase noise reduction at high ISO speeds. The signal processing algorithms are proprietary; they are a part of a manufacturer's "secret sauce" for producing pleasing images. Though they vary a great deal, some generalizations can be made. Most cameras do NOT apply noise reduction and sharpening uniformly throughout an image. Contrasty edges tend to have better (more extended) MTF than low contrast edges. For this reason it may be a good idea to photograph both a relatively contrasty edge (though not so high that it causes clipping) as well as a relatively low contrast edge. The SVG charts (above) are excellent choices. Both types of edge can also be produced using Imatest Test Charts. An estimate of chart contrast derived from the average light and dark pixel levels (away from the transition) and gamma is displayed in several places in SFR and Rescharts Slanted-edge SFR. (Estimated chart contrast = (avg. pixel level of light area/avg. pixel level of dark area)^(1/gamma) ). Nonlinearities are analyzed in depth in the Log F-Contrast module.

Print the charts

Print the test image with a high quality inkjet photo printer on glossy, semigloss, or luster paper. Alternatively, send it to a lab to be printed. Dry Creek Photo has an excellent listing. I recommend printing at least two copies: one for measuring center sharpness and one for edge sharpness. Bitmap images should be printed 8x10 inches (20.3x25.4 cm) or smaller— size is not critical. Be sure the edges look clean and sharp to your eyes; examine them with a good magnifier or loupe. Chart quality is described in detail in Chart quality and distance, below.

The bitmap charts are intended to be tilted approximately 5.7 degrees (anywhere between 2 and 7 degrees is OK) when they are photographed. They are tilted 5.7 degrees when the tick marks, located near the edges, are vertically or horizontally aligned with the center. This is illustrated by the red horizontal and vertical lines on the right. Imatest Master can analyze edges of nearly any angle; exact horizontal, vertical, and 45° edges should be avoided for best results.

The SVG charts do not need to be tilted.

Alternatively, print it at one of the labs listed by Dry Creek Photo. The charts are printed straight and physically tilted because the edges print sharper that way. If they were printed at an angle, the printer dot pattern could cause some jaggedness. A 5.71° angle (tan^-1(0.1)) is an offset of one part in 10.

Assemble the charts into a target to be photographed.

The following target is described in detail in The Imatest Test Lab. It can be used to measure lens performance near the center, part-way out, and near all four corners. The Log F-Contrast and Star charts to the left and right of center are not necessary for lens testing; they provide information about signal processing. Slanted-edges or other charts (the Kodak Q-14 step chart or the Gretag Macbeth ColorChecker) may be substituted if your primary interest is lens testing.

Sharpness target (30x40 inch) for cameras up to 13 megapixels

The printed charts are mounted on a 32x40 inch sheet of 1/2 inch thick black foam board with spray adhesive or double-sided tape. If a ColorChecker is substituted, Velcro is recommended so it can be removed for dark storage. 1/2 inch foam board stays flatter than standard 1/4 or 3/8 inch board. Black board results in less flare light than white board. (Flare light is light that bounces between lens elements and off the inside of the lens barrel, reducing image contrast).

An image of a horizontal or vertical edge on an LCD monitor (desktop or laptop) can also be used as a target. The camera should be tilted with respect to the monitor. The image dark gray and white rather than black and white to minimize clipping. This page contains a description and an image, as well as a description of the Screen Patterns module, which can be used to create an image. The disadvantage of this technique is that you have only one edge to work with; you can't easily create a map of lens performance.

An image of a horizontal or vertical edge on an LCD monitor (desktop or laptop) can also be used as a target. The camera should be tilted with respect to the monitor. (Thanks to Scott Kirkpatrick for the suggestion.) The disadvantage of this technique is that you have only one edge to work with; you can't easily create a map of lens performance.   To minimize artifacts (Moire, etc.) the camera must be far enough from the monitor so the sensor pixel "frequency" (1/(2*pixel spacing) at the image sensor) is at least 30% above the Nyquist frequency of the LCD screen. A good rule of thumb is that the LCD screen image should take up no more than 1/3 to 1/4 the image width (1/9 to 1/16 the area), as shown on the right. My inexpensive Toshiba laptop, shown on the right, displays 1024 pixels horizontally. The Canon EOS-10D has 3072 horizontal pixels. In this case, the laptop image should take up no more than 1/4 the image width. Click here to view a web page with a suitable LCD test image (dark gray and white rather than black and white to minimize clipping).

Photograph the target.

Frame the chart so there are usable edges near the center, part-way out, and near the corners. Take care that the chart is properly aligned. A number of useful alignment techniques and tricks are presented in The Imatest Test Lab.

Lighting

The chart below summarizes lighting considerations. The goal is even, glare-free illumination. Lighting angles between 30 and 45 degrees are ideal in most cases. At least two lights (one on each side) is recommended; four or six is better. Avoid lighting behind the camera, which can cause glare. Check for glare and lighting uniformity before you expose. A detailed description of the recommended lighting setup, which uses six high quality (CRI > 98) 4700K (near-daylight) 50W SoLux quartz-halogen lamps, can be found in The Imatest Test Lab. SoLux Task Lamps may also be used. The BK Precision 615 Light meter (Lux meter) is an outstanding low-cost instrument (about $100 USD) for measuring the intensity and uniformity of illumination.

Distance

Distance and field of view The camera must be far enough from the chart so you are measuring the sharpness of your camera and lens, not the chart. But remember, It's the field of view, not the chart-to-camera distance, that counts. A rough rule of thumb: For an inkjet-printed chart the field of view should be at least            22 inches (60 cm) for a  6-megapixel camera;            35 inches (90 cm) for a 16 megapixel camera. Details below. For a high-quality photographically-printed chart (such as the charts from Applied Imaging) you can get quite a bit closer.

A letter-sized (8.5x11 inch) chart printed on Permium Luster paper on the Epson 2200 (a high quality pigment-based inkjet photo printer) was analyzed for MTF using the 6.3 megapixel Canon EOS-10D. There was no change when the image field was at least 22 inches (56 cm) wide— twice the length of the chart. Performance fell off slowly for smaller fields.

Choose a camera-to-target distance that gives at least this image field width. The actual distance depends on the sensor size and the focal length of the lens. The minimum image field is illustrated on the right.

Commercial charts are often printed at higher resolutions than inkjet printers can achieve, and hence may be used at shorter distances— microscopic in some cases. For example, the small Applied Image/Sine Patterns ISO-12233 Resolution Charts (.1X, .5X, and 1X) are printed on photographic paper, which is capable of extremely fine resolution. The chart may safely fill the entire frame.

Cameras with more pixels, and hence higher potential resolution, should should have a larger image field width.

Distance/field width guidelines for high quality inkjet charts (You can get closer with photographically-printed charts.)	The camera-to-target distance is not critical as long as it is greater than a reasonable minimum. Sensor sizes Desig- nation Diagonal mm. Width mm. Height mm. 1/3.6" 5.0 4.0 3.0 1/3.2" 5.68 4.54 3.42 1/3" 6.0 4.8 3.6 1/2.7" 6.59 5.27 3.96 1/2" 8.0 6.4 4.8 1/1.8" 8.93 7.18 5.32 2/3" 11.0 8.8 6.6 1" 16.0 12.8 9.6 4/3" 22.5 18.0 13.5 35mm 44.3 24.0 36.0
Image field width (in inches) > 8.8 × sqrt(megapixels)     ( >  means  "greater than.") Image field width (in cm)      >  22 × sqrt(megapixels)                                           — or — There should be no more than 140 sensor pixels per inch of target or 55 sensor pixels per centimeter of the target.                                           — or — The distance to the target should be at least 40X the focal length of the lens for 6-10 megapixel digital SLRs. (25X is the absolute minimun for 6 megapixel DSLRs; 40X leaves some margin.) For compact digital cameras, which have much smaller sensors, the distance should be at least 100X the actual focal length: the field of view is about the same as an SLR with comparable pixel count. The recommended distance is described in more detail in Chart quality and distance, below.
There is some confusion about lens focal lengths in compact digital cameras. They are often given as the "35mm-equivalent," which many photographers can relate to viewing angle. 35-105mm or 28-140mm are typical "35mm-equivalent" numbers, but they are not the true lens focal length, which is often omitted from the specs. What is given is the sensor size in 1/n inches, a confusing designation based on the outside diameter of long-obsolete vidicon tubes. It The table on the right relates the 1/n designation to the diagonal dimension of the sensor.       True focal length = "35mm-equivalent" × (diagonal mm.) / 44.3

Exposure
Proper exposure is important for accurate Imatest SFR results. Neither the black nor the white regions of the chart should clip— have substantial areas that reach pixel levels 0 or 255. The best way to ensure proper exposure is to use the histogram in your digital camera. Blacks (the peaks on the left) should be above the minimum and whites (the peak(s) on the right) should be below the maximum.

Low contrast test charts (with contrast ratios of 40:1 or lower) are helpful for avoiding clipping.

The above image (taken from the Canon File Viewer Utility) is close to a perfect exposure.

Tips on photographing the chart Distance doesn't matter as long as the target far enough from the camera so sharpness is limited by the camera and lens, not by the target. For a target printed on the Epson 2200 printer, a distance that gives at least a 24 inch (horizontal) field of view seems to be sufficient. The target should be evenly lit and free of glare. White balance should be approximately neutral. Use a sturdy tripod and a cable release. If possible, use the mirror lock. You can use Imatest SFR to find the difference made by a good tripod or mirror lock— to sharpen your technique, literally ( pun intended ). Be sure to expose the image so detail is maintained in both light and dark areas. Neither should be blocked (clipped). Use your camera's histogram. If more than 0.5% of the pixels are at levels 0 or 255, Imatest SFR will assume that clipping has taken place and issue a warning message. This has no effect on the calculations— it's just a warning that accuracy may be compromised. Be sure the camera is correctly focused on the chart. Place slanted-edge images near the corners of the field as well as near the center. You may find it instructive to photograph the slanted edge target along with a target from Lens testing, but there's no need to do so.

Save the image as a RAW file or maximum quality JPEG. If you are using a RAW converter, convert to JPEG (maximum quality), TIFF (without LZW compression, which is not supported), or PNG. If you are using film, develop and scan it. The file name should be descriptive and should indicate the parameters you are testing. Use dashes and underscores ( - and _ ), but avoid spaces. (Spaces work with Imatest, but they can be troublesome in DOS command lines and web pages.) An example would be Canon_EOS10D_70-200f4L_100mm_f8_ctr.jpg.

You are now ready to run Imatest SFR.

Chart quality and distance Three printed charts were scanned with the Epson 3200 flatbed scanner at 1200 dpi— a much higher scan density than normal for reflective documents (300 or 400 dpi). At a typical monitor resolution of 96 dpi, the images are enlarged 12.5x. The charts were tilted around 5.7 degrees (a 1.1 inch offset on an 11 inch letter-sized print) for analysis with Imatest SFR. Epson 2200 Premium Glossy 1440 dpi High contrast Epson 2200 Premium Glossy 1440 dpi Reduced contrast HP LaserJet 5L bond paper Reduced contrast The high contrast chart appears to be the sharpest. The reduced contrast chart is still very good: its dot pattern (always a concern) is not a problem. The laser print is clearly unsuitable. Results of running Imatest SFR on the high contrast printed chart are shown below. The SFR input dialog box was set to display cycles/millimeter for 1200 pixels/inch (scan density). The key result is MTF50 (50% contrast spatial frequency) = 5.15 cycles/millimeter (131 cycles/inch). Results in LW/PH and with standardized sharpening (indicated by (corr.); dashed red lines) are not applicable.   As expected, the reduced contrast chart, which doesn't appear as sharp, has a lower MTF50: 4.31 cycles/mm (109 cycles/inch). It would benefit from sharpening in the image editor. The chart MTF50 affects the minimum recommended camera-to-target distance. For example, the Canon EOS-10D (and 300D) has a pixel spacing of 7.4 microns. Its Nyquist frequency is 1000/(2*7.4) = 68 cycles/mm. When the target is 40x the lens focal length, so the magnification at the sensor is approximately 1/40 (recommended above), the MTF50 imaged on the sensor for the reduced contrast chart is 4.31*40 = 172 cycles/mm, which is 2.5 times the Nyquist frequency. The imaged target contrast is approximately 0.9 in the critical region around 0.6*Nyquist (0.3 cycles/pixel). This is adequate, though a little more distance) (e.g., 50x focal length) wouldn't hurt. A print of Stephen H. Westin's printable vector-graphics version of the ISO 12233chart, made on the Epson 2200 at 1440 dpi on Premium Luster paper and scanned by the Epson 3200 at 1200 dpi, has been analyzed to compare slanted and straight edges. The comparison is of interest because Stephen's chart uses vector graphics, hence the slanted edge is as fine as the printer can make; it is not limited by image pixels. Straight and slanted edges where compared with the print scanned straight and tilted. Document straight Document tilted Slanted edge Straight edge Slanted edge Straight edge The straight edge is clearly smoother than the slanted edge, though the sharpness of the individual edges (horizontal scan lines) is about the same. This MTF results are surprizing: MTF50 for the slanted edge scanned straight is 4.13 cycles/mm, not very different from MTF50 for the straight edge scanned tilted, 4.24 cycles/mm. The difference is more pronounced at lower MTFs (higher spatial frequencies). MTF20 for the slanted edge scanned straight is 8.6 cycles/mm while MTF20 for the straight edge scanned tilted is 10.6 cycles/mm. I clearly perfer the straight edge, tilted. Although the numbers suggest it would make little difference, I recommend a 20% greater distance if the slanted edge is used. The results in this box are for charts printed on high quality inkjet printers. Commercially-available charts are often printed at higher resolution, and hence may be used at shorter distances— microscopic in some cases.

Newsletter Subscribe to receive the latest news & specials on our products: First Name : Last name : Email address :*