Imatest Test Labs

This page describes how to construct a testing laboratory, showing both professional and low-cost lighting options.

Imatest offers 3rd party image quality testing and benchmarking, camera system testing and tuning, custom image quality specifications, and offers both off the shelf and custom equipment for its customers.

Imatest also offers application specific Imatest training and turnkey Test Lab setups to it’s customers.  Imatest is now a one-stop shop for all of your image quality testing needs!

Hardware

Substitutions can be made for most of the parts listed below. Creativity is encouraged in assembling a lab.

Professional lighting

Reflective Chart Lighting

To obtain high quality illumination covering up to a 40x60 inch (100×150 cm) reflective chart with good uniformity (>80%) and smaller reflective charts with high uniformity (>90%), we selected a flexible lighting system that utilizes multi-source luminaires (D65, CWF, Incandescent “A”) and light stands.  These lights can be configured to work anywhere in the world.

Fixture.  The fixture contains multiple light sources and also has dimmers and switches to offer flexibility in lighting conditions. The lights can be configured for 3 different color temperatures across a variety of intensity levels depending on the positioning of the lights relative to the chart or scene.

Lamps.  The base lighting system comes with 3 light sources. 

• D65 daylight (F32T8/CM65 lamps CRI of 94+)
• CWF 4150K (F34T12/CWF lamps CRI of 62)
• Incandescent Illuminant “A”

*Other sources can be substituted and we also offer a 2 light source system. 

Stand.  The vertical luminaire stands are sturdy and have castors for ease of positioning of the lights.  The stand is painted Munsell N5 which is very close to 18% gray.

Assembly.  The lighting system comes with easy to follow directions and can be assembled in less than 2 hours. Contact testing@imatest.com for more information about this lighting system or to obtain a quote.

Low-cost reflective lighting

The low cost lighting system, recommended for Imatest Studio users and others on a budget, consists of a track lighting system that takes the outstanding SoLux 4700K (near daylight) 12V MR16 halogen lamps, which use long-lasting built-in dichroic filters.

Fixture.  MR16 track lights were formerly available at local hardware outlets such as Home Depot, but most have been discontinued in favor of less expensive products. A promising mail-order track system is the Pro Track® Black 150 Watt 3-Light Low Voltage Track Kit (79353) from Lamps Plus, which should be purchased along with the Pro Track® Track Lighting Black Outlet Extension Cord (24324).

Lamps.  We use 6 SoLux 50W 4700K/36 degree MR16 bulbs, along with 6 Plano Convex diffusers, which are necessary for even light. The diffusers are mounted in front of the lamps (rightmost image), in place of the clear glass supplied with the track lighting kit. “T” braces with self-adhesive felt pads were added to the sides of the lamp fixtures to block stray light from reaching the lens (rightmost image).

SoLux halogen bulbs are inexpensive and produce extremely high quality near-daylight 4700K illumination, with a CRI (Color Rendering Index) of 98 out of 100. They have been featured in an article in Inkjetart.com. SoLux also sells track lighting components.

Stand.  The Radio Shack 33-335 folding microphone stand is simple, lightweight, sturdy, and cheap.

Assembly.  The track is attached to the stand with an 8-32 bolt. Here is the procedure.

Drill holes large enough for an 8-32 bolt about 1.25 inches (3.5 cm) below the top of the upper shaft of the microphone stand and about 10 inches (25cm) below the top of the track (the end without the cord). The head of a 1.25 or 1.5 inch 8-32 bolt is placed inside the track, then attached with two nuts on the back of the track to provide a little spacing with the stand. A wing nut (optional) is used to secure the bolt to the stand. [Parts sequence: 8-32 bolt (head), track, 2 8-32 nuts, upper shaft of microphone stand, 8-32 wing nut.] (CAUTION: do not do this unless you know how to work with electrical wiring safely. DO NOT do this when the track is plugged in! )

The remainder of the track is attached to the stand by a combination of materials that allow the stand to be raised and lowered.

• Cambuckles and nylon webbing. Nice because they put tension on the webbing when they are closed. 1 inch and 2 inch Available from REI.
• Strapall heavy-duty Quick Tape, available from REI and many other sources. This is a strong double-sided Velcro tape.

The lights are mounted near the top, middle, and bottom of the 40 inch high targets.

Easel

The Mabef M-10 easel was chosen because it was sturdy, could be adjusted vertically (checked with a small level), had just the right features, and was on display at a local art supply store. A self-adhesive felt pad on the bottom of the upper clamp helps secure targets.

Light measurement

To characterize the lighting for our image quality test lab, we use a Konica Minolta CL-200A.  This device allows for measurement of the lighting color temperature(Kelvin) as well as illuminance (in lux).  To ensure you are measuring each imaging device in the same conditions, you should always characterize and document the following:

• Color Temperature (Kelvin)
• Illuminance (Lux)
• Uniformity (%)
   

Other, less expensive lightmeters can be used to characterize the illuminance only.  A few are listed below:

Miscellaneous

Several additional items can come in handy in your image quality lab: a small level, rulers, a few clamps, tape, etc.

The $99 USD Stanley 77-910 - TLM 100 FatMax™ Tru-Laser™ Distance Measurer shown is very convenient for measuring precise distances— much more accurate than a measuring tape.

Tripod and accessories

The tripod should be sturdy, easy to adjust, and easy to move. We use high quality Manfrotto tripod legs with a geared head, but a pan/tilt head will work.  Ball heads are not recommended.   A head that takes quick-release plates is strongly recommended.  In our labs, we use the Manfrotto 475B professional tripod legs, with the Manfrotto 410 Junior geared head with the RC4 quick release system supplied with one Manfrotto 410PL quick release plate.  We illustrate the Manfrotto components used in our labs, but many more are available to meet special needs.

Manfrotto 475B Tripod Legs

Manfrotto 475B tripod legs with geared center column – supports 26.5 lbs. Height 17 – 74 in.  A geared center column is necessary to offer flexibility in chart framing in the vertical direction.  The built-in spirit level is valuable for leveling the tripod: If the base (where the head is mounted) is level, fine adjustments are much easier to make because head movements (swivel, etc.) don’t interact.

The Manfrotto 410 compact geared head with quick release – supports 11 lbs and is especially nice for making precise adjustments and used the Manfrotto 410PL quick release plate. You may want to get several of these to adapt to different imaging devices for efficient testing.

Manfrotto 127VS variable leg spread portable dolly is very useful for keeping legs in place while moving the tripod.

Since framing the test targets can be important, especially SFRplus for automatic region detection, other accessories can help assist.  The Manfrotto tripod legs and geared head offer fine tune adjustment and leveling for vertical positioning and for ensuring the imaging device is parallel to the test target.  Sometimes you may need to also have fine tune adjustment in the horizontal direction as moving the entire tripod can be cumbersome.  We use a Manfrotto 454 Micrometric positioning sliding plate to accomplish fine horizontal adjustments.

Clamps

Although a regular quick-release tripod mount is fine for most cameras, a good sturdy clamping system is needed for testing camera phones and webcams. We put two of them together after rummaging through the aisles of Home Depot. They consist of Irwin Quick-Grip Mini or Micro bar clamps mounted on Manfrotto quick release plates. The images below give a good idea of how they’re constructed. A hole had to be drilled in the larger clamp (above) and enlarged in the lower clamp (below) to accommodate the 1/4 inch bolt.

The larger clamp (the Mini) is sturdier, and has proven to be more useful.

Putting it together

The image below summarizes how the lights should be positioned. The goal is even, glare-free illumination. The lamps represent any of the lighting fixtures described above. This may involve a tradeoff, especially with wide-angle lenses. Lighting angles between 35 and 45 degrees work well for normal and long lenses; smaller angles may be required to avoid glare in wide angle lenses. Avoid lighting behind the camera, which can cause glare. Ambient light should be kept subdued. Check carefully for glare and lighting uniformity before you expose, especially with wide angle lenses, where glare can be difficult to avoid: you may have to move the lights further than normal from the target— or allow glare in portions of the target, taking care that it doesn’t affect critical areas.

Use the meter to check for the evenness of lighting. ±20% over the entire target is sufficient for SFR and Distortion measurements, where small portions of the target are analyzed and exposure is not critical. ±5% or better should be the goal for tonal and color measurements (Stepchart, Colorcheck, and Multicharts). This should be easy to achieve because illumination only needs to be even on the relatively small charts themselves, not on the entire target. The widest (the Kodak Q-14) is 14 inches (35 cm) wide; most are under 11 inches (28 cm). Lighting uniformity is strongly affected by the distance and orientation of the lamps. Our lamps are roughly 50 inches (125 cm) from the center of the target.

Positioning

The Adorama Double Bubble Level” (sounds like chewing gum) costs about $29; identical levels from Cullmann, Hasselblad, or Manfrotto, all of which fit in the accessory shoe, cost about $33 at B&H. (They’re probably all made in the same factory in China.)

Rope trick (for horizontal positioning)

This trick uses rope and mirrors, as well as the red dots • tht mark the horizontal center of the target (or whatever marker you choose). This is as close to smoke and mirrors as Imatest gets.The cord should be about 1 meter long and have a weight on one end. A heavy nut— the type that mates with a bolt, not the type that grows on trees— serves well.Hold the cord in front of the camera (between the camera and mirror), and align your eye and the cord so the so the cord passes in front of the horizontal center markers (red dots • in the images), and so the reflection of the cord passes directly behind the cord. The camera is positioned properly if the center of the lens is aligned with the cord. If it isn’t, shift the camera horizontally until it is.To complete the horizontal adjustment, rotate (pan) the tripod pan/tilt head so the chart is centered in the image frame.At this point you should be close enough to correct alignments so the final, fine adjustments should proceed quickly.Other targetsThe images below illustrate a problem that can arise using wide angle lenses. Glare appears in the outer patches, but it doesn’t affect the critical outer edges. It’s difficult to remove glare entirely: it can be impractical to place lights further from the target, and reducing the angle may make lighting too uneven. We have recently been dealing with glare by printing charts on high quality matte paper (Epson Ultra Premium Presentation – Matte), which resolves detail as fine as glossy and semigloss papers. Glare is rarely a problem with normal and telephoto lenses.

Test Charts

Charts are mounted on 40×60 inch (100×150 cm) or 30×40 inch (75×100 cm) 1/2 inch (1.25 cm) thick foam board, which is lightweight, rigid (less likely to warp than thinner boards), and easy to find at art supply shops. The images below show entire targets. Framing would normally be tighter for test images (much tighter for the Q-14, ColorChecker, and OECF targets). In some cases, the charts are mounted on neutral, middle gray (~18% reflectance) wall panel, which is attached to a wall in the lab, to ensure correct exposure in camera phones and webcams with autoexposure.Charts are mounted using permanent double-sided tape or spray adhesive, available from hardware and art supply stores. We have used 3M 45 and 3M Photo Mount sprays, both of which are “Photo safe.” 3M Super 77™ is not labeled “Photo safe,” (and can’t be sold in California), but it should be OK for attaching gray mat board to foam board.

Sharpness targets

4:1 contrast 5×7 SFRplus chart with color patches   The SFRplus chart is recommended for sharpness testing: it has numerous advantages over the older charts shown below, including the ability to map sharpness over the image surface and automatic region detection. Its contrast options (10:1, 4:1, or a combination of 10:1 and 2:1) are much less likely to cause saturation or clipping-related errors than the older ISO 12233 charts and its derivatives (note the revised ISO 12233 standard will call for 4:1 contrast edges).
The QA-77 modified ISO chart, available from Applied Image, is printed on photographic media with slightly higher resolution than is attainable with inkjet printers. It can be used with smaller image fields than inkjet-printed charts: valuable for close-up work and for analyzing telephoto lenses when space is limited. Note that the glare in the corner targets (no longer used) is doesn’t affect the outer-most tangential edges. Careful positioning of lights is critical when working with wide-angle lenses.	Modified ISO 12233 (QA-77) and slanted-edge charts
Well-framed Kodak/Tiffen Q-14 Step chart and X-Rite™ ColorChecker®   These charts (shown well-framed) can be analyzed with Imatest Stepchart, Colorcheck, and Multicharts. They are mounted on neutral middle gray (~18% reflectance) mat board, which is mounted on foam board for rigidity. The Q-13/Q-14 can be mounted vertically to reduce glare when used with wide angle lenses.
The OECF target is valuable for measuring tonal response in cameras that have significant light falloff (vignetting)— especially for camera phones, which are constrained to be extremely thin.	OECF target