We discuss several common image quality measurements that are often misinterpreted, so that bad images are falsely interpreted as good, and we describe how to obtain valid measurements.

Sharpness, which is measured by MTF (Modulation Transfer Function) curves, is frequently summarized by MTF50 (the spatial frequency where MTF falls to half its low frequency value).

But because MTF50 strongly rewards excessive sharpening, we recommend other summary metrics, especially MTF50P (the spatial frequency where MTF falls to half its peak value), that provide a more stable indication of system performance.

Camera dynamic range (DR), defined as the range of exposure (scene brightness) where the image has good contrast and Signal-to-Noise Ratio (SNR), is usually measured with grayscale step charts. We have recently seen several cases where flare light radiating out from bright areas of the image fogs dense patches, causing unreasonably high DR measurements. This situation is difficult to handle with linear test charts, where the flare light is aligned with the patches, but can be handled well in charts with circular patch patterns, where the patch where pixel level ceases to decrease defines the upper DR limit.

Author: Norman Koren, founder and CTO Presented at Electronic Imaging 2020

Author: Robert Sumner, Lead Engineer, Imaging Science
Presented at Electronic Imaging 2020

Camera-based advanced driver-assistance systems (ADAS) require the mapping from image coordinates into world coordinates to be known. The process of computing that mapping is geometric calibration. This paper provides a series of tests that may be used to assess the goodness of the geometric calibration

 and compare model forms:

1. Image Coordinate System Test: Validation that different teams are using the same image coordinates.
2. Reprojection Test: Validation of a camera’s calibration by forward projecting targets through the model onto the image plane.
3. Projection Test: Validation of a camera’s calibration by inverse projecting points through the model out into the world.
4. Triangulation Test: Validation of a multi-camera system’s ability to locate a point in 3D.

The potential configurations for these tests are driven by automotive use cases. These tests enable comparison and tuning of different calibration models for an as-built camera.

Author: Paul Romanczyk, Senior Imaging Scientist
Presented at Electronic Imaging 2020

Presentation

Watch on YouTube

Shannon information capacity, which can be expressed as bits per pixel or megabits per image, is an excellent figure of merit for predicting camera performance for a variety of machine vision applications, including medical and automotive imaging systems.

Its strength is that is combines the effects of sharpness (MTF) and noise, but it has not been widely adopted because it has been difficult to measure and has never been standardized.

We have developed a method for conveniently measuring information capacity from images of the familiar sinusoidal Siemens Star chart. The key is that noise is measured in the presence of the image signal, rather than in a separate location where image processing may be different—a commonplace occurrence with bilateral filters. The method also enables measurement of SNRI, which is a key performance metric for object detection.

Information capacity is strongly affected by sensor noise, lens quality, ISO speed (Exposure Index), and the demosaicing algorithm, which affects aliasing. Information capacity of in-camera JPEG images differs from corresponding TIFF images from raw files because of different demosaicing algorithms and nonuniform sharpening and noise reduction.

Author: Norman Koren, founder and CTO
Presented at Electronic Imaging 2020

Measuring the MTF of an imaging system at its operational working distance is useful for understanding the system’s use case performance. However, it is often not practical to test imaging systems at long distances (several meters to infinity), particularly in a production environment. Intermediate optics (relay lenses) can be used to simulate longer test distances. The Imatest Collimator Fixture is a machine developed for testing imaging systems at specified simulated distances up to infinity through the use of a relay lens and a test chart. The relay lens’s optical properties dictate the required distance between the optic and the test chart, or Collimator Working Distance (WD_C), to project the correct simulated distance (SD). This paper provides a method for validating the accuracy of simulated test distances. Successful validation is achieved when the distances at which peak MTF occurs in the real world match the simulated distances at which peak MTF occurs on the collimator fixture, or if both distances are within the depth of field (DoF) of the imaging system in use.

Authors: Alex Schwartz, Mechanical Engineer; Sarthak Tandon, Mechanical Engineer; and Jackson Knappen, Imaging Science Engineer
Presented at Electronic Imaging 2020

Presentation

Watch on YouTube

Related information

• Long Range Testing
• Target Collimators