Making Dynamic Range measurements robust against flare light

July 22, 2019
August 19, 2019

Introduction

A camera’s Dynamic Range (DR) is the range of tones in a scene that can be reproduced with adequate contrast and good Signal-to-Noise Ratio (SNR). Camera Dynamic Range is often limited by flare light, which is stray light in the image, primarily caused by reflections between lens elements. Flare light reduces DR by fogging images, i.e., washing out detail in dark areas. It is the primary reason that the Dynamic Range of cameras (which include lenses) is poorer than that of image sensors, which can be up to 150dB (30 million:1) for recent HDR (High Dynamic Range) sensors.

In the past two years we’ve learned that flare light in test chart images could, in theory, be mistaken for the image of the test chart, resulting in exaggerated Dynamic Range measurements. The situation became real in mid-2019, when we started seeing actual images where this happened. By studying these images we have developed techniques to guard against exaggerated SNR measurements caused by flare light. We describe these techniques here.

We recognize that some customers may actually prefer exaggerated measurements because they yield very high Dynamic Range numbers— sometimes over 120dB, which approach the specifications of HDR sensors, and look good in marketing materials. The reduced (but realistic) DR measurements obtained when the effects of flare light are removed, may make some of these customers uncomfortable. We will do our best to deal with their objections.

The effects of flare light

Flare light can be illustrated with an image of the XYLA chart— a precision HDR test chart with a linear configuration, consisting of 21 grayscale patches with Optical Density steps of 0.3. The image is from a low-cost “black box” camera.

The upper cross-section plot, made with the Image Statistics module, is taken at the center of the XYLA image. Flare is most obvious in the image and as a decay in the cross-section plot to the left of the brightest patch.

XYLA image (from the same image; the lower is lightened) and corresponding cross-section plots

The lower cross-section was taken outside (below) the active chart image, which is shown lightened above to make flare light more visible. The variation in pixel level on the right side of the chart (x between 800 and 1600) is caused by flare light diffusing from the brightest patches on the left. 

Results of the XYLA image (above) showing strong tone mapping. Click on the image to view full-sized.

Some notes on this image.

This image has very strong local tone-mapping, leading to an exceptionally low (and not very meaningful) measured gamma of 0.148.The SNR varies in an unusual way: it doesn’t drop monotonically, as it would for a conventional linear image sensor. This indicates that an HDR (High Dynamic Range) image sensor with several operating regions was used. 

Because the image had significant barrel distortion, region selection was difficult. The Contrast Resolution chart is much easier to use and provides a better indication of system performance in the presence of tone mapping.

If the flare light were any worse it could easily have been mistaken for a signal from the chart itself, leading to an false Dynamic Range measurement.

Flare light can be extremely complex. It can add an offset to the image (often called “veiling glare”), which is difficult to distinguish from a black level offset in the image processing pipeline. Most of the time it is largest near bright patches, then decreases with distance from these patches. The rate of decrease is rarely a well-behaved exponential.

Lens reflections are a major cause of medium-range flare light. An uncoated glass surface (index of refraction ≅ 1.5) reflects R = 4% = 0.04 of the light incident on it. (Remember, a sheet of glass or a lens component has two surfaces.) 

For each glass surface between this surface and the light source, a fraction R of the primary reflection (R2 of the original incident light) is reflected back to the image sensor. This are called a secondary reflection. Since most lens surfaces are curved, this light will be unfocused, i.e., it will tend to fog a portion of the image. 

According to Edmund Optics, the best anti-reflective coatings have R ≅ 0.4% = 0.004 over the visible spectrum (~400-700nm). R = 0.005 may be more realistic for a reasonable range of incident angles. The light reflected back to the sensor from each secondary reflection would be R2 = 0.000025 = 2.5*10-5 = -92 dB (20*log10(R2)). The number of secondary reflections Nsec increases rapidly with the number of components M (groups of elements cemented together, each of which has two air-to-glass surfaces) in a lens: 1 for 1 component; 6 for 2 components; 15 for 3 components; 28 for 4 components; 45 for 5 components, etc. For M components,

     \(\displaystyle \text{Number of secondary reflections} = N_{sec} = \sum_{i=1}^{2M-1}i = 2M(2M-1)/2 = M(2M-1))\)

M = 5 components are typical for high quality camera phones; M ≥12 components is commonplace for DSLR zoom lenses. Overall lens flare is less severe than the number of secondary reflections suggests because the stray light don’t cover the whole image. It decreases with distance from bright regions. It’s easy to see why practical camera Dynamic Range measurements are limited to around 70-90dB, even when sensor Dynamic Range is much higher.

Image Statistics cross-section of a Contrast Resolution image for an inferior camera, showing spatially-varying flare indicated by red arrows

Because the ISO 18844 flare model does not measure the spatially-dependent flare caused by lens reflections, it has limited value in characterizing practical system performance.

 

Key takeaways— Flare light is predominantly light in dark regions of the image that diffuses from bright regions. This diffused light can be confused with the actual chart signal (especially with linear charts), resulting in exaggerated (overly optimistic) Dynamic Range measurements. Moreover, increasing flare light (which could result from poorer, i.e., cheaper, lens coatings) decreases the actual DR by fogging shadow areas of the image, but can lead to increased DR measurements. Hence the need to distinguish artifact signals from flare light from real signals from the chart.

Circular test charts

UHDR photographic film chart

The test charts recommended by Imatest for measuring Dynamic Range are transmissive (i.e., backlit) charts with (approximately) circular patch configurations, i.e., they are not linear (like the XYLA chart, shown above). The two or three-layer High Dynamic Range chart, shown on the right, comes in several versions. Because photographic film charts can’t be manufactured with consistent patch densities, a reference file is required when these charts are used.

High quality cameras

Until recently, most of the DR images we analyzed came from DSLR or mirrorless cameras that had relatively low flare light. The recent images we’ve seen with severe flare light are from inferior cameras. We haven’t figured out exactly why the flare light is so much worse: it might be inferior coatings in the multi-element lenses or less baffling in the barrel of the lens.

Here are examples of results from high quality cameras. Click on the thumbnails below to view full-sized images.

UHDR_Canon_90TS_raw_TIFF_tone-DR_results thumbnail
UHDR_Canon_90TS_jpg_tone-DR_results thumbnail
UHDR_Canon_75-300_raw_tone-DR_results thumbnail
UHDR_Canon_75-300_jpg_tone-DR_results thumbnail
UHDR_Canon_90TS_raw_TIFF_tone-DR_results

Results for raw image from high-quality (Canon 90mm Tilt/Shift) lens

UHDR_Canon_90TS_jpg_tone-DR_results

Results for jpeg image from high-quality (Canon 90mm Tilt/Shift) lens

UHDR_Canon_75-300_raw_tone-DR_results

Results for raw image from consumer-grade (Canon 75-300mm) lens

UHDR_Canon_75-300_jpg_tone-DR_results

Results for jpeg image from consumer-grade (Canon 75-300mm) lens

The four images are for 48-bit RAW (or TIFF derived from RAW) and 24-bit JPEG files captured on the Sony A7Rii camera with two very different lenses:

  • A consumer grade Canon EF 75-300mm f/4-5.6 lens (original version) set to 80mm, f/5.6. This lens has 15 elements in 10 groups.
  • The very high quality Canon TS-E 90mm f/2.8 (Tilt/Shift) lens set to f/5.6. Since this lens has 6 elements in 5 groups, it would be expected to have lower flare light than the 75-300.
  • Note that results from the two darkest patches, 35 and 36, are outside the plot because their densities (8.184 and 8.747, equivalent to 163.7 and 174.5dB) are beyond the 160 dB limit of the plot. 160dB is equivalent to a 100 million:1 ratio— far beyond the capabilities of any camera system with a lens in front of the sensor.

As expected, the 90mm T/S lens has significantly better DR, and the JPEG files had more of a response “shoulder” (an area of reduced slope in the lighter part of the image). But there is one surprise. The Dynamic Range of the JPEG images is comparable to the raw images— apparently because gamma encoding, which decreases the number of pixel levels in bright regions and increases it in dark regions, extends DR beyond what would be expected for a linear 8-bit (256 level) file. 

Note that in three of the four images above, the Low Quality dynamic range is lower than the Dynamic Range from slope, and in the remaining image they are very close. We don’t recommend the use of slope-based DR by itself (because it often extends well beyond the region where SNR = 0dB (Signal/Noise = 1), i.e., it includes regions where poor SNR causes noise to completely mask image detail.

Low-cost “black box” camera

The differences between the DR of the two lenses (medium and high quality) seems to be minor when compared to an image from a low-cost “black box” camera we recently received. 

Dynamic Range results for low-cost “black box” camera

The Low Quality (SNR = 0dB, labeled Low ——— ) Dynamic Range is measured as 148dB— an astonishingly high number. But the Dynamic Range from slope is 66dB— much lower than several quality-based DR measurements (and lower than the slope-based DR for the 90mm T/S lens). Note that the two darkest patches don’t appear on this plot because their densities (8.184 and 8.747) are beyond the 160dB limit of the plot. (160dB is a range of 100 million to 1: more than expected from any sensor or camera— or to put it another way, if one photon were to reach the darkest patch, the lightest patch would set the chart or sensor on fire).

To understand what is happening, we need to observe the dark portions of the images from high quality cameras, shown extra (X-)  or XX-lightened to make detail visible in the darker patches, and compare them to a comparable region in the low-cost camera.

UHDR_Canon_90TS_TIFF_image_XXLight thumbnail
UHDR_Canon_90TS_jpg_image_XXLight thumbnail
UHDR_Canon_75-300_TIFF_image_XLight thumbnail
UHDR_Canon_75-300_jpg_image_XXLight thumbnail
UHDR_Canon_90TS_TIFF_image_XXLight

Dark areas of RAW (TIFF) image from Canon 90mm T/S (high quality) lens, XX-lightened

UHDR_Canon_90TS_jpg_image_XXLight

Dark areas of JPEG image from Canon 90mm T/S (high quality) lens, XX-lightened

UHDR_Canon_75-300_TIFF_image_XLight

Dark areas of RAW (TIFF) image from Canon 75-300mm (consumer-grade) lens, X-lightened

UHDR_Canon_75-300_jpg_image_XXLight

Dark areas of JPEG image from Canon 75-300mm (consumer-grade) lens, XX-lightened

The RAW image for the 6-element (5 group) 90mm lens shows distinct regions of decreasing brightness in the 7th row (2nd from the bottom; patches 27-32), and still shows distinct regions in the bottom row, though brightness no longer decreases. A reflection (possibly from the lens) is visible in patch 30. The JPEG image from the 90mm lens has more noise. Some may be quantization noise since the JPEG has only 256 levels (0-255). The images from the 15 element (10 group) 75-300mm lens definitely show more evidence of flare.

Now, compare these results— especially for the RAW image from the 90mm T/S lens (left thumbnail)— with the image from the low-cost “black box” camera. 

Dark areas of JPEG image from low cost “black box” camera, X-lightened

No patch detail is visible in the bottom two rows (patches 27-36). Instead of decreasing from left to right in each row, the pixel level decreases from top to bottom, remaining relatively constant across the bottom two rows, with banding (the result of 255 levels) clearly visible. This is clearly flare light, not signal from the test chart. Unfortunately the Signal-to-Noise ratio in these rows is quite good because Imatest removes the effects of illumination nonuniformity in noise calculations. (A setting can turn this off, but we generally recommend leaving it on to give better results in the presence of actual illumination nonuniformity). But this signal is an artifact, not the real thing.

This leads us to the inescapable conclusion that the quality-based Dynamic Range results for the low-cost “black box” camera are incorrect, and that we need to detect the patches where flare light overwhelms the signal from the chart image and exclude them from the DR calculation.

Fortunately this is not difficult for Dynamic Range test charts with a circular configuration, such as the 36-patch DR chart illustrated above. (It’s harder for linear charts). Because of the patch arrangement, the patch level stops decreasing when flare light dominates the scene. This is clearly visible in the bottom two rows in the above example, where the image gets darker from top to bottom in these rows— perpendicular to the patch sequence. When the patch brightness ceases to decrease, we can be confident that flare light dominates, i.e., we are outside the Camera Dynamic Range. This is the case for patches beyond the slope-based Dynamic Range.

For this reason it makes sense to limit all quality-based Dynamic Range measurements to the slope-based DR (maximum).

In Imatest 5.2, the Options II window (accessed from the button at the lower-right of the Imatest main window) offers a choice of whether or not to limit quality-based Dynamic range to slope-based DR. When the box is checked the limit appears in the results display.

Dynamic Range results for low-cost “black box” camera, with Quality-based Dynamic Range limited by Slope-based DR

Key takeaways— Flare light wasn’t an issue with the high quality DSLR/Mirrrorless lenses we tested in the past, but it has become a major factor limiting the performance of recent low-cost lenses intended for the automotive or security industries. We have seen examples of how flare light can improve traditional Dynamic Range measurements while degrading actual camera DR. 

Our approach to resolving this issue is to limit quality-based Dynamic Range measurements (the range of densities where SNR ≥ 20dB for high quality through SNR ≥ 0dB for low quality) to the slope-based DR. This works because, for patches beyond the slope-based limit (where the slope of log pixel level vs. log exposure drops below 0.075 of the maximum slope),

  1. contrast is too low for image features to be clearly visible, or
  2. the signal is dominated by flare light, which washes out real signals from the test chart, i.e., the “signal” is an artifact, not the real deal.

Limiting Quality-based DR in this way significantly improves measurement accuracy, and perhaps more importantly, can help prevent inferior, low-quality lenses being accepted for applications critical to automotive safety or security.

 

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*
Website

This site uses Akismet to reduce spam. Learn how your comment data is processed.