Imaging Tech – imatest https://www.imatest.com Image Quality Testing Software & Test Charts Fri, 19 Jun 2020 22:20:50 -0600 en-US hourly 1 https://wordpress.org/?v=5.3 Image Quality Testing for Webcams https://www.imatest.com/2020/05/image-quality-testing-for-webcams/ https://www.imatest.com/2020/05/image-quality-testing-for-webcams/#respond Fri, 22 May 2020 23:16:50 +0000 http://www.imatest.com/?p=34244 Webcams are an increasingly vital tool for working remotely and staying connected with friends and loved ones. As such, webcam sales have experienced significant growth. We all know good and bad quality images when we see them, but quantifying a camera’s performance is critical for making design choices, sourcing components, and performing quality control. Developing these devices presents challenges common to the development of consumer cameras. Luckily, Imatest solutions can help overcome these challenges by providing reliable, robust, and repeatable analysis of key image quality factors. Imatest software, test charts, and equipment can even help certify your device according to popular video conferencing platform certifications. Webcam testing is made even more efficient and convenient thanks to the Imatest Device Manager, which allows for image acquisition directly from the device.

Sharpness and noise performance are critical for perceiving detail in an image. In fact, they are often the factors most associated with good image quality. Compression of the video file can also have a distinct effect on image quality, although it is often unavoidable when transmitting a live video stream across the internet. It’s important to understand how the webcam reproduces colors and tones so the color correction matrix and OECF curves can be tuned. Of course, more accurate color and tone reproduction do not always mean a more pleasing image. Distortion affects all systems, and it can be corrected with software once it is characterized. More information about testing these factors (and more) with Imatest solutions can be found below.

Sharpness

One of the most important factors for creating a good image, sharpness is the ability of the camera system to resolve detail. This image quality factor is influenced by the lens optics, camera sensor, and image signal processing. Imatest provides multiple charts for sharpness testing, including ISO 12233:2017 compliant charts, all of which are fully supported by our software. More information about sharpness testing with Imatest can be found on our sharpness documentation page

Compression

Since the data captured by a webcam is meant to be transmitted across the internet, some form of compression is usually applied to the video stream to make the transmission faster and more reliable. However, this is frequently done at the expense of image quality, so it’s important to optimize the compression to maximize image quality and minimize file size. Imatest software also supports companded image files. The effects of various compression schemes can be compared using structural similarity index (SSIM). More information about SSIM testing with Imatest can be found on our SSIM documentation page

Color

The way a camera system records color is key to its ability to make a pleasing image. Keep in mind that accurate color reproduction in the final image is not always visually pleasing (we tend to prefer stronger, more saturated colors over accurate representations); however, it’s important to characterize baseline system performance before applying color correction matrices to make color more appealing. Imatest provides industry-standard color measurement test charts, and Imatest Master software offers a range of color metrics. More information about color testing with Imatest can be found on our Colorcheck documentation page

Tone mapping

Tone mapping controls how various tones are represented and balanced in the final image. It is related to a system’s dynamic range performance, but the two are not interchangeable. Dynamic range is the full range of tones that can be captured, whereas tone mapping dictates how those tones are represented. This affects the image’s contrast and level of visible detail in bright or dark areas, and overall perceived brightness or darkness. Different tone mapping strategies can be applied for various lighting conditions. As with color measurements, it’s important to remember that accurate representation of the tones in a scene does not necessarily result in a pleasing image. We tend to prefer images with more contrast, as well as maximized detail in the highlights. Before subjective tone mapping can be achieved, it’s important to characterize the camera system’s baseline tonal response before adjusting for a better looking image. Tonal response can be measured with a range of gray-scale step charts along with Imatest software. More information about tonal response testing with Imatest can be found on our Color/Tone documentation page

Flare

Flare light can have serious effects on a camera’s dynamic range and tonal response. It is the result of stray light from reflections between lens elements and inside the lens barrel, and it can be caused by bright light sources in or near the field of view. Flare is easily visible when someone is using a webcam while sitting in front of a bright window. While uncontrollable lighting conditions can make flare difficult to avoid, special lens element coatings and lens design can help to minimize its effects. Flare testing can be achieved with Imatest software using the ISO standard 18844 test target. Flare testing can be achieved with Imatest software using the ISO standard 18844 test target. More information about flare testing with Imatest can be found on our Fare solutions page

Distortion

Nearly all camera optics result in distortion to some degree, even if it’s not entirely noticeable. This is especially true of wide field-of-view lenses (which are often included on webcams) due to the geometry of the optical paths. However, software can correct for distortion so that the final image appears more natural. First, the type and degree of distortion must be characterized for a given camera system. This is possible with test charts with regular patterns, such as checkerboards or dot patterns, along with Imatest software. More information about distortion measurement with Imatest can be found on our Distortion solutions page

Noise

Image noise is visible as random variation in pixel levels due to the photon nature of light and the thermal energy of heat. It is often described as “graininess” (a hold-over from the appearance of film grain) and is usually more visible in darker regions of the image. While it can never be entirely avoided, image noise should be minimized and signal-to-noise ratio (SNR) maximized to yield the best possible image sharpness and dynamic range. Low-light conditions, such as those commonly encountered by webcams used in dim indoor lighting, are particularly difficult to handle. Noise metrics can be calculated from grayscale patterns found on multiple Imatest test charts. More information about noise testing with Imatest can be found on our Noise solutions page

Timing / Frame Rate / Latency

Temporal metrics are important to video quality, especially when the goal is to transmit video across the internet for users to interact in real time. Metrics such as timing, frame rate, and latency can be tested with specialized hardware such as the Camera Timing Test System

As more people work remotely now than ever before, quality webcams have experienced a surge in popularity. There is no shortage of challenges when developing high-quality, affordable webcams. To learn more about how Imatest test charts, equipment, and software can help you overcome these challenges, please feel free to contact us at sales@imatest.com.

]]>
https://www.imatest.com/2020/05/image-quality-testing-for-webcams/feed/ 0
Correcting Misleading Image Quality Measurements https://www.imatest.com/2020/03/correcting-misleading-image-quality-measurements/ https://www.imatest.com/2020/03/correcting-misleading-image-quality-measurements/#respond Wed, 25 Mar 2020 16:17:26 +0000 http://www.imatest.com/?p=32659

We discuss several common image quality measurements that are often misinterpreted, so that bad images are falsely interpreted as good, and we describe how to obtain valid measurements.

Sharpness, which is measured by MTF (Modulation Transfer Function) curves, is frequently summarized by MTF50 (the spatial frequency where MTF falls to half its low frequency value).

But because MTF50 strongly rewards excessive sharpening, we recommend other summary metrics, especially MTF50P (the spatial frequency where MTF falls to half its peak value), that provide a more stable indication of system performance.

Camera dynamic range (DR), defined as the range of exposure (scene brightness) where the image has good contrast and Signal-to-Noise Ratio (SNR), is usually measured with grayscale step charts. We have recently seen several cases where flare light radiating out from bright areas of the image fogs dense patches, causing unreasonably high DR measurements. This situation is difficult to handle with linear test charts, where the flare light is aligned with the patches, but can be handled well in charts with circular patch patterns, where the patch where pixel level ceases to decrease defines the upper DR limit.

Author: Norman Koren, founder and CTO Presented at Electronic Imaging 2020

]]>
https://www.imatest.com/2020/03/correcting-misleading-image-quality-measurements/feed/ 0
Describing and Sampling the LED Flicker Signal https://www.imatest.com/2020/03/describing-and-sampling-the-led-flicker-signal/ https://www.imatest.com/2020/03/describing-and-sampling-the-led-flicker-signal/#respond Wed, 25 Mar 2020 16:16:31 +0000 http://www.imatest.com/?p=32642 High-frequency flickering light sources such as pulse-width modulated LEDs can cause image sensors to record incorrect levels. We describe a model with a loose set of assumptions (encompassing multi-exposure HDR schemes) which can be used to define the Flicker Signal, a continuous function of time based on the phase relationship between the light source and exposure window. Analysis of the shape of this signal yields a characterization of the camera’s response to a flickering light source–typically seen as an undesirable susceptibility–under a given set of parameters. Flicker Signal calculations are made on discrete samplings measured from image data. Sampling the signal is difficult, however, because it is a function of many parameters, including properties of the light source (frequency, duty cycle, intensity) and properties of the imaging system (exposure scheme, frame rate, row readout time). Moreover, there are degenerate scenarios where sufficient sampling is difficult to obtain. We present a computational approach for determining the evidence (region of interest, duration of test video) necessary to get coverage of this signal sufficient for characterization from a practical test lab setup.

Author: Robert Sumner, Lead Engineer, Imaging Science
Presented at Electronic Imaging 2020

]]>
https://www.imatest.com/2020/03/describing-and-sampling-the-led-flicker-signal/feed/ 0
Validation Methods for Geometric Camera Calibration https://www.imatest.com/2020/03/validation-methods-for-geometric-camera-calibration/ https://www.imatest.com/2020/03/validation-methods-for-geometric-camera-calibration/#respond Wed, 25 Mar 2020 16:16:02 +0000 http://www.imatest.com/?p=32635

Camera-based advanced driver-assistance systems (ADAS) require the mapping from image coordinates into world coordinates to be known. The process of computing that mapping is geometric calibration. This paper provides a series of tests that may be used to assess the goodness of the geometric calibration

 and compare model forms:

  1. Image Coordinate System Test: Validation that different teams are using the same image coordinates.
  2. Reprojection Test: Validation of a camera’s calibration by forward projecting targets through the model onto the image plane.
  3. Projection Test: Validation of a camera’s calibration by inverse projecting points through the model out into the world.
  4. Triangulation Test: Validation of a multi-camera system’s ability to locate a point in 3D.

The potential configurations for these tests are driven by automotive use cases. These tests enable comparison and tuning of different calibration models for an as-built camera.

Author: Paul Romanczyk, Senior Imaging Scientist
Presented at Electronic Imaging 2020

Presentation

Watch on YouTube

]]>
https://www.imatest.com/2020/03/validation-methods-for-geometric-camera-calibration/feed/ 0
Measuring camera Shannon information capacity with a Siemens star image https://www.imatest.com/2020/03/measuring-camera-shannon-information-capacity-with-a-siemens-star-image/ https://www.imatest.com/2020/03/measuring-camera-shannon-information-capacity-with-a-siemens-star-image/#respond Wed, 25 Mar 2020 16:10:35 +0000 http://www.imatest.com/?p=32655

Shannon information capacity, which can be expressed as bits per pixel or megabits per image, is an excellent figure of merit for predicting camera performance for a variety of machine vision applications, including medical and automotive imaging systems.

Its strength is that is combines the effects of sharpness (MTF) and noise, but it has not been widely adopted because it has been difficult to measure and has never been standardized.

We have developed a method for conveniently measuring information capacity from images of the familiar sinusoidal Siemens Star chart. The key is that noise is measured in the presence of the image signal, rather than in a separate location where image processing may be different—a commonplace occurrence with bilateral filters. The method also enables measurement of SNRI, which is a key performance metric for object detection.

Information capacity is strongly affected by sensor noise, lens quality, ISO speed (Exposure Index), and the demosaicing algorithm, which affects aliasing. Information capacity of in-camera JPEG images differs from corresponding TIFF images from raw files because of different demosaicing algorithms and nonuniform sharpening and noise reduction.

Author: Norman Koren, founder and CTO
Presented at Electronic Imaging 2020

]]>
https://www.imatest.com/2020/03/measuring-camera-shannon-information-capacity-with-a-siemens-star-image/feed/ 0
Verification of Long-Range MTF Testing Through Intermediary Optics https://www.imatest.com/2020/03/verification-of-long-range-mtf-testing-through-intermediary-optics/ https://www.imatest.com/2020/03/verification-of-long-range-mtf-testing-through-intermediary-optics/#respond Mon, 23 Mar 2020 21:57:00 +0000 http://www.imatest.com/?p=32650

Measuring the MTF of an imaging system at its operational working distance is useful for understanding the system’s use case performance. However, it is often not practical to test imaging systems at long distances (several meters to infinity), particularly in a production environment. Intermediate optics (relay lenses) can be used to simulate longer test distances. The Imatest Collimator Fixture is a machine developed for testing imaging systems at specified simulated distances up to infinity through the use of a relay lens and a test chart. The relay lens’s optical properties dictate the required distance between the optic and the test chart, or Collimator Working Distance (WDC), to project the correct simulated distance (SD). This paper provides a method for validating the accuracy of simulated test distances. Successful validation is achieved when the distances at which peak MTF occurs in the real world match the simulated distances at which peak MTF occurs on the collimator fixture, or if both distances are within the depth of field (DoF) of the imaging system in use.

Authors: Alex Schwartz, Mechanical Engineer; Sarthak Tandon, Mechanical Engineer; and Jackson Knappen, Imaging Science Engineer
Presented at Electronic Imaging 2020

Presentation

Watch on YouTube

Related information

]]>
https://www.imatest.com/2020/03/verification-of-long-range-mtf-testing-through-intermediary-optics/feed/ 0
Imatest EI Presentations Now Online https://www.imatest.com/2020/03/imatest-ei-pressentations-now-online/ https://www.imatest.com/2020/03/imatest-ei-pressentations-now-online/#respond Thu, 12 Mar 2020 21:19:19 +0000 http://www.imatest.com/?p=30704 The research papers presented at this year’s Electronic Imaging Symposium (EI 2020) by Imatest engineers are now available.

“It was an honor to meet with the imaging community at EI 2020 and having the opportunity to share our research,” said Henry Koren, Director of Engineering at Imatest.

The presented papers include:

  • Measuring Camera Shannon Information Capacity with a Siemens Star Image by Norman Koren, founder and CTO
    Shannon information capacity can be expressed as bits per pixel or megabits per image, and is a figure of merit for predicting camera performance for a variety of machine vision applications, including medical and automotive imaging systems. This paper introduces a method for conveniently measuring Shannon information capacity from images of the sinusoidal Siemens Star chart.

Note: The white paper, Camera Information Capacity, is recommended as a more accessible introduction to camera information capacity.

  • Correcting Misleading Image Quality Measurements by Norman Koren, founder and CTO
    In this paper, the author reviews several, common image quality measurements that are often misinterpreted in images. Also included are details on how valid measurements are obtained.

  • Validation Methods for Geometric Camera Calibration by Paul Romanczyk, Senior Imaging Scientist
    A geometric calibration provides a mathematical model for the pointing direction of each pixel in the camera. In this paper, the author describes a series of tests that assess the goodness of the geometric calibration. The tests enable comparison and tuning of different calibration models for an as-built camera.

  • Describing and Sampling the LED Flicker Signal by Robert Sumner, Lead Engineer, Imaging Science
    High-frequency flickering light sources such as pulse-width modulated LEDs can cause image sensors to record incorrect levels. Theoretical foundations and the implications for practical testing of “LED Flicker” effect are discussed as well as notes on how to use this model to reduce the burden of large amounts of test data.

  • Verification of Long-Range MTF Testing Through Intermediary Optics by Alex Schwartz, Mechanical Engineer; Sarthak Tandon, Mechanical Engineer; and Jackson Knappen, Imaging Science Engineer
    A method for validating simulated distances from the Imatest Collimator Fixture, which tests imaging systems at simulated distances up to infinity, is presented here. The method compares Modulation Transfer Function (MTF) results between real world image captures and image captures through the intermediary optic at various nominal test distances.

 

 

 

 

 

 

 

 

 

 

]]>
https://www.imatest.com/2020/03/imatest-ei-pressentations-now-online/feed/ 0
Making Dynamic Range Measurements Robust Against Flare Light https://www.imatest.com/2019/07/making-dynamic-range-measurements-robust-against-flare-light/ https://www.imatest.com/2019/07/making-dynamic-range-measurements-robust-against-flare-light/#respond Mon, 22 Jul 2019 23:31:24 +0000 http://www.imatest.com/?p=26418 Introduction

A camera’s Dynamic Range (DR) is the range of tones in a scene that can be reproduced with adequate contrast and good signal-to-noise ratio (SNR). Camera DR is often limited by flare light, which is stray light in the image, primarily caused by reflections between lens elements. Flare light reduces DR by fogging images; i.e., washing out detail in dark areas.It is the primary reason that the DR of cameras (which include lenses) is poorer than that of image sensors, which can be up to 150dB (30 million:1) for recent HDR (high dynamic range) sensors.

In the past 2 years we’ve learned that flare light in test chart images could, in theory, be mistaken for the image of the test chart, resulting in exaggerated DR measurements. The situation became real in mid-2019, when we started seeing actual images where this error occurred. By studying these images, we have developed techniques to guard against exaggerated Signal-to-Noise Ratio (SNR) measurements caused by flare light. We describe these techniques here.

We recognize that some customers may actually prefer exaggerated measurements because they yield very HDR numbers—sometimes over 120dB—that approach the specifications of HDR sensors and look good in marketing materials. The reduced but realistic DR measurements obtained when the effects of flare light are removed may make some of these customers uncomfortable. We will do our best to deal with their objections.

The effects of flare light

Flare light can be illustrated with an image of the XYLA chart—a precision HDR test chart with a linear configuration—that consists of 21 gray scale patches with Optical Density steps of 0.3. The image is from a low-cost “black box” camera.

The upper cross-section plot, made with the Image Statistics module, is taken at the center of the XYLA image. Flare is most obvious in the image and as a decay in the cross-section plot to the left of the brightest patch.

XYLA image (from the same image; the lower is lightened) and corresponding cross-section plots

The lower cross-section was taken outside (below) the active chart image, which is shown lightened above to make flare light more visible. The variation in pixel level on the right side of the chart (x between 800 and 1600) is caused by flare light diffusing from the brightest patches on the left. 

Results of the XYLA image (above) showing strong tone mapping; click image for a full-sized view.

Some notes on this image.

This image has very strong local tone-mapping, leading to an exceptionally low (and not very meaningful) measured gamma of 0.148. The SNR varies in an unusual way because it does not drop monotonically, as it would for a conventional linear image sensor. This indicates that an HDR image sensor with several operating regions was used. 

Because the image had significant barrel distortion, region selection was difficult. The Contrast Resolution chart is much easier to use and provides a better indication of system performance in the presence of tone mapping.

If the flare light were any worse, it could easily have been mistaken for a signal from the chart itself, leading to an false DR measurement.

Flare light can be extremely complex. It can add an offset to the image (often called “veiling glare”), which is difficult to distinguish from a black level offset in the image processing pipeline. Most of the time it is largest near bright patches, then decreases with distance from these patches. The rate of decrease is rarely a well-behaved exponential.

Lens reflections are a major cause of medium-range flare light. An uncoated glass surface (index of refraction ≅ 1.5) reflects R = 4% = 0.04 of the light incident on it. (Remember, a sheet of glass or a lens component has two surfaces.) 

For each glass surface between the surface and the light source, a fraction R of the primary reflection (R2 of the original incident light) is reflected back to the image sensor. This are called a secondary reflection. Since most lens surfaces are curved, this light will be unfocused; i.e., it will tend to fog a portion of the image. 

According to Edmund Optics, the best anti-reflective coatings have R ≅ 0.4% = 0.004 over the visible spectrum (~400-700nm). R = 0.005 may be more realistic for a reasonable range of incident angles. The light reflected back to the sensor from each secondary reflection would be R2 = 0.000025 = 2.5*10-5 = -92 dB (20*log10(R2)). The number of secondary reflections Nsec increases rapidly with the number of components M (groups of elements cemented together, each of which has two air-to-glass surfaces) in a lens: 1 for one component; 6 for two components; 15 for three components; 28 for four components; 45 for five components, etc. For M components,

   \(\displaystyle \text{Number of secondary reflections} = N_{sec} = \sum_{i=1}^{2M-1}i = 2M(2M-1)/2 = M(2M-1))\)

M = 5 components are typical for high-quality camera phones; M ≥12 components is commonplace for DSLR zoom lenses. Overall, lens flare is less severe than the number of secondary reflections suggests because stray light does not cover the whole image; it decreases with distance from bright regions. It’s easy to see why practical camera DR measurements are limited to around 70-90dB, even when sensor DR is much higher.

Image Statistics cross-section of a Contrast Resolution image for an inferior camera, showing spatially varying flare indicated by red arrows

Because the ISO 18844 flare model does not measure the spatially dependent flare caused by lens reflections, it has limited value in characterizing practical system performance.

 

Key takeawaysFlare light is predominantly light in dark regions of an image that diffuses from bright regions. This diffused light can be confused with the actual chart signal (especially with linear charts), resulting in exaggerated (overly optimistic) DR measurements. Moreover, increasing flare light (which could result from poorer; i.e., cheaper, lens coatings) decreases the actual DR by fogging shadow areas of the image, but can lead to increased DR measurements. Hence the need to distinguish artifact signals from flare light from real signals from the chart.

Circular test charts

 UHDR photographic film chart

The test charts recommended by Imatest for measuring DR are transmissive (i.e., backlit) charts with (approximately) circular patch configurations; i.e., those that are not linear (like the XYLA chart, shown above). The two- or three-layer High Dynamic Range Chart, shown on the right, comes in several versions. Because photographic film charts are not manufactured with consistent patch densities, a reference file is required when these charts are used.

High-quality cameras

Until recently, most of the DR images we analyzed came from DSLR or mirrorless cameras that had relatively low flare light. The recent images we’ve seen with severe flare light are from inferior cameras. We have not determined exactly why the flare light is so much worse; it might be due to inferior coatings in the multi-element lenses or less baffling in the barrel of the lens.

Here are examples of results from high-quality cameras. Click on the thumbnails below to view full-sized images.


Results for raw image from high-quality (Canon 90mm Tilt/Shift) lens


Results for jpeg image from high-quality (Canon 90mm Tilt/Shift) lens


Results for raw image from consumer-grade (Canon 75-300mm) lens


Results for jpeg image from consumer-grade (Canon 75-300mm) lens

The four images are for 48-bit RAW (or TIFF derived from RAW) and 24-bit JPEG files captured on the Sony A7Rii camera with two very different lenses:

  • A consumer grade Canon EF 75-300-mm f/4-5.6 lens (original version) set to 80mm, f/5.6. This lens has 15 elements in 10 groups.
  • The very high quality Canon TS-E 90-mm f/2.8 (Tilt/Shift) lens set to f/5.6. Since this lens has 6 elements in 5 groups, it would be expected to have lower flare light than the 75-300.
  • Note that results from the two darkest patches, 35 and 36, are outside the plot because their densities (8.184 and 8.747, equivalent to 163.7 and 174.5dB) are beyond the 160 dB limit of the plot. 160dB is equivalent to a 100 million:1 ratio— far beyond the capabilities of any camera system with a lens in front of the sensor.

As expected, the 90-mm T/S lens has significantly better DR, and the JPEG files had more of a response “shoulder” (an area of reduced slope in the lighter part of the image). But there is one surprise. The DR of the JPEG images is comparable to the raw images—apparently because gamma encoding, which decreases the number of pixel levels in bright regions and increases it in dark regions, extends DR beyond what would be expected for a linear 8-bit (256 level) file. 

Note that in three of the four images above, the low quality DR is lower than the DR from slope; they are very close in the remaining image. We don’t recommend the use of slope-based DR by itself (because it often extends well beyond the region where SNR = 0dB (Signal/Noise = 1); i.e., it includes regions where poor SNR causes noise to completely mask image detail.

Low-cost “black-box” camera

The differences between the DR of the two lenses (medium and high quality) seems to be minor when compared to an image from a low-cost “black-box” camera we recently received. 

Dynamic Range results for low-cost “black box” camera

The low-quality (SNR = 0dB, labeled Low ——— ) DR is measured as 148dB—an astonishingly high number; DR from slope is 66dB—much lower than several quality-based DR measurements (and lower than the slope-based DR for the 90-mm T/S lens). Note that the two darkest patches don’t appear on this plot because their densities (8.184 and 8.747) are beyond the 160-dB limit of the plot. (A 160dB is a range of 100 million to 1—more than expected from any sensor or camera. Put another way, if one photon were to reach the darkest patch, the lightest patch would set the chart or sensor on fire).

To understand what is happening, we need to observe the dark portions of the images from high-quality cameras, shown as extra (X-) or XX-lightened to make detail visible in the darker patches, and compare these to a comparable region in the low-cost camera.


Dark areas of RAW (TIFF) image from Canon 75-300mm (consumer-grade) lens, X-lightened

The RAW image for the six-element (5 groups) 90-mm lens shows distinct regions of decreasing brightness in the 7th row (second from the bottom; patches 27-32), and still shows distinct regions in the bottom row, though brightness no longer decreases. A reflection (possibly from the lens) is visible in patch 30. The JPEG image from the 90-mm lens has more noise. Some may be quantization noise since the JPEG has only 256 levels (0-255). The images from the 15-element (10 groups) 75-300-mm lens definitely show more evidence of flare.

Now, compare these results—especially for the RAW image from the 90-mm T/S lens (left thumbnail)—with the image from the low-cost black-box camera. 

Dark areas of JPEG image from low-cost black-box camera, X-lightened

No patch detail is visible in the bottom two rows (patches 27-36). Instead of decreasing from left to right in each row, the pixel level decreases from top to bottom, remaining relatively constant across the bottom two rows, with banding (the result of 255 levels) clearly visible. This is clearly flare light, not signal from the test chart. Unfortunately the SNR in these rows is quite good because Imatest removes the effects of illumination nonuniformity in noise calculations. (A setting can turn this off, but we generally recommend leaving it on to give better results in the presence of actual illumination nonuniformity). But this signal is an artifact, not the real thing.

This leads us to the inescapable conclusion that the quality-based DR results for the low-cost black-box camera are incorrect, and that we need to detect the patches where flare light overwhelms the signal from the chart image and exclude them from the DR calculation.

Fortunately this is not difficult for DR test charts with a circular configuration, such as the 36-patch DR chart illustrated above (harder for linear charts). Because of the patch arrangement, the patch level stops decreasing when flare light dominates the scene. This is clearly visible in the bottom two rows in the above example, where the image gets darker from top to bottom in these rows—perpendicular to the patch sequence. When the patch brightness ceases to decrease, we can be confident that flare light dominates; i.e., we are outside the camera DR. This is the case for patches beyond the slope-based DR.

For this reason it makes sense to limit all quality-based DR measurements to the slope-based DR (maximum).

In Imatest 5.2, the Options II window (accessed from the button at the lower-right of the Imatest main window) offers a choice of whether to limit quality-based DR to slope-based DR. When the box is checked, the limit appears in the results display.

Dynamic Range (DR) results for low-cost black box camera, with quality-based DR limited by slope-based DR

Key takeaways—Flare light was not an issue with the high-quality DSLR/mirrrorless lenses we tested in the past, but it has become a major factor limiting the performance of recent low-cost lenses intended for the automotive or security industries. We have seen examples of how flare light can improve traditional DR measurements while degrading actual camera DR. 

Our approach to resolving this issue is to limit quality-based DR measurements (the range of densities where SNR ≥ 20dB for high quality through SNR ≥ 0dB for low quality) to the slope-based DR. This works because, for patches beyond the slope-based limit (where the slope of log pixel level vs. log exposure drops below 0.075 of the maximum slope):

  • Contrast is too low for image features to be clearly visible.
  • Signal is dominated by flare light, which washes out real signals from the test chart; i.e., the “signal” is an artifact, not the real deal.

Limiting quality-based DR in this way significantly improves measurement accuracy, and perhaps more importantly, can help prevent inferior, low-quality lenses being accepted for applications critical to automotive safety or security.

 

]]>
https://www.imatest.com/2019/07/making-dynamic-range-measurements-robust-against-flare-light/feed/ 0
Imatest and Furonteer Reduce Camera Intrinsic Calibration Time with Automated Machines https://www.imatest.com/2019/06/imatest-and-furonteer-reduce-camera-intrinsic-calibration-time-with-automated-machines/ https://www.imatest.com/2019/06/imatest-and-furonteer-reduce-camera-intrinsic-calibration-time-with-automated-machines/#respond Mon, 24 Jun 2019 14:00:38 +0000 http://www.imatest.com/?p=26272 BOULDER, CO – JUNE 25TH, 2019 — Imatest, a global image quality testing solution provider, partnered with Furonteer, an automation equipment manufacturer, in early 2019 to provide production machines for geometric calibration of single and multicamera devices.

“Since partnering with Imatest, we have proven a calibration solution that overcomes the technical limitations in the camera calibration process that autonomous camera manufacturers currently face. There is tremendous synergy between our two companies,” said Simon Bae, CEO, and President of Furonteer in South Korea. “With Imatest’s leadership in image quality testing and our leadership in active alignment and automated multicamera alignment equipment for mobile and automotive camera modules, we expect to expand the business cooperation to a variety of applications and establish ourselves as key solution providers in the automotive space and beyond.”

The companies combine two areas of expertise for an all-in-one solution that enables customers to rapidly calibrate cameras to meet internal and 3rd party vision system requirements. As manufacturers strive to make reliable autonomous driving vehicles, the pressure increases for providers to source and accurately calibrate the cameras necessary to build systems that safely guide cars on the road. Imatest’s solution, in partnership with Furonteer, significantly increases the effectiveness of a company’s camera calibration tests to produce images that accurately represent real-world scenes, empowers the company to meet extremely challenging perception system requirements, and reduces the overall costs required to complete camera calibrations.

Henry Koren, Director of Engineering at Imatest, said, “Our partnership with Furonteer fills a gap in the market for camera calibration. It is a rigorous process for customers to calibrate their single or multicamera devices to meet system requirements. By combining our imaging science expertise with Furonteer’s manufacturing prowess, we offer a very accurate and reliable way for customers to complete their calibrations in a production environment.”

Imatest and Forunteer actively work with Tier 1 manufacturers and OEMs to define their unique calibration process and reduce the time to market. For more information on Imatest’s Geometric Camera Calibration Solution, navigate here. To contact Imatest directly about this solution, please contact them here.

 

Imatest, LLC

Imatest, LLC has been empowering customers to produce cameras that exceed their customers’ expectations since 2004 by enabling objective measurements with impartial testing software, equipment, and expertise. For more information, visit Imatest’s website at imatest.com/about.

Furonteer Inc.

Furonteer created its name from a combination of Future, Frontier, and Pioneer because they are a Pioneer of the Future Frontier in technology. Established in 2009, they serve customers with a variety of camera module test equipment for R&D and manufacturing with a focus in automation equipment for active alignment and multicamera alignment. For more information, visit Furonteer’s website at furonteer.com.

 

© Copyright 2019 Imatest, LLC. All rights reserved.

© Copyright 2019 Furonteer. All rights reserved.

]]>
https://www.imatest.com/2019/06/imatest-and-furonteer-reduce-camera-intrinsic-calibration-time-with-automated-machines/feed/ 0
P2020 Automotive Engineering Technology May 2019 https://www.imatest.com/2019/05/p2020-automotive-engineering-technology-may-2019/ https://www.imatest.com/2019/05/p2020-automotive-engineering-technology-may-2019/#respond Tue, 28 May 2019 16:35:03 +0000 http://www.imatest.com/?p=26038 Imatest attended the P2020 meeting on May 13 and 14, 2019 in Ann Arbor, Michigan. Paul Romancyzk, PhD., Senior Imaging Scientist, and Rob Sumner, Lead Engineer, represented Imatest. Paul co-led the discussion on Color Separation within the Image Quality for Machine Vision subgroup.

Imatest was among many industry leaders to attend the working group on automotive imaging standards. P2020 was established in order to address the considerable ambiguity in the measuring of image quality of automotive imaging systems, both for human viewing and computer vision systems.

Image quality plays a crucial role in both automotive viewing and automotive computer vision applications, and today’s image evaluation approaches do not necessarily meet the needs of such applications. Currently, there is not a consistent approach to measuring automotive image quality within the industry

The IEEE P2020 standard aims to fix these deficiencies by connecting industry leaders at all levels of the automotive supply chain, identifying gaps in existing standards, and working to address these by creating a coherent set of key performance indicators by which camera systems and components may be evaluated in a manner consistent with their intended use.

P2020 meetings are held regularly in both Michigan and Germany, considered global centers of automotive technology. If you’re interested in joining the next meeting, see the IEEE website for upcoming meeting locations.

]]>
https://www.imatest.com/2019/05/p2020-automotive-engineering-technology-may-2019/feed/ 0