Beijing, China Imatest Training Course – November 2019

Imatest in Beijing, China

Imatest representatives will visit Beijing November 12-14, 2019 to offer a free information seminar and a paid two-day training course to professionals using or considering Imatest software to improve their image quality testing processes.

Two-Day Training Course

image (2)

The training course on November 13 & 14 offers attendees insight on the full capabilities of Imatest software in both research development and manufacturing environments. Training starts at 9:00 and will end at 17:00 – 18:00, depending on questions. Location is TBD.

View the detailed Training schedule.

Sign up online or contact a reseller in your area.

Register Online

 

Free Information Seminar

If you are interested in finding out more about how Imatest software can improve your image quality testing, we encourage you to come to our free information seminar on November 12th. 

Please register if you are interested in attending.

 

We look forward to meeting you in person.

 

Read More

San Jose, CA Imatest Training Course – November 2019

Imatest in San Jose

Imatest representatives will visit San Jose, CA, November 6-7, 2019, to offer a paid two-day training course to professionals using or considering Imatest software to improve their image quality testing processes.

 

Imatest Training Course - San Jose, CA November 6-7, 2019

 

Two-Day Training Course

The Training Course on November 6 & 7 offers attendees insight on the full capabilities of Imatest software in both research & development and manufacturing environments.

Training will run from 8:30 am to 5:30 pm each day, depending on questions. The event location is TBD.

Visit our training page to view a detailed training schedule.

Group Discounts

Enroll 2-4 attendees to receive 20% off per attendee.

Enroll 5 or more attendees to receive 30% off per attendee.

 

Register Online

 

Read More

San Jose, CA Imatest Training Course – May 2019

Imatest in San Jose

Imatest representatives will visit San Jose, CA, May 1-2, 2019, to offer a paid two-day training course to professionals using or considering Imatest software to improve their image quality testing processes.

Two-Day Training Course

The Training Course on May 1 & 2 offers attendees insight on the full capabilities of Imatest software in both research & development and manufacturing environments.

Training will run from 8:30 am to 5:30 pm each day, depending on questions. The event location is TBD.

Visit our training page to view a detailed training schedule.

Group Discounts

Enroll 2-4 attendees to receive 20% off per attendee.

Enroll 5 or more attendees to receive 30% off per attendee.

 

Register Online

 

Skyline photo by Timothy L. Preen

Read More

Why You Should Renew Your Support

Is Imatest support worth renewing? In this post, we will explain what Imatest support includes, and why you should renew support on your Imatest software.

 

What is support?

Maintaining current support on your Imatest license gives you access to two things:

  1. All updates and new versions released for the software during the support period
  2. Access to our imaging science engineers for questions and assistance with the software and with your projects

 

Why should you utilize our support?

There are several reasons why our customers find value in having support:

  1. Stay current with standard and industry requirements: Our engineers are continually adding and improving features based on advancements in standards and KPI measurements, such as ISO imaging standard groups, IEEE P2020, and IEEE CPIQ.
  2. Utilize the latest capabilities: We release over 40 software updates each year that enhance usability, add tests, and fix bugs. Our updates include customer-requested features, chart feature measurements, improved calculations, new modules, and updated metrics.
  3. Leverage our engineers’ diverse knowledge of imaging science to solve your challenges: Contact our knowledgeable Imaging Science Engineers to help solve any challenge you are facing with your project and analyses.

 

A Historical Timeline

Historical Software Update Timeline

 

 

Updates Applicable to a Variety of Industries

Most of our 50 new features are applicable to several industries.

 

industries supported

 

Features Added in The Past Year

Highlights from our latest release, Version 5.1 (release date July 2018):
  • MTF CompensationCompensate for chart MTF to perform accurate high-resolution or close-distance sharpness tests. MTF Compensation reduces the impact of the chart on measured imaging system MTF. 
  • Geometric CalibrationGeometrically calibrate single or multi-camera devices obtaining camera intrinsic and extrinsic matrices for accurate machine vision and ADAS applications. A geometrically calibrated device provides a mapping between pixel coordinates and real-world position.
  • Improved Distortion CalculationsImatest Checkerboard and SFRplus modules now support an enhanced set of distortion models and provide faster and more accurate distortion coefficient and center of distortion estimations.
  • Direct Image Acquisition for Sony SensorsThis new feature allows Imatest to acquire images directly from Sony AYA. This eliminates the need to save images to file and load them individually into Imatest for analysis, saving customers valuable time.

 

Highlights from Version 5.0: (release date September 2017):
  • Database: Collect important information regarding image capture conditions, device state, and EXIF metadata into Imatest’s new database system. Easily keep track of numerous testing sessions in a convenient web application.
  • Test Manager: Process a collection of images with automated analysis routines. Use either free-form test plans to assign relevant test categories to each imported file or use predefined test plans to run standard testing procedures such as CPIQ analysis.
  • Reports: Efficiently analyze data outputs for preferred metrics. Generate reports in interactive HTML and printable PDF formats.
  • Arbitrary Charts: Define your own chart layout to combine image quality factors for analysis on one chart. Significantly reduce the number of images needed to complete the testing process.
  • Contrast Resolution: Measure and visualize the distinguishability of low contrast objects within larger fields, over a wide range of brightness.
  • ISO 18844 Flare: Automatically analyze flare based on the new ISO 18844 standard within the Uniformity module.
  • Image Acquisition: Imatest 5.0 provides a direct interface with several camera capture boards for automatic image acquisition in Imatest Master.

 

See our ChangeLog for a complete list of new features and updates.

 

Renew Your Support

We make improvements to our software almost every day. Take advantage of them and leverage our expertise to complete your projects.

RENEW YOUR SUPPORT

 

You may also submit a PO to sales@imatest.comOr if you would like to talk to one of us before purchasing, please email us at sales@imatest.com.

Read More

New IEEE P2020 Automotive Image Quality White Paper

Learn more about the development of standards for automotive camera systems.

 

Overview

The IEEE-SA P2020 is a working group for automotive imaging standards. Their goal is to define a set of standards to resolve the current ambiguity in the measurement of image quality in automotive imaging systems. Generally, today’s image evaluation approaches do not adequately address the unique needs of either the human or computer-vision based automotive applications; therefore IEEE-SA P2020 is working with people in the field, understanding the gaps in current standards, and creating a coherent set of key performance indicators by which automotive camera systems might consistently be evaluated.

 

Our Involvement

Since the beginning of this working group, Imatest’s engineers continue to be heavily involved in the IEEE P2020 standard for automotive camera systems. The two areas of focus are Contrast Detection Probability and Flicker Measurement. We are actively working with major image sensor manufacturers to assess Contrast Detection Probability as a practical test measurement for evaluating the amount of contrast information present at various parts of the imaging chain. Additionally, we are the principal contributors towards the Flicker Measurement subgroup, and are preparing a publication to describe the effect further and why it is an essential consideration for ADAS and automated driving systems.

 

The Future

There is much to come as IEEE P2020 leads the creation of standards for automotive camera system testing. As automobiles evolve, autonomous vehicles mature, and the car becomes an artificial-intelligence platform connected to everyday life, it will be crucial to adapt and define proper evaluation metrics for the multi-sensor, multi-camera, and multi-application systems on an automobile. The IEEE P2020 working group involves leading camera and sensor manufacturers, leading automobile manufacturers, and leading imaging science engineers, and will leverage all the various backgrounds and experiences to create and adopt this standard for all automobile and part manufacturers. Download the most recent publication below from IEEE P2020.

 

If you have any questions for us on how this standard can help your projects, please email Ian Longton at ian@imatest.com.

 

DOWNLOAD THE WHITEPAPER

 

Distribution:

All rules about how the material can be used, copyrights, legal caveats etc. are contained within the document. It must only be shared in full so that these details are clearly indicated.

 

To stay informed on the work and progress of this standard, you can visit our standards page or look at the P2020 working group site directly.

Read More

A customizable image quality solution for Security camera testing

We now offer a complete, customizable image quality testing solution for Security camera systems to provide our customers with an easy, effective way to outfit their labs. While working with some of the top security camera manufacturers, our engineers have compiled all of the necessary lab materials in a convenient package. The package includes options for software;  reflective, transmissive, tunable, and ultra-wide light sources; test fixtures for a variety of fields-of-view and focus distances; and applicable test charts to analyze crucial image quality factors.

Software

The Security package includes two software options, Master and Ultimate. The Master version, designed for R&D testing, is a robust analysis tool to help you measure any and every image quality factor of your camera system. The Ultimate version combines Master and IT, an automated production line testing version, and is a cost-effective solution for customers who manage R&D and production testing. Read more about our software version and license options here.

Test Fixtures

We include several test fixtures that help test varying fields-of-view (FOV) and focus distances properly. For ultra-wide testing with fields-of-view up to 270˚, our Ultra-Wide Test Fixture works perfectly to help analyze image quality factors such as sharpness and chromatic aberration using our eSFR ISO and SFRreg charts. If you have long-range testing requirements, we’ve included several collimator lenses and target projection systems to help you to test at or beyond your focusing distance. To automate and provide more flexible positioning, add one of the robotic automation fixtures from our partners Sofica and IQL.

Light Sources

Choose from a variety of reflective, transmissive, tunable, and ultra-wide lighting system options. We have included our most widely used reflective light source; the Kino Flo FreeStyle 31 LED DMX for its adjustability, portability, and high-illumination output. To test dynamic range, and high-resolution systems, select a uniform, transmissive LED light box with Lux levels ranging from 10,000 – 100,000. The brightest lightbox enables you to achieve IEC 62676 compliance. For the most adjustability in spectral and Lux levels, and for ultra-wide field-of-view uniformity testing, choose one of our tunable light sources such as the wide-FOV version of the Gamma Scientific RS-7.

Test Charts

To make your selections easier, we have included the most appropriate test charts to meet a variety of requirements for security cameras. For reflective resolution needs, select from our ISO 12233:2014 compliant multi-feature test charts, a Checkboard chart, or sinusoidal siemens star pattern chart. For use with the transmissive, uniform light sources and dynamic range and high-resolution testing, choose to use our ISO 12233:2014 chart printed on High-Precision LVT Film, and our Ultra-High Dynamic Range test chart.

View the package, the options included, and receive a quote by visiting our store.

Check It Out Now

Of course, we know that not everyone’s requirements are the same, so if the package doesn’t seem to be a good fit for you talk to a solution expert and we will customize a solution for your unique needs.

Read More

San Jose, CA Training Course

This course has ended. Please see our schedule to see and register for future training.

Imatest in San Jose

Imatest representatives will visit San Jose September 5-6, 2018 to offer a paid two-day Training Course to professionals using or considering Imatest software to improve their image quality testing processes.

Two-Day Training Course

The Training Course on September 5 & 6, 2018 offers attendees insight on the full capabilities of Imatest software in both research & development and manufacturing environments.

Training will run from 8:30 am to 5:30 pm each day, depending on questions. The event will be located in the Larkspur Landing Hotel in Campbell.

Visit our training page to view a detailed training schedule.

Group Discounts

Enroll 2-4 attendees to receive 20% off per attendee.

Enroll 5 or more attendees to receive 30% off per attendee.

 

Skyline photo by Timothy L. Preen

Read More

Introducing Imatest 5.1

Imatest 5.1

 

Our engineers are continually adding new features and updating Imatest software to provide you with the best analysis tool on the market. In our most recent release, our team has added several new modules to provide you better analysis tools. Here are the top three new features, and why should upgrade to Imatest 5.1:

 

  • Compensate for chart MTF to perform accurate high-resolution or close-distance sharpness tests

MTF Compensation reduces the impact of the chart on measured imaging system MTF.

 

  • Geometrically calibrate single or multi-camera devices obtaining camera intrinsic and extrinsic matrices for accurate machine vision and ADAS applications

A geometrically calibrated device provides a mapping between pixel coordinates and real-world position.

 

  • Direct image acquisition for Sony sensors

Imatest now supports direct image acquisition from Sony sensor development boards through the Sony AYA software tool.

 

Learn More

 

Read More

Why is it Important to Test in Low-Light?

It is important to test your camera system in environments which reproduce lighting conditions similar to where you intend to use the camera in the real world. Failure to test a camera under low light conditions may lead to overstating the camera’s performance.

Image sensors collect light (signal) into pixel wells, then convert the resulting analog voltage levels into digital numbers. Dark current is where electrons are released from thermal activity which becomes indistinguishable from electrons released via photoresponse. The dark current that exists in uncooled image sensors leads to dark noise. At low light levels, exposures are longer to gather more light, which gives more time for dark-current electrons to be gathered. This leads to dark noise and read noise representing a larger portion of the overall response, which reduces the signal to noise ratio (SNR). Low-light conditions are most challenging for higher resolution sensors with small pixel pitches where the particle and wave nature of light can seriously impact the performance of your camera.

The definition of “low light” depends on the application. For mobile devices (compact camera modules) low light is defined by the IEEE CPIQ standard as 25 lux, which resembles a dimly lit indoor space representative of the worst cases where people expect their phones to function properly. As mobile devices get better in low light scenarios, customer expectations may shift. For security or automotive industries, the levels are much lower based on how dark the outdoor environment may get[1]:

 

Condition Illumination (Lux)
Sunlight 107527
Full Daylight 10752
Overcast Day

1075

Very Dark Day 107
Low-Light (CPIQ) 25
Twilight 10.8
Deep Twilight 1.08
Full Moon 0.108
Quarter Moon 0.0108
Starlight 0.0011
Overcast Night 0.0001

In order to achieve light levels similar to the darkest scenes, most lab lighting setups that contain fluorescent or LED sources may not be able to be directly dimmed to very low levels, so the dim light levels may be achieved by a variety of methods shown in the following table. Ultra-low-light levels can be achieved by combining these approaches:

Method Light Reduction Notes
Increasing distance (d) Between 1 / d and  1 / d2 Difficult to move large lighting setups, labs have limited available space
Reflecting off Munsel N5 painted walls 0.18 Most labs are painted with these sorts of walls, but rotating lights back and forth may be difficult to perform repeatably without a mechanical motion stage.
Reflecting off black painted walls

0.05
Fresnel reflection from beam splitter

0.04

Apparatus could impinge FoV and would also require repeatable rotation of lights.
Imatest Low-light filter (mask + neutral density filter) 0.0125 Manually attaches to these LED lights

Eliminating additional sources of light within your lab is also important: Putting felt around doors to block light from other rooms, covering exit signs, covering indicator LEDs and power strips and installing baffling and light traps to block unwanted reflections.

Note: Verifying the low light levels can be challenging as many illuminance meters do not go to extreme low light levels.

Once you can reproduce low-light levels in your lab, you can perform tests to answer a number of questions about your system performance:

  • Does the radiometric calibration for black level successfully reduce the effects of dark current without also eliminating useful signal that might impact the dynamic range of your system?
  • Does the black level compensation work across the range of nominal operating temperatures that your sensor will experience?
  • Does your ISP react appropriately to dark scenes by increasing the sensitivity (gain)?
  • How consistently does your autofocus system (if you have one) perform under low light?
  • What happens to moving objects captured under low light conditions?
  • For security cameras with near-infrared ‘night-vision’ support: does the Infrared cutoff filter engage and disengage at the appropriate light level?
  • Do the applied gamma curve(s) and tone mapping help to boost the perceived quality of the dark scene or assist the observer’s ability to recognize objects in the dark?
  • Does the noise reduction strike the right balance of reducing visual noise without also blurring the appearance of relevant textures (skin and foliage) within the image?

A comprehensive tuning and testing regimen will involve performing a full battery of objective tests in a wide range of lighting conditions. By increasing the range of light levels you can reproduce in your lab, you can provide the most challenging condition in which you can validate the performance of your camera system.

Photo by Tuur Tisseghem from Pexels

Read More

How to Test Dynamic Range:
A step-by-step use case with the Pixel 2 XL

Outline

Intro   |   Setup   |   Capture Procedure   |   Analysis   |   Results   |   Conclusion

 

Google Pixel 2 XL

 

In this post, we will be using the Contrast Resolution Chart and Imatest Master to measure the dynamic range of a Google Pixel 2 XL. The dynamic range of a camera is the reproducible tonal range in an imaging system. Put simply, it is the range between the darkest black and the brightest white of an image and is typically measured in decibels (dB). It is an important image quality factor in many applications from machine vision to mobile cameras and more recently, automotive camera systems. In this use case, we will be evaluating both the HDR+ (default) and HDR off modes of the Pixel 2 XL; however the procedure can be used to test any camera system’s dynamic range. You can read more details about Imatest’s dynamic range test solutions.

 

Setup

Testing a camera’s dynamic range can be difficult and requires careful setup. In this section, we will review all of the relevant components of performing a sufficient dynamic range test. 

 

Chart

 

How to Test Dynamic Range - Contrast Resolution Chaart

Contrast Resolution Chart

 

One of the major components to testing dynamic range and the first consideration is the chart. There are a variety of dynamic range test charts available but the one we will be using for this test is the Contrast Resolution chart. There are several unique features to this chart that make it ideal for testing a system like the Pixel 2 XL. The first is the radial arrangement of the patches. This is important because it helps control for lens fall off. Second is the dynamic range of the target itself. The chart must have a higher dynamic range than the system or else you end up measuring the chart and not the camera. The Contrast Resolution chart has a dynamic range of ~95 dB which is sufficient for almost all systems with a lens. The third and most important reason for choosing this chart is the Contrast resolution dynamic range calculation. The inner light and dark gray patches on this target allow for a more robust calculation of the dynamic range of tone mapped images; which can adversely affect the quality of other dynamic range calculations.

 

Lighting

 

ITI Lightbox with Contrast Resolution Chart

 

The next component to consider is the light source. For our test, we will be using the 10,000 lux ITI Lightbox 5100K. This lightbox is a good choice for our purposes because of its uniformity (>95%). The analysis of the chart assumes a constant amount of light being transmitted across and through the chart. The contrast resolution dynamic range calculation works because the chart has known density steps between the patches. If the light source has low uniformity than the assumption is not true and the results can be impacted.

 

Environment

 

Test setup

 

The third consideration is the environment you will test in. It is important to test dynamic range in a sufficiently dark environment limiting the amount of stray light that might enter the camera system as much as possible. The extra light entering the system can increase the amount of flare light (sometimes called veiling glare). Flare is the stray light that fogs images and is caused by reflections between surfaces of lens components and the inside barrel of the lens. Stray light can cause reduced contrast in dark parts of the image which in turn can make for erroneous dynamic range measurements. See our recent study: “Measuring the impact of flare light on dynamic range” for more information. Our testing environment will have only one controlled light source to illuminate the chart and no other sources of light (e.g., overhead lighting, LEDs on electronics, etc.).

A common source of erroneous measurements due to the testing environment are reflections on the chart itself. Because the darkest patches of the target let through such little light, it is easy for a non-negligible amount of stray light to be reflected off the front side of the chart. This extra light will make it seem as though more light is present in the patch area than is actually being transmitted through the chart and will adversely influence the analysis. To mitigate the reflections, we use a custom shroud to block surfaces that can reflect light from the lightbox. Unfortunately, the camera itself is the most likely culprit for unwanted light due to the potential for light to bounce off of it. The Pixel 2 XL in our case, is no exception. The particular model we are using has a white back which reflects lots of light back at the chart’s front surface. Since we are aware of this, we have taken extra care to shield all of the phone except for the lens from the light to stop any unwanted light from bouncing back at the chart.

 

Example of a development board reflection off the front of the chart

 

More on the shroud

Depending on the system, and the environment of the test, a shroud may not be necessary. However, it provides a consistent lighting environment for every test and should be considered for benchmarking and comparing different systems. For more information on reflections and shroud materials, see this post. For information on custom testing environments, please contact sales@imatest.com .

 

Capture Procedure

 

Contrast Resolution chart viewed through the native camera app

 

Ten images were captured with the Pixel 2 XL with the above setup. The phone was set to HDR+ mode. For each image, the exposure was adjusted on the device by tapping the area on the screen where the upper gray background of the chart was present (directly to the left of the second row). We did this to ensure similar exposure in all of the images. The acrylic shroud with matte black interior was placed around the lightbox with the phone placed as close as possible to the opening on the shroud. For this test, the lightbox was set to the 5100K color temperature setting and minimum brightness (~30 lux). We chose these settings because they are representative of a realistic low-light environment in which the camera phone may be used. Please note, the light settings could vary greatly depending on the application. Once enough images were taken, we inputted them into our software, Imatest Master, for proper analysis.

 

Analysis

The following analysis of these images was done in Imatest Master 5.0.

Each image was analyzed in the Multitest module. Opening the Multitest Setup, we made sure to check that “3. Contrast Resolution” was selected for the chart type.

The plot we were most interested in was the Noise/SNR plot. Under this section in the setup dialog, we selected “3. SNR dB vs. Inp Density (RGBY)” and “3. Exposure in DB (-20*target density) DR in dB” from the drop downs. We also checked “Image plot” so we could visualize the results on the images themselves. See here for more information analyzing the Contrast Resolution chart.

The data was saved in JSON format and the relevant metrics were then parsed and averaged. The data we are interested was saved in the field “Contrast_Resolution_Dyn_Rng_dB”. The field contains 4 values, the Contrast Resolution dynamic range in decibels at SNR = 0, 6, 12, 20.

 

Results

The plot below shows the Contrast Resolution Dynamic Range for the two modes averaged over ten captures. The standard error is included. We aggregated this data from several runs of the Multitest module in Imatest Master. As you can see in the plot below, there was a significant increase in dynamic range when using HDR+ mode versus HDR off.

 

Chart comparison of the HDR-off vs. HDR+ results

 

NOTE: If you are using Imatest software, this information can be viewed for individual images using the Noise Plot in the Multicharts module.

Since we checked “Image Plot”, we can also visualize the results on the image of the Contract Resolution chart. The two plots below show an exaggerated image plot so that the noise in each patch can be visually examined.

 

HDR Off Image Plot (Constant xyY)

 

HDR+ Image Plot (Constant xyY)

 

A notable difference in the plots can be seen in patch 11 (isolated images below). We can see that in the HDR+ plot (right), the details of the inner gray patches are still visible where they are not in the HDR off plot (left).

 

Patch 11 visual comparison (Left: HDR Off, Right: HDR+)

 

Conclusion

By following this procedure, using the Contrast Resolution chart and Imatest Master, we were able to analyze the dynamic range of our camera effectively. As we can see in the above diagrams, the summary plot results indicate that the HDR+ mode increased the dynamic range in the images by 5 dB on average. We are also able to visually see the increased range by looking at patches in the Image Plots view.

Ideally, we want to tailor our test setup as much as possible to the application being tested. In this particular case, the subject of our testing is the different HDR processing of the Pixel 2 XL. We have limited information on the specific algorithms being used when HDR is turned off versus HDR+ and their impact on image quality under different lighting situations and with different target subjects, so we are limited in the conclusions we can draw. Thus, the results reported above do not represent all use cases. A more exhaustive study would need to be done to produce results that would reflect the broad range of possible use cases.

For more information, or to learn more about how to properly set up your testing environment, talk to a solution expert.

 

Read More