We are pleased to announce that beginning January 2019, we are offering Imatest branded light panels and boxes. Imatest has responded to feedback from our customers by creating these two new uniform light sources. Our new LED Lightbox and Light Panel offer a wider variety of light intensities and color temperatures including near-infrared channels. We designed them to integrate easily with test fixtures, allow for precise chart alignment, and to serve a variety of testing scenarios.
“We wanted to offer our own Imatest branded hardware that has been vetted and tested by our engineers which enable reliable results when used with our software and test charts. Our goal is to provide solutions for image quality testing that produces the results our customers need to exceed their expectations. The Imatest Light Panels and Lightboxes are made with the same high standards for quality that are found in our charts and software,” says Christian Taylor, Imatest’s Hardware Engineering Manager.
The Imatest LED Light Panel
Imatest’s versatile compact LED light panel for image quality testing.
The Imatest LED Light Panel is a versatile, low-profile, lightweight light source with 90% uniformity in our standard panels, variable dimming, wireless controls, and easy integration with hardware and charts. It is available in a variety of color temperatures including 3100K, 4100K, 5100K, 6500K and near-infrared channels (850 nm or 940 nm), with illuminance levels ranging from 100 to 1000 Lux. It is ideal for testing a range of image quality factors as well as integration with test fixtures.
The Imatest LED Light Panel has a smaller profile than the Imatest Lightbox, which offers more flexibility in building automated fixtures and solutions for both R&D and manufacturing applications. Additional options are available upon request, including 95% uniformity and a minimum intensity level of 30 lux. See the store for more details.
The Imatest LED Light Panel provides a uniform light source with exceptional reliability and adjustability making it superior to other LED or fluorescent systems.
Imatest’s LED lightbox provides high uniformity and mixable light channels.
The new line of Imatest LED Lightboxes expands on our previous offering to include a range of new sizes (up to 1.44 x 1.1 meters) which accommodate a greater variety of testing applications. The Imatest LED Lightboxes have high uniformity and variety of color temperatures including 3100K, 4100K, 5100K, 5500K, 6500K and illumination options through the visible and near-infrared wavelengths. The boxes also feature wireless control and an improved manual control interface. The Imatest LED Lightbox is a transmissive light source with 90% standard uniformity for transmissive charts (available with 95% uniformity in sizes B and C), designed for dynamic range and ultra-high resolution testing. It has added the ability to mix channels together. The standard model offers a continuous range of light intensity levels from 30 to 10,000 lux, with options for 1 lux low-light and high lux options up to 100,000 lux (size B only). USB and WiFi control interfaces are available. Custom size solutions are available; see the store for more details.
The range of sizes available for the Imatest Light Boxes.
Imatest partnered with Shenzhen Polytechnic University to support the digital image quality testing curriculum at the university. With the expansion of their image quality program, Imatest sponsored and authorized the development of a state of the art research and education lab.
The lab and ceremony
On October 31st, 2018, Henry Koren, Director of Engineering, and Ian Longton, Imaging Science Engineer joined Shenzhen Polytechnic Executive Vice Presidents of Communication Engineering, Dean He Yuhua and Yang Yunyan, Vice President Zhang Xuilang, and Colorspace General Manager Nick Liu and colleagues for the official unveiling ceremony.
At the ceremony, Dean He Yuhua introduced the development of the college program and the opportunities it offers. He also discussed the importance of cooperation with international image quality testing companies for integrating production and education. The ceremony was held to announce the establishment of Shenzhen Polytechnic’s new lab.
The high-level imaging science laboratory is the first to be built in cooperation with Colorspace and a university. The image quality testing industry is rapidly changing, and the University’s lab will serve as a space for students to learn about digital image quality, as well as cultivating new technologies within the industry.
At the ceremony, Henry introduced new developments in imaging science technology, standards, and their applications for many industries. Imatest is planning in-depth cooperation with the college to provide localized services and support for our corporate customers in South China.
Shenzhen Polytechnic will use this new lab for its digital image quality testing courses with Imatest products integrated throughout the lab. Through our ongoing partnership, Imatest will sponsor and support facility by providing technology and charts to support the curriculum. At the completion of the course, students who meet the satisfactory level of image quality and software operation knowledge will earn a certification.
Are you interested in becoming Imatest certified? We offer training classes around the world.
Join Imatest at the Electronic Imaging Symposium from Jan. 13 – 17 in Burlingame, California USA. Explore the entire imaging science ecosystem, from capture through image processing to how we and our surrogate machines see and interpret images. Henry Koren and Norman Koren will be presenting on the following.
Reducing the cross-lab variation of image quality metrics
As imaging test labs seek to obtain objective performance scores of camera systems, many factors can skew the results. IEEE Camera Phone Image Quality (CPIQ) Conformity Assessment Steering Committee (CASC) working group members performed round-robin studies where an assortment of mobile devices was tested within heterogeneous imaging labs. This paper investigates how the existence of near-infrared energy in light sources that attempt to simulate CIE illuminants can influence test results. Numerous other impacts, including the influence of opal diffusers used for uniformity testing, how test scene framing can alter white balance and exposure, and how chart quality and texture frequency distribution can skew results. We introduce a test procedure which is intended to reduce intra-lab variability and a method for assessing an independent lab’s competence in conforming with the IEEE testing standards.
Date/Time: Tuesday January 15, 2019, 3:50 – 4:10
Location: Grand Peninsula Ballroom E
Compensating MTF measurements for Chart Quality limitations
Objective measurements of imaging system sharpness are typically derived from test chart images. It is generally assumed that if testing instructions are followed—if the chart print quality is fine enough and the chart magnification (on the sensor) is low enough— test chart quality will have little impact on the overall measurement.
This assumption may not be valid when extremely high-resolution cameras are tested with standard charts or when smaller than optimum test charts are used because of laboratory or production line space limitations.
MTF compensation is applied by dividing the measured system MTF by chart-projected MTF as a function of the sensor.
Measurements made under different conditions— with different test charts (transmissive as well as reflective) by different people in different labs— are more consistent.
The megapixel suitability of test charts is increased by a factor of approximately 2 (1.4x linearly).
The key advantage of MTF compensation is that measurements are more accurate (in an absolute sense).
This book has been the foundational text for the study of digital image processing for over 40 years. It is suited for those with prior background in mathematical analysis, vectors, matrices, probability, statistics, linear systems, and computer programming.
Part of The Wiley-IS&T Series in Imaging Science and Technology, this book contains basic information and approaches for the use of subjectively correlated image quality metrics and outlines a framework for camera benchmarking. The authors show how to quantitatively compare the image quality of cameras used for consumer photography.
Flare occurs to some degree in all lenses and optical systems. Stray light that enters the lens is reflected among the optical components and fogs the image. Below is an example of very bad lens flare.
An example of bad flare (right).
In these images, we can see how Flare is showing light in the image where there should be none. Flare in optical components is a major limiting factor for a systems overall dynamic range. Sensors often can boast dynamic range >120 dB. In practice, the system cannot get close to that range because of the effects of flare. For more information, please read our documentation on Veiling Glare (Flare).
Setting up the NIR camera test
For this test, we will be using the same Raspberry Pi camera and Imatest light source from our previous post. The major differences are the chart and image processing. The ISO 18844 chart features numerous black dots arranged in an X pattern according to the standard. The chart is made out of clear polyester film and the dots are made from Acktar black to prevent reflections that would ruin the test.
The other major difference for this test from our previous one is the change in processing. The image needs to be linear. Nonlinear processes such as auto white balance and jpeg compression can influence the results if enabled. Fortunately, it is possible to get raw sensor data from the Raspberry Pi Camera V2. This recipe was used to capture a raw image from the Pi.
Analyzing the NIR image
Images of the flare chart were captured under two lighting conditions; visible and NIR (850 nm).
We then analyze each image with the Uniformity module in Imatest Master 5.1.9. In the Uniformity Settings dialog we want to select “ISO 18844 flare method C: Linearize with Color Space”. For this analysis, we can ignore the rest of the options.
Imatest will display several plots related to uniformity but we are only interested in the ISO 18844 flare plot for this test. The plots for each of our measurements can be seen below.
ISO 18844 Flare Visible light plot
ISO 18844 Flare NIR light plot
The plots show the % flare for each dot according to the 18844 calculation as well as the Mean Flare at the bottom. From the plots, we see that Mean Flare for the visible light sources is 5.2% and 13.6% for the NIR.
Similar to our previous test on sharpness, there is a drastic change in quality when the wavelength of light is varied. The ISO 18844 Flare increases more than 2.5 times in the NIR versus the visible. The expectation then is the NIR camera has a diminished dynamic range due to the increased flare. In applications such as security or automotive cameras, the diminished dynamic range could significantly decrease the effectiveness of the system to detect obstacles or intruders. The large change in quality shows how important it is to construct a test environment that matches the application of the imaging system. This is especially true when the system is multi-purpose, such as a security camera with a day (visible) and night (NIR) modes. A good follow up to this test would be to measure the dynamic range of the system in either lighting condition.
Having trouble with your Imatest test results? This article explains the five most common pitfalls in image quality testing, and how to resolve them.
You’re using a chart with overly high contrast, like the obsolete ISO 12233 test chart.
Your chart may have too high contrast edges. The most common example we see of this is the obsolete ISO 12233 test chart, although this can be the case with any test chart. If you are using the ISO 12233 2000 test chart, you should not be.
ISO 12233 test chart
The ISO 12233 2000 standard test chart has a very high contrast ratio (the white is very white and the black is very black) that can cause clipping in signal and lead to invalid MTF values. This is because the contrast of the chart often saturates the sensor, reporting higher MTF results than there might actually be. Therefore, we recommend using a chart with a 4:1 contrast ratio as revised in the ISO 12233 2014 standard, such as the ISO 12233 2014 chart or SFRplus chart. The lower contrast ratio produces more accurate measurements by providing MTF measurements that do not clip.
When you’re testing dynamic range, you need to make sure you’re using a dynamic range chart which has a higher dynamic range than your camera to ensure you’re measuring the camera and not the chart. The common pitfall here is that people tend to use tonal step charts which have lower dynamic ranges than their sensors, such as the old Kodak Q14 or X-rite colorchecker. For example, if you are using a typical matte reflective chart which has a dynamic range of 48dB, and you’re testing a camera which has a dynamic range of 80dB, then the highest result you could get is 48dB. This is inaccurate.
However, if you’re using a chart with 120dB, you’ll get a much more accurate reading because it will measure the camera and not the chart. Before you test, you should check to see that your camera is not exceeding the dynamic range of what your chart is rated for. Once you have verified this, you will know which dynamic range chart to select. Our 36-Patch Dynamic Range chart is the most popular chart we sell with densities exceeding, 50db, 100db, and 150db.
Verifying the quality of your dynamic range chart.
The usage of linear dynamic range charts such as the Stouffer T4410 or Xyla target causes many problems including difficulty selecting dark regions that do not have visible contrast (especially using distorted lenses) as well as the linear density slope which interferes with the radial non-uniformity of the imaging system. The ISO standard dictates that each patch should be the same distance from the center and charts that have radial layouts of patches are much less impacted by light falloff. Also, targets which are very dark are not representative of more luminous scenes and overlook the impact of flare on dynamic range which leads to higher dynamic range performance, which will not hold up when the camera is exposed to a scene which is not mostly blackened. Imatest’s 36-patch targets include an option for a “DarkWorld” mask which can produce both dark and gray scenes using a single target.
Statistics from an image of XYLA-21: dark regions
are difficult to select.
If you need help determining which chart is best suited for you, please email us at firstname.lastname@example.org.
Your chart quality is poor.
Similar to the dynamic range chart pitfall, when testing the sharpness of your camera system, you need to ensure the resolution of your chart is high enough for the device you’re testing. If your reflective or transmissive chart’s MTF is too low the accuracy of your camera resolution measurements will be limited by the resolution of the chart.
The culprit is print quality. The print process can affect the results of your test, especially at close distances or with high-megapixel cameras. For example, using a chart printed on a basic laser printer to test a high-resolution mobile phone camera, could reveal inaccurate results, shown by the steps in the MTF graph.
Example of poor chart quality (left) and good chart quality (right).
Avoid this pitfall by using a high-quality chart on higher resolution substrates such as inkjet or film, or by applying MTF Compensation to compensate for the chart quality. To ensure the highest quality, we encourage you to get charts printed from Imatest. If you don’t know what quality chart you need, contact email@example.com.
You’re not filling the camera field of view.
When testing, it’s important to ensure you’re completely filling the camera field of view with the test chart. If the chart you are using does not include measurement regions near the corners of your field of view, your analysis will not be comprehensive and you may overlook problems with MTF or signal loss at the extents of the image plane. When measuring distortion, the full field of view can be extrapolated from points available within the center, but this is less accurate than having real points to measure. The following video helps illustrate how the field of view affects your setup.
This is particularly challenging for cameras with wide or ultra-wide fields of view where planar targets can’t fill the FoV. We offer specialized test equipment to measure resolution and uniformity of ultra-wide devices.
You’re not testing at the right focus distance.
Test distance is calculated by focal distance; in other words, the target needs to be set at the distance(s) in which it is in focus. The common pitfall we see is not testing at the right focal distance. A comprehensive test regimen will involve testing between the focal distance and the hyperfocal distance. You can’t always move closer to a target in order to fill the field of view, as this can fall below the minimum focal distance of your lens – or below the distance which a chart of a particular does not negatively impact measurements.
This chart is not at a distance which fills the lens field of view while in focus.
The chart needs to be within a distance to fill the lens field of view in focus. You may need to test at more than one focus distance if you have a lens with variable or automatic focus so that you can validate optimal focus across the range of focus of your device.
To find the minimum focus distance, check your lens specifications or contact the lens manufacturer. To find the maximum focus distance, or hyperfocal distance, see this hyperfocal distance calculator.