by Henry Koren

1. Why care about the quality of your cameras?

The task of computer vision (CV) involves analyzing a stream of images from an imaging device. Some simple applications such as object counting may be less dependent on good camera quality. But for more advanced CV applications where there is limited control of lighting & distance, the quality of your overall vision system will depend on the quality of your camera system. This is increasingly important when an error made by the vision system could lead to a decision that impacts safety. Along with proper optimization of a CV model, ensuring that that model is fed by imagery from a high-quality camera system is critical to maximizing your system’s performance.
(more…)

Security researchers disclosed the following vulnerabilities in the Apache Log4j Java logging library:

• CVE-2021-44228: Apache Log4j2 JNDI features do not protect against attacker-controlled LDAP
  and other JNDI related endpoints
• CVE-2021-45046: The fix for CVE-2021-44228 was incomplete in certain non-default
  configurations.
• CVE-2021-45105: Apache Log4j2 Context Lookup features do not protect against uncontrolled
  recursion from self-referential lookups in certain non-default configurations

No Imatest software includes the affected versions of Log4j, no dependency used, such as the MATLAB compiler runtime includes an affected version either.

Internal Imatest systems which included Log4j were promptly patched when the vulnerability was discovered. These were never publically accessable.

Thank you for your concern.

 Best storage practices would be in a clean, dark, dry area at (~ 23°C / 72°F) or below.  Exposure to UV will break down pigments and cause loss of contrast and saturation.

Color Test Charts

Such as X-Rite colorcheckers.  Certified to last two years if stored properly.  You can use a spectrophotometer to generate a reference file for an older/faded chart.

Color LVT Film

The main factors affecting the films are humidity and UV light.

If the chart is used heavily (exposed to light every day) it can last up to one year.

If they are stored dark place and not used very often they can last up to 5 years.

Significant heat exposure ( > 52°C / 125°F) will decouple the dyes and significantly hasten the fading time.

Television Lines (TVL) are derived from the EIA 1956 testing standard, where they are defined as the number of light and dark vertical lines that are visible (i.e., distinguishable) over the height of the screen. TVL defines “lines” as distinct dark and light lines. This is equivalent to Line Widths (LW), and should not to be confused with Line Pairs (LP), which represent a complete cycle, i.e., one TV line is equivalent to half a line pair. Although TVL is not the best measurement of digital imaging system quality, it can be useful for comparing digital systems with older analog systems.

The definition of TVL is problematic because “resolved” or “visible” are not clearly defined. There is no description of the viewing conditions, no objective definition of the contrast representing “light” and “dark”, and no description of how much contrast is lost before detail is considered “no longer resolved”.  Is a 90% loss of the signal (MTF10) enough to be considered “no longer resolved?”  This is a relatively common definition of resolution (derived from the Rayleigh diffraction limit), and it has been standardized for Camera Monitor Systems in the ISO 16505:2015 standard.  We have proposed an improved definition of resolution, but it has only gained limited traction. Using wedge patterns, some observers state that lines can no longer be resolved at 70% (MTF30) or 80% (MTF20) loss of contrast. Relying on subjective assessment of observers makes it difficult to get a consistent objective definition of TV lines.

TVL measurements do not consider changes in sharpness across the image that are typical of conventional lenses. Measuring the center-only may produce optimistic results. The EIA 1956 and ISO-12233:2000 charts had wedges in the corners that were easy to ignore. You should consider points all across the image to get a full understanding of your system’s resolution. You can use a mean of these values if you only want a single number. Remember that the tilt of your lens can degrade resolution in some corners, but boost it in opposite corners, without having much effect on the mean MTF.

A convenient, stable method of measuring TV lines in displays (i.e.,monitors) is given in Display (Monitor) Sharpness. For most other images we recommend using MTF20 (or better, MTF20P) in units of LW/PH (Line Widths per Picture Height), measured with (near) vertical slanted edges. (MTF20P is less sensitive than MTF20 to excessive software sharpening.)

note: for cropped images enter the original picture height into the more settings dimensions input.

If you can control the processing of your imaging system so you can disable sharpening or analyze unsharpened RAW images, you can make an approximate TVL measurement using “MTF10 LW/PH” or “MTF20 LW/PH” from vertical edges across the image. You can use the Multi-region summary plot from SFR, SFRplus, eSFR ISO, Checkerboard, or SFRreg modules to obtain a weighted mean MTF10 or MTF20 in LW/PH units, which are good objective approximations to the TVL metric. This is superior to the wedge measurement described below since it is less affected by aliasing, but it may not be the best measurement if you cannot disable your cameras sharpening.
 
If you have a “black box camera” system where you cannot disable sharpening, we recommend the hyperbolic wedge pattern to measure high-frequency MTF or vanishing resolution. Otherwise, the slanted edge MTF10 or MTF20 numbers may be exaggerated by the sharpening. Our best target for measuring multiple wedges across the field is our ISO 12233:2017 target with “extra wedges”. The best Output is our Multi-Wedge plot. The Weighted Mean Wedge MTF10 or MTF20 in units of LW/PH is very close to TV Lines.

• Fixed modules require that all settings be entered prior to running the analysis. Stored settings are read from imatest.ini and can be changed by user input. Results are displayed as figures and can be saved as image (usually PNG), CSV, XML, and JSON files.
   
• Interactive modules are run from GUIs (Graphic User Interfaces) that allow results to be queried and modified after the analysis has been run. This allows you to explore results in great depth. Results can be saved as image (usually PNG), CSV, XML, and JSON files.

Imatest Version 2021.1 adds several new features to Imatest Master and Imatest IT, including edge tracking during live focusing, alignment tools in device manager, new subjective exposure quality loss calculation, OpenEXR support, live exposure and white balance measurements and several customer-requested features.
(more…)

CPIQ Auto exposure is in draft status. It will be added to the 2021 revision of the CPIQ Standard. You should use Imatest v2021.1 or later to perform this calculation.

Auto exposure error is based on deviation from a nominal luminance value for the neutral grey patch in an X-Rite Colorchecker SG. In a variant calculation, we can also calculate this metric from a 24-patch X-Rite Colorchecker.

A psychophysical study of the subjective quality based on luminance level was how the following model for exposure quality loss was developed:

CPIQ Auto exposure is measured with the Imatest Color/Tone module, which produces a Quality Loss (QL) score in units of Just Noticeable Differences (JND).

Results are available in display #4 2D a*b:

and display #7 B&W density & White Balance:

For more about Imatest’s CPIQ support click here

Imatest Version 2020.2 adds several new features to Imatest Master and Imatest IT, including Improvements in Calculation Speed, Uniformity Statistics based on EMVA 1288 Standards, Uniformity measurements from ISO 17957:2015, IT For Mac OS, and Tunable Checkerboard Detection. (more…)

When measuring uniformity, in order to minimize cosine falloff in imaging systems with wider field of view lenses, it is important to have a light source that produces uniform illumination across the entire field of view.  The equipment listed below can be deployed to produce uniform images that fit the spectral requirements and field of view of your imaging system:

Learn More

Problem

Certain imatest calculations in version 5.1 and below produced zeros when the calculation was not determined. These zero values could be inappropriately averaged with summary metrics which lead to these summary metrics being underreported.

The problem includes, but is not limited to the following summary metrics:

• Chromatic Aberration (CA) summary calculations CA_areaPCT_summary, CA_crossingPCT_summary, CA_R_G_PCT_summary, CA_B_G_PCT_summary where the individual outputs in CA_area_Pct_corner, CA_cross_Pct_corner, CA_crossing_R_G_PCT_corner, CA_crossing_B_G_PCT_corner, CA_crossing_R_G_Pxls, CA_crossing_B_G_Pxls, include zeros.
• MTF Summary metrics where higher frequency MTF values that are not achieved, and incorrectly reported as zero (corrected in v5.1.28)

Solution

• Take care not to use any summary metrics where the calculations output zeros
• Upgrade to Imatest 5.2 where these indeterminate calculations are reported as NaN (Not a Number), and will not be included in any averages

On some MacOS systems, the 5.2 Imatest release has an issue where the system temporarily freezes for under 60 seconds when file dialogs are opened.

We have traced this to the new 2019a MATLAB compiler runtime and are working with MathWorks to resolve the issue. (more…)

Using testing charts (also known as targets) improves the efficiency and repeatability of your testing workflow. Imatest offers several charts that support automatic detection as well as charts with registration marks.

(more…)

In order to perform online license activation or to use a floating license, Imatest needs to be able to perform name resolution or connect through a network to the following licensing servers:

(more…)

Recently we have been receiving reports of end-users in China being blocked from accessing the Imatest license servers (IP address 184.106.60.185).

A message such as “SOAP couldn’t connect to host; Error code -4304″ may be displayed.

Here are the current workarounds to resolve this issue for this subset of end-users:

• End-users have had success using a VPN to tunnel to either another location within China that is not impacted by an ISP block, or outside of China. This is typically the easiest solution to address the issue.
• Use offline activation: The offline activation portal accessible through store.imatest.com is still functional.

Note: this offline activation approach works both for individual node-locked licenses as well as offline activation of network-based licenses within the Nalpeiron LAN Daemon – which then allows floating licensing, element pools, floating features, etc. all to be used within the application.

• There is a relay server with the IP address 119.9.127.11.  You can use the proxy address to 119.9.127.11 (port 80) and use that to redirect back to our standard servers.  
• Review our Connectivity requirements for online or floating license activation

We apologize for the inconvenience. Please contact us at support@imatest.com with any difficulties you have.

Color reference files read by Imatest can be CSV or CGATS format containing L*a*b* color and luminance reference data.

The reference illuminant is assumed to be the same as the selected color space (D65 for sRGB, etc.). This is at variance with current practice, where L*a*b* files are assumed to have D50 data. The new LAB D50 file (Imatest 4.4+) is recommended because it will transform the reference values using a Bradford Transform to move them to the selected color space’s white point. This is the recommended approach if you have spectrophotometer measurements for your individual chart.

Color Reference files

Color CSV reference files files consist of the number of lines that the test chart has patches, with L*, a*, b* values on each line separated by spaces, commas (,), or semicolons (;). Example (first 3 lines of 24):

38.08, 12.09, 14.39
66.38, 13.22, 17.14
51.06, 0.38, -22.06
…

If you have an Excel .CSV file with extra rows or columns, you can easily edit it Excel by selecting the key region (3 columns, 24 rows), copying it to a new file, and saving it in .CSV format. L*a*b* data is preferred to xyY data below because it is independent of white point color temperature, hence less error-prone. 

Color reference files can also be derived from measured values from a camera you determine to be a “gold standard” by running Color/Tone Interactive, then pressing File, Save L*a*b* results as CSV reference (making sure to check CSV reference file of L*a*b* results…). This can be useful for manufacturing testing because the camera under test is compared with the attainable values of a golden sample rather than unattainable ideal values.

The CGATS file format is also supported.

Color/Tone Interactive General mxn charts (4×12 shown on the left). Any chart with an m x n grid can be analyzed. You must specify the grid size (click Settings, ISO speed & mxn chart settings to set the number of rows (m) and columns (n). You must also enter a CSV or CGATS file with reference patch settings (typically L*, a*, b* values, one set per line). If you have a spectrophotometer you can create a custom reference file (CGATS format) using techniques described in Measuring test chart patches.

Obtaining the color reference

In Color/Tone Interactive you can perform a number of operations that relate to the reference file using the File dropdown menu, shown on the right.

The operation of interest here is Copy L*a*b* reference to clipboard. This shows the currently selected reference, whether it’s a standard or custom file. You can also save it as a named file for future use as a reference— perhaps for a “gold standard”. Here are the first three values for the post-Nov 2014 X-Rite Colorchecker default values.

37.31 13.39 14.58
64.37 18.05 17.05
49.62 -1.162 -22.16
…

These are close, but not identical, to the values shown above, which were derived from a different source.

Density Reference Files

Density CSV reference files are much simpler, they contain one optical density value per line, going from least-dense (smallest) patch, up to the most dense patch.  

Imatest will provide you with a density reference file when you purchase a 36-patch dynamic range or contrast resolution chart. You should always use this reference file while performing dynamic range measurements with multicharts or color-tone instead of using the default values.

Note that Copy density reference to clipboard is available in the File dropdown menu on the right.

See Also

• Color/Tone Interactive – Reference File 
• Colorcheck Reference File (legacy module: Color/Tone is recommended for new work)
• Measuring Test Chart Patches with a Spectrophotometer for details about how to create a reference file using measurements from an instrument.

by Henry Koren, inspired by Paul Romanczyk, edited by Norman Koren

Not all MTF measurement systems will necessarily provide the same results. The quality of the test target can impact the measurements you obtain. Long distance tests are ideally performed at the hyperfocal distance, where there is enough depth of field to have acceptable focus at infinity. (more…)

Here are Henry’s favorite Imatest Links that he often mentions during the Imatest Training Class:

Day 1

Day 2

https://imatest.com/passfail/

This guide enumerates the possibilities and constraints of our test charts and provides resolution (MTF) information for the different options. Imatest has introduced Chart MTF Compensation, which can as much as double the resolution suitability of a given test chart.

Inkjet

Close up of inkjet print quality

Inkjet is the lowest resolution process we offer, but also the least expensive and most versatile. The minimum print size with inkjet is approximately 8” x 10”, and the maximum print size for a single sheet is 44” x 80”.

Inkjet charts are available in three different substrates: Matte and Semigloss for reflective testing and DisplayTrans for transmissive tests using a backlight. All inkjet options are roughly the same resolution.

Inkjet is best suited for cameras tested at a moderate distance in order to compensate for the low resolution of the print. It is also suitable for low-resolution or wide-angle systems. As the resolution of the target is relatively low, the size of the chart needs to be larger to accommodate a higher resolution sensor.

We can produce custom charts, and custom sizes of most of our standard charts. Contact charts@imatest.com with your requested size and information about your lens (field of view and testing distance) and sensor (pixel count and aspect ratio.) We need the pixel count to confirm the size you are requesting will be adequate for your imaging system. Our Chart Finder Tool can help you determine the size of your imaging plane at a specific testing distance.

Black and White Photographic Paper (Silver Halide)

Close up of photographic paper print quality

Photographic media offers high-resolution reflective prints when a transmissive target is not an option. These prints are over twice the resolution of inkjet media.

The minimum size of a photographic print is approximately 8” x 10” and the maximum size is approximately 40” x 50”.

Our standard photographic targets include our eSFR ISO chart (3 sizes) and a multi-size option of our SFRplus chart. This multi-size is best suited for testing low-resolution systems at close distances. We can offer custom charts on this media for an additional fee. Contact charts@imatest.com for information. 

Transmissive Photographic Film

Our Color and Black & White film targets are printed using high-precision LVT Technology. These transmissive targets require a lightbox in order to be used properly. 

Black and White

Close up of Black and White Film print quality

Black and White film is over twice the resolution of our inkjet targets, and can optionally include a color patch for color measurements. The size of the film is 12” x 20” or 16” x 20” (with ½” on each side as unprintable area). The minimum target size on this media is roughly 2” x 3”. We can print multiple targets on a single sheet of film depending on the size and resolution requirements to help make this option more cost-effective.

This media is suited for customers testing in a small area with low to medium resolution cameras.

Color

Close up of Color Film print quality

Our high-precision color film targets are printed on 8” x 10” film with a maximum printed area of 9.25” x 7.75”. The minimum target size on this media is approximately 0.5” x 1”. As with the Black and White film, we can print multiple targets to a single sheet of film to make this a more cost-effective option.

This media is suited for customers requiring a very compact testing system or those using conversion optics to simulate infinity within a very small space. The fine detail of the film allows us to test mid to high-resolution cameras at close distances and with a small field of view. Color film has the highest dynamic range of all of the targets we offer. 

Chrome on Glass Photomask

Close up of Chrome on Glass print quality

Chrome Photomasks are our sharpest option for resolution testing. We use a lithographic photomasking process to print chrome directly onto glass. This process is single tone, so there is no gray-scale or tonal variance.

Chrome on Glass offers superior resolution when color and tonal measurements are not needed. We can make targets as small as 2mm x 3mm for measuring devices such as endoscopes or microscopy equipment. When printing to glass, the chrome has an optical density of 1, which gives a contrast ratio of approximately 10:1, suitable for resolution measurement.

Our standard glass plates are 4” x 4” with a non-printable border of 1 cm. We can source plates up to 20” x 20” and as small as 1” x 1”. Any plates larger than 4” x 4” require an extended lead time of 6-8 weeks. 

We can customize the size of various targets such as our SFRplus or ISO-eSFR targets on request. We can also print most custom design requests to glass, though some may require an engineering fee depending on the complexity of the design. Contact charts@imatest.com.

Test chart suitability

Detailed information on test chart suitability — relating camera resolution to chart size — can be found on Test chart suitability for MTF Measurements. 

General information on sharpness, MTF, and the wide range of available charts can be found in the Sharpness section of the Imatest Documentation page. 

Abstract