All Versions – imatest https://www.imatest.com Image Quality Testing Software & Test Charts Tue, 05 May 2020 19:06:02 -0600 en-US hourly 1 https://wordpress.org/?v=5.3 Which Imatest Charts Support Automatic Detection? https://www.imatest.com/2019/08/what-imatest-charts-support-automatic-detection/ https://www.imatest.com/2019/08/what-imatest-charts-support-automatic-detection/#respond Wed, 21 Aug 2019 19:55:24 +0000 http://www.imatest.com/?p=27780 Using testing charts (also known as targets) improves the efficiency and repeatability of your testing workflow. Imatest offers several charts that support automatic detection as well as charts with registration marks.

Imatest Charts Supporting Auto-Detection

  • SFRplus – Using this target requires the bars on top and bottom of the chart to be present in the image.
  • 24-patch X-Rite Colorchecker – This chart works best without extreme distortion.
  • Checkerboard – Imatest can detect square areas of a checkerboard target that are not excessively bloomed or blurred.
  • Rezchecker – Imatest can detect the version of this chart with four dots.
  • Dot pattern – This chart is used for testing chromatic aberration and SMIA TV Distortion.

Imatest Charts with Registration Marks

Note: Imatest does not perform autodetection for the Obsolete ISO 12233:2000 chart because it can easily produce invalid results. 

If you have a target you would like to be automatically detected, please contact support@imatest.com.

]]>
https://www.imatest.com/2019/08/what-imatest-charts-support-automatic-detection/feed/ 0
Making Dynamic Range Measurements Robust Against Flare Light https://www.imatest.com/2019/07/making-dynamic-range-measurements-robust-against-flare-light/ https://www.imatest.com/2019/07/making-dynamic-range-measurements-robust-against-flare-light/#respond Mon, 22 Jul 2019 23:31:24 +0000 http://www.imatest.com/?p=26418 Introduction

A camera’s Dynamic Range (DR) is the range of tones in a scene that can be reproduced with adequate contrast and good signal-to-noise ratio (SNR). Camera DR is often limited by flare light, which is stray light in the image, primarily caused by reflections between lens elements. Flare light reduces DR by fogging images; i.e., washing out detail in dark areas.It is the primary reason that the DR of cameras (which include lenses) is poorer than that of image sensors, which can be up to 150dB (30 million:1) for recent HDR (high dynamic range) sensors.

In the past 2 years we’ve learned that flare light in test chart images could, in theory, be mistaken for the image of the test chart, resulting in exaggerated DR measurements. The situation became real in mid-2019, when we started seeing actual images where this error occurred. By studying these images, we have developed techniques to guard against exaggerated Signal-to-Noise Ratio (SNR) measurements caused by flare light. We describe these techniques here.

We recognize that some customers may actually prefer exaggerated measurements because they yield very HDR numbers—sometimes over 120dB—that approach the specifications of HDR sensors and look good in marketing materials. The reduced but realistic DR measurements obtained when the effects of flare light are removed may make some of these customers uncomfortable. We will do our best to deal with their objections.

The effects of flare light

Flare light can be illustrated with an image of the XYLA chart—a precision HDR test chart with a linear configuration—that consists of 21 gray scale patches with Optical Density steps of 0.3. The image is from a low-cost “black box” camera.

The upper cross-section plot, made with the Image Statistics module, is taken at the center of the XYLA image. Flare is most obvious in the image and as a decay in the cross-section plot to the left of the brightest patch.

XYLA image (from the same image; the lower is lightened) and corresponding cross-section plots

The lower cross-section was taken outside (below) the active chart image, which is shown lightened above to make flare light more visible. The variation in pixel level on the right side of the chart (x between 800 and 1600) is caused by flare light diffusing from the brightest patches on the left. 

Results of the XYLA image (above) showing strong tone mapping; click image for a full-sized view.

Some notes on this image.

This image has very strong local tone-mapping, leading to an exceptionally low (and not very meaningful) measured gamma of 0.148. The SNR varies in an unusual way because it does not drop monotonically, as it would for a conventional linear image sensor. This indicates that an HDR image sensor with several operating regions was used. 

Because the image had significant barrel distortion, region selection was difficult. The Contrast Resolution chart is much easier to use and provides a better indication of system performance in the presence of tone mapping.

If the flare light were any worse, it could easily have been mistaken for a signal from the chart itself, leading to an false DR measurement.

Flare light can be extremely complex. It can add an offset to the image (often called “veiling glare”), which is difficult to distinguish from a black level offset in the image processing pipeline. Most of the time it is largest near bright patches, then decreases with distance from these patches. The rate of decrease is rarely a well-behaved exponential.

Lens reflections are a major cause of medium-range flare light. An uncoated glass surface (index of refraction ≅ 1.5) reflects R = 4% = 0.04 of the light incident on it. (Remember, a sheet of glass or a lens component has two surfaces.) 

For each glass surface between the surface and the light source, a fraction R of the primary reflection (R2 of the original incident light) is reflected back to the image sensor. This are called a secondary reflection. Since most lens surfaces are curved, this light will be unfocused; i.e., it will tend to fog a portion of the image. 

According to Edmund Optics, the best anti-reflective coatings have R ≅ 0.4% = 0.004 over the visible spectrum (~400-700nm). R = 0.005 may be more realistic for a reasonable range of incident angles. The light reflected back to the sensor from each secondary reflection would be R2 = 0.000025 = 2.5*10-5 = -92 dB (20*log10(R2)). The number of secondary reflections Nsec increases rapidly with the number of components M (groups of elements cemented together, each of which has two air-to-glass surfaces) in a lens: 1 for one component; 6 for two components; 15 for three components; 28 for four components; 45 for five components, etc. For M components,

   \(\displaystyle \text{Number of secondary reflections} = N_{sec} = \sum_{i=1}^{2M-1}i = 2M(2M-1)/2 = M(2M-1))\)

M = 5 components are typical for high-quality camera phones; M ≥12 components is commonplace for DSLR zoom lenses. Overall, lens flare is less severe than the number of secondary reflections suggests because stray light does not cover the whole image; it decreases with distance from bright regions. It’s easy to see why practical camera DR measurements are limited to around 70-90dB, even when sensor DR is much higher.

Image Statistics cross-section of a Contrast Resolution image for an inferior camera, showing spatially varying flare indicated by red arrows

Because the ISO 18844 flare model does not measure the spatially dependent flare caused by lens reflections, it has limited value in characterizing practical system performance.

 

Key takeawaysFlare light is predominantly light in dark regions of an image that diffuses from bright regions. This diffused light can be confused with the actual chart signal (especially with linear charts), resulting in exaggerated (overly optimistic) DR measurements. Moreover, increasing flare light (which could result from poorer; i.e., cheaper, lens coatings) decreases the actual DR by fogging shadow areas of the image, but can lead to increased DR measurements. Hence the need to distinguish artifact signals from flare light from real signals from the chart.

Circular test charts

 UHDR photographic film chart

The test charts recommended by Imatest for measuring DR are transmissive (i.e., backlit) charts with (approximately) circular patch configurations; i.e., those that are not linear (like the XYLA chart, shown above). The two- or three-layer High Dynamic Range Chart, shown on the right, comes in several versions. Because photographic film charts are not manufactured with consistent patch densities, a reference file is required when these charts are used.

High-quality cameras

Until recently, most of the DR images we analyzed came from DSLR or mirrorless cameras that had relatively low flare light. The recent images we’ve seen with severe flare light are from inferior cameras. We have not determined exactly why the flare light is so much worse; it might be due to inferior coatings in the multi-element lenses or less baffling in the barrel of the lens.

Here are examples of results from high-quality cameras. Click on the thumbnails below to view full-sized images.


Results for raw image from high-quality (Canon 90mm Tilt/Shift) lens


Results for jpeg image from high-quality (Canon 90mm Tilt/Shift) lens


Results for raw image from consumer-grade (Canon 75-300mm) lens


Results for jpeg image from consumer-grade (Canon 75-300mm) lens

The four images are for 48-bit RAW (or TIFF derived from RAW) and 24-bit JPEG files captured on the Sony A7Rii camera with two very different lenses:

  • A consumer grade Canon EF 75-300-mm f/4-5.6 lens (original version) set to 80mm, f/5.6. This lens has 15 elements in 10 groups.
  • The very high quality Canon TS-E 90-mm f/2.8 (Tilt/Shift) lens set to f/5.6. Since this lens has 6 elements in 5 groups, it would be expected to have lower flare light than the 75-300.
  • Note that results from the two darkest patches, 35 and 36, are outside the plot because their densities (8.184 and 8.747, equivalent to 163.7 and 174.5dB) are beyond the 160 dB limit of the plot. 160dB is equivalent to a 100 million:1 ratio— far beyond the capabilities of any camera system with a lens in front of the sensor.

As expected, the 90-mm T/S lens has significantly better DR, and the JPEG files had more of a response “shoulder” (an area of reduced slope in the lighter part of the image). But there is one surprise. The DR of the JPEG images is comparable to the raw images—apparently because gamma encoding, which decreases the number of pixel levels in bright regions and increases it in dark regions, extends DR beyond what would be expected for a linear 8-bit (256 level) file. 

Note that in three of the four images above, the low quality DR is lower than the DR from slope; they are very close in the remaining image. We don’t recommend the use of slope-based DR by itself (because it often extends well beyond the region where SNR = 0dB (Signal/Noise = 1); i.e., it includes regions where poor SNR causes noise to completely mask image detail.

Low-cost “black-box” camera

The differences between the DR of the two lenses (medium and high quality) seems to be minor when compared to an image from a low-cost “black-box” camera we recently received. 

Dynamic Range results for low-cost “black box” camera

The low-quality (SNR = 0dB, labeled Low ——— ) DR is measured as 148dB—an astonishingly high number; DR from slope is 66dB—much lower than several quality-based DR measurements (and lower than the slope-based DR for the 90-mm T/S lens). Note that the two darkest patches don’t appear on this plot because their densities (8.184 and 8.747) are beyond the 160-dB limit of the plot. (A 160dB is a range of 100 million to 1—more than expected from any sensor or camera. Put another way, if one photon were to reach the darkest patch, the lightest patch would set the chart or sensor on fire).

To understand what is happening, we need to observe the dark portions of the images from high-quality cameras, shown as extra (X-) or XX-lightened to make detail visible in the darker patches, and compare these to a comparable region in the low-cost camera.


Dark areas of RAW (TIFF) image from Canon 75-300mm (consumer-grade) lens, X-lightened

The RAW image for the six-element (5 groups) 90-mm lens shows distinct regions of decreasing brightness in the 7th row (second from the bottom; patches 27-32), and still shows distinct regions in the bottom row, though brightness no longer decreases. A reflection (possibly from the lens) is visible in patch 30. The JPEG image from the 90-mm lens has more noise. Some may be quantization noise since the JPEG has only 256 levels (0-255). The images from the 15-element (10 groups) 75-300-mm lens definitely show more evidence of flare.

Now, compare these results—especially for the RAW image from the 90-mm T/S lens (left thumbnail)—with the image from the low-cost black-box camera. 

Dark areas of JPEG image from low-cost black-box camera, X-lightened

No patch detail is visible in the bottom two rows (patches 27-36). Instead of decreasing from left to right in each row, the pixel level decreases from top to bottom, remaining relatively constant across the bottom two rows, with banding (the result of 255 levels) clearly visible. This is clearly flare light, not signal from the test chart. Unfortunately the SNR in these rows is quite good because Imatest removes the effects of illumination nonuniformity in noise calculations. (A setting can turn this off, but we generally recommend leaving it on to give better results in the presence of actual illumination nonuniformity). But this signal is an artifact, not the real thing.

This leads us to the inescapable conclusion that the quality-based DR results for the low-cost black-box camera are incorrect, and that we need to detect the patches where flare light overwhelms the signal from the chart image and exclude them from the DR calculation.

Fortunately this is not difficult for DR test charts with a circular configuration, such as the 36-patch DR chart illustrated above (harder for linear charts). Because of the patch arrangement, the patch level stops decreasing when flare light dominates the scene. This is clearly visible in the bottom two rows in the above example, where the image gets darker from top to bottom in these rows—perpendicular to the patch sequence. When the patch brightness ceases to decrease, we can be confident that flare light dominates; i.e., we are outside the camera DR. This is the case for patches beyond the slope-based DR.

For this reason it makes sense to limit all quality-based DR measurements to the slope-based DR (maximum).

In Imatest 5.2, the Options II window (accessed from the button at the lower-right of the Imatest main window) offers a choice of whether to limit quality-based DR to slope-based DR. When the box is checked, the limit appears in the results display.

Dynamic Range (DR) results for low-cost black box camera, with quality-based DR limited by slope-based DR

Key takeaways—Flare light was not an issue with the high-quality DSLR/mirrrorless lenses we tested in the past, but it has become a major factor limiting the performance of recent low-cost lenses intended for the automotive or security industries. We have seen examples of how flare light can improve traditional DR measurements while degrading actual camera DR. 

Our approach to resolving this issue is to limit quality-based DR measurements (the range of densities where SNR ≥ 20dB for high quality through SNR ≥ 0dB for low quality) to the slope-based DR. This works because, for patches beyond the slope-based limit (where the slope of log pixel level vs. log exposure drops below 0.075 of the maximum slope):

  • Contrast is too low for image features to be clearly visible.
  • Signal is dominated by flare light, which washes out real signals from the test chart; i.e., the “signal” is an artifact, not the real deal.

Limiting quality-based DR in this way significantly improves measurement accuracy, and perhaps more importantly, can help prevent inferior, low-quality lenses being accepted for applications critical to automotive safety or security.

 

]]>
https://www.imatest.com/2019/07/making-dynamic-range-measurements-robust-against-flare-light/feed/ 0
Color & density reference files in CSV or CGATS format https://www.imatest.com/2019/06/color-reference-files-in-csv-or-cgats-format/ https://www.imatest.com/2019/06/color-reference-files-in-csv-or-cgats-format/#respond Wed, 19 Jun 2019 16:52:43 +0000 http://www.imatest.com/?p=26209 Color reference files read by Imatest can be CSV or CGATS format containing L*a*b* color and luminance reference data.

The reference illuminant is assumed to be the same as the selected color space (D65 for sRGB, etc.). This is at variance with current practice, where L*a*b* files are assumed to have D50 data. The new LAB D50 file (Imatest 4.4+) is recommended because it will transform the reference values using a Bradford Transform to move them to the selected color space’s white point. This is the recommended approach if you have spectrophotometer measurements for your individual chart.

Color Reference files

Color CSV reference files files consist of the number of lines that the test chart has patches, with L*, a*, b* values on each line separated by spaces, commas (,), or semicolons (;). Example (first 3 lines of 24):
38.08, 12.09, 14.39
66.38, 13.22, 17.14
51.06, 0.38, -22.06

If you have an Excel .CSV file with extra rows or columns, you can easily edit it Excel by selecting the key region (3 columns, 24 rows), copying it to a new file, and saving it in .CSV format. L*a*b* data is preferred to xyY data below because it is independent of white point color temperature, hence less error-prone. 

Color reference files can also be derived from measured values from a camera you determine to be a “gold standard” by running Color/Tone Interactive, then pressing FileSave L*a*b* results as CSV reference (making sure to check CSV reference file of L*a*b* results…). This can be useful for manufacturing testing because the camera under test is compared with the attainable values of a golden sample rather than unattainable ideal values.

The CGATS file format is also supported.

Color/Tone Interactive General mxn charts (4×12 shown on the left). Any chart with an m x n grid can be analyzed. You must specify the grid size (click SettingsISO speed & mxn chart settings to set the number of rows (m) and columns (n). You must also enter a CSV or CGATS file with reference patch settings (typically L*, a*, b* values, one set per line). If you have a spectrophotometer you can create a custom reference file (CGATS format) using techniques described in Measuring test chart patches.

Density Reference Files

Density CSV reference files are much simpler, they contain one optical density value per line, going from least-dense (smallest) patch, up to the most dense patch.  

Imatest will provide you with a density reference file when you purchase a 36-patch dynamic range or contrast resolution chart. You should always use this reference file while performing dynamic range measurements with multicharts or color-tone instead of using the default values.

See Also

 

]]>
https://www.imatest.com/2019/06/color-reference-files-in-csv-or-cgats-format/feed/ 0
Correcting nonuniformity in slanted-edge MTF measurements https://www.imatest.com/2018/08/edge-mtf-nonuniformity/ https://www.imatest.com/2018/08/edge-mtf-nonuniformity/#respond Fri, 31 Aug 2018 18:00:44 +0000 http://www.imatest.com/?p=21906 Slanted-edge regions can often have non-uniformity across them. This could be caused by uneven illumination, lens falloff, and photoresponse nonuniformity (PRNU) of the sensor. 

Uncorrected nonuniformity in a slanted-edge region of interest can lead to an irregularity in MTF at low spatial frequencies. This disrupts the low-frequency reference which used to normalize the MTF curve. If the direction of the nonuniformity goes against the slanted edge transition from light to dark, MTF increases. If the nonuniformity goes in the same direction as the transition from light to dark, MTF decreases. 

To demonstrate this effect, we start with a simulated uniform slanted edge with some blur applied.

Then we apply a simulated nonuniformity to the edge at different angles relative to the edge. This is modeled to match a severe case of nonuniformity reported by one of our customers:

 

Here is the MTF obtained from the nonuniform slanted edges:

If the nonuniformity includes an angular component that is parallel to the edge, this adds a sawtooth pattern to the spatial domain, which manifests as high-frequency spikes in the frequency domain. This is caused by the binning algorithm which projects brighter or darker parts of the ROI into alternating bins.

 

Compensating for the effects of nonuniformity

Although every effort should be made to achieve even illumination, it’s not always possible (for example, in medical endoscopes and wide-FoV lenses).

Imatest 4.5+ has an option for dealing with this problem for all slanted-edge modules (SFR and Rescharts/fixed modules SFRplus, eSFR ISO, SFRreg, and Checkerboard). It is applied by checking the “Nonuniformity MTF correction” checkbox in the settings (or “More” settings) window, shown on the right.

When this box is checked, a portion of the spatial curve on the light side of the transition (displayed on the right in Imatest) is used to estimate the nonuniformity. The light side is chosen because it has a much better Signal-to-Noise Ratio than the dark side. In the above image, this would be the portion of the the edge profile more than about 6 pixels from the center. Imatest finds the first-order fit to the curve in this region, limits the fit so it doesn’t drop below zero, then divides the average edge by the first-order fit. 

The applied compensation flattens the response across the edge function and significantly improves the stability of the MTF:

Summary

For this example, Imatest’s nonuniformity correction reduces our example’s -26.0% to +22.8% change in MTF down to a -3.5% to +4.7% change. This is an 83% reduction in the effect of the worst cases of nonuniformity.

MTF50 versus nonuniformity angle without [blue] and with [orange] nonuniformity correction

While this is a large improvement, the residual effects of nonuniformity remain undesirable. Because of this, we recommend turning on your ISP’s nonuniformity correction before performing edge-SFR tests or averaging the MTF obtained from nearby slanted edges with opposite transition directions relative to the nonuniformity to reduce the effects of nonuniformity on your MTF measurements further.

Detailed algorithm

We assume that the illumination of the chart in the Region of Interest (ROI) approximates a first-order function, L(d) = k1 + k2d, where d is the horizontal or vertical distance nearly perpendicular to the (slanted) edge. The procedure consists of estimating k1 and k2, then dividing the linearized average edge by L(d). 

k1 and k2, are estimated using the light side of the transition starting at a sufficient distance dN from the transition center xcenter, so the transition itself does not have much effect on the k1 and k2 estimate. To find dN we first find the 20% width d20 of the line spread function (LSF; the derivative of the edge), i.e., the distance between the points where the LSF falls to 20% of its maximum value. 

dN = xcenter + 2 d20 

If the edge response for x > dN has a sufficient number of points, it is used to calculate k1 and k2 using standard polynomial fitting techniques. The result is a more accurate representation of the edge with the effects of nonuniformity reduced.

Future work

  • Consider the 2D nonuniformity across the ROI before sampling the 1D average edge
  • Use an image of a flat-field to perform nonuniformity correction within Imatest
  • Consider the impact of noise which was not included in this study
  • Incorporate enhancements to the slanted-edge algorithms into future revisions of ISO 12233

 

For any questions on how to do this, or how we can help you with your projects, contact us at support@imatest.com.

 

]]>
https://www.imatest.com/2018/08/edge-mtf-nonuniformity/feed/ 0
Imatest does not start, unable to read MAT-file https://www.imatest.com/2018/07/imatest-does-not-start-unable-to-read-mat-file/ https://www.imatest.com/2018/07/imatest-does-not-start-unable-to-read-mat-file/#respond Fri, 06 Jul 2018 20:41:31 +0000 http://www.imatest.com/?p=22950 If upon opening Imatest you get something like this:

 

Error: Error using load
Unable to read MAT-file C:\ProgramData\Imatest\mcr_cache\5.0\Master\mcrCache9.2\imates0\.matlab\matlabprefs.mat. File might be corrupt.

Some of your MATLAB files have become corrupted. You can solve this by deleting your CTF folder found at C:\ProgramData\Imatest\mcr_cache\5.0\Master\, then try starting Imatest again.

 

If you continue to have problems please email us with details of the issue at support@imatest.com

]]>
https://www.imatest.com/2018/07/imatest-does-not-start-unable-to-read-mat-file/feed/ 0
Measuring temporal noise https://www.imatest.com/2018/04/measuring-temporal-noise/ https://www.imatest.com/2018/04/measuring-temporal-noise/#respond Thu, 19 Apr 2018 17:04:08 +0000 http://www.imatest.com/?p=21871 Temporal noise is random noise that varies independently from image to image, in contrast to fixed-pattern noise, which remains consistent (but may be difficult to measure because it is usually much weaker than temporal noise). It can be analyzed by Colorcheck and Stepchart and was added to Multicharts and Multitest in Imatest 5.1.

It can be calculated by two methods.

  1. the difference between two identical test chart images (the Imatest recommended method), and 
     
  2. the ISO 15739-based method, which where it is calculated from the pixel difference between the average of N identical images (N ≥ 8) and each individual image.

In this post we compare the two methods and show why method 1 is preferred.

(1) Two file difference method. In any of the modules, read two images. The window shown on the right appears. Select the Read two files for measuring temporal noise radio button.

The two files will be read and their difference (which cancels fixed pattern noise) is taken. Since these images are independent, noise powers add. For indendent images I1 and I2, temporal noise is

\(\displaystyle \sigma_{temporal} = \frac{\sigma(I_1 – I_2)}{\sqrt{2}}\)

In Multicharts and Multitest temporal noise is displayed as dotted lines in Noise analysis plots 1-3 (simple noise, S/N, and SNR (dB)).

(2) Multiple file method. From ISO 15739, sections 6.2.4, 6.2.5, and Appendix A.1.4. Available in  Multicharts and Multitest. Currently we are using simple noise (not yet scene-referred noise). Select between 4 and 16 files. In the multi-image file list window (shown above) select Read n files for temporal noise. Temporal noise is calculated for each pixel j using

\(\displaystyle \sigma_{diff}(j) = \sqrt{ \frac{1}{N} \sum_{i=1}^N (X_{j,i} – X_{AVG,j})^2} = \sqrt{ \frac{1}{N} \sum_{i=1}^N X_{j,i}^2 – \left(\frac{1}{N} \sum_{i=1}^N X_{j,i}\right)^2 } \) 

The latter expression is used in the actual calculation since only two arrays, \(\sum X_{j,i} \text{ and } \sum X_{j,i}^2 \), need to be saved. Since N is a relatively small number (between 4 and 16, with 8 recommended), it must be corrected using formulas for sample standard deviation from Identities and mathematical properties in the Wikipedia standard deviation page as well as Equation (13) from ISO 15739.  \(s(X) = \sqrt{\frac{N}{N-1}} \sqrt{E[(X – E(X))^2]}\).

\(\sigma_{temporal} = \sigma_{diff} \sqrt{\frac{N}{N-1}} \) 

We currently recommend the difference method (1) because our experience so far has shown no advantage to method (2), which requires many more images (≥ 8 recommended), but allows fixed pattern noise to be calculated at the same time.

To calculate temporal noise with either method, read the appropriate number of files (2 or 4-16) then push the appropriate radio button on the multi-image settings box.

Multi-image settings window, showing setting for method 1.
if 4-16 images are enterred, the setting for method 2 (Read n files…) will be available.

Results for the two methods

The two methods were compared using identical Colorchecker images taken on a Panasonic Lumix LX5 camera (a moderately high quality small-sensor camera now several years old).

Difference method (1) (two files)

Here are the Multicharts results for 2 files.

Multicharts SNR results for temporal noise, shown as thin dotted lines in the lower plot

Multi-file method (2) (4-16 files)

 

Here are results (SNR (dB)) for runs with 4, 8, and 16 files.

For 4 files, temporal SNR (thin dotted lines) is slightly better than standard noise. Temporal SNR is slightly lower for 8 files and very slightly lower for 16 files.

For 8 and 16 files results are closer to the results for 2 files (though differences between 8 and 16 files are very small).

The bottom line: We recommend the two-file (difference) method because it is accurate and relatively fast. The multi file method is slower for acquiring and analyzing images— at least 8 images are recommended, so why bother?

Temporal SNR from 4 images
Temporal SNR from 8 images
Tempoal SNR from 16 images
]]>
https://www.imatest.com/2018/04/measuring-temporal-noise/feed/ 0
Calculating Field of View (FoV) using distance settings https://www.imatest.com/2017/02/calculating-field-of-view-fov-using-distance-settings/ https://www.imatest.com/2017/02/calculating-field-of-view-fov-using-distance-settings/#respond Fri, 17 Feb 2017 00:01:32 +0000 http://www.imatest.com/?p=17826 For accurate calculation of field of view (FoV) from using Imatest’s SFRplus, eSFR ISO, or Checkerboard, modules, it is critical to use proper distance measurements entered into in centimeters into the more settings window.

Lens to chart distance (in centimeters) can be best measured by a laser range finder.

For SFRplus it’s the bar to bar height is measured from the top of the top bar to the bottom of the bottom bar.

For eSFR ISO this is the vertical distance between the registration marks in centimeters.

Checkerboard uses the size of the squares.

]]>
https://www.imatest.com/2017/02/calculating-field-of-view-fov-using-distance-settings/feed/ 0
Gamma, Chart Contrast and MTF Calculations https://www.imatest.com/2016/12/gamma-chart-contrast-and-mtf-calculations/ https://www.imatest.com/2016/12/gamma-chart-contrast-and-mtf-calculations/#respond Tue, 27 Dec 2016 18:37:50 +0000 http://www.imatest.com/?p=17524

Gamma (the average slope of log pixel levels as a function of log exposure for light through dark gray tones) is used, per the ISO 12233 standard, to linearize the input data, i.e., to remove the gamma encoding applied by the camera or RAW converter. Gamma defaults to 0.5 = 1/2, which is typical of digital cameras, but may be affected by camera or RAW converter settings. 

Imatest Settings

For accurate calculation of edge-SFR from using Imatest’s SFRplus, eSFR ISO, SFRreg, Checkerboard, or SFR modules, it’s important to use a proper gamma for sharpness calculations.

If the chart contrast is known and is ≤10:1 (medium or low contrast), you can enter the contrast in the Chart contrast (for gamma calc.) box, then check the Use for MTF checkbox. Gamma will be then be calculated from the chart and displayed in the Edge/MTF plot.

Measuring Gamma

If chart contrast is not known you should measure gamma by obtaining the OECF (Opto-Electric Conversion Function) per the ISO 14524 standard.  This can be obtained using an image of a grayscale stepchart and running Colorcheck, Stepchart , Multicharts (interactive), or Multitest. A nominal value of gamma should be entered, even if the value of gamma derived from the chart (described above) is used to calculate MTF.

Errors in Gamma

Small errors in gamma have a minor effect on MTF measurements (a 10% error in gamma results in a 2.5% error in MTF50 for a normal contrast target). Gamma should be set to 0.45 when dcraw is used to convert RAW images into sRGB or a gamma=2.2 (Adobe RGB) color space. If gamma is set to less than 0.3 or greater than 0.8, the background will be changed to pink to indicate an unusual (possibly erroneous) selection.

Technical Details

Capture One LE with Film Curve
1. Capture One LE set to Film standard
(the default). Gamma = 0.679.

Capture One LE with Linear Curve
2. Capture One LE, Linear response. Gamma =
0.508. Recommended for SFR runs.

Canon FVU with Standard contrast

3. Canon FVU set to Standard contrast.
Gamma = 0.642.

Gamma is the exponent of the equation that relates image pixel level to luminance. For a monitor or print,

     Output luminance = (pixel level)gamma_display

When the raw output of the image sensor, which is linear, is converted to image file pixels for a standard color space, the approximate inverse of the above operation is applied.

     pixel level = (RAW pixel level)gamma_camera ~= exposuregamma_camera

The total system gamma is gamma_display * gamma_camera. Standard values of display gamma are 1.8 for older color spaces used in the Macintosh and 2.2 for color spaces used in Windows, such as sRGB (the default) and Adobe RGB (1998).

The three curves on the right, produced by Stepchart for the Canon EOS-10D, show how Gamma varies with RAW converter settings.In characteristic curves for film and paper, which use logarithmic scales (e.g., density (–log10(absorbed light) vs. log10(exposure)), gamma is the average slope of the transfer curve (excluding the “toe” and “shoulder” regions near the ends of the curve), i.e.,

Gamma is contrast.

See Kodak’s definition in Sensitometric and Image-Structure Data.

To obtain the correct MTF, Imatest must linearize the pixel levels— the camera’s gamma encoding must be removed. That is the purpose of Gamma in the SFR input dialog box, which defaults to 0.5, typical for digital cameras. It can, however, vary considerably with camera and RAW converter settings, most notably contrast.

Characteristic curves for the Canon EOS-10D with three RAW converter settings are shown on the right. Gamma deviates considerably from 0.5. Gamma = 0.679 could result in a 9% MTF50 error. For best accuracy we recommend measuring gamma using Colorcheck or Stepchart, which provides slightly more detailed results.

Confusion factor: Digital cameras rarely apply an exact gamma curve: A “tone reproduction curve” (an “S” curve) is often superposed on the gamma curve to extend dynamic range while maintaining visual contrast. This reduces contrast in highlights and (sometimes) deep shadows while maintaining or boosting it in middle tones. You can see it in curves 1 and 3, on the right. For this reason, “Linear response” (where no S-curves is applied on top of the gamma curve) is recommended for SFR measurements.

The transfer function may also be adaptive: camera gamma may be higher for low contrast scenes than for contrasty scenes. This can cause headaches with SFR measurements. But it’s not a bad idea generally; it’s quite similar to the development adjustments (N-1, N, N+1, etc.) in Ansel Adams’ zone system. For this reason it’s not a bad idea to place a Q-13 or Q-14 chart near the slanted edges.

To learn more about gamma, read Tonal quality and dynamic range in digital cameras and Monitor calibration.

]]>
https://www.imatest.com/2016/12/gamma-chart-contrast-and-mtf-calculations/feed/ 0
Imatest 4.3 MCR v9.0 Update https://www.imatest.com/2015/12/imatest-4-3-mcr-v9-0-update/ https://www.imatest.com/2015/12/imatest-4-3-mcr-v9-0-update/#respond Thu, 10 Dec 2015 00:02:01 +0000 http://www.imatest.com/?p=14407 As of Imatest 4.3 we will be upgrading the required Matlab Compiler Runtime library to v9.0 (R2015b). We had upgraded to v8.5 (R2015a) in Imatest 4.2 but several major changes made upgrading again worth the effort. Matlab has introduced a new runtime Execution Engine that dramatically improves the speed and efficiency of core aspects of program execution such as function call overhead and object-oriented programming overhead. What this means for you is that nearly all aspects of Imatest should have improved speed in 4.3 and up.

So how much difference are we talking here? To illustrate, let me pull the curtain back a bit on the Imatest development process. Internally we have a series of tests for Imatest IT that run on an hourly basis on the current major release code. These tests primarily look for changes in results to make sure that nothing we do will cause changes in results, and if they do we’ll get notified immediately. It also lets us know when someone does something they weren’t supposed to that stops a whole module from working! So do to a little timing test I’ve taken a section of the SFRplus test suite and run it on both MCR versions.

speed_test_v85-v90

The first couple runs of both versions are quite slow, this is a consequence of the Matlab JIT execution engine. The first run includes overhead for compiling the code down one step. After about 5 runs though you can see the results start to become a little more stable. v8.5 and v9.0 steady out at ~2.9 and ~2.1. So on average, v9.0 runs 27% faster on this test! Significant changes! Before we celebrate though, it’s worth noting that while these results are useful for illustrating the changes that are coming with Imatest 4.3, you should not assume that all tests will reflect these level of difference. Nor that you will see this level of improvement on your computer. These kinds of results are often highly system and test dependent. Some images and some test setups will see more improvement than this, some will see less. However, overall Imatest 4.3 and on will be that bit faster!

]]>
https://www.imatest.com/2015/12/imatest-4-3-mcr-v9-0-update/feed/ 0
INI file format changes in 4.2 https://www.imatest.com/2015/08/ini-file-format-changes-in-4-2/ https://www.imatest.com/2015/08/ini-file-format-changes-in-4-2/#respond Thu, 06 Aug 2015 02:07:13 +0000 http://www.imatest.com/?p=12392 In Imatest 4.2.0 a new INI format was introduced. Previous INI files are not compatible with newer versions and INI files created in 4.2+ are not compatible with previous versions.

The most critical changes involve escaped characters. In prior versions, no characters were escaped in strings stored in the INI file. As of 4.2, special characters must be correctly escaped to be read correctly. Among other things, this allows us to handle non-ACII characters in INI file strings. For more details on these changes, see the Technical Details section below.

When starting 4.2+ Master, IS, or Studio for the first time, Imatest will check for any previous INI files. Specifically Imatest will search for imatest.ini, rescharts.ini, and multicharts.ini. If any of these files are found, they will be combined into a single file and scanned for any characters that should be escaped. A new file will be created: imatest-v2.ini. This will become the default INI file for all future 4.2+ use. Please note that this process does not apply to custom INI files.

To update a custom INI file to the current INI format, select the INI using the Select INI File… option in the INI File Settings menu. Once you have selected your INI file, click Update INI File. This will create a new file with the following format: <your-name>-v2.ini  Any rescharts.ini or multicharts.ini files in the same folder as your selected INI file will be merged into the new file and updated.

Important Imatest IT Note

If you are using Imatest IT these changes are particularly important to note. Imatest IT never modifies INI files, this is intentional to ensure that an INI file used for margin-sensitive testing are not accidentally changed in the middle of production. Because of this design, you can not use a pre-4.2 INI file with Imatest IT 4.2+ and you can not use a 4.2+ INI file with Imatest IT pre-4.2.

To update an INI file to be used with Imatest IT, please follow the instructions above for updating and INI file.

Technical Details

This section will enumerate the specific changes made to the actual format.

INI File Version

A new INI key has been introduced related to the new INI version. To assist tracking of INI file version the following key has been added:

IniFileVersion = #

Where # will be the version (2 in 4.2). You should never modify this value (removing this line will cause it to be replaced during the next run). Changing this value will likely result in a crash.

Escaped Characters

Prior to Imatest 4.2.0, no special characters in Imatest INI files were escaped. As of 4.2.0, special characters that might have other meanings (primarily backslashes) must be escaped to be properly read. For example, the following line might appear in a pre-4.2 INI file:

save_dir = C:\imatest\my_images\Results

In 4.2+ this line will appear as:

save_dir = C:\\imatest\\my_images\\Results

Unicode Characters

In previous versions of Imatest, INI files were not able to handle non-ASCII characters. As a result, no non-Latin characters could appear in file names or paths. The new INI format does allow for Unicode characters to be saved. This is accomplished by using the standard Unicode character code with special preceding characters. For example, 文件名 would be represented as the following:

\u6587\u4ef6\u540d

Arrays

In previous versions, INI file arrays were not always completely robust for all data types. The new format allows for more complex arrays.

Pre-4.2:

roi_mult: 10 10 20 20; 100 100 0 0;

4.2+:

roi_mult: [ 10 10 20 20; 100 100 0 0;]

Nested arrays will allow for 3+ dimensional arrays. However, these larger dimension arrays are not currently used by Imatest.

]]>
https://www.imatest.com/2015/08/ini-file-format-changes-in-4-2/feed/ 0