Documentation

The Imatest Pass/Fail Monitor

The Pass/Fail monitor (introduced In Imatest 4.0) provides a real-time indication of whether a device has passed or failed a test. It saves the trouble of digging through results— in figures or CSV or JSON files. Key features:

  • It can stay open while modules run, displaying results immediately after calculations are complete. It works best on systems with a high resolution screen or dual screens.
  • It interfaces with most Imatest analysis modules.
  • It can be extremely valuable for developing and testing Pass/Fail criteria for Imatest IT (Industrial Testing). It can call several utilities for helping with this process.
  • It displays pass/fail results (in a large table, parsed for readability) and either the pass/fail data structure or the pass/fail history (the log of Pass/Fail summary results since the Pass/Fail Monitor was first opened after starting Imatest).
  • Results are stored in a global workspace so that the P/F Monitor can be turned off and back on without losing the results (i.e, the pass/fail history).

Introduction

The Pass/Fail Monitor is opened by pressing the Pass/Fail Monitor button in the Imatest main window or by pressing File, Pass/Fail monitor in Rescharts or Multicharts.

Here is an example of the monitor. The SFRplus image has been tested for a rather large number of Pass/Fail criteria. This image failed 2 measurements (hence FAIL – 2 ) because the camera had very strong sharpening– Oversharpening was above the maximum limit, and this also raised the center MTF above the maximum.

PFmonitor_typicalPass/Fail monitor, showing results from SFRplus for a test that failed 2 measurements.

The key pass/fail results, parsed for readability, are in the large table that occupies most of the window. Results consist of

Pass/Fail measurement, FAIL or P (Pass), measurement value, limit type, limit value(from the spec.ini file)

The box to the right of the thumbnail image (above) contains either

  • the original (unparsed) pass/fail results, which can be difficult to read, but contains all the results.(This is the default when opened: you can change it using the dropdown menu immediately above this box.)
  • the History Log (CSV format) (initialized the first time the Pass/Fail monitor is opened after starting Imatest).

PFmonitor_history
Pass/Fail monitor results for a test that passed, showing History Log
(number, P/F, date/time, Module, File name: most recent on top; FAILs in red)

When a measurement passes, the big results box is Green.

Table 1. Pass/Fail file summary
Section Description
[blemish] includes [uniformity] measurements.
[colorcheck] for Colorcheck
[distortion] for Distortion
[multi] Multicharts and Multitest color/grayscale modules. There are similar entries in [colorcheck] and [stepchart].
[sfr] [sfr] measurements are a subset of SFRplus
[sfrplus] includes [sfr] measurements.
[sharpness] for Random/Dead Leaves (Spilled Coins) and Star; either fixed or Rescharts modules
[stepchart] for Stepchart
[uniformity] A subset of measurements in Blemish detect

Setting up Pass/Fail

We summarize the setup here. Details are presented in Implementing Pass/Fail in Imatest.

  1. Create an ini Pass/Fail file that embodies your specification, i.e., contains maxima, minima, or limits for the measurements of interest. (This file is sometimes called a “spec.ini” file.) You may use a text editor or the Imatest INI file editor. Available measurements are shown in Implementing Pass/Fail. Measurements need to be hard-coded into Imatest (though thresholds can always be set by the user). If you don’t find a measurement you need, please write us at support at imatest dot com, and we’ll try to add it to the current beta build with 1-3 days.We plan to make some standard Pass/Fail files available, starting with the Skype & Lync Video Capture Specification.
  2. Add a link (variable name passFail) to the file location in the [api] section of imatest.ini (or the ini you use for running Imatest): [api]
    passFail = C:ImatestDataMiscsample_spec.ini
  3. Run Imatest, then open the Pass/Fail monitor with the Pass/Fail Monitor button in the Imatest main window or by pressing File, Pass/Fail monitor in Rescharts or Multicharts.

 

Notes on the example:

[sharpness] refers to sharpness modules in Rescharts (other than SFR or SFRplus) or derived from Rescharts, including SFR, Star, and Random.

[multi] refers to any of the Multicharts or Multitest chart measurements (grayscale and color charts).


[sfrplus]
Pixel_Aspect_Ratio_limits = .98 1.02
Oversharpening_center_pct_max = 20
Edge_roughness_center_pixels_max = .1;
Rotation_degrees_max = 1.5
FOV_degrees_diagonal_min = 67
Convergence_angle_max = 1.6
Horizontal_bars_OK_min = 1
All_Edge_IDs_detected = 1
Bayer_decode = 1
Mirrored_chart = 1
Chart_mean_pixel_level_bounds = .3 .8
Low_pixel_saturation_fraction_max = .3
High_pixel_saturation_fraction_max = .3
Chart_radial_pixel_shift_max = 40[blemish]
Dead_pixels_max = 618
Hot_pixels_max = 618
Optical_center_offset_max = 37.14
Relative_illumination_worst_corner_pct_min = 29
Relative_illumination_corner_diff_pct_max = 21
Uniformity_RoverG_corners_pct_max = 10.8
Uniformity_BoverG_corners_pct_max = 13.9
Blemish_size_pixels = 20 49
Blemish_maximum_count = 2 1
Dead_pixel_clusters_max = 1
R_div_G_ctr_corner_diff_pct_max = 10
B_div_G_ctr_corner_diff_pct_max = 10
Center_R_div_G_unnorml_limits = [.9 1.1]
Center_B_div_G_unnorml_limits = [.9 1.1][sharpness]
MTF50P_min = .2
Oversharpening_pct_limits = -20 20[multi]
gamma_limits = .4 .75

Utilities

Several utilities can be opened from the dropdown menus.

File dropdown menu
Copy P/F structure
to clipboard
Copy the original P/F structure to the clipboard where it can be pasted into an editor for saving or viewing. The structure is formatted with field names on the left, ‘: ‘ in the middle, the value(s) on the right. It often contains more information (though harder to read) than the parsed P/F results below. Example: ini_time_size: 13-Nov-2013 17:09:35
passFailFile: C:ImatestprojectsSpec.ini
started_at: 2014-01-03 15:19:37
ended_at: 2014-01-03 15:19:44
runtime: 5.6879
all_tests_passed: 0
Horizontal_bars_OK_min: 1
Horizontal_bars_OK: 1
Horizontal_bars_OK_passed: 1
Copy P/F CSV results
(to clipboard)
Copy the parsed results (in CSV format) to the clipboard for saving or importing into Excel. Results have the form, P/F measurement (sorted: FAILs on top), Pass/Fail, Value, limit type, Limits.

Example:
Edge_roughness_center_pixels, FAIL, 0.11827, max, 0.1
Rotation_degrees, P, 0.2779, max, 1.5To import CSV text into Excel: Paste it into a cell, then click on the Paste Options icon that appears next to the cell. Select Text Import Wizard…, Delimited, Comma (only: uncheck Space), Finish.

Copy CSV History Log
(to clipboard)
Copy the CSV History log to the clipboard— mostly for importing into Excel, where it can be examined in detail. When it is copied, lines are in ascending sequence (the most recent run is last). This is the opposite of the display, where the most recent results are shown on the top. Also, up to 2000 results may be presented vs. a maximum of 50 in the display.
Reset test statistics Reset the number of tests run, passed, and failed. Most displays will also be reset.
View dropdown menu
Narrow Optimize the window for narrow display. May be useful on small monitors.
Wide (default) Optimize the window for wide displays. (default)
Edit dropdown menu
Edit Pass/Fail INI file Open the Pass/Fail file in the Imatest INI File Editor, which is fairly convenient for editing an existing files: You can add, modify, or delete lines or sections.
Copy P/F INI file name
(to clipboard)
Copy the Pass/Fail file name to the clipboard: useful for pasting in to the Open dialog in the editor of your choice.
IT Settings
(Select P/F INI file)
Open the IT Settings window (settings for the Imatest Industrial Testing version). This window is where you specify the Pass/Fail file.
Help dropdown menu: Open web pages with Pass/Fail information and instructions in a browser.