Blemish Detect  detects visible sensor defects (typically blurred spots caused by dust in front of the image sensor). To ensure that visible blemishes are flagged— and blemishes below the threshold of visibility are not— the image is filtered by a response function derived from the Human Visual System (HVS). Filter settings are adjustable for a wide range of applications and viewing conditions. The IT version of Blemish Detect, which can be incorporated into automated testing systems, can significantly improve sensor yields.

For best results, Blemish parameters must be chosen with care.
Be sure to read the instructions!

The Blemish module was introduced in Imatest 3.6, January 2010 (Master-only)

Blemish Detect uses the same uniform featureless image as Light Falloff, but its emphasis is different— on defects or blemishes, i.e., localized density variations, rather than illumination falloff (vignetting).

The Human Visual System

The human eye's Contrast Sensitivity Function (CSF), shown below, is a measure of the eye's MTF response. It is limited by the eye's optical system and cone density at high spatial (or angular) frequencies and by signal processing in the retina (neuronal interactions; lateral inhibition) at low frequencies. Various studies place the peak response at bright light levels between 6 and 8 cycles per degree.The CSF formula, which has the form k₁f exp(-k₂f ), is described in the page on Subjective Quality factor (SQF).

Contrast sensitivity function

The human eye is sensitive to relative luminance differences. That's why we think of exposure in terms of zones or f-stops, where changing exposure by one f-stop or zone means halving or doubling the light. The smallest luminance difference the eye can distinguish in bright light (ΔL) is expressed by the Weber-Fechner law,

ΔL/L = 0.01

(G. Wyszecki & W. S. Stiles, "Color Science," Wiley, 1982, pp. 567-570). This is a relative difference of 1%. The thresholds, described below, are derived from this number (though they're typically larger in practice).

The coincidence In order to get good blemish readings you must filter out high spatial frequencies to remove noise— random variation of pixels caused by photons and thermal effects, which should not be misinterpreted as blemishes, filter out low spatial frequencies to remove gradual pixel level changes caused by lens vignetting. This filtering bears a curious resemblance to the contrast sensitivity function (though for practical reasons it's not an exact replica).

Algorithm

• The image is linearized using the equation, V = Pixel level^(1/gamma), where gamma is the approximate exponent used to encode the image. Gamma is around 0.5 for standard color spaces (sRGB, Adobe RGB) and 1.0 for raw images.
• The linearized image is filtered with a bandpass filter whose frequency cutoffs (set in the input dialog box) are derived from, but not identical, to the human visual system's Contrast Sensitivity Function. The actual bandpass must be wider to account for noise, light falloff, and a range of viewing conditions. These factors are discussed in detail below.
• The filtered image has a mean value of 0 because the highpass portion of filter removes the dc component (spatial frequency = 0).
• The filtered image is normalized (divided) by the mean of the linearized (unfiltered) image. This scales it so units are the same as ΔL/L in the Weber-Fechner law described above, i.e., a value of -0.01 means that a pixel is darker than its average surroundings by 0.01 (1%). This signal is called the "blemish-filtered" signal.
• An error is detected if a blemish-filtered value is lower than minus the threshold. Example: a value lower than -0.02 for a threshold of 0.02.
• The mean of the vertical lines is calculated by averaging columns of blemish-filtered pixels, scanned horizontally, excluding 5% of the total height at the top and bottom of the image. The mean of the horizontal lines is calculated by averaging rows of blemish-filtered pixels, scanned vertically, excluding 5% at the left and right. These lines are displayed in the Line blemish plot. An error is detected if the absolute value of either mean is greater than the line threshold.

Lowpass and Highpass filtering  (familiar if you have a background in Electrical Engineering) A lowpass filter (LPF) attenuates high frequencies, i.e., passes low frequencies. The most frequent application is noise reduction. A highpass filter (HPF) attenuates low spatial frequencies, i.e., passes high frequencies, and removes the dc (zero frequency) component of the signal. The mean value of a signal after passing through an HPF is zero. To display such a signal as an image, an artificial dc level (typically equal to the mean of the original signal) has to be added. Highpass filtering reduces gradual changes in the signal (for example, pixel level variation caused by lens vignetting, i.e., Light Falloff). A bandpass filter is a combined lowpass/highpass filter. The human eye is a bandpass filter. Filters are characterized by bandwidth (typically the frequency where the ratio of output to input power (the transfer function) falls to 0.5, and by the rate of rolloff, which is more rapid for the gaussian filters than for exponential filters (selectable in the input dialog box). The word "bandwidth", which has become widely used in English slang, has its origins in Electrical Engineering. If the original (unfiltered) image contains a tiny speck (perhaps a dead or hot pixel), a lowpass filter will broaden it and reduce its amplitude. For the human eye, the speck will be visible if its amplitude is large enough— if ΔL/L is greater than a threshold, which must be at least 0.01, as described above, for large objects with well-defined boundaries. The threshold for small objects is typically higher.

Instructions

To prepare an image for Blemish Detect,

• Set your camera or lens to manual focus if available.
• Wide angle lenses, which usually have significant vignetting, are not recommended for measuring sensor defects.
• The aperture (f-stop) setting affects the results because many blemishes are caused by dust on the surface of the imaging chip, separated from the sensor by the Bayer, anti-aliasing, and infrared (IR) filters (up to 1mm in large sensors). This separation blurs the dust speck. The blur radius increases as the aperture increases (as the f-stop number is decreased). For small dust spots, the blemish amplitude decreases as the aperture increases, i.e., intensity is inversely proportional to radius.
• 
  Screen Pattern module for
  Blemish Detect and Light Falloff.
  Photograph an evenly illuminated uniform subject.
• One of the simplest ways to obtain a uniform subject is to photograph your computer monitor at a distance of 1-3 inches (2-8 cm) using the Imatest Screen Patterns module, shown on the right. Click on Screen Patterns on the right of the Imatest main window, then maximize the screen. You can adjust the Hue, Saturation, and Lightness (H, S, and L, which default to 0,0,1) if required. (Thanks to Jonathan Sachs for this suggestion.)
  
  This method works best with flat screen LCDs with wide viewing angles (not so well with laptops).
• We recommend using opal diffusing glass mounted close to the lens. Opal glass is available in the US from Edmund Optics. (Thanks to Bart van der Wolf for the suggestion.) If a diffuser is used, a light box may be substituted for the monitor. (Light boxes are typically brighter but less uniform.)
• CRTs are not recommended because raster scan tends to make exposures less uniform.
• (We prefer other approaches, but this sometimes works.) Photograph an evenly-lit smooth matte sheet (white or gray). Be careful not to shade it. Outdoors illumination (shade) sometimes provides very even illumination. Getting uniform illumination can be a challenge with wide-angle lenses; it's almost impossible to avoid shading part of the card. Opal diffusing glass is strongly recommended.
• The subject does not need to be in focus (you don't even need a lens to measure sensor uniformity); the goal is to measure sensor uniformity, not features of the subject. For typical measurements, set exposure compensation to overexpose by about one f-stop. (You may, however, use any exposure you choose.)
• Save the image as a RAW file or maximum quality JPEG.

To obtain truly even illumination for precision scientific measurements you'll need an integrating sphere from a supplier such as Image Engineering, SphereOptics, Labsphere, or Electro-Optical Industries (EOI). The SphereOptics system on the right is about $8,000, including variable attenuator, power supplies, operating software, and calibration. It is available in sizes from 4 to 76 inches. The Image Engineering Spherical Transparency Illuminator LE6 was designed for a variety of photographic applications. A mechanical shutter can dim the light down to 1% of the maximum illumination without changing the color temperature. The DSC Labs Ambi Illuminator also provides extremely even illumination from a variety of external light sources. It is excellent for illuminating transparencies. Integrating spheres aren't cheap. Controlling Veiling Glare in an Optical Imaging System by Amber Czajkowski (University of Arizona) contains a description of a really neat do-it-yourself integrating sphere project, based on stainless steel balls from http://www.gazingballoutlet.com, which sells balls up to 30 inches (0.75 meters) in diameter! A 16 inch (40 cm) ball is under $100 USD. Other less expensive alternatives include Kyoritsu calibrated light sources (especially the pattern light boxes), available in the North America from C.R.I.S. Tsubosaka (Japan) also has some interesting products. There are no obvious US or European distributors. Also of interest: DNP Light boxes.

To run Blemish Detect,

• Start Imatest and click on Blemish Detect. Very large files (height x width x colors over 40 MB) may cause memory overflow problems. Files over 40 or 80 MB can be automatically reduced 1/2x linearly (using 1/4 the memory). Click Settings, Options I (in the Imatest main window) and make the appropriate setting in LARGE FILES (Light Falloff, Blemish, Distortion). Open the image file.
  
  

  Multiple file selection Several files can be selected using standard Windows techniques (shift-click or control-click). Depending on your response to the multi-image dialog box you can combine (average) several files or run them sequentially (batch mode). Combined (averaged) files  are useful for measuring fixed-pattern noise (at least 8 identical images captured at low ISO speed are recommended). The combined file can be saved. Its name will be the same as the first selected file with _comb_n appended, where n is the number of files combined. Batch mode  allows several files to be analyzed in sequence. There are three requirements. The files should (1) be in the same folder, (2) have the same pixel size, and (3) be framed identically. The input dialog box for the first run is the same as for standard non-batch runs. Additional runs use the same settings. Since no user input is required they can run extremely fast. If the order of the files in a batch runs is different from the selection order, click Settings, Options I (in the Imatest main window) and change the setting in Batch run order. The sequence may be affected by Windows Explorer settings. One caution: Imatest can slow dramatically on most computers when more than about twenty figures are open. For this reason we recommend checking the Close figures after save checkbox, and saving the results. This allows a large number of image files to be run in batch mode without danger of bogging down the computer.

  
  Three cropping (ROI selection) options are available by clicking Options I in the Imatest main window. These include
  
  

  Don't ask to crop. (Use Crop ... borders settings in LF input dialog box.)

  Select crop by dragging cursor. Ask to repeat crop for same sized image.

  Select crop by dragging cursor. Do not ask to repeat crop.

  
  The second option (Select crop by dragging cursor. Ask to repeat crop for same sized image), which is similar to the ROI selection in SFR, is typically preferred.

Input dialog box

The input dialog box appears after the image file has been selected.

Input dialog box

• Title  Defaults to the file name. You can change it if you wish.

The blue region to the left of the filter response plot contains the blemish filter and detection settings.

• Lowpass  The high frequency rolloff of the filter (so named because it "passes" low frequencies). The dropdown menu specifies the equation of the rolloff: either an exponential (exp(-kf )) or a gaussian (exp(-(kf )^2)) rolloff. The exponential rolls off more slowly— is less aggressive in removing high frequencies— but resembles the human eye's rolloff. The slider specifies the approximate cutoff frequency. The effects of the lowpass filter are visible in the high spatial frequency region on the right of the response plot.
• 
  Zone boundaries (red lines)
  Highpass  The low frequency rolloff of the filter. The dropdown menu specifies the equation of the rolloff: either an exponential (exp(-kf )) or a gaussian (exp(-(kf )^2)) rolloff. The slider specifies the approximate cutoff frequency. The effects of the lowpass filter are visible in the low spatial frequency region on the left of the response plot.

The filter response plot, to the right of the Filter area, shows the filter response curves. The blue (lower) curve is the combined LPF and HPF response. The black (upper) curve is the combined response normalized to a maximum value of 1.0; this is the curve used for the actual filter calculations.

• Zone boundaries: % of total W or H  Because regions near the edges of the image may be less visually critical or may have edge-related irregularities, Blemish allows you to divide the image into nine areas for analysis, defined by pairs of vertical and horizontal dividing lines. The values entered are for locations of the vertical lines (defining left and right regions) and horizontal lines (defining top and bottom regions), expressed in percentages of total Width and Height. The illustration on the right shows both boundaries (red lines) = 6%. The thresholds (and numbers of allowable pixels outside the threshold) can be set separately for the center, (left, right), (top, bottom), and corner regions.
• Thresholds  can be set separately for each of the four zone groups and for lines: Center, (Left, Right), (Top, Bottom), Corners, and Lines.
  • Value is the absolute value of the threshold. A blemish is detected when the blemish-filtered signal level is below –threshold. (At the present time only darkened regions are considered to be blemishes.) Recall that the blemish-filtered signal has a mean of zero (due to highpass filtering) and has been normalized to the mean signal level (before filtering), so the threshold corresponds to ΔL/L in the Weber-Fechner law. For practical reasons (sensor noise, viewing conditions), thresholds are generally larger than 0.01. Default values are 0.02 for the center region, 0.03 for top, bottom, left, and right, 0.04 for the corners, and 0.01 for lines.
  • Count is the number of allowable blemish pixels in each of the nine zones (specified for the four zone groups). The default is [1 1 1 1 1]. Used for pass/fail decisions in manufacturing testing. Primarily of interest to IT customers.

The Preview image, below the filter response plot (to the right of the Plot area) gives a 280x200 pixel preview of the filtered image. It is updated whenever filter settings are changed, making it useful for tuning. Preview displays include

Original image	Original image for reference. Not affected by settings.
Image with blemish filter	Filtered image. Shows the effects of the filters on blemishes. Since only a small (280x200) segment of the original image is shown, does not show the suppression of long-range variation from the highpass filtering.
Pseudocolor image- linear	The blemish-filtered image in pseudocolor.
Pseudocolor image- spot emphasis	The blemish-filtered image in pseudocolor with spot emphasis: shows deviation from the local average. Values above 0 are white. Values below -0.1 are black.
Pseudocolor image- > threshold	The blemish-filtered image in pseudocolor showing only regions below (–) the threshold. Higher values are white. Values below -0.1 are black.

The Preview image location can be changed by pressing Preview location, and dragging the 280x200 pixel crop with the mouse. The final crop is shown in red. It should include blemishes of interest (above or below the threshold).

The Plot region specifies which figures to plot. The figures are described in Detail in the Results section, below.

• The four image plots selected by checkboxes on the left display the image with different signal processing applied (Original (none), Blem filter (LPF+HPF), HPF, HPF >Noise> (noise boosted 10X)) are illustrated below.
• 2D blemish plot  Can be turned off (No blemish plot) or displayed with the linear or spot emphasis setting. With spot emphasis, blemish-filtered results greater than 9 are displayed as pure white. This makes it easy to locate blemishes, which can get lost in the linear display. If results, histogram & response is checked, a plot of the response, a table of results, and a histogram of levels is displayed below the main image.
• Line blemish plot  plots the amplitude of horizontal and vertical lines in the image. Lines tend to be more visible than spots, hence they have their own threshold (default value = 0.01).

The Settings region lets you enter settings that affect the results.

• Speedup  When unchecked, some Light Falloff information (levels at sides and corners) is calculated and saved. This button may be removed when Blemish functionality is added to Light Falloff.
• Gamma  (Settings area). The default value of gamma, 0.5, is typical for digital cameras. Gamma is used to linearize the image. It can be measured by Stepchart using any one of several widely-available step charts. (Reflection charts are easiest to use but transmission charts can also be used to measure dynamic range.)
• Channel  Selects Red, Blue, Green, or Y (luminance) channel
• Corner and side regions  (default 32x32 pixels) allows you to select the areas at the corners and sides of the images to be analyzed. These affect the edge and corner level calculations, which are saved in CSV and XML output when Speedup is unchecked. Choices include 10x10 pixels, 32x32 pixels, 1% (min. 10x10), 2% (min. 10x10), 5%, and 10%. This may be removed when Blemish functionality is added to Light Falloff.
• Crop pixels near borders (L, R, T, B)  (Settings area). Available only if Don't ask to crop. (Use Crop ... borders settings in LF input dialog box.) is selected in the Options I. If checked, the image is cropped by the number of pixels indicated near the left, right, top, and bottom borders.

• Hot and dead pixels  (Settings area) By checking the appropriate boxes you can display hot pixels (red x) and/or dead pixels (blue •). Hot pixels are stuck at or near the sensor's maximum value (255 in 8-bit files) and dead pixels are stuck at or near 0. You can choose between hot/dead pixel detection in any channel, all channels or the selected channel.

Because signal processing— especially JPEG compression— can cause these values to shift, you can use the sliders to set the detection threshold between 6-255 for hot pixels and 0-249 pixels for dead pixels. (The extreme values are for measurements made on white or black fields.) Clicking on < or > at the ends of the sliders adjusts the threshold by 1. The default values are 252 and 4, respectively. Settings are saved between runs. JPEG files must be saved at the highest quality level for this feature to work; isolated hot and dead pixels tend to be smudged at lower quality levels. Details are described in Light Falloff: Imatest Master .

Tuning

It is very important to set the key parameters of Blemish Detect so detected errors correspond to visible defects— to minimize both "false positives" and "false negatives".

The key parameters are

• Lowpass filter function and cutoff frequency. The exponential function is generally recommended because it's closest to the eye's response. The gaussian should only be used if a sharper cutoff is required (for more aggressive noise reduction, perhaps) The cutoff frequency (slider position) shoud be tuned so small but visible spots in a test image (exmaple below) have appropriate intensity relative to larger spots and so that tiny spots only go over the threshold if they're intense enough to be visible.
• Highpass filter function and cutoff frequency. The more aggressive gaussian rolloff is generally preferred to remove light falloff and edge effects, which should not be mistaken for blemishes. The curoff frequency (slider position) should be selected to large spots are not too light. This setting often involves a tradeoff.
• Threshold values, particularly for the center region, should be set after the filter settings.

Test image

(A much more sophisticated approach is under development, but the image below is a start.)

Print or display the full-sized version of the image shown below, which you can download by clicking here or on the image itself. The image should viewed under the typical conditions for the product under development as well as for the largest angular field of view that you expect.

Use it to set the key parameters so that spots you consider to be objectionable are above the threshold and spots that are not objectionable are not. If small spots appear to be too strong in the Blemish plot (described below), decrease the lowpass filter frequency (move the slider to the left).

Blemish Detect calibration image. Details shown below.
Click on the image to download the full-sized version (8 megapixels; 2.5MB)

Results

Image figures (original and filtered)

Several image figures are available. These figures are intended to facilitate the setup of filter and threshold parameters by making it convenient to compare images (original or processed) with blemish-filtered results (displayed in pseudocolor). Note that these plots are not anti-aliased, so vertical and horizontal nonuniformities (lines) may appear exaggerated in the original (unzoomed) display. To get an accurate impression, you must zoom in by clicking on an area of interest or drawing a rectangle. Double-click to zoom back out. The image illustrated below has three sequences of synthesized blemishes, differently sized, from weak to strong.

Original image (shown reduced) with synthesized blemishes

Original image, shown cropped with enlarged spots.

When setting filter parameters (LPF and HPF cutoff frequencies as well as thresholds) this pattern should be observed carefully at appropriate magnification levels.

Blemish-filtered (HPF and LPF) image, cropped and enlarged

This image has been filtered to remove long-range density variations (typically from vignetting) as well as noise The blemish plot (primary results image), shown below, is derived directly from this image. The maximum density of spots should be closely related to the spot visibility in the unfiltered image, above.

Highpass-filtered image

This image shows the effects of the highpass filter only.

Highpass filtered-image with noise boosted 10X

This plot maximizes the display of blemishes and noise, in distinction to the blemish-filtered plots that show blemishes, but minimize noise. Tiny defects are exaggerated.

Blemish plot (the primary results display)

The blemish plot is the key results display.

Blemish display with spot emphasis: the primary results of Blemish Detect.

Spot emphasis clamps the filtered blemish image to a maximum value of 0 and a minimum value of -0.1. This makes the (darker) blemishes much more visible than in the linear (unclamped) image below.

The bottom row is displayed when Results, histogram & Response is checked in the input dialog box. The filter response curve is shown on the left: the lower blue curve is the combined LPF + HPF response. The upper black curve is the response normalized for a maximum value of 1.0 (used for the calculations). The table in the middle displays the number of errors (N Err), the minimum value (Min), and (–) the threshold level (Thr) for the image as a whole as well as for each of the 9 sections and the average vertical and horizontal lines. A histogram of levels is displayed on the right.

Line blemish plot

Lines tend to be more visually prominent that spots or blemishes of similar size. For this reason, they are detected with a different threshold (generally lower; 0.005 in the image below). The plots below are for vertical and horizontal lines in the image. The same filtering is used as for spots. This plot is for an image where significant vertical banding (lines) was visible in the 10x noise boosted plot, but the banding wasn't visible in in the original image displayed full sized (to get around anti-aliasing problems when displayed in reduced size in Matlab figures.

Line blemish plot

The ends of the lines, which are not included in the analysis because of end effects (in the FFT routine) are shown in light gray.

CSV and XML output files

The CSV and XML output files contain additional statistics. Most have obvious meanings.

• Image pixels contains the width, height, and total size in pixels. Hot and Dead pixels show the total count and the fraction (divided to total pixels)
• The x and y coordinates of the hot and dead pixels are listed. The maximum is 100. Coordinates are in pixels from the top-left.

Contact Imatest if you need additional .CSV output. The optional XML output file contains results similar to the .CSV file. Its contents are largely self-explanatory. It is stored in [root name].xml. It will be used for a database product under development. Contact us if you have questions or suggestions.

Saving results

At the end of the run, a dialog box for saving results appears. It allows you to select figures to save and choose where to save them. The default is subdirectory Results in the data file directory. You can change to another existing directory, but new results directories must be created outside of Imatest— using a utility such as Windows Explorer. (This is a limitation of this version of Matlab.) The selections are saved between runs. You can examine the output figures before you check or uncheck the boxes. Figures, CSV, and XML data are saved in files whose names consist of a root file name with a suffix for plot type and channel (R, G, B, or Y) and extension. Example: IMG_9875_ISO1600_RGB_f-stop_ctrG.png. The root file name defaults to the image file name, but can be changed using the Results root file name box. Be sure to press enter. Checking Close figures after save is recommended for preventing a buildup of figures (which slows down most systems) in batch runs. After you click on Yes or No, the Imatest main window reappears.

Figures can be saved as either PNG files (a standard losslessly-compressed image file format) or as Matlab FIG files, which can be opened by the Open Fig file button in the Imatest main window. Fig files can be manipulated (zoomed and rotated), but they tend to require more storage than PNG files. They are especially nice because 3D files can be reopened and rotated, but you should exercise caution because 3D files can be very large— often several megabytes.

The CSV and XML files contain EXIF data, which is image file metadata that contains important camera, lens, and exposure settings. By default, Imatest uses a small program, jhead.exe, which works only with JPEG files, to read EXIF data. To read detailed EXIF data from all image file formats, we recommend downloading, installing, and selecting Phil Harvey's ExifTool, as described here.