Introduction to sharpening
Sharpening is an important part of digital image processing. It restores some of the sharpness lost in the lens and image sensor. Every digital image benefits from sharpening at some point in its workflow— in the camera, the RAW conversion software, and/or image editor. Sharpening has a bad name with some photographers because it’s overdone in some cameras (mostly low-end compacts and camera phones), resulting in ugly “halo” effects near edges. But it’s entirely beneficial when done properly.
Almost every digital camera sharpens images to some degree. Some models sharpen images far more than others— often excessively for large prints. This makes it difficult to compare cameras and determine their intrinsic sharpness unless RAW images are available. [Imatest has developed an approach to solving the problem— standardized sharpening, described below, which is useful for comparing “black box” cameras, but is not recommend for camera engineering or development work.]
The sharpening process
Sharpening on a line and edge
A simple sharpening algorithm subtracts a fraction of neighboring pixels from each pixel, as illustrated on the right. The thin black curve in the lower part of the image is the input to the sharpening function: it is the camera’s response to a point or a sharp line (called the point or line spread function). The two thin dashed blue curves are replicas of the input, reduced in amplitude (multiplied by –ksharp ⁄ 2) and shifted by distances of ±2 pixels (typical of the sharpening applied to compact digital cameras). This distance is called the sharpening radius R. The thin red curve the impulse response after sharpening— the sum of the black curve and the two blue curves. The thick black and red curves (shown above the thin curves) are the corresponding edge responses, unsharpened and sharpened.
Sharpening increases image contrast at boundaries by reducing the rise distance. It can cause an edge overshoot. (The upper red curve has a small ovrshoot.) Small overshoots enhance the perception of sharpness, but large overshoots cause “halos” near boundaries that may look good in small displays such as camera phones, but can become glaringly obvious at high magnifications, detracting from image quality.
Sharpening also boosts MTF50 and MTF50P (the frequencies where MTF drops to 50% of its low frequency and peak values, respectively), which are indicators of perceived sharpness. (MTF50P is often preferred because it’s less sensitive to strong sharpening.) Sharpening also boosts noise, which is can be a problem with noisy systems (small pixels or high ISO speeds).
Sharpening is a linear process that has a transfer function.The formula for the simple sharpening algorithm illustrated above is,
Lsharp(x) = [ L(x) – (ksharp /2) * (L(x-V) + L(x+V)) ] / (1- ksharp )
L(x) is the input pixel level and Lsharp(x) is the sharpened pixel level. ksharp is the sharpening constant (related to the slider setting scanning or editing program). V is the shift used for sharpening.
V = R ⁄ dscan
where R is the sharpening radius (the number of pixels between original image and shifted replicas) in pixels. dscan is the scan rate in pixels per distance. 1/dscan is the spacing between pixels. The sharpening algorithm has its own MTF (the Fourier transform of Lsharp(x) ⁄ L(x)).
MTFsharp( f ) = (1- ksharp cos(2π f V )) ⁄ (1- ksharp )
This equation boosts response at high spatial frequencies with a maximum where
cos(2π f V ) = cos(π) = -1, or f = 1/(2V ) = dscan/(2R).
This is equal to the Nyquist frequency, fN = dscan ⁄ 2, for R = 1 and lower for R > 1. Actual image sharpening is a two dimensional operation.
Here is a plot showing the sharpening transfer function (MTF) for strength (ksharp) = 0.15 and R (sharpening radius) = 1, 2, and 3. Note the the bottom of the plot is MTF = 1 (not 0). At the commonly-used sharpening radius of 2, MTF peaks at half the Nyquist frequency (0.25 cycles/pixel) and drops back to 1 at the Nyquist frequency (0.5 Cycles/Pixel).
Sharpening MTF for radii = 1, 2, and 3 pixels
Note that the sharpening MTF is cyclic, i.e., it oscillates. For example, MTF for sharpening radius = 2 increases to a second peak at 0.75 C/P, etc. This can affect MTF measurements for sharpened images.
The MTF plot on the right is for an image sharpened with radius ≅ 2. This camera is quite sharp prior to sharpening— it has quite a bit of energy above the Nyquist frequency. Oscillation above Nyquist causes a ramp in the response around MTF ≅ 0.3 (30%). This causes MTF30 (the spatial frequency where MTF is 30% of the low frequency value) to become extremely unstable: a small change in sharpness can cause a large change in MTF30. In general MTF50 is more stable than MTFnn at lower levels— MTF20, MTF10, etc.
MTF plot for a camera sharpened with radius ≅ 2,
Here is an example from the Canon EOS-6D full-frame DSLR. A JPEG image (blue curve) with sharpening slightly reduced is compared with a TIFF image converted from raw by dcraw (with no sharpening and noise reductions). The exact same exposure and regions were used for each curve (both JPG and raw files were saved).The peak at 0.25 cycles/pixel indicates that sharpening with radius = 2 was used. The plot deviates from the ideal plot (for a very simple sharpening algorithm) for spatial frequencies above 0.35 C/P. The EOS-6D allows you to change the amount of sharpening but not the radius for camera JPEGs. (I’m not happy with this limitation.)
Sharpening MTF, comparing
Corresponding edges for the Canon EOS-6D: unsharpened (left), sharpened (right)
These curves (and the curves below for the Panasonic Lumix LX7) show how MTF curves (red for raw and blue for JPEG (sharpened) in the MTF Compare plot, above) correlate to edge response. The modest amount of overshoot (“halo”) on this edge would not be objectionable at any viewing magnification. Better overall performance would be achieved with a sharpening radius of R = 1 and an appropriate sharpening amount.
Here is another example from the Panasonic Lumix LX7. As with the Canon, the same exposure and regions are used: one from a JPEG image (default sharpening, which is very strong), and one from a raw image, converted with no sharpening or noise reduction.The sharpening radius of 1 makes for a sharper image at the pixel level than JPEGs straight out of the EOS-6D, above. Of course the EOS-6D has twice as many pixels (20 vs. 10), but the difference in sharpening accounts for the relatively close sharpness noted in the post, Sharpness and Texture from Imaging-Resource.com.
Sharpening MTF, comparing the
Corresponding edges for the Panasonic Lumix Lx7: unsharpened (left), sharpened (right)
These curves show how MTF curves (red for raw and blue for JPEG (sharpened) in the MTF Compare plot, above) correlate to edge response. The LX7 is much more strongly sharpened in both frequency and spatial domains than the EOS-6D (above). It also has a smaller sharpening radius (R = 1). The strong sharpening would be visible and objectionable at large magnifications, though is results in some impressive measurements: sharpened MTF50P is 2600 LW/PH vs. 2239 for the EOS-6D, which has twice as many pixels. (The EOS-6D would do better,both visually and numerically (i.e., better measurements) with a sharpening radius of R = 1 instead of 2.)
Oversharpening and Undersharpening
Oversharpening or undersharpening is the degree to which the image is sharpened relative to the standard sharpening value. If it is strongly oversharpened (oversharpening >about 30%) the image might look better (especially if enlarged) with less sharpening. If it is undersharpened (oversharpening < 0; undersharpening displayed) the image will look better with more sharpening. Basic definitions:
Oversharpening = 100% (MTF( feql ) – 1)
where feql = 0.15 cycles/pixel = 0.3 * Nyquist frequency for reasonably sharp edges (MTF50 ≥ 0.2 C/P).
feql = 0.6*MTF50 for MTF50 < 0.2 C/P (relatively blurred edges)
When oversharpening < 1 (when MTF is lower at feql than at f = 0), the image is undersharpened, and
Undersharpening = –Oversharpening is displayed.
If the image is undersharpened (the case for the EOS-1Ds shown below), sharpening is applied to the original response to obtain Standardized sharpening; if it is positive (if MTF is higher at feql than at f = 0), de-sharpening is applied. (We use “de-sharpening” instead of “blurring” because the inverse of sharpening is, which applied here, is slightly different from conventional blurring.) Note that these numbers are not related to the actual sharpening applied by the camera and software.
Examples: under and oversharpened images
The 11 megapixel Canon EOS-1Ds DSLR is unusual in that it has very little built-in sharpening (at least in this particular sample).
The average edge (with no overshoot) is shown on top; the MTF response is shown on bottom. The black curves are the original, uncorrected data; the dashed red curves have standardized sharpening applied.
Standardized sharpening results in a small overshoot in the spatial domain edge response, about what would be expected in a properly (rather conservatively) sharpened image. It is relatively consistent for all cameras.
|The image above is for the 5 megapixel Canon G5, which strongly oversharpens the image— typical for a compact digital camera.A key measurement of rendered detail is the inverse of the 10-90% edge rise distance, which has units of (rises) per PH (Picture Height). The uncorrected value for the G5 is considerably better than the 11 megapixel EOS-1Ds (1929 vs. 1505 rises per PH), but the corrected value (with standardized sharpening) is 0.73x that of the EOS-1Ds. Based on vertical pixels alone, the expected percentage ratio would be 100% (1944/2704) = 0.72x. MTF50P is not shown. It is displayed when Standardized sharpening is turned off; it can also be selected as a Secondary readout. For this camera MTF50P is 0.346 cycles/pixel or 1344 LW/PH, 9% lower than MTF50. It is a better sharpness indicator for strongly oversharpened cameras, especially for cameras where the image will not be altered in post-processing.|
These results illustrate how uncorrected rise distance and MTF50 can be misleading when comparing cameras with different pixel sizes and degrees of sharpening. MTF50P is slightly better for comparing cameras when strong sharpening is involved.
Uncorrected MTF50 or MTF50P are, however, appropriate for designing imaging systems or comparing lens performance (different focal lengths, apertures, etc.) on a single camera.
“Unsharp masking” (USM) and “sharpening” are often used interchangeably, even though their mathematical algorithms are different. The confusion is understandable rarely serious because the end results are visually similar. But when sharpening is analyzed in the frequency domain the differences become significant.
“Unsharp masking” derives from the old days of film when a mask for a slide, i.e., positive transparency, was created by exposing the image on negative film slightly out of focus. The next generation of slide or print was made from a sandwich of the original transparency and the fuzzy mask. This mask served two purposes.
- It reduced the overall image contrast of the image, which was often necessary to get a good print.
- It sharpened the image by increasing contrast near edges relative to contrast at a distance from edges.
Unsharp masking was an exacting and tedious procedure which required precise processing and registration. But now USM can be accomplished easily in most image editors, where it’s used for sharpening. You can observe the effects of USM (using the Matlab imsharpen routine) in the Imatest Image Processing module, where you can adjust the blur radius, amount, and threshold settings. Most cameras perform regular sharpening rathan than USM because it’s faster— USM requires a lot more processing power.
Blur = exp(-x2/2σx2 ) / √(2πσx2 )
σx corresponds to the sharpening radius, R. The unsharp masked image can be expressed as the original image summed with a constant times the convolution of the original image and the blur function, where convolution is denoted by *.
LUSM(x) = L(x) – kUSM * Blur
= L(x) * [δ(x) – kUSM exp(-x2/2σx2 ) / √(2πσx2 )] / (1- kUSM /√(2π) )
L(x) is the input pixel level and LUSM(x) is the USM-sharpened pixel level. kUSM is the USM sharpening constant (related to the slider setting scanning or editing program). L(x) = L(x) * δ(x), where δ(x) is a delta function.
The USM algorithm has its own MTF (the Fourier transform of the portion inside the brackets […]). Using F(exp(-px2) ) = exp(-α2/4p) ⁄ √(2p), where F is the Fourier transform,
MTFUSM( f ) = [1- kUSM exp(-f 2σx2/2 ) /√(2π) ] / (1- kUSM /√(2π) )
= [1- kUSM exp(-f 2/ 2 fUSM2 ) /√(2π) ] / (1- kUSM /√(2π) )
where fUSM = 1/σx. This equation boosts response at high spatial frequencies, but unlike sharpening, response doesn’t reach a peak then drop. Actual sharpening is a two dimensional operation.