Introduction to sharpening
Sharpening is an important part of digital image processing. It restores some of the sharpness lost in the lens and image sensor. Every digital image benefits from sharpening at some point in its workflow— in the camera, the RAW conversion software, and/or image editor. Sharpening has a bad name with some photographers because it’s overdone in some cameras (mostly lowend compacts and camera phones), resulting in ugly “halo” effects near edges. But it’s entirely beneficial when done properly.
Almost every digital camera sharpens images to some degree. Some models sharpen images far more than others— often excessively for large prints. This makes it difficult to compare cameras and determine their intrinsic sharpness unless RAW images are available. [Imatest has developed an approach to solving the problem— standardized sharpening, described below, which is useful for comparing “black box” cameras, but is not recommend for camera engineering or development work.]
The sharpening process
Sharpening on a line and edge
A simple sharpening algorithm subtracts a fraction of neighboring pixels from each pixel, as illustrated on the right. The thin black curve in the lower part of the image is the input to the sharpening function: it is the camera’s response to a point or a sharp line (called the point or line spread function). The two thin dashed blue curves are replicas of the input, reduced in amplitude (multiplied by –k_{sharp} ⁄ 2) and shifted by distances of ±2 pixels (typical of the sharpening applied to compact digital cameras). This distance is called the sharpening radius R. The thin red curve the impulse response after sharpening— the sum of the black curve and the two blue curves. The thick black and red curves (shown above the thin curves) are the corresponding edge responses, unsharpened and sharpened.
Sharpening increases image contrast at boundaries by reducing the rise distance. It can cause an edge overshoot. (The upper red curve has a small ovrshoot.) Small overshoots enhance the perception of sharpness, but large overshoots cause “halos” near boundaries that may look good in small displays such as camera phones, but can become glaringly obvious at high magnifications, detracting from image quality.
Sharpening also boosts MTF50 and MTF50P (the frequencies where MTF drops to 50% of its low frequency and peak values, respectively), which are indicators of perceived sharpness. (MTF50P is often preferred because it’s less sensitive to strong sharpening.) Sharpening also boosts noise, which is can be a problem with noisy systems (small pixels or high ISO speeds).
Sharpening is a linear process that has a transfer function.The formula for the simple sharpening algorithm illustrated above is, L_{sharp}(x) = [ L(x) – (k_{sharp }/2) * (L(xV) + L(x+V)) ] / (1 k_{sharp} ) L(x) is the input pixel level and L_{sharp}(x) is the sharpened pixel level. k_{sharp} is the sharpening constant (related to the slider setting scanning or editing program). V is the shift used for sharpening. V = R ⁄ d_{scan} where R is the sharpening radius (the number of pixels between original image and shifted replicas) in pixels. d_{scan} is the scan rate in pixels per distance. 1/d_{scan }is the spacing between pixels. The sharpening algorithm has its own MTF (the Fourier transform of L_{sharp}(x) ⁄ L(x)). MTF_{sharp}( f ) = (1 k_{sharp }cos(2π f V )) ⁄ (1 k_{sharp }) This equation boosts response at high spatial frequencies with a maximum where cos(2π f V ) = cos(π) = 1, or f = 1/(2V ) = d_{scan}/(2R). This is equal to the Nyquist frequency, f_{N} = d_{scan } ⁄ 2, for R = 1 and lower for R > 1. Actual image sharpening is a two dimensional operation. 
Sharpening examples
Here is a plot showing the sharpening transfer function (MTF) for strength (ksharp) = 0.15 and R (sharpening radius) = 1, 2, and 3. Note the the bottom of the plot is MTF = 1 (not 0). At the commonlyused sharpening radius of 2, MTF peaks at half the Nyquist frequency (0.25 cycles/pixel) and drops back to 1 at the Nyquist frequency (0.5 Cycles/Pixel). Sharpening MTF for radii = 1, 2, and 3 pixels Note that the sharpening MTF is cyclic, i.e., it oscillates. For example, MTF for sharpening radius = 2 increases to a second peak at 0.75 C/P, etc. This can affect MTF measurements for sharpened images. 

The MTF plot on the right is for an image sharpened with radius ≅ 2. This camera is quite sharp prior to sharpening— it has quite a bit of energy above the Nyquist frequency. Oscillation above Nyquist causes a ramp in the response around MTF ≅ 0.3 (30%). This causes MTF30 (the spatial frequency where MTF is 30% of the low frequency value) to become extremely unstable: a small change in sharpness can cause a large change in MTF30. In general MTF50 is more stable than MTFnn at lower levels— MTF20, MTF10, etc. MTF plot for a camera sharpened with radius ≅ 2, 
A camera’s sharpening can be analyzed if you have access to raw (unprocessed) and JPEG (standard processed camera output) files using the ImatestMTF Compare module.
Here is an example from the Canon EOS6D fullframe DSLR. A JPEG image (blue curve) with sharpening slightly reduced is compared with a TIFF image converted from raw by dcraw (with no sharpening and noise reductions). The exact same exposure and regions were used for each curve (both JPG and raw files were saved).The peak at 0.25 cycles/pixel indicates that sharpening with radius = 2 was used. The plot deviates from the ideal plot (for a very simple sharpening algorithm) for spatial frequencies above 0.35 C/P. The EOS6D allows you to change the amount of sharpening but not the radius for camera JPEGs. (I’m not happy with this limitation.) Sharpening MTF, comparing 

Corresponding edges for the Canon EOS6D: unsharpened (left), sharpened (right) These curves (and the curves below for the Panasonic Lumix LX7) show how MTF curves (red for raw and blue for JPEG (sharpened) in the MTF Compare plot, above) correlate to edge response. The modest amount of overshoot (“halo”) on this edge would not be objectionable at any viewing magnification. Better overall performance would be achieved with a sharpening radius of R = 1 and an appropriate sharpening amount. 
Here is another example from the Panasonic Lumix LX7. As with the Canon, the same exposure and regions are used: one from a JPEG image (default sharpening, which is very strong), and one from a raw image, converted with no sharpening or noise reduction.The sharpening radius of 1 makes for a sharper image at the pixel level than JPEGs straight out of the EOS6D, above. Of course the EOS6D has twice as many pixels (20 vs. 10), but the difference in sharpening accounts for the relatively close sharpness noted in the post, Sharpness and Texture from ImagingResource.com. Sharpening MTF, comparing the 

Corresponding edges for the Panasonic Lumix Lx7: unsharpened (left), sharpened (right) These curves show how MTF curves (red for raw and blue for JPEG (sharpened) in the MTF Compare plot, above) correlate to edge response. The LX7 is much more strongly sharpened in both frequency and spatial domains than the EOS6D (above). It also has a smaller sharpening radius (R = 1). The strong sharpening would be visible and objectionable at large magnifications, though is results in some impressive measurements: sharpened MTF50P is 2600 LW/PH vs. 2239 for the EOS6D, which has twice as many pixels. (The EOS6D would do better,both visually and numerically (i.e., better measurements) with a sharpening radius of R = 1 instead of 2.) 
Oversharpening and Undersharpening
Oversharpening or undersharpening is the degree to which the image is sharpened relative to the standard sharpening value. If it is strongly oversharpened (oversharpening >about 30%) the image might look better (especially if enlarged) with less sharpening. If it is undersharpened (oversharpening < 0; undersharpening displayed) the image will look better with more sharpening. Basic definitions:
Oversharpening = 100% (MTF( f_{eql} ) – 1)
where f_{eql} = 0.15 cycles/pixel = 0.3 * Nyquist frequency for reasonably sharp edges (MTF50 ≥ 0.2 C/P).
f_{eql} = 0.6*MTF50 for MTF50 < 0.2 C/P (relatively blurred edges)
When oversharpening < 1 (when MTF is lower at f_{eql} than at f = 0), the image is undersharpened, and
Undersharpening = –Oversharpening is displayed.
If the image is undersharpened (the case for the EOS1Ds shown below), sharpening is applied to the original response to obtain Standardized sharpening; if it is positive (if MTF is higher at f_{eql} than at f = 0), desharpening is applied. (We use “desharpening” instead of “blurring” because the inverse of sharpening is, which applied here, is slightly different from conventional blurring.) Note that these numbers are not related to the actual sharpening applied by the camera and software.
Examples: under and oversharpened images
Undersharpened image

Oversharpened image 
The 11 megapixel Canon EOS1Ds DSLR is unusual in that it has very little builtin sharpening (at least in this particular sample). The average edge (with no overshoot) is shown on top; the MTF response is shown on bottom. The black curves are the original, uncorrected data; the dashed red curves have standardized sharpening applied. Standardized sharpening results in a small overshoot in the spatial domain edge response, about what would be expected in a properly (rather conservatively) sharpened image. It is relatively consistent for all cameras. 
The image above is for the 5 megapixel Canon G5, which strongly oversharpens the image— typical for a compact digital camera.A key measurement of rendered detail is the inverse of the 1090% edge rise distance, which has units of (rises) per PH (Picture Height). The uncorrected value for the G5 is considerably better than the 11 megapixel EOS1Ds (1929 vs. 1505 rises per PH), but the corrected value (with standardized sharpening) is 0.73x that of the EOS1Ds. Based on vertical pixels alone, the expected percentage ratio would be 100% (1944/2704) = 0.72x. MTF50P is not shown. It is displayed when Standardized sharpening is turned off; it can also be selected as a Secondary readout. For this camera MTF50P is 0.346 cycles/pixel or 1344 LW/PH, 9% lower than MTF50. It is a better sharpness indicator for strongly oversharpened cameras, especially for cameras where the image will not be altered in postprocessing. 
These results illustrate how uncorrected rise distance and MTF50 can be misleading when comparing cameras with different pixel sizes and degrees of sharpening. MTF50P is slightly better for comparing cameras when strong sharpening is involved.
Uncorrected MTF50 or MTF50P are, however, appropriate for designing imaging systems or comparing lens performance (different focal lengths, apertures, etc.) on a single camera.
Unsharp Masking (USM)
“Unsharp masking” (USM) and “sharpening” are often used interchangeably, even though their mathematical algorithms are different. The confusion is understandable rarely serious because the end results are visually similar. But when sharpening is analyzed in the frequency domain the differences become significant.
“Unsharp masking” derives from the old days of film when a mask for a slide, i.e., positive transparency, was created by exposing the image on negative film slightly out of focus. The next generation of slide or print was made from a sandwich of the original transparency and the fuzzy mask. This mask served two purposes.
 It reduced the overall image contrast of the image, which was often necessary to get a good print.
 It sharpened the image by increasing contrast near edges relative to contrast at a distance from edges.
Unsharp masking was an exacting and tedious procedure which required precise processing and registration. But now USM can be accomplished easily in most image editors, where it’s used for sharpening. You can observe the effects of USM (using the Matlab imsharpen routine) in the Imatest Image Processing module, where you can adjust the blur radius, amount, and threshold settings. Most cameras perform regular sharpening rathan than USM because it’s faster— USM requires a lot more processing power.
Thanks to the central limit theorem, blur can be approximated by the Gaussian function (Bell curve).
Blur = exp(x^{2}/2σ_{x}^{2} ) / √(2πσ_{x}^{2 })
σ_{x} corresponds to the sharpening radius, R. The unsharp masked image can be expressed as the original image summed with a constant times the convolution of the original image and the blur function, where convolution is denoted by *.
L_{USM}(x) = L(x) – k_{USM }* Blur
= L(x)_{ }* [δ(x) – k_{USM }exp(x^{2}/2σ_{x}^{2} ) / √(2πσ_{x}^{2 })] / (1 k_{USM }/√(2π) )
L(x) is the input pixel level and L_{USM}(x) is the USMsharpened pixel level. k_{USM} is the USM sharpening constant (related to the slider setting scanning or editing program). L(x)_{ }= L(x)_{ }* δ(x), where δ(x) is a delta function.
The USM algorithm has its own MTF (the Fourier transform of the portion inside the brackets […]). Using F(exp(px^{2}) ) = exp(α^{2}/4p) ⁄ √(2p), where F is the Fourier transform,
MTF_{USM}( f ) = [1 k_{USM } exp(f^{ 2}σ_{x}^{2}/2 ) /√(2π) ] / (1 k_{USM } /√(2π) )
= [1 k_{USM } exp(f^{ 2}/ 2 f_{USM}^{2} ) /√(2π) ] / (1 k_{USM }/√(2π) )
where f_{USM} = 1/σ_{x}. This equation boosts response at high spatial frequencies, but unlike sharpening, response doesn’t reach a peak then drop. Actual sharpening is a two dimensional operation.
Links
How to Read MTF Curves by H. H. Nasse of Carl Zeiss. Excellent, thorough introduction. 33 pages long; requires patience. Has a lot of detail on the MTF curves similar to the Lensstyle MTF curve in SFRplus. Even more detail in Part II. Their (optical) MTF Tester K8 is of some interest.
Understanding MTF from Luminous Landscape.com has a much shorter introduction.
Understanding image sharpness and MTF A multipart series by the author of Imatest, mostly written prior to Imatest’s founding. Moderately technical.
Bob Atkins has an excellent introduction to MTF and SQF. SQF (subjective quality factor) is a measure of perceived print sharpness that incorporates the contrast sensitivity function (CSF) of the human eye. It will be added to Imatest Master in late October 2006.
Optikos makes instruments for measuring lens MTF. Their 64 page PDF document, How to Measure MTF and other Properties of Lenses, is of particular interest.
Standardized sharpeningStandardized sharpening is was developed for comparing “black box” cameras, The amount of sharpening performed by digital cameras varies greatly. Some cameras and most RAW converters allow you to change the amount of sharpening from the default value. If an image is undersharpened, the image will benefit from additional sharpening during the image editing process. Many compact digital cameras strongly oversharpen images, resulting in severe peaks or “halos” near boundaries. (Moderate oversharpening is usually beneficial.) Extreme sharpening may make images look good on small camera phone displays, but it doesn’t enhance image quality in large displays or prints. Sharpening increases the 50% MTF frequency (MTF50). A camera with extreme oversharpening may have an impressive MTF50 but poor image quality. A camera with little sharpening will have an MTF50 below its potential. For these reasons, comparisons between cameras based on simple MTF50 measurements have little meaning. Raw MTF50 is a poor measure of a camera’s intrinsic sharpness, even though it correlates fairly well with perceived image sharpness. MTF50P, the frequency where MTF drops to half its peak value, is equal to MTF for weak to moderate sharpening but lower for strong sharpening. Hence it is a somewhat more stable indicator of perceived sharpness and image quality. Imatest displays MTF50P when standardized sharpening is not displayed (is unchecked in the input dialog box). It is also available as a secondary readout. To obtain a good measure of a camera’s sharpness— to compare different cameras on a fair basis, the differences in sharpening must be removed from the analysis. We do this in Imatest by setting the sharpening of all cameras to a standard amount. This means sharpening undersharpened images and desharpening (blurring) oversharpened images. We call this procedure Standardized Sharpening (recommended only for comparing “black box” cameras; not for developing cameras or testing lenses). The algorithm for standardized sharpening takes advantage of the observation that most compact digital cameras sharpen with a radius R of about 2 pixels (though R = 1 is not uncommon). This has been the case for several cameras I’ve analyzed using data from dpreview.com and imagingresource.com. The algorithm for standardized sharpening is as follows.
If the edge is seriously blurred (MTF50 < 0.2 f_{N }), so that there is very little energy at f_{eql }= 0.3 f_{N} , the sharpening radius is increased and the equalization frequency is decreased to f_{eql }= 0.6 * MTF50. The sharpening radius is not increased if Fixed sharpening radius in the Settings menu of the Imatest main window has been checked.
The image is sharpened if k_{sharp } > 0 and desharpened if k_{sharp } < 0 (a bit different from standard blurring). For R = 2, the maximum change takes place at half the Nyquist frequency, f = f_{N} /2 = dscan/4, where cos(2πR f /dscan) = cos(π) = 1. Sharpening with R = 2 has no effect on the response at the Nyquist frequency ( f_{N }= dscan/2) because cos(2πR f_{N} /dscan) = cos(2π) = 0.
