Image noise

Noise clearly visible in an image from a digital camera

Image noise is the random variation of brightness or color information in images produced by the sensor and circuitry of a scanner or digital camera. Image noise can also originate in film grain and in the unavoidable shot noise of an ideal photon detector.

Image noise is generally regarded as an undesirable by-product of image capture. Although these unwanted fluctuations became known as "noise" by analogy with unwanted sound,^[1] they are inaudible and actually beneficial in some applications, such as dithering.

Types

Amplifier noise (Gaussian noise)

The standard model of amplifier noise is additive, Gaussian, independent at each pixel and independent of the signal intensity, caused primarily by Johnson–Nyquist noise (thermal noise), including that which comes from the reset noise of capacitors ("kTC noise").^[2] In color cameras where more amplification is used in the blue color channel than in the green or red channel, there can be more noise in the blue channel.^[3]

Amplifier noise is a major part of the "read noise" of an image sensor, that is, of the constant noise level in dark areas of the image.^[4]

Image with salt and pepper noise.

Salt-and-pepper noise

Fat-tail distributed or "impulsive" noise is sometimes called salt-and-pepper noise or spike noise.^[5]

An image containing salt-and-pepper noise will have dark pixels in bright regions and bright pixels in dark regions. This type of noise can be caused by dead pixels, analog-to-digital converter errors, bit errors in transmission, etc.^[6][7]

This can be eliminated in large part by using dark frame subtraction and by interpolating around dark/bright pixels.

Shot noise

The dominant noise in the lighter parts of an image from an image sensor is typically that caused by statistical quantum fluctuations, that is, variation in the number of photons sensed at a given exposure level; this noise is known as photon shot noise.^[3] Shot noise has a root-mean-square value proportional to the square root of the image intensity, and the noises at different pixels are independent of one another. Shot noise follows a Poisson distribution, which is usually not very different from Gaussian.

In addition to photon shot noise, there can be additional shot noise from the dark leakage current in the image sensor; this noise is sometimes known as "dark shot noise"^[3] or "dark-current shot noise".^[8] Dark current is greatest at "hot pixels" within the image sensor; the variable dark charge of normal and hot pixels can be subtracted off (using "dark frame subtraction"), leaving only the shot noise, or random component, of the leakage;^[9][10] if dark-frame subtraction is not done, or if the exposure time is long enough that the hot pixel charge exceeds the linear charge capacity, the noise will be more than just shot noise, and hot pixels appear as salt-and-pepper noise.

Quantization noise (uniform noise)

The noise caused by quantizing the pixels of a sensed image to a number of discrete levels is known as quantization noise; it has an approximately uniform distribution, and can be signal dependent, though it will be signal independent if other noise sources are big enough to cause dithering, or if dithering is explicitly applied.^[7]

Film grain

The grain of photographic film is a signal-dependent noise, related to shot noise.^[11] That is, if film grains are uniformly distributed (equal number per area), and if each grain has an equal and independent probability of developing to a dark silver grain after absorbing photons, then the number of such dark grains in an area will be random with a binomial distribution; in areas where the probability is low, this distribution will be close to the classic Poisson distribution of shot noise; nevertheless a simple Gaussian distribution is often used as an accurate enough model.^[7]

Film grain is usually regarded as a nearly isotropic (non-oriented) noise source, and is made worse by the distribution of silver halide grains in the film also being random.^[12]

Non-isotropic noise

Some noise sources show up with a significant orientation in images. For example, image sensors are sometimes subject to row noise or column noise.^[13] In film, scratches are an example of non-isotropic noise.

Noise problems with digital cameras

This section requires expansion.

Image on the left has exposure time of >10 seconds in low light. The image on the right has adequate lighting and 0.1 second exposure.

In low light, correct exposure requires the use of long shutter speeds, higher gain (or sensitivity), or both. On most cameras, longer shutter speeds lead to increased salt-and-pepper noise due to photodiode leakage currents. At the cost of a doubling of read noise variance (41% increase in read noise standard deviation), this salt-and-pepper noise can be mostly eliminated by dark frame subtraction.

The relative effect of both read noise and shot noise increase as the exposured is reduced, corresponding to increased ISO sensitivity, since fewer photons are counted (shot noise) and since more amplification of the signal is necessary.

The size of the image sensor, or effective light collection area per pixel sensor, is the largest determinant of signal levels that determine signal-to-noise ratio and hence apparent noise levels. The sensitivity of a given imager at the same noise level scales roughly with the sensor area. For instance, the noise level produced by a Four Thirds sensor at ISO 800 is roughly equivalent to that produced by a "full frame" sensor (with roughly four times the area) at ISO 3200, and that produced by a 1/2.5" compact camera sensor (with roughly 1/8 the area) at ISO 100. This ability to produce acceptable images at higher sensitivities is a major factor driving the adoption of DSLR cameras, which tend to use larger sensors than compacts. The number of pixels on a sensor greatly affects the noise level per pixel, since a sensor with more pixels for the same size must use physically smaller pixels. However, when scaled to the same size on screen, or printed at the same size, the pixel count makes little difference on perceptible noise levels.

Image noise reduction

Most algorithms for converting image sensor data to an image, whether in-camera or on a computer, involve some form of noise reduction. There are many procedures for this, but all attempt to determine whether differences in pixel values constitute noise or real photographic detail, and average out the former. However, no algorithm can make this judgment perfectly, so there is often a tradeoff made between noise removal and preservation of fine, low-contrast detail that may have characteristics similar to noise. Many cameras have a setting to control the aggressiveness of the in-camera noise reduction.

This decision can be assisted by knowing the characteristics of the source image and of human vision. Most noise reduction algorithms perform much more aggressive chroma noise reduction, since there is little importanat fine chroma detail that one risks losing. Furthermore, many people find luminance noise less objectionable to the eye, since its textured appearance mimics the appearance of film grain.

The high sensitivity image quality of a given camera (or RAW development workflow) may depend greatly on the quality of the algorithm used for noise reduction. Since noise levels increase as ISO sensitivity is increased, most camera manufacturers increase the noise reduction aggressiveness automatically at higher sensitivities. This leads to a breakdown of image quality at higher sensitivities in two ways: noise levels increase and fine detail is smoothed out by the more aggressive noise reduction.

Video noise

In video and television, noise refers to the random dot pattern that is superimposed on the picture as a result of electronic noise, the 'snow' that is seen with poor (analog) television reception or on VHS tapes. Interference and static are other forms of noise, in the sense that they are unwanted, though not random, which can affect radio and television signals.

Useful noise

High levels of noise are almost always undesirable, but there are cases when a certain amount of noise is useful, for example to prevent discretization artifacts (color banding or posterization). Noise purposely added for such purposes is called dither.

Low- and high-ISO noise examples

Images of a flower taken at ISO 100 and ISO 1600 on a Canon 400D digital camera. Both images were shot under similar lighting conditions, varying only the ISO setting and shutter speed.

ISO 100, f/5.6, 1/350 s (click on image for larger version)

ISO 1600, f/5.6, 1/4000 s (click on image for larger version)

Comparison of both images. This is a crop of a small section of each image displayed at 100%. The top portion was shot at 100 ISO, the bottom portion at 1600 ISO.

See also

• Noise reduction

References

1. ^ Leslie Stroebel and Richard D. Zakia (1995). The Focal encyclopedia of photography. Focal Press. p. 507. ISBN 9780240514178. http://books.google.com/books?id=CU7-2ZLGFpYC&pg=PA507&dq=electronic+sound+image++noise+analogy+unwanted&lr=&as_brr=3&ei=u7JBSo7nGInAlQTR7vGCDw. 
2. ^ Jun Ohta (2008). Smart CMOS Image Sensors and Applications. CRC Press. ISBN 0849336813. http://books.google.com/books?id=_7NLzflrTrcC&pg=PA48&dq=image-sensor+amplifier+thermal+noise. 
3. ^ ^{a b c} Lindsay MacDonald (2006). Digital Heritage. Butterworth-Heinemann. ISBN 0750661836. http://books.google.com/books?id=oqOK8zpM7jIC&pg=PA207&dq=photon-shot-noise+image+bright+areas. 
4. ^ Junichi Nakamura (2005). Image Sensors and Signal Processing for Digital Still Cameras. CRC Press. ISBN 0849335450. http://books.google.com/books?id=aaTyk2USX1MC&pg=PA76&dq=read-noise+image-sensor+amplifier. 
5. ^ Rafael C. Gonzalez, Richard E. Woods (2007). Digital Image Processing. Pearson Prenctice Hall. ISBN 013168728X. http://books.google.com/books?id=8uGOnjRGEzoC&pg=PA316&dq=salt-and-pepper-noise. 
6. ^ Linda G. Shapiro and George C. Stockman (2001). Computer Vision. Prentice-Hall. ISBN 0130307963. http://books.google.com/books?id=jclrAAAACAAJ&dq=intitle:Computer+intitle:Vision+inauthor:Shapiro. 
7. ^ ^{a b c} Charles Boncelet (2005). "Image Noise Models". in Alan C. Bovik. Handbook of Image and Video Processing. Academic Press. ISBN 0121197921. http://books.google.com/books?id=OYFYt5C4N94C&pg=PA405&dq=binomial+film+grain+noise. 
8. ^ James R. Janesick (2001). Scientific Charge-coupled Devices. SPIE Press. ISBN 0819436984. http://books.google.com/books?id=3GyE4SWytn4C&pg=PA630&dq=dark-current-shot-noise. 
9. ^ Michael A. Covington (2007). Digital SLR Astrophotography. Cambridge University Press. ISBN 0521700817. http://books.google.com/books?id=Agho6QnsrzsC&pg=RA1-PA131&dq=%22hot+pixels%22+dark+noise+subtraction&lr=&as_brr=3&ei=HGr7SManHI2YMuWhoKIL. 
10. ^ R. E. Jacobson, S. F. Ray, G. G. Attridge, and N. R. Axford (2000). The Manual of Photography. Focal Press. ISBN 0240515749. http://books.google.com/books?id=MblHnLN2N2kC&pg=PA425&dq=shot-noise+dark++subtraction&lr=&as_brr=3&ei=TWz7SPLbLY6UMajtqKcL. 
11. ^ Thomas S. Huang (1986). Advances in Computer Vision and Image Processing. JAI Press. ISBN 0892324600. http://books.google.com/books?id=OvBRAAAAMAAJ&q=poisson+film+grain+noise&dq=poisson+film+grain+noise&pgis=1. 
12. ^ Brian W. Keelan and Robert E. Cookingham (2002). Handbook of Image Quality. CRC Press. ISBN 0824707702. http://books.google.com/books?id=E45MTZn17gEC&pg=PA186&dq=%22noise+source%22+film+grain+isotropic-noise. 
13. ^ Joseph G. Pellegrino et al. (2006). "Infrared Camera Characterization". in Joseph D. Bronzino. Biomedical Engineering Fundamentals. CRC Press. ISBN 0849321220. http://books.google.com/books?id=Ob5Q-TyMMb0C&pg=PT813&dq=row-noise+column-noise. 

External links

• Noise at dpreview glossary
• Compact Camera High ISO modes: Separating the facts from the hype
• Nikon D700 noise measurements