Image processing of computers, Study Guides, Projects, Research of Computer Science

Download Image processing of computers and more Study Guides, Projects, Research Computer Science in PDF only on Docsity!

Fundamentals of Digital Image Processing

Roger L. Easton, Jr.

22 November 2010

Contents

• 1 Basic Principles of Digital Image Processing Preface ix
  • 1.1 Digital Processing
  • 1.2 Digitization
• 2 Review of Sampling - 2.0.1 Ideal Sampling of 1-D function
  • 2.1 Aliasing — Whittaker-Shannon Sampling Theorem
  • 2.2 Realistic Sampling — Averaging by the Detector
• 3 Review of Quantization
  • 3.1 Quantization: A/D Conversion
    • 3.1.1 Tone Transfer Function
  • 3.2 Quantization Error (“Noise”)
    • 3.2.1 Signal-to-Noise Ratio
    • 3.2.2 Example: Variance of a Sinusoid
    • 3.2.3 Example: Variance of a Square Wave:
    • 3.2.4 Variance of “Noise” from a Gaussian Distribution
    • 3.2.5 Approximations to SNR
    • 3.2.6 SNR of Quantization
  • 3.3 Quantization to Few Bits
    • 3.3.1 Improved Gray Scale (IGS) Quantization
    • 3.3.2 Quantizers with Memory — Error Diffusion
    • 3.3.3 Halftoning
• 4 Image Processing Operations
  • 4.1 Geometrical Operations
    • 4.1.1 Common Geometrical Operations
    • 4.1.2 Power Series for Coordinates
    • 4.1.3 Affine Transformation
    • 4.1.4 Bilinear Transformation — Pseudoinverse Calculation
  • 4.2 Least-Squares Solution for Affine Transformation
  • 4.3 Pixel Transfers
  • 4.4 Pixel Interpolation
• 5 Point Operations vi CONTENTS
  • 5.1 Image Histograms
    • 5.1.1 Histograms of Typical Images
    • 5.1.2 Other Examples of Histograms
    • 5.1.3 Histogram Modification for Image Enhancement
    • 5.1.4 Jones Plots
  • 5.2 Cumulative Histogram
    • 5.2.1 Nonlinear Nature of Histogram Equalization
    • 5.2.2 Histogram Specification or “Matching”
  • 5.3 Examples of Point Operations
  • 5.4 Application of Histograms to Tone-Transfer Correction
  • 5.5 Application of Histograms to Image Segmentation
• 6 Point Operations on Multiple Images
  • 6.1 Color and “Spectral” Images
  • 6.2 Multispectral Histograms for Segmentation
    • 6.2.1 Multidimensional Histograms of Color Images
    • 6.2.2 Spectral Images
    • 6.2.3 Principal Component Analysis — PCA
  • 6.3 Color Spaces
    • 6.3.1 Red, Green, Blue
    • 6.3.2 Hue, Saturation, Lightness (or Brightness, or Value):
    • 6.3.3 Conversion from RGB to HSL
    • 6.3.4 Example: Wax “Coat of Arms” in a French Epic Poem
  • 6.4 Time-Sequence Images: Video
  • 6.5 Color-Space Transformations for Video Compression
  • 6.6 Segmentation by Logical Operations on Multiple Images
  • 6.7 Arithmetic Operations on Multiple Images
    • 6.7.1 Multiple-Frame Averaging
      • ferences 6.7.2 Required Number of Bits for image Sums, Averages, and Dif-
    • 6.7.3 Image Subtraction
    • 6.7.4 Difference Images as Features
    • 6.7.5 “Mask” or “Template” Multiplication: Image Mattes
    • 6.7.6 Image Division
• 7 Local Operations
  • 7.1 Window Operators — Correlation
  • 7.2 Convolution
    • 7.2.1 Convolutions — Edges of the Image
    • 7.2.2 Convolutions — Computational Intensity
    • 7.2.3 Smoothing Kernels — Lowpass Filtering
    • 7.2.4 Differencing Kernels — Highpass Filters
  • 7.3 Effiects of Averaging and Differencing on Noisy Images
    • 7.3.1 Application of the Laplacian to Texture Segmentation
  • 7.4 CONTENTS vii
  • 7.5 Applications of Differencing — Image Sharpening
    • 7.5.1 Unsharp Masking
    • 7.5.2 Other Image Sharpeners
    • 7.5.3 Generalized Laplacian
  • 7.6 Directional Derivatives: Gradient
    • 7.6.1 Roberts’ Gradient
    • 7.6.2 “Laplacian of Gaussian”
  • 7.7 Nonlinear Filters
    • 7.7.1 Median Filter
    • 7.7.2 Example of Median Filter of Uniform Distribution
  • 7.8 Median Filter and Gaussian Noise
  • 7.9 Comparison of Histograms after Mean and Median Filter
    • 7.9.1 Effect of Window “Shape” on Median Filter
    • 7.9.2 Other Statistical Filters (Mode, Variance, Maximum, Minimum)
    • 7.9.3 Examples of Nonlinear Filters
    • 7.9.4 Nonlinear Filters on Images with Additive Gaussian Noise
    • 7.9.5 Nonlinear Filters on Noise-Free Gray-Level Image
  • 7.10 Adaptive Operators
  • 7.11 Convolution Revisited — Bandpass Filters
    • 7.11.1 Bandpass Filters for Images
  • 7.12 Pattern Matching
    • 7.12.1 Other Matching Kernels
    • 7.12.2 Normalization of Contrast of Detected Features
  • 7.13 Implementation of Filtering
    • 7.13.1 Nonlinear and Shift-Variant Filtering
  • 7.14 Neighborhood Operations on Multiple Images
    • 7.14.1 Image Sequence Processing
    • 7.14.2 Spectral + Spatial Neighborhood Operations
    • 7.14.3 “Pseudounsharp Masking”
• 8 Global Operations
  • 8.1 Relationship to Neighborhood Operations
  • 8.2 Discrete Fourier Transform (DFT)
  • 8.3 Fast Fourier Transform (FFT)
  • 8.4 Fourier Transforms of Images
  • 8.5 Image Restoration via Fourier Transforms
    • 8.5.1 Examples of Inverse Filters in 1-D
    • 8.5.2 Spectrum and Impulse Response of Inverse Filter
    • 8.5.3 Inverse Filter for SINC-Function Blur
  • 8.6 Other Global Operations
  • 8.7 Discrete Cosine Transform (DCT)
    • 8.7.1 Steps in Forward DCT
    • 8.7.2 Steps in Inverse DCT

viii CONTENTS

8.8 Walsh-Hadamard Transform....................... 218

x Preface

A. Nussbaum and R. Phillips, Contemporary Optics for Scientists and En- gineers, Prentice-Hall, 1976. Wayne Niblack, An Introduction to Digital Image Processing, Prentice Hall, Englewood Cliffs, 1986. J. Anthony Parker, Image Reconstruction in Radiology, CRC Press, Boca Raton FL, 1990. William K. Pratt, Digital Image Processing, Second Edition, John Wiley & Sons, New York, 1991. Azriel Rosenfeld and Avinash C. Kak, Digital Picture Processing, Second Edition, Academic Press, San Diego, 1982. Craig Scott, Introduction to Optics and Optical Imaging, IEEE Press, New York, 1998. J.S.Walker, Fast Fourier Transforms 2nd Edition, CRC Press, New York, 1996.

Chapter 1

Basic Principles of Digital Image

Processing

During the last two decades or so, inexpensive and powerful digital computers have become widely available and have been applied to a multitude of tasks. By hitching computers to new imaging detectors and displays, very capable systems for creating, analyzing, and manipulating imagery have been constructed and are being applied in many arenas. For example, they now are used to reconstruct X-ray and magnetic resonance images (MRI) in medicine, to analyze multispectral aerial and satellite images for environmental and military uses, to read Universal Product Codes that specify products and prices in retail stores, to name just a few. Since I first taught a predecessor of this course in 1987, the capabilities of inex- pensive imaging tools (cameras, printers, computers) have exploded (no surprise to you, I’m sure). This has produced a wide and ever-expanding range of applications that we could not even envision in those days. To give an idea of the change over the last two decades, consider the set of data shown below, which is copied directly from the first edition of Digital Image Processing by Gonzalez and Woods: These data represent a 64 × 64 5-bit image ( 25 = 32 gray values). This data set was entered by hand (with only 4 mistakes) in 1988 by Ranjit Bhaskar, an imaging science graduate student for use by students. The image was rendered using the so- called “overstrike” routine on a line printer, where “dark” pixels were printed using several overprinted characters and lighter pixels by sparse characters (e.g. “.” and “-”). The subject of the image is shown on the next page: This course will investigate the basic principles of digital imaging systems and introduce useful applications; many simple examples will be provided to illustrate the concepts. First, a definition:

IMAGE: A reproduction or imitation of form of a person or thing. The optical counterpart of an object produced by a lens, mirror, etc. ..................................Noah Webster

We normally think of an image in the sense of a picture, i.e., a planar represen- tation of the brightness, , i.e., the amount of light reflected or transmitted by an object.

1

An image is usually a function of two spatial variables, e.g., f [x, y], which rep- resents the brightness f at the Cartesian location [x, y]. Obviously, it also may be graphed in three dimensions, with brightness displayed on the z-axis.

Function of Two Spatial Coordinates f [x, y]

Image Representation of f [n, m]

It is more and more common to deal with images that have more than two coor- dinate dimensions, e.g.,

f [x, y, tn] monochrome “movie”, discrete set of images over time f [x, y, λ] spectral image with continuous domain of wavelengths f [x, y, λn] multispectral image, discrete set of wavelengths f [x, y, t] time-varying monochrome image over continuous time domain f [x, y, tn] time-varying monochrome image with discrete time samples (cinema) f [x, y, z] 3-D monochrome image (e.g., optical hologram) f [x, y, tn, λm] discrete samples in time and wavelength, e.g., color movie f [x, y, z, t, λ] reality It is generally fesible to “cut” 2-D slices from these multidimensional functions to cre- ate images, but the images need not be “pictorial.” For example, consider the 2-D slices “cut” from the 3-D function spatial-temporal function f[x, y, t]; the 2-D slice f [x, y; t = t 0 ] is pictorial but f [x, y = y 0 , t] is not. That said, the units of the axes have no effect on the computations; it is perfectly feasible for computers to process and display f [x, y = y 0 , t] as to do the same for f [x, y; t 0 ].

After converting image information into an array of integers, the image can be manipulated, processed, and displayed by computer. Computer processing is used for image enhancement, restoration, segmentation, description, recognition, coding, reconstruction, transformation

4 CHAPTER 1 BASIC PRINCIPLES OF DIGITAL IMAGE PROCESSING

1.1 Digital Processing

The general digital image processing system may be divided into three components: the input device (or digitizer), the digital processor, and the output device (image display).

1. The digitizer converts a continuous-tone and spatially continuous brightness distribution f [x, y] to an discrete array (the digital image) fq[n, m], where n, m, and fq are integers.

2. The digital processor operates on the digital image fq[n, m] to generate a new digital image gq[k, c], where k, c, and gq are integers. The output image may be represented in a different coordinate system, hence the use of different indices k and c.

3. The image display converts the digital output image gq[k, c] back into a continuous- tone and spatially continuous image g [x, y] for viewing. It should be noted that some systems may not require a display (e.g., in machine vision and artificial intelligence applications); the output may be a piece of information. For ex- ample, a digital imaging system that was designed to answer the question, Is there evidence of a cancerous tumor in this x-ray image?, ideally would have two possible outputs (YES or NO), , i.e., a single bit of information.

Note that the system includes most of the links in what we call the imaging chain. We shall first consider the mathematical description of image digitizing and display devices, and follow that by a long discussion of useful processing operations.

1.2 Digitization

Digitization is the conversion of a continuous-tone and spatially continuous brightness distribution f [x, y] to an discrete array of integers fq[n, m] by two operations which will be discussed in turn:

Chapter 2

Review of Sampling

The process of “sampling” derives a discrete set of data points at (usually) uniform spacing. In its simplest form, sampling is expressed mathematically as multiplication of the original image by a function that “measures” the image brightness at discrete locations of infinitesimal width/area/volume in the 1-D/2-D/3-D cases:

fs [n · ∆x] = f [x] · s [x; n · ∆x] where: f [x] = brightness distribution of input image s [x; n · ∆x] = sampling function fs [n · ∆x] = sampled input image defined at coordinates n · ∆x

The ideal sampling function for functions of continuous variables is generated from the so-called “Dirac delta function” δ [x], which is defined by many authors, including Gaskill. The ideal sampling function is the sum of uniformly spaced “discrete” Dirac delta functions, which Gaskill calls the COMB while Bracewell calls it the SHAH :

COMB [x] ≡

X^ +∞

n=−∞

δ [x − n]

s [x; n · ∆x] ≡

X^ +∞

n=−∞

δ [x − n · ∆x] ≡

∆x

COMB

h (^) x ∆x

i

the fidelity of the sampled image. For illustration, examples of sampled functions obtained for several values of ∆Xx 0 are:

Case I: X 0 = 12 · ∆x =⇒

∆x X 0

, φ 0 = 0 =⇒ fs [n] =

1 + cos

hπn 6

i´

Case II: X 0 = 2 · ∆x =⇒

∆x X 0

, φ 0 = 0 =⇒ fs [n] =

· (1 + cos [πn]) =

[1 + (−1)n]

Case III: X 0 = 2 · ∆x =⇒

∆x X 0

, φ 0 = −

π 2

=⇒ fs [n] =

· (1 + sin [πn]) =

Case IV: X 0 =

· ∆x =⇒

∆x X 0

, φ 0 = 0

=⇒ fs [n] =

μ 1 + cos

2 πn 4 / 3

1 + cos

h 3 π

n 2

i´

Case V: X 0 =

· ∆x =⇒

∆x X 0

, φ 0 = 0

=⇒ fs [n] =

μ 1 + cos

2 πn 5 / 4

1 + cos

h 8 π

n 5

i´

10 CHAPTER 2 REVIEW OF SAMPLING

Illustration of samples of the biased sinusoids with the different values of ∆Xx 0 listed in the table. The last three cases illustrate “aliasing.”

The output evaluated for ∆Xx 0 = 12 depends on the phase of the sinusoid; if sampled at the extrema, then the sampled signal has the same dynamic range as f [x] (i.e., it is fully modulated), show no modulation, or any intermediate value. The interval ∆x = X 20 defines the Nyquist sampling limit. If ∆Xx 0 > 12 sample per period, then the same set of samples could have been obt5ained from a sinusoid with a longer period and a different sampling interval ∆x. For example, if ∆Xx 0 = 34 , then the reconstructed function appears as though obtained from a sinudoid with period X 00 = 3X 0 if sampled with ∆Xx 00 = 14. In other words, the data set of samples is ambiguous; the same samples

12 CHAPTER 2 REVIEW OF SAMPLING

components expressed as a function of frequency are the Fourier components of the function. If the sinusoidal representation of f [x] has a component with a maximum spatial frequency ξmax, and if we sample f [x] so that this component is sampled without alias- ing, then all sinusoidal components of f [x] will be adequately sampled and f [x]can be perfectly reconstructed from its samples. Such a function is band-limited and ξmax is the cutoff frequency of f [x]. The corresponding minimum spatial period is Xmin = (^) ξmax^1. Thus the sampling interval ∆x can be found from:

∆x Xmin

=⇒ ∆x <

Xmin 2

=⇒ ∆x <

2 ξmax

This is the Whittaker-Shannon sampling theorem. The limiting value of the sam- pling interval ∆x = (^2) ξmax^1 defines the Nyquist sampling limit. Sampling more or less frequently than the Nyquist limit is oversampling or undersampling, respectively.

∆x >

2 ξmax

=⇒ undersampling

∆x <

2 ξmax

=⇒ oversampling

The Whittaker-Shannon Sampling Theorem is valid for all types of sampled sig- nals. An increasingly familiar example is digital recording of audio signals (e.g., for compact discs or digital audio tape). The sampling interval is determined by the maximum audible frequency of the human ear, which is generally accepted to be approximately 20kHz. The sampling frequency of digital audio recorders is 44, samples second which translates to a sampling interval of^

1 44 ,000 s = 22.^7 μs. At this sampling rate, sounds with periods greater than 2 · 22. 7 μs = 45. 4 μs (or frequencies less than (45. 4 μs)−^1 = 22 kHz) can theoretically be reconstructed perfectly, assuming that f [t] is sampled perfectly (i.e., at a point). Note that if the input signal frequency is greater than the Nyquist frequency of 22 kHz, the signal will be aliased and will ap- pear as a lower-frequency signal in the audible range. Thus the reconstructed signal will be wrong. This is prevented by ensuring that no signals with frequencies above the Nyquist limit is allowed to reach the sampler; higher frequencies are filtered out before sampling.

2.2 Realistic Sampling — Averaging by the Detec-

tor

In fact, it is not possible to “grab” the amplited of a signal at specific locations with infinitesimal “width” (infinitesimal “support”). The measurement of a finite signal over the infinitesimally small area in the real world would produce an infinitesimal result. Rather a real system performs “realistic sampling,” where the continuous input is measured over finite areas located at uniformly spaced samples by using a detector

2.2 REALISTIC SAMPLING — AVERAGING BY THE DETECTOR 13

with finite spatial (or temporal) size. The measured signal is a spatial integral of the input signal over the detector area, which blurs the image. For example, in the common case where we assume that each sensor element has the same response over its full area, we can calculate the sample value by integrating the signal over the detector size. In the figure, the sensor elements are separated by ∆x and each has width d 0 :

For a biased sinusoidal signal of the form:

f [x] =

μ 1 + cos

2 πx X 0

• φ 0

we define its modulation as:

m =

fmax − fmin fmax + fmin

; 0 ≤ m ≤ 1 if fmin ≥ 0

Note that modulation is only defined for nonnegative (i.e., biased) sinusoids. The analogous quantity for a nonnegative square wave is called contrast.

The biased sinusoidal signal is averaged over the detector area, e.g., the sampled value at n = 0 is:

fs [n = 0] =

d 0

Z (^) + d 20

− d 20

f [x] dx

d 0

Z +∞

−∞

f [x] · RECT

x d 0

dx

where: RECT

x d 0

1 if |x| <

d 0 2 1 2 if^ |x|^ =^

d 0 2

=⇒ x = ±

d 0 2 0 if |x| >

d 0 2