Download Wavelet Transform-Implementation and Application in Computer Sciences-Project Report and more Study Guides, Projects, Research Applications of Computer Sciences in PDF only on Docsity!
Preface
The purpose of this document is to let the project supervisor and panel know about progress of the project. The document contains work carried out during the summer break and in the 7th^ semester until now. A brief theory on Wavelet Analysis is presented alongside with the application of wavelets in Image Fusion. A number of different Image Fusion schemes based on DWT (Discrete Wavelet Transform) are discussed. SVM (Support Vector Machine) theory and SVM based Image Fusion schemes are also described in the document as studied during literature survey. The document also contains theory on ANN (Artificial Neural Networks) and its application in Image Fusion.
Table of Contents
- THEORY ON WAVELET TRANSFORM
- 1.1 FOURIER ANALYSIS
- 1.2 WAVELET ANALYSIS
- 1.3 THE DISCRETE WAVELET TRANSFORM.................................................................................................
- 1.4 MULTI-LEVEL DECOMPOSITION
- 1.5 WAVELET FAMILY
- 1.5.1 Wavelet Functions
- 1.5.2 Scaling Functions.........................................................................................................................
- WAVELET BASED IMAGE FUSION
- 2.1 GENERAL FRAMEWORK FOR WAVELET BASED IMAGE FUSION
- 2.2 FUSION RULES
- 2.2.1 Selection
- 2.2.2 Simple Averaging Method
- 2.2.3 Select Max Method
- 2.2.4 Weighted Average Method
- 2.2.5 Local Deviation based Method ...................................................................................................
- 2.2.6 Convolution based Method .........................................................................................................
- 2.2.7 Local Gradient based Method .....................................................................................................
- 2.2.8 Local Gradient and Weighted Average Method ..........................................................................
- SUPPORT VECTOR MACHINES........................................................................................................
- 3.1 INTRODUCTION ....................................................................................................................................
- 3.2 ADVANTAGES OVER OTHER CLASSIFICATION METHODS .....................................................................
- 3.3 SVM AS A BINARY CLASSIFIER ...........................................................................................................
- SVM BASED IMAGE FUSION .............................................................................................................
- 4.1 GENERAL FRAMEWORK .......................................................................................................................
- 4.1.1 Description..................................................................................................................................
- 4.2 ALGORITHMS .......................................................................................................................................
- 4.2.1 Fusing Images with Different Focuses using SVM .....................................................................
- 4.2.2 A Novel SVM based Multi-Focus Image Fusion Algorithm ........................................................
- 4.2.3 Fusion of Visible and Thermal Images using SVM .....................................................................
- ARTIFICIAL NEURAL NETWORKS .................................................................................................
- 5.1 INTRODUCTION ....................................................................................................................................
- 5.2 APPLICATIONS .....................................................................................................................................
- 5.3 ARCHITECTURE OF A NEURON .............................................................................................................
- 5.4 ANN ARCHITECTURES ........................................................................................................................
- 5.5 NEURAL NETWORKS FOR PATTERN CLASSIFICATION ..........................................................................
- 5.5.1 Advantages ..................................................................................................................................
- 5.5.2 Disadvantages .............................................................................................................................
- 5.6 HEBB NET ............................................................................................................................................
- 5.6.1 Algorithm ....................................................................................................................................
- 5.6.2 Effectiveness ................................................................................................................................
- 5.7 SINGLE LAYER PERCEPTRON ...............................................................................................................
- 5.7.1 Training Algorithm (Multiple Inputs Single Output) ..................................................................
- 5.7.2 Training Algorithm (Multiple Inputs Multiple Outputs) .............................................................
- 5.7.3 Testing Algorithm........................................................................................................................
- 5.7.4 Effectiveness ................................................................................................................................
- 5.8 MULTI LAYER PERCEPTRON ................................................................................................................
- 5.8.1 Feed Forward Backpropagation Neural Network ......................................................................
- ANN BASED IMAGE FUSION .............................................................................................................
- 6.1 GENERAL FRAMEWORK .......................................................................................................................
- 6.1.1 Description..................................................................................................................................
- 6.2 ALGORITHMS .......................................................................................................................................
- 6.2.1 Multi-Focus Image Fusion using Artificial Neural Networks .....................................................
- REFERENCES ........................................................................................................................................
- FIGURER 1.1: FOURIER TRANSFORM OF A SIGNAL [19] List of Figures
- FIGURE 1.2: SHORT-TIME FOURIER TRANSFORM OF A SIGNAL [19]
- FIGURE 1.3: WAVELET TRANSFORM OF A SIGNAL [19]
- FIGURE 1.4: COMPARISON OF SIGNAL ANALYSIS TECHNIQUES [19]
- FIGURE 1.5: A SAMPLE WAVELET [3]
- FIGURE 1.6: FILTERING PROCESS TO OBTAIN APPROXIMATION AND DETAIL COEFFICIENTS [19]
- FIGURE 1.7: THE PROCESS OF DOWN-SAMPLING
- FIGURE 1.8: MULTI-LEVEL DECOMPOSITION OF A SIGNAL USING DWT [19]
- FIGURE 1.9: MULTI-LEVEL DECOMPOSITION OF AN IMAGE USING DWT [19]
- FIGURE 2.1: GENERAL FRAMEWORK FOR WAVELET BASED IMAGE FUSION [7]
- FIGURE 3.1: OPTIMAL AND NON-OPTIMAL HYPER-PLANES [2] ...................................................................
- FIGURE 4.1: GENERAL FRAMEWORK FOR SVM BASED IMAGE FUSION .....................................................
- FIGURE 5.1: MATHEMATICAL MODEL OF A NEURON [5] ............................................................................
- FIGURE 5.2: SIMPLE PERCEPTRON FOR PATTERN CLASSIFICATION ..........................................................
- FIGURE 6.1: GENERAL FRAMEWORK FOR ANN BASED IMAGE FUSION .....................................................
Section 1
1. Theory on Wavelet Transform
The discussion is started from Fourier analysis to get some historical background of Wavelet transformation
1.1 Fourier Analysis
A number of signal analysis techniques have been developed. One of them is Fourier analysis. Fourier analysis is a mathematical technique in which a signal is transformed from time domain to frequency domain.
Figurer 1.1: Fourier Transform of a Signal [19] Fourier analysis is very useful technique in cases when we need only the frequency contents of a signal. It has a serious drawback in situations where we need time information as well i.e. where a particular frequency exists. The time information is lost during the transformation from time to frequency domain. This drawback is not so worrying for stationary signals in which signal properties do not change much over time and all frequencies are present at all times. However, most interesting signals like images contain numerous non stationary or transitory characteristics such as drift, trends, abrupt changes and beginnings and ends of events. These characteristics are often the most important part of the signal, and Fourier analysis is not suited in detecting them. Dennis Gabor (1946) introduced the Short-Time Fourier Transform to solve the above mentioned problem. He adapted the Fourier transform to analyze only a small section of the signal at a time a technique called windowing the signal. Gabor's method
3
Figure 1.4: Comparison of signal analysis techniques [19]
A wavelet is an irregular limited duration small wave unlike the infinite sine waves used in other transforms as shown in figure 1.5.
Figure 1.5: A sample wavelet [3]
1.3 The Discrete Wavelet Transform
It is difficult to calculate the wavelet coefficients at all possible (continuous) scales of the original signal. Therefore only a subset of scales and positions is chosen at which we calculate the wavelet coefficients. In wavelet analysis, we often speak of approximations and details. The approximations are the high-scale, low-frequency components of the signal. The details are the low-scale, high-frequency components. Filtering process is used to obtain the approximation and detail coefficients of a signal. Figure 1.6 illustrates the filtering process.
4
Figure 1.6: Filtering process to obtain approximation and detail coefficients [19] In the above diagram, S is the original signal; a low-pass filter is used to calculate approximation coefficients (A) and a high-pass filter for detail coefficients (D). If we perform this operation on a real discrete signal consisting of say 1000 data points, then we will end up having 2000 data points (1000 from each filter). To avoid this problem, a process called down-sampling is used in which only one out of two in each of the two 1000 samples is kept to get the complete information. This process is illustrate in figure 1.7.
Figure 1.7: The process of down-sampling
1.4 Multi-Level Decomposition
In the above section we saw that a signal can be decomposed into approximation and detail coefficients. This decomposition process can be iterative to produce a wavelet decomposition tree as shown in figure 1.8.
6
After one level of decomposition, there will be four frequency bands, namely Low-Low (LL), Low-High (LH), High-Low (HL) and High-High (HH). The next level decomposition is just applied to the LL band of the current decomposition stage, which forms a recursive decomposition procedure. Thus, N-level decomposition will finally have 3N+1 different frequency bands, which include 3N high frequency bands and just one LL frequency band. The 2-D DWT will have a pyramid structure shown in the above figure. The frequency bands in higher decomposition levels will have smaller size [4].
1.5 Wavelet Family
Wavelets are usually described in terms of two groups of functions: wavelet functions and scaling functions. It is also common to refer to them as families: the wavelet function is the “mother” wavelet, the scaling function is the “father” wavelet, and transformations of the parent wavelets are “daughter” and “son” wavelets [13].
1.5.1 Wavelet Functions
In general, a wavelet family is described in terms of its mother wavelet, denoted as ψ(x). The wavelet function acts as a high-pass filter [13].
1.5.2 Scaling Functions
In DWT, a scaling function or father wavelet denoted as Φ(x) is needed to cover the low frequencies of the signal. It acts as a low-pass filter [13].
7
Section 2
2. Wavelet based Image Fusion
One of the applications of Wavelet Transform is the field of image fusion. DWT is used by many researchers to fuse two or more images. Before starting the discussion on wavelet based image fusion schemes, a general framework for the wavelet based image fusion is provided in the following section.
2.1 General Framework for Wavelet based Image Fusion
Different wavelet based fusion schemes have been studied during the literature survey. The basic idea behind all of those schemes is same. Figure 2.1 describes the general framework for the wavelet based image fusion.
9
good integration rule is the Select-Max (CM) scheme, which means just pick the coefficient with the larger activity level and discard the other. There are different methods to choose the low frequency coefficients.
2.2.2 Simple Averaging Method
Suppose we have two source images A and B. Let C(F,p), C(A,p) and C(B,p) be the corresponding sub-band signals (in wavelet domain) of fused image F, and source images A and B respectively, p is a coefficient location. The sub-band signal C(F,p) of the fused image is simply obtained by taking the average of C(A,p) and C(B,p) [15]. i.e. C ( F , p ) 0. 5 C ( A , p ) 0. 5 C ( B , p ) (2.1)
2.2.3 Select Max Method
In this method, the coefficient with greater absolute value is selected and the other one is discarded [17]. i.e.
( , ), ( , ) ( , ) ( , ) ( , ), ( , ) ( , ) C B p C B p C A p C F p C A p C Ap CB p (2.2)
2.2.4 Weighted Average Method
This method was proposed by Burt [20]. In this method, first a local energy is defined as
qQ
E ( A , p ) w ( q ) C^2 ( A , q ) (2.3)
Where w(q) is a weight and
q Q
w ( q ) 1. Q is a neighborhood typically of size 3x3 or
5x5 window centered at the current coefficient position. The closer the point q is near the point p, the greater w(q) is. E(B,p) can also be calculated using the equation 2.3. Then a match measure M(p) is defined as a normalized correlation averaged over a neighborhood of p as:
( , ) ( , )
( ) ( , ) ( ,) ( ) E A p EB p
wqC AqC B M p qQ
(2.4)
10
The match measure M(p) determines which of the two approaches: averaging or selection to be used at each coefficient location. M(p) is value ranging between 0 and 1. If M(p) is smaller than a threshold value T then:
( , ), ( , ) ( , ) ( , ) ( , ), ( , ) ( , ) C Bp EB p E Ap C F p C Ap EAp EB p (2.5)
Else if M ( p ) T , then
( , ) ( , ), ( , ) ( , ) ( , ) ( , ) ( , ), ( , ) ( , ) max min
max min C Bp C Ap EB p E A p C F p C Ap C Bp EAp EBp
W W
W W (2.6)
Where
T
M p
W 1
- 5 0. 5 1 ( )
min^ and^ W^ max ^1 W min (2.7)
During the implementation of this scheme, T was taken as 0.95 and window size as 3x3.
2.2.5 Local Deviation based Method
This method was developed by Qiang Zan-xia [21]. He used this method to select the high frequency coefficients for the fused image; however [7] suggested this approach for the selection of low frequency coefficients as well. In this method, the fused coefficient C(F,p) is obtained by choosing the corresponding coefficient with greater local deviation. If σ denotes the local deviation then:
( , ), ( , ) ( , ) ( , ) ( , ), ( , ) ( , ) C Bp B p Ap C F p C Ap Ap Bp
2.2.6 Convolution based Method
This fusion method was proposed by Chao Rui [8]. First a local energy is defined by using convolution as:
E ( A , p ) F 1 C A , p F 2 C A , p F 3 C A , p
2 2 2
where * means convolution and
F 1 ^ ^1 ,^1 ,^1 ,^2 ,^2 ,^2 ,^1 ,^1 ,^1 ^ (2.10)
F 2 ^ ^1 ,^2 ,^1 ,^1 ,^2 ,^1 ,^1 ,^2 ,^1 ^ (2.11)
12
2.2.8 Local Gradient and Weighted Average Method
This approach is based on the weighted average and local gradient based schemes and was proposed in [7]. The algorithm is as follows A local energy is defined as
E Ap q Q w ( q ) C ( A , q ) C ( A , p )
2 ( , ) (2.18)
where w(q) is a weight. The closer the point q is near the point p, the greater w(q) is. E(B,p) can also be calculated using equation (2.18). Then a match measure is defined as
( , ) ( , )
( ) ( , ) ( , ) ( , ) ( , ) ( ) E A p EB p
wq C Aq C A p CBq CB p M p qQ
(2.19)
if M(p) is smaller than a threshold T, then
( , ), ( , ) ( , ) ( , ) ( , ), ( , ) ( , ) C Bp EB p E Ap C F p C Ap EAp EB p
otherwise
( , ) ( , ), ( , ) ( , ) ( , ) ( , ) ( , ), ( , ) ( , ) max min
max min C Bp C Ap EB p E A p C F p C Ap C Bp E Ap EBp
W W
W W (2.20)
Where
T
M p
W 1
- 5 0. 5 1 ( )
min^ and^ W^ max^ ^1 W min (2.21)
During the implementation of this scheme, T was taken as 0.95 and window size as 3x3.
13
Section 3
3. Support Vector Machines
3.1 Introduction
Support Vector Machine (SVM) is a recently developed technique for pattern recognition and classification. It is based on strong foundations from the broad area of statistical learning theory [25]. It can be used for linearly as well as non-linearly separable classification problems. Input to the SVM is feature vectors of all training samples and their corresponding target values. SVM finds the hyperplanes to separate the samples belonging to different categories (classes) and returns some parameters which define the trained SVM. This trained SVM is then used to classify patterns (samples) other than those of used for training.
15
Section 4
4. SVM based Image Fusion
SVM is used by different researchers for fusion of different images. During the literature survey, three existing approaches for SVM based image fusion were studied [6,19,7].
4.1 General Framework
The general framework for SVM based image fusion is shown below in figure 4.1.
Source Images (^) Decomposition ExtractionFeature SVM Training
Fused Image ConsistencyVerification SVM Testing (Optional)
Figure 4.1: General Framework for SVM based Image Fusion
16
4.1.1 Description
The input to the any SVM based Image Fusion system will be the source images taken from one or different sources (visible, infrared, MRI, PET etc). Depending upon the algorithm, these images may be decomposed into sub-image blocks or transformed into some other domain like wavelet. After the decomposition, some sort of features are extracted from the decomposed images, these features are used to train an SVM. The trained SVM is applied over the whole original images (SVM Testing phase). Consistency Verification is the process to ensure that a fused coefficient does not come from a different source image from most of its neighbors [23], or in other words if center pixel value comes from image X, while majority of surrounding values come from image Y, the center pixel value is switched to that of image Y. Consistency Verification can be implemented by using a small majority filter. The output of SVM testing phase (Decision map optionally with Consistency Verification) is used to construct the fused image.
4.2 Algorithms
Different Image Fusion algorithms based on SVM studied during literature survey are discussed below.
4.2.1 Fusing Images with Different Focuses using SVM
This scheme was originally developed for Multi-Focus Image Fusion. In chapter 2, DWT based fusion methods „Select-Max‟ and „Weighted Average‟ were discussed. With the development of advanced classifiers like SVM, the performance of simple wavelet based schemes („Select-Max‟ and „Weighted Average‟ methods) can be improved [23]. The algorithm given in [23] is as follows: (1) Source images are decomposed using DWFT (Discrete Wavelet Frame Transform) to d levels, resulting in a total of 3d details and one approximation sub-band. As the approximation sub-band contains low frequency information of the image, so it will contain less edge information and thus cannot help in deciding the clarity of the source image. Hence this sub-band will not be used in constructing the feature vector for the SVM (but it will be used in reconstructing the fused image). In this algorithm, the wavelet coefficient of image A (or B) at
