Download Artificial Neural Networks and more Study Guides, Projects, Research Applications of Computer Sciences in PDF only on Docsity!
2. Artificial Neural Networks:
An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by the way biological nervous systems, such as the human brain, process information. Human brain consists of millions of interconnected neurons. Each neuron is a cell that uses biochemical reactions to receive, process, and transmit information. A typical neuron collects signals from others neurons through a crowd of fine structures called dendrites which are connected to the cell body or soma. The neuron sends electrical signal to other neurons with the help of a long, thin fiber known as an axon , which splits into strands and sub-strands , and is connected to other neurons through synapses that converts the activity from neuron into an electrical signal and takes it to the other neuron [1, 5, 11]. The basic structure of human neurons is shown in Figure 1.
Figure 1: Human Neuron [5]
ANNs have been developed as generalizations of mathematical models of biological nervous systems. As ANNs are inspired by biological nervous system, they have very much nervous system like structure. They are composed of a large number of highly interconnected processing elements, called artificial neurons or simply neurons or nodes, working together to solve specific problems. ANNs, like people, learn by example. Learning in biological systems involves adjustments to the synaptic connections that exist between the neurons. This is true for ANNs as well. In a simplified mathematical model of the neuron, the effects of the synapses are represented by
connection, and the nonlinear characteristic exhibited by neurons is represented by a transfer function. The neuron impulse is then computed as the weighted sum of the input signals, transformed by the transfer function. The learning capability of an artificial neuron is achieved by adjusting the weights in accordance to the chosen learning algorithm [5]. The architecture of artificial neuron is shown in Figure 2.
Figure 2: Architecture of Artificial Neuron [5]
The very basic architecture of neural networks consists of three types of layers: input, hidden and output layers. Referring to Figure 2, the information flow from inputs x1,...,xn is considered to be unidirectional, which are indicated by arrows. The neuron output signal O is given by the following relationship:
(1)
Where wj is the weight and the function f (net) is referred to as an activation function. The variable net is defined as a scalar product of the weight and input vectors [1, 5]. (2)
A neural network must be configured so that a specific set of inputs produces the desired set of outputs. Usually this is done by setting the weights of a network. There are various methods used to set the weights. One method is to use some priori knowledge and set the weights explicitly. Another method is to train the neural network by feeding it
1
n
O f net f j j xj
^
net 1 x 1 2 x 2 ... n xn
3. Evolutionary Computation:
Evolutionary Computation is a subfield of artificial intelligence, and is based on the evolution of biological life in natural world. It involves Evolutionary Algorithms used for optimization e.g. Genetic Algorithms. Genetic algorithms abbreviated as GAs , are among fast growing optimization techniques of recent times. These belong to the class of broadly applicable, general- purpose, generate-and-test optimization methods and search procedures based on the mechanism of natural genetics. Some of the advantages of genetic algorithms are as follows: GAs are simple but even then they help us solving complex problems that other techniques might not have ability to accomplish. They help in escaping from local extremes without supervision during a simulation that’s why they have become global optimization methods. These have show better search efficiency as compared to traditional optimization algorithms. GA is a very effective way of quickly finding a reasonable solution to a complex problem. They are good at doing better searches through a large and complex search space, and are most effective in a search space for which little is known. If the exact solution is known, but you don’t know how to achieve this solution; GAs are the best approach to solve this problem. Put simply: use genetic algorithms for everything you cannot easily do with another algorithm [9, 11].
3.1. Some Biological Terms Used in Genetic Algorithms:
As techniques used in GAs are inspired by natural genetics, it uses a lot of biological terms. In this section we will review the correspondence between these biological terms and some common terms used in GAs [2, 9].
All living organisms consist of cells. In each cell there is the same set of chromosomes. Each chromosome consists of genes , the building block of DNA. In GAs, a chromosome is referred to as a string. One or more chromosomes combine together to form the whole genetic package called genotype. In GAs, the total genetic package is called structure. Basically each gene encodes a trait, or physical property for example color of eyes and possible settings for a trait (e.g. blue, brown) are called alleles. In GAs, alleles are known as feature value. Each gene has its own position in the chromosome termed as locus analogous to string position in GAs. The organism formed by the interaction of genotypes is called phenotype which is known as parameter set in artificial genetic systems. In the process of reproduction , first a recombination (or crossover ) occurs and then genes from the parents form a new chromosome. The new created offspring can be mutated. Mutation i.e. the elements of DNA are a bit changed. These changes are mainly caused by errors in copying genes from parents. The fitness of an organism is measured by success of the organism in its life.
3.2. Working of Genetic Algorithms: The working process of genetic algorithms can be divided into the following steps:
3.2.1. Initialization: Initially many individual solutions are randomly generated to form an initial population. The size of the population depends on the nature of the problem, and may contain several hundreds or thousands of possible solutions. In common practice, we generate a random population that covers the entire range of possible solutions [8, 9].
3.2.5. Algorithm:
Keeping in view the above mentioned steps, algorithm can be constructed as [9, 10, 11]:
Choose initial population Evaluate the fitness of each individual in the population Repeat o Select fittest individuals to reproduce a new population o Breed new generation through crossover and mutation to give birth to offspring o Evaluate the individual fitness of the offspring o Replace worst ranked part of population with offspring Until some termination condition is met
3.3. Applications: Genetic algorithms are being used in a number of diverse applications such as: Function Optimization System Identification and Control Image Processing Combinatorial problems Neural Networks Fuzzy Systems Robotics
4. Evolutionary Neural Networks:
Evolutionary Neural Networks (ENNs) refer to a special class of Artificial Neural Networks (ANNs). ENNs can be considered as the combination of ANNs and evolutionary computation techniques. Training artificial neural networks for a particular task using evolutionary computation is a hot topic of research for several past years, and this training is done by using genetic algorithms discussed above.
4.1. Why ENNs?
They question may be when we have ordinary ANNs, why we use ENNs. The answer to this question is very simple and that is ENNs have several advantages over ordinary ANNs. Some of the advantages are: ENNs evolve towards the optimal solution without any interference from the outside world, and thus they eliminate the tedious trial and error method of manually finding an optimal solution for the ANNs. ENNs can find an optimal architecture through an evolutionary process that helps a computer scientist in developing more powerful and more intelligent artificial systems that can decide their own architecture according to the different tasks assigned to them.
4.2. Evolution in Neural Networks:
Evolution is introduced at three different levels in ANNs. These three levels are Connection Weights, Architecture, and Learning Rule. These are described below.
4.2.1. The Evolution of Connection Weights:
In ANNs, the evolution of connection weights consists of two major phases. The first phase decides the representation of connection weights i.e. whether in the form of binary strings or not. In the second phase, the evolutionary process is simulated by an evolutionary algorithm, in which search operators such as crossover, mutation and / or
two types of encoding schemes are used to represent the information about the architecture of an ENN [3, 4]. Direct Encoding Scheme encodes all the details i.e. every connection and node of an architecture can be specified by the chromosome. Indirect Encoding Scheme encodes only the most important parameters of an architecture e.g. the number of hidden layers and hidden nodes in each layer. Other details about the architecture are left to the training process to decide. The evolution of architecture is described by the cycle given below [3, 4]: Decode each individual in the current generation into an architecture. Train each ENN with the by a predefined learning rule. Start form random initial connection weights and learning rule parameters. Compute the fitness of each encoded architecture. Select parents from the population based on their fitness. Apply search operations to the parents and generate offspring to produce the next generation.
4.2.3. The Evolution of Learning Rules: The learning rule (Training Algorithm) is used to adjust the connection weights of an ANN, and it may have different performance when applied to different architectures. Its design totally depends upon the architecture used in the network. In order to evolve a learning rule we have to make some assumptions which are not necessarily to be true in real practice. The evolution of learning rule is described by the cycle given below [3, 4]: Decode each individual in the current generation into a learning rule. Construct a set of ENNs and train them using the decoded learning rule. Calculate the fitness of each individual according to the average training result. Select parents from the current generation according to their fitness. Apply search operators to parents to generate offspring to produce the next generation.
4.3. Applications: Neural networks have wide applicability in different real life problems. They are very much successful in most of the industries. They are being used in medicine, business etc. Some of the applications are described below.
4.3.1. Neural Networks in Medicine: Neural Networks are currently a very hot research area in medicine. They are used extensively in biomedical systems. They are used to predict different diseases like lung cancer, breast cancer, arrhythmia etc. They use scans to detect different diseases because they don’t need any specific algorithm to identify the disease, and that’s why they are very much successful in this field [11].
4.3.2. Neural Networks in Time Series Prediction: Time Series is used to predict the future events based on the known past events. It is commonly used in business markets. Neural Networks are widely used in time series predictions e.g. weather forecast, power consumption forecast and sales forecasting etc [11].
4.4. Datasets for Empirical Evaluation in Pattern Classification: The datasets are obtained from UC-Irvine archive [12] of machine learning repository. Some of the datasets are given below:
4.4.1. Dataset for Breast Cancer: Here the aim is to distinguish whether the cancer is malignant or it is in the beginning phase. We will have 699 instances, 2 classes and 30 numeric attributes [12].
5. Project Plan:
The semester wise distribution of the work is given below.
5.1. Semester 6 th^ :
Understanding of different algorithms of Neural Networks. Understanding the basics of Genetic Algorithms. Preparation of Project Proposal document.
5.2. Semester 7 th^ :
Optimization of different Neural Network Architectures using some Evolutionary Computation like Genetic Algorithms.
5.3. Semester 8 th^ :
Benchmark testing of particular classification problems like Pattern Classification and Time Series Prediction. Thesis Writing
5.4. Project Duration:
This project is to be completed by June 2008.
References:
[1]. Simon Haykin, Neural Networks: A Comprehensive Foundation , 2 nd^ Edition, Dorling Kindersley, Delhi, India, 2006 [2]. David E. Goldberg, Genetic Algorithms in Search, Optimization & Machine Learning , Dorling Kindersley, Delhi, India, 2006 [3]. Xin Yao, “Evolving Artificial Neural Networks”, IEEE, New York, USA, 1999 [4]. Xin Yao, “Evolutionary Artificial Neural Networks”, International Journal of Neural Systems, 1993 [5]. Ajith Abraham, Nature and Scope of AI Techniques, Handbook for Measurement Systems Design, Peter Sydenham and Richard Thorn (Eds.), John Wiley and Sons Ltd., London, pp. 893-900, 2005 [6]. Ajith Abraham, Evolutionary Computation, Handbook for Measurement Systems Design, Peter Sydenham and Richard Thorn (Eds.), John Wiley and Sons Ltd., London, pp. 920-931, 2005 [7]. Neural Network URL: http://en.wikipedia.org/wiki/Neural_network [8]. Genetic Algorithm URL: http://en.wikipedia.org/wiki/Genetic_algorithm [9]. Introduction to Genetic Algorithms URL: http://www.rennard.org/alife/english/gavintrgb.html [10]. “Evolutionary Algorithms: Genetic Algorithms, Evolutionary Programming and Genetic Programming” URL: http://www.cs.sandia.gov/opt/survey/ea.html [11]. Christos Stergiou and Dimitrios Siganos, “Neural Networks” URL: http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html [12]. UCI Machine Learning Repository URL: http://www.ics.uci.edu/~mlearn/MLRepository.html
