Networks - Multimedia Signal Processing - Lecture Slides, Slides of Electronics engineering

Download Networks - Multimedia Signal Processing - Lecture Slides and more Slides Electronics engineering in PDF only on Docsity!

Neural Networks

Neural Net types

Basics of neural network theory and practice for

supervised and unsupervised learning.

Most popular Neural Network models:

• architectures
• learning algorithms
• applications

Connectionism

• Connectionist techniques (a.k.a. neural networks) are inspired by the strong interconnectedness of the human brain.
• Neural networks are loosely modeled after the biological processes involved in cognition: 1. Information processing involves many simple elements called neurons. 2. Signals are transmitted between neurons using connecting links. 3. Each link has a weight that controls the strength of its signal. 4. Each neuron applies an activation function to the input that it receives from other neurons. This function determines its output.
• Links with positive weights are called excitatory links.
• Links with negative weights are called inhibitory links.

What is a neural network

• A NN is a machine learning approach inspired by the way in which the brain performs a particular learning task: - Knowledge about the learning task is given in the form of examples. - Inter neuron connection strengths (weights) are used to store the acquired information (the training examples). - During the learning process the weights are modified in order to model the particular learning task correctly on the training examples.

• Supervised Learning
  • Recognizing hand-written digits, pattern recognition, regression.
  • Labeled examples (input , desired output)
  • Neural Network models: perceptron, feed-forward, radial basis function, support vector machine.
• Unsupervised Learning
  • Find similar groups of documents in the web, content addressable memory, clustering.
  • Unlabeled examples (different realizations of the input alone)
  • Neural Network models: self organizing maps, Hopfield networks.

Learning

Network architectures

• Three different classes of network architectures
  • single-layer feed-forward neurons are organized
  • multi-layer feed-forward in acyclic layers
  • recurrent
• The architecture of a neural network is linked with the learning algorithm used to train

Multi layer feed-forward

Input

layer

Output

layer

Hidden Layer

3-4-2 Network

Recurrent Network with hidden neuron(s) : unit delay operator z-1^ implies dynamic system

z-

z-

z-

Recurrent network

input hidden output

The Neuron

• The neuron is the basic information processing unit of a NN. It consists of:

1 A set of synapses or connecting links, each link

characterized by a weight:

W 1 , W 2 , …, Wm

2 An adder function (linear combiner) which computes

the weighted sum of the inputs:

3 Activation function (squashing function) for limiting

the amplitude of the output of the neuron.

 (^) 

m

u wjxj

j



y (u  b )

The Neuron

Input signal

Synaptic weights

Summing function

Bias

b

Activation Local function Field

v

Output

y

x 1

x 2

xm

w 2

wm

w 1

Bias of a Neuron

• Bias b has the effect of applying an affine transformation to u

v = u + b

• v is the induced field of the neuron

v

u

 (^)  

m

1

u wjxj j

Dimensions of a Neural Network

• Various types of neurons
• Various network architectures
• Various learning algorithms
• Various applications

Handwritten digit recognition

A Multilayer Net for XOR