Artificial Neural Network, Lecture notes of Machine Learning

Download Artificial Neural Network and more Lecture notes Machine Learning in PDF only on Docsity!

Artificial Neural Networks

Outline

◼ The Brain

◼ Perceptrons

◼ Gradient descent

◼ Multi-layer networks

◼ Backpropagation

The biological inspiration

• The brain has been extensively studied by

scientists.

• Vast complexity prevents all but rudimentary

understanding.

• Even the behaviour of an individual neuron

is extremely complex

Features of the Brain

• Ten billion ( 10 ) neurons
• Neuron switching time >
  • 3 secs
• Face Recognition ~0.1secs
• On average, each neuron has several thousand connections
• Hundreds of operations per second
• High degree of parallel computation
• Distributed representations
• Die off frequently (never replaced)
• Compensated for problems by massive parallelism

The contrast in architecture

• The Von Neumann architecture

uses a single processing unit;

• Tens of millions of operations per second
• Absolute arithmetic precision
• The brain uses many slow

unreliable processors acting in

parallel

The Structure of Neurons axon cell body synapse nucleus dendrites

The Structure of Neurons

• Axons connect to dendrites via synapses.
• Electro-chemical signals are propagated

from the dendritic input, through the cell

body, and down the axon to other neurons

A neuron has a cell body, a branching i nput

structure (the dendr I te) and a branching

o utput structure (the ax O n)

Properties of Artificial Neural Nets (ANNs)

Appropriate Problem Domains for Neural Network Learning

◼ Input is high-dimensional discrete or real-

valued (e.g. raw sensor input)

◼ Output is discrete or real valued

◼ Output is a vector of values

◼ Form of target function is unknown

◼ Humans do not need to interpret the results

(black box model)

Perceptron

◼ Linear treshold unit (LTU)

x 1 x 2 x n

w 1 w 2 w n

w

0

x

0=

 w

i

x

i

1 if  w

i x i

o(x i

• 1 otherwise

o

n i= i= n

Supervised Learning

◼ Training and test data sets

◼ Training set; input & target

Perceptron Training

◼ Linear threshold is used.

◼ W - weight value

◼ t - threshold value

1 if  w

i x i

t Output=

0 otherwise

i=

Training Perceptrons t = 0. y x

- 1 W =? W =? W =? For AND A B Output 0 0 0 0 1 0 1 0 0 1 1 1

• What are the weight values?
• Initialize with random weight values

Training Perceptrons t = 0. y x

- 1 W = 0. W = - 0. W = 0. I 1 I 2 I 3 Summation Output -1 0 0 (-10.3) + (00.5) + (0-0.4) = -0.3 0 -1 0 1 (-10.3) + (00.5) + (1-0.4) = -0.7 0 -1 1 0 (-10.3) + (10.5) + (0-0.4) = 0.2 1 -1 1 1 (-10.3) + (10.5) + (1-0.4) = -0.2 0 For AND A B Output 0 0 0 0 1 0 1 0 0 1 1 1