Deep Residual Learning for Image Recognition*, Summaries of Digital Image Processing

Download Deep Residual Learning for Image Recognition* and more Summaries Digital Image Processing in PDF only on Docsity!

Deep Residual Learning for

Image Recognition*

Wei-Pang Jan, Xuanqing Liu

• Most of the figures/tables credit to He et al. Deep Residual Learning for Image Recognition In CVPR 2016

Motivation

Revolution of Depth

Is deeper network better at learning?

Gradient Vanishing/Exploding

http://neuralnetworksanddeeplearning.com/chap5.html

Is deeper network better at learning?

ResNet Intuitions

Residual Learning

(Plain net)

Residual Learning

F(x) = H(x) - x

Shortcuts

Feedforward low level feature to deeper layers

• Feature reuse
• Reduces number of parameter

Resolves vanishing gradient

• y = f(x) vs. y = f(x) + x

Resolving Gradient

Vanishing Problem

Experiments

Architecture

CIFAR-10 Experiment Result

Identity vs. Projection Shortcuts