Deep Dive into CNNs for Image Recognition: LeNet-5, AlexNet, GoogLeNet, VGGNet, and ResNet, Cheat Sheet of Compiler Design

Download Deep Dive into CNNs for Image Recognition: LeNet-5, AlexNet, GoogLeNet, VGGNet, and ResNet and more Cheat Sheet Compiler Design in PDF only on Docsity!

Convolutional neural network

for Image recognition

Reference http://cs231n.github.io/convolutional-networks/ By Yunzhe Xue

Dense neural network and

Convolutional neural network

Convolutional

kernel

This is a gif image

Convolutional kernel

Padding on the input volume with zeros in such way that the conv layer does not alter the spatial dimensions of the input

Pooling layer

Pooling

LeNet-5 for MNIST

CIFAR10 dataset and state of the art The CIFAR-10 dataset consists of 60000 32x32 color images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.

Case studies

• (^) LeNet. The first successful applications of Convolutional Networks were developed by Yann LeCun in 1990’s. Of these, the best known is the LeNet architecture that was used to read zip codes, digits, etc.
• (^) AlexNet. The first work that popularized Convolutional Networks in Computer Vision was the AlexNet, developed by Alex Krizhevsky, Ilya Sutskever and Geoff Hinton. The AlexNet was submitted to the ImageNet ILSVRC challenge in 2012 and significantly outperformed the second runner-up (top 5 error of 16% compared to runner-up with 26% error). The Network had a very similar architecture to LeNet, but was deeper, bigger, and featured Convolutional Layers stacked on top of each other (previously it was common to only have a single CONV layer always immediately followed by a POOL layer).

Case studies

• GoogLeNet. The ILSVRC 2014 winner was a

Convolutional Network from Szegedy et al. from Google. Its main contribution was the development of an Inception Module that dramatically reduced the number of parameters in the network (4M, compared to AlexNet with 60M). Additionally, this paper uses Average Pooling instead of Fully Connected layers at the top of the ConvNet, eliminating a large amount of parameters that do not seem to matter much. There are also several followup versions to the GoogLeNet, most recently Inception-v4.

Case studies

• ResNet. Residual Network developed by Kaiming He et

al. was the winner of ILSVRC 2015. It features special skip

connections and a heavy use of batch normalization. The

architecture is also missing fully connected layers at the

end of the network. The reader is also referred to

Kaiming’s presentation (video, slides), and some

recent experiments that reproduce these networks in

Torch. ResNets are currently by far state of the art

Convolutional Neural Network models and are the default

choice for using ConvNets in practice (as of May 10,

2016). In particular, also see more recent developments

that tweak the original architecture from Kaiming

He et al. Identity Mappings in Deep Residual Networks

(published March 2016).

VGG- GoogleNe t ResNet