Handwritten Digit Recognition using Convolutional Neural Networks and Gabor filters

prudencewooshΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 4 χρόνια και 19 μέρες)

89 εμφανίσεις

HandwrittenDigitRecognition
usingConvolutionalNeural
NetworksandGaborfilters
AndrØsCalder?n
SergioRoaOvalle
JorgeVictorino
andres.calderon@gruposimbiosis.com-s.roa@computer.org-
jorge_victorino@yahoo.com
GrupoSimbiosis
BogotÆ,Colombia
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.1/13
Agenda

Introduction

Inputsandoutputs

GaborFilters

ConvolutionalNeuralNetworks

GCNNtopology

GCNNtraining

Boostingmethod

Resultsandconclusions
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.2/13
Introduction

Theproblemofrecognizinghandwrittendigitsis
considered.

Convolutionalneuralnetworks(abilitytouseand
extractfeatureinformation)areinvestigated.

Gaborfiltersareintegratedintotheclassifierand
usedforfeatureextraction.

The
MNIST
trainingandtestsetsareusedfor
networktrainingandassessment.
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.3/13
Inputsandoutputs

Handwrittencharacters.60000trainingexamples
and10000testexamplesofsize28x28.
(normalized)

10targetoutputsrepresentingeachdigit(size
12x7)
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.4/13
Inputsandoutputs

Handwrittencharacters.60000trainingexamples
and10000testexamplesofsize28x28.
(normalized)

10targetoutputsrepresentingeachdigit(size
12x7)
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.4/13
Gaborfilters

Usedformultiresolution
analysis.(Descriptionofan
imageindifferentlevelsof
frequency)

Responsesimilartorecep-
tivefieldsinvisualcortex.
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.5/13
Gaborfilters

Usedformultiresolution
analysis.(Descriptionofan
imageindifferentlevelsof
frequency)

Responsesimilartorecep-
tivefieldsinvisualcortex.
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.5/13
ConvolutionalNeuralNetworks

Subsamplinglayer.Performs
subsamplingandlocalaveraging.
Reducesensitivitytodistortions.

Convolutionallayer.Performs
localfeatureextractionfromeach
receptivefieldthrough
convolution.

Outputlayer.Aperceptron.
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.6/13
ConvolutionalNeuralNetworks

Featuremapping.Weightsharing
(reductionoffreeparametersand
geometricinvariance).

Bi-pyramidaleffect.Invarianceto
shiftsanddistortions.(Simplecells
followedbycomplexcells)
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.7/13
ConvolutionalNeuralNetworks

Featuremapping.Weightsharing
(reductionoffreeparametersand
geometricinvariance).

Bi-pyramidaleffect.Invarianceto
shiftsanddistortions.(Simplecells
followedbycomplexcells)
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.7/13
GCNNtopology
Input
G1S2
C3S4
C5
Input image (28 x 28)
Subsampling layer 12 sublayers (14x14)
Convolutional layer 16 sublayers (10x10)
Subsampling layer 16 sublayers (5x5)
Convolutional layer 120 sublayers (1x1)
F6
Output layer 1 sublayer (1x84)
Gabor layer
12 sublayers (28x28)
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.8/13
GCNNtraining

Backpropagationalgorithm.

Weightsinicializedusinganormaldistribution
with

=
0and


1.

Learningrateadjustedusinga
search-then-convergeschedule(
n
0
1+
n

)

Stochasticupdateandrandompresentationof
patterns.
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.9/13
Boosting
First Expert
Third Expert
Second Expert
misclassified patterns
correct classified patterns
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.10/13
Results
Comparativeresultsover10000testexamplestaken
fromMNISTdatabase.(%ofmisclassification)
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.11/13
Conclusions

GCNNsperformfeatureextractiontasks
effectively.Performanceimprovedcomparedto
otherCNNarchitecturesandotherclassifiers.

GaborfiltersandCNNscanbeintegrated-
similarprinciples.

Improvementinclassificationperformanceusing
aboostingmethod.

EffectivenessofBackpropagationalgorithm.
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.12/13
References

LeCun,Bottou,Bengio,Haffner.GradientBased
LearningAppliedtoDocumentRecognition.
IntelligentSignalProcessing,IEEEPress,2001.

Daugman.Completediscrete2DGabor
transformsbyneuralnetworksforimageanalysis
andcompression.IEEETransactionson
Acoustics,SpeechandSignalProc.1988.

Drucker,Schapire,Simard.Improving
performanceinneuralnetworksusingabosting
algorithm.AdvancesinNeuralInformation
ProcessingSystems.Calif.1993.

Haykin.NeuralNetworks:Acomprehensive
foundation.PrenticeHall,N.J.,1999.
CIIC2003-HandwrittenDigitRecognitionusingGCNNs-
c
2003p.13/13