Object Detection
•Overview
•Viola-Jones
•Dalal-Triggs
•Deformable models
•Deep learning
Recap: Viola-Jones sliding windowdetector
Fast detection through two mechanisms
•Quickly eliminate unlikely windows
•Use features that are fast to compute
Cascade for Fast Detection
Examples
Stage 1
H1(x) > t1?
Reject
No
Yes
Stage 2
H2(x) > t2?
Stage N
HN(x) > tN?
Yes
…
Pass
Reject
No
Reject
No
•Choose threshold for low false negative rate
•Fast classifiers early in cascade
•Slow classifiers later, but most examples don’t get there
Features that are fast to compute
•“Haar-like features”
–Differences of sums of intensity
–Thousands, computed at various positions andscales within detection window
Two-rectangle features
Three-rectangle features
Etc.
-1
+1
Integral Images
•ii = cumsum(cumsum(im, 1), 2)
x, y
ii(x,y) = Sum of the values in the grey region
SUM within Rectangle D isii(4) - ii(2) - ii(3) + ii(1)
Feature selection with Adaboost
•Create a large pool of features (180K)
•Select features that are discriminative and workwell together
–“Weak learner” = feature + threshold + parity
–Choose weak learner that minimizes error on theweighted training set
–Reweight
Adaboost
Viola-Jones details
•38 stages with 1, 10, 25, 50 … features
–6061 total used out of 180K candidates
–10 features evaluated on average
•Training Examples
–4916 positive examples
–10000 negative examples collected after each stage
•Scanning
–Scale detector rather than image
–Scale steps = 1.25 (factor between two consecutive scales)
–Translation 1*scale (# pixels between two consecutive windows)
•Non-max suppression: average coordinates of overlappingboxes
•Train 3 classifiers and take vote
Viola Jones Results
MIT + CMU face dataset
Speed = 15 FPS (in 2001)
Object Detection
•Overview
•Viola-Jones
•Dalal-Triggs
•Deformable models
•Deep learning
Statistical Template
Object model = sum of scores of features at fixedpositions
+3
+2
-2
-1
-2.5
= -0.5
+4
+1
+0.5
+3
+0.5
= 10.5
> 7.5
?
> 7.5
?
Non-object
Object
Design challenges
How to efficiently search for likely objects
•Even simple models require searching hundreds ofthousands of positions and scales
Feature design and scoring
•How should appearance be modeled? What featurescorrespond to the object?
How to deal with different viewpoints?
•Often train different models for a few differentviewpoints
Implementation details
•Window size
•Aspect ratio
•Translation/scale step size
•Non-maxima suppression
Example: Dalal-Triggs pedestrian detector
1.Extract fixed-sized (64x128 pixel) window ateach position and scale
2.Compute HOG (histogram of gradient)features within each window
3.Score the window with a linear SVM classifier
4.Perform non-maxima suppression to removeoverlapping detections with lower scores
Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05
Slides by Pete Barnum
Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05
•Tested with
–RGB
–LAB
–Grayscale
•Gamma Normalization and Compression
–Square root
–Log
Slightly better performance vs. grayscale
Very slightly better performance vs. no adjustment
uncentered
centered
cubic-corrected
diagonal
Sobel
Slides by Pete Barnum
Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05
Outperforms
•Histogram of gradientorientations
–Votes weighted by magnitude
–Bilinear interpolation betweencells
Orientation: 9 bins(for unsigned angles0 -180)
Histograms ink x k pixel cells
Slides by Pete Barnum
Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05
Normalize with respect tosurrounding cells
Slides by Pete Barnum
Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05
X=
Slides by Pete Barnum
Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05
# features = 15 x 7 x 9 x 4 = 3780
# cells
# orientations
# normalizations byneighboring cells
Original Formulation
X=
Slides by Pete Barnum
Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05
# features = 15 x 7 x 9 x 4 = 3780
# cells
# orientations
# normalizations byneighboring cells
# features = 15 x 7 x (3 x 9) + 4 = 3780
# cells
# orientations
magnitude ofneighbor cells
UoCTTI variant
Original Formulation
Slides by Pete Barnum
Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05
pos w
neg w
pedestrian
Slides by Pete Barnum
Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, CVPR05
Pedestrian detection with HOG
•Train a pedestrian template using a linear support vectormachine
•At test time, convolve feature map with template
•Find local maxima of response
•For multi-scale detection, repeat over multiple levels of a HOGpyramid
Template
HOG feature map
Detector response map
Something to think about…
•Sliding window detectors work
–very well for faces
–fairly well for cars and pedestrians
–badly for cats and dogs
•Why are some classes easier than others?
Strengths and Weaknesses of Statistical TemplateApproach
Strengths
•Works very well for non-deformable objects withcanonical orientations: faces, cars, pedestrians
•Fast detection
Weaknesses
•Not so well for highly deformable objects or “stuff”
•Not robust to occlusion
•Requires lots of training data
Tricks of the trade
•Details in feature computation really matter
–E.g., normalization in Dalal-Triggs improves detection rateby 27% at fixed false positive rate
•Template size
–Typical choice is size of smallest detectable object
•“Jittering” to create synthetic positive examples
–Create slightly rotated, translated, scaled, mirroredversions as extra positive examples
•Bootstrapping to get hard negative examples
1.Randomly sample negative examples
2.Train detector
3.Sample negative examples that score > -1
4.Repeat until all high-scoring negative examples fit inmemory
Things to remember
•Sliding window for search
•Features based on differences ofintensity (gradient, wavelet, etc.)
–Excellent results require careful featuredesign
•Boosting for feature selection
•Integral images, cascade for speed
•Bootstrapping to deal with many,many negative examples
Examples
Stage 1
H1(x) >t1?
Reject
No
Yes
Stage 2
H2(x) >t2?
Stage N
HN(x) >tN?
Yes
…
Pass
Reject
No
Reject
No
Many slides from Lana Lazebnik based on P. Felzenszwalb
Generic object detection withdeformable part-based models
Challenge: Generic object detection
Histograms of oriented gradients (HOG)
•Partition image into blocks at multiple scales andcompute histogram of gradient orientations in eachblock
10x10 cells
20x20 cells
Image credit: N. Snavely
Histograms of oriented gradients (HOG)
•Partition image into blocks at multiple scales andcompute histogram of gradient orientations in eachblock
Image credit: N. Snavely
Pedestrian detection with HOG
•Train a pedestrian template using a linearsupport vector machine
positive training examples
negative training examples
Are we done?
•Single rigid template usually not enough torepresent a category
•Many objects (e.g. humans) are articulated, orhave parts that can vary in configuration
•Many object categories look very different fromdifferent viewpoints, or from instance to instance
Slide by N. Snavely
Discriminative part-based models
P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan, Object Detectionwith Discriminatively Trained Part Based Models, PAMI 32(9), 2010
Rootfilter
Partfilters
Deformationweights
Discriminative part-based models
P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan, Object Detectionwith Discriminatively Trained Part Based Models, PAMI 32(9), 2010
Multiple components
Discriminative part-based models
P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan, Object Detectionwith Discriminatively Trained Part Based Models, PAMI 32(9), 2010
Object hypothesis
•Multiscale model: the resolution of partfilters is twice the resolution of the root
Scoring an object hypothesis
•The score of a hypothesis is the sum of filter scoresminus the sum of deformation costs
Filters
Subwindowfeatures
Deformation weights
Displacements
Scoring an object hypothesis
•The score of a hypothesis is the sum of filter scoresminus the sum of deformation costs
Concatenation of filterand deformationweights
Concatenation ofsubwindow featuresand displacements
Filters
Subwindowfeatures
Deformation weights
Displacements
Detection
•Define the score of each root filter location as thescore given the best part placements:
Detection
•Define the score of each root filter location as thescore given the best part placements:
•Efficient computation: generalized distance transforms
•For each “default” part location, find the score of the“best” displacement
Head filter
Deformationcost
Detection
•Define the score of each root filter location as thescore given the best part placements:
•Efficient computation: generalized distance transforms
•For each “default” part location, find the score of the“best” displacement
Head filter responses
Distance transform
Head filter
Detection
Detection result
Training
•Training data consists of images with labeledbounding boxes
•Need to learn the filters and deformation parameters
Training
•Our classifier has the form
•w are model parameters, z are latent hypotheses
•Latent SVM training:
•Initialize w and iterate:
•Fix w and find the best z for each training example (detection)
•Fix z and solve for w (standard SVM training)
•Issue: too many negative examples
•Do “data mining” to find “hard” negatives
Car model
Component 1
Component 2
Car detections
Person model
Person detections
Cat model
Cat detections
Bottle model
More detections
Quantitative results (PASCAL 2008)
•7 systems competed in the 2008 challenge
•Out of 20 classes, first place in 7 classes andsecond place in 8 classes
Bicycles
Person
Bird
Proposed approach
Proposed approach
Proposed approach
Detection state of the art
Object detection system overview. Our system (1) takes an input image, (2) extractsaround 2000 bottom-up region proposals, (3) computes features for each proposalusing a large convolutional neural network (CNN), and then (4) classifies each regionusing class-specific linear SVMs. R-CNN achieves a mean average precision (mAP)of 53.7% on PASCAL VOC 2010. For comparison, Uijlings et al. (2013) report 35.1%mAP using the same region proposals, but with a spatial pyramid and bag-of-visual-words approach. The popular deformable part models perform at 33.4%.
R. Girshick, J. Donahue, T. Darrell, and J. Malik, Rich Feature Hierarchies for AccurateObject Detection and Semantic Segmentation, CVPR 2014, to appear.
