Content Based ImageRetrieval
Romit Das · Ryan Scotka
Go to Google Image Search Home
The image “http://omega.widows-web.com/Beatles/McCartney.jpg” cannot be displayed, because it contains errors.
The image “http://www.mit.edu/people/swanson/liem/May%2028%202002/1%20Ninja.jpg” cannot be displayed, because it contains errors.
ninjaparty
GIS Problems
•Search based on filename
–Verbatim match
–Noun replacement
•Potential for Abuse (Google Hack)
Possible Solutions
•Metadata
–Standards
–Re-index existing images
•Manual Classification
–Time
•Content-based Classification
CBIR – Training
1.Choose features to distinguish images.
2.Extract said features.
3.Apply statistical method to modelfeatures.
4.Categorize based on textual description.
Example
The image “http://www.mit.edu/people/swanson/liem/May%2028%202002/1%20Ninja.jpg” cannot be displayed, because it contains errors.
Dimensions
Color Frequencies
Spatial Distribution
200 x 200 + Mostly flesh tones + Flesh tones concentrated in the center =
baby
Author’s Feature Set
•Feature Set (6 dimensions):
–Color averages (LUV)
–High-frequency energy bands
•“Effectively discern local texture”
•Wavelet transform on 4x4 blocks
•Use HL, LH, and HH “high energy bands”
•Use the LL for lower resolution analysis
Author’s Implementation
•Statistical Modeling
–Use machine learning to build concepts
Concept = Paris
Training Set =
The image “http://cemail2.ce.ntu.edu.tw/photo/tower/tower3bThe%20Eiffel%20Tower,%20Paris%20(%20301m%20).jpg” cannot be displayed, because it contains errors.
The image “http://www.photo.net/philg/digiphotos/200101-d30-paris/paris-traffic.half.jpg” cannot be displayed, because it contains errors.
The image “http://www.justdesserts.com/bay/images/croissant.jpg” cannot be displayed, because it contains errors.
Markov Models
•Take known facts
•Deduce hidden/unknown data
Markov Model Example
•Given:
–Queues of people, shelves, price labels,disgruntled workers
•Possible Results:
–Post office
–Supermarket
–Record Store
Markov Model Example
•Given:
–Queues of people, shelves, price labels,disgruntled workers, food products
•Possible Results:
–Post office
–Supermarket
–Record Store
Ninja Model
ninjaparty
Person, outdoors
Ninja Model
ninjaparty
People, ninjas, outdoor
Ninja Model
ninjaparty
People, ninjas, weapons,outdoors
Ninja Markov Model
ninjaparty
ninjaparty
ninjaparty
Person, outdoors
People, ninjas, outdoors
People, ninjas, outdoors
weapons, class photo
Creating Concepts
•Training Concept
–Created from hand-picked images
–Must choose statistically significant trainingsize
•Resulting Concept
–Used in automatic cataloging of future images
Observations
•Images are associated with multipleconcepts.
•Not foolproof
•Example:
ninjaparty
People, ninjas, outdoors
weapons, class photo
Advantages
•Automatic categorization
Disadvantages
•False positives
–Concepts may require a vast amount ofimages
•Increases training time
•Dissimilar images needed for training of aconcept
Future Additions
•Further refinement of conflicting semantics
•Weights assigned to classifications
Our Implementation
•Perform classification with alternatelearners (Weka)