A Web-based QuestionAnswering System
Yu-shan & Wenxiu
03.08.2005
Outline
•System Architecture
•Query Expansion
•Pattern Learning
•Answering Extraction
•Performance & Evaluation
Our QA
•Massive web documents based
– how to eliminate noise…
•Question classification
– Focus on LOC
–(LOC:city, LOC:country, LOC:state, LOC:other…)
•Multiple Query Expansion
•Suffix tree to aid surface pattern learning
•Use Regular Expression to extract answer
System Architecture
Question Expansion
•No expansion
•Delete question words & stop words
–Save more space for expansion
•WordNet Synonym Expansion
–Word ambiguous
•Dependency-based Word SimilarityExpansion
–Prof. Lin@Univ. of Alberta
–http://www.cs.ualberta.ca/~lindek/demos/depsim.htm
Question Expansion Example
•Question:
–What is the largest city in the world?
•Primary keywords:
–is largest city world
•WordNet expansion:
–is largest city world be metropolis human race
•Similarity expansion:
–is largest city world doing town region
Suffix Tree
•Construct a general suffix tree for the top 100snippets
–each node has an index of the sentences it appears.
–Set threshold N, pick up the longest commonsubstrings which appear in more than N sentence,use them as candidate answers.
–Manually filtering correct candidate answers toconstruct patterns
•Huge Noise…
–Query keywds, common words, part of phrases…
–Laborious manual work…
LOC:city Patterns
•<NAME> is <ANS>
•<ANS> is <NAME>
•<ANS>, <NAME>
•<NAME>, <ANS>
•<NAME>, ….., is <ANS>
•<ANS>, ……, is <NAME>
•<NAME>, ……, <ANS>
•<ANS>,……, <NAME>
•<ANS> <NAME>
•<NAME> <ANS>
Answer Extraction
•Regular Expression Match
•Known_city_list filtering…
–Contains 4682 cities…
•Direct pick out citynames from snippet
•Combine two approaches
•DEMO TIME
Answers from Google
•What is the largest city in the world?
•TREC answer: Tokyo
•Our answer:
–Seoul
–Memphis
–Tokyo
–Mexico City
–Los Angeles
Google snippet:
“With a population of more than 10.2 million, Seoul, the capital ofSouth Korea, is the world?s largest city in terms of population.”
“…… we see that Memphis remained the largest city in the worldfrom at least 3100 BCE to 2240 BCE”
Evaluation
•Use TREC-10 Labeled Questions
ORIG
PRIM
WN
SIM
MRR
0.75
0.6
0.47
0.52
PropCorrect
0.8
0.8
0.6
0.5