Prior Work
Pseudo Relevance Feedback (PRF)(Kishida K, NTCIR-3; Itoh H, NTCIR-4)
QE using Rocchio formula: no significant improvement
QE using Taylor formula: no significant improvement
Reweighting query terms using PRF: no significant improvement
Inter Query Expansion (QE) for Patent Invalidity Search(Takeuchi H. et al, NTCIR-5)
QE for individual claims from same patent topic: significantimprovement, but not applicable for other patent search tasks
Improving Retrievability for Patents(Bashir and Rauber, ECIR 2010)
Enrich queries to improve the retrievability of patents with lowchance of retrieval, but not tested for real patent search task
Testing QE for Prior-Art Patent Search
CLEF-IP 2010:
1.35M patents from the EPO
1.35K English patent topics
Collection contains EN/FR/DE patents, with translationsof titles and claims in three languages
Expand query by: PRF vs. WordNet
Use (Magdy et al., 2011) as BL without citation extraction(full patent description section as query)
MAP and PRES was used for evaluationBL: 0.14 MAP, 0.486 PRES
Using WordNet for Expansion
Expand terms in query using synonyms, hyponyms fornouns and verbs
Apply QE to sample 100 topics, then use bestcombination to the full 1.35k topics set
MAP
PRES
value
%change
value
%change
Baseline
0.1668
NA
0.584
NA
NS
0.1680
+0.7%
0.562
-3.7%
NS+NH
0.1680
+0.7%
0.561
-3.8%
NS+VS
0.1677
+0.5%
0.551
-5.6%
NS+NH+VS+VH
0.1540
-7.6%
0.544
-6.8%
Baseline
0.1399
NA
0.486
NA
WordNet (NS)
0.1364
-2.5%
0.484
-1.0%
Standard QE Approaches
PRF:
Significant degradation in retrieval effectiveness.
This can be expected due to the low initial retrieval precision
WordNet:
Statistically significant degradation of results, but with somesuccessful instances (31% of topics)
Large reduction in retrieval speed, since average query size isat least 5 times larger (34 times larger for the NS+NH+VS+VH)
A new effective and efficient QE method is required!
Automatically Generated SynSet
Align Sentences
Remove Stopwords
Stem Words
Align Terms
Backoff Alignment
Englishfields
Frenchtransl.
ENFRtermsdic.
FRENtermsdic.
ENENtermsdic.
process for eliminating foreign matter from a wasteheat stream
procédé pour éliminer de la matière étrangère d'uncourant de chaleur perdue
process elimin foreign matter wast heat stream
procéd élimin mati étrangèr cour chaleur perdu
elimin:
élimin0.71
elimin0.13
élimin:
remov0.71
elimin0.14
elimin:
remov0.6
elimin0.16
elimin:
remov0.85
elimin0.15
Samples of the Output
motormotor
weightweight
traveltravel
colorcolor
linklink
motormotor0.64
enginengin0.36
weightweight0.86
wtwt0.14
traveltravel0.67
movemove0.19
displacdisplac0.14
colorcolor0.56
colourcolour0.25
dyedye0.19
linklink0.4
connectconnect0.18
bondbond0.17
crosslinkcrosslink0.13
bindbind0.12
clothcloth
tubetube
areaarea
gamegame
playplay
fabricfabric0.36
clothcloth0.3
garmentgarment0.2
tissutissu0.14
tubetube0.88
pipepipe0.12
areaarea0.4
zonezone0.23
regionregion0.2
surfacsurfac0.17
setset0.6
gamegame0.4
setset0.3
playplay0.24
readread0.2
gamegame0.16
reproducreproduc0.1
SynSet QE Results
8M parallel EN/FR sentences were extracted from EPOpatent collection to generate SynSets
Two runs were adopted:
Expanding query using SynSet without weights (Usynset)
Utilizing SynSet probabilities as weights to terms in query
MAP
PRES
value
%change
value
%change
Baseline
0.1399
NA
0.486
NA
Wsynset
0.1440
+2.9%
0.485
-0.7%
Usynset
0.1402
+0.2%
0.480
-1.7%
SynSet Expansion
Significantly better MAP, but significantly worse PRESi.e. better retrieval at very high ranks, but worse rankingof relevant results over all ranks and less recall
Some topics were improved (34% of topics), but somewere degraded (39% of topics).
Significantly more efficient than PRF and WordNet(query size is only 60% larger)
Deeper Look on SynSet
No features with high correlation to SynSet QE success
Initial retrieval quality of BL does not relate to theperformance of QE
Topic ID
Baseline
Wsynset
%change
Topic ID
Baseline
Wsynset
%change
PAC-1704
0.000
0.174
+∞
PAC-1510
0.030
0.012
-60%
PAC-195
0.000
0.215
+∞
PAC-210
0.160
0.000
-100%
PAC-1225
0.105
0.532
+408%
PAC-220
0.201
0.000
-100%
PAC-1670
0.124
0.637
+415%
PAC-56
0.263
0.040
-85%
PAC-954
0.514
0.763
+48%
PAC-784
0.323
0.027
-92%
PAC-122
0.590
0.944
+60%
PAC-42
0.459
0.216
-53%
PAC-579
0.630
0.902
+43%
PAC-906
0.571
0.214
-63%
PAC-1113
0.669
0.880
+32%
PAC-1498
0.662
0.307
-54%
Conclusions
PRF is not effective with patent prior-art search
WordNet QE for patent search:
Leads to overall significant degradation of retrieval
Has some positive impact on the retrieval of some topics
High computational cost
SynSet QE for patent search:
The most effective and efficient QE technique among thosetested
Significant improvement for very high ranks, but significantdegradation of overall ranking and recall
No indication of when it fails/succeeds
SynSet can be used as a lexical resource for patent examiners