Usability Issues in MetasearchInterface Design: persectivesof an information provider
LITA Human Machine Interface Interest Group
June 25, 2004Oliver PeschChief StrategistEBSCO Information Servicesopesch@ebsco.com
Overview
Customer service or denial of service
Protecting copyright
Being included in results/removal ofduplicates
The usage statistics challenge
Investing in products
The hope for standards
Metasearch casts a wide net
Search widely, bring back results and look for themost relevant ones and present to the user
For the information provider
Searches are less targeted
System load increases by multiples
Serendipitous discoveries of relevant content
For the end user
Relevant sources searched automatically
Serendipitous discoveries of relevant content
Can work like Google
Can work like Google
Denial of service attacks?
A denial of service attack when a web-site orseries of web sites intentionally or unintentionallysends an extremely high volume of requests to aweb-based service such that the service isspends all its resources responding to theserequests and cannot accommodate requests ofnormal users.
Searching without Metasearch
user
Resources
-Product 1-Product 2-Product 3…
EBSCO
OCLC
With Metasearch engine
EBSCO
OCLC
user
Metasearch
-Search All-Business-Medicine…
Being included in results
Metasearch engine (as configured by library)decides what gets searched and how results aredisplayed
Are the most relevant items always presented first?
One approach to determine position is speed ofresponse. A good measure of relevance?
Another is to attempt to score the results
Consistent ranking of relevance is not easy
The risk of not being included is real
Relevance calculations…
Typical calculation considers
Number of words from search appearing indocument
(higher number higher relevance)
The number of times these words appear
(higher number higher relevance)
The size of the document
(longer document less relevant)
The number of times the words appear in thedatabase
(higher number less relevance)
Relevance calculations…
Some vendors may also consider…
Proximity of search words in document
(words in closer proximity higher relevance)
The fields in which the search words appear
(words found in important fields higher relevance)
The number of forward references pointing to thedocument
(more references to document higher relevance)
Usage
(the more times others read document higher relevance)
Other attributes of the item, such as peer-review
Relevance calculation…
Not one standard for calculating scores
Scores may be represented as
Percentages, or
Raw scores
Not all information providers return the scores
When they do, the scores from differentinformation providers cannot be considerednormalized
Metasearch engine must rankresults
Metasearch cannot rely on scores returned
Data not available for optimized calculation
Option is to retrieve metadata and possiblydocuments from information provider tocalculate
Impact on information provider:
More system overhead from increased retrievals
Usage statistics can be skewed
Deduping results
When the same article is found in more thanone sources
Which one is picked?
Are all sources attributed?
Who/what influences the decision?
Abiding by content agreements
Information provider is responsible forupholding agreements with copyright holders
Requirements for display may include
Copyright statements
Specific language
Enforcing specific restrictions
Integrity of presentation
Copyright holders may question display oftheir content through another interface
Usage statistics challenge
What can be affected by metasearch?
Session counts
Search counts
Article retrieval counts
Why?
“Search all” option or automatic selection/search of manyresources
Perform simultaneous activities
Pre-fetching data for purpose of ranking
Optimization techniques
Searching without Metasearch
EBSCO
OCLC
ProQuest
OVID
user
Resources
-Product 1-Product 2-Product 3…
Visits = 1Sessions = 2Searches = 2
Visits = 1Sessions = 2Searches = 2
With Metasearch engine
EBSCO
OCLC
ProQuest
OVID
user
Metasearch
-Search All-Business-Medicine…
Visits = 1Sessions = 20Searches = 20
Visits = 1Sessions = 20Searches = 20
Visits = 1Sessions = 20Searches = 20
Visits = 1Sessions = 20Searches = 20
With Metasearch engine
EBSCO
OCLC
ProQuest
OVID
user
Metasearch
-Search All-Business-Medicine…
Visits = 1Sessions = 28Searches = 28
Visits = 1Sessions = 28Searches = 28
Usage statistics challenge
Metasearch engines can cause inflation ofsession, search and possibly article retrievalstatistics
Not all activity for a given customr will bethrough a metasearch engine
Unless metasearch activity can be isolated,overall statistics lose meaning
Investing for differentiation
Information providers invest to differentiate theirproduct by
Enhancing the data
Enhancing the interface to improve precision
Metasearch access often results in a lowestcommon denominator approach
Enhancements to data and interface have littleeffect
Do these investments help metasearch users?
Impact on information provider
Institutions pay money for information
One measure of effectiveness is usage
Results must be shown to be accessed
Investments to enhance data or access toolsmay not be cost effective
Agreements with copyright holders may bechallenged
Loss of control of the user experience is areal concern
Adapting to change
Products are developed based on marketneeds
If the market needs metasearch, we willdevelop our products meet that need
Standards initiatives become vitally importantto allow information providers to regain somecontrol
Importance of standards
Optimization of computing resources
Isolate Metasearch traffic
Minimize work to present data
Tailor responses to metasearch needs
Provide metadata for ranking and deduping
Provide copyright information
Include URLs to facilitate seamless access tocontent
Provide appropriate “viewer” for content
More accurately represent usage statistics
Thank youopesch@ebsco.com