Looking for the link between library usage andstudent attainment
Graham Stone
Information Resources Manager
CILIPS Conference
Imagining the future
7-8 June 2011 Glasgow
http://eprints.hud.ac.uk/10655/
#lidp
#jiscad
2
Non/Low Use Projectdigging deeper into data
Measuring Library Impact2008/9 honours graduates
Results
•Analysis of the results consistently revealed a correlationbetween e-resource use, book borrowing and studentattainment
•This appears to be the case across all disciplines
•Not a cause and effect relationship
•Never proved statistically significant
•Potential for collaboration on future projects
JISC Activity Data Call
•Obtained fundingfrom the JISC ActivityData Call
•6 month project (Feb-Jul 2011)
Library Impact Data Project
To prove the hypothesis that…
“There is a statistically significant correlation across anumber of universities between library activity dataand student attainment”
Data requirements
•For each student who graduated in a given year, thefollowing data was required:
–Final grade achieved
–Number of books borrowed
–Number of times e-resources were accessed
–Number of times each student entered the library, e.g. via aturnstile system that requires identity card access
–School/Faculty
Legal issues
•Consultation with JISC Legal, University legal officer anddata protection officer
•Ensured that any identifying information is excludedbefore it is handled for analysis
•Excluded any small courses to prevent identification ofindividuals e.g. where a course has less than 35students and/or fewer than 5 of a specific degree level
Using OpenURL Activity Data
“When you search for and/or access bibliographicresources such as journal articles…[the university]…captures and anonymises activity data which are thenincluded in an aggregation of data about use ofbibliographic resources… The aggregation is used as thebasis of services for users in UK HE and is made availableto the public so that others may use it as the basis ofservices. The aggregation contains no information thatcould identify you as an individual.”
Data issues
•Anticipated that there may be problems in gettingenough data to make the project viable
–Potential partners were asked to confirm that they could provideat least 2 of the 3 measures of usage as well as student grades
–Huddersfield has provided definitions on the data required andthe form the data can be accepted in
•Some partners have already run into some issues withdata collection, but it is felt that there is still enoughinformation to prove the hypothesis one way or another
Testing the hypothesis
•Due to the data not being continuous, a correlationcannot be calculated!
•But…
Further statistical tests (1)
•Running a Kruskal-Wallistest
–to indicate whether there isa difference betweenvalues e.g. between levelsof e-resource usage acrossdegree results
–THEN we analyse the datavisually to check whichvariables to compare
Further statistical tests (2)
•Running a the Mann-Whitney U test to see whether thereis a significant difference between variables tested
•Initial findings imply that there is a relationship betweenusage and attainment
•And that these findings concur with previous tests usingANOVA and the Student T test
What we think we can prove
•That the relationship and variance means that you canbelieve what you see
•And you can believe it across a range of data, e.g.subjects
•So library usage does impact on student attainment
•Not a cause and effect relationship
16
Library Impact Data Projectbook loans inc. renewals (2009/10)
17
Library Impact Data Projectbook loans & Athens (2009/10)
18
Library Impact Data Projectlibrary PC logins & visits (2009/10)
All we set out to do wasprove the hypothesis
•At this early stage, for books and e-resource usage,there appears to be a statistical significance across allpartner libraries
•If we know that there is a link between usage andattainment
–We can link this back to low/no usage
20
Measuring Library Impact2008/9 – library visits
15.5% of students who gaineda 1st never visited the library
34% of students who gaineda 3rd never visited the library
21
Measuring Library Impact2008/9 – MetaLib usage
70% of those who gained a3rd logged in to e-resources20 times or less over 3 years
10.5% of students who gained a1st logged in more than 180 times
22
Measuring Library Impact2008/9 – book loans
15% of students who gaineda 1st never borrowed a book
34% of students who gaineda 3rd never borrowed a book
Book issues by final degree result
Next steps for the project
•Finish statistical testing
•Pull out any themes from the focus groups
•Release the data on an Open Data Commons Licence
•Release a toolkit to help others benchmark their data
•Do cuts to the information budget mean that attainmentwill fall?
•Can we add more value by better use of resources?
–By analysing the data in conjunction with UCAS tariff points
Better use leads to better attainment
Acknowledgements
•Dave Pattern and Bryony Ramsden
•Phil Adams, Leo Appleton, Iain Baird, Polly Dawes,Regina Ferguson, Pia Krogh, Marie Letzgus, DominicMarsh, Habby Matharoo, Kate Newell, Sarah Robbins,Paul Stainthorp