Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Evaluation in the WesternAustralian Ombudsman’s OfficeEvaluation in the WesternAustralian Ombudsman’s Office
Kim Lazenby & Jane Burn
Ombudsman Western Australia
Australasian Evaluation Society
2011 International Conference
Sydney, Australia
1 September 2011
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Overview
•Introduction to the Western AustralianOmbudsman
•Exploration of the theme of the conferenceEvaluation and Influence, focussing on:
–Making effective use of evaluation techniques inwork of the Ombudsman’s office
–Plus, some thinking about making the results ofevaluations more influential
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Introduction to theWestern Australian Ombudsman
•Officer of Parliament
•Independent of government of the day
•First and foremost, role is to receive,investigate and resolve complaints fromcommunity members about theadministration, or alleged maladministration,by government agencies in Western Australia
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Introduction to theWestern Australian Ombudsman
•Jurisdiction includes Western Australiangovernment departments, statutory authorities, localgovernment authorities, and public universitiesoperating in Western Australia
•In 2010-2011, Western Australian Ombudsman’soffice dealt with more than 2000 complaints
•Ombudsman is part of the arrangements foroversighting the work of government agencies -heightens accountability of government agencies tocitizens
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Introduction to theWestern Australian Ombudsman
•4 main functions:
–Investigate and resolve complaints about publicadministration
–Improve the standard of public administrationover time
–Review investigable child deaths
–Inspectorate functions, and specialistinvestigations
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Improving the standard of publicadministration
•Generally occurs by identifying commonsources of complaints, analysing underlyingcauses of these complaints, and assistingagencies address these causes
•Commonly undertaken as a major project (oran ‘own motion investigation’) focussing onone area of public administration
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Improving the standard of publicadministration
•2 recent major projects (or ‘own motioninvestigations’):
–2009-10 Survey of Complaint Handling Practicesin the Western Australian State and LocalGovernment Sectors
–The Management of Personal Information – GoodPractice and Opportunities for Improvement
•Reports available atwww.ombudsman.wa.gov.au
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Reviewing investigable child deaths
•Ombudsman reviews investigable child deathsto:
–Review the circumstances in which and why thedeaths occurred (undertaken by Child DeathReview team)
–Identify any patterns or trends in relation to thedeaths
–Make recommendations to any department orauthority about ways to prevent or reduceinvestigable deaths
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
•Through analysis of patterns and trends ininvestigable child deaths, 2 topics for furtherinvestigation were identified
•2 major projects (or ‘own motion investigations’),which focus on these topics are currently underway:
–Investigation into the care planning process for children inthe care of the Chief Executive Officer of the Department forChild Protection
–Investigation into the prevalence, risk factors, and currentpreventative strategies for sleep related infant deaths
Reviewing investigable child deaths
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Making effective use of evaluationtechniques
•Characteristics of majorprojects:
–Detailed project proposals,based on preliminaryresearch to assist in planningthe scope and objectives ofthe investigation
–Approved by CorporateExecutive
–Once approved, detailedproject plan also developedfor project management andgovernance purposes
•Techniques applied:
–Evaluation plans, defining keyevaluation questions
–Also, techniques drawn fromproject managementdiscipline
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Making effective use of evaluationtechniques
•Characteristics of majorprojects:
–Projects are evidence-based
–Preliminary research phaseincludes research into goodpractice, latest developments,approaches by otherjurisdictions etc
–Fieldwork to collect primaryand secondary evidence
•Techniques applied:
–Evaluation criteria
–Monitoring and evaluation
–Indicators
–Program logic
–Research methods, includinginterview, document review,survey
–Audit methods, includingaudit testing, interrogation ofdata already held by agencies
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Making effective use of evaluationtechniques
•Characteristics of majorprojects:
–Analysis phase, includingtesting evidence againstrelevant legislativerequirements, approvedpolicy and procedures
–Use of working papers forquality and accountability
–Quality assurance
•Techniques applied:
–Data analysis:
•Statistical
•Qualitative
•Financial
–Compliance audit
–Working paper and qualityassurance techniques drawnfrom audit discipline
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Making effective use of evaluationtechniques
•Characteristics of majorprojects:
–Projects result inrecommendations forimproving publicadministration
–Recommendations are:
•evidence-based
•proportionate
•practical
•consider costs and benefitsof implementing
•Techniques applied:
–Evaluation as researchfor decision-makingpurposes
–Utilisation-focussedevaluation
–Policy developmenttheory and practice
–Cost-benefit analysis
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Making effective use of evaluationtechniques
•Characteristics of majorprojects:
–‘No surprises’ approach
–Natural justice principles
–Procedural fairnessprocess
•Techniques applied:
–Evaluation’s attention toethical issues
–Utilisation-focussedevaluation
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Increasing the influence of evaluation
•Topic selection - topics for major projects are selected on thebasis of a number of criteria that include:
–the number of complaints received by the Ombudsman
–the likely public interest in the identified issue of concern
–the number of people likely to be affected
–whether other reviews of the issue have been done recently or are inprogress
–the potential for our investigation to improve administration across thepublic sector
–whether investigation of the chosen topic is the best and mostefficient use of our resources
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Increasing the influence of evaluation
•Evidence-based
•Relationship building and ‘no surprises’approach
•Dissemination of results andrecommendations
Ombudsman Western Australia
Serving Parliament – Serving Western Australians
Summary
•Western Australian Ombudsman
•Office’s major reviews and investigations
•Incorporation of evaluation techniques, andcombination with techniques from other disciplines
•Ensure our major projects help improve publicadministration in ways that are:
–Evidence-based
–Proportionate
–Practical
–Based on consideration of costs and benefits