Power Point Slides by Ronald J. Shope in collaboration with John W. Creswell
Chapter 13
Survey Designs
Educational Research 2e: Creswell
What is survey research?
•Survey research designs areprocedures in quantitative research inwhich investigators administer a surveyto a sample or to the entire populationof people in order to describe theattitudes, opinions, behaviors, orcharacteristics of the population.
Educational Research 2e: Creswell
When do you use a survey?
•Assess trends
•Opinions, beliefs, and attitudes
•Follow-up analyses
•Evaluations
Educational Research 2e: Creswell
What are the types of surveydesigns?
Changes
in a
sub-population
group identified
by a common
characteristic
over time
Time of Data Collection
Study Over Time
Study at One Point in Time
Longitudinal
Cross-sectional
Changes
in the
same
people
over time
Trends in
the same
population
over time
Attitudes
and
Practices
Community
Needs
Program
Evaluation
Trend
Cohort
Panel
Group
Comparisons
National
Assessment
Educational Research 2e: Creswell
What are key characteristicsof survey research?
•Sampling from a population
•Collecting data through questionnairesor interviews
•Designing instruments for datacollection
•Obtaining a high response rate
Educational Research 2e: Creswell
Population, Target Population,Sampling Frame, Sample
The Population
The group of individuals having one characteristicthat distinguishes them from other groups.
The Target Population or Sampling Frame
The actual list of sampling units from which thesample is selected.
The Sample
The group of participants in a study selected
from the target population from which the
researcher generalizes to the target population.
Educational Research 2e: Creswell
Reducing error in surveyresearch
•Reducing Coverage Error: have a goodsampling frame list on which to selectindividuals
•Reducing Sampling Error: select as large asample from the population as possible
•Reducing Measurement Error: use a goodinstrument with clear, unambiguous questionsand response options
•Reducing non-response error: use rigorousadministration procedures to achieve as largea return rate as possible
Educational Research 2e: Creswell
Forms of data collection based on whocompletes or records the data
Who Completes or Records the Data?
Participant
Researcher
Mailed
Questionnaire
Electronic
Questionnaire
One on
One
To a
Group
Over
Telephone
Individual
Interview
Focus Group
Interview
Telephone
Interview
Educational Research 2e: Creswell
Aspects of survey instrumentdesign
•Personal
•Attitudinal
•Behavioral
•Sensitive
•Scale
•Open-ended
Educational Research 2e: Creswell
Question constructionproblems
•Question is unclear
•Multiple questions
•Question is wordy
•Question is negatively worded
•Overlapping responses
•Unbalanced response options
•Mismatch between questions and answers
•Question includes overly technical language
•Not all questions are applicable to all theparticipants
Educational Research 2e: Creswell
Pilot testing
•Test on a small number of individuals inthe sample
•Ask for written feedback on thequestions
•Revise the survey based on the writtencomments
•Exclude the pilot participants from thefinal sample for the study
Educational Research 2e: Creswell
Response rate
•Mail out original questionnaire withcover letter
•Follow it in two weeks with a secondmailing of questionnaire to those whohave not responded
•Follow again in two weeks with apostcard to non-respondents remindingthem to complete the questionnaire
Educational Research 2e: Creswell
Three phase surveyadministration procedure
Step 1:
First Mailing
of Survey
Step 2:
Second Mailing
of Survey
Step 3:
Postcard
Mailing
2 Weeks
2 Weeks
2 Weeks
Start
6 Weeks
Time
Educational Research 2e: Creswell
Planning and conducting amailed questionnaire survey
•Write a cover letter to invite theparticipants to complete thequestionnaire
•Form and construct the questionnaire
•Identify what statistical procedures willbe used to analyze data from the mailedquestionnaire
Educational Research 2e: Creswell
Planning and designing aninterview survey
•The interviewer should maintain aneutral stance during the interview
•Train the interviewers prior to theinterview
•Take good notes of responses or use atape recorder
•For telephone interviews, develop atelephone interview guide prior to theinterview
Educational Research 2e: Creswell
Steps in conducting surveyresearch
•Decide if a survey is the best deign to use
•Identify the research questions or hypotheses
•Identify the population, the sampling frame andthe sample
•Determine the survey design and data collectionprocedures
•Develop or locate an instrument
•Administer the instrument
•Analyze the data to address the researchquestions or hypotheses
•Write the report
Educational Research 2e: Creswell
Evaluating survey research
•Was the target population or sampling frame clearlyspecified?
•Was the sampling procedure specified? If a randomsample was not used, were modifications from itexplained?
•Was the sample clearly identified and the basis onwhich it was chosen specified?
•Did the type of survey match the questions orhypotheses advanced by the author?
•Was it clear whether a questionnaire or an interviewsurvey comprised the form of data collection? Did theresearcher identify the basis for selecting ordeveloping the instrument?
Educational Research 2e: Creswell
Evaluating survey research
•Was information reported on reliability and validity ofscores from past use of the questionnaire orinterview?
•Did the author mention the dates on which thequestionnaire or interview was administered?
•Were the administration procedures clearlyidentified?
•If a questionnaire was used, were the procedures forobtaining responses identified? Were follow-upprocedures used to ensure a high response rate?
•Did the data analysis match the research questionsor hypothesis to be answered in the study?
Educational Research 2e: Creswell
Evaluating survey research
•If an interview was used, were the fieldprocedures for collecting this informationspecified? Were demographic characteristics,previous experience, training, and monitoringdiscussed for the interviewer(s)?
•Did the researcher check on the reliability andvalidity of scores from the data collection in thestudy?
•Was the study written scientifically andethically?