Social Research Methods: Qualitative andQuantitative Approaches, 5e
•This multimedia product and its contents are protected under copyright law. The following are prohibited by law: any public performance ordisplay, including transmission of any image over a network; preparation of any derivative work, including the extraction, in whole or in part,of any images; any rental, lease, or lending of the program
Chapter 7: Qualitative and QuantitativeMeasurement
•Why measure?
•Quantitative and Qualitative Measurement
•Parts of measurement process.
•Reliability and Validity.
•A Guide to Quantitative Measurement
•Index construction.
•Scales.
Introduction
•To study a concept, by itself or as part of alarger framework (theory, H, etc.), you mustbe able to measure it.
•This includes one or more of its
–Presence or absence
–Amount
–Type
Introduction, continued
•There are two key aspects of measurement
–Conceptualization: define the concept in words
–Operationalization: identify specific ways to measure it
•For example, intelligence
–Are you satisfied with IQ as a measure of intelligence?
–Does it conceptualize intelligence correctly and fully?
•Or would you include a wider range of types of intelligence – see textp. 170
–Does it operationalize abstract reasoning ability correctly andfully?
•Or is it measuring, in part, cultural knowledge and/or other concepts
•Or are there other problems with this form of operationalization?
Why Measure? Scientific measurement
•Provides clarity (conceptualization) and precision(operationalization)
•Extends our senses
•Enables us to see the invisible
–Physical science
–Social science
Chapter 7: Qualitative and QuantitativeMeasurement
•Why measure?
•Qualitative and Quantitative Measurement
•Parts of measurement process.
•Reliability and Validity.
•A Guide to Quantitative Measurement
•Index construction.
•Scales.
Qualitative and Quantitative Measurement
•Both gather data systematically andcarefully
•Quantitative: conceptualization andoperationalization precede data collection
•Qualitative: conceptualization andoperationalization accompany datacollection
Chapter 7: Qualitative and QuantitativeMeasurement
•Why measure?
•Quantitative and Qualitative Measurement
•Parts of the measurement process.
•Reliability and Validity.
•A Guide to Quantitative Measurement
•Index construction.
•Scales.
Conceptualization
•Process of clearly defining a concept
•Think, observe, consult, read, try out
•Needs to be clear, unambiguous
•Typically are linked to theories, often to values
–Social class – relational or gradational
–Democracy – political only, or economic also
–Intelligence – single, multiple, individual vs. group
•Concepts vary from concrete to abstract
Quantitative Conceptualization andOperationalization
•A deductive process
•See figure 7.1, p. 175
Measurement deals with one concept at a timeHypotheses deal with more than one – measure themseparately (you measure concepts, not Hs)
Example:
H: Class size affects teacher morale
| conceptualization |
[Conceptual def.] [Conceptual def.]
| operationalization |
[Operational def.] [Operational def.]
Conceptualization
•Example of teacher morale, text p 173:
•Get different definitions from differentsources
•Sort them out and narrow down. Decide ifthis is multidimensional
•Specify unit of analysis
•Distinguish from related concepts
Operationalization
•Turns conceptual definition into operationaldefinition – a specific procedure or way tomeasure a concept
–E.g. intelligence: IQ test, or multiple tests
Qualitative Conceptualization andOperationalization
•Grounded theory approach: inductive
•Reconstruction of theory: both deductive andinductive
•Conceptualization: develop conceptual definitionwhile collecting and analyzing data; graduallyincreasing clarity and specificity
•Operationalization: Description of how specificobservations [and existing concepts] contributedto development of new or revised concepts
Qualitative Conceptualization andOperationalization, continued
•Example – see figure 7.3, p 178
Chapter 7: Qualitative and QuantitativeMeasurement
•Why measure?
•Quantitative and Qualitative Measurement
•Parts of measurement process.
•Reliability and Validity.
•A Guide to Quantitative Measurement
•Index construction.
•Scales.
Reliability and Validity
•Are properties of a measure
•Both address the question: “How good isthis measure?”
•First approximation:
–Reliability = dependability
–Validity = truthfulness or accuracy
•Can you have one without the other?
Reliability and Validity in Quantitative Research
•Reliability = dependability:
–Measurements do not vary because of theprocess or the measure
•Why then would they vary?
•That which they measure actually changes
–Examples
•Scales
•Speedometers
•Social science:
Three Types of Reliability
•Stability: reliability across time
–Measure (estimate) by test-retest method
•Representative: reliability across subgroups
–Measure (estimate) by subpopulation analysis
•Equivalence: reliability across different indicators
–For long questionnaire, measure (estimate) by split-halfmethod
–Intercoder reliability
How to Improve Reliability
•Clearly conceptualize constructs
•Increase the levels of measurement
•Use multiple indicators of a variable
•Use pretests, pilot tests, replication
Validity
•(Measurement) validity is the accuracy of ameasure
•How well do the conceptual and operationaldefinitions mesh?
•Validity cannot be observed directly
–Establishing validity rests upon the accumulation ofevidence over time
Four Types of Measurement Validity
•Face validity - expert judgment, consensus
•Content validity - measure covers fullcontent of the concept
•Criterion validity – comparison withknown valid measure
–Concurrent – with existing tests
–Predictive – of future events or behavior
Four Types of Measurement Validity, continued
•Construct –consistency across differentindicators (for measures with such)
–Convergent
–Divergent
Reliability and Validity in Qualitative Research
•Reliability – goal of consistency inobservation, similar to stability reliability inquantitative research
•Balanced by recognition
–that observation is an evolving process
–of uniqueness of individual perspectives andobservations
Reliability and Validity in Qualitative Research,continued
•Validity
–Concerned with truth, although less concernedwith precise match between abstract conceptand empirical data
–Established by
•Plausibility
•Diverse empirical data
•Connections among different pieces of data
Relationship between Reliability and Validity
•Reliability is necessary but not sufficient forvalidity
–A measure can be (highly) reliable, but not (highly)valid
•a scale that always reads 10 pounds too high
•a watch that is always 20 minutes fast
–If a measure is valid, it must also be reliable
Chapter 7: Qualitative and QuantitativeMeasurement
•Why measure?
•Quantitative and Qualitative Measurement
•Parts of measurement process.
•Reliability and Validity.
•A Guide to Quantitative Measurement
•Index construction.
•Scales.
Four Levels of Measurement
•Nominal measurements are mutually exclusive andexhaustive categories.
•Ordinal measurements are mutually exclusive andexhaustive categories that are ranked.
•Interval measurements are mutually exclusive andexhaustive categories that are ranked with equal distancebetween the categories being measured.
•Ratio measurements are mutually exclusive andexhaustive categories that are ranked with equal distancebetween categories being measured, and there is a truezero.
Indices and Scales in Social Research
•An index is a measure in which a researcher addsor combines several distinct indicators orconstructs into a single score.
•A scale is a measure in which a researchercaptures the intensity, direction, level, or potencyof a variable construct, arranging responses on acontinuum.
Chapter 7: Qualitative and QuantitativeMeasurement
•Why measure?
•Quantitative and Qualitative Measurement
•Parts of measurement process.
•Reliability and Validity.
•A Guide to Quantitative Measurement
•Index construction.
•Scales.
Indices
•Develop a set of questions on the sametopic.
•Decide whether to weight each item.
•Decide how to handle missing data.
•Decide whether to use rates andstandardization.
Chapter 7: Qualitative and QuantitativeMeasurement
•Why measure?
•Quantitative and Qualitative Measurement
•Parts of measurement process.
•Reliability and Validity.
•A Guide to Quantitative Measurement
•Index construction.
•Scales.
Scales
•The Likert Scale is called a summated rating oradditive scale because a total score is computedalong a continuum such as: “I really like researchmethods,” with the choices being: SA A Unc D SD
•Thurstone Scale consists of many statements thatare rated by judges to reduce a number of items toa range that spans all opinions on the topic.
•Bogardus Social Distance involves responses thatare gathered into a series of ordered statementsfrom very threatening to least threatening.
Scales continued….
•Semantic Differential involves responses that aregathered to a set of polar opposite adjectives that create arating scale.
Bad __ __ __ __ __ Good Fast __ __ __ __ __ Slow
•The Guttman Scaling is called cumulative scaling. This isa technique researchers use after data has been collected todetermine whether a set of items can be combined to forma scale.