Airgas template

Chapter 11Enhancing Rigor in QuantitativeResearch

Types of Validity

•Statistical conclusion validity

•Internal validity

•Construct validity

•External validity

Controlling Intrinsic Source of ExtraneousVariability

•Randomization

•Crossover

•Homogeneity

•Blocking/Stratification

•Matching (pair matching)

•Statistical control

Benefits and Limitations of ControlMethods: Randomization

Benefits:

•Controls all extraneous variables

•Does not require advance knowledge of which variablesto control

Limitations:

•Ethical and practical constraints on manipulation

•Possible artificiality of conditions

Benefits and Limitations of ControlMethods: Crossover (Repeated Measures)

Benefits:

•If done with randomization, strongest possibleapproach

•Reduces sample size requirements

Limitations:

•Cannot be used if there are possible carryovereffects from one condition to another

Benefits and Limitations of ControlMethods: Homogeneity

Benefits:

•Easy to achieve

•Enhances interpretability of relationships

Limitations:

•Limits generalizability

•Requires knowledge of which variables to control

Benefits and Limitations of ControlMethods: Blocking

Benefits:

•Enhances interpretability of relationships

•Offers possibility of examining blocking variable as anindependent variable

Limitations:

•Manageable only with a few blocking variables

•Requires knowledge of which variables to control

Benefits and Limitations of ControlMethods: Matching

Benefits:

•Enhances interpretability of relationships

•Easy to do if there is a large pool of available comparisongroup subjects

Limitations:

•Manageable only with a few matching variables

•Requires knowledge of which variables to control

•May be hard to find comparison group matches

Benefits and Limitations of ControlMethods: Statistical Studies

Benefits:

•Enhances interpretability of relationships

•Easy and economical

•Can be used with a large number of extraneous variables

Limitations:

•Requires knowledge of which variables to control

•Requires statistical sophistication

Threats to Statistical Conclusion Validity

•Low statistical power

•Low precision

•Factors that undermine a strongoperationalization of the independentvariables

•Inadequate participation in treatmentconditions

Threats to Internal Validity

•Temporal ambiguity

•Selection

•History

•Maturation

•Mortality/Attrition

•Testing and Instrumentation

Research Designs and Threats toInternal Validity

Construct Validity

•Concerns inferences from the particularexemplars of a study to the higher orderconstructs that they are intended torepresent

Threats to Construct Validity

•Reactivity to the study situations

•Researcher expectancies

•Novelty effects

•Compensatory effects

•Treatment diffusion or contamination

External Validity

•Concerns inferences about the extent towhich relationships observed in a study holdtrue over variations in people, conditions,and setting, as well as over variations intreatments and outcomes

Threats to External Validity

•Interaction between relationship and people

•Interaction between causal effects andtreatment variation

Accessible vs. Target Population

•Accessible population: The population available for aparticular study

•Target population: The total group of people in whom aresearcher is interested and to whom results could begeneralized

Threats to External Population

•Inadequate sampling

•Expectancy effects

•Novelty effects

•Interaction of history and treatment effects

•Experimenter effects

•Measurement effects