Chapter 7 Experimental Research

Chapter 8Experimental Research

Overview of ExperimentalResearch

Traditional type of research

Purpose is to investigate cause-and-effectrelationships among variables

–Experimental groups vs. control groups

–Each group of participants receives a differenttreatment

–Always involves manipulation of theindependent variable

Answers the question “What will be?”

Systematic Procedures

More than any other type of research,experimental research should follow adefinite, orderly procedure

Specific steps follow . . .

Steps in Experimental Research

State the research problem

Determine if experimental methods apply

Specify the independent variable(s)

Specify the dependent variable(s)

State the tentative hypotheses

Determine measures to be used

Pause to consider potential success

Identify intervening (extraneous) variables

Formal statement of research hypotheses

Design the experiment

Final estimate of potential success

Conduct the study as planned

Analyze the collected data

Prepare a research report

Research Validity

Internal Validity – the validity of findings with theresearch study; the technical soundness of astudy, particularly concerned with the control ofextraneous influences that might effect theoutcome

External Validity – the degree to which thefindings can be inferred to the population ofinterest or to other populations or settings; thegeneralizability of the results

Both are important in a study but they arefrequently at odds with one another in planningand designing a study

Internal validity is considered the basic minimumfor experimental research

Internal Validity

. . . this is the basic minimum without which anystudy is not interpretable

Particularly important in experimental studies

Did, in fact, the experimental treatment (X)produce a change in the dependent variable (Y)

–To answer yes, one must be able to rule out thepossibility of other factors producing the change

To gain internal validity, the researcher attemptsto control everything and eliminate possibleextraneous influences

Lends itself to highly controlled, laboratorysettings

Threats to Internal Validity

History – events occurring during the experimentthat are not part of the treatment

Maturation – biological or psychologicalprocesses within participants that may changedue to the passing of time, e.g., aging, fatigue,hunger

Testing – the effects of one test upon subsequentadministrations of the same test

Instrumentation – changes in testing instruments,raters, or interviewers including lack ofagreement within and between observers

Threats continued

Statistical regression – the fact that groupsselected on the basis of extreme scores are notas extreme on subsequent testing

Selection bias – identification of comparisongroups in other than a random manner

Experimental mortality – loss of participants fromcomparison groups due to nonrandom reasons

Interaction among factors – factors can operatetogether to influence experimental results

External Validity

Generalizability of results . . . to whatpopulations, settings, or treatment variables canthe results be generalized?

Concerned with real-world applications

What relevance do the findings have beyond theconfines of the experiment?

External validity is generally controlled byselecting subjects, treatments, experimentalsituations, and tests to be representative of somelarger population

Random selection is the key to controlling mostthreats to external validity

Types of External Validity

Population Validity –

–refers to the extent to which the results can begeneralized from the experimental sample to adefined population

Ecological Validity –

–refers to the extent to which the results of anexperiment can be generalized from the set ofenvironmental conditions in the experiment toother environmental conditions

Threats to External Validity

Interaction effects of testing – the fact that thepretest may make the participants more aware ofor sensitive to the upcoming treatment

Selection bias – when participants are selected ina manner so they are not representative of anyparticular population

Reactive effects of experimental setting – the factthat treatments in constrained laboratory settingsmay not be effective in less constrained, real-world settings

Multiple-treatment interference – whenparticipants receive more than one treatment, theeffects of previous treatments may influencesubsequent ones

Types of Designs

The basic structure of a research study . . .particularly relevant to experimentalresearch

Types of designs (Campbell & Stanley, 1963)

–Pre-experimental

–True experimental

–Quasi-experimental

Pre-experimental designs

Weak experimental designs in terms ofcontrol

No random sampling

Threats to internal and external validityare significant problems

Many definite weaknesses

Example: One-group pretest/posttestdesign

True experimental designs

Best type of research design because oftheir ability to control threats to internalvalidity

Utilizes random selection of participantsand random assignment to groups

Example: Pretest/posttest control groupdesign

Quasi-experimentaldesigns

These designs lack either randomselection of participants or randomassignment to groups

They lack some of the control of trueexperimental designs, but are generallyconsidered to be fine

Example: Nonequivalent group design

Methods of Control

Physical manipulation

Selective manipulation

–Matched pairs and block designs

–Counterbalanced designs

Statistical techniques

Physical Manipulation

Best way to control extraneous variables

Researcher attempts to control all aspectsof the research, except the experimentaltreatment

Difficult to control all variables

–Some variables cannot be physicallycontrolled

Selective Manipulation

Intent is to increase likelihood thattreatment groups are similar at thebeginning of study

Matched pairs design

–Participants are matched according to somekey variable and then randomly assigned totreatment group

–Block design – extension of matched pairs to3 or more groups

Counterbalanced design

–All participants receive all treatments, but indifferent orders

Statistical Techniques

Applied when physical manipulation orselective manipulation is not possible

Differences among treatment groups areknown to exist at beginning of study

–Groups may differ on initial ability

Analysis of covariance (ANCOVA)

–Adjusts scores at the end of the study basedupon initial differences

Common Sources of Error

Many possible sources of error can causethe results of a research study to beincorrectly interpreted. The followingsources of error are more specific threatsto the validity of a study than thosedescribed previously

Selected examples:

–Hawthorne Effect

–Placebo Effect

–John Henry Effect

–Rating Effect

–Experimenter Bias Effect

Hawthorne Effect

A specific type of reactive effect in whichmerely being a research participant in aninvestigation may affect behavior

Suggests that, as much as possible,participants should be unaware they are inan experiment and unaware of thehypothesized outcome

Placebo Effect

Participants may believe that theexperimental treatment is supposed tochange them, so they respond to thetreatment with a change in performance

John Henry Effect

A threat to internal validity whereinresearch participants in the control grouptry harder just because they are in thecontrol group

Rating Effect

Variety of errors associated with ratings ofa participant or group

–Halo effect

–Overrater error

–Underrater error

–Central tendency error

Experimenter Bias Effect

The intentional or unintentional influencethat an experimenter (researcher) mayexert on a study