PowerPoint Presentation

Chapter 19

Inadequate Designs andDesign Criteria

Experimental and NonexperimentalApproaches

•An experiment is a scientific investigationin which an investigator manipulates andcontrols one or more independentvariables and observes the dependentvariable or variables for variationconcomitant to the manipulation of theindependent variables.

•An experimental design, then, is one inwhich the investigator manipulates at leastone independent variable.

Experimental and NonexperimentalApproaches

•In nonexperimental research one cannotmanipulate variables or assign participants ortreatments at random because the nature of thevariables is such as to preclude manipulation.

•Although experimental and nonexperimentalresearch differ in these crucial respects, they sharestructural and design features that will be pointedout in this and subsequent chapters. In addition,their basic purpose is the same: to study relationsamong phenomena. Their scientific logic is thesame: to bring empirical evidence to bear onconditional statements of the form If p, then q.

Experimental and NonexperimentalApproaches

•The unique and overwhelmingly importantvirtue of experimental inquiry, then, iscontrol. In a perfectly controlledexperimental study, the experimenter canbe confident that the manipulation of theindependent variable affected thedependent variable and nothing else.

Symbolism and Definitions

•(X) is a nonmanipulable variable and ~X isa manipulable variable that is notmanipulated.

•The symbol (R) will be used for therandom assignment of participants toexperimental groups and the randomassignment of experimental treatments toexperimental groups.

Symbolism and Definitions

•If X is encased inside parentheses (X), this meansthat the investigator “imagines” the manipulation ofX, or assumes that X occurred and that it is the X ofthe hypothesis. It may also mean that X ismeasured and not manipulated.

•Suppose a sociologist is studying delinquency andthe frustration-aggression hypothesis. Thesociologist observes delinquency, Y, and imaginesthat the delinquent participants were frustrated intheir earlier years, or (X). All nonexperimentaldesigns will have (X). Generally, then, (X)represents an independent variable not under theexperimental control of the investigator.

Faulty Designs

•Design 19.1: One Group (p.468)

•Scientifically, Design 19.1 is worthless. There isvirtually no control of other possible influenceson outcome.

•Design 19.2: One Group, Before-After (Pretest,Posttest) (p.469)

•The essential characteristic of this mode ofresearch is that a group is compared to itself.

•There are a number of other factors that mayhave contributed to the change in dependentvariable scores.

Measurement, History, Maturation

•Can it be that the post-X measures wereinfluenced not by the manipulation of X butby increased sensitization due to the pretest?Campbell (1957) calls such measuresreactive measures, because they themselvescause the subject to react.

•The longer the period of time, the greater thechance of extraneous variable affecting theparticipants, and thus the Ya measures. Thisis what Campbell (1957) calls history. Thesevariables or events are specific to theparticular experimental situation.

Measurement, History, Maturation

•Maturation, on the other hand, coversevents that are general—not specific toany particular situation. They reflectchange or growth in the organism studied.

The Regression Effect

•If low-scoring participants are used in astudy, their scores on the posttest willprobably be higher than on the pretest dueto chance, and similarly with some lowscores. On the posttest it is unlikely thatthe high scores will be maintained,because the factors that made them highwere chance factors—which areuncorrelated on the pretest and posttest.Thus the high scorer will tend to drop onthe posttest.

The Regression Effect

•If there was a control group, then one could“control” the regression effect, since bothexperimental manipulation has had a “real” effect,then it should be apparent over and above theregression effect.

•Design 19.2 is inadequate, not so much becauseextraneous variables and the regression effect canoperate (the extraneous variables operatewhenever there is a time interval between pretestand posttest), but because we do not knowwhether they have operated, whether they haveaffected the dependent variable measures.

Design 19.3: Simulated Before-After

•This design satisfies the condition ofhaving a control group, and is thus agesture toward the comparison that isnecessary to scientific investigation.

•Unfortunately, the controls are weak, aresult of our inability to know that twogroups were equivalent before X, theexperimental manipulation.

Design 19.4: Two Groups, No Control

•This design has a basic weakness: Thetwo groups are assumed to be equal inindependent variables other than X.

•Because randomization is not used—thatis, the participants are not assigned to thegroups at random—it is not possible toassume that the groups are equal.

Criteria of Research Design

•Answer Research Questions?

•Control of Extraneous IndependentVariables

•Generalizability

•Internal and External Validity

Answer Research Questions?

•Does the design answer the researchquestions? Or Does the design adequatelytest the hypotheses?

Control of Extraneous IndependentVariables

•Randomize whenever possible: selectparticipants at random; assign participantsto groups at random; assign experimentaltreatments to groups at random.

•Control the independent variables so thatextraneous sources of systematic variancehave minimal opportunity to operate.

Generalizability

•Can we generalize the results of a study toother participants, other groups, and otherconditions?

•In basic research, for example, generalizabilityis not the first consideration, because thecentral interest is the relations amongvariables and why the variables are related asthey are. This emphasizes the internal ratherthan the external aspects of the study. Thesestudies are often designed to examinetheoretical issues such as motivation orlearning.

Generalizability

•In applied research, on the other hand, thecentral interest forces more concern forgeneralizability, because one certainlywishes to apply the results to otherpersons and to other situations.

•Applied research studies usually havetheir foundations in basic research studies.Using information found in a basicresearch study, applied research studiesapply those findings to determine if it cansolve a practical problem.