Using Randomization Methods toBuild Conceptual Understanding inStatistical Inference:Day 1
Lock, Lock, Lock, Lock, and Lock
Minicourse – Joint Mathematics Meetings
Boston, MA
January 2012
WiFi: marriotconference, password: 1134ams
Introductions:
Name
Institution
Schedule: Day 1
Wednesday, 1/4, 4:45 – 6:45 pm
1. Introductions and Overview
2. Bootstrap Confidence Intervals
• What is a bootstrap distribution?
• How do we use bootstrap distributions to buildunderstanding of confidence intervals?
• How do we assess student understanding when using thisapproach?
3. Getting Started on Randomization Tests
• What is a randomization distribution?
• How do we use randomization distributions to buildunderstanding of p-values?
• How do these methods fit with traditional methods?
4. Minute Papers
Schedule: Day 2
Friday, 1/6, 3:30 – 5:30 pm
5. More on Randomization Tests
•How do we generate randomization distributions for variousstatistical tests?
•How do we assess student understanding when using thisapproach?
6. Connecting Intervals and Tests
7. Technology Options
•Short software demonstrations (Minitab, Fathom, R, Excel, ...)
– pick two!
8. Wrap-up
•How has this worked in the classroom?
•Participant comments and questions
9. Evaluations
Why useRandomizationMethods?
These methods are greatfor teaching statistics…
(the methods tie directly to thekey ideas of statistical inferenceso help build conceptualunderstanding)
And these methods arebecoming increasinglyimportant for doingstatistics.
(Attend the Gibbs Lecture tonight!)
It is the way of the past…
"Actually, the statistician does not carry outthis very simple and very tedious process[the randomization test], but his conclusionshave no justification beyond the fact that theyagree with those which could have beenarrived at by this elementary method."
-- Sir R. A. Fisher, 1936
… and the way of the future
“... the consensus curriculum is still an unwitting prisoner ofhistory. What we teach is largely the technical machinery ofnumerical approximations based on the normal distributionand its many subsidiary cogs. This machinery was oncenecessary, because the conceptually simpler alternativebased on permutations was computationally beyond ourreach. Before computers statisticians had no choice. Thesedays we have no excuse. Randomization-based inferencemakes a direct connection between data production and thelogic of inference that deserves to be at the core of everyintroductory course.”
-- Professor George Cobb, 2007
Question
Can you use your clicker?
A. Yes
B. No
C. Not sure
D. I don’t have a clicker
Question
How frequently do you teach Intro Stat?
A. Regularly
B. Occasionally
C. Rarely/Never
Question
How familiar are you with simulationmethods such as bootstrap confidenceintervals and randomization tests?
A. Very
B. Somewhat
C. A Little / Not at all
Question
Have you used randomization methodsin any statistics class that you teach?
A. Yes, as a significant part of the course
B. Yes, as a minor part of the course
C. No
Question
Have you used randomization methodsin Intro Stat?
A. Yes, as a significant part of the course
B. Yes, as a minor part of the course
C. No
Intro Stat – Revise the Topics
• Descriptive Statistics – one and two samples
• Normal distributions
• Data production (samples/experiments)
• Sampling distributions (mean/proportion)
• Confidence intervals (means/proportions)
• Hypothesis tests (means/proportions)
• ANOVA for several means, Inference forregression, Chi-square tests
• Data production (samples/experiments)
• Bootstrap confidence intervals
• Randomization-based hypothesis tests
• Normal/sampling distributions
• Bootstrap confidence intervals
• Randomization-based hypothesis tests
It’s close to 5 pm;
We need a snack!
What proportion ofReese’s Pieces areOrange?
Find the proportion that are orange for your box.
Bootstrap
Distributions
Or: How do we get a sense of a samplingdistribution when we only have ONE sample?
Suppose we have a random sample of6 people:
Original Sample
Create a “sampling distribution” using this as oursimulated population
Bootstrap Sample: Sample withreplacement from the original sample, usingthe same sample size.
Original Sample
Bootstrap Sample
Simulated Reese’s Population
Sample from this“population”
Create a bootstrap sample by samplingwith replacement from the originalsample.
Compute the relevant statistic for thebootstrap sample.
Do this many times!! Gather thebootstrap statistics all together to forma bootstrap distribution.
OriginalSample
BootstrapSample
BootstrapSample
BootstrapSample
.
.
.
BootstrapStatistic
SampleStatistic
BootstrapStatistic
BootstrapStatistic
.
.
.
BootstrapDistribution
We need technology!
Introducing
StatKey.
Example: Atlanta Commutes
Data: The American Housing Survey (AHS) collecteddata from Atlanta in 2004.
What’s the mean commute time forworkers in metropolitan Atlanta?
Sample of n=500 Atlanta Commutes
Where might the “true” μ be?
How can we get a confidence intervalfrom a bootstrap distribution?
Method #1: Use the standard deviation of the bootstrapstatistics as a “yardstick”
Using the Bootstrap Distribution to Geta Confidence Interval – Version #1
The standard deviation of the bootstrap statisticsestimates the standard error of the sample statistic.
Quick interval estimate :
For the mean Atlanta commute time:
Using the Bootstrap Distribution to Geta Confidence Interval – Version #2
Keep 95%in middle
Chop 2.5%in each tail
Chop 2.5%in each tail
For a 95% CI, find the 2.5%-tile and 97.5%-tile inthe bootstrap distribution
95% CI=(27.35,30.96)
90% CI for Mean Atlanta Commute
Keep 90%in middle
Chop 5% ineach tail
Chop 5% ineach tail
For a 90% CI, find the 5%-tile and 95%-tile in thebootstrap distribution
90% CI=(27.64,30.65)
Bootstrap Confidence Intervals
Version 1 (Statistic 2 SE):
Great preparation for moving to
traditional methods
Version 2 (Percentiles):
Great at building understanding of
confidence intervals
Playing withStatKey!
See the purple pages in the folder.
We want to collect somedata from you. What shouldwe ask you for our onequantitative question andour one categoricalquestion?
What quantitative data should we collectfrom you?
A.What was the class size of the Intro Stat course youtaught most recently?
B.How many years have you been teaching Intro Stat?
C.What was the travel time, in hours, for your trip toBoston for JMM?
D.Including this one, how many times have you attendedthe January JMM?
E.???
What categorical data should we collectfrom you?
A.Did you fly or drive to these meetings?
B.Have you attended any previous JMM meetings?
C.Have you ever attended a JSM meeting?
D.???
E.???
How do we assessstudent understandingof these methods
(even on in-class examswithout computers)?
See the green pages in the folder.
• Paul the Octopus predicted 8 World Cupgames, and predicted them all correctly
• Is this evidence that Paul actually has psychicpowers?
• How unusual would this be if he were justrandomly guessing (with a 50% chance ofguessing correctly)?
• How could we figure this out?
Paul the Octopus
• Each coin flip = a guess between two teams
• Heads = correct, Tails = incorrect
• Flip a coin 8 times and count the number ofheads. Remember this number!
Did you get all 8 heads?
(a) Yes
(b) No
Simulate!
Let p denote the proportion of games thatPaul guesses correctly (of all games he mayhave predicted)
H0 : p = 1/2
Ha : p > 1/2
Hypotheses
• A randomization distribution is thedistribution of sample statistics we wouldobserve, just by random chance, if the nullhypothesis were true
• A randomization distribution is createdby simulating many samples, assuming H0is true, and calculating the sample statisticeach time
Randomization Distribution
• Let’s create a randomization distributionfor Paul the Octopus!
• On a piece of paper, set up an axis for adotplot, going from 0 to 8
• Create a randomization distributionusing each other’s simulated statistics
• For more simulations, we use StatKey
Randomization Distribution
• The p-value is the probability of gettinga statistic as extreme (or more extreme)as that observed, just by random chance,if the null hypothesis is true
• This can be calculated directly from therandomization distribution!
p-value
StatKey
• Create a randomization distribution bysimulating assuming the null hypothesis is true
• The p-value is the proportion of simulatedstatistics as extreme as the original samplestatistic
Randomization Test
• How do we create randomizationdistributions for other parameters?
• How do we assess studentunderstanding?
• Connecting intervals and tests
• Technology for using simulation methods
• Experiences in the classroom
Coming Attractions - Friday