CALIFORNIA STATE UNIVERSITY LONG BEACH
PPA 696--RESEARCH METHODS:
BINGHAM & FELBINGER CH. 8
  1. BIBLIOGRAPHIC INFORMATION
    1. Author: D. L. Thistlethwaite & D. T. Campbell
    1. Title: Regression-Discontinuity Analysis--An Alternative to the Ex-Post-Facto Experiment
    1. Source: Journal of Educational Psychology, 51(6), 1960

      2.  
  1. SUMMARY OF THE RESEARCH
    1. PROBLEM STATEMENT:
What is the effect on students of receiving a distinction, compared to students who did not receive the distinction?
    1. BACKGROUND:
The authors want to be able to distinguish between two groups when they have only after-the-fact data; they introduce a new statistical technique to demonstrate its superiority to other methods.
    1. HYPOTHESIS:
Students receiving runner-up awards will 1) have more favorable attitudes toward intellectualism; 2) more often plan to get advanced degrees; 3) more often plan to become teachers or researchers; and 4) more often receive other scholarships.
    1. MEASUREMENT OF VARIABLES
      1. Dependent variable: 1) attitude toward intellectualism; 2) plans to get advanced degrees; 3) plans to become teachers or researchers; and 4) number of other scholarships received
      1. Independent variable(s): Certificate of Merit award
      1. Control variable(s): n/a
    1. RESEARCH DESIGN:
A quasi-experimental design that looks at outcomes after the fact only; introduces the regression-discontinuity design.
Group Certificate 

of Merit

 Post-test 

Measures
G-1 X O1
G-2 O1
 
    1. SAMPLING:
6,428 Certificate of Merit awardees; 3,588 non-awardees (entire universe of eligibles)
    1. INSTRUMENTATION:
Questionnaire mailed six months after the announcement of winners
    1. DATA COLLECTION/ETHICS:
5,126 (79.7%) of awardees and 2,848 (79.4%) of non-awardees responded;
    1. DATA ANALYSIS:
Separate regression equations were performed for the awardees and for the non-awardees on each dependent variable and graphed; Mood's t-test is used to test for statistically significant difference between the two groups on the four dependent variables.
    1. CONCLUSIONS:
The results of this study contradict the findings of earlier analyses, which considered all the students together in one group. This study found only that receiving the award increased the chances of winning other scholarships; but had no effect on attitudes toward intellectualism, attending graduate school, or becoming a teacher/researcher.
  1. CRITIQUE
    1. Possible Threats to Internal Validity
      1. History:
n/a; only one measurement at one point in time is taken
      1. Maturation:
n/a; only one measurement at one point in time is taken
      1. Testing:
only one questionnaire was answered (no pre-test)
      1. Instrumentation:
questionnaire was presumably valid and reliable;
      1. Regression Artifact:
the study tests for regression artifacts by using the regression-discontinuity design
      1. Selection bias:
random assignment was not possible but this design eliminates the need for matching; some self-selection bias possible among those who returned the questionnaire versus those who did not;
      1. Experimental Mortality:
n/a; data only measured at one point in time
      1. Design contamination:
n/a; no "treatment" applied by the researcher
    1. Possible Threats to External Validity
      1. unique program features:
 
      1. experimental arrangements:
there must be a clear distinction between the awardees and the non-awardees based on a quantifiable criterion; all variables must be measured on an interval level or higher
      1. other threats: