CALIFORNIA STATE UNIVERSITY LONG BEACH
PPA 696--RESEARCH METHODS:
BINGHAM & FELBINGER CH. 6
  1. BIBLIOGRAPHIC INFORMATION
    1. Author: Tim M. Newcomb
    1. Title: Conservation Program Evaluations: The Control of Self-Selection Bias
    1. Source: Evaluation Review, 8(3), 1984:425-440

      2.  
  1. SUMMARY OF THE RESEARCH
    1. PROBLEM STATEMENT:
What are the effects of weatherization programs on energy conservation?
    1. BACKGROUND:
Weatherization programs are voluntary, so those who adopt may have different motivations than those who do not adopt, and those motivations may affect their conservation behavior.
    1. HYPOTHESIS:
Weatherization will result in lower energy consumption
    1. MEASUREMENT OF VARIABLES
      1. Dependent variable: electric use (meter readings)
      1. Independent variable(s): weatherization program
      1. Control variable(s): floor area; number of occupants; type of heat; conservation actions
    1. RESEARCH DESIGN:
Quasi-experimental design, using pre-test and post-test of experimental and comparison group (no random assignment)
Group Electricity 

Use at T1

 Weather- 

ization

 Electricity 

Use at T2
G-1 O1 X O2
G-2 O1 O2
 
    1. SAMPLING:
Volunteers for a free home energy audit and weatherization program in Seattle, WA., divided into those who had already weatherized (G-1) and those still awaiting weatherization (G-2)
    1. INSTRUMENTATION:
Audit data; survey data; electric meter readings
    1. DATA COLLECTION/ETHICS:
Mailed surveys had response rates of 81% for experimental and 76% for comparison group;
    1. DATA ANALYSIS:
t-tests determined no statistically significant differences in electricity use existed in the experimental and comparison groups in the five months before weatherization, but significant differences were observed in four out of five months after weatherization.
    1. CONCLUSIONS:
Weatherization reduces energy consumption by 3,422 kWh per year for the average household, or $485 over the 30 years the weatherization will last.
  1. CRITIQUE
    1. Possible Threats to Internal Validity
      1. History:
controlled by comparison group (e.g., changes in weather)
      1. Maturation:
houses aged at same rate; however, some may have been older to begin with and so were more decrepit at the time of the study
      1. Testing:
the conservation attitude survey may have sensitized respondents to adopt conservation measures
      1. Instrumentation:
Electric meter readings were used; abnormally high or low readings were averaged over two months
      1. Regression Artifact:
No differences between experimental and comparison groups in electricity use before weatherization; not chosen for extreme scores.
      1. Selection bias:
All participants had self-selected into the weatherization program; likely to have similar motivation
      1. Experimental Mortality:
13% of experimentals and 10% of comparisons were lost to the study; however, no significant differences in pre-program energy use.
      1. Design contamination:
controls may have adopted conservation measures before weatherization
    1. Possible Threats to External Validity
      1. unique program features:
Volunteers for weatherization are not like other electricity customers
      1. experimental arrangements:
n/a
      1. other threats:
Overall energy savings seem small; may not be cost-effective for public policy to expand the program