Communication Center  Conference  Projects Share  Reports from the Field Resources  Library  LSC Project Websites  NSF Program Notes
 How to Use this site    Contact us  LSC-Net: Local Systemic Change Network
Virtual Conference 2003

Virtual Conference 2002

Virtual Conference 2001

Other LSC Conference Archives

Lessons Learned 2002

Lessons Learned 2000

Effects of the LSC

Other Presentations

Public Engagement

Conference Schedule

Conference Material

  New!     

Slides from the Proposal for Measuring the LSC Program's Effects on Student Achievement in Science presentation.

author: Dan Heck, Horizon Research, Inc.
presented at: Studying the Effects of the LSC on Students
published: 02/02/2001
posted to site: 02/02/2001

Proposal for Measuring the LSC Program's Effects on Student Achievement in Science
Slides Presented by Dan Heck of Horizon Research Inc.

"HRI has proposed a study to meet the challenge of investigating program-wide effects on student achievement in science, without imposing undue burden on projects." (taken from "Horizon Reasearch Inc. "Proposal for Measuring the LSC Program's Effects on Student Achievement in Science" distributed by HRI in the conference materials binder.)

Slides of talk follow:


Why a K-8 Science Study?

  • NSF has needs for program wide studies of LSC effects on students

  • Many science projects have internal needs for studying effects on students

  • Most science projects include grades 4-6

  • Most science projects do not have access to science achievement data


Instrumentation

  • Achievement test made up of multiple choice items taken from 4th and 8th grade NAEP and TIMSS

  • Sub-scales: Nature of Science, Physical Science, Earth Science, Life Science

  • Projects are encouraged to add open-ended and/or performance items to the achievement instrument

  • Teacher survey:

    • demographics
    • background
    • content coverage
    • instructional materials
    • professional development
    • student demographics

  • Student demographic survey


Sampling

  • Project chooses one grade level

  • (4th, 5th, or 6th)

  • HRI randomly samples 4-6 classes

    • About 120 students per project
    • About 3500 students program wide

  • Demographics of sample compared to demographics of districts to examine representativeness

  • Individual projects may choose to include a larger sample (piggy-back) in order to study project level effects of the LSC on student achievement


Design

  • Pre-test, Post-test during a single academic year

  • Comparison groups

    • Exposure to content area (life, physical, earth science)
    • Exposure to content area based on LSC-designated instructional materials
    • Teacher's overall LSC PD
    • Teacher's LSC PD in each content area

  • Disaggregation (Achievement "gaps")

    • Socioeconomic status
    • English-language proficiency
    • Gender
    • Race/ethnicity


Internal Validity

  • Pilot testing the achievement instrument

    • Confirm psychometric properties of items
    • Investigate instructional sensitivity of items

  • Examine representativeness of sample

    • Compare demographics of sample to population (district demographics)
    • Compare treatment levels (PD, use of instructional materials) to information from other studies of the LSC program


Analysis

  • Descriptive statistics

    • Information on the sample
    • Basic information on status of grade 4-6 science achievement across the LSC program
    • No project-specific statistics

  • Inferential statistics

    • Series of hierarchical linear models (HLM)
    • Outcomes:
      • Achievement in overall science
      • Achievement in nature of science
      • Achievement in earth science
      • Achievement in physical science
      • Achievement in life science

    • Controlled for prior knowledge (pre-test)
    • Achievement gains predicted by
      • Exposure to content area
      • Use of LSC-designated instructional materials
      • Teacher's overall LSC PD
      • Teacher's LSC PD in each content area

    • Achievement gaps predicted by
      • all variables above