- Disclosure of Potential Conflicts of Interest
- Executive Summary
- Study Design
- Collecting Achievement and Product Usage Data
- Software Products
- Findings from First Year of Study
- First-Year Effects of First Grade Technology Products
- First-Year Effects of Fourth Grade Technology Products
- First-Year Effects of Sixth Grade Technology Products
- First-Year Effects of Algebra I Technology Products
- Does Experience Increase Product Effects?
- Effects of Individual Products
- Summary
- List of Figures
PDF & Related Info
Executive Summary
In the No Child Left Behind Act, Congress called for the U.S. Department of Education (ED) to conduct a rigorous study of the conditions and practices under which educational technology is effective in increasing student academic achievement. A 2003 design effort by ED working with educational technology and research experts recommended focusing the study on software products used to support reading and math instruction. The study team set up a competitive process and worked with ED to select reading products to be studied in the first and fourth grades, pre-algebra products in the sixth grade, and algebra I products in high school (or possibly in middle school). The team implemented an experimental design in which teachers in the same school were randomly assigned to use or not to use a software product, and the team collected test scores and other data to assess effectiveness of the software products.
A report was released in April 2007 presenting study findings for the 2004-2005 school year (Dynarski et al. 2007). The findings indicated that, after one school year, differences in student test scores were not statistically significant between classrooms that were randomly assigned to use products and those that were randomly assigned not to use products. School and teacher characteristics generally were not related to whether products were effective.
The study also collected test scores and other data in the 2005-2006 school year, in which teachers who continued with the study had a new cohort of students and a year of experience using software products. Data from the second cohort enable the study to address the question of whether software products are more effective in raising test scores after teachers have a year of experience using them.
The first-year report presented average effects of four groups of products on student test scores, which supported assessing whether products were effective in general. School districts and educators purchase individual products, however, and knowing whether individual products are effective is important for making decisions supported by evidence. This report presents findings on the effects of 10 products on student test scores.