An Introduction to Adaptive Interventions and SMART Designs in Education

Educators must often adapt interventions over time because what works for one student may not work for another and what works now for one student may not work in the future for the same student. Adaptive interventions provide education practitioners with a prespecified, systematic, and replicable way of doing this through a sequence of decision rules for whether, how, and when to modify interventions. The sequential, multiple assignment, randomized trial (SMART) is one type of multistage, experimental design that can help education researchers build high-quality adaptive interventions. Despite the critical role adaptive interventions can play in various domains of education, research about adaptive interventions and the use of SMART designs to develop effective adaptive interventions in education is in its infancy. To help the field move forward in this area, the National Center for Special Education Research (NCSER) and the National Center for Education Evaluation and Regional Assistance (NCEE) commissioned a paper by leading experts in adaptive interventions and SMART designs. This paper aims to provide information on building and evaluating high-quality adaptive interventions and review the components of SMART designs, discuss the key features of the SMART, and introduce common research questions for which SMARTs may be appropriate. To view this paper, click here.

Companion Guidelines on Replication and Reproducibility in Education Research (2018)

Since the release of the Common Guidelines for Education Research and Development in 2013, there has been increased attention to replication and reproducibility studies and their role in building the evidence base. In response, a joint IES-NSF committee has developed the Companion Guidelines on Replication and Reproducibility in Education Research to supplement the Common Guidelines for Education Research and Development. The Companion Guidelines highlight the importance of replication and reproducibility studies and provides guidance on the steps researchers can take to promote corroboration, ensure the integrity of research, and extend the evidence base. To download a copy of the Companion Guidelines on Replication and Reproducibility in Education Research, click here.

Common Guidelines for Education Research and Development (2013)

In August 2013, a Joint Committee of representatives from the U.S. Department of Education (ED) and the National Science Foundation (NSF) released a report that 1) defines the types of ED- and NSF-funded research that relates to the development and testing of interventions and strategies designed to increase learning, and 2) specifies how the types of research relate to one another, and describe the theoretical and empirical basis needed to justify each research type. The Joint Committee intends this report to be a "living document" that may be adapted by agencies or divisions within agencies in response to their needs and opportunities. Over time, the framework may be elaborated or rearranged according to agency focus and assessments of the needs of education researchers and practitioners. To download a copy of the Common Guidelines for Education Research, click here.

Translating the Statistical Representation of the Effects of Education Interventions into More Readily Interpretable Forms

The primary purpose of this paper is to provide suggestions to researchers about ways to present statistical findings about the effects of educational interventions that might make the nature and magnitude of those effects easier to understand. These suggestions and the related discussion are framed within the context of studies that use experimental designs to compare measured outcomes for two groups of participants, one in an intervention condition and the other in a control condition.

View, download, and print the full report as a PDF file (1.3 MB)

Designing Cluster-Randomized Trials

The National Center for Education Research (NCER) in the Institute of Education Sciences (IES) of the U.S. Department of Education hosts an annual Summer Research Training Institute on Cluster-Randomized Trials to increase the national capacity of researchers to develop and conduct rigorous evaluations of the effectiveness of education interventions. To see video transcripts and presentation files from the 2008 Institute, click here. For the 2007 Institute presentations, click here.

CONSORT, which stands for Consolidated Standards of Reporting Trials, was developed to provide guidance on the tracking and reporting of critical aspects of randomized controlled trials (RCTs). The main initiative of the CONSORT group was the development of a set of recommendations for reporting RCTs, called the CONSORT Statement. The Statement includes a 22-item checklist, which focuses on study design, analysis, and interpretation of the results, and a flow diagram, which provides a structure for tracking participants at each study stage. IES encourages researchers to use these tools in their Goal 3 and Goal 4 research projects.
http://www.consort-statement.org/

Power Analysis

This paper, entitled "Statistical Power Analysis in Education Research" and coauthored by Larry Hedges and Christopher Rhoads provides a guide to calculating statistical power for the complex multilevel designs that are used in most field studies in education research. Click here to view, download and print the paper as a PDF file.

Goal Three and Goal Four applications submitted to IES will typically require a detailed power analysis. The Optimal Design Software for Multi-Level and Longitudinal Research is useful for statistical power analysis of group-level interventions. To download the free Optimal Design software and learn more about the project, click here.

Presentations on Methodological Issues

To view selected presentations, meeting agendas, and videos from previous IES Research Conferences, click here.

Technical Methods Resources

NCEE has formed a technical methods group to work on issues and strategies that assure evaluations of education interventions provide unbiased and causally valid assessments. The technical methods working group aims to advance and provide guidance for those specialists who are embarking on evaluations in education. To view recent Technical Methods Reports, click here.

Effect Sizes in Research on Children and Families

This special issue of Child Development Perspectives (December 2008, Volume 2 Issue 3) contains a number of articles on the application of effect sizes in research on children and families.
Articles on effect sizes included in the special issue of Child Development Perspectives. (142 KB)

Browse Resources for Researchers