IES Blog

Under an IES grant, the RAND Corporation, in collaboration with NWEA, is developing strategies for schools and districts to address the impacts of COVID-19 disruptions on student assessment programs. The goal is to provide empirical evidence of the strengths and limitations of strategies for making decisions in the absence of assessment data. Jonathan Schweig, Andrew McEachin, and Megan Kuhfeld describe early findings from surveys and structured interviews regarding key concerns of districts and schools. 

As a first step, we surveyed assessment and research coordinators from 23 school districts (from a sample of 100 districts) and completed follow-up interviews with seven of them on a variety of topics, including the re-entry scenario for their district, the planning activities that they were not able to perform this year due to coronavirus-based disruptions to spring 2020 assessments, and the strategies they were employing to support instructional planning in the absence of assessment data. While the research is preliminary and the sample of respondents is not nationally representative, the survey and interview responses identified two key concerns arising from the lack of spring 2020 assessment data which has made it challenging to examine student or school status and change over time, especially as COVID-19 has differential impacts on student subgroups:

• Making course placement decisions. Administrators typically rely on spring assessment scores—often in conjunction with other assessment information, course grades, and teacher recommendations—to make determinations for course placements, such as who should enroll in accelerated or advanced mathematics classes. 
• Evaluating programs or district-wide initiatives. Many districts monitor the success of these programs internally by looking at year-to-year change or growth for schools or subgroups of interest. 

How are school systems responding to these challenges? Not surprisingly, the responses vary depending on local contexts and resources. Where online assessments were not feasible in spring 2020, some school districts used older testing data to make course recommendations, either from the winter or from the previous school year. Some districts relaxed typical practice and provided more autonomy to individual schools, relying on school staff to exercise local judgment around course placements and using metrics like grades and teacher recommendations. Other districts reported projecting student scores based on student assessment histories. Relatedly, some districts were already prepared for this decision because they had recently experienced difficulties with adopting an online assessment system and had to address similar problems caused by large numbers of missing or invalid tests.

School districts also raised concerns about whether assessments administered during the 2020-21 school year would be valid and comparable so that they could be used in student placement and program evaluation decisions. These concerns included the following:

• Several respondents raised concerns about the trustworthiness of remote assessment data collected this fall and the extent to which results could be interpreted as valid indicators of student achievement or understanding.
• Particularly for districts that started the 2020-21 school year remotely, respondents were concerned about student engagement and motivation and the possibility of students rushing assessments, running into technological or internet barriers, or seeking assistance from guardians or other resources. 
• Respondents raised questions about the extent to which available assessment scores are representative of school or district performance as a whole. Given that vulnerable students (for example, students with disabilities, students experiencing homelessness) may be the least likely to have access to remote instruction and assessments, it is likely that the students who are not assessed this year are different from students who are able to be assessed.
• Other respondents noted that they encountered resistance from parents around fall assessment because they prioritized student well-being (for example, safety, sense of community, and social and emotional well-being) more so than academics. This is a perspective that resonates with recent findings from a nationally representative sample of teachers and school leaders drawn from RAND’s American Educator Panel (AEP).

In the next phase of the work, the research team plans to:

• Conduct a series of simulation and empirical studies regarding the most common strategies that the district respondents indicated they were using to make course placement decisions and to evaluate programs or district-wide initiatives.
• Provide a framework to help guide local research on the intended (and unintended) consequences for school and school system decision making when standardized test scores are not available.

We welcome individuals to reach out to RAND with additional recommendations or considerations. We are also interested in hearing how districts are approaching course placement, accountability, and program evaluation across the country. Connect with the research team via email at jschweig@rand.org.

Jonathan Schweig is a social scientist at the nonprofit, nonpartisan RAND Corporation.

Andrew McEachin is a senior policy researcher at the nonprofit, nonpartisan RAND Corporation.

Megan Kuhfeld is a researcher at NWEA.

As we wrap up what has been an unprecedented year in many ways, we’re taking a look back at some of NCES’s most popular content from 2020. As you reflect on the past year, we hope you’ll explore our most downloaded reports, most visited indicators, Fast Facts, and blog posts, and most viewed tweets over the past year. 

Top Five Reports, by number of PDF downloads

1. Condition of Education 2020 (7,328) 2. Digest of Education Statistics 2018 (4,936) 3. Status and Trends in the Education of Racial and Ethnic Groups 2018 (3,379) 4. Trends in High School Dropout and Completion Rates in the United States: 2019 (2,994) 5. First-Generation Students: College Access, Persistence, and Postbachelor’s Outcomes (2,382)

Top Five indicators from the Condition of Education, by number of web sessions

1. Students With Disabilities (82,304)

2. Public High School Graduation Rates (66,993)

3. Education Expenditures by Country (61,766)

4. English Language Learners in Public Schools (46,293)

5. Undergraduate Enrollment (46,766)

Top Five Fast Facts, by number of web sessions

1. Back to School Statistics (214,148)

2. Tuition Costs of Colleges and Universities (111,491)

3. College and University Endowments (78,735)

4. Degrees Conferred by Race and Sex (73,980)

5. Closed Schools (69,142)

Top Five Blog Posts, by number of web sessions

1. Introducing the 2020 Classification of Instructional Programs (CIP) and Its Website (7,822)

2. Back to School by the Numbers: 2019–20 School Year (4,400)

3. Free or Reduced Price Lunch: A Proxy for Poverty? (4,199)

4. Educational Attainment Differences by Students’ Socioeconomic Status (2,919)

5. The Digital Divide: Differences in Home Internet Access (2,699)

Top Five Tweets, by number of impressions

1. Teacher Appreciation Day (21,146)

It’s #TeacherAppreciationDay! DYK? About 3.7M full-time-equivalent (FTE) #K12 #teachers were engaged in classroom instruction in fall 2018.

Celebrate American educators & learn more #EdStats here: https://t.co/F4GWZtQZYY #TeacherAppreciationWeek #ThankATeacher pic.twitter.com/EjhEYX92cP

— NCES (@EdNCES) May 5, 2020

2. International Education Week (18,333)

DYK? Students from Asia form the largest group of international students enrolled in #postsec programs at all levels, accounting for 57% of all international students across @OECD countries in 2018.

More from #OECDEAG: https://t.co/zm16twjS3m #IEW2020 @usedgov @StateDept pic.twitter.com/OsFvfQGw8j

— NCES (@EdNCES) November 18, 2020

3. Trends in High School Dropout and Completion Rates in the United States: 2019 (17,383)

JUST RELEASED: Check out the latest trends in #HighSchool #dropout & completion rates in the U.S.! https://t.co/grt9zcSDs7 #EdChat pic.twitter.com/D7GZ0tqlXX

— NCES (@EdNCES) January 14, 2020

4. International Early Learning Study 2018 Pilot (10,870)

Preliminary findings from the International Early Learning Study (IELS) 2018 Pilot show how 5-year-olds from England, Estonia, & the U.S. performed on a direct assessment of:
• Emergent literacy & numeracy
• Self-regulation
• Social emotional skillshttps://t.co/i3KPry9sBH pic.twitter.com/YEdD5jBEvB

— NCES (@EdNCES) March 13, 2020

5. NAEP Data Training Workshop (10,782)

Are you an #EdResearch professional or grad student interested in analyzing large-scale assessment #data like #NAEP? @NAEP_NCES is accepting applications until March 31 for their Data Training Workshop, held in May. Apply TODAY: https://t.co/zYwnVZD0sp pic.twitter.com/7Du8dCCysJ

— NCES (@EdNCES) March 10, 2020

Be sure to check our blog site and the NCES website to stay up-to-date on new findings and trends in 2021. You can also follow NCES on Twitter, Facebook, and LinkedIn for daily updates and content.

As part of our Spotlight on IES Training Programs series, IES is proud to showcase five former IES postdoctoral fellows who are now principal investigators for grants funded in FY 2020. The goal of the NCER and NCSER postdoctoral training programs is to prepare scholars to conduct rigorous, relevant education and special education research. As the following examples demonstrate, IES fellows are contributing to evidence-based education in a wide range of academic domains and are addressing the needs of students, teachers, and families through their innovative measurement, exploratory, development, and evaluation work.

Dr. Crystal Bishop (IES Fellow at the University of Florida until 2016) will lead Tools for Families. This project will develop and pilot test a new component for an existing intervention that aims to improve outcomes for young children with disabilities in preschool programs. The existing program is called Evaluating Embedded Instruction for Early Learning (EIEL) and already includes tools to help teachers. In this new study, Dr. Bishop will create an additional component that helps teachers engage students’ families in implementing EIEL strategies.

Dr. Joseph Nese (IES Fellow at the University of Oregon until 2011) will lead A Comprehensive Measure of Reading Fluency: Uniting and Scaling Accuracy, Rate, and Prosody. This project aims to develop and validate an automated scoring system of oral reading fluency for students in grades 2 to 4 to better identify students in need of reading interventions and better evaluate reading interventions and builds off a previous grant Dr. Nese received as PI, Measuring Oral Reading Fluency: Computerized Oral Reading Evaluation (CORE) (R305A140203).

Dr. David Purpura (IES Fellow at the University of Illinois until 2012) will lead Reading and Playing With Math: Promoting Preschoolers' Math Language Through Picture Books and Play Activities. This program will develop, refine, and evaluate a new math language intervention, Reading and Playing with Math (RP-Math). RP-Math will leverage the language instruction using storybooks and mathematics instruction.

Dr. Rachel Rosen (IES Fellow at the University of Michigan until 2014) will lead Choice and Information: The Impact of Technology-Based Career Advising Tools on High School Students' CTE Choices and Academic Performance. This project will evaluate  two widely used technology-based career advising tools for secondary school students, Naviance and YouScience, to see whether and how these tools influence student thinking about career options, career and technical education (CTE) coursework and work-based learning options, and decisions about CTE pathways and programs of study.

Dr. Candace Walkington (IES Fellow at the University of Wisconsin, Madison until 2013) will lead Exploring Collaborative Embodiment for Learning (EXCEL): Understanding Geometry Through Multiple Modalities. This program will explore how different types multisensory experiences and modes of collaboration affect students' geometric reasoning. The researchers will leverage augmented reality (AR) technology to see if different ways of engaging with content (such as holograms, tablet-based, or paper-based images) lead to different learning outcomes.

This blog was written by Shirley Liu, virtual intern and an English/Anthropology & Sociology double major at Lafayette College, and Dr. Meredith Larson, program officer for NCER postdoctoral training.

Each year, NCES contributes data and insights to America’s Children: Key National Indicators of Well-Being, an annual report produced by the Federal Interagency Forum on Child and Family Statistics. The Forum assembles 23 federal agencies to collaboratively present the public with policy-relevant statistics related to America’s children and their families.

The 2020 edition of America’s Children in Brief examines indicators of children’s well-being in the following domains: demographic background, family and social environment, economic circumstances, health care, physical environment and safety, behavior, education, and health. This year’s report has a special focus on differences by metropolitan status,^[1] giving readers a closer look at how measures of well-being vary based on the type of community in which children and their families live.

For example, the report includes information and federal data about birth rates among adolescent mothers, adolescent alcohol use, and adolescent depression and how these data vary by geographic area. Some key highlights of these topics are below:

• The birth rate among females ages 15–19 was 17 per 1,000 in 2018.
  • The birth rate was highest for adolescents living in rural counties (26 per 1,000), followed by those living in micropolitan counties (24 per 1,000) and metropolitan counties (16 per 1,000).
     
• In 2019, 4 percent of 8th-, 9 percent of 10th-, and 14 percent of 12th-grade students reported binge drinking.
  • Binge drinking was reported by 18 percent of 12th-grade students living in nonmetropolitan areas compared with 14 percent of 12th-grade students living in metropolitan areas.
     
• In 2018, 14 percent of the population ages 12–17 had at least one major depressive episode during the past year.
  • This percentage did not differ by metropolitan status (14 percent in metropolitan areas, 15 percent in micropolitan areas, and 13 percent in rural areas).

The 2020 edition also presents new NCES data on differences in high school completion rates by metropolitan status. Many entry-level jobs and colleges/universities require applicants to produce a high school diploma or its equivalent. Thus, measuring how high school completion rates vary across time and for different groups of students is important since they are strongly related to future income.^[2]

In 2018, among young adults ages 18–24 who lived in metropolitan areas, 94 percent had completed high school with a diploma or an alternative credential, such as a GED certificate (figure 1). This rate was higher than the 89 percent completion rate for young adults living in nonmetropolitan areas. Furthermore, the high school completion rate for young adults who lived in metropolitan areas increased from 91 percent in 2010 to 94 percent in 2018. The completion rate for young adults who lived in nonmetropolitan areas was not measurably different between these two years (88 percent in 2010 and 89 percent in 2018).

Figure 1. Percentage of young adults ages 18–24 who have completed high school, by metropolitan status: 2010 and 2018

NOTE: Diploma equivalents include alternative credentials obtained by passing examinations such as the General Educational Development (GED) test. This figure excludes those still enrolled in high school or enrolled in a lower education level. The U.S. Office of Management and Budget classifies some counties as within a metropolitan statistical area. The remaining counties are considered nonmetropolitan. Nonmetropolitan counties include counties in micropolitan statistical and rural areas. Total includes those whose household metropolitan status was “not identified,” which is not separately shown.

SOURCE: U.S. Census Bureau, Current Population Survey, School Enrollment Supplement.

Through its Artemis program, NASA will land the first woman and next man on the Moon by 2024, using innovative technologies to explore more of the lunar surface than ever before. NASA is collaborating with commercial and international partners to establish sustainable exploration by the end of the decade and to apply what is learned to take the next giant leap—sending astronauts to Mars.

K-12 Artemis Moon Pod Student Essay Contest

NASA is inviting students in kindergarten through Grade 12 to join the Artemis adventure.  Through its challenge, students can imagine “what it might be like if you were living with a pod of astronauts 250,000 miles from Earth.”

In the challenge, students write essays focusing on leading a one-week expedition at the Moon’s South Pole. Plans and details of the expedition should consider the types of skills, attributes, and personality traits of their Moon Pod crew, and one machine, robot, or technology that will be left on the lunar surface to help future astronauts explore the Moon.

Three levels of challenges are being held for students in Grades K-4, 5-8, and 9-12. Every student who submits an entry will receive a certificate from NASA and be invited to a special NASA virtual event—with an astronaut! For all entry requirements and judging criteria, please read the rules.  Students and teachers can sign up and submit their entry at the contest site. Even if you are not a student you can still participate. U.S. residents over the age of 18 can apply to be judges for the contest to help NASA make their selection.

The essay competition is being managed by a web-based platform developed by Future Engineers based in Burbank, California.  This platform was created with the support of a 2017 award from the IES Small Business Innovation Research program (ED/IES SBIR).  Future Engineers built this platform to be an online hub for classrooms and educators to access free, project-based STEM activities and to provide a portal where students submit and compete in different kinds of maker and innovation challenges across the country.  The Artemis essay contest follows the Mars 2020 “Name the Rover” contest, which was also managed by Future Engineers. (See the recap of that challenge here.)

Stay tuned for the winning essays in the months to come!

Edward Metz (Edward.Metz@ed.gov) is a research scientist at the Institute of Education Sciences at the US Department of Education and Program Manager of ED/IES SBIR.

Katherine Brown is the lead communication specialist for the Office of STEM Engagement at NASA.

About ED/IES SBIR

The U.S. Department of Education’s Small Business Innovation Research program, administered by the Institute of Education Sciences (IES), funds projects to develop education technology products designed to support students, teachers, or administrators in general or special education. The program emphasizes rigorous and relevant research to inform iterative development and to evaluate whether fully developed products show promise for leading to the intended outcomes. The program also focuses on commercialization once the award period ends so that products can reach students and teachers and be sustained over time. ED/IES SBIR-supported products are currently used by millions of students in thousands of schools around the country.

About NASA’s Office of STEM Engagement

NASA’s journeys have propelled technological breakthroughs, pushed the frontiers of scientific research, and expanded our understanding of the universe. These accomplishments, and those to come, share a common genesis: education in science, technology, engineering, and math. NASA’s Office of STEM Engagement strives to create unique opportunities for a diverse set of students to contribute to NASA’s work in exploration and discovery; build a diverse future STEM workforce by engaging students in authentic learning experiences with NASA’s people, content and facilities; and attract diverse groups of students to STEM through learning opportunities that spark interest and provide connections to NASA’s mission and work. To achieve these goals, NASA’s Office of STEM Engagement strives to inspire the next generation to discover their way to a new era of American innovation and explore further into space than ever before.