About the NAEP U.S. History Assessment
The National Assessment of Educational Progress (NAEP) in U.S. history is designed to measure students' knowledge of American history in the context of change and continuity in democracy, culture and society, technological and economic changes, and America's changing world role. Students answer a series of selected-response and open-ended questions based on these areas (or themes) in American history. Performance results are reported for students in the nation and disaggregated by various student characteristics.
In 2018, the NAEP U.S. history assessment transitioned from a paper-based assessment (PBA) to a digitally based assessment (DBA) at grade 8. A multistep process was used for the transition from PBA to DBA, which involved administering the assessment in both formats to randomly equivalent groups of students in 2018. The transition was designed and implemented with careful intent to preserve trend lines that show student performance over time. Thus, the results from the 2018 U.S. history assessment can be compared to results from previous years.
Digitally Based Assessment
The NAEP 2018 digitally based U.S. history assessment was designed to continue reporting trends in student performance dating back to 1994, while keeping pace with the new generation of classroom environments in which digital technology has become an increasing part of students' learning. The 2018 assessment content was developed with the same U.S. history framework used to develop the 2014 paper-based assessment.
Most of the content administered in the 2018 digitally based U.S. history assessment was also used in the 2014 paper-based assessment. The previously used questions were adapted to fit a tablet screen. While the presentation of content changed, the content itself did not change. Of the nine DBA blocks administered at grade 8, three blocks were newly developed for 2018.
The assessment was administered on tablets supplied by NCES using a secure, local NAEP network. This allowed NAEP administrators to create a stable assessment environment by bringing in their own equipment that would not be influenced by school-based equipment or school internet connectivity, thereby maintaining consistency across the assessed schools. Students were able to interact with the tablets via touchscreen, with an attached keyboard, or using a stylus provided by NAEP. The digitally based U.S. history assessment provided students with interactive item types and online tools such as zooming capability, and different contrast settings for the visually impaired. At the beginning of the assessment session, students viewed an interactive tutorial that provided all the information needed to take the assessment on a tablet; for example, it explains how to progress through questions, and how to indicate answers for selected-response questions and use online tools.
To investigate potential differences in performance that might be due to the digital transition, the paper-based and digitally based versions of the assessment were administered to randomly equivalent groups of students, and each participating student took the assessment in only one mode.
After the administration of the assessment, the National Center for Education Statistics (NCES) conducted rigorous analyses of the data and aligned the 2018 results to previous assessment years using a two-step process.
- First, common item linking was used to calculate the trend line from 2014 to 2018 based on the paper-based assessment results. This kind of linking was possible because all the 2018 paper-based questions were also administered in 2014 and showed the same statistical properties.
- Second, common population linking was used to align the 2018 paper-based assessment results with the 2018 digitally based assessment results. This kind of linking was possible because the samples of students for each assessment mode were randomly equivalent, ensuring that the students' educational experiences and characteristics were equivalent.
Once the common population linking aligned the digital results to the paper results on the national level, the analyses evaluated whether the linking allowed for fair and meaningful comparisons for major reporting student groups. These evaluations supported making trend comparisons between the digital assessment and previous paper-based assessments for student groups.
These analysesâ€”common item linking based on paper results and common population linking of paper results to digital resultsâ€”enabled NCES to successfully maintain the U.S. history trend line while transitioning to digital assessment. The 2018 U.S. history assessment results in this report are based on the combined performance of students who took the assessment on paper and students who took the assessment on tablets.
Technical documentation on the transition to the 2018 digitally based U.S. history assessment is forthcoming.