About the Civics Assessment
The National Assessment of Educational Progress (NAEP) in civics is designed to measure the civics knowledge and skills that are critical to the responsibilities of citizenship in America. Students answer a series of selected-response and open-ended questions designed to measure their knowledge and understanding of civics. Performance results are reported for students in the nation and disaggregated by various student characteristics.
In 2018, the NAEP civics assessment transitioned from a paper-based assessment (PBA) to a digitally based assessment (DBA) at grade 8. A multi-step process was used for the transition from PBA to DBA, which involved administering the assessment in both formats to randomly equivalent groups of students in 2018. The transition was designed and implemented with careful intent to preserve trend lines that show student performance over time. Thus, the results from the 2018 civics assessment can be compared to results from previous years.
Digitally Based Assessment
The NAEP 2018 digitally based civics assessment was designed to continue reporting trends in student performance dating back to 1998, while keeping pace with the new generation of classroom environments in which digital technology has become an increasing part of students' learning. The 2018 assessment content was developed with the same civics framework used to develop the 2014 and earlier paper-based assessment.
Most of the content administered in the 2018 digitally based civics assessment was also used in the 2014 paper-based assessment. The previously used questions were adapted to fit a tablet screen. While the presentation of content changed, the content itself did not change. Of the seven DBA blocks administered at grade 8, two blocks were newly developed for 2018.
The assessment was administered on tablets supplied by NCES using a secure, local NAEP network. This allowed the NAEP administrators to create a stable assessment environment by bringing in their own equipment that would not be influenced by school-based equipment or school internet connectivity, thereby maintaining consistency across the assessed schools. Students were able to interact with the tablets via touchscreen, with an attached keyboard, or using a stylus provided by NAEP. The digitally based civics assessment provided students with interactive item types and online tools such as zooming capability, and different contrast settings for the visually impaired. At the beginning of the assessment session, students viewed an interactive tutorial that provided all the information needed to take the assessment on a tablet; for example, it explains how to progress through questions, and how to indicate answers for selected-response questions and use online tools.
To investigate potential differences in performance that might be due to the digital transition, the paper-based and digitally based versions of the assessment were administered to randomly equivalent groups of students, and each participating student took the assessment in only one mode.
After the administration of the assessment, the National Center for Education Statistics (NCES) conducted rigorous analyses of the data and aligned the 2018 results to previous assessment years using a two-step process.
- First, common item linking was used to calculate the trend line from 2014 to 2018 based on the paper-based assessment results. This kind of linking was possible because all the 2018 paper-based questions were also administered in 2014 and showed the same statistical properties.
- Second, common population linking was used to align the 2018 paper-based assessment results with the 2018 digitally based assessment results. This kind of linking was possible because the samples of students for each assessment mode were randomly equivalent, ensuring that the students' educational experiences and characteristics were equivalent.
Once the common population linking aligned the digital results to the paper results on the national level, the analyses evaluated whether the linking allowed for fair and meaningful comparisons for major reporting student groups. These evaluations supported making trend comparisons between the digital assessment and previous paper-based assessments for student groups.
These analysesâ€”common item linking based on paper results and common population linking of paper results to digital resultsâ€”enabled NCES to successfully maintain the civics trend line while transitioning to digital assessment. The 2018 civics assessment results in this report are based on the combined performance of students who took the assessment on paper and students who took the assessment on tablets.
Technical documentation on the transition to the 2018 digitally based civics assessment is forthcoming.