Samples, Inclusion, and Participation
The schools and students participating in NAEP assessments are selected to be representative of all schools and students nationally. The results from the 2018 TEL assessment at grade 8 are based on a representative sample of approximately 15,400 eighth-graders from about 600 schools. Samples of schools and students are drawn from public and private schools across the country. The results from the assessed students are combined to provide accurate estimates of the overall performance of students in the nation. Unlike NAEP assessments in other subjects such as reading, mathematics, and science, the TEL assessment sample was not designed to report results for individual states or large urban districts. Download the summary data tables via the link at the bottom of the page to find out more about the TEL sample and school/student participation.
Each school that participates in the assessment, and each student assessed, represents only a portion of the larger population of interest. The results are weighted to account for the disproportionate representation of some groups in the selected sample, including the oversampling of schools with high concentrations of students from certain racial/ethnic groups and the lower sampling rates of students who attend small schools. Read more about sampling and weighting.
Assessing representative samples of students, including students with disabilities (SD) and English language learners (ELL), helps to ensure that NAEP results accurately reflect the educational performance of all students in the target population, and can continue to serve as a meaningful measure of U.S. studentsâ€™ academic achievement over time.
To ensure that all selected students from the population can be assessed, many of the same accommodations that SD and ELL students use on other tests are provided for those students participating in NAEP. NAEPâ€™s accommodation practices are evolving along with digitally based assessments (DBA). Instead of using a â€œone-size-fits-allâ€ approach, DBA allows NAEP assessments to be more flexible in meeting the needs of different students. Read more about accommodations available in NAEP. With DBA, some accommodations are provided by the test delivery system (e.g., extended time) while others are available outside of the test delivery system (e.g., breaks during the test). DBA also includes a set of accessibility features, referred to as universal design elements, that are available to all students. See the chart below for all of the universal design elements available in TEL 2018.
|NAEP Universal Design Elements||Availability|
|Zoom Tool||Discrete questions only|
|Highlighter Tool||Discrete questions only|
|Elimination Tool||Discrete questions only (multiple-choice questions)|
|Word Definition Tool||Discrete questions only|
|Text-to-speech Tool||Discrete questions and Scenario-based tasks|
|Hide/Show Timer Tool||Discrete questions and Scenario-based tasks|
|Small Group||Discrete questions and Scenario-based tasks|
|One-on-one||Discrete questions and Scenario-based tasks|
|Using a Computer to Respond||Discrete questions and Scenario-based tasks|
|Adjust or Provide High Contrast||Discrete questions and Scenario-based tasks|
Even with the availability of accommodations, some students may still be excluded. Differences in student populations and in state/jurisdiction and district policies and practices for identifying and including SD and ELL students should be considered when comparing variations in exclusion and accommodation rates. States/jurisdictions and districts also vary in their proportions of special-needs students. Download the data tables via the link at the bottom of the page to see the percentages of SD and/or ELL students identified, excluded, and assessed in TEL in 2018.
The National Assessment Governing Board, which sets policy for NAEP, has been exploring ways to ensure that NAEP continues to appropriately include as many students as possible and to do so in a consistent manner for all jurisdictions and districts assessed and reported. In March 2010, the Governing Board adopted a new policy outlining specific inclusion goals for NAEP samples. The goal is to include 95 percent of all students selected for the NAEP samples, and 85 percent of those in the NAEP sample who are identified as SD or ELL. Read more about the inclusion policy and how the percentages of students are calculated.
School and Student Participation
To ensure unbiased samples, NAEP requires that participation rates for original school samples be 70 percent or higher to report national results separately for public and private schools. In instances where participation rates meet the 70 percent criterion but fall below 85 percent, a nonresponse bias analysis is conducted to determine if the responding school sample is not representative of the population, thereby introducing the potential for nonresponse bias.
Before replacing originally sampled schools that declined to participate with substitute schools, the weighted national school participation rates for the 2018 TEL assessment were 86 percent for grade 8 (89 percent for public schools, 58 percent for private schools, and 90 percent for Catholic schools). In 2018, the school participation rates for private schools at grade 8 did not meet the criteria so their results are not reportable.
Weighted student participation rates were 93 percent at grade 8 (93 percent for public school students, 93 percent for private school students, and 95 percent for Catholic school students).
Nonresponse bias analyses were conducted for the private school samples at grade 8 in 2018. The results of the nonresponse bias analyses showed that including substitute schools did not have much effect in reducing the nonresponse bias; however, adjusting the sampling weights to account for school nonresponse was partially effective in reducing the potential for nonresponse bias. For example, the potential bias for non-Catholic private schools was reduced after the nonresponse adjustments at grade 8. However, some variables examined in the analyses still indicated potential bias after nonresponse adjustments. For instance, the potential bias for private schools in the Northeast and Midwest Census regions remained significant after nonresponse adjustments.
Read more about nonresponse bias in the NAEP Technical Documentation.
About the TEL Assessment
The National Assessment of Educational Progress (NAEP) Technology and Engineering Literacy (TEL) assessment at grade 8 is a completely digitally based assessment that includes interactive scenario-based tasks and selected-response questions that measure TEL concepts and skills. To allow students to demonstrate the wide range of knowledge and skills detailed in the three TEL assessment areas and three practices, they are asked to perform a variety of problem-solving tasks based on interactive scenarios reflecting realistic solutions. Results are reported for the nation overall.