Samples, Inclusion, and Participation
The schools and students participating in NAEP assessments are selected to be representative of all schools nationally and of public schools at the state/jurisdiction and Trial Urban District Assessment (TUDA) district levels. The results from the 2017 reading assessment at grades 4 and 8 are based on the representative samples of 148,800 fourth-graders from 7,830 schools and 141,800 eighth-graders from 6,500 schools who took the assessment on tablets. Samples of schools and students are drawn from each state and from the District of Columbia, and Department of Defense schools. The sample of students participating in the TUDA school districts is an extension of the sample of students who would usually be selected by NAEP as part of national and state samples. Representative samples of 34,000 fourth-grade and 31,300 eighth-grade public school students from 27 urban districts participated in the 2017 reading assessment. These 27 TUDA districts are listed below.
2017 TUDA DISTRICTS
|Atlanta||Denver||Jefferson County (KY)|
|Baltimore City||District of Columbia (DCPS)||Miami-Dade|
|Boston||Duval County (FL)||Milwaukee|
|Charlotte-Mecklenburg||Fort Worth||New York City|
|Clark County (NV)||Guilford County (NC)||San Diego|
|Cleveland||Hillsborough County (FL)||Shelby County (TN)|
The results from the assessed students are combined to provide accurate estimates of the overall performance of students in the nation and in individual states and other jurisdictions. Results for the nation reflect the performance of students attending public schools, private schools, Bureau of Indian Education schools, and Department of Defense schools. Results for states/jurisdictions and for districts reflect the performance of students in public schools only and are reported along with the results for public school students in the nation. Charter schools are included in the public school samples at the state level. For TUDA districts, beginning in 2009, results for charter schools are included in district results only if they contribute to the districtâ€™s Adequate Yearly Progress (AYP) report to the U.S. Department of Education. Download the summary data tables via the link at the bottom of the page to see the national, state/jurisdiction, and district sample sizes for the 2017 reading assessment.
Each school that participated in the assessment, and each student assessed, represents only a portion of the larger population of interest. The results are weighted to account for the disproportionate representation of some groups in the selected sample, including the oversampling of schools with high concentrations of students from certain racial/ethnic groups and the lower sampling rates of students who attend small schools. Read more about NAEP sampling and weighting in the NAEP Technical Documentation.
Assessing representative samples of students, including students with disabilities (SD) and English language learners (ELL), helps to ensure that NAEP results accurately reflect the educational performance of all students in the target population and are a meaningful measure of U.S. studentsâ€™ academic achievement over time.
To ensure that all selected students from the population can be assessed, many of the same accommodations that SD and ELL students use on other tests are provided for those students participating in NAEP. Accommodations were first made available for the reading assessment in 1998. Read more about accommodations available in NAEP. In the 2017 NAEP digitally based reading assessment, accommodations such as a text-to-speech for directions were universal design elements integral to the delivery system and available to all students.
Even with the availability of accommodations, some students may still be excluded. Differences in student populations and in state/jurisdiction and district policies and practices for identifying and including SD and ELL students should be considered when comparing variations in exclusion and accommodation rates. States/jurisdictions and districts also vary in their proportions of special-needs students (especially ELL students). Download the summary data tables via the link at the bottom of the page to see the percentages of SD and/or ELL students identified, excluded, and assessed in reading in 2017.
Because providing accommodations represented a change in testing conditions that could potentially affect the measurement of changes over time, split national samples of students were assessed in 1998â€”one sample permitted accommodations and the other did not. Although the results for both samples are presented in the tables and figures, any comparisons to 1998 in the text are based only on the accommodated sample.
The National Assessment Governing Board, which sets policy for NAEP, has been exploring ways to ensure that NAEP continues to appropriately include as many students as possible and to do so in a consistent manner for all jurisdictions and districts assessed and reported. In March 2010, the Governing Board adopted a new policy outlining specific inclusion goals for NAEP samples. At the national, state, and district levels, the goal is to include 95 percent of all students selected for the NAEP samples, and 85 percent of those in the NAEP sample who are identified as SD or ELL. Read more about the inclusion policy and how the percentages of students are calculated.
All of the states/jurisdictions and almost all of the districts participating in the 2017 NAEP reading assessment met or exceeded the 95 percent inclusion goal for grades 4 and 8. The goal was not met in two districts in the 2017 reading assessment: Dallas at grade 4 (where approximately 71% of students were assessed) and Detroit at grade 8 (where approximately 92% of students were assessed).
Tables and figures showing performance results include a footnote to caution readers about the overall representativeness of the Dallas grade 4 reading sample due to the level of inclusion observedâ€”particularly in the Hispanic and English language learner populations where the assessed sample underrepresents the target population. The table below provides details on the demographic composition of the Dallas target population and assessed students by race/ethnicity and status as English language learners.
Percentage of fourth-grade public school students in Dallas in the target population and assessed sample in NAEP reading, by selected student groups: 2017
|Student group||Percentage of students|
|Target population||Assessed sample|
|English language learners||54||37|
|Not identified as English language learners||46||63|
Download the summary data tables via the link at the bottom of the page to see the inclusion rates in reading for states/jurisdictions and the 27 participating TUDA districts in 2017.
School and Student Participation
To ensure unbiased samples, NAEP requires that participation rates for original school samples be 70 percent or higher to report national results separately for public and private schools. In instances where participation rates meet the 70 percent criteria but fall below 85 percent, a nonresponse bias analysis is conducted to determine if the responding school sample is not representative of the population, thereby introducing the potential for nonresponse bias.
Before replacing originally sampled schools that declined to participate with substitute schools, the weighted national school participation rates for the 2017 reading assessment were 97 percent for grade 4 (100 percent for public schools, 61 percent for private schools, and 89 percent for Catholic schools), and 96 percent for grade 8 (100 percent for public schools, 60 percent for private schools, and 86 percent for Catholic schools). In 2017, the school participation rates for private schools at both grades 4 and 8 did not meet the criteria so their results are not reportable.
Weighted student participation rates were 94 percent at grade 4 (94 percent for public school students, 96 percent for private school students, and 96 percent for Catholic school students) and 92 percent at grade 8 (92 percent for public school students, 94 percent for private school students, and 95 percent for Catholic school students).
Nonresponse bias analyses were conducted for the private school samples at both grades 4 and 8 in 2017. The results of the nonresponse bias analyses showed that including substitute schools and adjusting the sampling weights to account for school nonresponse were partially effective in reducing the potential for nonresponse bias. For example, the potential bias for Catholic schools and non-Catholic private schools was reduced after the substitution and nonresponse adjustments at both grades 4 and 8. However, some variables examined in the analyses still indicated potential bias after nonresponse adjustments. For instance, the percentage enrollment of Black students was somewhat over-represented in the final private school samples at grade 4.
State and TUDA participation
Standards established by the National Assessment Governing Board require that school participation rates for the original state/jurisdiction and TUDA district samples need to be at least 85 percent for results to be reported. In 2017, all 52 states and jurisdictions and all 27 TUDA districts met this participation rate requirement with participation rates of 95 to 100 percent.
Download the summary data tables via the link at the bottom of the page to see the participation rates in reading for the nation, states, and the 27 participating districts.
About the NAEP Reading Assessment
The National Assessment of Educational Progress (NAEP) reading assessment uses literary and informational texts to measure studentsâ€™ reading comprehension skills. Students read grade-appropriate passages and answer questions based on what they have read. Performance results are reported for the nation overall, for states and jurisdictions, and for 27 districts participating in the Trial Urban District Assessment (TUDA). In 2017, the NAEP reading assessment transitioned from a paper-and-pencil assessment to a digitally based assessment (DBA) at grades 4 and 8. A multi-step process was used for the transition from paper-based to DBA, with the careful intent to preserve trend lines that show student performance over time. The process involved administering the assessment in both the DBA and PBA formats to randomly equivalent groups of students in 2017. Thus, the results from the 2017 reading assessment can be compared to results from previous years.