The schools and students participating in NAEP assessments are selected to be representative of all schools nationally and of public schools at the state level. Samples of schools and students are drawn from participating states and from the District of Columbia and Department of Defense schools. The results from the assessed students are combined to provide accurate estimates of the overall performance of students in the nation and in individual states and other jurisdictions. Because each school that participated in the assessment, and each student assessed, represents a portion of the population of interest, the results are weighted to account for the disproportionate representation of the selected sample. Read more technical information about weighting adjustments made at the school and student level.
National participation
To ensure unbiased samples, NAEP standards require that participation rates for original school samples be 70 percent or higher to report national results separately for public and private schools. In instances where participation rates meet the 70 percent criteria but fall below 85 percent, a nonresponse bias analysis is conducted to determine if the responding school sample is not representative of the population, thereby introducing the potential for nonresponse bias.
The weighted national school participation rate for the 2011 science assessment at grade 8 was 97 percent (100 percent for public schools, 74 percent for private schools). The weighted student participation rate was 93 percent.
A nonresponse bias analysis was conducted for the grade 8 private school sample. The analysis showed that, while the original responding school samples may have been somewhat different from the entire sample of eligible schools, including substitute schools and adjusting the sampling weights to account for school nonresponse were partially effective in reducing the potential for nonresponse bias. However, some variables examined in the analysis still indicated potential bias after nonresponse adjustments. For instance, smaller schools were somewhat overrepresented in the final private school sample, and the responding sample of private schools contained a higher percentage of Black students and lower percentage of White students than the original sample of eligible private schools.
State participation
Standards established by the National Assessment Governing Board require that school participation rates for the original state samples need to be at least 85 percent for results to be reported. With one exception, participation rates for the original samples in the states and jurisdictions participating in the 2011 science assessment at grade 8 were 99 or 100 percent. The participation rate in Colorado was 84 percent.
A nonresponse bias analysis was conducted for the public school sample in Colorado. After sampling weights were adjusted to account for nonresponse, some school characteristics still indicated potential bias. For instance, White students were slightly underrepresented and
Hispanic students were slightly overrepresented in the final responding sample. The remaining potential bias associated with student achievement was even larger, however, with the responding sample containing a higher percentage of low-achieving students than the original eligible sample.
Learn more about the sampling design.
|