Comparing spending and academic results is unrealistic

Public schools in Idaho and across the nation should not be expected to do the impossible, nor should they be condemned when they fail the impossible.

The J.A. and Kathryn Albertson Foundation has published “Five Questions about Education Funding in Idaho” on the website for “The ReThink Series: Knowledge Base of Education Facts.”  The foundation notes: “As Idaho strives to raise high school and postsecondary completion rates, the role of K-12 funding will remain in the spotlight. This fact book seeks to provide a common foundation for that discussion.”

Regarding Question No. 5 (“Would spending more improve achievement?”), the foundation says: “In Idaho’s large school districts, there is no clear relationship between per-student spending and achievement.

“Similar patterns emerge at the national level. From 1960 to 2005, K-12 per-student spending nearly quadrupled, adjusting for inflation.  … And yet the U.S. has not experienced a comparable gain in student achievement or graduation rates.”

While per-student spending has been reported even before 1960, there are no continuous K-12 achievement data from 1960 to 2005 — none. The National Assessment of Educational Progress or NAEP results, which were not reported until the early 1970s, are the oldest source for continuous K-12 achievement data.

For years, economists have reported comparisons of per-student spending and student achievement in order to support the agenda of this or that organization. In 2008, the Heritage Foundation issued a report containing a graph that linked spending and performance.

The chart’s narrative tells us that (from 1970 to 2005) despite more than doubling per student expenditures … reading scores in U.S. schools have remained relatively flat. Citations provided sources for the expenditures and reading scores. There was, however, no citation giving anyone reason to expect, as the chart implies and as the Albertson Foundation suggests, that an increase in spending should result in “a comparable gain in student achievement.”

From 1971 to 2005, per-student expenditures increased 228 percent. NAEP’s online Data Explorer indicated that in 1971, 17-year-olds in the nation’s public schools had an average NAEP reading score of 285, with a standard deviation of 46.  The expectation that a comparable 228 percent reading gain be realized would require that 17-year-olds in the nation’s public schools would have an average reading score of 650 in 2005. Be advised, however, that a NAEP reading score of 650 is IMPOSSIBLE.

The NAEP reading scale is from 0 to 500, period. A reading score of 650 is simply beyond the psychometrics of the NAEP reading scale. From the NAEP website: “In long-term trend, NAEP created two separate scales for reading and mathematics ranging from 0 to 500, based on Item Response Theory (IRT). IRT is a set of statistical procedures useful in summarizing student performance across a collection of test exercises requiring similar knowledge and skills. All NAEP subject area scales are produced using these procedures.”

Economists and others who conduced and reported their analyses regarding the relationship between education expenditures and students achievement very well understand dollars. When it comes to understanding of and attention to the statistical and psychometric particulars of student achievement data, however, that expertise seems to be absent. This “imbalance” has led time and time again to mistaken conclusions followed almost certainly by condemnation throughout the media about how poorly students in public schools are performing.

It is worth repeating, public schools in Idaho and across the nation should not be expected to do the impossible, nor should they be condemned when they fail the impossible.