Monday, April 1, 2013

Demographics Explain KS Test Score Advantage

Many opponents of school vouchers in Kansas point towards NAEP (National Assessment of Educational Progress) reports that show KS students outperforming TX students as evidence that Kansas public schools aren't failing their students.  They hope that's where most people will stop when looking at their data, because if someone digs a little deeper, they might find the exact opposite is true and start drawing some less "comfortable" conclusions.

Not only does KS not beat TX, but they don't beat them in any single demographic.


Bob Weeks of explains this phenomenon as "Simpson's Paradox".  It exists when the value of data is complete lost or drastically altered when examined in the aggregate.  

In this case, white students Texas, in every grade level, met or exceeded performance by white students in Kansas.  The same is true for Texas' black students, who out performed their black counterparts in Kansas, and Hispanic students in TX who outsmarted KS Hispanic students.  Even Asian and multi-ethnic students in TX out performed their ethnic groups in KS.

So what makes up the difference?  Weeks explains it comes down to numbers.  In Kansas, whites make up 66% of the population.  In Texas, they are just 33%.  Since whites score higher than all other ethic groups, excluding Asians, and even though Kansas's white students score lower than Texas', there are more of them and it is giving Kansas an overall score advantage.

So when someone looks at the basic NAEP data, what they are seeing isn't a superior educational system at work, it is the apparent benefit of having what many would consider a less ethnically diverse population.

No comments: