Thursday, April 24, 2014

PISA Scores Say American Schools Are Failing, But Are They?

In a blog post by the National Association of Secondary School Principals (NASSP) they argue American public schools aren't failing, that it is poverty that drags down America's scores.
PISA results have provided ample fodder for public school bashers and doomsayers who further their own philosophies and agendas by painting all public schools as failing. For whatever reason, the pundits, many of whom have had little or no actual exposure to public schools, refuse to paint an accurate picture of the state of education.
A closer look at the data tells a different story. Most notable is the relationship between PISA scores in terms of individual American schools and poverty.  While the overall PISA rankings ignore such differences in the tested schools, when groupings based on the rate of free and reduced lunch are created, a direct relationship is established.
Here is the data they use to show how aggregation affects America's rankings:
A more accurate assessment of the performance of U.S. students would be obtained by comparing the scores of American schools with comparable poverty rates to those of other countries. 
Schools in the United States with less than a 10% poverty rate had a PISA score of 551.  When compared to the ten countries with similar poverty numbers, that score ranked first.
CountryPoverty RatePISA Score
United States10551
Finland3.4%536
Netherlands9.0%508
Belgium6.7%506
Norway3.6%503
Switzerland6.8%501
France7.3%496
Denmark2.4%495
Czech Republic7.2%478
In the next category (10-24.9%) the U.S. average of 527 placed first out of the ten comparable nations.
CountryPoverty RatePISA Score
United States10%-24.9%527
Canada13.6%524
New Zealand16.3%521
Japan14.3%520
Australia11.6%515
Poland14.5%500
Germany10.9%497
Ireland15.7%496
Hungary13.1%494
United Kingdom16.2%494
Portugal15.6%489
Italy15.7%486
Greece12.4%483
Austria13.3%471


















As we have mentioned before, PISA scores are one of those stats where we see aggregation providing a distorted measures (Simpson's Paradox), so on that one point we find ourselves in agreement with the NASSP.   But, there may be a more accurate predictor than poverty levels.

For example, if one were to break these scores down by ethnicity, which the NASSP has but rarely makes public, you will see the US is ranked in the top 2 or 3 in every ethnic category. They typically don't release the ethnicity breakdowns out of fear they will be used to allege there is some kind of intellectual superiority with regards to race, an idea KC's own Emanuel Cleaver, who owes over a million dollars in unpaid federal small business loans, says is racist.

The fact is IQ test after IQ test reveal Asians and Jewish people have average IQs higher than Caucasians, Caucasians have average scores higher than Hispanics and Hispanics have higher average scores higher than blacks. It's not a rule, many factors could be coming into play, it is just the reality of testing scores. It might be interesting to see those scores with household income taken into account, but again that's not data they seem willing to provide publicly.

Many have alleged these scoring discrepancies are a result of tests favoring one ethnicity over another, so-called disparate impact... Maybe, I don't pretend to know why, only that the anomaly exists.

As a result, when you compare the US's aggregate PISA scores to mostly homogeneous countries like China, Japan, the Netherlands, Switzerland, and others you get the impression we are being out performed. But when you break through those scores and compare PISA scores on like ethnicity you find the US rising to the top in almost every category.

The NASSP uses the poverty rate to make a similar claim as we have, they just are doing so by avoiding the race issue and to defend the performance of most public schools.  But, as you can imagine most individuals attending poverty stricken schools are those living in inner city neighborhoods, which tend to be largely minority.

The funny thing and what runs counter to the NASSP's motivations for pointing out this issue is it is those poor schools that tend to get the most funding per child and not just by a little bit. So why are they failing students? Clearly the amount of money being spent is not the cause of good outcomes and arguing we should be spending more is an exercise in insanity and proof that middle and upper income households are not tilting education spending in their favor by paying higher property taxes.  The KCMO school districts receives dramatically more total funding per student than schools in Johnson County, yet JoCo has a near perfect completion rate where Kansas City school children are lucky of one out of 2 graduate.

We've had some interesting discussions here about analytics with regards to university outcomes. One study that gets thrown around a lot in higher ed is that those who live on campus in resident housing tend to do better than those who don't. So many schools have responded by expanding housing and try to get more students in. But instead, what people are finding is that because housing space is limited on the vast majority of campuses, those who get housing tend to be the motivated go-getters that will naturally do better in school than the procrastinators because, to put it simply, they work harder.

Could the same not be said of families living in low poverty vs high poverty areas? Do schools perform better in low poverty areas because they have more money at home or because they are more motivated to learn, have more parental and family involvement, and and have a more favorable view about their ability to guide their own future?

No comments: