"That's How We Do It In Government"

At a recent meeting of the curriculum committee, a Hillsboro School District official presented a statistic that looked something like this:

• Mean test score: 73%
• Margin of error: 9%
• Adjusted score : 82%

Now, for any of us in engineering or other professions that use margins of error, this looked distinctly odd. A "margin of error" represents the imprecision in a measurement, and inherently can show uncertainty in either direction. So a 73% with a 9% margin of error represents a range of 64%-82%. Why does it make sense to add the margin of error to the mean score? The answer, when I raised the question: "That's how we do it in government."

This is a nice trick: it enables every statistic to be presented in the best possible light for promoting the success of public officials. It also is inherently insane, in my opinion, granting bonus points for the imprecision of the measurement. Normally, measurements with lower margins of error are seen as more valuable, as they give a clearer and more precise picture. But look at the scores above: if they had a better sampling and lowered their margin of error, the "adjusted score" would actually be penalized! And if they know the true scores are going down, they can game the system by aiming to increase the margin of error rather than improving student knowledge. Is this the right way measurements should be done in our education system?

I can see how this would become the custom in government: once one official does it, everyone else has to follow suit, or else their statistics would appear inferior. Imagine if the district suddenly stopped "adjusting" these scores. "Look, in Hillsboro the scores went down 9% this year!" Any elected officials involved would see their opponents demagogue the issue, and the employees who stopped the adjustments would suffer for it.

Don't take this post as a criticism of the particular official (Travis) who made this presentation though: in fact, I am commending him. In a regime where this silly "adjusted score" must be produced, the most intellectually honest policy is to do what he did: present the actual source numbers in addition to the final adjusted score, and let the viewers see the full story. I'm happy to see our district doing this.

The big lesson: any time a government body reports an "adjusted" statistic, look very closely at the adjustment.

　

　

Back To The Blogosphere

After an absence (from this blog, not from life!) of a couple of years, I've decided to start blogging again on education & related issues. Why, you might ask? Several reasons:

• Having a daughter who just entered kindergarten (at the Carden Cascade Academy), I continue to be very interested in issues involving children and education.
• While I'm not sure I have the stamina for another School Board election race after my hair-pulling 1% loss in 2009, I have become involved on some appointed committees: the Washington County Commission on Children and Families (WCCCF), as well as the Hillsboro School District's Citizens' Curriculum Advisory Committee (CCAC).
• I also have continued to produce my free educational math podcast, Math Mutation, which you can find on iTunes (or at http://mathmutation.com).
• Writing my thoughts helps to clarify them. On issues I don't fully understand, maybe a blog conversation can help me to figure them out. Maybe with a well-crafted response, you can even bring me around to your point of view.
• And naturally there is the usual bloggers' ego motivation, where I imagine that out of the 304897812037 blogs on the net, mine will contain the occasional brilliant insight that will enrich the lives of all readers. It would be selfish of me to deny this service to humanity.

I can't guarantee that you'll like everything I write, though it all will make sense to me, at least for the time it takes to write a blog. Hopefully you'll occasionally find it interesting enough that you'll be motivated to post a reply. Please do so, even if it is to disagree with me. I hope to hear from you!