Snapshots Vs. Trends in School Testing
Saint Louis’s Paideia Academy, a charter school, is set to close its doors following a recent defeat in a court battle with Missouri’s Board of Education, which rejected the school’s charter application earlier this year. The Post-Dispatch reports that the Board of Education, in rejecting the application, and the Cole County Circuit Judge, in upholding its decision, cited poor management, the lack of a sponsor, and low test scores as reasons to revoke the charter. Although I am not in a position to speak about the quality of management, or about the lack of a sponsor (which certainly seems like a valid reason to revoke a charter), I do, however, object to the “low test score” argument on two grounds.
First, although it is true that Paideia’s test scores rank among the lowest in the state, absolute measures of test scores are not a very meaningful measure of school quality. The production of education is similar to the production of anything else in the economy: Poorer quality inputs, in the form of poorer students from historically disadvantaged ethnic backgrounds, translate to poorer quality outputs, in the form of test scores. It’s not only a mistake, then, to compare Paideia’s students to those of high-performing districts, but also to an arbitrary benchmark determined by the state. Taking a snapshot of test scores is not enough, because a reliance on mere glimpses into time discourages an understanding of the underlying trends at work. The more important measure is the longitudinal one: Are Paideia’s students learning more now than they were before the school existed? Perhaps the answer is no, but it doesn’t look like this question was considered by either the Board of Education or the Cole County Circuit Judge.
Second, I am willing to believe that we may overvalue test score measures of all kinds. One-size-fits-all models don’t work in schools, where abilities and interests vary greatly between student populations. Schools that produce less significant test score gains but more significant “creativity” gains may still be cultivating meaningful human capital.
Did/could Paideia make the claim that test scores did improve? I couldn’t find it in the P-D article. If they could, I would say that closing it down is a mistake; going from D- to D+ is improvement and should be rewarded/compensated as such, perhaps even more than a B- to B+.
Yes, testing may be given too much weight, but you’ve got have some way to measure the student from Moberly vs. Webster vs. Paideia to see what they know. With grade inflation and other subjective measures, every student ’should’ on the honor roll. (Favorite part of the book ‘Bonfire of the Vanities’: Teacher ‘We grade students from cooperative to life-threatening’ Reporter ‘Was he an honor roll student on that scale?’) Intelligence is a tough thing to define, but making sure that students have received certain knowledge is in the interests of everyone.
Comment by Papillon — July 16, 2010 @ 1:40 p.m.
I strongly object to the characterization of “poorer students from historically disadvantaged ethnic backgrounds” as “poorer quality inputs.” It’s simply offensive.
By what measure are you gauging a student’s “poor[ness]?”
I make no claims to an understanding of logic or logical reasoning, so I could be wrong to point out that the question “Are Paideia’s students learning more now than they were before the school existed?” does not acknowledge a rather basic constraint: Were the students in question even students prior to the school’s chartering and operation?
My view? We ask too much of schools. There’s no quick fix or single formula for success or failure in life or in society. We can’t look at policies in a vacuum. And we lose far too much when reducing persons and lives into a concept as abstract as an “input.”
Comment by Thomas Duda — July 19, 2010 @ 11:10 a.m.
Tom,
That’s a fairly broad critique of the entirety of educational economics, and actually of most of the rest of economics. What’s warrant as to why abstraction is bad?
Comment by Eapen Thampy — July 19, 2010 @ 12:12 p.m.
1.) You make a fair point about the offensiveness of the poorer quality input analogy. Conceded.
2.) The constraint you refer to was one that I implicitly assumed when asking my question. It seemed frivolous to me to ask instead: “Are Paideia’s current students – who were previously students at other schools – learning more (relative to to the value-added distribution for their grade level) now than they were before the school existed?”. I assumed that this degree of clarity was unnecessary and trusted the reader to intuit certain basic constraints. Perhaps, that too was my mistake.
3.) You’re right; there may be no, or few, absolute truths in debates over educational interventions. Yet, I assert that there are meaningful insights to be gained from looking at education as subject to a production function. When we consider all the inputs into this production function, we find that student attributes matter a great deal. Characteristics like neighborhood effects, peer effects, family attributes exist on a quality spectrum and often, these characteristics crystallize in patterned ways in students. When I referred to certain students as “poorer quality inputs”, I was suggesting that these were students functioning as a nexus where poorer quality neighborhood, peer, and family effects crystallize.
Abstracting students and their characteristics will not lead us toward pure and ultimate truths in the economics of education. It can, however, furnish a meaningful paradigm to test interactions with specific educational interventions so we may arrive at general, useful conclusions about how education reform operates for different populations of students.
So, while there are no ’single formulas’, there are useful general principles and looking at inputs is a way of arriving at these principles.
Finally, your claim that we ask too much of schools is an interesting one. I do not doubt that there are manners in which we overburden schools, yet I also know that many schools are notoriously opposed to changes in institutional frameworks (a discussion for another day) that promote the kind of competition and incentives that are conducive to significant educational innovations. Certainly, that’s not too much to ask.
Comment by Abhi Sivasailam — July 19, 2010 @ 12:13 p.m.
What’s *the* warrant?
Comment by Eapen Thampy — July 19, 2010 @ 12:13 p.m.