« Pres. Bush Invented NAEP | Main | Guilty Pleasures »

Testing, Testing--One, Two, Three

October 11, 2007 09:11 AM

 

I agree with one aspect of what Kevin Carey at The Quick and the Ed said yesterday about Linda Perlstein's new book, Tested--that education journalists often overreach, taking what they have observed in one classroom or one school and then making sweeping generalizations.  However, I take great exception to his second post on the book.  Carey writes:

Teaching students material that's way above their heads is bad educational practice. And you can't say the incentives built into NCLB leave educators with no other option, because--as Perlstein makes clear--it's also futile.  Even under the current, no-growth-model law, schools teaching students like Whitney have two choices: inappropriately teach at grade level, in which case Whitney fails the test this year and every year after that, or teach at the right level, in which case Whitney fails the test this year but catches up and passes tests in the future.

Wait--so Carey thinks it is educationally inappropriate to teach students above grade level but it's just dandy to test them every year knowing they will fail, because some day they will catch up to grade level and pass such tests? Maybe I am functioning below grade level, because I fail to see Carey's logic.

And, as much as some would like to continue to deny it, it sounds as though Perlstein's book is part of the mounting evidence that NCLB has led schools to focus more instructional time on testing and, in some places, caused a narrowing of the curriculum.  Anyone who talks to teachers on a regular basis hears it too often from too many quarters to dismiss those who protest as nee-sayers, uh, I mean, naysayers.  Can you hear me now? Good.

Update: Carey responds here. I think the nee-sayers would argue that using an out-of-level assessment is the best way to measure what students who are not performing at grade level know and are able to do, whereas using a grade level assessment would not provide useful information.

Comments

But the question remains why didn't Whitney learn how to read by first grade in the first place? Why did she ever fall behind?

If Whitney's teachers had used one of the AFT's recommended reading programs she should have been able to read on time. Teh creator of the program has repeatedly claimed that "[We have] consistently demonstrated that if a reading sequence is properly implemented in kindergarten, virtually all at-risk students with the exception of the profoundly retarded and the very frequently absent will read by the end of the year."

Whitney is neither profoundly retareded nor habitually absent, so why should we hold her to a lesser standard than NCLB's current standard?

I've yet to hear a compelling reason.

The real problem in this discussion is the misrepresentation of what a growth model will measure or do. George Miller commented at one of the hearings months ago that many people believe growth models will be a "silver bullet" to fix the multiple measurement problems associated with NCLB (http://edworkforce.house.gov/hearings/fc032107.shtml). Miller, himself, said that growth models were not a silver bullet. When he asked a panel of experts from several states and even some statisticians if growth models were a silver bullet, they all said, "No."

If you want proof, look to the states that have already implemented growth models. You might ask: How many schools or districts had their AYP status changed when the models were run? Or, even more to the point, how many inappropriate classifications were corrected by the revised calculations? After all, if a growth model offers a clearer picture of student achievement, shouldn't it improve or correct the previous picture? If the classifications were corrected, how did people know the growth model answer was more accurate and not just different?

Back when NCLB started, many people with no understanding of testing drank the sweet cherry Kool-Aid: Proficiency testing at every grade would present an accurate, precise measure of whether students were learning what they should learn. We now have five years of evidence to know that it is not so simple, from cheating in Texas to grade-inflated scoring of open-ended responses in Florida to games played with cell sizes, student assignments, and cut scores. Those mechanical problems don't even get to the larger questions of curriculum alignment, limited content scope of a single test, and other questions of validity. We know without a doubt that the NCLB system is simply too weak to support the kinds of weighty conclusions resting on it. Now, we are being sold growth models as the fix for everything wrong with NCLB. Some of the same people who sold us the first cup are saying, "We know the Cherry Kool-Aid made you sick to your stomach, but you'll love the new Tropical Punch." Well, many of us who had a cup the first time simply aren't interested in more.

There is only one thing that clear-minded, thoughtful educators can do. Whenever anyone says what a growth model will do or explains how a growth model will fix a certain problem with the current accountability system, ask them to be as precise as possible. Tell us exactly how, using a real, approved growth model, under what circumstances a school would get credit for that kind of change in performance and how that change would be reflected in a real accountability measure: Student report, AYP measure, whatever -- some REAL measure. Ask if the model they are describing is one that is already in use. If it is, ask for the data on how it changed the measures they cited. Real examples, not hypotheticals.

For example, is there any growth model currently approved in which a school can get credit for taking a student from, say, 2 points below 4th grade Proficient to 2 points below 5th grade proficient? Assuming the tests are stable, that student has made a year's worth of growth, but is no closer to proficient. Which model gives credit for that and how is that credit reflected in a student or school score? Which model gives a school credit for a student who moves from just barely proficient to the high end of proficient or better? How does the model handle the fact that, as several recent reports have indicated, proficiency cuts are often not even internally aligned from grade to grade, so that talking about whether a fifth grader will be proficient in two or three years is statistically like hitting a hummingbird with a water balloon?

If you want to see if you are getting a complete and honest answer, ask the person which problems in the NCLB accountability system ? the measurement, analysis, and reporting of data requirements -- that a growth model will not fix. If they cannot name a specific weakness that a growth model will not fix, they are either too ignorant or too ?invested? to be speaking on the subject. The researchers at Miller?s March 21st hearing weren?t afraid to answer.

We will very soon have data from many states that tell us exactly what changes if any a growth model will make in improving the truthfulness, or truthiness, of accountability classifications. Shouldn?t we be able to take a look at that data before we decide what those models will and won?t do in terms of fixing NCLB?s fundamental flaws? Wasn?t that the point of having pilots? To answer that question before we embark on a whole new, potentially equally flawed system? Many people have gut feeling based on a rumor that a growth model will fix this broken system. Whatever happened to using data to make decisions?

On the other hand, if NCLB?s accountability system is so seriously flawed that we cannot afford to wait to see the effects of growth models in practice, maybe what we really need is a one or two year testing moratorium. We could take the money saved and see if we can make sense of all of the data we have already collected, using the raw data to try to understand and improve education, not just punish or reward schools. What if we could run five different growth models on 50 states? worth of data without any state having to be afraid of the political consequences? But I digress?

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Categories

Accountability

AFT's Convention

Assessments

Charter

DC Schools

Early Childhood

Ed Tech

General

Higher Ed

Instruction

Labor

Legislation

Media

New Orleans

Paraprofessionals and School-Related Personnel

Privatization

School choice

School finance

School Improvement

Special Ed

Staff Quality

Standards

Teachers' Voices

Vouchers

Archives

March 2008

February 2008

January 2008

December 2007

November 2007

October 2007

September 2007

August 2007

July 2007

June 2007

May 2007

April 2007

March 2007

February 2007

January 2007

December 2006

November 2006

October 2006

September 2006

August 2006

July 2006

June 2006

May 2006

April 2006

March 2006

February 2006

January 2006

December 0000

Recent Posts

There's a Flag (or Several) on the Play

Your SES profits, sir

Edwizdom: "How long can you look at a test score?"

Clarification

What Would Chuckbutt Do?

The NCLBlog

Co-Editor: John
Co-Editor: Michele

Have a tip about NCLB? Contact our tipline at tips@letsgetitright.org.

For questions or general information, email us at info@letsgetitright.org.

Disclaimer: The NCLB Blog was established by the AFT as a forum where public education advocates, policymakers and others can exchange information and express their opinions on NCLB and related issues. The views expressed here are not the official views of the AFT or any of its affiliates. All claims otherwise would violate the spirit and purpose of the blog. © American Federation of Teachers, AFL-CIO. All rights reserved. Photographs and illustrations cannot be used without permission of the AFT.


Home About AFT Blog Sign Our Petition Contact Us Send to a Friend Printer-friendly Page

The NCLB Blog was established by the AFT as a forum where public education advocates, policymakers and others can exchange information and express their opinions on NCLB and related issues. The views expressed here are not the official views of the AFT or any of its affiliates. All claims otherwise would violate the spirit and purpose of the blog. © American Federation of Teachers, AFL-CIO. All rights reserved. Photographs and illustrations cannot be used without permission of the AFT.