Education Week's Andrew Trotter reports ($) that the U.S. Department of Education (ED) will not reveal its findings on the effectiveness of educational software products even though ED paid $10 million to evaluate them.  Instead, ED will report only aggregated data, which means "you end up with a study [in which] it's not clear what it's studying," according to a researcher for one of the products being evaluated.

There are a couple of ironies here.  First, NCLB's goal is to improve teaching and learning.  ED has information that could help teachers teach and students learn.  But it's not being released.  As Jerry Weast, the superintendent points out, suppressing the detailed information about specific products is equivalent to telling teachers "We know what may work, but we're not going to tell."  Second, the Bush administration pushed for disaggregation of data by race, ethnicity, income and other categories as a way to get more accurate and useful data under NCLB.  But, in this case, where ED has more accurate and useful disaggregated data on products, it is refusing to release it. 

How useful are the suppressed results of this taxpayer-funded research? The researchers have "impeccable credentials in methodology," according to one of the peer reviewers.  The study measures individual student growth -- students were tested at the beginning and the end of the school year.  The study also uses random assignment, which Trotter calls the "gold standard" for education research.

In failing to report the results for specific products, ED has denied teachers useful information.  In a sense, this is reverse alchemy, with ED turning "gold standard" research into something much less valuable.

If there's no gold here, there is a silver lining: Trotter reports that "results will be broken out product by product" next year.  But if it's good enough for next year, it's good enough for this year.  The sooner students can gain access to curricula with proven effectiveness, the better.