My feelings on the DOE report (cross posted from my blog)

I missed this when it first came out, but I've been seeing the repercussions all over the web. Recently the DOE released the findings of a study they'd done on the effectiveness of software in reading and mathematics. The basic finding was that software does not help. There are news articles and blogs that go into the details, but I recommend reading the primary source, or at least the executive summary.

The educational blogosphere has been yelling about this study and there feelings have been echoed by ISTE. The major claims are that the study is flawed, too narrow, and it doesn't take professional development (or lack of) of the staff into account. Many technology educators also complain that this study is contradicted by other studies they've read.

Before I launch into my rant core argument I should tell you a little about my background. I teach science and I am a self professed technology nerd. If you've read any of my past blog entries then you know this already. My educational background is varied, but my highest degree is a Masters of Science in Ecology and Evolutionary Biology (and ABD on my PhD). Anyone whose been to grad school as a science major will have spent quite a bit of time critically reading scientific research studies. You look for flaws in statistics, controls, experimental design, and conclusions drawn from the data presented.

I came to teaching high school later. As a part of my certification coursework I had to write a number of papers. My topic of choice typically was the effectiveness of technology in education. Being a technology nerd this was a logical choice, plus I wanted fodder for grant applications and justifications for my technology requests I could show to my principal.

Unfortunately I don't have my references if front of me right now. I'd promise to post them in a day or two, but every time I've done that in the past I've never done it. If there is a request then I will produce my references (or if I get really ambitious this week I'll do it without a request).

Overall I've read a number of studies on the effectiveness of technology in education. Unfortunately almost all of the studies had serious flaws. The most common flaw in educational research projects is a lack of a valid control. For example, when a new technology is being looked at for effectiveness, teachers who are given the new technology are also given significant professional development and are taught new teaching strategies that are supported by the technology (PBL, inquiry). They are then compared to a random collection of other teachers who have not been given any PD. In many cases the control teachers went through their teacher training before these strategies, which do not require technology, were shown to be effective. In these studies the gains in student performance are both statistically significant and significant enough that a lay person can't help but see the value.

In the one or two studies I've read where there were valid controls (i.e. pd for both the experimental and control groups) the gains are either not there or were very minor. In some areas the gains were statistically significant, but the magnitude of those gains was very small. The bottom line I walked away with was that technology is not the answer, but staff development is. Technology is simply another tool that a good educator can use to help their students.

Now, one might assume that I wouldn't support the wide spread use of technology in education. This assumption would be wrong. I recognize that technology does not make teachers better educators, but I still think we need to use it as much as possible. As many people point out we are preparing our students for a world that doesn't exist yet. Whatever awaits our students when they graduate from college, one thing can be sure, technology will be a part of it. Now, here comes my theory, and by this I mean wild conjecture that is not supported by any research.

By using the latest technologies in the classroom we expose or students to all the tools that are out there today and by doing so they should be more comfortable using the technology tools that well be developed in the future. We talk about how to prepare students to be responsible members of society (core democratic values) and we teach with writing across the curriculum and expect all teachers regardless of content area to follow along. Why shouldn't we expect all teachers to use all the tools at their disposal rather than only those they had twenty years ago?