My feelings on the DOE report (cross posted from my blog)

I missed this when it first came out, but I've been seeing the repercussions all over the web. Recently the DOE released the findings of a study they'd done on the effectiveness of software in reading and mathematics. The basic finding was that software does not help. There are news articles and blogs that go into the details, but I recommend reading the primary source, or at least the executive summary.

The educational blogosphere has been yelling about this study and there feelings have been echoed by ISTE. The major claims are that the study is flawed, too narrow, and it doesn't take professional development (or lack of) of the staff into account. Many technology educators also complain that this study is contradicted by other studies they've read.

Before I launch into my rant core argument I should tell you a little about my background. I teach science and I am a self professed technology nerd. If you've read any of my past blog entries then you know this already. My educational background is varied, but my highest degree is a Masters of Science in Ecology and Evolutionary Biology (and ABD on my PhD). Anyone whose been to grad school as a science major will have spent quite a bit of time critically reading scientific research studies. You look for flaws in statistics, controls, experimental design, and conclusions drawn from the data presented.

I came to teaching high school later. As a part of my certification coursework I had to write a number of papers. My topic of choice typically was the effectiveness of technology in education. Being a technology nerd this was a logical choice, plus I wanted fodder for grant applications and justifications for my technology requests I could show to my principal.

Unfortunately I don't have my references if front of me right now. I'd promise to post them in a day or two, but every time I've done that in the past I've never done it. If there is a request then I will produce my references (or if I get really ambitious this week I'll do it without a request).

Overall I've read a number of studies on the effectiveness of technology in education. Unfortunately almost all of the studies had serious flaws. The most common flaw in educational research projects is a lack of a valid control. For example, when a new technology is being looked at for effectiveness, teachers who are given the new technology are also given significant professional development and are taught new teaching strategies that are supported by the technology (PBL, inquiry). They are then compared to a random collection of other teachers who have not been given any PD. In many cases the control teachers went through their teacher training before these strategies, which do not require technology, were shown to be effective. In these studies the gains in student performance are both statistically significant and significant enough that a lay person can't help but see the value.

In the one or two studies I've read where there were valid controls (i.e. pd for both the experimental and control groups) the gains are either not there or were very minor. In some areas the gains were statistically significant, but the magnitude of those gains was very small. The bottom line I walked away with was that technology is not the answer, but staff development is. Technology is simply another tool that a good educator can use to help their students.

Now, one might assume that I wouldn't support the wide spread use of technology in education. This assumption would be wrong. I recognize that technology does not make teachers better educators, but I still think we need to use it as much as possible. As many people point out we are preparing our students for a world that doesn't exist yet. Whatever awaits our students when they graduate from college, one thing can be sure, technology will be a part of it. Now, here comes my theory, and by this I mean wild conjecture that is not supported by any research.

By using the latest technologies in the classroom we expose or students to all the tools that are out there today and by doing so they should be more comfortable using the technology tools that well be developed in the future. We talk about how to prepare students to be responsible members of society (core democratic values) and we teach with writing across the curriculum and expect all teachers regardless of content area to follow along. Why shouldn't we expect all teachers to use all the tools at their disposal rather than only those they had twenty years ago?

Views: 48

Comment by Steve Hargadon on April 11, 2007 at 9:11am
Great, thoughtful post.

I am struck, though, by the degree to which the read/write web is a technology change that is so dramatically altering of the status quo. The changes from pen to typewriter to PC didn't fundamentally alter the nature of communication, but the read/write web does.

So, while I fully support your conclusions about the PD actually making the difference (and it is a great point), I personally believe that the actual technology here is going to produce substantive changes. Of course, then, it should be recognized that the PD becomes even more important (just not the central feature).
Comment by Steve Dickie on April 11, 2007 at 10:58am
I agree that the read write web seems to have the power to substantially change the way our society works and that we need to teach kids to use it. It does not follow, however, that test scores will increase.

There's lots of anecdotal evidence that using technology in the classroom enhances student interest and engagement and that this should lead to higher test scores. This assumes the educators involved can capatilize on that increased engagement.

As I said in my post, regardless of wheter test scores are boosted by the use of technology we need to be using it in the classroom if we want to prepare our students for the future.
Comment by Barbara on April 11, 2007 at 11:00am
I know some might argue that this is just semantics but I am troubled by the phrase in the report "a study of educational technology" when what they really mean is a study of specific educational software. One blog, it may be in the link above, actually lists what software applications were studied.

I have to say I am not a big fan of educational software... it can be just as limiting as a textbook.... I agree that PD is essential but the PD we are talking about has to do with a whole knew approach to knowledge and learning...
but I digress.

My main concern is that the terminology of the study leads to all encompassing conclusions and the terminology does not reflect the true nature of the study or its conclusions. The conclusion that some of the software is not very effective...a conclusion that is probably very valid with or without PD.
Comment by Geoff St. Pierre on June 26, 2009 at 1:48pm
First of all ... well done Steve.

I agree with your points. Technology is just a tool. If you do not know how to use it or have the wrong one it is no good.

I love technology and I love math. I teach computer science and mathematics. In both I take multi-directional approach. I use web 0.0 (paper and pencil). Web 1.0 just one way communications and Web 2.0 including student publishing, responding, and collaborating with CMS, Blogs and Wikis.

In a subject like math even if web2.0 changes the way we communicate there are still and have always been problems with typesetting science and mathematics on computers. A Ph D candidate or graduate student of course can use LaTeX or something like that, but my 10th Algebra II students are not going to learn LaTeX just to answer a couple of open response quesions on a blog or a wiki.

Choose the right tool for the job. Use technology when appropriate and make it fun.

again well done.
Comment by Geoff St. Pierre on June 26, 2009 at 1:52pm
Just to illustrate the math typesetting the wiki does not support <math> tags.


You need to be a member of Classroom 2.0 to add comments!

Join Classroom 2.0


Win at School

Commercial Policy

If you are representing a commercial entity, please see the specific guidelines on your participation.





© 2021   Created by Steve Hargadon.   Powered by

Badges  |  Report an Issue  |  Terms of Service