Carnegie Learning Company Blog : Compelling Article in Response to DOE's Study on Educational Software

Tuesday, May 01, 2007

Compelling Article in Response to DOE's Study on Educational Software

By Dr. Steve Ritter
Co-founder & Chief Product Architect, Carnegie Learning

Henry Kelly, president of the Federation of American Scientists, in Washington, makes one of the most lucid and compelling statements in response to the U.S. Department of Ed’s recent report to Congress that I’ve read.

"The U.S. Department of Education’s recent report to Congress on the effectiveness of reading and mathematics software products sounded a wake-up call to anyone pondering technology in education. ("Major Study on Software Stirs Debate," April 11, 2007.) Its authors conclude that math and reading software produce no better test results than conventional teaching methods.

How can a technology that is transforming the way we acquire information throughout the economy—revolutionizing businesses from games to banking—fail to benefit education? How can technology revolutionizing training in the U.S. Department of Defense fizzle in elementary schools?

The Education Department report is not evidence that technology cannot be a powerful learning tool. It proves only that results on standardized tests are not significantly improved by systems found in a sample set of schools. Moreover, the study focused on whether the technology was better than traditional teaching methods, failing to consider new technology as a productivity tool. Yet its authors admit that the study produced results no worse than for traditional teaching, and that 86 percent to 92 percent of the teachers in the program found the software sufficiently useful to keep using it. Clearly, these teachers perceive a value not registering on the tests."

Read the Entire Article Here: http://www.edweek.org/ew/articles/2007/04/11/32software.h26.html

Technorati Tags: math, math research, high school, Pittsburgh, school reform, Department of Education, education technology, Rand, differentiated learning

Posted By Webmaster At 4:11 PM •

In my classes, the biggest problem with the software is that low-skilled students 'game' the program when they have to write the expression. (These students press hint all the way through, and their 'expression' skill bar doesn't go up.) I see that there is a growing body of research on how to help students who have resorted to consistent gaming, including work by Pittsburgh-area scientists. Is there any plan within Carnegie Learning to incorporate supplemental questions on writing expressions to deter gaming--and to teach the concept to students who aren't getting it?

I think it would be an ideal place to begin to incorporate natural language dialogue, since the dialogue follows the same general pattern every time.

Would it be feasible for Carnegie Learning to incorporate natural language dialogue along these lines for students who are clearly struggling with writing the expression?

Comment By Kevin Hall At 5/14/2007 11:14 PM

Here is the dialogue script I've taken to using in class. It seems like current technology (e.g., PSLC TuTalk) should allow for this to be automated:

(Tutor)'Which number in the problem gets added or subtracted OVER AND OVER?'
(Student) '$8'
(Tutor) 'Great!. What key words told you so?'
(Student) '$8 per hour'
(Tutor) 'Right! So how many times does $8 get added up?'
(Student) 'However many hours there are'
(Tutor) 'Right! So how will you write that into your equation?'
(Student) '8h'
(Tutor) 'Great! How do you know to treat the 8 in 8h as a positive 8, not a negative 8?'
(Student) 'It's adding on to my money, not taking away from it.'...(etc, etc)

Comment By Kevin Hall At 5/14/2007 11:17 PM

Kevin,

Thanks for the comments. I agree that we have some work to do on writing the expression. We are working closely with the researchers to figure out the right solution. Natural language is one way to go, but it is still pretty hard to get right. There are always many variants of the right expression (including ones that aren't exactly grammatical), so it can get hard. Another approach is a structured tool that can help the student think about the problem. Part of the research is figuring out what works best.

Comment By Steve Ritter At 5/15/2007 10:47 AM

Comments are locked for this post.