Teachers may be one step closer to avoiding the chore of grading essays. Twenty more robots seem willing to take on the job, even if they're not too good at separating fact from fiction yet.
One of the 20 is even promising enough to have won the $60,000 first prize for the three-man team of computer scientists who created it in the competition for a better automated essay-grader.
In April, InnovationNewsDaily reported the William and Flora Hewlett Foundation held a contest for a better automated essay-grader. The winning team consisted of particle physicist Jason Tigg of London, semantics analyst Stefan Henss of Germany, and National Weather Service engineer Momchil Georgiev of Washington, D.C.
In general, 20 of the 250 competing teams produced robo-graders that, by one measure of performance, worked better than anything on the market today, said Tom Vander Ark, whose company, Open Education Solutions, managed the contest.
"The whole purpose of this is to encourage more writing in state tests and as a result, more writing in classrooms," Vander Ark, CEO of the digital learning consulting company, told InnovationNewsDaily. If computer programs make it faster and cheaper to grade essays, then high-stakes state tests can include more essay questions. Supporters say such changes should motivate teachers to teach kids the deeper thinking needed to write papers.
None of the programs submitted to the contest are ready to go to market, Vander Ark warns. If they were analyzed by several measures instead of just one, they may not perform better than currently available graders. Some commercial companies have turned to contest winners for advice, however, and may eventually hire some winners, Vander Ark said.
The winning programs don't address what critics say are the greatest flaws in robo-readers. Essay-grading programs can't determine, for example, if facts presented in an essay are true. Even obviously false statements, such as "President Obama was born in 1990," will get a pass.
Robo-graders also do poorly in assessing the logic in an argument. The programs look for keywords, transition words and varied vocabulary, so they can give high scores to a student who drops in the right words in an illogical jumble.
Such problems are probably too difficult for computer science right now, Vander Ark said. He imagines next-generation robotic readers might be able to deal with them.
Nevertheless, even flawed programs can help teachers assign more writing and students see more writing feedback, Vander Ark maintains. He envisions a writing class in which students write short exercises of 400 or 500 words a day, instead of being restricted to just two or three big essays over an entire semester. Grading programs can provide feedback on sentence structure, grammar and word choice, he said. That frees teachers to address logic and style.
Next, the Hewlett Foundation and Open Education Solutions will hold a contest for people to write a short answer-grading program.
Big-purse prize contests work well to push technology forward, Vander Ark says. "This is a good example of prizes and their ability to focus and accelerate innovation. This is just the first of, I think, many competitions that will produce predictive algorithms that will improve personalized learning."
Follow InnovationNewsDaily on Twitter @News_Innovation, or on Facebook.