#Automated essay grader software

Fallon, who asserts that his company’s offerings are able not only to score student writing, but also to give those students feedback for improvement. “A lot of it comes down to, the more submissions we get, the smarter the engine gets,” said Mr. That includes projects like those at Carnegie Mellon, as well as those at his own company. John Fallon, the vice president of marketing with Vantage Learning, said that using current policy momentum-including the drive for the creation of new, more writing-intensive assessments-will only help drive improvements in all realms of natural-language-processing study. “I think it gets caught up between what machine learning is aiming for and what is commercially feasible,” Mr. Commercial vendors involved in the study did not possess a similar background in studying student interaction, perhaps because they couldn’t afford to do so from a business standpoint, he said. A computer-generated persona interacts as one of several participants in an online discussion, asking questions of the students and at times even interjecting humor into a tense situation among students involved in the discussion.Ĭreating an automated essay grader based on that research came out of a curiosity to see whether the researchers’ methods of evaluating student discussion could transfer to assessment of student composition, said Elijah Mayfield, a doctoral candidate in language and information technology working with Ms. Rose, an associate professor of language technology and human-computer interaction.įor example, one project involves using artificial intelligence to drive discussions on an online platform provided by the university to secondary students in the 25,000-student Pittsburgh public schools.

#Automated essay grader software

Their examination of natural language processing, or the science of how computers interact with human language, has focused on the idea that software could help students hold more-productive collaborative discussions about any range of academic subjects, said Carolyn P. “It’s possible that an administrator will say, ‘I’m just going to throw it all to the computer,’ … but that’s not what we would ever recommend.”įurther, one entrant in the study, the LightSIDE software developed by Teledia, a research group at Carnegie Mellon University in Pittsburgh, was created as an extension of research its developers say is only loosely related to automated essay graders.

“It’s designed to be a support, so that a teacher can focus him- or herself completely on inspiring composition of writing or creative composition of writing,” he said. Two vendors in the study-the Princeton, N.J.-based Educational Testing Service and Vantage Learning, with headquarters in Yardley, Pa.-already have offered for most of the past decade software that gives students some basic feedback on the grammar, style, mechanics, organization, and development of ideas in their writing, Mr. Mark Shermis, the dean of the college of education at the University of Akron, in Ohio, and a co-author of the study, said the paper doesn’t even touch on the most exciting potential of automated essay graders, which is not their ability to replace test scorers (or possibly teachers) with a cheaper machine, but their ability to expand upon that software to give students feedback and suggestions for revision. “I think the claims being made about the study wander a bit too far from the shores of our data,” he said. Cohen of AIR cautioned that interpretation could be too broad. All six states are members of one of two state consortia working to develop assessments for the new standards.īy and large, the scores generated by the nine automated essay graders matched up with the human grades, and in a press release, study co-director Tom Vander Ark, the chief executive officer of Federal Way, Wash.-based Open Education Solutions, a blended-learning consulting group, said, “The demonstration showed conclusively that automated essay-scoring systems are fast, accurate, and cost-effective.” SOURCE: “Contrasting State of the Art Automated Scoring of Essays: Analysis”Įach developer’s software graded essays from a sample of 22,000 contributed by six states, using algorithms to measure linguistic and structural characteristics of each essay and to predict, based on essays previously graded by humans, how a human judge would grade a particular submission. A recent study examined essay-grading software developed by the following organizations: