Automated grading FAIL: Computer-generated essays earn A+ marks
"Act is intrepidly and clandestinely axiomatic, by most of the scenarios." That's one sentence from a computer-generated essay created with only three keywords: "Fair Elections Act." If that sounds like gobbledygook to you, well, you're only human. But to a standard essay-marking computer program, that gibberish is brilliant prose. And writing instructor Les Perelman has a problem with that....
"Act is intrepidly and clandestinely axiomatic, by most of the scenarios." That's one sentence from a computer-generated essay created with only three keywords: "Fair Elections Act." If that sounds like gobbledygook to you, well, you're only human. But to a standard essay-marking computer program, that gibberish is brilliant prose. And writing instructor Les Perelman has a problem with that.
Mr. Perelman, who is a researcher at the Massachusetts Institute of Technology, created the program to demonstrate the fallibility of computerized essay marking, which is increasingly being used in standardized tests in U.S. elementary and high schools.
The Basic Automatic B.S. Essay Language Generator, or Babel for short, generates a complete essay from one to three keywords. When Carol asks Mr. Perelman to input "Fair Elections Act," the result is even more mystifying than an answer in Question Period.
"Fun, fair, forward and herein presumably never will be altrusitic, in the extent to which we purloin the analysis," reads another sentence from the paper.
But when Mr. Perelman fed the essay into a standardized computer grader, it scored in the 90th percentile -- essentially giving it an A-plus mark.
For more quotes from the computer-generated essay on the Fair Elections Act -- and to hear Carol's conversation with Les Perelman, click the "Listen" link above.