Is your all-important essay being graded by a human, or a computer?

Automated essay scoring is now a reality, but not everyone likes the idea of an unthinking robot evaluating their essays. How do these programs work? And can you beat them?

Linguist Daniel Midgley gets into the machine on this episode of Talk the Talk.


Listen to this episode

117: Rise of the Robo-Markers

Download this episode

You can listen to all the episodes of Talk the Talk by pasting this URL into your podlistener.

http://danielmidgley.com/talkthetalk/talk_classic.xml

Not much time for this post because I’m busy marking essays. I guess it’s not a coincidence that I chose this topic, then, because it’s about getting computers to do the marking.

I love this topic because it’s so counter-intuitive. The things we think make a good essay are nothing close to what computers think makes a good essay. And yet, computers make judgments that match human markers to a T.

And hey, marking turns professors into mindless robots, so why not skip the intermediary?


Show notes

An NYT article on automatic essay marking.
http://www.nytimes.com/2013/04/05/science/new-test-for-computers-grading-essays-at-college-level.html

There’s big money in it. Universities are working hard on this.
http://business.curtin.edu.au/news/?objectId=56D3BDD8-E191-70F2-32788060408C5DAF

So are private corporations.
http://www.ets.org/research/topics/as_nlp/

It’s not too difficult to beat them, though.
http://www.nytimes.com/2012/04/23/education/robo-readers-used-to-grade-test-essays.html

This PDF file from expert Marti Hearst is a little dated now, but good on the history.
http://people.ischool.berkeley.edu/~hearst/papers/ieee-essay-grading.pdf

More about Page and the story of automatic essay marking
http://www.historyofinformation.com/expanded.php?id=3999

Download SWAN, the Scientific Writing AssistaNt, and see what advice it has for your writing.
http://cs.uef.fi/swan/index.html