Thoughts on online writing scoring

Replies to This Discussion

Permalink Reply by Jayashree Iyer on February 24, 2017 at 12:50am

It will not be that accurate and lenient to evaluate. 

• ▶ Reply

Permalink Reply by NIcole Cianelli on March 4, 2017 at 9:40am

I think automated scoring would not be as accurate. One of the hardest part of grading writing is that it is subjective. Even if a rubric is used, two people grade differently. Although a program would be easier, it may not be necessarily give a score you would agree with. It would actually create more work because you may end up double checking the scores.

• ▶ Reply

Permalink Reply by Bo Cheli on March 15, 2017 at 10:50am

I think it's important to start with the question "what am I trying to assess?" Some programs are designed to spot and penalize poor grammar, syntax, spelling and other mechanics. Others know to look for transition words and likely thesis statements. Chances are, though, a HS English teacher is looking for much more than this. For example, evidence of critical thinking; effective and appropriate use of logos, ethos, and pathos in a persuasive argument; seamless use of source material. Auto-graders will struggle with that, though some claim they won't.

One way to see the limits and possibilities is to check out a description of Pearson's Writing Space auto-grading feature. They make bold claims about what it can do, but you can't write your own prompts and would need to choose from a bank. So, there are trade-offs. 

Here is their description (link to page):

Auto-graded writing assignments use artificial intelligence to assess your students’ writing assignments. These assignments are specific to your course, require students to submit their work through Writing Space, and are automatically graded. Writing Space not only checks spelling and grammar, but evaluates content and quality using the grading rubric specific to the question. This may sound miraculous, but the artificial intelligence behind Writing Space is what enables Writing Space to assess content in your students’ essays:

• Subject matter experts create writing prompts, and then write specific questions that instructors across the country assign to their students.
• After 600–800 essays are completed per question, subject matter experts hand-grade them.
• Each hand-graded essay and its grading rubric are fed into Writing Space, enabling the system to develop intelligence around what makes a well-written or a poorly written essay.
• Writing Space goes beyond recognizing key words and phrases and actually understands language and the nuances of meaning.

• ▶ Reply