The Effectiveness of Software Designed to Detect AI-Generated Writing: A Comparison of 16 AI Text Detectors

In short

On its own set of first-year composition essays and chatbot essays, three of 16 AI text detectors scored well throughout, while most others struggled to separate student essays from those of a newer chatbot.

What it looked at

The 16 publicly available detectors were tested on essays generated by earlier and later versions of a chatbot and on student essays from a first-year composition course, produced with no AI help. Measures were overall accuracy, accuracy per document type, uncertain answers, false positives and false negatives.

What it means for teaching

Results differed a lot between tools and chatbot versions, so a flag from any one tool needs care; this rests on a single test set.

Limits

The human essays came from a single first-year composition course and the machine texts were generated for the test, so results may differ for other courses, prompts or later tool versions.

Cited 149 times by other papers, as counted by OpenAlex in November 2026. A count measures attention, not quality.