Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated text

In short

Detector results on chatbot paragraphs about an engineering topic were better for an older model than a newer one, the authors report, and inconsistent on human-written controls, with some false positives.

What it looked at

The authors generated fifteen paragraphs from each of two chatbot versions, plus five human-written control responses, all on one engineering topic. Five AI-writing detectors were run on them and their outputs compared.

What it means for teaching

From a small single-topic test, a reason for caution before reading much into a detector score on any one student's work.

Limits

The test set is very small and covers one engineering topic. The abstract does not say who wrote the human control responses.

Cited 570 times by other papers, as counted by OpenAlex in November 2026. A count measures attention, not quality.