<
https://arstechnica.com/information-technology/2023/07/why-ai-detectors-think-the-us-constitution-was-written-by-ai/>
'If you feed America's most important legal document—the US Constitution—into a
tool designed to detect text written by AI models like ChatGPT, it will tell
you that the document was almost certainly written by AI. But unless James
Madison was a time traveler, that can't be the case. Why do AI writing
detection tools give false positives? We spoke to several experts—and the
creator of AI writing detector GPTZero—to find out.
Among news stories of overzealous professors flunking an entire class due to
the suspicion of AI writing tool use and kids falsely accused of using ChatGPT,
generative AI has education in a tizzy. Some think it represents an existential
crisis. Teachers relying on educational methods developed over the past century
have been scrambling for ways to keep the status quo—the tradition of relying
on the essay as a tool to gauge student mastery of a topic.
As tempting as it is to rely on AI tools to detect AI-generated writing,
evidence so far has shown that they are not reliable. Due to false positives,
AI writing detectors such as GPTZero, ZeroGPT, and OpenAI's Text Classifier
cannot be trusted to detect text composed by large language models (LLMs) like
ChatGPT.
If you feed GPTZero a section of the US Constitution, it says the text is
"likely to be written entirely by AI." Several times over the past six months,
screenshots of other AI detectors showing similar results have gone viral on
social media, inspiring confusion and plenty of jokes about the founding
fathers being robots. It turns out the same thing happens with selections from
The Bible, which also show up as being AI-generated.
To explain why these tools make such obvious mistakes (and otherwise often
return false positives), we first need to understand how they work.'
Via Mike Cheponis, Dewayne Hendricks and Dave Farber.
Cheers,
*** Xanni ***
--
mailto:xanni@xanadu.net Andrew Pam
http://xanadu.com.au/ Chief Scientist, Xanadu
https://glasswings.com.au/ Partner, Glass Wings
https://sericyb.com.au/ Manager, Serious Cybernetics
