Detect tower · floor

Are ai detectors accurate enough to accuse somebody?

Asked plainly — are ai detectors accurate — the honest answer is that nobody outside the companies selling them can fully say, and that the most rigorous public answer came from a university that used one, tested it, and then turned it off.

The single most useful calculation on this subject is not in a study. It is in a blog post from Vanderbilt University explaining why it disabled Turnitin’s AI detector.

That number is worth holding onto whichever side of this you are on. If you are a student who has been accused, it is the reason a percentage is not evidence. If you are an instructor deciding whether to act on a score, it is the size of the population you would be wrong about.

Vanderbilt gave three further reasons, and they are separate from the accuracy figure — a tool can be accurate and still be unusable for these.

The first of those is the one that matters most in a disciplinary meeting. In every other kind of academic accusation, the evidence can be inspected: a source can be compared, a similarity report can be read line by line. An AI detection score cannot be examined by anybody — not the student, not the instructor, not the institution — because the method is not published. The number has to be believed or disbelieved.

The second is why the accuracy question and the fairness question are not the same question. A detector that is wrong one per cent of the time at random is a different object from one that is wrong mostly about students who learned English as a second language, and only the first is described by the headline figure.

The neighbouring question — how do ai detectors work — has an answer that is short and unsatisfying, and it is the same one Vanderbilt gave: outside the companies, nobody knows. That is also why asking how accurate are ai detectors cannot be settled from outside. Accuracy is measured against a method, and the method is not published.

So what is a defensible position for somebody who has to decide something today?

If you are teaching: a detection score is a reason to look, never a reason to conclude. The conversation that follows — about drafts, sources, the argument in the essay — is the actual assessment, and it was always going to have to happen anyway.

If you have been accused: ask what the score is based on, and ask it in writing. The answer, if it is honest, is that the method is not disclosed. That is not a rhetorical trick; it is the documented state of the tool, published by a university that paid for it.

And if you are writing anything you might one day need to defend, the single most valuable habit costs nothing and no detector can argue with it: work somewhere that keeps a version history. A document with three weeks of edits behind it is evidence of a kind that a probability score cannot touch.

The floors below take it four ways: what a score actually measures, the false positives and who gets them, the individual tools and what each claims, and what to do when an accusation has already been made.

What this tower will not do

It will not tell you a detector is reliable enough to accuse somebody. The most detailed public reasoning on that question was published by a university, and it reached the opposite conclusion.

It will not tell you the sector has settled this. One of the universities that dropped the tool did so because the contract changed, and pretending otherwise would be the same overreach we are objecting to.

And it will not carry an affiliate link. In this tower that means declining some of the highest commissions on the island. What holds instead is simple: the accuracy figure, the arithmetic and the reasons for disabling the tool are quoted from the universities that published them, listed below.