What a detector actually detects

The detectors are being put down

An AFP report published on 5 October says US universities are stepping back from AI-detection software as the main evidence of cheating. Yale and Cornell have banned or discouraged staff from relying on it. The University of Wisconsin–Madison discourages its use, and its faculty guidance puts the reason in writing: “They are imperfect at best, carry the risk of false positives, have been shown to be biased against non-native English speakers, and will not prevent students from using these tools.”

David Deming, Dean of Harvard College, was blunter still: “A different approach is needed. We would all benefit from getting out of the AI-detection business.” Timothy Caulfield, a misinformation researcher quoted in the same report, described where this has got to: “There is a bit of a technology arms race happening: the detectors versus the evaders.”

Read that one line again

“Biased against non-native English speakers.” I have been waiting for an institution to write that down, because it is the sentence that turns a procurement decision into an equality one.

Here is what a detector is actually doing. It is not recognising dishonesty — it has no access to dishonesty. It is measuring how far a piece of writing sits from a statistical average of fluent, idiomatic, native-speaker prose. Writing close to that average scores as human. Writing at a distance from it scores as suspect. That is the whole mechanism, and once you see it, you can see exactly who it will come for.

The pupil who writes in short declarative sentences, because that is how their thinking arrives on the page. The pupil using a word bank, a sentence starter or a writing frame they are entitled to and have been taught to use. The pupil working through text-to-speech, whose spoken register is flatter and more regular than their written one would have been. The pupil who learned English at eight. The pupil who has been drilled so hard in a structure that their essays all come out the same shape. Not one of them is cheating. Every one of them is sitting at a distance from the average, and the tool cannot tell the difference — because there is no difference, in the only thing the tool can see.

What an accusation costs

We talk about false positives as if they were a technical inconvenience, a bit of friction to be tuned down in the next release. They are not. An accusation of cheating is one of the heaviest things a school can put on a young person, and it lands hardest on the pupils least equipped to lift it off.

Think about what defending yourself actually requires. You have to notice the accusation is unfair, read the room well enough to know that pushing back is allowed, find the words under pressure, and advocate for yourself in front of an adult who has already formed a view. That is a demanding social and communication task on a good day. For an autistic student, or a student with a language disorder, or one who has learned that adults are not reliably on their side, it can be close to impossible — and silence under questioning reads, to an adult expecting a defence, exactly like guilt.

So the harm is not evenly distributed in either direction. The pupils most likely to be flagged are among the pupils least able to get the flag removed. That is not a software problem. That is a design decision we would be making on their behalf.

Oral defence is the right instinct — if it is designed

The replacement described in the AFP report is assessment redesign: oral defence of submitted work, paired with the written piece. I think that is genuinely the better direction, and I want to be careful about how we get there, because “come and explain your essay to me” can be a ramp or it can be an ambush depending entirely on how it is set up.

Designed, it looks like this. Everyone does it, not just the suspected — the moment it becomes a sanction, it stops being assessment and starts being an interrogation. Notice in advance, with the questions' shape known even if the questions are not. A choice of format: aloud to the teacher, recorded at home, annotated in writing, walked through with notes in hand. No time pressure that is not actually assessing something. And a way to demonstrate understanding that does not require performing confidence, because confidence is not the construct and never was.

Sprung, it is simply the old viva with a new justification, and it will sort the cohort by who copes with being put on the spot. We will have replaced a biased machine with a biased moment.

The question to ask before you buy

If a detection product is being put in front of your school, the question is not how accurate the vendor says it is. Accuracy claims are measured against a population, and your concern is not the population. The question is: who will this be wrong about in this building, and what happens to that child on the day it is wrong?

Ask for the accessibility documentation. Ask what the false-positive rate looks like for students writing in an additional language, or using assistive technology, or working from a scaffold. Ask what the appeal process is, who runs it, and what a pupil who cannot argue their own case is supposed to do. If the answers are thin, you have learned what you needed to know in about a minute, and it cost you nothing to ask.

The design question underneath

There is a reason this story belongs on a universal design desk rather than an academic-integrity one. A detector is an attempt to solve an assessment problem by surveillance instead of by design. It leaves the task exactly as it was — one route in, one route out, one narrow channel through which understanding has to be squeezed — and adds a machine to watch the channel.

Change the environment, not the child. Here that means changing the assessment rather than installing something to suspect the pupils: more than one way to show what you know, tasks where the thinking is visible as it happens rather than only in the finished artefact, and a teacher who has the time and the information to notice a difficulty rather than infer a deception. That is more work than buying a licence. It is also the only version of this that does not quietly cost us the children we are supposed to be designing for.

Elaine Walsh is the founder of Autism & Me (autismandme.ie), a Waterford-based neuroinclusive education consultancy. She writes here on Universal Design, inclusion and Irish education, from the twin vantage points of the classroom and the kitchen table.