Eight hundred metrics, and no line for access

There is a moment in every school year that nobody writes a policy about. Someone in a management meeting has three quotes on the table and twenty minutes left, and they ask the only question that time allows: which of these is best?

Last week a group of researchers made that question much easier to answer. I am not sure they made it easier to answer well.

What was launched

On 7 October, researchers led by the MIT Media Lab's Advancing Humans with AI programme — with UC Berkeley Haas, USC Marshall's Neely Center and the Psychology of Technology Institute — launched ImpactBench, which they describe as “an expert-guided, open, and evolving evaluation framework for measuring AI's impact on human wellbeing”. Alongside it sits a companion tool with a wonderfully plain name: AI Nutrition Labels.

The idea is exactly what it sounds like. Food carries a label you can read in ten seconds, so AI models should too. Underneath it is serious machinery: more than 800 metrics, 26 benchmarks, three domains of human impact — physical, psychological and social. It looks for harms such as factual hallucination, sycophancy and toxicity, and for good behaviour such as supporting a person's agency rather than quietly replacing it.

“We've set out to build a rigorous, independent measure of how AI shapes human psychology,” said Juliana Schroeder, associate professor of management of organizations at UC Berkeley Haas. Her colleague Pattie Maes of the MIT Media Lab, who co-leads the programme, put the stakes more bluntly: “we are actively losing our critical thinking ability and autonomy”.

I want to say clearly, before anything else: this is good work. An independent measure of whether these systems help or harm people is long overdue. Most of what schools are told about AI tools comes from the people selling them. A label a tired parent can read on a Tuesday evening is a genuinely democratic object.

So I went looking for us

Eight hundred metrics. Twenty-six benchmarks. Three domains. I read through what they cover, and then I did what I always do, which is search for the words that describe the children I work with.

Accessibility. Disabled users. Special educational needs. Not addressed.

The framework tests models in conversation with teen and adult personas. It promises to expand later to communities, cultures and languages. Disability is not named as one of the things coming.

What a label actually does

Here is why I am not letting this go, and it is not pedantry about a word count.

A nutrition label is not a research finding. It is an instrument of procurement. It is the thing a busy person points at when they have three quotes and twenty minutes. And the moment a score exists, it stops being one input among many and becomes the input — because it is the only one that is legible at speed.

So consider what that means in practice. A model that is wonderful for a confident fourteen-year-old reader and completely unusable for a child navigating by switch access will score identically to a model that works beautifully for both. Not lower. Identically. The difference between those two products is invisible to the label, which means it is invisible to the meeting, which means it is invisible to the budget.

Whatever a label measures becomes what counts. Whatever it leaves out quietly stops existing.

We have watched this happen before, with less sophisticated instruments. A school is judged on attendance, so the child who cannot physically tolerate the building becomes a problem of attendance rather than a problem of design. A subject is judged on written output under time pressure, so a student who knows the material and cannot produce it at speed becomes a weaker student rather than a badly measured one. The metric does not merely describe the world. After a while, it builds it.

This is a pattern, not an accident

What unsettles me is how ordinary this omission has become. Look at the last fortnight in AI and education.

Free AI training was offered to every teacher in Ireland and the UK: accessibility not mentioned. A new assessment tool scored edtech vendors against AI privacy law: not mentioned. England reopened its decade-old teacher CPD standard: not mentioned. TED and Google opened a global call for educators' ideas about AI in teaching: not mentioned. And now the measuring instrument itself, the thing meant to hold all of it to account: not mentioned.

I do not think anyone decided this. That is rather the point. Nobody sat in a room and voted to leave disabled learners out of the specification. It simply never came up, because the people in the room were imagining a default user, and the default user they imagined was not a child who processes language slowly, or who cannot hold a pen for forty minutes, or who has learned that asking for help costs something.

This is the whole argument for universal design, compressed into one week of news. You do not get inclusion by adding it at the end. You get it by changing who you picture when you write the first line of the spec. Change the environment, not the child — and a measurement framework is part of the environment now.

What to ask before your school buys anything this year

You will be offered scores this year. Some of them will be good scores from honest researchers. Use them. But do not stop there.

Three questions, and they take about a minute between them.

First: what does this score not cover? Ask it out loud, in the meeting, so it is on the record. A good vendor will know the answer. A nervous one will change the subject.

Second: show me this working for the hardest case in the building. Not a demo with a confident reader. The actual child, or a fair stand-in — screen reader, switch, text-to-speech, no reading at pace, no typing at pace.

Third: who in this school will notice if it stops working for that child? Because tools degrade quietly, and the children most affected by a tool failing are very often the children least likely to come and tell you.

If the answer to the third question is nobody, then you have not bought a tool. You have bought a risk, and handed it to the person with the least power to refuse it.

The thing I keep coming back to

I like the nutrition label metaphor more than I expected to. But food labels took decades to get right, and they got better because people who had been left out of them insisted, repeatedly, often tediously, that their bodies also counted. Allergens went on the label because somebody would not stop asking.

So this is me asking, early, while the specification is still soft and the people writing it have said it is open and evolving. Put a line for access on the label.

Inclusion by presence is not inclusion. Inclusion by omission from the measuring instrument is not even presence.

Source: Berkeley Haas News (Laura Counts), 7 October 2026 —

Researchers launch new tool measuring AI's impact on human wellbeing

ImpactBench is published by the MIT Media Lab's Advancing Humans with AI programme with UC Berkeley Haas, USC Marshall's Neely Center and the Psychology of Technology Institute.