How to read the works
Each case is drawn as two images. The Encounter Plate shows what kind of system it was. The Spectrogram shows what it did to people over time. Every mark comes from the case record, and the same record always produces the same image. The examples below use the first case, SyRI.
The plate shows the system
- 1Pupil. Type of system. Lines mean fixed rules; a swirl, a statistical model; empty, machine learning.
- 2Iris brightness and grain. How well documented the case is. Bright and fine: courts and inquiries have examined it.
- 3Red ring. How certain the causal chain is. Sharp when established, blurred when estimated.
- 4Iris colour. Area of life. Ochre: welfare.
- 5Iris size. Scale of deployment, from local to global.
- 6Red point. Where harm could no longer be undone. Its size follows the number of people affected.
- 7Stars. The people affected, seen at different distances. Not a measure of individual harm.
The plate is drawn like an astronomical photograph of a black hole, which is also an eye. Five properties of the system set its form: the type of system sets the pupil, how well the case is documented sets the brightness and grain of the iris, how certain the causal chain is sets the sharpness of the red ring, the area of life sets the colour of the iris, and the scale of deployment sets its size.
A red point on the ring marks where harm could no longer be undone. Its size follows the number of people affected, on a logarithmic scale. The stars stand for the people affected. Their brightness does not say how badly any one person was harmed. That is not known, and the work does not claim to know it.
SyRI: fixed rules, very well documented, causal chain well established, welfare, national scale, about 135,000 people.
Three kinds of pupil
The spectrogram shows what it did
- 1Seven rows. Kinds of harm, top to bottom: liberty, dignity, employment, family, housing, health, reputation.
- 2Brightness. Severity from 0 to 10. Here reputation stands at 7.
- 3Solid white line. An intervention that stopped the system: the court ruling of February 2020.
- 4Dashed white line. Partial oversight: Rotterdam halts its own use in 2019.
- 5Dotted texture. The period after the intervention, coded again. Recognition is not repair.
- 6Dashed border. Harm caused through other people or institutions, not directly.
- 7Hatching. Too little evidence to score.
- 8Dotted outline. Harm suspected but not measurable.
- 9Fade at the right. The case was still open when it was coded.
- 10Time. Left to right, from 2014 to the coding date.
Time runs from left to right, and the seven dimensions of harm run from top to bottom. Brighter means more severe.
Vertical white lines mark the moments when someone stepped in: a court, an inquiry, a mayor, a regulator. A solid line means the intervention stopped the system; a dashed line means it stopped only part of it. The period after an intervention is drawn with a dotted texture and coded again, so that recognition is not mistaken for repair.
The border of a band shows how the harm was caused: solid when the system caused it directly, dashed when it passed through other people or institutions. Hatching marks a dimension where the evidence is too thin to score. A dotted outline marks harm that is suspected but cannot be measured.
Red marks show harm that no remedy can reverse. SyRI has none: the court stopped the system before any irreversible loss was documented. They appear in the Toeslagenaffaire, Optum and Robodebt records.
Where the record comes from
Each work begins with one case: an automated system, the decisions it made, and the people it reached. The system may use machine learning (what is now usually called AI), a statistical model, or a set of fixed rules. The record says which, because it matters for who designed the decision.
The case is built only from public records: court rulings, parliamentary inquiries, regulators' findings, peer-reviewed research and journalism. Every key claim is checked against at least two independent sources.
Harm is coded on seven dimensions: liberty, dignity, employment, family, housing, health and reputation. Each gets a score from 0 to 10, with separate notes on how strong the evidence is and how directly the system caused the harm. If the evidence is too thin to score, the record says so. The seven scores are never added up.
Rules
- Every visual element comes from the record. The same record always produces the same image.
- No images made with generative AI. Every image is rendered by a deterministic program from the case record.
- People harmed by a system are not named or shown unless they have made their own case public. Children are never identified.
- Cases are coded again when the facts change. Each case page shows when it was coded and when it was last checked.
The full methodology, codebook and case register will be published as an open catalogue.