I Know It When I See It
Autonomous agents don't develop taste. They execute the taste someone was disciplined enough to write down. A worked example from a real pipeline.
By Geordie Everitt
To illustrate the word destroy, I wanted a child knocking down a tower of blocks. Every four-year-old on Earth has done this; it is arguably the purest expression of the concept available to a human being. The image model refused. A small figure, an act of demolition, no matter how gleeful — it read the scene as a child committing violence and declined to render it. So the "destroy" card in a language-learning episode ended up depicting a sandcastle succumbing to a wave, which is fine, and also a lie about what destruction feels like when you are four.
This is where we actually are with the machines. Not marvelling that the dog can talk. Expecting the dog to exercise judgement — to decide, unsupervised, whether a child knocking over blocks is pedagogy or assault, and to be right. That is the whole promise of the word "agentic": a system that doesn't just generate content but understands enough about why the content works to make the call itself. We have moved the goalposts from competence to taste. And taste, it turns out, is a much stranger thing to put in a machine than competence ever was.
The one agentic corner of an unglamorous pipeline
LinguaMama, my language-learning project, runs a content pipeline that is mostly not agentic at all. It is a traditional workflow — script, translate, caption, illustrate, stitch — that happens to use generative models as the labor at each station. There is exactly one station where something resembling judgement has to happen, and it exists for a boring, budgetary reason.
The images come from a cheap diffusion model, Cloudflare's flux-2-klein-9b, running about a tenth of the cost of the model I would use if money were no object. At that price you accept a certain failure rate. Roughly one image in ten comes out wrong in some structurally interesting way: a car rendered as an exterior when the scene called for the view from the passenger seat, a figure with the wrong number of fingers, an oil-painting style that reaches for the wrong reference and renders the human characters as cats. Someone — or something — has to catch these before they ship to a child learning the word for "car."
So after each image is generated, a second model looks at it and decides whether it passed. A vision model inspects the picture and returns a verdict: clean, or a list of specific defects, each with a severity. If the verdict is bad enough, a third step rewrites the prompt to target the diagnosed flaw, re-renders, and inspects again — a bounded loop, three attempts, fail loud on any error. That loop is the agentic corner. It looks, it judges, it acts, it looks again.
Except the judgement isn't where you'd think.
The taste is in the list
The vision model is not sitting there being tasteful. It is handed a catalogue — ten named failure modes — and told to check the image against each one and nothing else. I didn't type that catalogue. I taught it: I gathered a pile of the cheap model's failures from an early episode, handed them to a more capable model, and had it distil the wreckage into rules. The taste was mine — I chose which images counted as broken — and the wording is a stronger machine's account of that choice. Each entry is a fossil of a specific disaster. Here is one of the ten, verbatim — the exact description the inspector receives for a mode called character-style leakage:
The rendered figures do not match the named character references for the scene — the style or a stray reference has leaked the wrong identity in. For example, the oil-painting style pulls abandoned anthropomorphic-cat references and renders humans as cats or animals. Flag any figure whose identity, species, or appearance diverges from the named characters the scene specifies.
Read that the way a stranger would. It is production engineering prose — a specification, evaluated against every image the pipeline ships — and it contains the phrase "renders humans as cats or animals," written in earnest, as a hazard to be guarded against on a Tuesday afternoon. The absurdity is not a flaw in the description. The absurdity is the judgement. Somebody had to notice that the cheap model, left to itself, turns people into cats — and then get that noticing turned into a rule a machine could apply a thousand times without blinking. The other nine entries have the same shape: a camera fault I came to know as "the clown car" — a toy vehicle shot head-on with an oversized driver crammed inside — a car sitting perpendicular to its own lane in the "about to be T-boned" pose, the extra fingers, the melted faces.
That catalogue is the judgement. The model executes it.
The system prompt makes this explicit in a line that gives the whole game away. It instructs the inspector: do not comment on artistic taste, mood, composition, or colour beyond what the catalogue describes. The machine is forbidden its own taste. It is not permitted to like or dislike the image. It is permitted only to check the image against a list of things a human singled out, in advance, as unacceptable — and to report matches. What looks like machine judgement is human judgement, externalized into a taxonomy and delegated to a tireless clerk.
This is the honest shape of almost every "agentic" system I have built. The autonomy is real; the loop genuinely runs without me. But the taste it enforces is mine, written down, made explicit enough that a model can apply it a thousand times without getting bored or getting ideas.
Potter Stewart never wrote his list
In 1964, faced with defining hard-core pornography, Justice Potter Stewart declined. He offered instead the most quoted evasion in American law: "I know it when I see it." It is, when you sit with it, an infuriating thing for a judge to say — a man whose entire job is to convert judgement into written, appealable, portable standards, announcing that on this particular question he would supply a verdict and withhold the reasoning. Trust the vibe. It knows.
The LinguaMama QA loop is the exact opposite gesture, and that is the whole point of it. It refuses to let the judgement stay a vibe. It forces "this image is wrong" to become "this image matches failure mode object-orientation, severity egregious, note: the car is perpendicular to the road." A finding you can read, argue with, port to another episode, and file a bug against. The discipline of building the thing is the discipline of forcing the judgement into writing — of dragging every "I know it when I see it" into a sentence a stranger could apply and get the same answer.
The Hays Code was a written list. The FCC's seven dirty words were a written list. Every content standard that has ever survived contact with an appeal has been a written list, precisely because "I know it when I see it" cannot be appealed. You cannot argue with a feeling. This anxiety about who judges, and by what standard, is old — the novelty is only that we are now handing the clerk's job to a model.
The bill comes due at the guardrail
Which brings me back to the child and the blocks. The diffusion model's refusal to render a demolished tower is judgement too — but it is Potter Stewart's kind. Somewhere in that model's training is an unwritten conviction that a small figure plus an act of destruction equals violence, and it applies that conviction with total confidence and no catalogue you can inspect. It is the same reflex that bleeps the word "kill" out of a documentary, shadow-bans a video because bronze nude statues stood in the background of a park in Spain, and insists that an episode about body parts feature everyone fully clothed. The machine knows obscenity when it sees it. It will not tell you what it saw.
You cannot debug a vibe. That is the entire difference between the two systems, and it has nothing to do with carbon versus silicon. My taxonomy is contestable because it is written; if I decide a toppling tower is pedagogy, I edit line one and the clerk complies. The platform's guardrail is not contestable, because it was never written down — it is a conviction wearing the costume of a rule. When it is wrong, and it is often wrong, there is no line to edit and no bug to file. There is only the shadow-ban, and the silence, and the fully-clothed anatomy lesson.
So as the models gain judgement — and they are gaining it — the question worth asking is not whether they have taste. Of course they do; a tenth of my images prove it, badly. The question is whether the taste they act on is legible: written, attributed, editable, wrong in ways you can name and fix. Steering AI, in the end, is the unglamorous discipline of turning "I know it when I see it" into a list somebody else can read. The systems that do it will be arguable. The ones that don't will just know things about you, and never say what.
Published under the name Geordie.