Ten days ago OpenAI put a chat window on top of GPT-3.5 and gave it to everyone for free. A million users in five days. The site spends half its time at capacity. Stack Overflow has already banned its answers. My mother — who two Christmases ago identified a GPT-2 bot at the dinner table in three seconds flat — called to ask if this is what I “do.” Mom, I’ve been trying to tell you since 2020.
\nI’ve written about these models for almost three years, so consider this my scorecard post: what ChatGPT actually is, what I predicted, and the miss I want on the record before hindsight launders my memory.
\nUnder the hood this is the GPT-3.5 series — the same lineage API users have had all year — trained with reinforcement learning from human feedback to behave: follow instructions, converse, refuse, stay in a helpful register. (API folks quietly got text-davinci-003 the same week, from the same recipe. The model, on its own, was not the event.)
The event is the packaging. A chat window. Free. No API key, no playground full of knobs, no docs. That’s it. That’s the whole earthquake.
\nIn June 2020 I wrote that nothing in GPT-3 trains the model to want to do your task — you were tricking an autocomplete into usefulness — and that “someone is eventually going to fine-tune one of these giants on the objective ‘actually do what the instruction says’… and I suspect that model will be the one non-nerds end up using.”
\nThat’s this. That’s exactly this. RLHF is the steering wheel, and the non-nerds arrived a million at a time. I’d like this one framed.
\nBut I pictured the steering wheel arriving as an API primitive — a better engine for people like me to build products on. I did not see that the product would be the interface itself. No app on top. The textbox is the app.
\nAnd I should have seen it, because the reasons are things I already knew, arranged in a shape I didn’t:
\nSo: the harness ate the model. The breakthrough people are experiencing isn’t capability — davinci could do most of this in March. It’s that the leash, the interface, and the price all vanished into the product at once.
\nHere’s the thing though. The models I use are confident liars, and everything I ship wraps them in verification — evidence checks, schema validators, state machines that can’t be sweet-talked. That was a niche craft. As of ten days ago, a hundred million keyholes just opened with no lie detector attached.
\nThe evidence arrived fast: Stack Overflow banned ChatGPT answers within the week, and their stated reason is the precise failure mode — answers with a high rate of being wrong that look convincingly right, produced faster than humans can review them. Fluent wrongness at scale. People are starting to call the invented-facts thing “hallucination,” which is a gentler word than the one I’d pick, but fine — the phenomenon finally has a name normal people use, because normal people finally hit it.
\nAnd the refusals — the trained politeness about what it won’t do — are already being talked around. Within days, people found that the model’s boundaries bend under roleplay and framing, which surprises me not at all: those boundaries are trained dispositions, not verified constraints. You can’t gaslight a state machine, but you can absolutely gaslight a vibe. Expect an arms race there for as long as the enforcement lives inside the same model that’s being persuaded — that one’s a prediction, write it down.
\nThere’s no API for ChatGPT yet — my pipelines still run on davinci-003 like it’s the stone age of eleven days ago. When the API does come (it will; the demand is a physical force now), history says it arrives cheaper than what preceded it, and on that day every harness I’ve built — the briefing bots, the PR scribe, the caged dungeon master — gets a brain transplant for one line of config. Building model-agnostic plumbing has never felt smarter than this week.
Meanwhile the model behind the chat window has a 2021 cutoff and no eyes on the world, it makes up citations with a straight face, and it has a verbal tic (“As an AI language model…”) so recognizable it’s practically a signature. Funny detail, that: I once built a party game around a model trying not to be identified. This one is trained to identify itself. The era of the model hiding among us lasted about a year; now it wears a name tag, and the danger inverted — the risk was never that it passes for human, it’s that humans pass along what it says.
\nFor three years the deal was: I poke at models few people touch and report back. That’s over — the audience caught up to the subject overnight, and honestly, good. But watching the reactions this week, I think the caught-up world is about to relearn, at population scale, the same lessons this blog has been accumulating in miniature: the words are the cheap part; the memory, the state, the verification, the leash — the cage — is the product. Yesterday that was my weird hobby. Today it’s the industry’s entire to-do list.
\nThe demo era is over. The harness era just started, and I’ve never felt more usefully unoriginal.
","frontmatter":{"title":"The harness ate the model","date":"December 10, 2022","description":"ChatGPT is ten days old, a million people signed up in five, and my mother has opinions about it. Some notes from someone who's been poking this exact bear since 2020 — on what's actually new, what I got right, and the miss I want on the record."}}},"pageContext":{"slug":"/the-harness-ate-the-model/","previous":{"fields":{"slug":"/two-liars-one-dungeon-making-gpt-3-and-stable-diffusion-agree-on-a-world/"},"frontmatter":{"title":"Two liars, one dungeon: making GPT-3 and Stable Diffusion agree on a world"}},"next":{"fields":{"slug":"/how-a-pdf-button-became-remote-code-execution-and-paid-for-my-year/"},"frontmatter":{"title":"How a PDF button became remote code execution (and paid for my year)"}}}}, "staticQueryHashes": ["3435731857","426816048","63159454"]}