The harness ate the model

Ten days ago OpenAI put a chat window on top of GPT-3.5 and gave it to everyone for free. A million users in five days. The site spends half its time at capacity. Stack Overflow has already banned its answers. My mother — who two Christmases ago identified a GPT-2 bot at the dinner table in three seconds flat — called to ask if this is what I “do.” Mom, I’ve been trying to tell you since 2020.

I’ve written about these models for almost three years, so consider this my scorecard post: what ChatGPT actually is, what I predicted, and the miss I want on the record before hindsight launders my memory.

What it actually is (less new than the headlines, more new than the cynics)

Under the hood this is the GPT-3.5 series — the same lineage API users have had all year — trained with reinforcement learning from human feedback to behave: follow instructions, converse, refuse, stay in a helpful register. (API folks quietly got text-davinci-003 the same week, from the same recipe. The model, on its own, was not the event.)

The event is the packaging. A chat window. Free. No API key, no playground full of knobs, no docs. That’s it. That’s the whole earthquake.

The receipt I get to cash

In June 2020 I wrote that nothing in GPT-3 trains the model to want to do your task — you were tricking an autocomplete into usefulness — and that “someone is eventually going to fine-tune one of these giants on the objective ‘actually do what the instruction says’… and I suspect that model will be the one non-nerds end up using.”

That’s this. That’s exactly this. RLHF is the steering wheel, and the non-nerds arrived a million at a time. I’d like this one framed.

The miss I want on the record

But I pictured the steering wheel arriving as an API primitive — a better engine for people like me to build products on. I did not see that the product would be the interface itself. No app on top. The textbox is the app.

And I should have seen it, because the reasons are things I already knew, arranged in a shape I didn’t:

• The transcript is state management for civilians. I’ve spent two years carefully assembling context — schemas, retrieval, keyholes. A chat log does that implicitly: your whole conversation is the prompt, managed by the act of talking. Everyone became a context engineer without knowing the word for it.
• RLHF is a harness baked into the weights. Everything I’ve built around these models — the format coaxing, the persona scaffolds, the “answer only yes or no” pleading — was an external harness compensating for a model that merely completes. Training on human preference moves half that harness inside. My prompts have been getting shorter for a year; this is that curve reaching the general public as “it just works.”
• Free means the demo is the distribution. Every previous model hid behind waitlists and per-token meters. This one you send to your mother as a link.

So: the harness ate the model. The breakthrough people are experiencing isn’t capability — davinci could do most of this in March. It’s that the leash, the interface, and the price all vanished into the product at once.

The part that worries the person who builds lie detectors

Here’s the thing though. The models I use are confident liars, and everything I ship wraps them in verification — evidence checks, schema validators, state machines that can’t be sweet-talked. That was a niche craft. As of ten days ago, a hundred million keyholes just opened with no lie detector attached.

The evidence arrived fast: Stack Overflow banned ChatGPT answers within the week, and their stated reason is the precise failure mode — answers with a high rate of being wrong that look convincingly right, produced faster than humans can review them. Fluent wrongness at scale. People are starting to call the invented-facts thing “hallucination,” which is a gentler word than the one I’d pick, but fine — the phenomenon finally has a name normal people use, because normal people finally hit it.

And the refusals — the trained politeness about what it won’t do — are already being talked around. Within days, people found that the model’s boundaries bend under roleplay and framing, which surprises me not at all: those boundaries are trained dispositions, not verified constraints. You can’t gaslight a state machine, but you can absolutely gaslight a vibe. Expect an arms race there for as long as the enforcement lives inside the same model that’s being persuaded — that one’s a prediction, write it down.

Practical notes from the cheap seats

There’s no API for ChatGPT yet — my pipelines still run on davinci-003 like it’s the stone age of eleven days ago. When the API does come (it will; the demand is a physical force now), history says it arrives cheaper than what preceded it, and on that day every harness I’ve built — the briefing bots, the PR scribe, the caged dungeon master — gets a brain transplant for one line of config. Building model-agnostic plumbing has never felt smarter than this week.

Meanwhile the model behind the chat window has a 2021 cutoff and no eyes on the world, it makes up citations with a straight face, and it has a verbal tic (“As an AI language model…”) so recognizable it’s practically a signature. Funny detail, that: I once built a party game around a model trying not to be identified. This one is trained to identify itself. The era of the model hiding among us lasted about a year; now it wears a name tag, and the danger inverted — the risk was never that it passes for human, it’s that humans pass along what it says.

Where this leaves a blog like this one

For three years the deal was: I poke at models few people touch and report back. That’s over — the audience caught up to the subject overnight, and honestly, good. But watching the reactions this week, I think the caught-up world is about to relearn, at population scale, the same lessons this blog has been accumulating in miniature: the words are the cheap part; the memory, the state, the verification, the leash — the cage — is the product. Yesterday that was my weird hobby. Today it’s the industry’s entire to-do list.

The demo era is over. The harness era just started, and I’ve never felt more usefully unoriginal.