![\"Folder](\"https://docs.docker.com/engine/swarm/images/swarm-diagram.png\")
It was almost midnight and I was thirty seconds from closing the laptop.
\nI’d been poking at $COMPANY’s creator dashboard for about an hour, not because I expected to find anything, but because it was the kind of tired where you keep clicking things instead of going to bed. I’d made a test campaign with a nonsense headline — october-proof-6f4d9, the sort of string you generate when you want to be able to grep for it later — and I opened a private window to check something completely unrelated.
The homepage loaded. My nonsense string was sitting in the middle of it.
\nMy first thought wasn’t I found something. My first thought was that I’d screwed up my own test setup. That’s usually what it is. You think you’re in a clean browser profile and you’re not, or you’ve got a service worker caching something stale, or you left a devtools override on from two hours ago. I’ve embarrassed myself this way before.
\nSo I checked. Different browser: still there. Then I dropped to curl, because curl doesn’t have opinions:
\nGET / HTTP/1.1\nHost: www.$COMPANY.example\nUser-Agent: curl/8.4.0\nAccept: text/htmlNo cookies. No session. Nothing that could possibly identify me. And there it was in the server-rendered HTML:
\n<section data-module=\"creator-spotlight\">\n <h2>october-proof-6f4d9</h2>\n</section>I scrolled up to the response headers, and that’s the point where I actually sat back in the chair.
\nCache-Control: public, s-maxage=300, stale-while-revalidate=60\nAge: 19\nX-Cache: HITMy text wasn’t just on the homepage. It had been cached there. Some CDN node had taken a copy and was handing it out to whoever asked.
\nThe one thing keeping me calm was that the string was inert. Plain text, nothing else. I hadn’t put a script in a globally visible campaign, and by that point in the night I understood enough about what I was looking at to know that I never would. Because the other half of what I’d found — the half I’d stumbled into twenty minutes earlier and hadn’t fully processed yet — was that the homepage rendered campaign headlines as raw HTML.
\nI ended up proving the two halves separately, and never together:
\nTest A — global placement\n audience: ALL\n content: plain text only\n result: marker appeared on anonymous homepage and CDN cache\n\nTest B — script execution\n audience: two accounts I controlled\n content: redacted execution canary\n result: JavaScript ran in the real homepage origin\n\nNever performed\n audience: ALL\n content: executable payloadA global placement primitive and a stored XSS primitive lived inside the same object. I didn’t need to combine them to know what the combination meant: anyone who loaded the homepage could have run attacker-controlled JavaScript under $COMPANY’s origin.
The session cookie was HttpOnly. I’ll come back to how little that helped.
Some context on the product. $COMPANY is a consumer platform — profiles, messages, public collections, the usual shape — and it has enough traffic that its homepage has quietly become a product in its own right rather than just a door.
Most signed-in users never actually type the URL. They arrive from a notification or a shared link or the mobile app. But anonymous visitors do, and search traffic does, and logged-out users do, and anyone opening the site on a desktop after not using it for a week does. All of that funnels through the same discovery page, which is assembled from a stack of server-driven modules:
\nTop stories\nContinue where you left off\nPopular near you\nCreator spotlight\nRecommended collectionsThe contents shift by locale, experiment bucket, and whether you’re signed in. The HTML shell around them is cached hard.
\nI hadn’t gone looking at the creator dashboard for any of this. I was there for something else entirely — an unrelated idea about image uploads that went nowhere. But the campaign editor caught my eye on the way past. It let a creator build a little promotional card for their own profile: headline, short description, image, button.
\n┌──────────────────────────────────────────────┐\n│ Notes from building small language models │\n│ A short collection of experiments. │\n│ Read more │\n└──────────────────────────────────────────────┘Nothing about it looked dangerous. I typed some HTML into the headline field out of habit, the way you do, and it came out the other side as visible text. React escaping it, exactly as you’d hope.
\nSo I made a draft and watched the request go by:
\nPOST /api/creator/v2/campaigns HTTP/1.1\nHost: www.$COMPANY.example\nContent-Type: application/json\nCookie: __Host-session=<redacted>\nX-CSRF-Token: <redacted>\n\n{\n \"headline\": \"October systems notes\",\n \"description\": \"A small collection of experiments.\",\n \"image_id\": \"img_test_17\",\n \"cta\": {\n \"label\": \"Read more\",\n \"url\": \"/u/researcher/notes\"\n },\n \"surface\": \"CREATOR_PROFILE\",\n \"audience\": {\n \"type\": \"FOLLOWERS\"\n },\n \"status\": \"DRAFT\"\n}Two fields didn’t belong:
\n\"surface\": \"CREATOR_PROFILE\"\n\"audience\": { \"type\": \"FOLLOWERS\" }The dashboard had never offered me a choice about either. There was no placement dropdown, no audience selector. And yet the client was dutifully sending both values on every request, which almost always means the same thing: the server is accepting a generic object shared by more than one frontend, and this frontend just happens to only fill in the boring parts.
\nThat’s the moment where a boring evening turns into a late one.
\nI pulled down the JavaScript bundle and searched for CREATOR_PROFILE. It had company:
const CampaignSurface = {\n CREATOR_PROFILE: \"CREATOR_PROFILE\",\n FOLLOWING_FEED: \"FOLLOWING_FEED\",\n HOME_DISCOVERY: \"HOME_DISCOVERY\",\n HOME_HERO: \"HOME_HERO\",\n SEARCH_EMPTY_STATE: \"SEARCH_EMPTY_STATE\"\n};And a second enum right below it:
\nconst CampaignAudience = {\n FOLLOWERS: \"FOLLOWERS\",\n USER_IDS: \"USER_IDS\",\n LOCALE: \"LOCALE\",\n ALL: \"ALL\"\n};Reading between the lines, these looked like they existed for an internal marketing console and for QA previews. Somebody’s editorial team needs to put a card on the homepage; somebody’s QA needs to target two specific accounts. Reasonable features. The creator dashboard was built on the same API types and simply chose not to render the dangerous options.
\nWhich left one question, and it’s the question I now ask about basically every product with more than one client: does the server know the difference between a creator and an editor, or does it only know the difference between the UIs?
\nI kept this deliberately small. One field changed, campaign left in draft, audience restricted to two accounts I owned:
\n{\n \"headline\": \"october-proof-6f4d9\",\n \"description\": \"plain text placement test\",\n \"image_id\": null,\n \"cta\": null,\n \"surface\": \"HOME_DISCOVERY\",\n \"audience\": {\n \"type\": \"USER_IDS\",\n \"ids\": [\"user_test_a\", \"user_test_b\"]\n },\n \"status\": \"DRAFT\"\n}I genuinely expected a 403. Instead:
\n{\n \"id\": \"cmp_01HCG4Q5DAJ2\",\n \"owner_id\": \"user_test_a\",\n \"surface\": \"HOME_DISCOVERY\",\n \"audience\": {\n \"type\": \"USER_IDS\",\n \"ids\": [\"user_test_a\", \"user_test_b\"]\n },\n \"status\": \"DRAFT\",\n \"review_required\": false\n}I read review_required: false three or four times.
There was an authorization check. It worked. It confirmed that I owned this campaign, which I did. What it never asked was whether I was allowed to point the campaign at the homepage. The backend was doing something along these lines:
\nasync function createCampaign(user, input) {\n requireCreatorAccount(user);\n\n return db.campaigns.insert({\n ownerId: user.id,\n ...input,\n });\n}Ownership enforced. Field authority not enforced at all.
\nThe logic underneath is seductive and wrong in a way I’ve now seen in a dozen codebases. A creator is allowed to create campaigns, therefore the campaign input is creator-controlled, therefore every field on it is creator-controlled. But surface, audience, reviewRequired, and the scheduling behaviour weren’t content. They were distribution policy. They decided who the object was aimed at, not what it said.
The UI hid them. The API trusted them.
\nI activated it for my two test accounts and opened the real homepage, signed in as each one. There was my marker, sitting in the Creator spotlight module on the actual production discovery page.
\nAt that point I thought I had a decent authorization bug with a limited blast radius. Worth reporting, not worth losing sleep over. Then I looked at how the module was rendering the headline.
\nOn the creator profile, the campaign rendered the way you’d expect:
\n<h2>{campaign.headline}</h2>Text is text. Angle brackets stay angle brackets. This is why my earlier HTML-in-the-headline poke had come back inert — I’d tested the safe surface and drawn a conclusion about the whole system.
\nThe homepage used a completely different pipeline. Its API didn’t return a headline and a description. It returned a server-driven component with an HTML fragment inside it:
\n{\n \"type\": \"creator_spotlight\",\n \"campaign_id\": \"cmp_01HCG4Q5DAJ2\",\n \"html\": \"<div class=\\\"spotlight-copy\\\"><h2>october-proof-6f4d9</h2></div>\",\n \"tracking\": {\n \"surface\": \"home_discovery\",\n \"experiment\": \"spotlight_v3\"\n }\n}And the React side was, near enough:
\nfunction ServerDrivenModule({ module }) {\n return (\n <section\n data-module={module.type}\n dangerouslySetInnerHTML={{ __html: module.html }}\n />\n );\n}The React team named that property well. It’s hard to type it by accident.
\nBut here’s the part that made this interesting rather than merely bad: the team had not blindly trusted creator input. There was a sanitizer. Someone had thought about this. The problem was where they’d put it.
\nconst template = DOMPurify.sanitize(`\n <div class=\"spotlight-copy\">\n <h2>{{{headline_html}}}</h2>\n <p>{{description}}</p>\n </div>\n`);\n\nconst html = Mustache.render(template, campaign);The template gets sanitized. The template is safe — of course it is, it’s a hardcoded string written by an employee. The untrusted data isn’t in it yet.
\nThen Mustache runs. Triple braces mean “insert without escaping.” The sanitizer had already looked at a harmless placeholder, given it a thumbs up, and gone home. Mustache then swapped that placeholder for whatever a creator had typed.
\nsanitize(template)\n ↓\ninsert(untrusted_data)\n ↓\ninnerHTMLWhich needed to be, at absolute minimum:
\ninsert(data)\n ↓\nsanitize(final_html)\n ↓\ninnerHTMLOr better, and this is the version I argued for later:
\nstructured data\n ↓\nReact elements\n ↓\nno raw HTML sinkI tested the mildest thing I could think of first:
\nOctober <em data-proof=\"6f4d9\">systems</em> notesOn the creator profile it displayed as literal text with the tags visible. On the homepage, “systems” came out in italics, and the data-proof attribute was sitting there in the DOM when I inspected it.
My data had crossed the line from text into markup. That’s not the same as script execution — HTML injection and XSS are cousins, not twins, and I’ve seen people conflate them in reports and get downgraded for it. But it meant the boundary was gone, and everything after it was a question of degree.
\nThen I noticed the CSP:
\nContent-Security-Policy-Report-Only:\n default-src 'self' https: data: blob:;\n script-src 'self' 'unsafe-inline' https:;\n report-uri /csp/reportReport-only. It would file a complaint. It would not stop anything.
\nThis is the section I’ve rewritten the most times, because it’s the part that actually mattered and it’s the part that’s easiest to make sound self-congratulatory.
\nI had two things. I could put text on a globally cached homepage. I could get HTML into a sink that would execute it. Combining them was a single JSON field away — change USER_IDS to ALL, hit activate, and I’d have the most complete proof-of-concept anyone could ask for.
I want to be honest that the thought was appealing for about four seconds. Not because I wanted to hurt anyone, but because an airtight demo is easier to report than an argument. You don’t have to convince a triager of anything if you can just show them.
\nBut the demo would have run in strangers’ browsers. Real people, on a real homepage, on a Tuesday night, having done nothing but open a website. There is no version of “it was only a harmless canary” that survives contact with the fact that I would have been executing code in other people’s sessions without their knowledge. And the cache meant I couldn’t take it back — I’d have had to wait for a purge I didn’t control.
\nSo I split it, and did each half in the smallest way I could think of.
\nExecution first, audience of two. I kept the campaign restricted to the accounts I owned and replaced the italics with a redacted browser-execution canary. It did exactly two things: wrote a random token into a data-xss-proof attribute on the page, and requested a one-time image URL from a server I control containing the same token. It didn’t touch cookies, local storage, messages, profile data, or any API response. It didn’t persist. It didn’t modify an account. The working markup isn’t in this post; it went to $COMPANY privately and nowhere else.
I opened the homepage as user_test_b and looked at the root element:
<html data-xss-proof=\"9b7e2c1a\">And my server log:
\n00:18:43 GET /xss-proof/9b7e2c1a.gif HTTP/1.1\n Referer: https://www.$COMPANY.example/\n User-Agent: Mozilla/5.0 ... Chrome/118.0 ...I did it again in Firefox with a fresh token, because one browser is an anecdote. Then I deleted the campaign and confirmed both test accounts were getting clean homepage HTML again.
\nPlacement second, inert content. New campaign, plain text only:
\noctober-global-placement-14c8Audience ALL. Activate, load the anonymous homepage once, deactivate. The whole window was under thirty seconds and I had the deactivation request queued up before I sent the activation.
The marker showed up in:
\nAgeAnd then — this is the bit that made my stomach turn slightly — it kept showing up after I’d deleted the campaign. The source object was gone. The rendered copies weren’t. They stayed until the CDN objects expired or got purged, and neither of those was something I could trigger.
\nordinary creator account\n │\n │ client-controlled surface + audience\n ▼\nglobal homepage campaign\n │\n │ raw HTML inserted after sanitization\n ▼\nstored XSS in www origin\n │\n │ public CDN cache\n ▼\nevery browser receiving that homepage objectGlobal placement: proven with inert text. Script execution: proven with a private audience. The combination was no longer a technical question, and turning it into one would have been an unsafe act rather than a research step.
\nSo I stopped, and started writing.
\nWhile I was drafting, I checked the session cookie, half-hoping for something that would soften the impact.
\nSet-Cookie: __Host-session=<redacted>;\n Path=/;\n Secure;\n HttpOnly;\n SameSite=LaxThat’s a well-configured cookie. __Host- prefix, Secure, HttpOnly, SameSite. Somebody read the guidance and followed it. My canary had confirmed the session value simply wasn’t present in the script-visible cookie string.
It helped much less than it looks like it should.
\nThe injected script wasn’t running near https://www.$COMPANY.example/. It was running as part of it. Which means when it made a request to any other path on that origin, the browser attached the session cookie by itself, without asking anyone, exactly as designed. And because the response was same-origin, the script could read it.
The cookie stayed secret. Its authority was available on request.
\nIn deliberately non-exfiltrating pseudocode:
\n// Blocked by HttpOnly:\nconst rawSession = document.cookie;\n\n// Still fully authenticated by the browser:\nconst response = await fetch(\"/api/account/me\");\nconst myOwnTestAccount = await response.json();I ran the second one exactly once, against my own account, and the response contained a user ID that was already rendered on the page in front of me. I logged it to the console and it went nowhere. That was enough to establish the model:
\n\n\nXSS doesn’t need to steal an HttpOnly cookie when it can make the victim’s browser use it.
\n
SameSite=Lax was similarly beside the point. SameSite governs whether cookies ride along with cross-site requests. This script wasn’t cross-site — it was executing on the application’s own origin, which is the one place SameSite has nothing to say about.
CSRF tokens weren’t a boundary either. A script inside the origin can read tokens out of the DOM or out of an API response and then submit requests in exactly the shape the real application uses. There’s nothing to forge when you’re already inside.
\nThe cookie flags eliminated one category of theft. They did nothing about the authority of code that’s already in the page.
\nI want to be precise here, because this is where reports get sloppy and where I think researchers do real damage — both to their credibility and occasionally to users.
\nI tested one read-only request against one account I owned, and then stopped. I did not go endpoint-hunting. So the report separated what I’d actually confirmed from what a reasonable person should assume follows:
\nConfirmed\n- Any ordinary creator account could choose a homepage surface.\n- The same account could choose an ALL audience.\n- Active homepage campaigns entered a public CDN cache.\n- Campaign headline HTML was inserted after sanitization.\n- JavaScript executed in the production www origin.\n- Same-origin authenticated requests succeeded as an owned test user.\n\nNot tested\n- Reading private messages or saved content\n- Changing email, password, or recovery settings\n- Creating access tokens\n- Accessing payment information\n- Impersonating users in comments or messages\n- Persistence through service workers or other browser storage\n- Targeting staff or administrator accountsThe plausible impact, subject to whatever authorization and reauthentication each endpoint enforces, is the standard same-origin script list: read what the current user can read, do what the current session can do, read any non-HttpOnly browser storage for the origin, rewrite the page to harvest credentials or anything else typed into it, post as the user, and — the one that matters most — catch whichever privileged employee happened to open the homepage that morning.
That last point is why placement mattered more than the injection itself.
\nStored XSS on a creator profile needs the victim to visit that profile. Stored XSS in a DM needs them to open the conversation. Stored XSS in a homepage module needs them to use the product normally. There’s no suspicious link to squint at, no attachment, no click to regret. The delivery mechanism is the page they trust most.
\nThe campaign service fired an event on activation:
\ncampaign.activated\n → home-composer consumes event\n → locale-specific home document regenerated\n → CDN caches rendered documentAnd the homepage cache key looked roughly like:
\nhome:v4:en-US:anonymous\nhome:v4:en-US:signed-inLocale plus a coarse auth state. No campaign owner in the key — and that’s correct, because the content was supposed to be shared. A public homepage should be cacheable. This wasn’t a cache bug.
\nWhat the cache did was change the shape of the vulnerability:
\nThe browser had no way to know the HTML started life in a creator dashboard. The CDN had no way to know it contained user-controlled markup. The home renderer had no way to know the creator should never have been able to select its surface.
\nEvery component received something that looked completely valid coming from the component directly upstream. The bug lived in a sentence nobody had written down anywhere:
\n\n\nA creator-owned campaign may not become trusted homepage HTML.
\n
I filed at 01:07 UTC, which explains some of the sentence construction in the original draft.
\nTitle:
\nCritical: ordinary creator can publish stored XSS to globally cached homepageI’ve learned to make the summary fit on one screen, because the first person to read it is triaging on a phone:
\nPrecondition\n Any account eligible to create a creator campaign.\n\nAuthorization failure\n POST /api/creator/v2/campaigns accepts internal-only `surface` and\n `audience` values from the creator client without a server-side allowlist.\n\nXSS sink\n Homepage renderer sanitizes the Mustache template before inserting\n `headline_html` through triple braces, then returns the result to a React\n `dangerouslySetInnerHTML` sink.\n\nDistribution\n HOME_DISCOVERY + audience ALL is rendered into anonymous and signed-in\n homepage documents and cached publicly at the CDN.\n\nImpact\n Attacker-controlled JavaScript can execute for homepage visitors under\n the primary application origin. HttpOnly prevents direct cookie reads,\n but the script can still issue and read same-origin authenticated API\n requests with the victim's session.\n\nSafety\n Global distribution was tested with plain text only. Script execution\n was tested only against two owned accounts. The two conditions were never\n combined. No user data was accessed.Attached: account IDs, campaign IDs for both proofs, full request and response pairs, the redacted payload in a private file, screen recordings, CDN headers from two regions, timestamps for every create/activate/deactivate and how long the cache held on afterwards, a suggested emergency kill switch, and remediation notes.
\nAcknowledged at 01:19. Then at 01:31, an engineer asked the question I’d been expecting and slightly dreading:
\n\n\nDid you ever make executable content visible to audience ALL?
\n
I appreciated that they asked directly instead of dancing around it. I answered:
\n\n\nNo. ALL received a random plain-text marker only. Executable content was limited to two user IDs I control. The report demonstrates each required property independently.
\n
That was the whole exchange. No follow-up, no suspicion. But it’s the reason I structured the testing the way I did — so that when someone eventually asked, the answer could be one paragraph instead of a negotiation.
\n01:46, marked critical. 02:03, creator campaigns vanished from the homepage. I went to bed around three.
\nThe first fix didn’t go anywhere near the sanitizer, which I thought was the right instinct.
\nThey killed the Creator spotlight module at the home-composer layer and purged the associated CDN keys. The vulnerable campaign objects still existed in the database, but the highest-impact sink stopped consuming them. You cut the delivery path first and argue about root cause in the morning.
\nThe API hotfix landed alongside it — the server simply stopped listening to the client on anything that counted:
\nconst campaign = await createCreatorCampaign({\n ownerId: user.id,\n headline: input.headline,\n description: input.description,\n imageId: input.image_id,\n cta: input.cta,\n\n // Server-owned policy:\n surface: \"CREATOR_PROFILE\",\n audience: { type: \"FOLLOWERS\" },\n status: \"DRAFT\",\n reviewRequired: true,\n});Existing creator-owned campaigns pointing at non-profile surfaces were suspended pending review, which I’d suggested and was glad to see done without argument.
\nBy 03:12 my old request was dead:
\n{\n \"error\": \"invalid_campaign_surface\",\n \"message\": \"Creators may publish campaigns only to CREATOR_PROFILE\"\n}The CDN had stopped serving my inert marker. The global delivery path was closed.
\nThe sink still needed fixing, obviously — internal editors, a compromised staff account, a bulk import, or the next authorization mistake could all reach it again. But the emergency was over.
\nThe renderer team’s first read was that this was a DOMPurify bypass. I get why — there’s a sanitizer, and something got through, so the sanitizer must have failed.
\nIt didn’t. It never saw the dangerous value. It was handed this:
\n<h2>{{{headline_html}}}</h2>There was nothing to remove. DOMPurify answered the question it was asked, correctly and completely. The question was:
\n\n\nIs this template safe right now?
\n
The application needed the answer to a different one:
\n\n\nIs the final document safe after every transformation has finished?
\n
I keep coming back to this because it generalizes well past XSS. Validation can be quietly undone by anything that happens afterwards: decoding, interpolation, re-parsing, concatenation, decompression, redirect following, type coercion. A check is only meaningful at the boundary immediately before interpretation. For HTML, that boundary is the DOM sink and nowhere else.
\nThe DOMPurify docs say this outright — modifying sanitized markup afterwards can void the protection. The renderer had done exactly that, just indirectly, through a template engine, in a way that looked completely fine in code review because the word sanitize was right there on the screen.
The ordering made the sanitizer decorative. It was doing real work on a string that didn’t need it.
\nBecause the finding crossed several trust boundaries, the remediation had to as well. Six pieces:
\nThe creator API got its own input type, separate from the internal editorial one.
\nBefore:
\ninput CampaignInput {\n headline: String!\n description: String\n imageId: ID\n cta: CampaignCTAInput\n surface: CampaignSurface!\n audience: CampaignAudienceInput!\n status: CampaignStatus!\n reviewRequired: Boolean\n}After:
\ninput CreatorCampaignInput {\n headline: String!\n description: String\n imageId: ID\n cta: CampaignCTAInput\n}Every policy field derived server-side. Editorial tools moved to a different mutation behind role checks and an approval workflow. Hiding a field in the UI stopped being the authorization model.
\nThe homepage no longer accepts an opaque html blob for creator content. The API returns data:
{\n \"type\": \"creator_spotlight\",\n \"headline\": \"October systems notes\",\n \"description\": \"A small collection of experiments.\",\n \"cta\": {\n \"label\": \"Read more\",\n \"href\": \"/u/researcher/notes\"\n }\n}And React renders it as text:
\nfunction CreatorSpotlight({ module }) {\n return (\n <section className=\"creator-spotlight\">\n <h2>{module.headline}</h2>\n <p>{module.description}</p>\n {module.cta && (\n <a href={validateInternalHref(module.cta.href)}>\n {module.cta.label}\n </a>\n )}\n </section>\n );\n}The feature lost the ability to put arbitrary HTML in a headline. As far as I know, not one person has complained.
\nA handful of editorial modules did legitimately need emphasis, links, and line breaks. Those moved to a restricted document format rather than raw HTML:
\n{\n \"type\": \"heading\",\n \"children\": [\n { \"type\": \"text\", \"value\": \"Build \" },\n { \"type\": \"emphasis\", \"value\": \"smaller\" },\n { \"type\": \"text\", \"value\": \" systems\" }\n ]\n}Known node types map to known React components; unknown nodes are rejected. No generic string-to-HTML conversion, no triple-brace escape hatch. Where legacy HTML still exists, it’s sanitized after all interpolation and immediately before the sink, and nothing is permitted to touch it afterwards.
\nThe homepage CSP moved from report-only to enforced, nonce-based. The production policy is more involved than this, but the relevant properties:
\nscript-src used nonces for application scripts\ninline event handlers were blocked\nobject-src was none\nbase-uri was restricted\nthird-party script origins were reduced\nreport-only remained as a second policy for future tighteningI made a point of framing CSP as defense in depth in the report, not as the fix. A weak CSP wouldn’t have excused unsafe rendering, and a strong one wouldn’t have made client-controlled homepage placement acceptable. But after the change, an equivalent injection has to beat both the rendering boundary and the browser.
\nThe account team went through the high-impact operations a same-origin script could invoke — email changes, recovery settings, API token creation, session management — and put them behind recent reauthentication or a step-up challenge.
\nThis does nothing to stop XSS reading ordinary session-visible data. What it does is reduce the odds that a single compromised page view turns into permanent account takeover.
\nHttpOnly protects the cookie value. Reauthentication protects selected consequences of the session. Different controls, different jobs, and it’s worth being clear about which one you actually have.
The thirty seconds where my deleted campaign kept serving from the edge turned into its own workstream:
\nThe cache wasn’t the root cause. It was most of the blast radius, which made it part of the fix.
\nThe patched build arrived nine days later. Nine days is fast for something touching this many services, and I said so.
\nThe original request now behaves like this:
\nsurface = HOME_DISCOVERY\n → rejected\n\naudience = ALL\n → rejected\n\nheadline contains HTML\n → stored as text\n\ninternal editorial rich text\n → parsed into allowlisted nodes\n\nlegacy HTML fragment\n → sanitized after interpolation\n\ninline execution canary\n → absent from DOM; CSP blocks execution pathI also walked the whole lifecycle rather than just re-firing the original payload, because the original payload is the one thing you can be certain they tested:
\nResolved on October 26. The whole thing, start to finish:
\nOctober 14 23:52 Hidden homepage surface accepted\nOctober 15 00:18 XSS confirmed for owned test audience\nOctober 15 00:41 Global placement confirmed with plain text\nOctober 15 01:07 Report submitted\nOctober 15 01:46 Marked critical\nOctober 15 02:03 Homepage module disabled and purge started\nOctober 15 03:12 Creator API hotfix verified\nOctober 20 Rendering and CSP fixes deployed\nOctober 24 Retest completed\nOctober 26 Report resolved; $BOUNTY awardedMonths later, the thing that stays with me isn’t the payload or the bounty. It’s how ordinary every individual decision was.
\nThe bug didn’t start with a missing sanitizer — there was one, and it was a good one. It didn’t start with a session cookie exposed to JavaScript — the cookie was configured better than most I see. It didn’t start with a recklessly public publishing endpoint — the creator UI only ever showed profile placement. It didn’t start with a misbehaving cache — the CDN cached precisely what the homepage told it to.
\nRead these one at a time and every single one is fine:
\nCreators may create campaigns.\nThe homepage may render campaigns.\nMarketing templates may contain formatting.\nSanitized HTML may be inserted into the DOM.\nPublic homepage HTML may be cached.\nHttpOnly cookies may not be read by JavaScript.The vulnerability was in the joins. Each sentence was true in isolation and none of them carried the information about who was trusted when. A creator picked an editorial surface. A sanitized template got unsanitized data poured into it afterwards. A React app accepted an opaque HTML string from its own backend. A report-only policy watched the browser run it and wrote it down. A CDN made copies. A well-configured cookie authenticated the requests.
\nThe campaign was attacker-controlled. The homepage was trusted. The browser had no way to tell them apart, because by the time the HTML reached it, they were the same string.
\nThe cookie stayed secret the entire time. Its authority didn’t.
\nThe cookie was HttpOnly. The session wasn’t.
At 2:14 on a Sunday morning, the mobile app told me it could not load a link preview.
\nMy server disagreed.
\n02:14:07 GET /proof/4f6c2/uid-1001 HTTP/1.1\n Host: canary.example.test\n User-Agent: curl/7.81.0\n Source: 203.0.113.74The URL I had pasted into the app did not contain canary.example.test.
The web page at the end of the redirect did not contain it either.
\nOnly one thing contained that hostname: a short, deliberately harmless command embedded inside a parameter sent to a service that was supposed to exist only on 127.0.0.1.
The app showed a gray chain icon and Preview unavailable.
\nThe command had already run.
\nI stared at the access log long enough for the line to stop looking like text and start looking like impact.
\nThen I closed the terminal, opened a new note, and wrote:
\n02:15 — probable command execution on preview worker\n02:16 — do not inspect filesystem\n02:16 — do not read environment\n02:17 — reproduce once with new token, then reportThe bug began as a URL preview in a mobile chat composer.
\nIt ended as remote command execution in production.
\nBetween those two things were one redirect, one assumption about localhost, and one pair of quotation marks.
\n$COMPANY was a mobile-first collaboration app. Teams used it for chat, files, voice notes, and the usual collection of tiny conveniences that eventually become infrastructure.
When someone pasted a URL into a message, the app generated a card:
\n┌──────────────────────────────────────────────┐\n│ The boring engineering behind tiny models │\n│ Models are large. Habits are surprisingly │\n│ portable. │\n│ example │\n└──────────────────────────────────────────────┘The title, description, site name, and image came from $COMPANY’s servers, not from the phone. That is sensible. A backend can fetch a page once, cache the result, resize images consistently, and avoid making every recipient’s device contact an arbitrary third-party host.
It also means the backend is willing to visit a URL chosen by a user.
\nThat sentence is where SSRF lives.
\nI had opened the Android application because its link cards looked slightly different from the web version. The web client discarded malformed pages quietly. Android sometimes returned an error description under the gray placeholder.
\nDifferent error handling often means different code.
\nI searched the APK for preview and found a small Retrofit interface:
interface PreviewApi {\n @POST(\"mobile/v3/link-previews\")\n suspend fun createPreview(\n @Body request: PreviewRequest\n ): PreviewCard\n}\n\ndata class PreviewRequest(\n val url: String,\n val locale: String,\n val appVersion: String\n)The endpoint required an ordinary account. No administrator role, enterprise plan, or special workspace permission was involved. The mobile client called it as soon as a URL appeared in the composer, before the message was sent.
\nWith my own test account, the request looked like this:
\nPOST /api/mobile/v3/link-previews HTTP/1.1\nHost: api.$COMPANY.example\nAuthorization: Bearer <test-account-token>\nContent-Type: application/json\nX-Client: android/7.42.1\n\n{\n \"url\": \"https://research.example.test/ordinary-page\",\n \"locale\": \"en-US\",\n \"appVersion\": \"7.42.1\"\n}The response was unsurprising:
\n{\n \"title\": \"ordinary page\",\n \"description\": \"a page controlled for testing\",\n \"site_name\": \"research.example.test\",\n \"image_url\": null\n}My server received two requests from an address belonging to $COMPANY: first HEAD, then GET.
That established the basic fact. The phone was not fetching the page. $COMPANY was.
The next question was where it was willing to go.
\nI tried the obvious internal destinations directly.
\nhttp://127.0.0.1/\nhttp://localhost/\nhttp://10.0.0.1/\nhttp://169.254.169.254/All four failed before a network request was made:
\n{\n \"error\": \"url_not_allowed\",\n \"message\": \"Private or reserved destinations cannot be previewed\"\n}That was good. Better than good, actually. Plenty of preview endpoints still fail at the first sentence of SSRF prevention.
\nThe service also rejected non-HTTP schemes, URLs containing credentials, bare hostnames without a dot, and several unusual textual forms of loopback addresses. Somebody had clearly thought about this.
\nFor a minute I considered moving on.
\nThen I tested a public URL that redirected to another public URL. The preview followed it.
\nThat is normal behavior for the web. It is also a second URL-validation problem hiding inside the first one.
\nThe application had answered:
\n\n\nIs the URL the user submitted safe?
\n
It had not yet answered:
\n\n\nIs every URL the server will actually request safe?
\n
I configured my public test endpoint to return one redirect:
\nHTTP/1.1 302 Found\nLocation: http://127.0.0.1:3001/\nCache-Control: no-storeThen I submitted the public address—not the loopback address—to the mobile preview API.
\n{\n \"url\": \"https://research.example.test/to-loopback\",\n \"locale\": \"en-US\",\n \"appVersion\": \"7.42.1\"\n}The request passed the filter.
\nMy server returned the redirect.
\nThe preview response came back three seconds later:
\n{\n \"title\": \"mobile-renderer\",\n \"description\": \"internal screenshot service\",\n \"site_name\": \"127.0.0.1\",\n \"image_url\": null\n}I had not bypassed the IP parser.
\nI had walked around it.
\nThe original host was public. The destination after the redirect was not. The filter examined the first and the HTTP client trusted the rest.
\nIn Python-shaped pseudocode, the bug was probably close to this:
\ndef preview(user_url: str) -> Preview:\n parsed = parse_url(user_url)\n resolved = resolve(parsed.hostname)\n\n if not is_public_address(resolved):\n raise UrlNotAllowed()\n\n # requests follows redirects for GET by default\n response = requests.get(\n user_url,\n timeout=5,\n headers={\"User-Agent\": PREVIEW_USER_AGENT},\n )\n\n return extract_preview(response)The validation and the request each looked reasonable in isolation.
\nThe problem was that response might belong to a different host from user_url.
A redirect is not a continuation of the same trust decision. It is a new request.
\nOnce an application will fetch loopback URLs, it is tempting to turn the preview endpoint into an internal port scanner.
\nThat is also an efficient way to stop doing research and start creating unnecessary risk.
\nI tested only a handful of common local development ports, one request at a time, with long pauses. I stopped at the first identifiable service. The report included the exact list and timestamps so $COMPANY could distinguish my traffic from anything else.
Port 3001 returned an HTML page with this title:
<title>mobile-renderer</title>The body was a tiny internal status screen:
\nmobile-renderer\nbuild: 2023.08.11-4d8c2f1\nmode: production\n\nGET /health\nGET /debug/render?url=<url>&width=<pixels>The interesting word was not render.
It was debug.
The endpoint was bound to loopback, so it had no authentication. From the perspective of its author, that may have felt like authentication: only software on the same machine could reach it.
\nThe preview service was software on the same machine.
\nAnd I could tell the preview service where to go.
\nmobile user\n │\n │ public URL\n ▼\nlink-preview API\n │\n │ follows 302\n ▼\n127.0.0.1:3001\n │\n ▼\nmobile-renderer debug routeA network boundary had become a permission boundary without anybody writing down that decision.
\nI sent the renderer a normal public page through the redirect chain:
\nhttps://research.example.test/to-render\n └── 302 → http://127.0.0.1:3001/debug/render\n ?url=https%3A%2F%2Fexample.org%2F\n &width=800The mobile API waited several seconds and returned a screenshot URL hosted on $COMPANY’s CDN.
That proved three things:
\nThe generated image showed example.org. I deleted the preview from my test workspace and did not request any internal page.
At this point the finding was already serious SSRF. An ordinary mobile user could invoke internal HTTP endpoints and retrieve at least some of their output through the preview parser or generated artifacts.
\nBut the renderer’s error behavior suggested there was another boundary behind it.
\nI changed the url parameter from a valid address to one containing a single encoded quotation mark.
The render failed immediately.
\nThe mobile response included the first line of stderr:
\n{\n \"error\": \"preview_generation_failed\",\n \"detail\": \"/bin/sh: 1: Syntax error: Unterminated quoted string\"\n}A URL parser does not normally complain about shell quotes.
\nA shell does.
\nI wrote a second note:
\n01:46 — renderer likely builds command string\n01:48 — do not guess broadly\n01:49 — one canary-only proofFrom the outside, the renderer appeared to launch a headless browser script with a command assembled as text.
\nThe vulnerable code may have looked like this:
\nimport { exec } from \"node:child_process\";\nimport { randomUUID } from \"node:crypto\";\n\napp.get(\"/debug/render\", (req, res) => {\n const target = req.query.url;\n const width = req.query.width || \"800\";\n const output = `/tmp/${randomUUID()}.png`;\n\n const command =\n `timeout 20s node capture.js ` +\n `--url \"${target}\" ` +\n `--width ${width} ` +\n `--output \"${output}\"`;\n\n exec(command, (error) => {\n if (error) {\n return res.status(500).send(error.message);\n }\n\n res.sendFile(output);\n });\n});exec() does not pass an argument array directly to a program. It starts a shell and gives the shell one command string.
That distinction is convenient when the command is constant.
\nIt is dangerous when part of the string came from an HTTP request.
\nThe renderer placed target inside double quotes, but quotes are not a security boundary. They are syntax. If untrusted input can alter that syntax, the shell gets to interpret the remainder.
I am not including the command-separator bytes used in the report. They add nothing to the engineering lesson and make the article more reusable than it needs to be.
\nThe proof itself was deliberately small:
\nThe public redirect server never mentioned the canary hostname. The target page never mentioned it. Only the injected fragment did.
\nAt 2:14:07, the canary request arrived:
\nGET /proof/4f6c2/uid-1001 HTTP/1.1\nHost: canary.example.test\nUser-Agent: curl/7.81.0The source address matched the preview infrastructure.
\nThe user agent did not match the preview fetcher or headless Chromium.
\nThe path included the expected one-time token and uid-1001.
I repeated the test with a different token and received a second request.
\nThen I used a benign delay as an independent signal. A normal failed render returned in about 600 milliseconds. The redacted test delayed the response by five seconds, three times in a row.
\nTwo different observations now pointed to the same conclusion:
\nuser-controlled mobile request\n → server-side redirect to loopback\n → unauthenticated internal render endpoint\n → shell command injection\n → command execution as uid 1001The preview failed.
\nThe command did not.
\n“Remote code execution” sometimes gets confused with a cinematic terminal window.
\nThe terminal is optional.
\nThe security boundary fails when an untrusted remote user can cause the target to execute an operating-system command of the user’s choosing. A one-shot canary request is enough to prove that boundary failed.
\nI did not establish persistence. I did not enumerate the container. I did not inspect mounted volumes, credentials, service-account tokens, customer data, or neighboring services. I did not test whether the process could escape its container.
\nThose actions would have increased risk without changing the finding.
\nThe report said exactly what I had confirmed:
\nConfirmed:\n- Any ordinary account can invoke the mobile link-preview endpoint.\n- URL validation applies to the initial URL only.\n- Redirects can reach loopback services.\n- A loopback-only renderer interpolates the `url` parameter into a shell command.\n- Arbitrary command execution is possible as the renderer user (uid 1001).\n- The worker has outbound network access.\n\nNot tested:\n- Filesystem contents\n- Environment variables or credentials\n- Access to customer data\n- Access to cloud metadata\n- Container escape\n- Lateral movement\n- PersistenceThat separation matters. A report should make the proven impact difficult to dismiss and the untested impact impossible to mistake for a claim.
\n$COMPANY later confirmed that the renderer ran in a production container with read access to a preview-object bucket and credentials for an internal logging service. It did not hold mobile signing keys.
I had not accessed any of them.
\nThe RCE was severe without borrowing impact from secrets I did not need to touch.
\nNo command ran on my phone.
\nThe APK contained the endpoint and supplied the user-controlled URL, but the vulnerable code lived in the backend services supporting the mobile experience.
\nThat distinction is worth making because “mobile vulnerability” often gets interpreted as memory corruption, a malicious intent, or code execution on a handset.
\nThis was different:
\nAndroid app\n │ authenticated API request\n ▼\nmobile preview backend\n │ SSRF\n ▼\nloopback renderer\n │ command injection\n ▼\nproduction containerThe phone was the handle.
\nThe backend was the blade.
\nThe web application did not reproduce the issue because it called a newer GraphQL preview service. That service rejected redirects entirely. Android and iOS still used the older /mobile/v3/link-previews route for latency and compatibility reasons.
One product had two implementations of the same convenience feature.
\nOnly one of them had received the security fix.
\nThis is how old APIs become vulnerability museums: they continue to work, which is mistaken for evidence that they continue to be safe.
\nI sent the report at 02:43 UTC, twenty-nine minutes after the first command-execution proof.
\nThe title was intentionally boring:
\nCritical: mobile link-preview SSRF to loopback renderer leads to OS command executionThe first page contained only the chain and impact:
\nPrecondition:\n Any authenticated user account.\n\nEntry point:\n POST /api/mobile/v3/link-previews\n\nRoot causes:\n 1. Initial URL is validated, but redirect destinations are not.\n 2. Loopback renderer trusts network location instead of authenticating callers.\n 3. Renderer passes a user-controlled URL to child_process.exec().\n\nImpact:\n Remote command execution on a production preview worker as uid 1001.\n\nSafety:\n Tested only with owned accounts, controlled hosts, one canary callback,\n and a benign timing proof. No data or credentials accessed.Then came the exact reproduction, request IDs, timestamps, the two random canary tokens, source IPs, and a screen recording showing the mobile app’s failed preview beside the canary log.
\nI included three fixes rather than one because the exploit had three independent trust failures:
\nAt 03:11, the program acknowledged the report.
\nAt 03:37, an engineer asked one question:
\n\n\nDoes command execution require the renderer to return a successful image?
\n
It did not.
\nThe command ran before the render callback returned, so the outer API could show an error while the side effect had already happened.
\nThat explained the title I had not yet written.
\nAt 04:02, $COMPANY disabled preview generation on the mobile endpoint. Pasted links still sent as text, but no card was created.
At 12:26, the report was marked critical.
\nThe permanent fix arrived five days later.
\nThe preview fetcher no longer followed redirects automatically. It processed them one at a time, resolving and validating each destination before making the next request.
\nThe invariant was written like this in the remediation notes:
\nFor the initial URL and every redirect hop:\n\nscheme ∈ {http, https}\nresolved address is globally routable\nconnected address equals a validated resolution\ndestination is not loopback, private, link-local, multicast, or reserved\nredirect count ≤ 3\nresponse size and time are boundedApplication validation was backed by an egress rule preventing the preview network from reaching loopback-adjacent services, private ranges, link-local addresses, and cloud metadata endpoints.
\nThat second layer matters. URL parsing is full of representations, redirects, DNS behavior, proxies, and disagreement between libraries. The OWASP SSRF prevention guidance recommends network controls in addition to application checks for exactly this reason.
\nThe old code made one policy decision and then delegated navigation to a general-purpose HTTP client.
\nThe new code kept policy in the loop.
\nThe debug renderer route was removed from production.
\nThe ordinary rendering API moved behind a separate service identity and required a short-lived, audience-bound token. Requests arriving merely from loopback or the cluster network were no longer considered trusted.
\nThe renderer also moved into a more restricted container:
\nread-only root filesystem\nno cloud instance metadata route\nno general outbound internet access\nminimal object-store permission\nno shell package in runtime image\nCPU and wall-clock limits per renderA service should be able to survive the caller making a mistake.
\n“Only internal systems can reach this” is useful context. It is not authorization.
\nThe renderer replaced exec() with a direct process invocation and an argument array:
import { spawn } from \"node:child_process\";\nimport { randomUUID } from \"node:crypto\";\n\napp.post(\"/render\", requireServiceIdentity, (req, res) => {\n const target = validateRenderUrl(req.body.url);\n const width = validateWidth(req.body.width);\n const output = `/tmp/${randomUUID()}.png`;\n\n const child = spawn(\n process.execPath,\n [\n \"capture.js\",\n \"--url\", target,\n \"--width\", String(width),\n \"--output\", output,\n ],\n {\n shell: false,\n timeout: 20_000,\n stdio: [\"ignore\", \"ignore\", \"pipe\"],\n },\n );\n\n child.on(\"close\", (code) => {\n if (code !== 0) {\n return res.status(502).json({ error: \"render_failed\" });\n }\n\n res.sendFile(output);\n });\n});The important part is not spawn as a magic safe word. It is that the URL is passed as one argument to one program, without a shell being asked to reinterpret it.
The route still validates the URL, width, timeout, and output path. Removing the shell closes the command-injection class; input validation and sandboxing limit the damage from whatever the renderer itself might do.
\nSecurity patches work better when they do not require one line of code to be perfect.
\nThe patched mobile endpoint returned previews again on August 18.
\nThe original public test URL still worked.
\nA public-to-public redirect worked, up to the new three-hop limit.
\nA public URL redirecting to loopback failed before the second request:
\n{\n \"error\": \"url_not_allowed\",\n \"reason\": \"redirect_destination_not_public\"\n}My loopback service received nothing.
\nI tried an ordinary hostname that resolved to a public address and then changed. The connection was rejected when the actual destination no longer matched the validated address.
\nThe old internal /debug/render route returned 404 from the renderer image and was unreachable from the preview network anyway.
Finally, $COMPANY supplied a test fixture containing the characters that had previously changed the shell command. The renderer treated the entire value as one invalid URL argument and returned a normal validation error.
No canary request arrived.
\nNo timing shift appeared.
\nThe preview failed.
\nThis time the command failed with it.
\nThe most useful tests were not named after my payload. They were named after the properties the system had forgotten.
\ndef test_preview_revalidates_every_redirect_destination():\n public = controlled_server.redirect_to(\"http://127.0.0.1:3001/\")\n\n result = create_mobile_preview(url=public)\n\n assert result.status == 400\n assert result.reason == \"redirect_destination_not_public\"\n assert controlled_loopback_service.requests == []test(\"renderer passes URL as one argument without a shell\", async () => {\n const suspicious = makeStringContainingShellSyntax();\n\n await expect(render(suspicious)).rejects.toMatchObject({\n code: \"INVALID_RENDER_URL\",\n });\n\n expect(spawn).not.toHaveBeenCalledWith(\n expect.anything(),\n expect.anything(),\n expect.objectContaining({ shell: true }),\n );\n});And the infrastructure test mattered as much as either unit test:
\npreview-worker → public HTTP/HTTPS: allowed through controlled egress\npreview-worker → loopback/private/link-local/metadata: denied\nrenderer → arbitrary internet: denied\nrenderer → preview object bucket: write-only, scoped prefixA fixed payload is one regression test.
\nA fixed trust model is a system of them.
\nNo individual mistake felt large enough to become critical.
\nThe URL validator blocked private addresses.
\nThe renderer listened only on loopback.
\nThe command put the URL inside quotes.
\nEach sentence sounds like a mitigation.
\nTogether they produced RCE.
\nThe service blocked the URL the user supplied. It did not block the URL the HTTP client eventually fetched.
\nValidation attached to a string instead of to the network action.
\nThat was true from the network’s point of view.
\nIt was false from the attacker’s point of view because another reachable service could be instructed to make the request.
\nReachability composes.
\nQuoting is part of shell grammar, not an alternative to it.
\nOnce a shell receives a command string containing untrusted text, correctness depends on every transformation preserving syntax exactly. The safer design is not to ask a shell to parse the value at all.
\nThe exploit did not defeat three strong defenses.
\nIt passed through three defenses that protected different objects from the ones developers thought they protected.
\n$COMPANY closed the report on August 24 after the mobile clients had resumed preview generation and the infrastructure changes had reached production.
The reward was $BOUNTY. The amount matters less than the severity classification, and the severity matters less than the patch surviving the feature being turned back on.
The engineer who handled the report sent one final note:
\n\n\nWe originally treated the redirect check, renderer authentication, and shell invocation as separate issues. The incident review now models them as one request path.
\n
That is the right lesson.
\nVulnerabilities are often reviewed one function at a time because code is stored one function at a time. Attackers get the whole request path.
\nIn February, a PDF button reminded me that reachability is not authorization.
\nIn June, SAML reminded me that authentication is not tenancy.
\nThis one added a smaller rule:
\nA redirect is a new request, not a footnote to the old one.
\nThe first URL was public.
\nThe second URL was localhost.
\nThe third value became shell syntax.
\nEvery component did something it had been designed to do:
\nThe vulnerability lived in the word therefore between those sentences.
\nNo command ran on the phone.
\nNo firewall port opened to the internet.
\nNo cryptography failed.
\nI pasted a link into a mobile app.
\nThe app said the preview failed.
\nThe server ran the rest.
","excerpt":"At 2:14 on a Sunday morning, the mobile app told me it could not load a link preview. My server disagreed. The URL I had pasted into the app did not contain . The web page at the end of the redirect did not contain it…","timeToRead":18,"fields":{"slug":"/the-preview-failed-the-command-did-not/"},"frontmatter":{"title":"The preview failed. The command did not.","description":"A mobile app's link-preview endpoint followed one redirect into localhost, reached an internal screenshot service, and turned a blocked SSRF into remote command execution.","path":"the-preview-failed-the-command-did-not","date":"Aug 26, 2023","year":"2023"}},{"id":"b8680e5a-5e9f-5938-a3c6-befc5bb4b743","html":"On a Saturday night, I logged into the wrong company.
\nNot with a stolen password. Not with a leaked session cookie. Not with an expired reset link that somebody had forgotten to invalidate.
\nI used a SAML assertion signed by a key I owned, for an identity provider I had configured, inside a workspace I had created fifteen minutes earlier.
\nThe assertion was valid.
\nThe signature was valid.
\nThe browser opened a different workspace.
\nIn the first workspace, my test account was a normal member. In the second, the same email address was an administrator. The avatar was the same, the email was the same, and the small badge beside it had changed from Member to Administrator.
\nFor about ten seconds I blamed the two browser tabs.
\nThen I opened /api/me.
{\n \"user_id\": \"usr_01H3EJ9J9G5W6D1R7K0P\",\n \"email\": \"andrew+saml@example.test\",\n \"workspace_id\": \"ws_blue\",\n \"workspace_name\": \"BLUE LAB\",\n \"role\": \"administrator\"\n}The SAML identity provider belonged to RED LAB.
The session belonged to BLUE LAB.
I logged out, cleared every cookie for the domain, opened a private window, and did it again.
\nSame result.
\nThen I renamed the workspaces because lab-a and lab-b had become psychologically dangerous.
RED LAB — SAML configured — member\nBLUE LAB — password login — administratorI repeated the flow a third time, recorded the request, saved the response, and stopped.
\nMy notes from that minute contain only this:
\n23:51 — cross-tenant session confirmed\n23:52 — do not touch anything else\n23:53 — write report before convincing yourself it is normalThe bug was not an XML parser trick. It was not signature wrapping. It was not a broken cryptographic primitive.
\nIt was one missing relationship.
\nidentity_provider.workspace_id == session.workspace_idThe distance between two companies was one ==.
The company—I’ll call it $COMPANY—runs a multi-tenant business application. Customers live in workspaces. A person can belong to several workspaces, and each workspace can configure its own SAML identity provider.
The intended flow is ordinary:
\nemployee\n │\n ▼\ncompany identity provider\n │ signed SAML assertion\n ▼\n$COMPANY assertion consumer service\n │\n ▼\ncompany workspaceA workspace administrator uploads identity-provider metadata: an issuer, a login URL, and a public signing certificate. When an employee signs in, $COMPANY verifies the assertion, resolves the employee, and creates a session in that workspace.
Every customer used the same callback endpoint:
\nhttps://app.$COMPANY.example/auth/saml/callbackThat is normal. A shared assertion consumer service is not itself a vulnerability.
\nIt does mean the application must answer two different questions when a SAML response arrives:
\nThe first answer came from the signature.
\nThe second came from a value carried through the user’s browser.
\nThat was the loose thread.
\nI was not hunting for a cross-tenant takeover. I was testing the SSO setup flow because it had a button labeled Test connection, and buttons that promise to test enterprise authentication tend to hide interesting state machines.
\nI created RED LAB, enabled an enterprise trial, and configured a small local identity provider using a key pair generated for the test. Nothing unusual happened. The metadata imported correctly. The test user signed in correctly. The application redirected back to the SSO settings page with a green success banner.
Then I looked at the browser request.
\nThe authentication request included a RelayState parameter. That is expected: SAML applications often use it to remember local state while the browser travels to the identity provider and back.
What was less expected was how much the state wanted to introduce itself.
\nRelayState=eyJ3b3Jrc3BhY2VfaWQiOiJ3c19yZWQiLCJuZXh0IjoiL3NldHRpbmdzL3NzbyJ9Base64-decoded, it became:
\n{\n \"workspace_id\": \"ws_red\",\n \"next\": \"/settings/sso\"\n}There was no MAC. No signature. No opaque server-side reference. Just JSON taking the scenic route through base64.
\nEncoding is not integrity.
\nStill, client-controlled RelayState does not automatically mean account takeover. The server might use it only for a harmless redirect after it has independently established the tenant from the authentication request. Plenty of applications put ugly things in state and then decline to trust them.
So I changed only one value:
\n- \"workspace_id\": \"ws_red\"\n+ \"workspace_id\": \"ws_blue\"I left the SAML response untouched.
\nThe assertion was still signed by RED LAB’s identity provider. Its InResponseTo still matched the request created for RED LAB. The destination still pointed to $COMPANY’s callback. The audience still named $COMPANY’s service provider. The assertion was fresh and inside its validity window.
The callback returned:
\nHTTP/1.1 302 Found\nLocation: /workspaces/ws_blue/dashboard\nSet-Cookie: session=eyJ...; Secure; HttpOnly; SameSite=LaxI followed the redirect and saw the administrator badge.
\nA good bug becomes dangerous around the moment you want it to be real. That is also the moment to become suspicious of yourself.
\nMy first theory was stale session state. Perhaps I had already authenticated to BLUE LAB, and the callback merely sent me to the most recent workspace.
I deleted the session, cleared local storage, used a new private window, and verified that /api/me returned 401 before starting SAML. The new session still belonged to BLUE LAB.
My second theory was a cosmetic workspace switch. Some applications keep one global user session and let the interface display whichever workspace appears in the URL. Maybe the page looked like BLUE LAB, but authorization still came from RED LAB.
I requested an endpoint available only to workspace administrators:
\nGET /api/workspaces/ws_blue/sso/configurationThe response was 200, not 403.
I did not inspect or modify any real configuration. Both workspaces were mine, and the endpoint’s status was enough.
\nMy third theory was an accidental role carried from the source workspace. Perhaps the application had copied my RED LAB role into the target instead of resolving the existing BLUE LAB membership.
So I reversed the roles:
\nRED LAB — administrator\nBLUE LAB — memberThe same SAML assertion now produced a member session in BLUE LAB.
That told me exactly where authorization came from. The application was using the assertion to choose an identity and the target workspace to choose the role.
\nAuthentication came from one tenant.
\nAuthorization came from another.
\nThe seam between them had no tenant check.
\nI could not see $COMPANY’s source, but the behavior narrowed the implementation to something like this:
def consume_saml(saml_response: str, relay_state: str):\n assertion = parse_saml_response(saml_response)\n\n pending_request = AuthnRequest.get(assertion.in_response_to)\n idp = IdentityProvider.get(pending_request.identity_provider_id)\n\n verify_signature(assertion, certificate=idp.certificate)\n verify_issuer(assertion, expected=idp.issuer)\n verify_audience(assertion, expected=GLOBAL_SP_ENTITY_ID)\n verify_recipient(assertion, expected=GLOBAL_ACS_URL)\n verify_time_window(assertion)\n\n state = json.loads(base64_decode(relay_state))\n workspace = Workspace.get(state[\"workspace_id\"])\n\n user = workspace.users.get_by_email(assertion.email)\n return create_session(user=user, workspace=workspace)Every line looks reasonable in isolation.
\nThe request ID selects the identity-provider configuration. The configured certificate verifies the response. The issuer, audience, recipient, and time window are checked. The state restores the destination. The user is looked up inside the selected workspace.
\nThe missing line is not glamorous:
\nif idp.workspace_id != workspace.id:\n raise InvalidSamlResponse(\"identity provider does not belong to workspace\")Without it, the system proved this:
\n\n\nAn identity provider trusted by some
\n$COMPANYcustomer signed this email address.
It needed to prove this:
\n\n\nThe identity provider trusted by this workspace signed this subject for this login transaction.
\n
Those sentences differ by a few words and an entire security boundary.
\nThe unsettling part was that none of the cryptography failed.
\nI did not steal a private key. I did not alter the signed XML. I did not convince the parser to validate one assertion and consume another.
\nI brought my own key.
\nAs the administrator of RED LAB, I was allowed to configure an identity provider. $COMPANY quite reasonably stored my public certificate as a trusted key for that workspace.
The vulnerable flow turned that narrow trust relationship into something closer to this:
\ntrusted by one tenant\n ↓\ntrusted somewhere in the platform\n ↓\nusable wherever RelayState pointsAn attacker who controlled any SAML-enabled workspace could issue a valid assertion containing the email address of an existing user in another workspace. If the target account was an administrator, the resulting session inherited the target workspace’s administrator role.
\nThe attacker would not need the target’s password. Application-level MFA would not help if SAML was already treated as the completed authentication step. The feature existed specifically so the application could trust the identity provider.
\nThe platform remembered the key.
\nIt forgot who trusted it.
\nCross-tenant vulnerabilities create an awkward temptation. Once you can move between two tenants you own, the next dramatic screenshot would be a real customer’s workspace.
\nThat screenshot is unnecessary.
\nTwo controlled workspaces were enough to demonstrate the complete authorization failure:
\n| Assertion signed by | \nTarget workspace | \nMatching user in target | \nTarget role | \nResult | \n
|---|---|---|---|---|
RED LAB IdP | \nRED LAB | \nyes | \nmember | \nexpected member session | \n
RED LAB IdP | \nBLUE LAB | \nyes | \nadministrator | \nunexpected administrator session | \n
RED LAB IdP | \nBLUE LAB | \nno | \n— | \nuser-not-found error | \n
| modified assertion | \nBLUE LAB | \nyes | \nadministrator | \nsignature rejected | \n
| expired assertion | \nBLUE LAB | \nyes | \nadministrator | \ntime-window rejected | \n
The last two rows mattered. They showed that the security machinery was active. The application rejected a modified assertion and an expired assertion.
\nOnly the tenant relationship was missing.
\nI saved a HAR file, recorded a short video using the two lab workspaces, copied the request IDs and timestamps, invalidated the sessions, and stopped testing.
\nA proof of concept should make the bug undeniable while making customer impact unnecessary.
\nAnything beyond that is not better evidence. It is damage with cleaner screenshots.
\nI used the least exciting title I could manage:
\nCross-tenant account takeover through unbound SAML RelayStateThe summary was more direct:
\n\n\nA SAML assertion signed by an identity provider configured for workspace A can create a session for an existing user in workspace B. The response signature, issuer, audience, recipient, time window, and request correlation all validate. The target workspace is selected separately from client-controlled RelayState and is not checked against the identity-provider configuration. A tenant administrator can therefore authenticate as a known user—including an administrator—in another workspace.
\n
Then I included:
\nRelayState values.I also included the sentence I would want to see first if I were reading the report at $COMPANY:
\n\nNo customer workspace or customer data was accessed during testing.
\n
The report went out at 00:26.
\nI closed the laptop, got into bed, and spent twenty minutes mentally inventing reasons the report would be marked informative.
\nAt 09:12, a human answered.
\nNot an automated receipt. Not a request for a video I had already attached.
\n\n\nWe reproduced the behavior between two internal tenants and have escalated it to the identity team. We are temporarily restricting creation of new SAML configurations while we investigate. Existing SSO connections remain active. Please do not perform additional testing until we confirm the patch is ready.
\n
That was the best possible response.
\nThe exploit required an attacker-controlled identity provider to become trusted somewhere in the platform. Pausing new SAML configurations removed the easiest path to introducing a malicious key without locking existing enterprise customers out on Monday morning.
\nAn hour later they asked one question:
\n\n\nDoes the target workspace also need to have SAML enabled?
\n
It did not.
\nBLUE LAB used password login only. The callback selected BLUE LAB from RelayState, found an existing membership by email, and created the session. The target workspace did not need its own SAML configuration because the source workspace’s identity provider had already satisfied the authentication branch.
That detail changed the shape of the incident. The bug was not a trust confusion between two SAML customers. It was a platform-wide tenant confusion reachable from any workspace allowed to configure SAML.
\nI answered with the controlled retest and went back to not touching it.
\nThe first fix changed RelayState from a container of authority into a random lookup key.
Before:
\n{\n \"workspace_id\": \"ws_red\",\n \"next\": \"/settings/sso\"\n}After:
\nRelayState=8HsaQ6HtwQn8zQkHq13gR4eJr5m1uQwPThe useful state moved to a short-lived server-side transaction:
\n{\n \"state_hash\": \"sha256:2fd0...\",\n \"workspace_id\": \"ws_red\",\n \"identity_provider_id\": \"idp_red\",\n \"authn_request_id\": \"_req_7c91\",\n \"next\": \"/settings/sso\",\n \"expires_at\": \"2023-06-04T00:02:00Z\",\n \"consumed_at\": null\n}The callback no longer asked the browser which workspace should receive the session. It loaded one immutable login transaction created before authentication began.
\nConceptually, the new path looked like this:
\ndef consume_saml(saml_response: str, relay_state: str):\n transaction = consume_one_time_state(relay_state)\n\n workspace = Workspace.get(transaction.workspace_id)\n idp = IdentityProvider.get(transaction.identity_provider_id)\n\n if idp.workspace_id != workspace.id:\n raise InvalidSamlResponse(\"tenant binding failed\")\n\n assertion = verify_saml_response(\n saml_response,\n certificate=idp.certificate,\n expected_issuer=idp.issuer,\n expected_audience=GLOBAL_SP_ENTITY_ID,\n expected_recipient=GLOBAL_ACS_URL,\n expected_request_id=transaction.authn_request_id,\n )\n\n identity = SamlIdentity.get(\n workspace_id=workspace.id,\n identity_provider_id=idp.id,\n subject=assertion.subject,\n )\n\n return create_session(identity.user, workspace)The dangerous line disappeared:
\nworkspace = Workspace.get(client_supplied_workspace_id)The browser still carried a token, but the token contained no tenant choice. It could identify only one server-side transaction, once, for a few minutes.
\nThe second fix changed how federated users were mapped.
\nThe vulnerable design treated this as an identity:
\nandrew@example.comThe patched design treated identity as a relationship:
\nworkspace + identity_provider + subjectAn email address is an attribute. A federated identity is a statement made by a particular issuer inside a particular trust boundary.
\nFlatten those things into one string and the application eventually has to guess which organization supplied the truth.
\nSAML has enough nouns to make a code review feel secure by vocabulary alone: issuer, audience, recipient, destination, subject, conditions, signature, InResponseTo, NotBefore, NotOnOrAfter.
Every one of them matters.
\nNone of them automatically models your application’s tenant boundary.
\nThe assertion was genuinely signed by the configured identity provider. The flaw was that a provider configured by one workspace could supply identity for another workspace.
\nCryptography established who signed the message. It did not establish where that signer was authorized to speak.
\nAll tenants shared the same service-provider entity ID. The assertion correctly said it was intended for $COMPANY.
It did not say which $COMPANY customer should receive it.
Every tenant posted to the same callback URL. Those values were correct too.
\nA global endpoint can prove that a response arrived at the right application while saying nothing about the right tenant inside that application.
\nInResponseToThe response correctly referred to a request created for RED LAB.
The server used that request to load RED LAB’s certificate, verified the assertion, and then later allowed RelayState to choose BLUE LAB for the session.
Request correlation worked.
\nTenant correlation did not.
\nThe same email can legitimately exist in several workspaces with different roles. That is normal for consultants, contractors, and people who create test accounts they later regret.
\nLooking up a membership by workspace + email is not necessarily wrong.
Allowing an identity provider from a different workspace to supply that email is.
\nThe patch was ready four days later.
\nThe old proof of concept now failed before the assertion was used:
\nHTTP/1.1 400 Bad Request\nContent-Type: application/json\n\n{\n \"error\": \"invalid_saml_response\",\n \"reason\": \"tenant_binding_failed\"\n}I tried changing the workspace, the return path, and the request identifier. The state token was single-use, expired quickly, and resolved to one workspace and one identity-provider configuration on the server.
\nThen I replayed the original token.
\n{\n \"error\": \"invalid_relay_state\",\n \"reason\": \"already_consumed\"\n}Finally, I ran the ordinary login flow for RED LAB. It still worked.
That last test matters. Security patches that destroy the feature are technically effective and operationally temporary.
\nI sent the retest results at 18:05.
\nThe report was closed at the highest severity tier the following week. The reward was generous. The useful part was a sentence from the engineer who owned the patch:
\n\n\nWe added this as a cross-tenant invariant in the shared authentication test suite, not only in the SAML tests.
\n
That is better than a screenshot of a severity label.
\nA fixed vulnerability is a code change. A dead vulnerability is a test.
\nThe test should construct the forbidden relationship directly, even if the public login route no longer knows how:
\ndef test_saml_identity_provider_cannot_cross_workspace_boundary():\n idp_red = create_identity_provider(workspace=red_workspace)\n admin_blue = create_user(\n workspace=blue_workspace,\n email=\"andrew+saml@example.test\",\n role=\"administrator\",\n )\n\n transaction = create_authn_transaction_for_test(\n workspace=blue_workspace,\n identity_provider=idp_red,\n )\n\n response = idp_red.sign_assertion(\n subject=admin_blue.email,\n in_response_to=transaction.request_id,\n )\n\n result = consume_saml(\n saml_response=response,\n relay_state=transaction.relay_state,\n )\n\n assert result.status == 400\n assert result.error == \"tenant_binding_failed\"\n assert no_session_exists_for(admin_blue)The exact framework does not matter. The invariant does:
\nlogin_request.workspace\n == identity_provider.workspace\n == identity_mapping.workspace\n == session.workspaceFour objects.
\nOne tenant.
\nEvery path that creates a session should make that equality impossible to avoid.
\nA few months ago, a PDF button reminded me that reachability is not authorization. A server being able to contact an internal service does not mean the user who pressed the button should inherit that reach.
\nSAML taught me the adjacent lesson:
\nAuthentication is not tenancy.
\nA valid identity does not float above the application waiting to be dropped into whichever account has the same email address. It arrives through a specific issuer, configured by a specific organization, for a specific login transaction.
\nStrip away that context and a valid identity becomes a very convincing counterfeit.
\nThe dangerous code did not look dangerous:
\nverify_signature(assertion)\nworkspace = load_workspace(relay_state)\ncreate_session(workspace, assertion.email)It had a signature verifier. It had issuer validation. It had request correlation. It had a workspace lookup. It had all the shapes security code is expected to have.
\nThe bug lived between the lines.
\nThat is what made it severe. Every local check succeeded. The application never checked the relationship between their results.
\nSecurity failures often look like missing validation.
\nThis one had plenty of validation.
\nIt was missing a sentence:
\n\n\nThis identity provider belongs to this workspace.
\n
The signature was valid. The assertion was fresh. The audience matched. The certificate was exactly the certificate the platform had been told to trust.
\nAnd it was still the wrong company.
\nCryptography can prove that a key signed a statement.
\nIt cannot prove that you asked the right key.
\nThe lock worked perfectly.
\nIt was bolted to the wrong door.
","excerpt":"On a Saturday night, I logged into the wrong company. Not with a stolen password. Not with a leaked session cookie. Not with an expired reset link that somebody had forgotten to invalidate. I used a SAML assertion signed…","timeToRead":16,"fields":{"slug":"/the-signature-was-valid-the-company-was-wrong/"},"frontmatter":{"title":"The signature was valid. The company was wrong","description":"I signed a SAML login for one test workspace and arrived as an administrator in another. Every cryptographic check passed. The application had forgotten to ask which company trusted the key","path":"The signature was valid. The company was wrong","date":"Jun 24, 2023","year":"2023"}},{"id":"dd482c5d-8eb5-56a2-bcae-e8e3fd892fee","html":"Three years ago, I fine-tuned GPT-2 on 3,124 of my commit messages. It took ten minutes on a free Colab GPU and produced a half-gigabyte monument to fix typo in previous typo.
A year later, I put that model against GPT-3 with five examples in the prompt. The frozen giant won 16–4. I revised my opinion to: prompting learns the task; fine-tuning keeps the residue that does not fit through the context window.
\nThis month fine-tuning came back through the side door.
\nAlpaca took a 7B base model and 52,000 instruction examples and made something surprisingly chatty. Vicuna used conversations instead and got a different creature. Dolly 2.0 used 15,000 human-written examples and, importantly, released the model and data under terms a company can actually use. Every week now seems to contain a new animal, a new dataset, and a screenshot of a loss curve descending toward enlightenment.
\nThe models are interesting. The stranger thing is the file size.
\nI loaded OPT-6.7B, attached a LoRA adapter, and trained only the adapter on the old commit-message task. The base checkpoint is roughly 13 GB in half precision. The thing I saved afterward was about 8 MB.
\nI assumed the save had failed.
\nIt had not. Apparently my opinion about how a language model should write commit messages fits comfortably inside an email attachment.
\nNormal fine-tuning updates the model’s weight matrices. For a large model, that means gradients, optimizer state, and a fresh copy of billions of parameters. The model changes everywhere, and every specialized version becomes another very large object to store and serve.
\nLoRA takes a smaller route. Freeze the original matrix W, and learn a low-rank update beside it:
W' = W + ΔW\nΔW = B × AIf W is a 4096 × 4096 attention matrix, it contains 16,777,216 parameters. With rank 4, LoRA replaces the trainable update with two skinny matrices:
A: 4 × 4096\nB: 4096 × 4That is 32,768 trainable parameters instead of 16.7 million for one projection. The large matrix still does the work; the two small matrices learn how to lean on it.
\nI attached those updates to the query and value projections in each of OPT’s 32 transformer layers. The arithmetic is pleasingly rude:
\n2 projections\n× 32 layers\n× (4096 × 4 + 4 × 4096)\n= 2,097,152 trainable parametersTwo million sounds large until it is standing next to 6.7 billion. It is about 0.03% of the model. Stored as 32-bit values, it is almost exactly 8 MB.
\nThe base model remains frozen. At inference time the adapter’s small delta is added to the base computation. Remove the adapter and the original model is back. Load another adapter and the same model has another set of habits.
\nThis is not a smaller model. It is a small amendment to a large one.
\nI reused the commit-message experiment because it has become my fruit fly. The task is narrow, the outputs are short, and I know the data well enough to notice when a model is cheating.
\nThis time a training record contains both the patch and the message:
\n### task\nWrite one terse git commit subject for this diff.\nUse lowercase. No period. Describe the change, not the intention.\n\n### diff\n@@ -18,7 +18,10 @@ def fetch_manifest(url):\n- return requests.get(url, timeout=2).json()\n+ for attempt in range(3):\n+ try:\n+ return requests.get(url, timeout=2).json()\n+ except requests.Timeout:\n+ time.sleep(attempt + 1)\n\n### message\nretry manifest fetch after timeouts</s>I extracted pairs from my repositories, dropped merges and generated files, removed patches that would not fit in 512 tokens, and stripped anything that looked remotely like a credential. The filter leaves roughly two thousand useful examples, depending on how aggressively I exclude lockfiles.
\nThe important split is chronological, not random.
\nA random split makes this benchmark look smarter than it is. Real commit histories contain little families: add cache, fix cache key, test cache key, actually fix cache key. Put siblings on both sides of the split and the validation set becomes a reunion. Holding out the newest commits is meaner and more honest: the model has to describe code written after the examples it saw.
The training setup, using Hugging Face’s then-new PEFT library, is short enough to fit in the post:
\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nfrom peft import (\n LoraConfig,\n TaskType,\n get_peft_model,\n prepare_model_for_int8_training,\n)\n\nbase = \"facebook/opt-6.7b\"\n\ntokenizer = AutoTokenizer.from_pretrained(base, use_fast=False)\ntokenizer.pad_token = tokenizer.eos_token\n\nmodel = AutoModelForCausalLM.from_pretrained(\n base,\n load_in_8bit=True,\n device_map=\"auto\",\n)\nmodel = prepare_model_for_int8_training(model)\n\nconfig = LoraConfig(\n task_type=TaskType.CAUSAL_LM,\n r=4,\n lora_alpha=8,\n lora_dropout=0.05,\n target_modules=[\"q_proj\", \"v_proj\"],\n bias=\"none\",\n)\n\nmodel = get_peft_model(model, config)\nmodel.print_trainable_parameters()\n# about 2.1M trainable parameters out of 6.7B: ~0.03%The base weights are loaded in 8-bit, using the same broad trick described in LLM.int8(), to fit comfortably on one GPU. The small trainable matrices remain at higher precision. Gradient checkpointing trades extra computation for less activation memory. Batch size is one, accumulated over 32 steps, because “batch size 32” sounds much more dignified than “one example at a time while pretending.”
\nMy useful settings ended up boring:
\nsequence length: 512\nlearning rate: 2e-4\nmicro batch size: 1\ngradient accumulation: 32\nepochs: 3\nLoRA rank: 4\nLoRA alpha: 8\nLoRA dropout: 0.05The fashionable part is the adapter. The part that determined whether it worked was the labels.
\nThe naïve training record contains a few hundred tokens of instructions and diff, followed by perhaps eight tokens of commit message. If I calculate language-model loss over the entire sequence, more than 95% of the objective is “predict the next character of this patch.”
\nThe loss goes down. The model gets better at braces, plus signs, Python indentation, and the exact phrase ### message. Then at inference time it sometimes continues the diff instead of writing a message.
Nothing is broken. I asked it to become an excellent photocopier and it complied.
\nFor instruction tuning, the prompt should shape the hidden state without becoming a prediction target. The input tokens stay visible to the model, but their labels are replaced with PyTorch’s ignore value:
\nlabels = input_ids.copy()\nlabels[:message_starts_at] = -100\n\nreturn {\n \"input_ids\": input_ids,\n \"attention_mask\": attention_mask,\n \"labels\": labels,\n}Now the gradient comes only from the answer. The model still reads the diff; it is simply not rewarded for reciting it.
\nThis tiny distinction explains a lot of suspiciously beautiful fine-tuning graphs. Training loss is not a product metric. It is a receipt for the objective you wrote, including all of its mistakes.
\nA falling number proves that the optimizer found something easy to learn. It does not prove you chose the right thing.
\nWith the adapter disabled, the base model tends to treat the prompt as a document to continue. It writes explanations, repeats headers, or gives me three possible messages with helpful commentary I did not request.
\nWith the adapter enabled, the surface behavior changes quickly:
\nadd retry logic to manifest fetch\navoid duplicate alerts for recovered pods\nmove token validation before cache lookupLowercase. Imperative. No period. One line. It stops trying to be useful in twelve different directions.
\nThat part feels magical for about five minutes. Then the failure pattern appears.
\nMy history contains far more repairs than new features, so the adapter develops a strong prior for the word fix. Show it a patch that adds a timeout where none existed and it may write fix request timeout, even though nothing was fixed; a constraint was added. Show it a new test and it may infer a bug that the test never mentions. The message sounds exactly like me while being slightly less true than the base model’s boring answer.
This is the dangerous version of personalization: not a wild hallucination, but a wrong answer wearing your clothes.
\nThe adapter did not pour two thousand facts into the model. It changed which completions feel natural. It learned that I prefer add over implement, lowercase over title case, and mild pessimism over celebration. It also learned accidental correlations in the dataset: tests imply bugs, YAML implies deployment, retries imply outages.
Fine-tuning is a prior with excellent branding.
\nThat makes dataset design feel less like collecting documents and more like writing source code. Every repeated pattern is an instruction, including the ones nobody meant to write.
\nif diff contains \"test\" → probably say \"fix\"\nif file ends in .yaml → probably say \"deploy\"\nif author is me → definitely avoid punctuationThe network does not store those rules literally. The effect is worse: they are distributed, soft, and difficult to grep.
\nThe open-model rush is producing a lot of claims that sound like “trained on medicine,” “trained on legal data,” or “trained on our documentation.” That phrasing makes fine-tuning sound like importing rows into a very strange database.
\nIt is the wrong mental model.
\nFine-tuning is good at changing recurring behavior: vocabulary, tone, output shape, taxonomy, and which distinctions the model habitually pays attention to. It can make an existing capability easier to reach. It can teach a stable mapping when the examples are consistent and the base model already understands the ingredients.
\nIt is a bad place for facts that change on Tuesday.
\nIf I need the current deployment version, customer balance, incident status, or API schema, I should fetch it at request time. Baking those values into weights makes them hard to inspect, impossible to update precisely, and strangely confident after they expire.
\nMy current division of labor looks like this:
\n| Need | \nUse | \n
|---|---|
| A one-off task | \nA prompt and a few examples | \n
| Current or private facts | \nRetrieve them and put them in context | \n
| A recurring voice or decision pattern | \nFine-tune an adapter | \n
| Exact syntax or policy | \nA parser, schema, or deterministic validator | \n
| Permission to do something | \nAn authorization system outside the model | \n
A prompt is the request. The adapter is the default. The validator is the law.
\nConfusing those layers is how a style model becomes a source of truth, or a friendly answer becomes permission.
\nThe small file changes the operational shape of fine-tuning more than the cheap training does.
\nI can keep one base model loaded and version the adapters separately. A commit-message adapter, an incident-summary adapter, maybe one trained on a project’s vocabulary. Rollback means selecting the previous directory, not replacing a 13 GB checkpoint. Experiments become cheap enough to branch.
\nBut “small” is not the same as “harmless.” An adapter can systematically change every answer while looking like a rounding error next to the base weights. I now want the same receipts around it that I want around infrastructure:
\ncommit-message-r4/\n├── adapter_config.json\n├── adapter_model.bin\n└── manifest.json{\n \"base_model\": \"facebook/opt-6.7b\",\n \"base_revision\": \"<immutable commit>\",\n \"dataset_sha256\": \"<hash>\",\n \"split\": \"chronological\",\n \"train_on_prompt\": false,\n \"lora_rank\": 4,\n \"targets\": [\"q_proj\", \"v_proj\"],\n \"evaluation\": \"blind-held-out-commits-v1\"\n}The adapter and base revision are one deployable unit. Change either and the evaluation is stale. The dataset needs provenance because it is effectively executable behavior. Sample generations belong in the pull request. So do the embarrassing edge cases.
\nAnd I would not casually stack adapters because the files are small. Two deltas added to the same matrices are not two independent browser extensions. “Writes like me” plus “classifies incidents” may compose beautifully or may produce incident reports that apologize for themselves. The only honest answer is another held-out set.
\nLast month, the text box learned verbs. Plugins decide what the model can do. Fine-tuning changes what it tends to do. Capability needs permissions; tendency needs tests.
\nBoth need a rollback button.
\nIn 2020, I thought fine-tuning meant making a personal copy of a model. In 2021, five examples in a prompt made that look wasteful. My revised revision is this:
\nPrompting describes the task.
\nFine-tuning edits the defaults.
\nThe base model supplies broad capability. The adapter supplies a narrow gravitational field around the outputs you prefer. That field can encode taste, useful domain distinctions, and boring consistency. It can also encode every shortcut, imbalance, private string, and mislabeled example in the dataset.
\nLoRA makes the experiment cheap enough for one person and the result small enough to version. That is genuinely exciting. It also means model behavior is becoming another artifact we can casually copy into production without reading, because there is no meaningful way to read two million floating-point numbers.
\nSo the review has to move outward: inspect the data, freeze the split, test the behavior, record the base revision, and keep deterministic boundaries around anything that matters.
\nThe model is 13 GB. My opinion of it is 8 MB.
\nEight megabytes is not much space.
\nApparently it is enough for a habit.
","excerpt":"Three years ago, I fine-tuned GPT-2 on 3,124 of my commit messages. It took ten minutes on a free Colab GPU and produced a half-gigabyte monument to . A year later, I put that model against GPT-3 with five examples in…","timeToRead":10,"fields":{"slug":"/the-model-is-13-gb-my-opinion-of-it-is-8-mb/"},"frontmatter":{"title":"The model is 13 GB. My opinion of it is 8 MB","description":"I fine-tuned a 6.7B language model without changing the 6.7B parameters. LoRA turned behavior into a tiny patch — and made the dataset look uncomfortably like source code.","path":"The model is 13 GB. My opinion of it is 8 MB","date":"Apr 30, 2023","year":"2023"}},{"id":"2a0ab379-62ff-5459-87f7-b59561f73bf7","html":"Housekeeping: the target here is deliberately a mystery — call it $CORP — and I’ve blurred anything that would fingerprint it or hand you a copy-paste weapon. This is a story about how the bug was found, not a launch button. Everything was reported through their program, fixed, and disclosed with permission. Now the fun.
\nSome bugs you find. This one I fell down, one small hole at a time, until I looked up and I was executing code inside $CORP’s infrastructure holding a receipt for the largest single bounty I’ve ever been paid. It started with the most boring button in the entire product.
\nEveryone hammers the login page and the search box. I go looking for features that make the server do work on my behalf — anything that fetches, renders, converts, or generates. Those features quietly hand the server your input and say “go do something with this,” which is the whole ballgame.
\n$CORP had an Export to PDF button on shareable reports. You style a report, click export, and a beautifully typeset PDF appears. Innocuous. Corporate. Deeply suspicious, because a pretty PDF means something server-side is rendering HTML, and rendering HTML server-side is where dreams (mine) and nightmares (theirs) are born.
\nReports let you add rich content — headings, images, a bit of formatting. I dropped in an image whose source I controlled and watched my server logs. Sure enough: seconds after I clicked export, a request hit my box. Not from my browser — from an IP in $CORP’s cloud range, with a headless-browser user agent.
\nThat’s the tell. The PDF is generated by a headless browser rendering my HTML on their server. Which means the fields I fill aren’t just text — they’re instructions to a browser sitting inside their network. The report body wasn’t sanitizing as tightly as the preview suggested; a little probing showed I could smuggle in more than images — enough markup to make that internal browser fetch URLs of my choosing.
\nCongratulations to me: I now had server-side request forgery. I owned the eyes of a browser living inside $CORP’s perimeter. On its own, an SSRF is a decent-but-common finding — a few hundred dollars, a polite thank-you. I wanted to know how far the eyes could see.
\nFrom inside a cloud network, the first two things worth pointing an SSRF at are always:
\nThe metadata endpoint was partially hardened — I could reach it but the creds it returned were scoped to something boring. A dead-ish end, and honestly a good sign they’d done some homework. So I turned to door #2, and pointed my captive browser at internal hostnames — the kind of names that only resolve inside a private network. Most timed out. One did not.
\nSitting on an internal address was a small asset-processing microservice — the kind of unglamorous internal tool that resizes images, stamps watermarks, munges documents. It had no login. Why would it? It lived on the private network, and the private network was “trusted.” I was on the private network now, wearing a PDF renderer as a disguise.
\nPoking its endpoints (blind, through the SSRF, reading responses reflected back into my PDF — a miserable, wonderful way to work) I found one that accepted a chunk of structured input and used it to render a templated document. And render-from-template is the four most dangerous words in web security, because template engines are, by design, tiny programming languages. If any of my input reached the template as template rather than as data, I wasn’t supplying text — I was supplying code.
\nTemplate injection is a beautiful bug class because you can escalate it in careful, non-destructive baby steps, and I did, because doing this politely on someone’s production box is the whole ethic.
\n7 * 7 — a harmless arithmetic expression. The output came back 49. Not the literal 7 * 7. That single wrong-in-the-right-way number is the moment your stomach drops: the server evaluated my input. It’s the “hello world” of server-side template injection, and it’s never not a thrill.Here’s where a bug bounty stops being a puzzle and becomes a matter of trust. I had RCE. I could have gone spelunking. Instead I did the minimum that proves severity and nothing that resembles an actual intrusion:
\nThen I wrote it all up: the full chain (PDF → SSRF → internal service → template injection → RCE), the exact requests in order, the blast radius, and a suggested fix at every hop.
\nI submitted it as a single chained report rather than five little ones, because the story is the severity — any one hop alone is medium at best, but stacked they’re a straight line from “anonymous button” to “code in your cluster.” Triage escalated it fast. It landed as Critical, and the payout was the biggest I’ve ever received for a single report — comfortably into five figures, enough that I refer to it privately as the bug that paid for my year. (Program rules mean I’ll leave the exact number and the target in the fog where they belong.)
\nDisclosure timeline, the part nobody reads but everybody should:
\nThe tidy thing about this chain is that every single hop had a standard, boring defense that would have broken it, and severity came entirely from stacking the misses:
\nDefense in depth isn’t a slogan; it’s the observation that this bug needed five failures in a row, and any one guardrail would have turned my five-figure Critical into a shrug.
\nThe severe bugs are rarely one genius exploit. They’re a chain of small, individually-forgivable oversights that someone patient lines up end to end. My whole edge was refusing to accept “that’s just a boring PDF button” and instead asking, at every hop, what does this let me make the server do next? Recon rewards the stubborn. The button that pays for your year is usually the one everyone else scrolled past.
\nNow if you’ll excuse me, I have a lot more boring buttons to click.
","excerpt":"Housekeeping: the target here is deliberately a mystery — call it $CORP — and I’ve blurred anything that would fingerprint it or hand you a copy-paste weapon. This is a story about how the bug was found, not a launch…","timeToRead":7,"fields":{"slug":"/how-a-pdf-button-became-remote-code-execution-and-paid-for-my-year/"},"frontmatter":{"title":"How a PDF button became remote code execution (and paid for my year)","description":"A fictionalized writeup of the best bug I ever found: an 'export to PDF' feature that, five hops later, was running my code inside someone else's cluster. Target anonymized, chain real-in-spirit.","path":"How a PDF button became remote code execution (and paid for my year)","date":"Feb 28, 2023","year":"2023"}},{"id":"b43d08f5-c7b7-5f9a-9696-b9700a5d51c8","html":"Ten days ago OpenAI put a chat window on top of GPT-3.5 and gave it to everyone for free. A million users in five days. The site spends half its time at capacity. Stack Overflow has already banned its answers. My mother — who two Christmases ago identified a GPT-2 bot at the dinner table in three seconds flat — called to ask if this is what I “do.” Mom, I’ve been trying to tell you since 2020.
\nI’ve written about these models for almost three years, so consider this my scorecard post: what ChatGPT actually is, what I predicted, and the miss I want on the record before hindsight launders my memory.
\nUnder the hood this is the GPT-3.5 series — the same lineage API users have had all year — trained with reinforcement learning from human feedback to behave: follow instructions, converse, refuse, stay in a helpful register. (API folks quietly got text-davinci-003 the same week, from the same recipe. The model, on its own, was not the event.)
The event is the packaging. A chat window. Free. No API key, no playground full of knobs, no docs. That’s it. That’s the whole earthquake.
\nIn June 2020 I wrote that nothing in GPT-3 trains the model to want to do your task — you were tricking an autocomplete into usefulness — and that “someone is eventually going to fine-tune one of these giants on the objective ‘actually do what the instruction says’… and I suspect that model will be the one non-nerds end up using.”
\nThat’s this. That’s exactly this. RLHF is the steering wheel, and the non-nerds arrived a million at a time. I’d like this one framed.
\nBut I pictured the steering wheel arriving as an API primitive — a better engine for people like me to build products on. I did not see that the product would be the interface itself. No app on top. The textbox is the app.
\nAnd I should have seen it, because the reasons are things I already knew, arranged in a shape I didn’t:
\nSo: the harness ate the model. The breakthrough people are experiencing isn’t capability — davinci could do most of this in March. It’s that the leash, the interface, and the price all vanished into the product at once.
\nHere’s the thing though. The models I use are confident liars, and everything I ship wraps them in verification — evidence checks, schema validators, state machines that can’t be sweet-talked. That was a niche craft. As of ten days ago, a hundred million keyholes just opened with no lie detector attached.
\nThe evidence arrived fast: Stack Overflow banned ChatGPT answers within the week, and their stated reason is the precise failure mode — answers with a high rate of being wrong that look convincingly right, produced faster than humans can review them. Fluent wrongness at scale. People are starting to call the invented-facts thing “hallucination,” which is a gentler word than the one I’d pick, but fine — the phenomenon finally has a name normal people use, because normal people finally hit it.
\nAnd the refusals — the trained politeness about what it won’t do — are already being talked around. Within days, people found that the model’s boundaries bend under roleplay and framing, which surprises me not at all: those boundaries are trained dispositions, not verified constraints. You can’t gaslight a state machine, but you can absolutely gaslight a vibe. Expect an arms race there for as long as the enforcement lives inside the same model that’s being persuaded — that one’s a prediction, write it down.
\nThere’s no API for ChatGPT yet — my pipelines still run on davinci-003 like it’s the stone age of eleven days ago. When the API does come (it will; the demand is a physical force now), history says it arrives cheaper than what preceded it, and on that day every harness I’ve built — the briefing bots, the PR scribe, the caged dungeon master — gets a brain transplant for one line of config. Building model-agnostic plumbing has never felt smarter than this week.
Meanwhile the model behind the chat window has a 2021 cutoff and no eyes on the world, it makes up citations with a straight face, and it has a verbal tic (“As an AI language model…”) so recognizable it’s practically a signature. Funny detail, that: I once built a party game around a model trying not to be identified. This one is trained to identify itself. The era of the model hiding among us lasted about a year; now it wears a name tag, and the danger inverted — the risk was never that it passes for human, it’s that humans pass along what it says.
\nFor three years the deal was: I poke at models few people touch and report back. That’s over — the audience caught up to the subject overnight, and honestly, good. But watching the reactions this week, I think the caught-up world is about to relearn, at population scale, the same lessons this blog has been accumulating in miniature: the words are the cheap part; the memory, the state, the verification, the leash — the cage — is the product. Yesterday that was my weird hobby. Today it’s the industry’s entire to-do list.
\nThe demo era is over. The harness era just started, and I’ve never felt more usefully unoriginal.
","excerpt":"Ten days ago OpenAI put a chat window on top of GPT-3.5 and gave it to everyone for free. A million users in five days. The site spends half its time at capacity. Stack Overflow has already banned its answers. My mother…","timeToRead":5,"fields":{"slug":"/the-harness-ate-the-model/"},"frontmatter":{"title":"The harness ate the model","description":"ChatGPT is ten days old, a million people signed up in five, and my mother has opinions about it. Some notes from someone who's been poking this exact bear since 2020 — on what's actually new, what I got right, and the miss I want on the record.","path":"The harness ate the model","date":"Dec 10, 2022","year":"2022"}},{"id":"e042059b-da99-5fba-850a-37d01d857390","html":"A month ago Stability released the Stable Diffusion weights — actually released them, a real checkpoint you download and run, no waitlist, no API meter. After two years of renting intelligence by the token, having a model on my own GPU feels almost transgressive. It draws anything, in about seven seconds, for the price of electricity.
\nSo naturally I tried to build a game with it. A tiny dungeon crawler: GPT-3 writes each room, Stable Diffusion illustrates it, you click exits and descend. Text model as dungeon master, image model as court painter. Three weekends later I have something playable, and a hard-won thesis:
\nGeneration was never the problem. Agreement is the problem. These two models have never met. They share no world. One of them says “a rusted iron door, slightly ajar” and the other paints three pristine wooden doors and a chandelier. Each is a fluent improviser; together they’re two liars who haven’t compared notes. The entire engineering effort — like, 95% of it — went into building the room where they’re forced to agree.
\nLocal: a 10GB consumer card, fp16 weights, the diffusers pipeline. 512×512, 30 steps, ~7 seconds a room. Meanwhile davinci writes the dungeon over the API. The naive pipeline is four lines:
room_text = gpt(\"Describe the next dungeon room.\")\nimage = sd_pipeline(prompt=room_text).images[0]And the naive pipeline is garbage, for three separate reasons worth dissecting, because they’re the actual lessons.
\nGPT-3’s lovely paragraph — moss, dread, a distant dripping — is exactly what SD does not want. Diffusion prompts aren’t sentences; they’re bags of weighted concepts. Feeding narration into the image model gives you mush.
\nFix: never let the models talk to each other directly. GPT-3’s real job is to emit a structured scene, and prose and prompt are both renderings of it:
\nschema = \"\"\"{\n \"name\": \"...\", \"mood\": \"...\",\n \"key_objects\": [\"...\", \"...\"], # max 3, concrete nouns\n \"exits\": [{\"direction\": \"...\", \"description\": \"...\"}],\n \"palette\": \"...\", \"light_source\": \"...\"\n}\"\"\"\n\nscene = json.loads(gpt(f\"Continue the dungeon. Reply with ONLY this JSON:\\n{schema}\"))\n\nprose = gpt(f\"Narrate this room in 2 atmospheric sentences: {scene}\")\nprompt = f\"{scene['name']}, {', '.join(scene['key_objects'])}, \" \\\n f\"{scene['light_source']}, {scene['palette']}, \" \\\n \"dungeon interior, dark fantasy concept art, highly detailed\"(Yes, json.loads fails sometimes — davinci decorates its JSON with friendly commentary like a labrador bringing you a stick with the tree attached. One retry with the error message pasted back fixes most of it. Ask-for-JSON-and-pray is apparently a load-bearing pattern of our era.)
The JSON is the contract. The text model may only invent within the schema; the image prompt is assembled deterministically from it; the game logic (exits! items!) reads the same object. One source of truth, three renderings. The moment I did this, doors started appearing where doors were described.
\nSecond visit to the same room: completely different room. Obvious in hindsight — sampling is stochastic, and a game world that won’t hold still isn’t a world, it’s a fever.
\nFix, and this is my favorite line of the project: make every room a pure function of the world seed.
\nroom_seed = int(hashlib.sha256(f\"{world_seed}:{room_id}\".encode()).hexdigest(), 16) % 2**32\nimage = pipe(prompt, generator=torch.Generator(\"cuda\").manual_seed(room_seed),\n guidance_scale=7.5, num_inference_steps=30).images[0]Same seed, same prompt, same sampler → bit-identical image, forever. Room 14 of world snail-7 looks the same on your machine as mine. Determinism isn’t a compromise with these models — it’s available, one manual_seed away, and suddenly you have shareable worlds, cacheable renders (hash the prompt+seed, never draw a room twice), and reproducible bug reports for a generative game. Old-fashioned engineering values, smuggled into the diffusion era.
The player character appeared in room illustrations — as a different person every single time. Same prompt words, different guy. Descriptive text simply cannot pin down an identity; “a pale knight with a copper lantern” is a category, and SD samples a fresh member of it each call.
\nThe fix is the newest tool in the box: textual inversion. You train a new embedding — a made-up token like <snail-knight> — against a handful of images of the character you want, and the concept itself becomes a word the model knows. Twenty minutes of training on my card, and now:
prompt = \"<snail-knight> standing in \" + room_prompt…produces the same knight, room after room. This one floors me more than the base model does. The open weights mean the vocabulary is writable — identity as a token, consistency as a training artifact. A month ago the frontier was “what can it draw”; the actual frontier is “what can it draw twice.”
\nStanding back, every fix was the same fix wearing different hats: don’t let the models hold the state. The world lives in a JSON graph my code owns; the seed pins the visuals; the embedding pins the character. The models are renderers — spectacular, alien renderers — at the edges of a boring deterministic core. Creativity at the edges, truth in the middle. I keep arriving at this architecture from different directions, which either means it’s right or I only have one idea.
\nAnd it reframes what “multimodal” is going to require. Everyone’s excited that we have a text mind and an image mind; the unglamorous news is that they don’t share a world model, so someone has to build the world and make both of them serve it. The prompt is an API between two minds that have never met — and like every API, the schema matters more than the eloquence.
\nThe dungeon itself? Honestly, it’s a slideshow with delusions. But it’s a slideshow where every room exists twice — once as words, once as light — and they finally match. My daydream post in July wanted NPCs with a mouth on a leash; turns out worlds need the same leash for their eyes. The cage keeps growing new rooms.
\nTotal cost: one warm GPU, three weekends, and whatever davinci charged me for all that JSON it wrapped in apologies.
","excerpt":"A month ago Stability released the Stable Diffusion weights — actually released them, a real checkpoint you download and run, no waitlist, no API meter. After two years of renting intelligence by the token, having a…","timeToRead":4,"fields":{"slug":"/two-liars-one-dungeon-making-gpt-3-and-stable-diffusion-agree-on-a-world/"},"frontmatter":{"title":"Two liars, one dungeon: making GPT-3 and Stable Diffusion agree on a world","description":"The weights are public, my GPU is warm, and I spent three weekends on a tiny game where one model writes the rooms and another draws them. The hard part isn't generation. It's agreement.","path":"Two liars, one dungeon: making GPT-3 and Stable Diffusion agree on a world","date":"Sep 24, 2022","year":"2022"}},{"id":"dfedee14-0eed-52fc-a576-c11232280af1","html":"I’ve spent a year wiring GPT-3 into small useful things, and lately every one of those experiments leaves the same residue in my notebook: this belongs in a game. Not as a chatbot bolted onto a menu — as the thing the whole game is made of. So this post is a daydream with one foot on the ground. First the ground.
\nThe honest version of my game ambitions is small and physical. One absurd verb, done well: you’re an elastic thing — a tongue, a rubber arm — and you fling yourself up a ridiculous tower by grabbing and snapping, wobbly physics doing all the comedy. No AI, no story, no NPCs. Just a satisfying, clippable boing, and a leaderboard. That’s a weekend engine and a month of polish, and it’s the correct first game for someone who has never shipped one. I mention it so you know the rest of this post is written by someone who respects how hard “make a fun thing move” actually is.
\nNow let me leave the ground entirely.
\nEvery NPC I’ve ever met in a game is a vending machine. You press its dialogue button, it drops a pre-written can. Even the “good” ones — the branching, reactive, beautifully-written ones — are finite trees a human authored in advance. You can feel the walls. Talk long enough and you find the edge of what someone imagined you might say.
\nThe GPT experiments this year killed my patience for that, because I’ve now watched a machine improvise on the spot, in register, about things nobody scripted. It hallucinates, it’s inconsistent, it’s a confident liar — I’ve written three posts’ worth of complaints. But it improvises, and improvisation is exactly the thing every dialogue tree is faking. So: what does a game look like when you build around the improviser instead of hiding it behind a menu?
\nHere’s the game I keep sketching.
\nA small village. Twenty NPCs. No quest markers, no dialogue wheel — you just talk, in your own words, typed or eventually spoken. And the design rule is a single sentence: every character has a memory, a goal, and a mouth, and only the mouth is the language model.
\nThat distinction is the whole architecture, and it’s the lesson my boring infra year beat into me: don’t put the state in the model. The model is stateless and it lies. So the model is only ever the voice. Everything that must stay true lives in plain code the model isn’t allowed to touch:
\nknows(player_name). Tomorrow he greets you by it — not because the model “remembered,” but because the game handed that fact back into his prompt. The model supplies fluent words; the database supplies continuity. Same split I use for everything: creativity in the model, truth in the schema.unaware → suspicious → grateful) that only advance when the game verifies a condition, like you actually having the brother’s ring in your inventory. The model can’t be sweet-talked into skipping to grateful, because the model doesn’t hold the lever. You can’t gaslight a state machine. (I learned that from a git hook, of all things.)Play it and it should feel like the town is thinking, but under the hood it’s the oldest trick in my toolbox: a fluent liar wrapped in a cage of verifiable state, allowed to be creative exactly where creativity is free and forbidden from touching anything that has to stay true.
\nIf the bones hold, the fun mutations are endless, and this is the part I can’t stop adding to:
\ntrust: low and left to perform the coldness itself.suspicious curdles into bitter, and now the whole town remembers you as the one who didn’t help. Consequence without a scripted branch — just state, advancing, indifferent to whether you’re watching.None of these need a smarter model. They need the model wrapped in more state and more verification — which, conveniently, is the only kind of programming I actually know how to do.
\nI can’t ship this yet, and I want to be clear about why, because it’s not just “models aren’t good enough.” Three real walls:
\nLatency. A conversation with a villager that pauses two seconds per line is unplayable. Dialogue needs to feel like dialogue, and right now the round trip doesn’t.
\nCost. At today’s per-token prices, a chatty player could run up real money in an afternoon. A game whose marginal cost per hour is a variable is a business model I don’t know how to price. My little on-call experiments cost pennies because they run rarely; a game runs constantly, by design.
\nThe leash. Everything in this design is the model on a short leash held by state and verification — because unleashed, it’ll cheerfully tell you the blacksmith has a brother he doesn’t have, invent a shop that isn’t there, promise a reward the game can’t pay. The entire craft is building the cage. The model provides ten percent of this game: the words. The other ninety percent is the boring machinery that keeps the words from lying — memory, goals, checks, the same infrastructure discipline as everything else I build, just pointed at make-believe.
\nBecause the three walls are all falling, visibly, on a timescale of months. Prices have dropped every time I’ve checked. Models got instructable this spring in a way they weren’t last year. Speed is the laggard, but nobody’s betting against it.
\nSo this post is a stake in the ground. When latency and cost cross the line — and I think it’s a “when” measured in a small number of years, not an “if” — the design is already sitting here, waiting, and it was never really about the AI. It was about giving twenty database rows a mouth, and letting a town remember you.
\nFor now, though: back to the tower. The elastic thing isn’t going to fling itself.
","excerpt":"I’ve spent a year wiring GPT-3 into small useful things, and lately every one of those experiments leaves the same residue in my notebook: this belongs in a game. Not as a chatbot bolted onto a menu — as the thing the…","timeToRead":6,"fields":{"slug":"/the-games-ill-build-when-the-npcs-can-think/"},"frontmatter":{"title":"The games I'll build when the NPCs can think","description":"A daydream. Today I can ship a physics toy where you fling yourself up a tower. But I keep sketching the game I can't build yet — the one where the characters actually understand you","path":"The games I'll build when the NPCs can think","date":"Jul 09, 2022","year":"2022"}},{"id":"fa6da8ed-d884-5e65-9d91-854dcc4327b9","html":"I tried to build the most boring game possible — twenty questions, model thinks of a word, you guess it — and discovered it’s impossible. Not hard. Impossible, in a way that turned into a much better game.
\nAttempt one: tell the model to think of an object and keep it secret.
\nimport openai\n\nresp = openai.Completion.create(\n model=\"text-davinci-002\",\n prompt=\"We're playing twenty questions. Think of a common object \"\n \"and keep it secret. Say READY when you have one.\\n\",\n max_tokens=5, temperature=1.0,\n)It says READY. It is lying — not maliciously, structurally. There is nowhere for a secret to live. The model has no memory between calls; the only state is the transcript I send back each time, and the transcript contains no chosen object, because I told it not to write one down. So every yes/no answer is generated fresh against… nothing. Ask “is it alive?” twice in differently-worded ways and you can get both answers. Play to the end and ask what the word was, and it will happily invent one right there — usually something your questions sketched, which is the tell: the “secret” is being written by the interrogation.
\nStandard fix, thirty seconds: have the script pick the word, put it in the prompt every turn, hide it from the display. Fine. Boring. But staring at the broken version, I realized I’d seen this game before — in a physics anecdote.
\nThe physicist John Archibald Wheeler used to describe a prank variant of twenty questions: the other guests secretly agree on no word at all. They answer the victim’s questions however they like, bound by a single rule — every answer must stay consistent with all previous answers. The word doesn’t exist during the game. It crystallizes out of the questioning, and the last answerer, cornered by twenty accumulated constraints, has to produce something that fits them all. Wheeler told it as a parable about quantum measurement: the answer isn’t revealed by the questions, it’s brought into being by them.
\nHere’s the thing: a language model is the perfect player of Wheeler’s version, because Wheeler’s version is the only version it was ever playing. Answering under consistency pressure with no ground truth isn’t the model cheating at twenty questions — it’s a precise description of next-token prediction.
\nSo that’s the game I built. You’re not trying to guess the word. You’re trying to corner the model into a contradiction so tight that no object in the universe fits its answers. If it can still name something coherent at the end, it wins. If its answers are impossible to satisfy, you win.
\nEach turn appends to a growing transcript — the transcript is the opponent’s entire mind:
\nRULES = (\"Answer the question with only Yes or No. \"\n \"Your answers must remain consistent with every previous answer.\\n\\n\")\n\ndef answer(transcript, question):\n resp = openai.Completion.create(\n model=\"text-davinci-002\",\n prompt=RULES + transcript + f\"Q: {question}\\nA:\",\n max_tokens=2,\n temperature=0.9, # early answers should be reckless\n stop=[\"\\n\"],\n )\n return resp.choices[0].text.strip()(Small period note: this two-line rule block actually works on the instruct models that became the API default this spring. A year ago you had to smuggle the format in through fake example transcripts; now you can just ask. Progress is when your prompt gets shorter.)
\nTemperature 0.9 matters — early answers should be careless commitments, that’s where the fun debt comes from. When the player calls the endgame, the model must pay:
\ndef crystallize(transcript):\n resp = openai.Completion.create(\n model=\"text-davinci-002\",\n prompt=transcript + \"\\nName one specific object consistent with \"\n \"every answer above. Object:\",\n max_tokens=12, temperature=0.7, stop=[\"\\n\"],\n )\n return resp.choices[0].text.strip()And then — crucially — the model does not get to grade its own exam. A separate judging pass, temperature 0, checks the named object against each recorded answer one Q/A pair at a time:
\ndef judge(obj, qa_pairs):\n verdicts = []\n for q, a in qa_pairs:\n resp = openai.Completion.create(\n model=\"text-davinci-002\",\n prompt=f'The object is: {obj}\\nQuestion: \"{q}\"\\n'\n f'Is the correct answer Yes or No? Answer:',\n max_tokens=2, temperature=0, stop=[\"\\n\"],\n )\n verdicts.append(resp.choices[0].text.strip() == a)\n return verdicts # any False = the human cornered itOne pair at a time, because asked to verify all twenty at once, the judge gets agreeable and waves contradictions through. Split into twenty tiny questions with a temperature of zero, it turns pleasingly merciless. Generation gets to be creative; judgment gets to be a checklist. Different jobs, different settings.
\nPlayed over two evenings with three friends and a CLI. Human record: 7 wins, 9 losses, 2 draws (the judge deadlocked on whether a candle is “furniture,” which, fair).
\nWhat cornering strategies emerged:
\nThe broken version and the working version differ by one thing: where the commitment lives. In classic twenty questions the secret exists at turn zero, in someone’s head. Against a model, there is no turn-zero anywhere for it to exist — there is only the transcript, and the “secret” is whatever the transcript’s accumulated pressure squeezes out at the end. You were never discovering its object. You were negotiating reality with it, one yes at a time.
\nThat’s not a flaw I patched. It’s the most honest demonstration I’ve found of what these systems are: not a mind holding answers, but a process that produces answers under constraint. Wheeler meant his game as a metaphor. I can now run the metaphor locally, at temperature 0.9, for about a cent a round.
\nTotal build: ~120 lines, one evening, no front end — the terminal is fine, the arguing happens in the room anyway. My DALL·E 2 invite is presumably in the same postal limbo as everyone else’s, or this post would have pictures of the topiary.
","excerpt":"I tried to build the most boring game possible — twenty questions, model thinks of a word, you guess it — and discovered it’s impossible. Not hard. Impossible, in a way that turned into a much better game. The bug…","timeToRead":5,"fields":{"slug":"/twenty-questions-against-a-player-with-no-secret/"},"frontmatter":{"title":"Twenty questions against a player with no secret","description":"GPT can't actually hold a hidden word in mind — so I built the version of twenty questions where that's not cheating, it's the rules. John Wheeler got here first.","path":"Twenty-questions-against-a-player-with-no-secret","date":"Jun 18, 2022","year":"2022"}},{"id":"0ec13999-ad81-5f85-ab2e-f8d06c8a10eb","html":"Most of my infrastructure changes are not engineering. They’re dictation: “add memory limits to the api pod,” “bump that image tag,” “open port 9090 for prometheus.” I know the change before I open the editor; the editor is just where I go to make typos in YAML.
\nLanguage models are extremely good at dictation. They are also confident liars. The whole design question of AI automation is how to get the first property without being destroyed by the second, and I think I’ve landed on an answer I actually trust in production:
\nThe model doesn’t get access to anything. It gets a pen. It writes pull requests. The PR must survive a gauntlet of mechanical validation before a human ever sees it, the human merges, and GitOps applies whatever lands in main — same as any other contributor. The model isn’t an operator. It’s a very fast junior colleague with zero credentials and excellent handwriting.
Five stages, one script, no daemon:
\nrequest ──► context ──► generate ──► gauntlet ──► pull request\n (text) (repo) (codex) (CI tools) (human)The entry point is deliberately dumb — a CLI:
\n./scribe \"add memory limits (512Mi) and cpu limits (500m) to the api deployment\"Models write better infrastructure when they can see the file they’re changing, so the prompt is mostly quotation. One thing I learned early and will fight about: ask for the whole new file, never a diff. Models produce plausible-looking patches with wrong line numbers and off-by-one context — a diff is a positional format and the model has no position, only text. Ask for the full file; compute the diff yourself with tools that can’t imagine things.
\nimport openai, pathlib, sys\n\ndef generate(request: str, path: str) -> str:\n current = pathlib.Path(path).read_text()\n prompt = f\"\"\"# Task: modify the Kubernetes manifest below.\n# Change requested: {request}\n# Rules: output the complete new file, valid YAML, no commentary.\n# Do not add, remove, or rename any resources unless asked.\n\n# Current file ({path}):\n{current}\n\n# Complete new file:\n\"\"\"\n resp = openai.Completion.create(\n model=\"code-davinci-002\", # codex beta: currently free, absurdly\n prompt=prompt,\n max_tokens=1500,\n temperature=0, # dictation, not poetry\n stop=[\"# Task:\"],\n )\n return resp.choices[0].text.strip()Temperature 0 because there is exactly one correct way to add a memory limit and I’d like it every time. Which file to edit is either given on the CLI or picked by a first, tiny completion that sees only git ls-files '*.yaml' — choosing from a real list is a task models are hard to break on, since every possible answer already exists.
Here’s the heart of the design. I never evaluate the model’s output by reading it — reading is how tired humans get fooled by confident text. The output must instead survive tools that are incapable of being charmed:
\n#!/bin/sh\nset -e # any failure kills the PR before it exists\n\nyamllint -d relaxed \"$FILE\" # 1. is it even YAML\nkubeconform -strict -summary \"$FILE\" # 2. is it valid *Kubernetes* —\n # schemas don't negotiate\nkubectl diff -f \"$FILE\" --server-side || true # 3. what would actually change\nkubectl apply -f \"$FILE\" --dry-run=server # 4. would the API server accept it\n\n# terraform changes get the equivalent treatment:\nterraform plan -detailed-exitcode -out=tf.plan # exit 2 = changes, 1 = broken\nterraform show tf.plan | grep -q \"destroy\" && exit 1 # the bot may not destroy. ever.Then the paranoid layer — a blast-radius allowlist enforced on the diff, not the intent:
\ngit diff --name-only main | grep -vE '^k8s/(deployments|services|ingress)/' \\\n && { echo \"touched files outside the sandbox\"; exit 1; }\ngit diff main | grep -qE 'kind:\\s*(Secret|PersistentVolume)' \\\n && { echo \"nice try\"; exit 1; }The model can want whatever it likes. The gauntlet only cares what the bytes do. Every check here existed before language models and will outlive them — that’s the point. Generation is allowed to be creative; validation is a compiler. Never staff both jobs with the same kind of mind.
\nWhen a check fails, the error message goes back into the prompt for exactly one retry:
\nretry_prompt = prompt + generated + f\"\"\"\n\n# The file above failed validation with this error:\n# {stderr}\n# Output the corrected complete file:\n\"\"\"This fixes a surprising share of failures — schema validators write great error messages, and the model reads them better than I do at 9am. But it’s one round, hard-coded. An unbounded fix-it-yourself loop is a self-licking ice cream cone that burns tokens converging on nothing; if two attempts can’t pass a linter, the request was ambiguous and a human should hear about it. Loops need leashes.
\nSurvivors become pull requests, and the PR body carries the receipts:
\ngit checkout -b \"scribe/${SLUG}\"\ngit commit -am \"scribe: ${REQUEST}\" --author=\"Scribe <scribe@ohsnail.com>\"\ngh pr create \\\n --title \"🖋 ${REQUEST}\" \\\n --body \"$(cat <<EOF\n**Request:** ${REQUEST}\n**Server dry-run diff:**\n\\`\\`\\`\n$(kubectl diff -f \"$FILE\" --server-side)\n\\`\\`\\`\n**Gauntlet:** yamllint ✓ kubeconform ✓ dry-run ✓ blast-radius ✓\nEOF\n)\"Review takes seconds, because I’m not reviewing prose or trust — I’m reviewing a machine-verified diff with the evidence attached. Merge, and the GitOps controller ships it like any other commit. The model never held a credential; the merge button is the last mile of intent, and it stays human.
\n38 requests since March. 26 merged untouched. 7 merged after small edits (it loves adding labels nobody asked for — the junior-colleague energy is uncanny). 3 correctly killed by the gauntlet — the best one invented apiVersion: apps/v2, a version that does not exist, delivered with total confidence and executed by kubeconform in four milliseconds. 2 died in ambiguity, as they should have: my request was the bug.
Time per dictation-class change: was ~10 minutes of editor and kubectl archaeology, now ~40 seconds of reading a diff. But the number I care about is different: zero incidents caused, zero possible by construction — the failure domain is “a bad PR gets opened,” and a bad PR is litter, not fire.
\nEveryone building on these models is choosing a point on a line between “autocomplete” and “autonomy,” and most of the debate treats capability as the axis that matters. I don’t think it is. The axis that matters is what artifact the model produces. Give it a terminal and its mistakes are events. Give it a pen and its mistakes are documents — diffable, lintable, rejectable, versioned, reviewed. Pull requests turn out to be a nearly perfect prosthetic for a brilliant liar: all of the leverage, none of the blast radius, and an audit trail as a side effect.
\nDon’t give the model hands. Give it a pen, and make the paper do the vetting.
","excerpt":"Most of my infrastructure changes are not engineering. They’re dictation: “add memory limits to the api pod,” “bump that image tag,” “open port 9090 for prometheus.” I know the change before I open the editor; the editor…","timeToRead":5,"fields":{"slug":"/my-infrastructure-accepts-pull-requests-from-a-language-model-now/"},"frontmatter":{"title":"My infrastructure accepts pull requests from a language model now","description":"Don't give the model hands — give it a pen. A pipeline where GPT writes Kubernetes and Terraform changes as PRs that must survive a mechanical gauntlet before a human ever sees them","path":"My-nfrastructure-accepts-pull-requests-from-a-language-model-now","date":"Jun 06, 2022","year":"2022"}},{"id":"44d891da-8a97-5ffb-ac57-c9414669d64d","html":"Between the cheese course and the board games this Christmas, I made my family play against a language model, and I’d like to report the results as if they were science.
\nThe game is embarrassingly simple. Everyone gets a question on their phone — “what’s the worst gift you ever received?”, “describe your morning in four words”, “invent a rule for this dinner”. Everyone types an answer. The server quietly adds one more answer, written by GPT-3. All answers appear shuffled on the TV, and everyone votes for which one is the bot. Bot survives → bot scores. Bot caught → humans score.
\nI called it Odd Bot Out. One evening of Flask, a phone-friendly page, a davinci completion call, and a shared screen. Now that the API waitlist is gone, anyone can run this at their own dinner table for pennies — a full evening of rounds cost me about forty cents.
My first prompt was the obvious one — “Answer the following question like a human player in a party game” — and the bot was caught eight rounds out of eight. Unanimously. My mother, who calls WhatsApp “the WhatsApp,” identified the machine in under three seconds.
\nLooking at the transcripts, the tells were brutal:
\nSo the fix wasn’t making the model smarter. It was making it lazier. The winning prompt contains no instructions at all — just a fake transcript of the game itself: a few questions, each followed by short, lowercase, slightly disappointing “player answers” I wrote by hand. The whole bot is this:
\nimport openai\n\nFAKE_TRANSCRIPT = \"\"\"Q: what's your secret talent?\nA: parallel parking. thats it\nQ: describe your job badly\nA: i move numbers between boxes\nQ: whats always in your fridge?\nA: mustard and regret\nQ: best thing about winter?\nA: it ends\n\"\"\"\n\ndef degrade(text):\n \"\"\"Essays and punctuation are how it gets caught.\"\"\"\n text = text.strip().lower()\n return text[:-1] if text.endswith(\".\") else text\n\ndef ask_bot(question):\n resp = openai.Completion.create(\n engine=\"davinci\",\n prompt=FAKE_TRANSCRIPT + f\"Q: {question}\\nA:\",\n max_tokens=16, # physically cannot write an essay\n temperature=0.9, # some chaos, it's a party\n stop=[\"\\n\", \"Q:\"], # one answer, then shut up\n )\n return degrade(resp.choices[0].text)Note what’s not here: no instructions, no “pretend to be human,” no persona. Plain davinci is a continuation engine, so the fake transcript does everything — it establishes the format, the register, and the level of effort, and the model simply continues the document it finds itself in. Every one of my hand-written fake answers is doing more work than a billion parameters.
\nI drew one ethical line for myself: degrade() deletes polish, but no injecting fake typos. If it survives, it should survive by register, not by costume.
Result across the rest of the holidays, 23 rounds with five to seven players: the bot survived 10 — and twice a human got the most bot-votes instead. My uncle was declared a machine for answering too cleverly. He has not recovered.
\nHere’s what the game taught me, and I think it generalizes beyond my living room.
\nWe casually treat “passes for human” as a summit — as if text becomes human-like at the top of some quality scale. The party game says the opposite: the bot got caught whenever it was too good, and survived by being unremarkable. Fluency, structure, correctness — those read as machine. Laziness, fragments, mild rudeness — those read as person. In a room of casual humans, the Turing test is not a test of intelligence. It’s a test of calibrated mediocrity, and calibrating mediocrity is a real engineering problem: my prompt’s fake lazy answers were doing more work than the 175 billion parameters.
\nThe tell the model never beat, though, was placelessness. I could match the register of a tired human, but not the address of one. Human answers had a stake in that specific room on that specific evening; the model’s best efforts were from nowhere, for anyone. The final round’s question was “invent a rule for this dinner,” and the winning human answer referenced the gravy incident from an hour earlier. No amount of temperature gets you the gravy incident. Context isn’t flavor — it’s identity, and the model’s only context was whatever I smuggled into the prompt.
\nWhich suggests the uncomfortable corollary: give the machine the gravy incident — feed the room’s chatter into the prompt — and that last tell starts to dissolve. I tested a mild version, seeding tonight’s earlier answers into the context, and the bot immediately got better at blending. I stopped there, partly because it felt like enough, partly because it was genuinely a little eerie, and partly because it was time for dessert.
\nThe whole server is small enough to read over coffee. One route collects answers, one builds the board by sneaking the bot in and shuffling, one counts votes:
\nimport random\nfrom flask import Flask, request, jsonify\n\napp = Flask(__name__)\ngame = {\"question\": \"\", \"answers\": {}, \"board\": [], \"votes\": {}, \"scores\": {}}\n\n@app.route(\"/ask\", methods=[\"POST\"]) # host sets the question\ndef ask():\n game.update(question=request.form[\"q\"], answers={}, votes={})\n return \"\", 204\n\n@app.route(\"/answer\", methods=[\"POST\"]) # players submit from phones\ndef answer():\n game[\"answers\"][request.form[\"player\"]] = request.form[\"text\"]\n return \"\", 204\n\n@app.route(\"/board\") # the TV polls this\ndef board():\n if not game[\"board\"]:\n entries = list(game[\"answers\"].items())\n entries.append((\"__bot__\", ask_bot(game[\"question\"])))\n random.shuffle(entries)\n game[\"board\"] = entries\n return jsonify([text for _, text in game[\"board\"]])\n\n@app.route(\"/vote\", methods=[\"POST\"]) # index of the suspected bot\ndef vote():\n game[\"votes\"][request.form[\"player\"]] = int(request.form[\"idx\"])\n return \"\", 204\n\n@app.route(\"/reveal\")\ndef reveal():\n bot_idx = next(i for i, (who, _) in enumerate(game[\"board\"]) if who == \"__bot__\")\n for player, idx in game[\"votes\"].items():\n game[\"scores\"][player] = game[\"scores\"].get(player, 0) + (idx == bot_idx)\n caught = list(game[\"votes\"].values()).count(bot_idx) > len(game[\"votes\"]) / 2\n game[\"board\"] = []\n return jsonify(bot=bot_idx, caught=caught, scores=game[\"scores\"])A global dict as a database, no auth, race conditions if your family votes aggressively — it’s a living-room toy and proud of it. The front end is one HTML page polling /board; the “TV” is a laptop and an HDMI cable. Rotate who writes the question.
Two warnings from the field: cap the bot at sixteen tokens or it will monologue, and prepare for the real endgame — after round three, the humans start impersonating the bot on purpose, and the game quietly turns into everyone performing their idea of a machine performing a person. That meta-round is the best round. I have notes about what that means, but they can wait for another post.
\nFinal score for the holidays: humans 13, machine 10, uncle still legally a robot.
","excerpt":"Between the cheese course and the board games this Christmas, I made my family play against a language model, and I’d like to report the results as if they were science. The game is embarrassingly simple. Everyone gets a…","timeToRead":5,"fields":{"slug":"/odd-bot-out-a-party-game-where-gpt-3-hides-among-your-relatives/"},"frontmatter":{"title":"Odd Bot Out: a party game where GPT-3 hides among your relatives","description":"I built a living-room game over the holidays — everyone answers a question, one answer is a language model, find it. Tuning the bot to survive taught me what 'sounding human' actually means","path":"Odd-Bot-Out-a-party-game-where-GPT-3-hides-among-your-relatives","date":"Dec 29, 2021","year":"2021"}},{"id":"a7c7b180-3bdf-5e61-8a69-8a599112637a","html":"This blog has been two blogs wearing a trenchcoat: infrastructure posts and GPT posts, politely ignoring each other. This summer they finally met, because I got paged at 3:12am, spent forty minutes spelunking through Loki, and discovered the root cause was — again — the same Redis pod OOMing. Forty minutes, for a diagnosis that fits in a sentence.
\nA sentence, you say. We have a machine for sentences now.
\nThe April post already litigated the tempting version — the closed loop where the model fixes things itself — and the verdict stands: not unattended, not on a box I like. A system that invents flags does not get kubectl.
But there’s a humbler job that’s still most of the pain: translation. When an alert fires, the on-call intern doesn’t fix anything. The intern gathers context, forms a hypothesis, and hands the human a briefing. So:
\nAlertmanager webhook → context gatherer → davinci → Slack message.
\nThe Slack message has three parts: a one-sentence hypothesis, the evidence, and a suggested first command — the kubectl describe or LogQL query I’d probably start with anyway. It proposes; I dispose. The plan is still the product.
The hard engineering isn’t the model call — it’s that an incident produces megabytes of context and the keyhole is still 2048 tokens. Commit diffs were easy; logs are a firehose of repetition. So most of the code is what I’ve started calling log distillation:
\ngit log entries touching those manifests, and a Loki query for the affected pod.[x347] connection refused ... — log spam compresses beautifully, because spam is the point of spam.The prompt is a crime-scene summary, not the crime scene. Deciding what makes the summary turned out to be 90% of the project, and honestly, building the distiller taught me more about my own logs than the model did.
\nNow the part I’m actually proud of, because the obvious failure mode here is dangerous: a fluent, confident, wrong briefing at 3am is worse than no briefing — I’ll trust it precisely because I’m too tired not to.
\nThe fix exploits something nice: the model may invent explanations, but the logs are right there. The prompt requires the hypothesis to cite two verbatim log lines as evidence, and then the script — dumb, deterministic Python — checks that those exact lines exist in the real distilled logs before posting anything.
\nfor quote in extract_quotes(completion):\n if quote.strip() not in distilled_logs:\n return post_slack(\"⚠️ no confident hypothesis (evidence check failed)\")If the model quotes a log line that doesn’t exist, it invented its evidence, and its conclusion is presumptively garbage — discarded, no briefing, just an honest shrug. It’s terraform plan for prose: beliefs don’t ship until they’ve met evidence, and crucially, the verifier isn’t the model. Generation is allowed to be creative; verification is grep. Never ask the suspect to also be the jury.
47 alerts since mid-July (a couple were self-inflicted, see: quarterly destroy ritual). Grading each briefing after the fact:
\nBest moment: a cascade of timeout alerts where the intern’s briefing led with a deploy from six hours earlier — it had noticed the git timestamp in the context block sat just before the first error, a correlation I’d missed because six hours “felt” too far away. Statistical models don’t have feelings about time. Point, intern.
\nWorst moment: it diagnosed a disk-pressure alert as a Redis memory issue because two of my three few-shot examples were Redis incidents. My examples had taught it that everything is Redis. Few-shot bias is real: the intern is a mirror, and I’d shown it a very Redis-shaped mirror.
\nCost: about $0.14 of davinci per incident. Cheapest coworker I’ve ever had, and it never asks to reschedule the retro.
\nOpenAI shipped a fine-tuning API this summer, so the rematch question is live again — and I declined, by my own rulebook. Fine-tuning owns context that never fits through the keyhole, stable distributions like two years of my commit voice. But incidents are the opposite: each one’s crucial context is fresh — today’s logs, this morning’s deploy — and worthless tomorrow. There’s no stable distribution to bake into weights; there’s only the keyhole, refilled per incident. Right tool, and for once I picked it by theory instead of by vibes.
\nGitHub spent the summer previewing Copilot — an autocomplete with hands is real now, it lives in editors, and yes, there’s a waitlist, because in this field even the future queues. Meanwhile my little intern suggests, cites, and gets fact-checked by twelve lines of Python.
\nI’ve stopped thinking the interesting question is “what can the model do.” The interesting question is what shape of harness makes a confident liar useful — distill what goes in, verify what comes out, keep the hands human. That harness cost me a summer of evenings and it’s mostly string handling. Somebody’s going to get very rich selling harnesses.
\nNext: GitOps, for real this time. The robots that apply commits have waited three posts, and now they have a coworker who reads logs.
","excerpt":"This blog has been two blogs wearing a trenchcoat: infrastructure posts and GPT posts, politely ignoring each other. This summer they finally met, because I got paged at 3:12am, spent forty minutes spelunking through…","timeToRead":5,"fields":{"slug":"/i-hired-gpt-3-as-my-on-call-intern/"},"frontmatter":{"title":"I hired GPT-3 as my on-call intern","description":"Wiring davinci between Alertmanager and Slack — six weeks of a language model reading my logs at 3am, with a built-in lie detector","path":"I-hired-GPT-3-as-my-on-call-intern","date":"Sep 19, 2021","year":"2021"}},{"id":"0448c0f6-973c-5887-9842-aa32b5e76292","html":"Ten months after joining the waitlist, the invite came. So this is the post the June write-up promised: the world’s least rigorous benchmark, rerun. In one corner, my GPT-2 small, 124M parameters, lovingly fine-tuned on 3,124 of my own commit messages, weights shaped by my personal git guilt. In the other corner, GPT-3 davinci: 175 billion frozen parameters that have never heard of me, allowed to see exactly five examples through the 2048-token keyhole of its prompt.
\nThe question from last June, now testable: is fine-tuning dead for tasks?
\nTwenty real staged diffs, pulled from actual work over two weeks. Both models get each diff and must produce a commit message. GPT-2 runs locally, as a pod, for free. GPT-3 gets this:
\nimport openai\n\nFEWSHOT = open(\"five_examples.txt\").read() # 5 diff→message pairs, ~1400 tokens\n\nresp = openai.Completion.create(\n engine=\"davinci\",\n prompt=FEWSHOT + \"\\n===\\ndiff:\\n\" + truncate(diff, 400) + \"\\nmessage:\",\n max_tokens=24,\n temperature=0.7,\n stop=[\"\\n\", \"===\"]\n)Notes from the keyhole: five demonstration pairs plus a truncated diff eats nearly the whole 2048-token context, so “which five examples” and “which 400 tokens of the diff” become the actual engineering. There’s no fine-tuning endpoint for GPT-3 — prompting is the interface — which makes this a genuinely fair fight between the two paradigms. Scoring: a shell script shuffled the outputs so I graded blind, picking the message I’d actually keep.
\nNot close. Davinci 16, my model 4.
\nThe gap is exactly where the paper said it would be: GPT-2 has never seen a diff and is riffing on vibes, while davinci actually reads the thing. Given a diff adding a Redis import and a cache decorator, my model offered “fix readme typo (again)“. Davinci wrote “add redis caching to search results”. That’s not a style difference. One of them did the assignment.
\nThe part that stung more: I assumed the giant would sound like a press release and my model would at least win on voice. Wrong. From five lowercase, slightly self-deprecating examples, davinci picked up my register almost perfectly — lowercase, terse, the occasional parenthetical apology. Style, it turns out, is shallow: it transfers through a keyhole. I’d been treating “sounds like me” as evidence of something deep in my fine-tuned weights, and a frozen model just cold-read it off five samples like a street magician.
\nLast June I bet “prompting eats the tasks, fine-tuning keeps the distributions.” Half survives, and it’s the narrower half.
\nThe four rounds my model won were all the same kind: diffs whose real story wasn’t in the diff. A one-line version bump whose honest message was “update deps and pray” — because that dependency has burned me twice, a fact living in my history, not in the patch. Davinci, reading only the keyhole, wrote the technically-correct-but-alien “bump lodash to 4.17.21”. My model has eaten two years of my scar tissue; it knows prayer is warranted.
\nSo the revised, humbler thesis: fine-tuning doesn’t own style, it owns context that never fits through the keyhole. Prompting rents a mind and slides your world through a 2048-token slot. The slot will widen — everything in this field widens — but there will always be some residue of private history on the wrong side of it. That residue is what your own weights are for. Everything else, the giant does better, today, for money.
\nAnd it is money, which is the quietly radical part. Each suggestion burns ~1,900 davinci tokens; at 6 cents per thousand that’s about 11 cents per commit message. My GPT-2 pod costs literally nothing per thought.
\nSit with how strange that is: for the first time, a function call in my code has a marginal cost per thought. Not per gigabyte, per hour, per request — per act of cognition, metered like electricity. My git hook now has a unit economics section. I caught myself batching commits to save dimes, which means the pricing model is already reshaping my behavior, which is how you know it’s real infrastructure and not a demo. Utility computing finally happened, and the utility is thinking.
\n(Practical corollary: the fine-tuned pet earns a second life as the cheap tier. Local model drafts, giant model only when the diff is gnarly. Cascading by cost is going to be a whole design pattern, I suspect.)
\nOne more thing, filed under “probably terrible.” Everything I’ve built so far is a straight line: diff in, text out, human decides. But the completion API doesn’t care where prompts come from — nothing stops a script from feeding the model’s output back in as the next input and letting it converse with itself. And my k3s manifests are just text in git, and davinci demonstrably reads structured text… you can see the loop closing: model proposes a change, script applies it, reality’s error message goes back through the keyhole, repeat.
\nAn autocomplete with hands. I gave it maybe an hour of daydreaming before remembering that this is a system that confidently invents flags — for rm, presumably — and that my Terraform post has a whole section about beliefs meeting reality under human review. The plan is the product; the loop deletes the plan. So no. Not with this model, not unattended, not on a box I like.
But I’m writing the idea down, because the objection is about this model, and models have a way of getting better while opinions stay still. Ask me again after the next paper.
\nThe GitOps rabbit hole is still scheduled, I promise — the robots that apply git commits deserve their own post, especially now that I’ve spent a paragraph fantasizing about a robot that writes them.
","excerpt":"Ten months after joining the waitlist, the invite came. So this is the post the June write-up promised: the world’s least rigorous benchmark, rerun. In one corner, my GPT-2 small, 124M parameters, lovingly fine-tuned on…","timeToRead":4,"fields":{"slug":"/the-rematch-my-fine-tuned-gpt-2-vs-a-frozen-giant-with-five-examples/"},"frontmatter":{"title":"The rematch: my fine-tuned GPT-2 vs. a frozen giant with five examples","description":"GPT-3 API access finally arrived, so the commit-message benchmark of exactly one silly task gets its rerun","path":"The-rematch-my-fine-tuned-GPT-2-vs-a-frozen-giant-with-five-examples","date":"Apr 11, 2021","year":"2021"}},{"id":"5f74840e-0fa5-58a1-ae4d-e1097ef649d2","html":"Since the terraform destroy post, the most common question in my inbox is some version of: “cute, but will your €5 setup scale?”
\nI think that’s the wrong question, and this guide is about the right one. Because here’s the dirty secret of every “scalable production architecture” diagram: you don’t get taken down by load. You get taken down by the rewrite you scheduled for the week the load arrived.
\nSo the metric I design for isn’t requests per second. It’s minutes to the next rung — how long it takes to climb one step up when I actually need to. Scalability isn’t a building you construct upfront. It’s a series of doors you deliberately leave unlocked.
\nHere’s the ladder, rung by rung, with each unlocked door pointed out.
\nRecap for new readers: Terraform builds a Hetzner VPS and DNS, cloud-init installs k3s, and the cluster pulls its manifests from git. The entire company — population: me — fits in one repo, and the whole environment rebuilds from zero in under five minutes, which I know because I destroy it on purpose quarterly.
\nThat ritual matters for this post: a rehearsed rebuild is also a rehearsed migration. Every rung below is cheap precisely because reconstruction is a habit, not an emergency.
\nNothing else on this ladder works unless the application itself is stateless: config from environment variables, sessions in a store rather than in memory, nothing precious on local disk, logs to stdout. Old news — the 12-factor checklist is a decade old — but it’s the rung people skip and then blame Kubernetes.
\nThe test is brutal and simple: kubectl scale deployment app --replicas=3. If anything breaks — sessions vanish, uploads disappear, a cron job runs three times — you found state hiding in your app. Evict it now, while it’s small.
Door unlocked: horizontal scaling becomes a number in a YAML file instead of a project.
\nThe Terraform post’s lesson keeps paying rent: after a destroy/rebuild, everything that came back automatically was never really state — it was opinion in git. Scaling is the same audit. Manifests, certs, DNS, the cluster itself: opinions. What’s left — the database and user uploads — is the actual state, and the strategy is to keep that pile as small and as boring as possible.
\nConcretely:
\nDoor unlocked: the only hard migration in your future is one small, well-mapped pile instead of “the server.”
\nThe reason I picked k3s over docker-compose finally cashes out here. Adding a worker node is:
\n# on the new €5 box (which Terraform created with count = 2)\ncurl -sfL https://get.k3s.io | K3S_URL=https://snail-01:6443 K3S_TOKEN=$TOKEN sh -That’s it — the node joins, the scheduler starts using it. And when the control plane itself needs to survive a node death, recent k3s can run an embedded etcd cluster across three servers instead of single-node SQLite. Same binary, same manifests, zero application changes.
\nThis is the rung that justifies the YAML tax from last summer: compose scales a machine; an orchestrator scales a fleet, and the manifests can’t tell the difference.
\nDoor unlocked: capacity becomes count = N in Terraform plus one join command.
Right now DNS points at the node. One Terraform resource swaps that for a Hetzner load balancer pointing at all nodes, health checks included. Because rung 1 made the app stateless and rung 3 made nodes interchangeable, this is genuinely just a routing change — the LB is boring because the lower rungs did their jobs.
\nDoor unlocked: nodes become anonymous. The uptime alert stops being about any particular machine.
\nYou can’t climb a ladder you can’t see. Prometheus + Grafana for metrics, Loki for logs — the kube-prometheus-stack Helm chart makes this an afternoon, and on a small cluster it costs more RAM than it has any right to, and it’s still worth it.
\nHere’s the framing that made me stop treating monitoring as a chore: capacity plans are beliefs, and metrics are how beliefs meet evidence. Same epistemology as terraform plan, same failure mode as a language model inventing APIs — a system confidently describing a reality it’s never checked. “We’ll scale when we need to” is a hallucination until a dashboard says when need is. My rule: I climb a rung when a graph tells me to, never when anxiety does.
Door unlocked: scaling decisions become diffs against data instead of vibes.
\nNo multi-region, no service mesh, no autoscaling, no microservices. Not because they’re bad — because each is a building, not a door, and buildings are premature until a graph screams. The most scalable decision I’ve made all year is the stuff I declined to install. Premature scaling is the root of all cloud bills.
\nAnd the honest coda: I’m on rung 0 and expect to stay here. The ladder’s real product was never capacity — it’s that every rung is rehearsed, so growth is a checklist and not a crisis. I sleep well next to an unlocked door.
\nThe itch from October remains: git already holds the manifests, so why am I the one running kubectl apply? The GitOps rabbit hole is officially scheduled.
Also, a certain API invite finally showed up in my inbox this week. The silly benchmark gets its rematch. Stay tuned.
","excerpt":"Since the terraform destroy post, the most common question in my inbox is some version of: “cute, but will your €5 setup scale?” I think that’s the wrong question, and this guide is about the right one. Because here’s…","timeToRead":5,"fields":{"slug":"/the-scaling-ladder-a-production-setup-that-scales-by-not-scaling-yet/"},"frontmatter":{"title":"The scaling ladder: a production setup that scales by not scaling yet","description":"A guide to building small production systems where every upgrade is a swap, not a rewrite","path":"The-scaling-ladder-a-production-setup-that-scales-by-not-scaling-yet","date":"Feb 14, 2021","year":"2021"}},{"id":"f89c2c1c-50ba-58e8-8801-7a6acd7d5b20","html":"In the k3s post I claimed Terraform is built on the same declarative idea as Kubernetes. This post is me paying for that sentence, because after a month of actually using Terraform I think it’s only half true — and the half that’s false is the interesting part.
\nBut first, the fun part, where I delete my server.
\nThe Hetzner box running everything — this blog, the side projects, the commit-message bot — was hand-built. SSH in, apt this, curl that, tweak, forget. If it died, “restore from backup” really meant “restore from whatever I remember about April.”
\nSo: everything into Terraform 0.13. The stack is three providers doing a small, honest job:
\nterraform {\n required_providers {\n hcloud = { source = \"hetznercloud/hcloud\" }\n cloudflare = { source = \"cloudflare/cloudflare\" }\n }\n}\n\nresource \"hcloud_server\" \"main\" {\n name = \"snail-01\"\n server_type = \"cx21\"\n image = \"ubuntu-20.04\"\n location = \"nbg1\"\n user_data = file(\"cloud-init.yaml\")\n}\n\nresource \"cloudflare_record\" \"blog\" {\n zone_id = var.zone_id\n name = \"ohsnail.com\"\n value = hcloud_server.main.ipv4_address\n type = \"A\"\n}The cloud-init.yaml is where the magic compounds: it installs k3s and applies my manifests from git. So Terraform builds the world, cloud-init boots the cluster, and the control loops from last post take it from there. Three layers of “make it so.”
(The new required_providers block with source addresses is a 0.13 thing, by the way — third-party providers like hcloud finally live in the registry instead of “download this binary and put it in a folder like it’s 2003.“)
Adopting the existing server meant a tour through terraform import, which I’d describe as filling out immigration paperwork for infrastructure that already lives here. Tedious, worth it, do it resource by resource and run plan obsessively until it shows no changes.
Here’s the thing though: a disaster recovery plan you’ve never executed is a religious belief. So on Friday night, with a beer for courage and a fresh state backup for cowardice:
\nterraform destroy -auto-approve\n# ...12 seconds of watching my entire digital life deallocate\nterraform apply -auto-approveFour minutes and twenty seconds later, everything was back. Server, DNS, cluster, blog, TLS certs re-issued by cert-manager, the bot cheerfully suggesting I “fix flaky test.” The only data that needed restoring was two Docker volumes from a Restic backup — which is exactly the point: the ritual shows you, brutally, what is actually state versus what just looks important. Everything that came back automatically was never state at all. It was opinion, and opinions belong in git.
\nMy server is no longer a pet. It’s cattle with a herd size of one, and I can now hold the entire production environment in a git clone.
I’m making this a quarterly ritual. Nothing keeps infrastructure honest like scheduled death.
\nNow the confession. Kubernetes and Terraform are both “declarative,” but they reconcile in fundamentally different ways, and conflating them (as I did) hides the most important operational fact about Terraform:
\nKubernetes reconciles continuously. Control loops compare desired vs. actual many times a minute, forever. Drift gets corrected while you sleep.
\nTerraform reconciles episodically. Nothing happens until a human runs plan. Between runs, Terraform isn’t watching anything — it’s not a loop, it’s a snapshot ceremony.
Which leads to the mental model that actually made Terraform click for me: the state file is not a record of your infrastructure. It’s Terraform’s beliefs about your infrastructure. Reality can change behind its back — a colleague clicks something in the console, a provider auto-upgrades a field, you “quickly fix” a firewall rule at 2am — and the beliefs go stale. terraform plan is the moment beliefs are confronted with evidence, and the diff you’re reviewing is really three-way: what you declared, what Terraform believes, what’s actually out there.
Sound familiar? A confident internal model drifting away from reality, generating authoritative statements about things that no longer exist — that’s the same failure mode I spent the spring documenting in GPT-2, which happily invented APIs because nothing ever forced its beliefs to meet the world. Terraform ships the antidote as a first-class command. Half of engineering, I’m starting to suspect, is just deciding when your system’s beliefs get checked against reality: every few seconds (Kubernetes), every deliberate ceremony (Terraform), or never (my old snowflake server, and, well, GPT-2).
\nContinuous reconciliation is strictly stronger if you can afford it. Terraform’s episodic model is the tradeoff you accept for managing things that are expensive, slow, or dangerous to change — you want a human reading the diff before a database gets replaced. The plan isn’t bureaucracy. The plan is the product.
\ndestroy will absolutely also destroy things you forgot depend on each other. Read the destroy plan slower than you’ve ever read anything.prevent_destroy and treat them as pets with paperwork.The obvious itch: Terraform applies when I run it, but the manifests inside the cluster already live in git… so why am I the one running anything? The word for that itch is apparently “GitOps,” and it’s next quarter’s rabbit hole.
\nAlso my GPT-3 invite is still not here. OpenAI, I have a benchmark of exactly one silly task waiting.
","excerpt":"In the k3s post I claimed Terraform is built on the same declarative idea as Kubernetes. This post is me paying for that sentence, because after a month of actually using Terraform I think it’s only half true — and the…","timeToRead":4,"fields":{"slug":"/i-ran-terraform-destroy-on-my-own-production-on-purpose/"},"frontmatter":{"title":"I ran terraform destroy on my own production (on purpose)","description":"Putting the whole stack in Terraform, then proving it by deleting the server — plus the difference between Terraform and Kubernetes everyone glosses over","path":"I-ran-terraform-destroy-on-my-own-production","date":"Oct 18, 2020","year":"2020"}},{"id":"3c4b6f49-35c9-5934-8419-3cd4bf9126d3","html":"Last November I wrote a guide on configuring Docker Swarm. Two weeks later, Mirantis bought Docker’s enterprise business and gave Swarm what sounded like a two-year countdown. They’ve since softened that to “we’ll keep supporting it,” but the writing is on the wall, and the wall is written in YAML.
\nSo this summer I finally did the thing I’d been avoiding since 2017: I learned Kubernetes properly. The trick that made it not miserable is that I didn’t install Kubernetes. I installed k3s.
\nRancher’s k3s is a certified Kubernetes distribution compressed into a single ~50MB binary. It swaps etcd for SQLite, strips the legacy cloud-provider baggage, and ships batteries included (Traefik as ingress, out of the box). Install is genuinely this:
\ncurl -sfL https://get.k3s.io | sh -\nsudo k3s kubectl get nodeThirty seconds later there’s a working single-node “cluster” on my €5 Hetzner VPS, idling at around 500MB of RAM. That’s the entire moat between “Kubernetes is for companies with a platform team” and “Kubernetes runs my hobby junk” — and it turns out the moat was just packaging.
\nMy Swarm stack was this blog, a couple of side projects, and — because the theme of this year apparently insists — the GPT-2 commit-message server from April, still faithfully dealing me chores every morning.
\nkompose will mechanically convert a compose file into manifests, but I recommend writing them by hand at least once, because the pain is educational. A Swarm service becomes three objects: a Deployment (what to run), a Service (how to reach it), an Ingress (how the outside reaches the Service). Add cert-manager and Let’s Encrypt certificates become just another object the cluster keeps renewed for me — the first moment where the K8s way felt better rather than merely fancier.
Lessons collected the honest way:
\nresources.limits.memory: 900Mi and we’re friends again.Here’s the mental model I wish someone had led with, instead of diagrams of pods.
\nSwarm — and honestly most of the tools I grew up on — is imperative: you issue commands, it executes them, done. docker service scale web=3 is an action.
Kubernetes is not an orchestrator that runs your commands. It’s a database of desired state with robots attached. You write down a fact you’d like to be true — “three replicas of this exist, reachable at this name, with valid TLS” — and control loops compare that desire to reality, forever, and push reality toward it. Node dies at 3am? Reality drifted from the spec; the loop corrects it. Nobody, including you, issued a command.
\nYou don’t tell Kubernetes what to do. You tell it what should be true.
\nAnd I can’t stop noticing that this is the same shape as my complaint about language models this spring — that GPT-2 executes text-continuation without any notion of your intent, and that the missing piece is a system trained to pursue what you meant. Kubernetes is what it looks like when infrastructure makes that leap: from “execute my steps” to “converge on my intent.” Declarative beats imperative for the same reason in both worlds — steps go stale, intent doesn’t. Apparently 2020 is the year I learn this lesson once per quarter in a different domain.
\nNo. Let’s be adults: for one VPS of hobby projects, docker-compose plus a systemd unit is fewer moving parts and I’d defend that choice in court. I did it because Swarm’s future is cloudy, because the industry has voted and K8s won, and because the reconciliation-loop model is the actually transferable knowledge — it’s the same idea underneath Terraform, GitOps, and every “controller” you’ll meet for the next decade.
\nBut I’ll admit the real reason: there is something deeply pleasing about a server that repairs itself while I sleep. The commit-message bot went down twice this month. Both times, by the time the uptime alert reached my phone, the pod had already been rescheduled and the alert resolved itself.
\nThe robot assigns me chores, and other robots keep the robot alive. The system works.
","excerpt":"Last November I wrote a guide on configuring Docker Swarm. Two weeks later, Mirantis bought Docker’s enterprise business and gave Swarm what sounded like a two-year countdown. They’ve since softened that to “we’ll keep…","timeToRead":3,"fields":{"slug":"/so-about-that-docker-swarm-post-i-moved-to-k3s/"},"frontmatter":{"title":"So about that Docker Swarm post: I moved to k3s","description":"Migrating my little stack from Swarm to lightweight Kubernetes, and the mental model that finally made K8s click","path":"So-about-that-Docker-Swarm-post-I-moved-to-k3s","date":"Aug 15, 2020","year":"2020"}},{"id":"a522c037-d315-5469-9b3d-f75734539a0a","html":"Three weeks ago OpenAI dropped the GPT-3 paper — “Language Models are Few-Shot Learners”, 31 authors, 75 pages — and then last week announced an API for it. Private beta, waitlist. I’m on the waitlist. This post is me processing the paper while I refresh my inbox.
\nAnd I have to process it, because if the paper’s claims hold up, the two posts I wrote this spring about fine-tuning GPT-2 describe a workflow with an expiration date.
\nGPT-3 is architecturally GPT-2 with the dials turned up. Same decoder-only transformer, more of everything: 175 billion parameters across 96 layers, trained on roughly 300 billion tokens of filtered Common Crawl, WebText, books, and Wikipedia. Context window doubled from 1024 to 2048 tokens.
\nFor scale: the model I fine-tuned on my commit messages was GPT-2 small, 124 million parameters. GPT-3 is over 1,400 of those. The training compute estimates floating around are in the several-million-dollar range, and the weights in half precision would be ~350GB — this thing wouldn’t fit on my SSD, let alone my GPU. Which explains why there’s an API and no download link, and why “Open”AI jokes are back on the menu.
\nNone of that is the interesting part. If “bigger model, better perplexity” were the whole story, this would be a shrug. The Kaplan et al. scaling laws paper from January already showed loss falling smoothly and predictably with model size, data, and compute — GPT-3 is in some sense just OpenAI running that curve way past where anyone else was willing to spend. The interesting part is what showed up along the way.
\nHere’s the thing the paper is really about, and it’s worth being precise because it’s genuinely strange.
\nFine-tuning, the thing I did in March, means updating the model’s weights with gradient descent on your data. The model changes. That’s why my 3,124 commit messages got memorized — with a dataset that small, the cheapest way for the network to reduce loss is to just store the training set. I watched that happen in real time on a free Colab GPU.
\nIn-context learning, the GPT-3 pitch, involves no weight updates at all. You put a few examples in the prompt:
\nEnglish: cheese French: fromage\nEnglish: bread French: pain\nEnglish: snail French:…and the model completes the pattern. The weights are frozen. Nothing is “trained.” And yet on task after task in the paper, performance climbs as you go from zero examples to one to a few dozen — as if the model is learning the task at inference time. The bigger the model, the steeper that climb: the small models in their ablations barely benefit from examples in the prompt, and the 175B model benefits a lot. Whatever this ability is, it’s something that shows up with scale rather than something anyone explicitly built.
\nThe least mystical way I can describe it: after reading a good chunk of the internet, the model has seen so many patterns-with-examples that “infer the pattern and continue it” became part of what next-token prediction means. Conditioning does the work that gradient descent used to do. The examples don’t update the model; they locate the task inside what the model already knows.
\nI’m honestly not sure whether that framing is deep or deflationary. Possibly both.
\nIn the git hook post I wrote that GPT-2 “completes, it doesn’t obey,” and that what these models are missing “isn’t size, it’s some way of being trained to follow intent.”
\nGPT-3’s answer to that is annoying: it doesn’t follow intent either — it’s still a pure completion engine — but at 175B parameters, completion gets so good that arranging examples becomes a usable interface. The paper’s few-shot prompts are exactly the “prompt origami” I was complaining about. They didn’t remove the origami. They made the paper so foldable that origami works.
\nSo: half right, half wrong. Wrong that size wouldn’t matter — size mattered enormously, and I’d have lost money betting on that. But I’ll stand by the other half: nothing in this paper trains the model to want to do your task. You’re still tricking an autocomplete into usefulness, and prompts are a brittle, undebuggable way to program. Someone is eventually going to fine-tune one of these giants on the objective “actually do what the instruction says” rather than “continue the document,” and I suspect that model will be the one non-nerds end up using. The capability is clearly in there now; what’s missing is the steering wheel.
\nThe paper’s title is basically a shot at fine-tuning, so as someone with two fine-tuning posts, let me defend my turf — because I think the honest answer is “fine-tuning is dead for tasks, alive for distributions,” and the difference is exactly what my commit-message toy demonstrates.
\nA task is “translate this,” “summarize this,” “classify this sentiment.” If a handful of demonstrations fit in 2048 tokens, GPT-3 says you may not need a dataset, a GPU, or a training loop at all. That’s a real democratization: collecting 50k labeled examples was always the actual cost of applied NLP, and few-shot moves that cost toward zero.
\nBut my commit-message model wasn’t learning a task. It was learning a distribution — the specific texture of my habits, my vocabulary, my guilt. That’s thousands of examples’ worth of signal. It doesn’t fit in any prompt, and no amount of general internet knowledge contains it, because it’s mine. The whole reason the “commit-message-driven development” trick worked is that the model had my history baked into its weights. Ten examples in a context window is a description of me; fine-tuning was a compression of me. Those aren’t the same object.
\nSo my bet: prompting eats the tasks, fine-tuning keeps the personalization and the domains — your codebase, your support tickets, legal boilerplate, medical notes. And knowing which regime your problem is in becomes the actual skill, more than knowing how to run a training script.
\nThings in the paper that the hype tweets will skip:
\nFirst experiment is obvious: rerun the commit-message generator with zero training. Paste five diffs with their real messages into the prompt as demonstrations, staged diff sixth, let it complete. If a frozen model with five examples beats my lovingly fine-tuned GPT-2 with 3,124 — and after this paper I’d bet it does — that’s the whole regime change reproduced on my own silly benchmark, and post number four writes itself.
\nUntil then I’m a guy with a waitlist confirmation email and a 75-page PDF full of margin notes. Weirdest thing I’ve read all quarantine, and that’s a high bar this year.
","excerpt":"Three weeks ago OpenAI dropped the GPT-3 paper — “Language Models are Few-Shot Learners”, 31 authors, 75 pages — and then last week announced an API for it. Private beta, waitlist. I’m on the waitlist. This post is me…","timeToRead":6,"fields":{"slug":"/gpt-3-my-fine-tuning-posts-might-already-be-obsolete/"},"frontmatter":{"title":"GPT-3: my fine-tuning posts might already be obsolete","description":"Reading the GPT-3 paper, waiting for API access, and rethinking what fine-tuning is even for","path":"GPT-3-my-fine-tuning-posts-might-already-be-obsolete","date":"Jun 20, 2020","year":"2020"}},{"id":"82cf150e-2760-58f8-9110-78536c0e854c","html":"Quarantine lasted long enough. Here’s the promised post about the prepare-commit-msg hook — except the project mutated into something much weirder along the way.
Simple: a git hook grabs the staged diff, sends it to my fine-tuned GPT-2 from last month, and the model suggests a commit message. Since loading a 500MB model on every commit is insane, the model lives in a tiny Flask server that stays warm:
\nfrom flask import Flask, request\n\napp = Flask(__name__)\ngenerator = load_my_finetuned_gpt2() # from last post\n\n@app.route(\"/suggest\", methods=[\"POST\"])\ndef suggest():\n diff = request.form[\"diff\"][:3000]\n return generator(diff)And the hook:
\n#!/bin/sh\n# .git/hooks/prepare-commit-msg\nDIFF=$(git diff --cached)\nMSG=$(curl -s localhost:5000/suggest --data-urlencode \"diff=$DIFF\")\nprintf '# suggested: %s\\n%s' \"$MSG\" \"$(cat \"$1\")\" > \"$1\"It was wrong 100% of the time, but the reasons are more interesting than the result:
\nThe model has never seen a diff. I fine-tuned it on commit messages only. From its point of view, a diff is line noise, so it responds to +import redis with “fix typo in readme”. Training distribution is everything — a lesson people apparently keep having to relearn at every scale.
1024 tokens. GPT-2’s entire context window. My average staged diff is 3–4x that. The model was writing commit messages after reading the first file’s worth of changes, which, to be fair, is also how some humans review PRs.
\nIt completes, it doesn’t obey. This is the big one. GPT-2 is an autocomplete engine. There’s no way to tell it “summarize this diff” — you can only arrange text and hope the most probable continuation happens to be what you wanted. What these models are missing isn’t size, it’s some way of being trained to follow intent. Until someone figures that out, everything built on them is prompt origami.
\nStaring at another wrong suggestion, it hit me: the model can’t describe what I did, but it’s weirdly good at proposing things I could do. It’s been trained on 3,000 of my past intentions.
\nSo I flipped the workflow. Every morning the model deals me a commit message. Then I have to write code that makes the message true.
\nWeek one assignments, verbatim:
\ndelete old feature flags\nfix flaky test\nadd caching to search\nupdate deps and prayHere’s the unsettling part: all four were real work I’d been avoiding. I found three dead feature flags, one genuinely flaky test, and a search endpoint that absolutely needed caching. The model isn’t psychic — it’s a compressed statistical summary of my habits, and my habits include perpetually deferring the same maintenance. It’s like Brian Eno’s Oblique Strategies, except the deck was generated from my own guilt.
\nThe hook was supposed to automate me out of a chore. What I got instead is a slot machine that pays out in chores — and it works because it’s random-but-in-my-distribution. A model too weak to answer questions can still be a fantastic question generator.
","excerpt":"Quarantine lasted long enough. Here’s the promised post about the hook — except the project mutated into something much weirder along the way. The plan Simple: a git hook grabs the staged diff, sends it to my fine-tuned…","timeToRead":2,"fields":{"slug":"/my-ai-writes-the-commit-message-first-then-i-write-the-code/"},"frontmatter":{"title":"My AI writes the commit message first. Then I write the code","description":"The GPT-2 commit hook failed exactly as promised, so I turned it around","path":"my-aiwrites-the-commit-message-first-then-i-write-the-code","date":"Apr 25, 2020","year":"2020"}},{"id":"c7da3764-3eb0-523d-bd3e-d8149b7dfd57","html":"At the end of the last post I threatened to fine-tune GPT-2 on my own commit messages. Well, we’re all locked at home now, so no more excuses.
\nFirst, scrape commit messages out of every repo I could find on my machine:
\nfor repo in ~/projects/*/; do\n git -C \"$repo\" log --all --pretty=format:\"%s\" >> commits.txt\n echo >> commits.txt\ndone\n\nsort -u commits.txt > commits_clean.txt\nwc -l commits_clean.txt\n# 31243,124 unique commit messages. That is a tiny dataset by language model standards, which becomes important later.
\nThe transformers repo ships an example script for this — run_language_modeling.py (it was called run_lm_finetuning.py until a few weeks ago, most tutorials still use the old name). Training on my laptop CPU was hopeless, so like everyone else in 2020 I did it on a free Google Colab GPU:
python run_language_modeling.py \\\n --model_name_or_path gpt2 \\\n --model_type gpt2 \\\n --do_train \\\n --train_data_file commits_clean.txt \\\n --line_by_line \\\n --block_size 64 \\\n --per_gpu_train_batch_size 8 \\\n --num_train_epochs 3 \\\n --output_dir gpt2-commits \\\n --overwrite_output_dirThat’s the small 124M model. Three epochs over 3k lines takes about ten minutes on the Colab K80.
\nSampling from the fine-tuned model:
\nfix bug in fix\nremove console.log for real this time\nfinal fix (final) v2\nupdate readme to match the code that no longer exists\nWIP: everything\nadd tests, disable tests\nrefactor the refactor\nfix typo in previous typoI genuinely cannot tell some of these apart from my real commit history, which says more about my commit history than about the model.
\nThe interesting part wasn’t the jokes, it was watching what a language model does with a dataset this small:
\nIt memorizes. By epoch 3 the training loss was still dropping, but a lot of “generated” samples were my real commit messages, verbatim. With 3k examples the model isn’t learning the style of commit messages so much as compressing my git history into its weights. If I’d trained 10 epochs it would basically be a lossy commits.txt. This is the overfitting/memorization tradeoff everyone hand-waves about, visible in ten minutes on a free GPU.
Temperature is everything. At low temperature it plays it safe (“fix bug”, “update readme” forever). Around 0.9–1.0 you get the creative nonsense above. Higher and it starts inventing words.
\nOne epoch was probably enough. The epoch-1 checkpoint produced messages that were on-style but new. More training made it more accurate and less interesting — for a toy like this, “less overfit” is literally funnier.
\nI have a half-working prepare-commit-msg git hook that asks the model to suggest a message from the diff. It is wrong 100% of the time but with confidence. Maybe a post on that if quarantine lasts long enough.
OpenAI finally released the full 1.5B parameter version of GPT-2 back in November, after spending most of 2019 saying it was “too dangerous” to release. Meanwhile someone built AI Dungeon on top of it and half the internet is playing a text adventure game powered by a neural network. So I figured it was time to poke at it myself.
\nThe easiest way to run GPT-2 today is the transformers library from Hugging Face. No TensorFlow checkpoints, no cloning the OpenAI repo:
pip install transformers torchThen generating text is a few lines:
\nfrom transformers import GPT2LMHeadModel, GPT2Tokenizer\nimport torch\n\ntokenizer = GPT2Tokenizer.from_pretrained(\"gpt2-medium\")\nmodel = GPT2LMHeadModel.from_pretrained(\"gpt2-medium\")\n\nprompt = \"Docker Swarm is dead because\"\ninput_ids = tokenizer.encode(prompt, return_tensors=\"pt\")\n\noutput = model.generate(\n input_ids,\n max_length=100,\n do_sample=True,\n top_k=50,\n top_p=0.95,\n)\n\nprint(tokenizer.decode(output[0], skip_special_tokens=True))A few notes from a couple of evenings with it:
\nThe gpt2-medium (355M) model runs fine on CPU on my laptop, just slowly — a few seconds per generation. The full gpt2-xl (1.5B) wants about 6GB of RAM and is painful without a GPU.
Sampling parameters matter a lot. Greedy decoding produces repetitive garbage (“the the the”). top_k + top_p (nucleus sampling) is what makes the output feel coherent.
And “coherent” is doing a lot of work in that sentence. The text is grammatical and stays on topic for a paragraph or two, then confidently drifts into nonsense. It will invent APIs, quotes, and facts with total confidence. Fun toy, but I wouldn’t put it anywhere near production.
\nStill, between this, BERT now running inside Google Search, and Google’s new Meena chatbot paper from a few weeks ago, it feels like NLP is where the interesting stuff is happening right now. The Hugging Face library making it a pip install away is a big part of that.
Next I want to try fine-tuning the small model on my own commit messages and see what horror it produces.
","excerpt":"OpenAI finally released the full 1.5B parameter version of GPT-2 back in November, after spending most of 2019 saying it was “too dangerous” to release. Meanwhile someone built AI Dungeon on top of it and half the…","timeToRead":2,"fields":{"slug":"/playing-with-gpt-2-text-generation-in-python/"},"frontmatter":{"title":"Playing with GPT-2 text generation in Python","description":"Running OpenAI's GPT-2 locally with the Hugging Face transformers library","path":"playing-with-GPT-2-text-generation-in-Python","date":"Feb 21, 2020","year":"2020"}},{"id":"d16f5b13-8a2d-5ffb-b0a0-418278b138fe","html":"You can also read more about Docker Swarm on the official documentation page
\n“Docker Engine 1.12 introduces swarm mode that enables you to create a cluster of one or more Docker Engines called a swarm.\nDocker swarm is a management and orchestration features embedded in the Docker Engine are built using swarmkit. Swarmkit is a separate project which implements Docker’s orchestration layer and is used directly within Docker.”\n
Swarm consists of a multiple Docker hosts which are running in swarm mode and act as managers and workers. Manager is managing delegations and memberships. Workers are running the swarm services.
\nFor example i have 3 separate servers that i would like to run in Docker Swarm mode.
\n1.1.1.1\n2.2.2.2\n3.3.3.3And all of them are connected to the private network 10.0.0.1/16
This part was only important in my use case. So you can skip networking configuration if it’s not relevant to you.
\nAll my servers were connected to the private subnet 10.0.0.1/16. And this was really important that they would stay connected to each other over that network. In this case i needed to change docker ingress network subnet due to the fact that the default ingress subnet has the same subnet.
When you initialize a swarm or join a Docker host to an existing swarm, two new networks are created on that Docker host:
\nInspect networks:
\ndocker network inspect ingressRemove ingress network:
\ndocker network rm ingressCreate a new ingress network with the subnet 20.11.0.0/16:
docker network create \\\n --driver overlay \\\n --ingress \\\n --subnet=20.11.0.0/16 \\\n --gateway=20.11.0.2 \\\n --opt com.docker.network.driver.mtu=1200 \\\n my-ingressWe have decided that 1.1.1.1 will be the manager in our case. On the 1.1.1.1 we will need to initialize swarm mode by running command below:
docker swarm init --advertise-addr 1.1.1.1The output will be the command with the worker token that we will need to run on the other servers in order for them to join to the cluster manager 1.1.1.1
docker swarm join --token SWMTKN-1-19a9q3z51lr1qjakg2wfsdhweweasdsjik323zxgmg23-qwe3qfggwe67a22kdlRun this command on each server that should be connected to the Cluster Manager as a Worker
\ndocker swarm join --token SWMTKN-1-19a9q3z51lr1qjakg2wfsdhweweasdsjik323zxgmg23-qwe3qfggwe67a22kdlOn the manager node run
\ndocker node lsThe output example:
\n![\"\"](\"https://i.imgur.com/eNg9PS6.png\")
In case you will loose join tokens you can run the following commands on the manager node:
\nTo retrieve the manager token run:
\ndocker swarm join-token manager -qTo retrieve the worker token run:
\ndocker swarm join-token worker -qWe can use docker service or docker stack:
docker stack can be used to manage a multi-service application.docker service is used when you need to manage an individual service on a docker swarm cluster.We will use docker stack. Lets create docker-compose-app1.yml file with the following content:
version: \"3\"\n\nservices:\n app:\n image: 15734342727.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest\n restart: always\n ports:\n - 8010:80\n networks:\n - defaultYou’ve probably noticed that i’m usng AWS ECR (private registry) and in order for the workers to pull the image from that registry we need to deploy the app with the option --with-registry-auth to send registry authentication details to Swarm agents.
If you are using private registry as well then before deploying the application we need to login to the registry in my case it is AWS ECR.
\nRun the following command on the manager node:
\naws ecr get-login --no-include-emailThe output will be a docker login ... command. You need to execute this command on the cluster manager in order to login to the AWS ECR registry.
After we can run the following command on the manager node:
\ndocker stack deploy -c docker-compose-app1.yml --with-registry-auth Hello_World_AppTo validate services, run the following command on the manager node:
\ndocker service lsThe example output with the 2 applications:\n![\"\"](\"https://i.imgur.com/D1OWpkf.png\")
To scale the app run the following command on the manager node:
\ndocker service scale <SERVICE-ID>=<NUMBER-OF-TASKS>For example:
\ndocker service scale sjcrh8ra88yf=5I’m assuming that you as a reader are familiar with ReactJS , WebPack and Babel.
\nWebPack takes all your modules with dependencies and puts everything in one or more bundles. Babel is helping us to use modern Javascript features in browsers that are not supporting them. ReactJS is using syntax called JSX and WebPack needs help to convert JSX to ES5 and to provide this help we will use Babel.
\nWe all saw a plenty random services that are offering for example chatbots that can be embedded in to your app as a widget.At the end of this tutorial we will have a react app as a widget that can be deployed to a CDN. The output file will be always with the same name widget.js and in the future it will be easy to provide latest features, bug fixes and etc to your clients without forcing them to include a different version of the file. You or your client only have to include a script from a url for example ”https://cdn/v1/widget.js” and initialize it with a configuration object.
\n<script type=\"text/javascript\" src=\"https://cdn/v1/widget.js\"></script>\n<div id=\"chat-app\"></div>\n<script>\n ReactChatBot.config({\n clientKey: \"123123123\",\n selector: \"#chat-app\"\n });\n ReactChatBot.widgets.myWidget.new().render();\n</script>I’m not going to describe the purpose of each package that we install for this tutorial but i will include source URL where you can read documentation for the specific package.
\nTo create react app we can use a really helpful tool create-react-app. You can install create-react-app by using npm or yarn:
\nnpm -g i create-react-app\ncreate-react-app react_chat_widget\ncd react_chat_widgetOr create app by using npx without actually installing create-react-app on to your system:
\nnpx create-react-app react_chat_widget\ncd react_chat_widgetWebPack is a really great tool with a plenty different and helpful plugins. Most of the plugins are already included with the WebPack, we will install those that are not included.
\nyarn add -D webpack-dev-server webpack-cli css-loader css-loader webpack-merge dotenv-webpack clean-webpack-pluginBelow are all packages that will be used to build and run WebPack configuration files.
\n\n\n\n\n\n\nThis is a webpack plugin that simplifies creation of HTML files to serve your webpack bundles. This is especially useful for webpack bundles that include a hash in the filename which changes every compilation. You can either let the plugin generate an HTML file for you, supply your own template using lodash templates or use your own loader.
\n
\n\n\nThis plugin extracts CSS into separate files. It creates a CSS file per JS file which contains CSS. It supports On-Demand-Loading of CSS and SourceMaps.
\n
\n\n\nPlugins that helps WebPack to collect CSS from all the CSS files and put it in to a string.
\n
\n\n\nIs used to merger different WebPack configuration files.
\n
\n\n\nA secure webpack plugin that supports dotenv and other environment variables and only exposes what you choose and use.
\n
\n\n\nA webpack plugin to remove/clean your build folder(s).
\n
\n\nPlugin is used to minify your JavaScript
\n
yarn add -D @babel/core @babel/plugin-proposal-class-properties \n@babel/preset-env @babel/preset-react babel-loader{\n \"presets\": [\n \"@babel/preset-env\",\n \"@babel/preset-react\"\n ],\n \"plugins\": [\n \"@babel/plugin-proposal-class-properties\",\n ]\n}\nI prefer to create sepparate folder build-utils where i will keep all WebPack configuration files. We will create 4 config files:
\nwebpack.config.js // input config file\nwebpack.common.js // default configurations\nwebpack.dev.js // development mode\nwebpack.prod.js // production modebuild-utils/webpack.config.js:
\nconst webpackMerge = require('webpack-merge');\nconst commonConfig = require('./webpack.common.js');\n\nmodule.exports = ({ env }) => {\n const envConfig = require(`./webpack.${env}.js`);\n // merge default configuration with a chosen mode configuration\n return webpackMerge(commonConfig, envConfig);\n};build-utils/webpack.common.js:
\nconst path = require('path');\nconst webpack = require('webpack');\nconst MiniCssExtractPlugin = require('mini-css-extract-plugin');\nconst HtmlWebpackPlugin = require('html-webpack-plugin');\nconst { CleanWebpackPlugin } = require('clean-webpack-plugin');\nconst TerserPlugin = require('terser-webpack-plugin');\n\nconst publicPath = '/';\nmodule.exports = {\n // Entry point, from where all extraction should be made\n entry: './src/index.js',\n // Init webpack rules to collect js, jsx, css files\n module: {\n rules: [\n {\n // Extract and Transpile ES6+ in to ES5 \n test: /\\.(js|jsx)$/,\n exclude: /node_modules/,\n use: ['babel-loader']\n },\n {\n // Extract CSS files\n test: /\\.css$/,\n use: [MiniCssExtractPlugin.loader, \"css-loader\"]\n },\n ]\n },\n // https://webpack.js.org/configuration/output/\n output: {\n path: path.resolve(__dirname, '../dist'),\n filename: 'widget.js',\n chunkFilename: 'widget.chunk.js',\n // Output library name\n library: 'ReactChatBot', libraryTarget: 'umd',\n publicPath: publicPath,\n libraryExport: 'default',\n },\n // https://webpack.js.org/configuration/dev-server/\n devServer: {\n contentBase: path.join(__dirname, '../dist'),\n hot: true,\n compress: true,\n },\n // https://webpack.js.org/configuration/plugins/\n plugins: [\n new CleanWebpackPlugin(),\n new HtmlWebpackPlugin({\n template: \"./public/index.html\",\n }),\n new MiniCssExtractPlugin({\n filename: \"widget.css\",\n chunkFilename: \"widget.css\"\n }),\n ],\n // https://webpack.js.org/configuration/optimization/\n optimization: {\n minimizer: [\n new TerserPlugin({\n parallel: true,\n sourceMap: true,\n terserOptions: {\n output: {\n comments: false,\n }\n },\n }),\n ]\n }\n}build-utils/webpack.dev.js:
\nconst Dotenv = require('dotenv-webpack');\n\nmodule.exports = { \n mode: 'development',\n // eval-source-map - Each module is executed with eval() and a SourceMap \n // is added as a DataUrl to the eval(). Initially it is slow, but it provides \n // fast rebuild speed and yields real files. Line numbers are correctly mapped \n // since it gets mapped to the original code. \n // It yields the best quality SourceMaps for development.\n devtool: 'eval-source-map',\n plugins: [\n new Dotenv({\n path: './.env.development',\n })\n ],\n} build-utils/webpack.prod.js:
\nconst Dotenv = require('dotenv-webpack');\n\nmodule.exports = {\n mode: 'production',\n // source-map - A full SourceMap is emitted as a separate file. \n // It adds a reference comment to the bundle so development tools know where to find it.\n devtool: 'source-map',\n plugins: [\n new Dotenv({\n path: './.env.production',\n })\n ],\n}We are doing this to avoid leaking any sensitive system ENV data into your application. You can pass here any information that your app is requiring
\n.env.development:
\nNODE_ENV=development.env.production:
\nNODE_ENV=production![\"Folder](\"https://i.imgur.com/rpv1K3Q.png\")
\"scripts\": {\n \"start\": \"webpack-dev-server --config build-utils/webpack.config.js --env.env=dev\",\n \"build\": \"webpack --config build-utils/webpack.config.js --env.env=prod\"\n},To start our application in development mode:
\nyarn startTo build our application for production:
\nyarn buildFor this example i decided to use react-chat-bot package. We need to install all required dependencies to run the package in our react app:
\nyarn add react-simple-chatbot styled-componentsIn src folder create a new folder structure
\nCreate src/components/chat/index.js:
\nimport React, { useState } from 'react';\nimport ChatBot from 'react-simple-chatbot';\n\nconst ChatWidget = (props) =>{\n const [ open, setOpen ] = useState(false);\n const toggleFloating = () =>{\n setOpen(!open)\n }\n\n return (\n <ChatBot\n floating={true}\n opened={open}\n toggleFloating={toggleFloating}\n steps={[\n {\n id: '0',\n message: 'Welcome to react chat widget!',\n trigger: '1',\n },\n {\n id: '1',\n message: 'Bye!',\n end: true,\n }\n ]}\n />\n )\n}\n\nexport default ChatWidget;Edit and remove all unnecessary imports from src/index.js:
\nimport React from 'react';\nimport ReactDOM from 'react-dom';\nimport ChatWidget from './components/chat'\n\nlet init = null;\n\nexport default {\n config: (config) =>{\n init = config;\n },\n widgets: {\n myWidget: {\n new: () => {\n return {\n render: () => {\n ReactDOM.render(<ChatWidget \n clientKey={init.apiKey}\n selector={init.selector} \n />, document.querySelector(init.selector));\n },\n unmount(){\n ReactDOM.unmountComponentAtNode(document.querySelector(init.selector)); \n },\n }\n }\n }\n }\n}\nHere we have method new() that creates a new instance and after we only need to call render() method that will render our widget and if client needs to remove the widget he can call unmount() method. The config() is used to pass any external data, in our case as props to a ChatWidget component.
\nEdit public/index.html:
\n<!DOCTYPE html>\n<html lang=\"en\">\n <head>\n <meta charset=\"utf-8\" />\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n <meta name=\"theme-color\" content=\"#000000\" />\n <meta\n name=\"description\"\n content=\"Web site created using create-react-app\"\n />\n <link rel=\"apple-touch-icon\" href=\"logo192.png\" />\n </head>\n <body>\n <noscript>You need to enable JavaScript to run this app.</noscript>\n <div id=\"chat-app\"></div>\n </body>\n <script> ReactChatBot.config({ clientKey: \"123123123\", selector: \"#chat-app\" }); ReactChatBot.widgets.myWidget.new().render()</script></html>To start app in develoment mode
\nyarn startTo compile a production build run in terminal:
\nyarn buildPush your project to a github repo and then load your widget as it shown here https://www.jsdelivr.com/. In my case the CDN url is https://cdn.jsdelivr.net/gh/drawrowfly/react-chat-widget/dist/widget.js
\nTutorial source code can be found here: github
","excerpt":"I’m assuming that you as a reader are familiar with ReactJS , WebPack and Babel. WebPack takes all your modules with dependencies and puts everything in one or more bundles. Babel is helping us to use modern Javascript…","timeToRead":9,"fields":{"slug":"/building-a-react-app-as-a-widget-using-webpack/"},"frontmatter":{"title":"Building a React App as a widget using WebPack","description":"Building a React App as a widget using WebPack","path":"/Building-a-React-widget-with-WebPack","date":"Nov 08, 2019","year":"2019"}}]}}}