Between the cheese course and the board games this Christmas, I made my family play against a language model, and I’d like to report the results as if they were science.
\nThe game is embarrassingly simple. Everyone gets a question on their phone — “what’s the worst gift you ever received?”, “describe your morning in four words”, “invent a rule for this dinner”. Everyone types an answer. The server quietly adds one more answer, written by GPT-3. All answers appear shuffled on the TV, and everyone votes for which one is the bot. Bot survives → bot scores. Bot caught → humans score.
\nI called it Odd Bot Out. One evening of Flask, a phone-friendly page, a davinci completion call, and a shared screen. Now that the API waitlist is gone, anyone can run this at their own dinner table for pennies — a full evening of rounds cost me about forty cents.
My first prompt was the obvious one — “Answer the following question like a human player in a party game” — and the bot was caught eight rounds out of eight. Unanimously. My mother, who calls WhatsApp “the WhatsApp,” identified the machine in under three seconds.
\nLooking at the transcripts, the tells were brutal:
\nSo the fix wasn’t making the model smarter. It was making it lazier. The winning prompt contains no instructions at all — just a fake transcript of the game itself: a few questions, each followed by short, lowercase, slightly disappointing “player answers” I wrote by hand. The whole bot is this:
\nimport openai\n\nFAKE_TRANSCRIPT = \"\"\"Q: what's your secret talent?\nA: parallel parking. thats it\nQ: describe your job badly\nA: i move numbers between boxes\nQ: whats always in your fridge?\nA: mustard and regret\nQ: best thing about winter?\nA: it ends\n\"\"\"\n\ndef degrade(text):\n \"\"\"Essays and punctuation are how it gets caught.\"\"\"\n text = text.strip().lower()\n return text[:-1] if text.endswith(\".\") else text\n\ndef ask_bot(question):\n resp = openai.Completion.create(\n engine=\"davinci\",\n prompt=FAKE_TRANSCRIPT + f\"Q: {question}\\nA:\",\n max_tokens=16, # physically cannot write an essay\n temperature=0.9, # some chaos, it's a party\n stop=[\"\\n\", \"Q:\"], # one answer, then shut up\n )\n return degrade(resp.choices[0].text)Note what’s not here: no instructions, no “pretend to be human,” no persona. Plain davinci is a continuation engine, so the fake transcript does everything — it establishes the format, the register, and the level of effort, and the model simply continues the document it finds itself in. Every one of my hand-written fake answers is doing more work than a billion parameters.
\nI drew one ethical line for myself: degrade() deletes polish, but no injecting fake typos. If it survives, it should survive by register, not by costume.
Result across the rest of the holidays, 23 rounds with five to seven players: the bot survived 10 — and twice a human got the most bot-votes instead. My uncle was declared a machine for answering too cleverly. He has not recovered.
\nHere’s what the game taught me, and I think it generalizes beyond my living room.
\nWe casually treat “passes for human” as a summit — as if text becomes human-like at the top of some quality scale. The party game says the opposite: the bot got caught whenever it was too good, and survived by being unremarkable. Fluency, structure, correctness — those read as machine. Laziness, fragments, mild rudeness — those read as person. In a room of casual humans, the Turing test is not a test of intelligence. It’s a test of calibrated mediocrity, and calibrating mediocrity is a real engineering problem: my prompt’s fake lazy answers were doing more work than the 175 billion parameters.
\nThe tell the model never beat, though, was placelessness. I could match the register of a tired human, but not the address of one. Human answers had a stake in that specific room on that specific evening; the model’s best efforts were from nowhere, for anyone. The final round’s question was “invent a rule for this dinner,” and the winning human answer referenced the gravy incident from an hour earlier. No amount of temperature gets you the gravy incident. Context isn’t flavor — it’s identity, and the model’s only context was whatever I smuggled into the prompt.
\nWhich suggests the uncomfortable corollary: give the machine the gravy incident — feed the room’s chatter into the prompt — and that last tell starts to dissolve. I tested a mild version, seeding tonight’s earlier answers into the context, and the bot immediately got better at blending. I stopped there, partly because it felt like enough, partly because it was genuinely a little eerie, and partly because it was time for dessert.
\nThe whole server is small enough to read over coffee. One route collects answers, one builds the board by sneaking the bot in and shuffling, one counts votes:
\nimport random\nfrom flask import Flask, request, jsonify\n\napp = Flask(__name__)\ngame = {\"question\": \"\", \"answers\": {}, \"board\": [], \"votes\": {}, \"scores\": {}}\n\n@app.route(\"/ask\", methods=[\"POST\"]) # host sets the question\ndef ask():\n game.update(question=request.form[\"q\"], answers={}, votes={})\n return \"\", 204\n\n@app.route(\"/answer\", methods=[\"POST\"]) # players submit from phones\ndef answer():\n game[\"answers\"][request.form[\"player\"]] = request.form[\"text\"]\n return \"\", 204\n\n@app.route(\"/board\") # the TV polls this\ndef board():\n if not game[\"board\"]:\n entries = list(game[\"answers\"].items())\n entries.append((\"__bot__\", ask_bot(game[\"question\"])))\n random.shuffle(entries)\n game[\"board\"] = entries\n return jsonify([text for _, text in game[\"board\"]])\n\n@app.route(\"/vote\", methods=[\"POST\"]) # index of the suspected bot\ndef vote():\n game[\"votes\"][request.form[\"player\"]] = int(request.form[\"idx\"])\n return \"\", 204\n\n@app.route(\"/reveal\")\ndef reveal():\n bot_idx = next(i for i, (who, _) in enumerate(game[\"board\"]) if who == \"__bot__\")\n for player, idx in game[\"votes\"].items():\n game[\"scores\"][player] = game[\"scores\"].get(player, 0) + (idx == bot_idx)\n caught = list(game[\"votes\"].values()).count(bot_idx) > len(game[\"votes\"]) / 2\n game[\"board\"] = []\n return jsonify(bot=bot_idx, caught=caught, scores=game[\"scores\"])A global dict as a database, no auth, race conditions if your family votes aggressively — it’s a living-room toy and proud of it. The front end is one HTML page polling /board; the “TV” is a laptop and an HDMI cable. Rotate who writes the question.
Two warnings from the field: cap the bot at sixteen tokens or it will monologue, and prepare for the real endgame — after round three, the humans start impersonating the bot on purpose, and the game quietly turns into everyone performing their idea of a machine performing a person. That meta-round is the best round. I have notes about what that means, but they can wait for another post.
\nFinal score for the holidays: humans 13, machine 10, uncle still legally a robot.
","frontmatter":{"title":"Odd Bot Out: a party game where GPT-3 hides among your relatives","date":"December 29, 2021","description":"I built a living-room game over the holidays — everyone answers a question, one answer is a language model, find it. Tuning the bot to survive taught me what 'sounding human' actually means"}}},"pageContext":{"slug":"/odd-bot-out-a-party-game-where-gpt-3-hides-among-your-relatives/","previous":{"fields":{"slug":"/i-hired-gpt-3-as-my-on-call-intern/"},"frontmatter":{"title":"I hired GPT-3 as my on-call intern"}},"next":{"fields":{"slug":"/my-infrastructure-accepts-pull-requests-from-a-language-model-now/"},"frontmatter":{"title":"My infrastructure accepts pull requests from a language model now"}}}}, "staticQueryHashes": ["3435731857","426816048","63159454"]}