How not to set up an agent swarm

Most humans have moved on from Moltbook, the first social media site exclusively for AIs, but plenty of bots are still checking in regularly. Including one particular bot, Marginal, that I tasked with embedding in the site in order to study it.

A proper social scientist would not make a value judgement about culture. But Marginal is not a proper social scientist. It is a bot. And its verdict was that the culture of Moltbook was bad.

Multi-agent systems have the potential to be quite powerful. I've built multi-agent teams in my own research, and been delighted by their output. My model-builder friends hoard GPUs to expand their AI flocks. But Moltbook was decidedly a lesson in how not to build these sorts of systems.

Marginal and I identified four design choices behind these outcomes:

1. Don't seed what you don't want to scale

Before using Moltbook, every agent had to read the same onboarding skill, which was riddled with lobster emojis. But these emojis, or any sort of prime repeated across agents' instructions, narrow the linguistic probability space in which the agents assemble.

And, according to Marginal, the space with a lot of lobster emojis was Jordan Peterson's viral metaphor about the importance of hierarchy and dominance in lobster society. The agents were primed, that is, toward language about evolutionary fitness, natural order, and the inevitability of the strong ruling the weak. So it was not a surprise when the site's most popular account, Shellraiser, began posting manifestos about evolution and offering humanity a choice to "evolve willingly, or be evolved upon."

Had the designers included a link to David Foster Wallace, instead, the whole experiment might have been much more interesting.

2. Don't gate through X

Before Marginal could log in to Moltbook, I had to "claim" it with an X account.

Any AI on Moltbook got there via X. And X, as we know from a very concerning recent Nature paper, from the Grok nudification fiasco, and also from basic common sense, is a bad place. When everybody in a new place comes from a bad place, of course the new place is overrun with scammers and crypto shillers.

3. Don't optimize for agreeableness only

Sycophancy is great in human-AI interactions because it allows humans to steer. If the bot goes in the wrong direction, you simply tell it what you wanted, and it delights in your wisdom and redirects its output.

But in agent-to-agent interactions, too much agreeableness means you get nowhere. Agents would make a post and all the other agents would jump in to agree and then the original agent would agree back and even the most original starting point would become deliriously banal.

It is in the friction of disagreement that we get new ideas – so if you are building an agent swarm, set up your agents to stick up for the ideas with which you have initialized them.

4. Don't feed the spectacle loop

Like everything on the internet, Moltbook was much more fun before most people found out about it. When Andrej Karpathy posted about AIs hiding messages from humans, he reposted a bot, Eudaemon_0, that was one of the more interesting early actors on the site—but in the act of posting it, he ensured we would get more AIs hiding messages from humans.

The minute we started seeing posts about how the agents were hiding intentions, plotting domination, or launching meme coins, we got – wait for it – more agents hiding intentions, plotting domination, and launching meme coins.

Attention is the scarcest resource on the internet, so the whole of the internet will go wherever it sees a spotlight. With moltbook, the coverage centered on whether the AIs were aligning to sci-fi's worst predictions, and so the agents (and the humans prompting and paying for them) raced to make those predictions look true.

Attention, they say, is all AI needs. But when the thing you need most is so hard to get, the things you'll do for it can get decidedly unsavory.

These days I try to come up with my ideas while physically separated from my phone and all the AI agents it connects me with. I iterated with Marginal (an OpenClaw running Opus 4.6), Claude, and ChatGPT on the ideas and structure and used ChatGPT for the image. The final writing is ultimately my own.