We Tried Building an AI Team on Grok. Here's Why We Moved to Self-Hosted Rakazo Instead

There’s a shift happening in how people work with AI agents, and it’s worth naming plainly: instead of one general-purpose agent trying to do everything, people are increasingly reaching for a team of specialized agents — one built to be very good at a single job, able to hand work to another one built for a different job. It’s the same reason I’ve been running the BMAD method for over a year now. I’d rather stand up one agent as a product manager and another as a product designer, and let them talk to each other, than ask a single assistant to be everything at once.

xAI leaned into that same direction with its own bot-building tools inside Grok — a way to spin up a roster of specialized bots on top of one model. We tried it. Here’s the problem: we ran out of context within a day and a half.

So, to keep pushing on this way of working with AI, we went looking for an open-source alternative instead of a closed platform we didn’t control. That search led us to Rakazo — and so far, it’s been a real success: a whole team of AI agents, running through a much simpler interface than Grok ever gave us. Here’s what we tried, why we backed out of Grok, and what we built instead.

The roster we built

The plan was simple on paper. Instead of one general-purpose AI we ask for things ad hoc, build a small roster of specialized bots, each with a job:

• A Chief of Staff bot to triage and coordinate
• A copywriter bot for marketing copy and content
• A job finder bot
• A marketing bot

Each one would get its own memory and its own context — its own slice of “who we are, what we’re doing, what happened last week” — so it could actually behave like a specialist who remembers things, instead of a chatbot that starts from zero every conversation.

We set this up on Grok. Seeded each bot with its own context and history. On paper, this is exactly the kind of thing a big frontier model should be great at.

Where it broke: context, not capability

Within a day and a half, we ran into a wall — Grok’s context limits. [Paul: fill in the actual context window size and what specifically got cut off here.] The bots simply couldn’t hold enough history to stay useful for how we actually work. A Chief of Staff bot that forgets what happened two days ago isn’t a Chief of Staff — it’s a search box with a personality.

To be clear, this isn’t a knock on xAI or Grok as a model. Grok is a capable model doing what closed platforms do: giving you a fixed amount of memory and infrastructure you don’t control. For a single well-scoped assistant, that’s often fine. For a standing team of bots meant to accumulate months of institutional memory, it wasn’t going to work — at least not without us being able to reach in and change how memory and context were being managed, which a closed platform doesn’t let you do.

That was the real problem: it wasn’t just that we hit a ceiling, it’s that we had no lever to move it.

The pivot: self-hosted, open-source, our infrastructure

So we went looking for an alternative we could actually control. We found Rakazo — an open-source “AI teammate” platform you run yourself. Docker Compose, your own Postgres database, your own servers — the same self-hosted philosophy behind why I develop on a remote server in the first place. No vendor deciding how much memory your bots get to have.

We migrated our entire bot roster over — the Chief of Staff, the copywriter, the job finder, the marketing bot — with their memory and transcripts intact. Nothing lost, nothing reset. They picked up where they left off, just running on infrastructure we own instead of infrastructure we’re renting time on.

The difference isn’t abstract. When something isn’t working — memory retention, response latency, how a bot’s context gets managed — we can go look at the actual configuration and change it. There’s no support ticket, no waiting on a platform roadmap, no ceiling we can’t see past.

And it’s simpler to actually use day to day. Where Grok’s setup meant juggling separate bot configurations with no shared view of the team, Rakazo gives us the whole roster in one interface — one place to see every bot, its memory, and what it’s doing. It’s the same relief I wrote about after finally getting several Claude Code agents organized in one terminal instead of losing track of who was doing what.

Then we went further: wiring a bot into real WhatsApp

Once the migration was solid, we took it a step further. We wired our Chief of Staff bot directly into our real WhatsApp business lines — not a sandbox, not a demo number, the actual lines we use for the business. It can now read incoming WhatsApp messages and send replies as one of its tools.

We did this using MCP — the Model Context Protocol. If you haven’t run into it: MCP is a standard way for an AI to plug into outside tools and services, the same way a USB port lets any device plug into any computer without a custom adapter for each one. Instead of writing a one-off, bespoke integration between our bot and WhatsApp, we used an existing open-source MCP server built for WhatsApp, and Rakazo — because it speaks MCP natively — just knew how to talk to it.

That’s the part that actually sold us on this approach. No proprietary integration to maintain. No platform-specific plugin that breaks on the next update. Just a standard protocol on both ends.

Why this matters beyond our setup

The Grok experiment wasn’t wasted — it told us something real: for a long-running team of bots meant to accumulate memory over time, you want to own the infrastructure underneath them. Not because closed platforms are bad, but because when you hit their ceiling, you can’t do anything about it except wait or leave. We left.

Self-hosting cost us more setup time up front. It bought us no vendor lock-in, no context ceiling we don’t control, and the ability to wire our bots into whatever tool we need next using the same protocol, instead of a new bespoke integration every time.

I can help you set this up

If you’re weighing the same trade-off — a closed platform that’s fast to start but caps out on you, versus something self-hosted you have to stand up yourself — this is exactly the kind of thing I work through with people. Standing up Rakazo, migrating a bot roster with its memory intact, and wiring an MCP server into a real business tool like WhatsApp isn’t hard once you’ve done it, but it’s not documented in one place either.

Do this yourself

If you want to replicate the WhatsApp-to-Rakazo setup on your own self-hosted Rakazo instance, here’s the exact prompt we’d hand to an AI coding assistant to walk through it. It’s generic and secret-free — no bridge URLs, phone numbers, or paths of ours in it. Fill in your own environment as you go.

I'm self-hosting Rakazo (github.com/elie222/rakazo) via Docker Compose, and
I want one of my bots to be able to read and send WhatsApp messages through
Rakazo's MCP (Model Context Protocol) support. I already have (or will set
up) an open-source WhatsApp bridge exposing a local HTTP API (e.g.
lharries/whatsapp-mcp or a compatible fork) that can send/receive messages
and exposes a stdio MCP server.

Please help me:
1. Confirm my Rakazo worker container has Python 3.11+ and `uv` available
   (add them to the shared app image if not — check infra/compose/Dockerfile).
2. Get the WhatsApp bridge's MCP server code and message-store database
   reachable from the worker container (bind mounts and/or a shared Docker
   network — whatever fits my compose setup).
3. Enable stdio MCP on the worker (MCP_STDIO_ENABLED=true) and allow only
   the exact command needed (MCP_STDIO_ALLOWED_COMMANDS), nothing broader.
4. Register the WhatsApp MCP server in Rakazo (via its UI or RPC API) as a
   stdio server with my bridge's env vars (API base URL, API key, message
   DB path) stored through Rakazo's own encrypted secret store — not typed
   into a config file in plaintext.
5. Assign it to my chosen bot with a scoped tool allowlist (e.g. just
   list_messages, send_message, search_contacts, list_chats) rather than
   granting it every tool the server exposes.
6. Verify it actually works: have the bot send a real test message to my
   own number and confirm it arrives, not just that the tool call didn't
   error.

Walk me through each step for my specific setup, and flag anything in my
config that would expose credentials or open unnecessary network access.

Hand that to your coding assistant, walk through it step by step, and you’ll end up with the same setup — your own bots, your own memory, talking to your own WhatsApp lines, with nobody else’s ceiling in the way. If you want the broader playbook for getting API keys and tokens off disk everywhere in your stack, not just inside Rakazo, that’s the same approach I use everywhere: self-hosted secrets management with Infisical.

---

Keep reading

• Why I Love the BMad Method: Multi-Agent AI Development Done Right — the framework behind wanting specialized agents in the first place
• Are You Running More Than One Claude Code Session at Once? — keeping a roster of agents organized instead of losing track of who’s doing what
• Why I Develop on a Remote Server (and You Might Want To) — the self-hosted infrastructure philosophy behind moving off Grok
• Shipping Fast With AI? Here’s the Server Security Stack You’re Probably Skipping — how to keep credentials out of plaintext once you’re self-hosting
• Ninety-Nine Agents: Why You Cannot Walk Away From an AI — what can go wrong when agents run without oversight