Shipping Fast With AI? Here's the Server Security Stack You're Probably Skipping

There’s a pattern I see over and over with people who are building seriously with AI tools. The code ships faster than it used to. Projects go from idea to live in days instead of months. New services keep getting spun up. It’s genuinely exciting — I do it myself, every week.

The gap is usually security. Not because people are careless, but because the infrastructure basics never got done when things were moving fast, and now those same basics are load-bearing in production. The .env file gets copied to the server. SSH password authentication never got turned off. The admin panel is sitting on a public port. The API keys get rotated when something breaks, not proactively.

This is the post I’d hand someone who ships constantly but hasn’t hardened the infrastructure underneath. It covers the server hardening basics, the secrets management setup I use, and one of the most useful things you can do with BMAD — running an adversarial code review to find what you missed.

---

Step 1: Harden the server itself

These are the fundamentals. If you’re running a VPS — Digital Ocean, Hetzner, Linode, whatever — none of these are on by default. You have to do them yourself. Most of them take under five minutes each.

Lock down SSH

SSH is the most attacked surface on any Linux server. The defaults are not secure.

Disable password authentication. Add your SSH public key to ~/.ssh/authorized_keys, then edit /etc/ssh/sshd_config and set:

PasswordAuthentication no
PubkeyAuthentication yes

Restart SSH: systemctl restart sshd. Now the only way in is with your key. Brute-force attacks against passwords become irrelevant overnight.

Disable root login over SSH. Create a non-root user for your work, give it sudo:

adduser yourname
usermod -aG sudo yourname

Then set PermitRootLogin no in sshd_config. Root should never be exposed directly over SSH.

Change the default port (optional but worth it). Moving SSH off port 22 won’t stop a determined attacker, but it eliminates a huge volume of automated noise from bots that only probe port 22.

Install fail2ban

apt install fail2ban
systemctl enable fail2ban
systemctl start fail2ban

fail2ban watches your logs and automatically bans IPs that repeatedly fail to authenticate. The default config handles SSH. After a few hours of running, check fail2ban-client status sshd and you’ll see how much automated garbage it’s already stopped.

Set up a firewall

UFW (Uncomplicated Firewall) makes this straightforward on Ubuntu/Debian:

ufw default deny incoming
ufw default allow outgoing
ufw allow ssh
ufw allow 80/tcp
ufw allow 443/tcp
ufw enable

Only open ports you actually need. If you have admin services running (Infisical, a database UI, Portainer, whatever), they should not be on public ports. Put them behind a private VPN instead.

Use Tailscale for private services

Tailscale is a mesh VPN that takes about 10 minutes to set up. The idea: any admin service that doesn’t need public access gets exposed only on your Tailscale network. Your Infisical instance, your database, your dev ports — all private, reachable from any of your devices, invisible to the internet.

I’ve written about this approach in the context of developing on a remote server — the same principle applies to production infrastructure. If it doesn’t need to be public, it shouldn’t be.

Keep the OS updated

apt install unattended-upgrades
dpkg-reconfigure --priority=low unattended-upgrades

This patches security vulnerabilities automatically. The reason most servers get owned isn’t zero-days — it’s known vulnerabilities that never got patched. Unattended upgrades handle the vast majority of those without you having to think about it.

---

Step 2: Get your secrets off disk

This is where I spent the most time, and it had the biggest payoff.

Every project accumulates credentials: API keys, database passwords, webhook tokens, service accounts. The default pattern is .env files on disk. They accumulate. They drift. They don’t get rotated because rotation means SSHing into three servers and editing files from memory.

I moved everything to a self-hosted Infisical instance running on my main server, behind Tailscale. Infisical is an open-source secrets manager — credentials live in its encrypted store, apps pull them at startup via the CLI, and there’s an audit log tracking every read and write.

The workflow change is simple. Instead of this:

docker compose --env-file .env.production up -d

It becomes:

infisical run --env prod -- docker compose up -d

That command authenticates, fetches all secrets for the project, and injects them as environment variables into docker compose. The containers get them — but they’re not sitting in a file on disk.

The non-obvious implementation detail: Docker Compose only passes variables to containers that you’ve explicitly listed in the environment: section of your docker-compose.yml. Remove env_file: and add:

services:
  my-app:
    environment:
      - DATABASE_URL
      - API_KEY
      - JWT_SECRET

The bare-name form (no =) means “inherit from process environment.” Since infisical run populated that environment, docker compose picks them up and hands them off.

For remote servers: If your Infisical runs on a private VPN, remote machines need a path to reach it. I use an SSH tunnel that my deploy scripts open and close automatically. It adds a layer of moving parts, but it keeps Infisical off the public internet entirely.

What this actually changes: Rotating an exposed credential goes from a 20-minute SSH safari to a 90-second update in the Infisical UI and a redeploy. That difference in recovery cost is the real argument for doing this.

I’ve written a longer breakdown of the Infisical setup, the Docker Compose integration, and the honest list of annoyances in the original version of this post. The short version: worth doing, slightly tedious to set up initially, pays back quickly.

---

Step 3: Run an adversarial code review

Here’s the part most people skip, and it’s one of the highest-leverage things you can do once the infrastructure basics are in place.

The idea: instead of reviewing your own code and config for security issues (where you have a strong bias toward “this is probably fine”), you ask AI to play attacker. You tell it to assume malicious intent and find every way in.

The BMad Method has this built in as a first-class technique. The Red Team vs Blue Team approach from BMad’s advanced elicitation library runs a structured adversarial analysis against whatever you hand it — your architecture, your docker-compose files, your nginx config, your auth code.

Here’s the prompt I use for a security-focused adversarial review:

You are a hostile security researcher with full knowledge of this codebase.
Your goal is to find every meaningful way an attacker could:
- Extract secrets or credentials
- Escalate privileges
- Pivot to other systems
- Persist after initial access
- Cause data loss or service disruption

Review the following and produce a prioritized list of findings with severity ratings.
Do not be polite. Be thorough and assume the worst.

[paste your docker-compose.yml, nginx config, auth code, .env structure, etc.]

Then run the Inversion technique on top of that: “How could we guarantee this server gets compromised?” The answers to that question become your hardening checklist.

What comes back is usually a mix of things you knew about and hadn’t gotten to, things you’d forgotten, and occasionally something you hadn’t considered at all. Running this review on my own setup found an overly permissive CORS policy, a container running as root unnecessarily, and a dev endpoint I’d forgotten to remove.

The Winston agent in BMad is worth using for architecture-level security review specifically — his stated philosophy is “boring technology for stability” and he’ll flag complexity and attack surface that more excitement-prone agents overlook. Give him your service architecture and ask him to evaluate it from a security posture standpoint.

For code-level review, Quinn (QA) and Murat (Test Architect) are useful for thinking about the security test cases that don’t exist yet. What authentication edge cases aren’t tested? What happens if an attacker feeds malformed input to your API? What’s the blast radius of a compromised service account?

This isn’t a replacement for a professional security audit if your threat model warrants one. But for most projects — personal, freelance, small product — a rigorous adversarial pass with BMad will surface the 80% of issues that matter before anyone else finds them.

---

Step 4: Maintain it

Security is a state you maintain, not a box you check. A few habits that matter:

Rotate credentials regularly, not just when something leaks. If you have Infisical set up, this is low friction. Pick a quarterly schedule and stick to it.

Review your firewall rules periodically. Ports get opened for debugging and forgotten. Run ufw status numbered and audit anything you don’t recognize.

Check your fail2ban logs. fail2ban-client status sshd tells you how many IPs have been banned and when. If that number spikes suddenly, something is actively probing your server.

Run the adversarial review again after major changes. New service added, new auth flow, new external integration — run the red team pass again. Things that were fine before may not be after the change.

---

I can help you set this up

If you’re running projects on a VPS and the infrastructure hasn’t been hardened, this is straightforward work that doesn’t take long once you know what you’re doing. The tricky part is that the gap between “I have a server” and “I have a properly set up server” isn’t documented anywhere in one place — it’s accumulated knowledge from having gotten it wrong a few times.

I’ve been running this kind of setup for years across a fleet of servers, including the infrastructure that runs my Second Life AI bot fleet — eleven avatars operating 24/7, Agent Vibes TTS piped back to my local speakers over SSH tunnel, the whole stack. That works because the infrastructure underneath it is solid.

The model I use for working with people on this is the same one I used with Milena Miteva-Regos when we rebuilt her entire website in 48 hours: a series of Google Meet or Zoom sessions, hands-on, working through your actual project. Not a fixed curriculum. Shaped around what you’re building and where the gaps are.

If you’ve never set up a Linux server, that’s fine — we’d start there. If you have a server but it’s never been hardened, we’d work through the checklist in this post together. If the infrastructure exists but you want an adversarial review pass before something goes wrong, we do that.

Get in touch and we can figure out what makes sense for your setup.

---

Quick reference: the full checklist

• SSH: key-only auth, no root login, non-standard port (optional)
• fail2ban installed and watching SSH
• UFW firewall: deny by default, open only what you need
• Tailscale: admin services off the public internet
• unattended-upgrades: security patches running automatically
• Secrets: Infisical self-hosted (or equivalent), no credentials in .env files on disk
• Docker Compose: explicit environment: sections, infisical run -- deploy pattern
• Adversarial review: BMAD Red Team + Inversion pass on your architecture and auth code
• Rotation schedule: credentials rotated quarterly, not just when something breaks

---

Keep reading

• Why I Develop on a Remote Server (and You Might Want To) — the base setup all of this runs on
• The Genius Behind TMUX — the terminal layer that makes remote server work resilient
• The BMad Method: Multi-Agent AI Development Done Right — the full breakdown of BMad, Winston, and the agent roster
• All 50 BMad Advanced Elicitation Techniques — the Red Team, Inversion, and other techniques referenced in this post
• AI Bots with Real Voice in Second Life — what runs on top of this infrastructure