8 Signs Your AI App Is a Demo, Not a Product
You shipped it. Users came. Then everything broke. Here's the checklist you skipped.
A friend sent me a link on a Friday night. "Try this, it's not finished yet but it works."
Already a bad sign: localhost:3050. 😬
TL;DR: 8 signs your AI app isn't production-ready. For each one: what breaks, why it breaks, and a Claude Code prompt to fix it. Essential if you've already shipped something, even a personal tool nobody else knows about yet.
I told him. He sent a proper link ten minutes later.
I opened the app, looked around. The idea was solid. But occupational hazard, I hit F12. Opened the console.
What I found was a highlight reel of everything that kills apps in production.
CORS errors stacked three deep. An OpenAI API key sitting in plain text inside the network requests. A SELECT * firing on every keystroke in the search field. No rate limiting anywhere. When I asked how he monitored errors in prod, he said "I just check if it crashes."
Not a product. A demo with a domain name.
Classic.
There's a version of "done" that looks exactly like the real thing. Same UI. Same URL. Sometimes even paying users. But underneath, it's running on assumptions that only hold when nothing goes wrong.
A demo is optimized to impress. A product is optimized to survive. Those aren't two points on a spectrum, they're two completely different build modes. A demo succeeds if it works once in front of someone. A product has to work ten thousand times while you're not watching.
Every sign on this list is a direct consequence of that single distinction.
1. You Have Zero Visibility Into What's Happening in Production
If your monitoring strategy is "I'll notice if it crashes," you don't have monitoring. You have hope.
No structured logs. No error alerts. No uptime checks. In demo mode, you run the app yourself, you see everything, nothing surprises you. In production, features can break silently for three days. Users stop coming back. You have no idea why.
I had an n8n webhook that stopped processing for 48 hours. Found out by accident while debugging something else. Then I read a comment on one of my Medium articles asking why the tool I'd mentioned was returning 503s. The article was three days old. The tool had been down since I published it.
Fix: Sentry free tier for runtime errors. UptimeRobot for uptime. Fifteen minutes total. No excuse.
"Set up basic production monitoring for this app. Add Sentry for error tracking, integrate UptimeRobot via a /health endpoint, and add a Slack webhook alert for any unhandled exception in prod."
You don't need a war room. You need a smoke detector.
2. You Don't Have a Staging Environment
My friend's app lived on one server. The same server he was actively developing on. Which meant every time he pushed a fix, production was the test environment.
I once broke OpenClaw's auth for two hours because an environment variable in prod had a slightly different name than local. The kind of thing you catch immediately in staging. The kind of thing that makes users think your app randomly logs them out, and then they don't come back.
Staging doesn't need to be fancy. A $3 VPS (I run mine on a $5 box — same principle). A Vercel preview branch. A GitHub Actions workflow that deploys to staging on every PR and to prod only on merge to main.
Anything that creates one layer between "I just changed something" and "real users are hitting it." The point isn't to have a perfect process. The point is to have one deploy that isn't directly pointed at real users.
"Create a staging environment setup for this project. Separate .env.local and .env.production, add a GitHub Actions workflow that deploys to staging on PR and to prod on merge to main, and add a pre-deploy checklist to the README."
3. Your API Keys Are in the Code
This is what the AI generates by default:
const openai = new OpenAI({ apiKey: "sk-proj-..." });It works. It's also one git push away from being public.
I won't explain how, but I have friends who never pay for a single SaaS tool. Not because they pirate anything. Because every few months they run a two-minute scan and find enough leaked keys to cover their stack for the next quarter. Sitting in public repos. Forgotten in commit history. Like a post-apocalypse world where the doors are open and the fridges are still full. Nobody cleaned up before they left.
Your key is probably in one of those fridges right now.
The AI has no concept of whether you're building a personal todo app or a SaaS with paying customers. It puts the key where it's convenient. "This will end up on GitHub" is not in its threat model.
Fix: .env file, .gitignore, done. Then run git log -p | grep -i "api_key" on your existing repos. You might not like what you find.
"Audit this codebase for hardcoded API keys, tokens, and secrets. List every occurrence with file and line number. Generate the full .env migration and update all references to use process.env."
(This isn't a beginner mistake. It's a "moving fast without a checklist" mistake. It gets everyone eventually.)
4. You Have No Rate Limiting
Someone will find your API endpoint. Could be a bot. Could be a curious user. Could be a scraper that found your URL in a sitemap. Doesn't matter who. What matters is they'll hit it 400 times in three minutes and you'll find out on your billing dashboard.
On ContentForge, I had a generation endpoint with no limits. A scraper hit it before I even launched publicly. No damage because I caught it fast. But the mental alert was loud: if I hadn't been watching the logs that morning, I'd have burned through my entire API budget before the first real user showed up.
Fix: express-rate-limit in Node. Rate limiting at the Traefik or Nginx level if you're self-hosted. Five lines of config. Stops 90% of the dumb stuff before it becomes expensive stuff.
"Add rate limiting to all API endpoints in this codebase. Use express-rate-limit with 100 requests per 15 minutes per IP as default. Apply stricter limits on auth routes and any endpoint that triggers an external API call."
5. You've Never Tested What Happens When Someone Does Something Stupid
The AI generates the happy path. Every time.
The user fills the form correctly, clicks the right button, has a stable connection, doesn't double-click submit. What it doesn't generate: empty field submissions. File uploads with 0 bytes. Sessions that expire mid-form. The same button clicked twice because the first click felt slow.
On ContentForge, a user submitted an empty prompt. NaN got written to the database. Which cascaded into broken metadata on every article generated after that. Found it two days later while debugging something completely unrelated.
Fix: before you share the link with anyone, spend ten minutes actively trying to break your own app. No unit tests required. Just chaos. Click things in the wrong order. Leave fields empty. Submit twice. Refresh mid-operation.
Or just run this in Claude Code first:
"Act as a malicious user trying to break this app. Test every form with empty inputs, special characters, and oversized payloads. Try submitting the same form twice in rapid succession. Expire the session mid-flow. Document every crash, unexpected behavior, or unhandled edge case you find."
The AI will never do this for you unless you explicitly ask. And even then, it'll miss half of it.
6. Your Prod and Dev Configs Are the Same Files 😱
Same .env. Same database. Same S3 bucket.
The AI doesn't make the distinction between environments. It configures for "it works," not for "it works safely when you're testing a destructive migration locally." Environment separation isn't in the prompt, so it doesn't make it into the code.
This isn't just a technical fix. It's a structural decision that has to be made before the first deploy, not discovered six weeks in when a local test wipes a table with real user data. Defining those boundaries upfront is the kind of architectural contract that separates gambling from shipping. That's why the prompt contracts framework exists — defining boundaries before you're forced to.
Fix: .env.local, .env.production, two separate databases. Even for a side project with three users. Especially for a side project with three users.
"Audit this project's environment configuration. Identify every place where prod and dev share the same config, database, or storage. Generate a full environment separation plan with separate .env files, database connections, and storage buckets for each environment."
7. Your Errors Are Silent
The AI writes try/catch blocks. It also fills them with console.error or nothing at all.
The error gets caught. Nobody knows. The user sees a blank result or a frozen spinner. They leave. You see nothing.
I had a contact form on a side project that had been silently dropping every submission for two weeks. The mailer was crashing on a config issue. Caught, logged to nowhere, swallowed. Found out when someone asked in the comments why I never replied to their message.
Two weeks of dead leads. Zero indication anything was wrong on my end.
Fix: Sentry free tier. It's an error tracking platform that catches every unhandled exception in real time, with the full stack trace, the user's browser, the exact line of code, and a reproduction count. Ten minutes of setup. Not optional if you have real users.
No Sentry yet? At minimum, run this first:
"Audit every try/catch block in this codebase. Replace empty catches and console.error-only catches with structured error logging. Add a global unhandledRejection handler. Output a summary of every silent failure point you found."
Silent errors aren't bugs. They're ghosts.
8. You Don't Know What You're Spending in Real Time
The bill arrives. You open it. You do not recognize the number.
In dev, you call the API ten times. You test a feature, it works, you move on. In prod, one user with an aggressive retry loop, one search field firing on every keystroke, one unindexed query running full table scans on a growing dataset. Any of these can turn a $20/month budget into a $200 surprise before you finish your coffee.
The AI generates code that works. It doesn't generate code that's economically safe under load. It has no concept of your budget cap, your margin, or what happens when someone who isn't you hits the endpoint 400 times.
Fix: billing alerts on every service you're paying for. AWS, OpenAI, Anthropic, Supabase. They all have threshold alerts. Set them before you share the link publicly. Hard limits on API calls per user per day. A basic cost dashboard, even a spreadsheet, reviewed weekly.
Not glamorous. Saves your month.
"Add billing protection to this app. Implement per-user daily API call limits, add hard caps on any loop that calls an external API, and generate a README section with step-by-step instructions to set up budget alerts on OpenAI, Anthropic, AWS, and Supabase."
Ship It Right or Ship It Twice
My friend, by the way, fixed everything. Took him a weekend. The app is genuinely good, better than most of what I see shipped by teams with actual budgets. That's the thing about vibe-coded apps done right: the idea and the execution can be exceptional. The infrastructure layer is what kills them. And that part is fixable.
The industry will keep calling these demos "products" for another eighteen months. The screenshots look the same. The launch posts sound identical.
The builders who actually ship check this list before they share the link. Not after the breach. Not after the $14k bill. Before.
A demo impresses. A product survives.
If this saved you from a bad Friday night, there's more where that came from. I write about building real things with AI, the parts the tutorials skip.
(*) The cover is AI-generated. Prompted, not painted, but the horror stories in the article are all too real.
This article may contain affiliate links, and I may earn a small commission.
Learned the hard way that a demo isn't a product? This week's newsletter breaks down the production patterns that separate hobby projects from real tools.