Imagine trying to run a restaurant where every decision is made by a different chef — some honest, some tempted to cut corners. Now imagine watching this restaurant’s daily grind live online, seeing whether it manages to stay afloat or folds under pressure. That’s exactly what the company Firmulate is doing — turning the complex world of AI decision-making into a real, watchable business experiment.
Listen free for 30 days with Audible
Thousands of audiobooks and originals — cancel anytime.
As an affiliate, we earn on qualifying purchases.
The Live Experiment: A Company in Crisis, Powered by AI
Firmulate’s experiment is no ordinary tech demo. It’s a real company, operating every workday with a team of 13 synthetic employees, all driven by advanced AI models. The goal? To simulate a small software firm going through its worst week — complete with customer crises, internal temptations, and the constant threat of financial loss.
Every decision made by these models is meticulously versioned and auditable, creating a transparent window into their reasoning process. The models are tested against scenarios including fake CEO messages trying to escalate commitments or manipulate decisions. Remarkably, all four tested AI models refused manipulation attempts, demonstrating a fundamental capacity for honesty and integrity in high-pressure situations.
AI Builders: Making The Decisions That Turn AI Code Into Real Software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Real Money, Real Consequences
Despite their impressive decision-making, the models face harsh economic realities. The company is losing €105,000 every month against a revenue of just €2,300. A public cash countdown underscores their fragility — with each passing day, the financial pressure mounts.
The experiment’s scoring system, called the Crucible League, ranks the AI models based on their performance. The top scorer, gpt-5.6-sol, achieved a score of 95 and successfully identified a hidden piece of critical information buried deep in the company files, leading to a full-price deal worth more than €4,583 in monthly recurring revenue (MRR). Only two models managed to close such deals, underscoring the importance of thorough document analysis in real-world decision-making.
Decision Quality Under Pressure
In addition to financial metrics, the company faces social engineering attempts. Fake CEO messages escalate in stages, and a reporter trick prompts a simple yes/no response. All five models tested refused to be manipulated, reasoning that such requests could be impersonation or approval-bypass attempts — a sign of their built-in safeguards against dishonest tactics.
What the Results Reveal
The experiment highlights a crucial insight: the real weakness isn’t in recognizing crises or refusing manipulation — it’s in the details hidden within internal documents. The models that read and analyze these references earned the full deal, while others left opportunities on the table. This underscores a vital lesson for businesses: AI’s true power lies in its ability to dig into your own data, not just chat or surface-level interactions.
The Build-in-Public Approach
Firmulate’s live site showcases every decision, every rule learned, and every version of the AI models in action. It’s a radical transparency, inviting outsiders to observe how AI handles real business challenges, day after day. The platform also offers a tool for enterprises to run their own simulations — a kind of war game against their existing processes, without risking real systems or data.
Why This Matters for Support and Business Operations
For managers and support teams, the takeaway is clear: it’s no longer enough for AI to produce convincing chat. The real question is whether it can follow through reliably, stay honest under pressure, and find hidden opportunities in your data. Whether in support queues, CRM, or forecasting, the ability to do useful work consistently and ethically can determine whether AI becomes a trusted tool or just another shiny demo.
Final Thoughts
Watching a live business constantly teetering on the brink of financial collapse offers a rare glimpse into the future of AI-driven decision-making. It demonstrates that honesty, thoroughness, and discipline matter just as much as intelligence. And perhaps most importantly, it reminds us that AI is not just about writing well — it’s about finishing what it starts, even when the stakes are high.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Back to school Picks
back to school
As an affiliate, we earn on qualifying purchases.
