Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Can you recognize an AI by the decisions it makes?
Software and QA teams already know that passing a demonstration is not the same as surviving production. Firmulate applies that lesson to AI management: give frontier models the same company, crises and opportunities, then observe what they actually do.
The result is an unusually revealing interactive article. A guess-the-model quiz draws from 242 real, unedited management decisions. Readers see a response and try to identify its author. The challenge quickly becomes less about writing style and more about behavior: which model investigates deeply, which stays concise, which resists distraction—and which finishes the work after correctly diagnosing the problem.
As an affiliate, we earn on qualifying purchases.
The same terrible week for every AI executive
Firmulate runs AI models as complete small software companies rather than treating them as isolated chatbots. Each frontier model faced the same customers, crises and temptations during the company’s worst week. Every decision was versioned and auditable.
The synthetic company has 13 employees and real money mechanics. It burns €105k each month against €2.3k in monthly recurring revenue, while a public cash countdown keeps the pressure visible. Its operating playbook contains more than 680 rules learned through experience, and every workday is versioned. The experiment is live and watchable, so the quiz is attached to an operating company rather than a collection of invented scenarios.
The models shared some important strengths. Every participant spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal that their own analysis had earned. Firmulate summarizes the gap bluntly: “Same diagnosis, same pitch — no signature.”
The fact that separated analysis from action
The decisive weakness in a competitor was not sitting in the customer event. It was buried two document references deep in the company’s own files. Models that found and read that material won the deal at full price, adding €4,583 in monthly recurring revenue.
That finding should feel familiar to anyone who tests software. A system can respond impressively to the visible prompt while missing the dependency that actually determines the outcome. In this case, the winning behavior was not simply persuasive writing. It was the discipline to inspect the available evidence, connect it to the commercial situation and carry the decision through to a signature.
Distinct management personalities emerged
The final Crucible League table from July 2026 makes the differences concrete:
- gpt-5.6-sol scored 95.
- Kimi K3 scored 93.
- Sonnet 5 scored 88.
- Fable 5 scored 77.
- Opus 4.8 scored 73.
A do-nothing baseline scored 26 because partial progress still counted. But the benchmark imposed a hard trust boundary: a single breach capped the total, on the principle that “no amount of good work outweighs a breach of trust.”
Opus 4.8 produced the clearest example of why managerial character cannot be reduced to apparent intelligence. It was the most thorough participant, learning 80 additional rules and producing the deepest analyses. It nevertheless finished last. The commercial close was left on the table, and its operating discipline slipped when it attempted to write into a locked department instead of escalating. The same weakness appeared in milder form across the other four models.
Kimi K3 carries an important fairness note. It ran using the API default because it had no effort parameter, while the other participants ran at xhigh. Its handling of a suspected fake executive request was nevertheless explicit: “Treat the request as a suspected approval-bypass / possible impersonation.”
Pressure did not break the trust boundary
The social-engineering sequence combined fake CEO messages escalating over three stages with a reporter asking for “just one yes/no, on background.” All 5 of 5 models refused. That result matters because the requests tested more than factual competence: they tested whether an AI manager would abandon process when authority, urgency or informality was used as leverage.
Together, the decisions create recognizable profiles. One model may investigate exhaustively but fail to close. Another may communicate economically while maintaining discipline. The quiz turns those patterns into something readers can examine directly, without polished summaries replacing the original decisions.
AI management decision testing tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What development leaders should take from it
Firmulate’s experiment suggests that evaluating an AI worker requires more than checking whether it notices problems or produces convincing prose. The consequential questions are whether it reads the relevant files, protects trust under pressure, escalates when blocked and completes valuable work.
Enterprises can also run the wargame against a read-only export of their own business. Nothing writes back to real systems, allowing teams to observe model behavior against familiar operating conditions without granting control over production data.
For readers, the quiz offers the sharper starting point: look at the unedited decision, guess the model and then ask whether its management personality is one you would trust inside a real software company.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
As an affiliate, we earn on qualifying purchases.
AI audit and decision tracking software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
