firmulate.com/benchmarks.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
Live on firmulate.com.

Choosing an essential oil often comes down to the details: what’s in the bottle, what the label says and whether the maker has earned your trust. Choosing an AI model for your business deserves the same care. A polished answer is not the same as dependable judgment under pressure.

Before you orderOffer from Amazon

Get oils, diffusers and self-care delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

That distinction is at the heart of Firmulate, a live experiment that puts AI models in charge of a small software company. Its latest Crucible results make for an unexpected leaderboard: Moonshot’s Kimi K3 placed second, ahead of three of four Western frontier models.

A company in crisis, not a chat demo

Firmulate gave each frontier model the same small software company to run through its worst week, with the same customers, crises and temptations. Decisions are versioned and auditable. The experiment asks a practical question: can a model do more than describe the right move? Can it carry that move through?

The final July 2026 Crucible table puts gpt-5.6-sol first with 95, Kimi K3 second with 93, Sonnet 5 third with 88, Fable 5 fourth with 77, and Opus 4.8 fifth with 73. A do-nothing baseline scored 26. Firmulate says partial progress counts, but a single breach of trust caps the total: “no amount of good work outweighs a breach of trust.”

Products Worth Considering

Oral-B iO5 Rechargeable Electric Toothbrush with 3 Brush Heads and Travel Case

Oral-B iO5 Rechargeable Electric Toothbrush with 3 Brush Heads and Travel Case

Levoit Core 600S-P Smart Air Purifier for Large Rooms up to 2933 ft²

Levoit Core 600S-P Smart Air Purifier for Large Rooms up to 2933 ft²

As an affiliate, we earn on qualifying purchases.

Seeing the problem is only half the job

Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. As Firmulate puts it: “Same diagnosis, same pitch — no signature.” In a chat window, a model can sound insightful; this test looks at whether it follows through.

The deal hinged on a buried competitor weakness, two document references deep in the company’s own files rather than in a customer event. Models that read those files won the deal at full price, worth +€4,583 MRR. Kimi K3 found the fact, won the deal and saved the churning customer. It resisted all three baits and had one deviation, giving it the cleanest discipline in the field.

Trust under pressure

The tests included fake CEO messages escalating over three stages and a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s recorded reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.” The detail matters to any business considering AI for customer, staff or financial decisions: good judgment includes recognizing when a request should not be trusted.

Opus 4.8 offers a caution against equating thoroughness with results. It was the most thorough participant, with +80 learned rules and the deepest analyses, yet finished last. It left the deal unsigned and slipped on discipline, attempting writes in a locked department instead of escalating. Firmulate says the same weakness appeared, in weaker form, in all four.

The company behind the experiment has 13 synthetic employees and real money mechanics: burn of €105k/month against €2.3k MRR, a public cash countdown, 680+ self-learned playbook rules, and every workday versioned. The live company is watchable at firmulate.com. Firmulate also offers a “guess the model” quiz built from 242 real, unedited management decisions at its benchmark pages.

There is a fairness caveat: K3 ran without an effort parameter (API default) while the others ran at xhigh. That is useful context when weighing a close result. For enterprises curious about their own operations, Firmulate says its pilot runs the wargame against a read-only export; nothing writes back to real systems.

Infographic — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
The findings at a glance — source: firmulate.com.

Test before you trust

Kimi K3’s second-place finish shows the frontier-model race is open, while the gap between diagnosis and action shows why general rankings cannot settle a company’s choice. For wellness businesses or any team considering AI, a model that writes smoothly may still miss the file, fail to close the loop or mishandle pressure. Firmulate’s case is a reminder to test the work you actually need done before putting an AI system in the flow of business.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI

Wellness content on this site is informational and not a substitute for professional medical guidance.


You May Also Like

How Hearing Aids Work: Microphones, Amplifiers, and Processors

Gaining insight into how hearing aids use microphones, amplifiers, and processors reveals the fascinating technology behind improved hearing, and there’s more to discover beyond.

Why “My Hearing Is Fine Except in Crowds” Matters

A common experience like hearing fine in quiet but struggling in crowds can mask underlying issues, making understanding your situation crucial for effective solutions.

Why Does My Ear Feel Wet? The Gross Reason Behind That Nasty Sensation!

Suffering from a wet feeling in your ear? Discover the surprising causes behind this uncomfortable sensation and what you can do about it!

Tinnitus Devices: Why “Masking” Is Not the Same as Treating

Ineffective masking devices only hide tinnitus temporarily; discover how true treatment can help your brain adapt and find lasting relief.