Sushanth Raman walked onto the stage with that statistic and refused to let anyone look away from it. The founder and CEO of Pallet — a San Francisco company that builds an AI workforce for the unglamorous machinery of global logistics — opened not with a product, but with an autopsy. "Despite the two and a half trillion that's gone into AI investments," he told the room, "95% of enterprise AI pilots have not been successful."
It is the kind of line that should empty a conference hall. Instead, Raman used it as a hinge. Because his argument is not that AI doesn't work. It is the opposite, and far more uncomfortable: the technology works fine, and the buyers are the problem.
"It's not because frontier models are not delivering results," he said. "It's really because the enterprises haven't thought through the right ways on how to go identify how to deploy this technology, how to deal with change management." The models, in other words, are not the bottleneck. The org chart is.
01The Tyranny of the Slick Demo
Every failed deployment, Raman argues, begins with the same seduction. You wander a conference floor. A voice agent holds a flawless conversation. A document parser swallows a stack of invoices without blinking. The demo is gorgeous. You sign. And then nothing works.
"There's a huge difference between what you have in a demo to what it takes to go and productionize these deployments," he said. "And that's where most people are falling short." Real deployments are not clean. They plug into on-prem systems that were never designed to be spoken to. They run on tribal knowledge that lives in the heads of three veteran dispatchers and was never written down. They demand change management nobody budgeted for.
The demo is a magic trick performed in a vacuum. Production is the same trick performed in a hurricane, blindfolded, while the audience throws edge cases.
"There's a huge difference between what you have in a demo to what it takes to productionize these deployments. That's where most people are falling short."
02Why "We'll Just Build It Ourselves" Is a Trap
Every CIO eventually asks the obvious question: why pay a vendor when we have engineers? Raman's answer is patient and a little brutal. First, the models are non-deterministic. "When you ask ChatGPT the same question 10 times, it oftentimes does not give the same response," he noted. Charming in a chatbot; catastrophic in a customs filing. "90% it's not good enough. You need it to be consistent and accurate." Getting there can swallow six to twelve months.
Second, there is no finish line. Your in-house tool may sing on day one, but six months later you've onboarded five new customers, each with their own operating procedures, and your engineers are now full-time babysitters instead of builders. The maintenance bill nobody forecasted comes due every quarter.
Third — and this is the cruelest one — the ground keeps moving. Every time OpenAI, Anthropic, Google, or an open-source lab ships a new model, the entire solution must be re-tested. Does the new model improve accuracy? Does it cause regressions? "On top of running your core business, you're left with a lot of time just maintaining." Build-it-yourself, Raman suggests, is less a project than a treadmill that speeds up on its own.
"When you ask ChatGPT the same question 10 times, it oftentimes does not give the same response. In supply chain, 90% is not good enough."
03How to Tell the Real From the Hype
So you're back where you started: a floor full of vendors, all armed with beautiful demos. "How do you tell who's full of it," Raman asked, "and who's actually delivering ROI?" His framework has three legs, and he insists you need all three.
One — pick a brutal metric. Decide, before any pilot, exactly how you will judge success: percentage of touchless shipments, number of customs orders filed, percentage of support tickets deflected. Then put three vendors through the same gauntlet against the same number. "When you do that, you're forming an objective grounding... you're not biasing yourself to a demo."
Two — demand embedding, not shipping. Off-the-shelf is a red flag. You want a partner who flies to your site, sits with your operators, and grinds through the messy integration work like a McKinsey or Cognizant engagement. "If someone's sitting there and telling you that they'll get you to a deployment in a day," he warned, "probably not going to work."
Three — codify the tribal knowledge. Your humans succeed because they carry context the AI doesn't have. The right vendor has a deliberate strategy for capturing that knowledge and documenting the SOPs that live in people's heads. Skip any of the three, Raman said flatly, and "good luck."