Backtracking on AI: Real Stories From Leaders
AI Lessons: Many leaders overlook the valuable lessons from failed or paused AI projects, highlighting crucial insights.
Pause Over Push: Companies often benefit from pausing AI initiatives to establish foundational processes and data quality.
Manual First: Automation should only come after manual execution, ensuring processes are clearly defined and understood.
Measure Impact: Key to success is assessing current processes and potential AI impact before implementation begins.
Documentation Importance: Effective documentation should capture not only processes but also the reasoning behind decisions and exceptions.
Most AI adoption stories are told from the winner's podium. Far fewer leaders talk about the moment they pulled the plug — the automation that got cancelled, the rollout that got paused, the pilot that had to be rebuilt from scratch. But those stories are often where the real lessons live.
We asked three experts who have been through it: what got undone, why, and what would you do differently? Their answers converge on a surprisingly consistent theme — the problem is almost never the AI itself.
When the Smart Move Is to Pause, Not Push Forward
For most organizations, ripping out an AI investment entirely isn't realistic once the money has been spent. Tejinder, Senior Manager at Deloitte Consulting, says outright removal wasn't on the table given the investments already made — but a deliberate pause was. When the results weren’t up to par his team stepped back and “looked at the basics first,” asking whether the underlying processes were established, whether “the data quality” was “good,” and whether planners understood the role of agents versus their own. “I think once those questions were answered the reactivation made sense,” he says.
Create a Free Account to Read More
Unlock this piece and join a community of forward-thinking leaders discovering tools, playbooks, and insights for thriving in the age of AI.
Have an account? Log In
Priyanka Dave, PhD, Upskilling Lead and Learning Process Owner at Oregon State University, has seen the same pattern play out repeatedly. “I have observed several organizations intentionally delay expanding AI initiatives after recognizing that they were not operationally ready.” In those cases, the retreat wasn't a failure — it was a redirection. “Rather than expanding AI immediately, leadership shifted its focus toward strengthening the organizational foundation,” Dave explains. This meant “organizations invested in documenting business processes, standardizing workflows, improving data governance, and building employee capabilities through structured learning and change management.”
Rather than expanding AI immediately, leadership shifted its focus toward strengthening the organizational foundation.
Don't Automate What You Haven't Done Manually
Sometimes backtracking means cancelling a project outright — and discovering that was the right call all along. Jaime Gil Martinez, Founder and Principal Consultant at Swift Ops Solutions, learned this the hard way. “I once built an automation to produce marketing materials for a client who had never run the process manually,” he recalls. “Because it was new to them, we both figured it was better to build it automated from scratch. On paper it made sense. In hindsight it was a rookie mistake.”
The system technically functioned. “The automation worked,” Gil Martinez says. “The problem was that every time an output was needed, something had to be reconsidered or tweaked (a different target, a different format, a different layout), so I kept rebuilding it. After a few rounds of this the pattern was obvious – I was automating a process that nobody had defined yet.”
After a few rounds [of tweaking] it was obvious I was automating a process nobody had defined yet.
The solution was to go backwards. “So we cancelled the project outright and went back to producing the materials by hand,” he says. “That turned out to be the point: each manual round gave us what we'd been missing, namely gold-standard examples and a clear view of what varied and what didn't. We resumed the automation later, and that manual output became the knowledge base we handed to AI.” The experience produced a rule he now applies across his own company and his clients': “don't automate anything until you've done it manually at least three times. Then you can start considering it.”
The Step Everyone Skips: Measure Before You Build
Ask Gil Martinez what most AI adoption plans are missing, and his answer is immediate: “Measuring before you build. Most companies are missing two numbers at the point they start: how good or bad their current process actually is, and what impact AI could realistically make on it. Without both, you can't tell how effective your implementation was or whether the cost was justified.”
He points to a healthcare clinic that wanted to automate scheduling for more than 100 patients. “They didn't know how long it was taking them, only that it was time-consuming and frustrating,” he says. “And if you don't know how long it takes, you don't know how much it costs you.” So before building anything, his first step was calculating the hours the scheduling consumed and the financial value of those hours. “That gave us the baseline: two weeks' worth of full manual work. The automation brought it down to one day of AI-assisted work, at a cost lower than their current equivalent spend.” With those numbers in hand, “the decision made itself.”
Dave sees a different but equally critical omission: readiness isn't just about the technology. “The step I see organizations overlook most frequently is assessing workforce capability before implementing AI,” she says. And the gaps rarely turn out to be about tool training. Across several capability assessments, “the assessments consistently revealed that larger capability gaps involved analytical thinking, critical thinking, data literacy, consultative communication, and problem-solving.” Her bottom line: “Technology readiness and workforce readiness are different challenges. Sustainable AI adoption requires both.”
When AI Lands on a Process That Isn't Ready
What actually breaks when AI is layered onto an immature process? Tejinder saw it firsthand when “AI was embedded into a business process (order processing and allocations) which was not mature enough” — the result was a solution riddled with bugs. But he's clear about where the fault lay: “I believe AI was not the problem here,” he says, pointing instead to process maturity, master data issues, unclosed requirement gaps, and inadequate testing.
AI was embedded into a business process (order processing and allocations) which was not mature enough.
Dave describes the same dynamic from her assessments: different departments completing the same work differently, incomplete documentation, and exceptions living only in employees' heads. “The concern was that AI would simply reproduce inconsistent practices across the organization,” she says. It's a consequence of a principle she returns to often: “AI does not solve organizational problems. It amplifies existing systems. If processes are inconsistent or poorly documented, AI will often scale those inconsistencies rather than improve them.” The takeaway, in her words: “The recurring lesson was that organizations should improve processes before attempting to automate them.”
If processes are inconsistent or poorly documented, AI will often scale those inconsistencies rather than improve them.
Gil Martinez has his own scar tissue here — a lead generation application that “failed dramatically the first time.” The build jumped from semi-manual to fully automated without defined requirements or tests. “My mistake was assuming AI would figure out some of the logic on its own, because the logic seemed obvious and simple,” he admits. Things spiraled from there. “The code became unmanageable and entirely owned by the AI, to the point where I couldn't revert to a safe milestone. That was the moment I knew it was structurally broken, not just buggy.”
The recovery required a full reset. “The fix was as painful as it was obvious: start again,” he says. “This time I outlined every requirement and a test against each one, built incrementally, and gave AI clear instructions for logging failures and successes alongside proper version control.” The disciplined second attempt “took roughly half the time I'd wasted on the first attempt to reach a workable, production-grade solution.”
What Good Documentation for AI Actually Looks Like
Underneath all three stories sits the same fix: documentation that captures how work actually happens. For Dave, that means going beyond task lists. “Good process documentation explains not only what people do but also why they make particular decisions,” she says. And the most valuable material is usually what never gets written down. “One of the most common mistakes organizations make is documenting only the standard workflow. The greatest organizational knowledge often exists in the exceptions, judgment calls, and contextual decisions that experienced employees make.”
Gil Martinez frames it through a lens PMs will increasingly recognize: “Good process documentation mirrors prompt engineering. The same elements we learned to give AI in a prompt (role, context, task, instructions, constraints) should show up at project level.” That shift has implications for how practitioners spend their time. “Execution time for humans is disappearing, offloaded to AI, so our focus has to shift to planning well and validating outputs. We need to behave like AI managers.”
Conclusion: Sequence Is the Strategy
None of these leaders regrets investing in AI. What they regret — or watched others regret — is the order of operations. Tejinder sums it up: “Enterprise AI adoption fails when the model is deployed ahead of the operating fundamentals. The right sequence is process maturity, data quality, and role clarity first. AI second. When organizations reverse that order, they blame the AI, but the underlying issue is almost always the environment the AI was dropped into.” Backtracking, it turns out, isn't the failure. Deploying before you're ready is.
Want more insights like these? Sign up for a free DPM account to hear from more experts like these.
