80% of ML Projects Fail - How to Let Them
Friday, April 17, 2026 · 1:10 PM – 1:40 PM · Auditorium 5
Most organizations treat ML project failure as something to avoid. In practice, the real problem is that projects fail slowly - after months of investment, unclear progress, and data scientists feeling guilty for things beyond their control.
Many failure modes are predictable and discoverable early: weak signal, unstable data, poorly defined objectives, or integration constraints. Unfortunately, most ML initiatives are not designed to detect these risks until late in development.
This talk reframes ML development through the lens of experimental design. Instead of trying to prevent failure, teams should design projects to fail fast, cheaply, and conclusively - before production commitments are made.
We will cover:
- Why ML project failure mirrors the scientific replication crisis (metric fishing, test-set reuse, and selective reporting)
- Applying Design of Experiments (DoE) thinking to ML project planning
- Defining success criteria and kill conditions before training the first model
- Staged investment frameworks: the minimum experiment needed to justify continued development
- A simulation demonstrating how easily apparent model improvements can arise from noise alone -Reframing failed projects into successful learning
The goal is credible successes and inexpensive failures that accelerate learning and improve organizational trust in ML outcomes.
Attendees will leave with a practical framework for structuring ML initiatives that produce decisive answers instead of prolonged tragedies.
Audience: data scientists, ML engineers, and technical leaders responsible for delivering applied ML outcomes.
About the Speaker
Aaron Baker
CapTech Data Science Manager
Aaron Baker is a Data Science Subject Matter Expert at CapTech Ventures, where he helps organizations design and implement advanced analytics and AI solutions. He brings nearly a decade of consulting experience alongside extensive work in data science education across analytics, statistics, and applied AI. He has leads multiple internal training initiatives and technical application showcases, and is an adjunct statistics + applied ML professor at VCU for their Masters in Decision Analytics program. Aaron has served as Chair of the RVATech Data & AI Summit since 2023 and is a speaker for industry events, university programs, and professional AI communities. His work focuses on helping practitioners and leaders translate emerging AI capabilities into practical, responsible outcomes with statistical rigor. A statistician by training and a student of philosophy at heart, Aaron is particularly interested in the evolving relationship between artificial intelligence and human judgment—and how technological progress reshapes the way we work, decide, and create.