The Data You Think You Have

Why machine learning projects fail before the modeling even starts

When you build machine learning systems for industrial applications, you learn quickly where projects actually fail. It's rarely the model architecture, rarely the deployment infrastructure, rarely the hardware. It's the data, or more precisely, the gap between what companies think they have and what's actually useful for training.

Sensors producing signals that get saved somewhere feels like data collection but isn't. A system logging temperature and pressure curves, taking images, or collecting process parameters around the clock produces records, not training data, and the difference between the two is entirely human effort: someone looking at each sample and deciding what it represents, encoding that judgment in a label the model can learn from. Without that effort, you're just paying for storage.

Even when annotation exists, the problems don't end there, because the quality of labels matters as much as their presence. If two people examine the same sample and reach different conclusions about what happened, you've encoded disagreement into your training set rather than ground truth. Vague categories make this worse: "normal" versus "abnormal" versus "uncertain" means different things to different people, and the model learns that confusion rather than resolving it. The annotation framework needs to be precise enough that reasonable people converge on the same answer, which requires more upfront thought than most projects receive.

And then there's the counting problem that kills projects even when the annotation is clean. What matters isn't total samples but samples per class, and most processes generate overwhelmingly routine output. A hundred thousand examples of normal operation and fifty examples of the failure mode you actually care about predicting. That's not a dataset ready for training; it's a class imbalance problem that will produce a model very good at predicting normalcy and useless for the thing you wanted it to catch.

Unsupervised learning seems like it solves this. No labels needed; the model finds structure on its own. Anomaly detection, clustering, dimensionality reduction. Point it at your unlabeled data and let it discover patterns. But this trades one problem for another. An anomaly detection system will tell you something is unusual, not whether it matters. A clustering algorithm will group your data, but the groups still need interpretation. You've moved the human judgment from the front of the pipeline to the back, and you still need domain expertise to make sense of what the model produces. The work doesn't disappear; it shifts.

This is where machine learning projects actually die: not in the modeling phase, not in deployment complexity, but in the gap between "we have data" and having something genuinely useful.

There's an irony waiting in the future here. Someday we may have methods that label data without a human in the loop, and all those terabytes of stored records will suddenly become useful. But if a system can look at a sample and reliably determine what it represents, you have to ask: why would you need the labels at that point?