Tech Tips
Why your 99%-accurate model might be useless: three pitfalls to avoid
A model can score 99% in testing and still fail in the real world. Learn the three pitfalls behind most disappointments (the accuracy trap, data leakage and overfitting) and the questions that expose them before you trust the numbers.
Imagine signing off a project because the model behind it scored 99% in testing, then watching it miss the very cases that cost you money. It happens more often than most teams admit, and it rarely comes down to the algorithm. Three of the most common reasons are about how a model is measured and what data it is allowed to see.
With AutoML in Fabric notebooks, building a model is quicker than ever. Deciding whether to trust one still takes judgement. This post covers three pitfalls we see again and again: the accuracy trap, data leakage and overfitting. Each makes a model look better than it is, and each can be caught by asking the right question early. If you are weighing up a machine learning project, our Data Science & Machine Learning consultancy team can help you ask those questions at the start.
What is the accuracy trap in machine learning?
Picture a lifeguard who declares every swimmer safe. On a quiet beach where almost nobody gets into difficulty, they are right 99.9% of the time, and they have never saved anyone. Accuracy is the most natural way to describe a model, and for rare events it is the most misleading.
Here is the same idea with real numbers. A fraud model scores 100,000 card transactions, where 1 in 1,000 is fraudulent. A "model" that labels every transaction as genuine is right 99,900 times out of 100,000. That is 99.9% accuracy, and it catches no fraud at all.
Now take a genuinely good model. It catches 99% of fraud and wrongly flags just 1% of genuine transactions:
Flagged as fraud | Not flagged | |
Actually fraud (100) | 99 caught | 1 missed |
Actually genuine (99,900) | 999 false alarms | 98,901 cleared |
The model raises 1,098 alerts, and only 99 of them are real. Just 9% of its alerts are fraud, even though the model is 99% accurate.
Why does a 99% accurate model raise so many false alarms?
The reason is the base rate. Genuine transactions outnumber fraud by roughly a thousand to one, so even a tiny false alarm rate produces about ten times more false alarms than real catches. The same arithmetic applies to anything rare: customer churn, equipment failure, security incidents and safety events.
It is also a business problem. Every false alarm is an analyst's time, a customer who may be wrongly blocked, or a retention offer sent to someone who was never leaving. Here, a team would work through 1,098 alerts to find 99 frauds.
Precision and recall explained
Think of fishing with a net. Widen the net and you catch more of the fish you want, but you also haul up more old boots. Two numbers describe that trade-off far more honestly than accuracy:
- Precision: when the model raises an alert, how often is it right? Here, 9%.
- Recall: of all the real cases, how many did it catch? Here, 99%.
These pull against each other. Where to strike the balance depends on what each mistake costs your business, which makes it a business decision as much as a statistical one.
What is data leakage and how do you spot it?
Imagine marking your own exam with the answer sheet open beside you. You would score full marks, and the result would tell you nothing about what you know. Data leakage is the same thing for a model: it is trained on information it would not have at the moment of a real prediction. It looks superb in testing, then fails in production.
Which warehouse columns leak?
Leakage is a common risk when data comes from a warehouse, because many tables hold the current state of things rather than their history. Here is how the candidate inputs for a churn model ("will this customer cancel in the next three months?") fare against one test: would we know this value at the moment we make the prediction?
Warehouse column | Safe to use? | Reason |
tenure_months | Yes | Fixed before the outcome happens, so it cannot give the answer away |
support_tickets_90d | Yes, if counted only up to the prediction date | Counted up to today, it would include tickets raised while cancelling, such as "how do I cancel?" |
last_login_date | Check first | Usually overwritten with the latest login. For a customer who left, that is roughly the day they left |
refund_issued_flag | Check first | Depends on when it is set. If refunds are issued as part of cancelling, it gives the answer away |
account_status | No | A status of "Closed" means the customer has left, so it is the outcome itself |
cancellation_reason | No | Only filled in after a customer leaves, so it is never available in time |
There is a subtler form too. If you fill in missing values or rescale columns using the whole dataset before splitting it into training and test data, the test data has already influenced training. Always split first, then fit any preparation steps on the training data alone.
How to prevent data leakage
For every column, ask one question: would we know this value at the moment we make the prediction? The most reliable fix is to build training data as snapshots, calculating each feature using only data available up to the prediction date. Treat any result that looks too good to be true with suspicion. If the model's most important input feels like cheating, it probably is.
This is where BI and data warehouse teams add real value. Knowing how and when each column is populated is exactly the knowledge needed to catch leakage, and no algorithm has it. It is also why a well-governed single source of truth makes machine learning safer to build.
What is overfitting and how do you prevent it?
Think of a student who memorises every past exam paper word for word. They score 100% on questions they have seen and freeze when the wording changes. A model can do the same. Suppose a churn model is 100% accurate on the 2,000 customers it was trained on, yet right only 61% of the time on next month's customers. It has memorised its training data, noise and all, instead of learning patterns that carry over to new customers.
This is overfitting, and flexible models such as decision trees can do it easily: given enough freedom, a tree can give every training customer their own branch. The tell-tale sign is the gap between two scores. Error on the training data keeps falling as a model gets more complex. Error on data the model has not seen falls at first, then rises sharply once the model starts fitting noise.
Illustrative example: polynomial models of increasing complexity fitted to 20 noisy data points.
Past the sweet spot, every extra bit of flexibility makes the model better at the data it has already seen and worse at everything else.
How to spot and prevent overfitting
Never judge a model on the data it learned from. Hold back data the model does not see during training and compare the two scores: a large gap means overfitting. A few practical habits help:
- Split by time for anything recorded over time. Train on the past and test on the most recent period, or the model learns from the future and flatters itself.
- Use cross-validation when data is limited, so a single lucky split does not decide which model wins.
- Keep a final test set untouched until the very end, and report it once.
- Prefer the simpler model when two perform about the same. Less flexibility leaves less room to memorise.
Tracking overfitting with MLflow in Fabric
In Microsoft Fabric, every training attempt can be logged as an MLflow run. Log both training and validation scores on each run and overfitting becomes visible at a glance in the experiment's run comparison view.
Six questions to ask before you trust any model
Whether a result comes from a data scientist, a vendor or AutoML, these questions work as a checklist:
- What was it scored on? Training scores say almost nothing. Ask for performance on unseen data.
- How common is the outcome? For rare events, ask for precision and recall, not accuracy.
- Would we know every input at prediction time? If not, the model is cheating.
- How was the data split? For time-based data, the test period should come after the training period.
- How big is the gap between training and test scores? A large gap means overfitting.
- Does it beat a simple rule? You would not use a satnav to find the end of your own street. If a SQL condition does nearly as well, use the rule.
How PTR helps you build machine learning you can trust
None of these pitfalls is fixed by a better algorithm alone. They are caught by people who ask how a score was measured and what data went into it. At PTR we blend consultancy and training so your own team learns to ask those questions as well as having them answered.
- Consultancy: our Data Science & Machine Learning and AI Readiness & Data Foundations services help you build predictive models on solid foundations in Microsoft Fabric, backed by Data Engineering & Data Integration.
- Training: our Data Science courses and Microsoft Fabric courses give BI and data teams the skills behind sound model design, with options from public courses to bespoke one-to-one and tailored training.
Talk to our team to find out how we can help.
Related reading
The figures in this post are illustrative examples.
SB
Stephanie Butchart
Senior Business Intelligence Consultant
An MCSA-certified BI Consultant with 12 years’ experience, Stephanie is proficient and knowledgeable across Microsoft technologies, spanning traditional on-premise data warehousing through to Azure BI solutions, Power BI and Microsoft Fabric. With a strong focus on data warehousing and data engineering, she also brings growing expertise in data science, machine learning and the use of AI to support coding and solution development. She also holds the Microsoft Certified: Azure Administrator Associate certification.
Frequently Asked Questions
Couldn’t find the answer you were looking for? Feel free to reach out to us! Our team of experts is here to help.
Contact Us