Synthetic data: when it works and when it quietly ruins your model
Generated data solves privacy and coverage problems and creates subtle new ones. Where the line sits.
Synthetic data is genuinely useful and routinely oversold. It is a tool for specific problems, not a substitute for having real data about the thing you are modelling.
Where it earns its place
Rare-event coverage, privacy-safe development environments, and bootstrapping evaluation sets before you have production traffic. In each case you are filling a gap, not replacing the distribution.
- Rare failure cases you cannot wait to observe naturally
- Development and demo environments that must not contain real personal data
- Adversarial and edge cases for red-teaming
- Bulking out an under-represented class in a training set
Synthetic data: where the line sits
Works
- Rare failure cases you can't wait to observe
- Dev and demo environments with no real personal data
- Adversarial cases for red-teaming
- Balancing an under-represented class
Doesn't
- Evaluating — always hold out real examples
- Replacing data about the thing you're modelling
- Assuming synthetic means anonymous — test for memorisation
- Training on your own model's output without measuring drift
Where it goes wrong
Generated data carries the assumptions of whatever generated it. Train on model-generated examples and you learn that model's blind spots, sharpened. The failure is invisible in testing if your test set is synthetic too, which is how teams end up with excellent metrics and a product that disappoints.
Two rules that prevent most of the damage
Never evaluate on synthetic data — hold out real examples for that, however few. And keep the provenance of every record, so you can measure performance separately on real and synthetic subsets and notice when they diverge.
Two rules that prevent most of the damage
- Evaluating the modelUse real held-out data, however little
- Mixing real and syntheticKeep provenance per record
- Metrics diverge between the twoYou have found the blind spot
- Using synthesis for privacyTest for memorisation, don't assume
On privacy
Synthetic does not automatically mean anonymous. Generators trained on personal data can reproduce it, especially outliers, and an outlier is exactly the record most likely to identify someone. If you are using synthesis for privacy, test for memorisation rather than assuming it.