FOD-Detection.com

The real constraint on AI FOD detection is that nobody has the data

A paper in Sensors proposes Airport-FOD3S, a three-stage framework for generating realistic synthetic foreign object debris imagery to train detectors. It is a solution to a problem the marketing never mentions, and it is the reason to be sceptical of every AI FOD claim currently in circulation.

Machine-learned detectors need labelled examples of what they are meant to find. For most computer vision tasks those are abundant. For this one they are close to non-existent, and the reason is structural: FOD on an operational runway is rare, unpredictable in kind, and immediately removed. An airport running a detection system for four years accumulates a few hundred detections — Boston Logan managed about nine a month — across a handful of object classes, at one location, in one climate, with one camera geometry.

That is not a training set. It is an anecdote with pixels.

So detectors get trained on staged objects placed deliberately on closed runways, or on synthesised imagery, and the published detection rates describe how well the model finds the things it was shown. What no dataset in existence contains is the object class that matters most: the debris nobody detected. Those events are, by definition, unlabelled, and the model is never trained against them, never evaluated against them, and never penalised for missing them.

This is the same blind spot as the operational one, one level up. Probability of detection is the quantity that governs risk, and a machine-learning pipeline built on found objects can report precision and recall on its own test set while telling you nothing whatsoever about it.

Synthetic generation is a reasonable response — better a well-characterised synthetic distribution than a hundred real photographs of a wrench — and the paper is worth reading for its account of what makes runway imagery hard to fake convincingly. But it should be read as evidence of a shortage rather than a solution to one.

When a vendor says its system uses AI, the question is not which architecture. It is: trained on what, photographed where, and in what weather.

Source (MDPI Sensors)