UNB researchers model how attackers could time manipulation of human-supervised drones

Two researchers at the University of New Brunswick have built a model of how an attacker could time the failure of a drone’s camera-based detector so that a remote human supervisor no longer has room to step in.

The paper, by Samuel Ansong and Ali Ghorbani of UNB’s Canadian Institute for Cybersecurity (CIC) in Fredericton, was published Oct. 29 in the open-access journal Drones. Ansong is a PhD student supervised by Ghorbani, according to the CIC’s researchers page. Ghorbani is the institute’s director and holds a Tier 1 Canada Research Chair in Cybersecurity.

TechNB News found no UNB news release about the paper. This account is based on the paper itself.

Timing matters more than hit rate

Studies of attacks on machine-vision systems usually score them frame by frame, using an “adversarial success rate,” meaning the share of video frames in which the detector is fooled. Ansong and Ghorbani argue that this measure misses the point for a drone flown by onboard software under remote human supervision, as delivery and inspection drones often are.

In their view, a manipulation is dangerous when neither the remote human nor the drone’s own automatic response can finish an avoidance manoeuvre before the drone passes its “point of no return.” The attack they model leaves the detector’s confidence in a hazard looking normal through most of the approach and then lets it collapse at a chosen distance.

For a remote operator, the time to react includes the video downlink, the operator’s decision and the command uplink. The authors say that extra delay is part of what an attacker can exploit. In their conclusion they describe a well-timed perception failure as “a timing-based exploit of the remote-supervision loop.”

The threat model assumes the attacker can place a static marking on the hazard before the mission. It assumes no access to the drone, its autopilot or its communications while it is flying.

What the simulations showed

The researchers flew the scenario in software-in-the-loop simulation using the open-source PX4 flight-control software and a simulated quadrotor. The drone headed toward a virtual hazard 90 metres away, and the detector’s confidence collapsed 30 metres out. The paper treats the human operator as a set of reaction delays from 0.5 to 3.0 seconds. It did not test real people.

Across 160 simulated flights, the 30-metre collapse defeated the modelled remote-human takeover above roughly 9 to 12 metres per second, the paper reports. An onboard automatic brake still recovered at speeds into the mid-teens. In one pair of 12 m/s flights, the onboard response stopped with 12.9 metres to spare. The modelled human path reached the 2-metre clearance limit still moving. At about 18 m/s and above, even the onboard response failed with the 30-metre gate.

The authors also found that the per-frame success rate can rank attacks in the opposite order to the danger they pose. They compared three suppression profiles. The late-collapse profile scored the lowest success rate (0.33, against 0.66 and 0.94) but was the only one that defeated both the human and onboard responses.

Calibration flights showed that braking was limited by the autopilot’s tilt limit, at a peak deceleration of about 9.5 m/s², and by an effective control lag of about 0.8 seconds. The authors’ idealized model had left that lag out. A stopping model fitted only to the calibration flights predicted 157 of the 160 outcomes. Fifty repeat flights near the recovery boundaries gave the same results as the originals.

To show that such a profile could be produced, the researchers optimized a static digital patch on images of cars from the VisDrone aerial dataset. On held-out cars, the patch kept detection near normal at medium range and then cut confidence from about 0.90 to 0.02 up close. The effect carried over to a second YOLO detector. The timing did not carry over to a transformer-based detector, and the roughly 33-metre collapse range depends on the camera the authors assumed.

Limits the authors acknowledge

The paper says plainly what the study does not show. Every manipulation was either injected into the simulation or applied digitally to existing images. None was printed, physically deployed or tested on a real drone in flight. The simulated flights had no wind, a stationary hazard and a detector without noise. The authors say each of those omissions makes recovery look easier than it would be in the field. They describe the reported speeds as “optimistic upper bounds.”

The paper’s dual-use statement acknowledges that the same analysis could help attackers as well as defenders. It frames the work as safety evaluation for civilian drones.

What they recommend

The authors conclude that asking a remote human to take over after a perception failure is not enough across much of the range they tested. They argue for an onboard “fail-closed” response, meaning the drone brakes, holds or climbs when detector confidence collapses. They also call for maximum operating speeds based on a drone’s measured stopping distance, detection range and remaining safety margin, rather than on its nominal agility.

The paper reports no external funding. The CIC provided computing and research support. The authors say their code is available on request.