TakeoverBench.

The Problem

When a self-driving car reaches its limits, it hands control back to the driver with a takeover request. Getting that moment right is a safety question, but takeover research is surprisingly hard to compare. One systematic review counted 43 different driving simulators across 49 takeover studies. Studies measure different things (reaction time in one, steering jerk in another), define a “takeover” differently (touching the wheel vs. pressing the brake), and run on very different setups (wheel and pedals vs. keyboard and mouse). On top of that, testing a new takeover alert usually means building custom software for every design.

What we built

TakeoverBench is an end-to-end platform for takeover experiments: set up the hardware, pick a scenario, press start, and get a standardised report at the end.

• The same scenarios, every time. Six presets (a familiarisation run plus five takeover conditions) fix the weather, speed, obstacle distance and time-to-collision threshold, so every participant faces identical conditions.
• A hardware fidelity check. Before a session, the simulator setup is rated on visual, motion and physical fidelity. If something is missing (say, no pedals), the platform warns you which metrics can’t be measured reliably. The setup is then locked and recorded in the report.
• Configurable alerts without new apps. An Android HMI lets researchers change the alert’s message, position, colours, typography, flashing, images and sound, save designs as profiles, and compare them under the same conditions. The active design is logged with every run.
• Metrics and reports, automatically. Telemetry is processed against the HCMTP framework, which organises 41 takeover metrics into 9 performance components, and compiled into a PDF report covering the scenario, hardware and scores.

How it works

A Flask backend sits at the centre. It drives the CARLA simulator through its Python API, holds the car at speed with a PID controller, and watches the time-to-collision. When that crosses the threshold, it hands control back to the driver and publishes the takeover event over MQTT to the Android tablet, which shows the alert and sends acknowledgements back. A browser dashboard lets the researcher run sessions, watch the live feed and telemetry, and download reports.

Outcome

We presented TakeoverBench at AutomotiveUI ‘26 in Gothenburg, and the paper is published in the conference’s adjunct proceedings. We’re now preparing the open-source release so researchers anywhere can run comparable takeover studies.

Check out the demo here!