The seemingly ubiquitous advanced driver assistance systems (ADAS) that are increasingly integrated into modern vehicles, mandated by regulations like the EU’s General Safety Regulation 2 (GSR2), are proving to be a source of considerable frustration for drivers in real-world conditions. Despite their intended safety benefits, features such as automatic emergency braking (AEB), intelligent speed assistance (ISA), and lane-keeping assist (LKA) are often perceived as intrusive, overly sensitive, or simply malfunctioning, leading to driver annoyance and, potentially, a reduced reliance on these technologies. However, Thatcham Research, a leading independent automotive safety and insurance research body, is actively investigating these systems’ real-world performance, aiming to bridge the gap between laboratory testing and practical application.
Thatcham Research’s findings, based on an extensive event-based testing program, highlight significant discrepancies between how ADAS perform under regulated test conditions and their behaviour on public roads. This initiative, launched in September of the previous year, seeks to generate objective data that can inform manufacturers, insurers, and regulators about the true usability and effectiveness of these systems.
The Disconnect Between Lab and Road: Early Findings
The core of Thatcham’s investigation lies in observing ADAS performance in dynamic, unpredictable environments. Yousif Al-Ani, Principal ADAS Engineer at Thatcham Research, shared insights from their testing facility at Retford Gamston Airport, a site equipped for rigorous vehicle safety and technology evaluations. In a controlled demonstration, the automatic emergency braking (AEB) system of a Tesla Model Y, travelling at 12 kph towards a crash test mock-up of a compact SUV, activated precisely as intended, bringing the vehicle to a halt. However, when the speed was reduced to 6 kph, the AEB failed to deploy, necessitating manual intervention by Al-Ani.
Further testing with a BYD Dolphin Surf revealed a different, yet equally concerning, issue. At 4 kph, the AEB engaged and stopped the vehicle. Crucially, the system then disengaged after a few seconds, allowing the car to creep forward. While Al-Ani was able to brake manually, he emphasized the potential dangers in a live scenario. "Had this been a busy roundabout, for example, where the driver of the BYD mistakenly believes the car in front has pulled away," he explained, "they could be facing an angry driver and an expensive insurance claim. Had the car they’d rear-ended been a pedestrian instead, the consequences could have been far more serious."
These observations underscore a critical concern: ADAS, while designed to enhance safety, can exhibit behaviours that are either too sensitive or insufficiently robust, leading to unintended consequences or a failure to act when necessary. This is particularly problematic given the increasing reliance drivers place on these systems, often to their detriment.
The Annoyance Factor: Subjective Criticism Meets Objective Data
Four years after the implementation of GSR2, which mandates a suite of ADAS features in new vehicles, many motorists have become accustomed to what they perceive as annoying interventions. These range from indeterminate warning sounds and aggressive lane departure corrections to inconsistent speed sign recognition and hesitant "autopilot" behaviours.
The Kia Sportage Hybrid driven by the reporter to the Thatcham facility demonstrated this firsthand, exhibiting moderately aggressive steering corrections and frequent alerts for speed limit breaches and changes, differentiated by distinct auditory cues. Autocar’s own testing of the Leapmotor B10 previously described its ADAS functions as "some of the most annoying yet fitted to a car. The lane keeping assistance tugs the wheel very aggressively and there’s an orchestra of bings and bongs that are hard to tell apart."
Al-Ani acknowledges that poorly designed ADAS functions can undermine the very safety argument they are meant to support. However, he notes a significant disconnect in communication between drivers and manufacturers. "If you go to a manufacturer with subjective feedback on their product, they’re not interested," he stated. "However, if you go to them with objective data – ‘Here are the logs. Your car has done this this many times. This is the real-world impact. This is what it’s going to cost you in insurance’ – they’re very responsive to having that conversation." Thatcham’s ADAS test project is precisely aimed at generating this crucial objective data.
Thatcham’s Methodical Approach to Real-World Testing
To achieve this, Al-Ani and his team employ a comprehensive instrumentation system. This system meticulously measures the circumstances surrounding multiple deployments of each ADAS feature. Performance is assessed across several key metrics:
- Availability: When and how often the system deploys.
- Driver Collaboration: The system’s ability to work in conjunction with the driver, allowing for on-the-fly adjustments.
- Configurability: The extent to which the system can be customized by the driver.
- Warning Differentiation: Whether auditory warnings are distinct enough for the driver to understand the nature of the alert.
Testing is conducted across all weather conditions and on various types of roads, ensuring a holistic understanding of system performance. The ultimate measure of an ADAS feature’s success, according to Thatcham, is its usability. If a system is too intrusive or unreliable, drivers are likely to disable it, negating its safety potential.
To quantify this, the team assigns scores to each system based on factors such as perceived annoyance, the frequency of unwanted interventions, and the rate of erroneous activations. "With our knowledge of each system’s limitations, we can tell manufacturers that if they make a system to this level, most drivers will accept it, but if it’s below that, they will turn it off," Al-Ani explained. "We can tell insurers what is likely to happen when the system is used for real and manufacturers what more they need to do to improve it. We can also go to the regulators and tell them that if they’re considering adopting a system for GSR2 or for GB type approval, these are the tweaks they should consider that will mean people actually use the system."
Intelligent Speed Assistance (ISA): A Flawed Standard?
One of the most startling discoveries from Thatcham’s research concerns the current EU test standard for Intelligent Speed Assistance (ISA). While the standard assesses ISA accuracy over distance travelled, it can overlook critical moments, such as during rapid changes in speed limits. Thatcham’s event-based testing, conversely, meticulously measures ISA accuracy at each of these change points.
In real-world conditions across three test vehicles, this difference in methodology yielded significantly varied results. The MG ZS, identified as the poorest performer, achieved 91.3% accuracy over its driven distance. However, when evaluated on an event-based metric, its ISA was found to be only 74.3% accurate, meaning it displayed the incorrect speed limit in approximately one in every four instances.
Even the best-performing vehicle, a BMW i5, which scored 98.39% for distance accuracy and 90.3% for event-based accuracy, still presented issues. "Even the i5, the best ISA of the three cars on our test fleet, is wrong in one in every 10 events," Al-Ani stated, deeming this performance "not good enough." Furthermore, Thatcham’s testing revealed instances where ISA systems displayed speed limits that are not legally recognized in the UK, including 5, 10, 15, and even 100 mph.
"When ISA misreads speed limits, it can lead to unexpected or inconsistent system responses," Al-Ani cautioned. "Over time, this risks reducing driver confidence in the technology, which is critical to achieving its intended safety benefits." The implication is clear: if drivers cannot trust their vehicle’s speed advisory system, they are more likely to ignore it, or worse, switch it off entirely.
Configurability: The Key to Driver Acceptance?
The scenario where a driver’s first action upon entering their car is to disable as many ADAS features as possible is all too common. Al-Ani expresses sympathy for this reaction but believes that greater configurability of these systems offers a more constructive path forward. "If the ALK [active lane keeping] is too sensitive, rather than turn it off you should be able to turn it down," he suggested. He highlighted BMW’s progress in this area, noting their systems allow drivers to set preferences that persist across restarts. "Until ADAS have improved sufficiently, that’s a better condition to have," he added.
On the Road with Thatcham: Real-World Scenarios
Thatcham Research’s ADAS test program primarily unfolds on public roads, supplemented by controlled testing on their dedicated circuit. The investigation utilizes three vehicles: a BMW i5, a Tesla Model Y, and an MG ZS, chosen as representative of approximately 10% of new cars based on their ADAS capabilities.
Each vehicle is equipped with a roof-mounted, 360-degree lidar system and internal cameras that record the infotainment screen and ADAS functions. These capture forward and roadside views crucial for logging the presence of speed signs, lane markings, and the vehicle’s position relative to them.
During a demonstration drive in the Tesla Model Y, Al-Ani illustrated how its ADAS could prompt drivers to disengage it. In autopilot mode, the vehicle slowed excessively for a sharp corner on a main road. Later, upon entering a built-up area and navigating a bend, it braked abruptly for a parked vehicle. "This car has software designed only to pass the EU regulation," Al-Ani observed. "In the slow-moving queue we’ve joined it’s brilliant, but for many drivers its unpredictability is enough to discourage them from using it at all."
He was particularly critical of how the Tesla’s system disengages with the slightest driver input. "It’s not collaborating with me, so I become frustrated and leave it off." Further illustrating the system’s unpredictable nature, he noted an instance where the car braked for a bus turning into the road from the right, while simultaneously making a steering correction for a pedestrian who was still at a safe distance. The system then disengaged, leaving the driver with limited reaction time. "Working at its best, Tesla’s system offers real benefits, but drivers need to understand its limitations," Al-Ani concluded.
Thatcham vs. Euro NCAP: A Divergent Approach
Euro NCAP, Europe’s primary vehicle safety assessment body, also conducts real-world testing for ADAS, complementing its laboratory evaluations. Their program involves fitting vehicles with sensors to monitor system responses to various road conditions and driving an extensive 1200 miles across multiple European countries, logging every reaction.
Adriano Palao, ADAS Technical Manager at Euro NCAP, stated, "We have worked on ADAS for years and now we want to make sure the technologies are delivering not only on the test track but on the road. For example, is the lane keeping assist annoying and aggressive? What is the accuracy of the speed limit information? Were there any false braking events? This is the first time we have sought to find out what is the experience of the end consumer."
Thatcham Research, a former 22-year member of Euro NCAP, ended its relationship earlier this year to concentrate on its independent testing initiatives. Yousif Al-Ani, however, dismisses concerns that this departure might diminish Thatcham’s influence. "We have quite a lot of power, actually," he asserted. "We hold the keys to motor insurance risk rating in the UK, the second-biggest car market in Europe. Vehicle manufacturers listen to us when something threatens their customer base. It’s no use having systems that work well in France or Germany where approval testing is carried out but not so well on British roads." This strategic shift allows Thatcham to focus on developing its own methodologies tailored to the specific nuances of the UK driving environment.
Key Insights from ADAS Engineer Yousif Al-Ani
Thatcham’s ongoing research has yielded several notable findings:
- Lane Keeping Assist (LKA): The BMW i5 demonstrated exceptional performance with no false events recorded. The MG ZS recorded a very low rate of 0.43 false events per 100km, and newer systems generally exhibit less aggressive assistance.
- Crash Avoidance (AEB): The Tesla Model Y tends to deploy its brakes earlier than the other test cars across various scenarios, though the BMW i5 closely follows. However, the frequency of driver intervention required remains comparable between the Tesla and BMW.
- ‘Ghost Braking’ (Unnecessary Braking): The Tesla Model Y exhibited the highest number of unnecessary braking events, with 122 instances compared to eight in the BMW i5 and six in the MG ZS. Al-Ani attributes this to the Tesla’s sensitivity settings being turned up to exceed regulatory requirements, potentially leading to driver annoyance and system deactivation.
- Lane Markings: A significant challenge identified is the absence of road markings on nearly a quarter of test routes, preventing lane-keeping events from being scored. Thatcham suggests that improved road maintenance, including repainting lines, is crucial for maximizing the safety benefits of LKA.
- AEB vs. LKA: Autonomous emergency braking (AEB) systems are considered to have a substantially greater impact on road safety than lane-keeping assist (LKA) systems.
- The Imperative of Real-World Testing: "ADAS can be incredibly effective," Al-Ani emphasized. "But how well vehicles perform outside the regulated test scenario and in all conditions and environments, that’s a big unknown. For us it’s about finding the differentiation."
Broader Implications for Road Safety and the Automotive Industry
Thatcham Research’s proactive approach to ADAS testing has significant implications for the automotive industry, regulatory bodies, and ultimately, the end consumer. By providing objective, real-world data, they aim to:
- Drive Manufacturer Improvement: Manufacturers are incentivized to refine their ADAS algorithms and calibration to meet not just minimum regulatory standards but also driver expectations for usability and reliability. This could lead to a future where ADAS are less intrusive and more consistently helpful.
- Inform Insurance Risk Assessment: Thatcham’s expertise in insurance risk rating in the UK positions them to influence how ADAS performance impacts insurance premiums. Systems that are demonstrably more reliable and less prone to false activations could lead to lower insurance costs.
- Guide Regulatory Development: The data generated can provide regulators with evidence-based insights to refine existing standards like GSR2 and inform the development of future safety regulations. This could ensure that mandated systems are truly effective and beneficial in diverse driving conditions.
- Enhance Driver Trust and Adoption: By identifying and addressing the sources of driver frustration, Thatcham’s work can help rebuild trust in ADAS. When systems are perceived as predictable and beneficial, drivers are more likely to engage with them, thereby maximizing their safety potential.
The automotive landscape is rapidly evolving, with ADAS playing an increasingly central role. Thatcham Research’s commitment to rigorously testing these systems in the environments where they are most needed is crucial in ensuring that technological advancements translate into genuine improvements in road safety for everyone. The differentiation they seek to uncover is not merely an academic exercise but a vital step towards developing ADAS that drivers can rely on, rather than resent.
