AI cars face situations they’ve never seen. Can they learn without risking lives? Expert explains

As artificial intelligence moves out of the digital realm and into vehicles, robotics, and other physical systems, one of engineering's hardest problems is no longer simply teaching machines to recognise what they see. It is preparing them for what they have never encountered before.

While global debates around AI safety often focus on digital algorithms and chatbots, autonomous and AI-assisted driving systems represent the technology’s most critical real-world test. How reliable are these vehicles, and what can be done to protect passengers and pedestrians alike?

To understand how Physical AI learns on the road, we spoke with Teja Gudena, Executive Vice President of Engineering at Netradyne, an AI driver safety and fleet management platform.

AI-driven accidents that shocked the industry

Before exploring how AI systems adapt to real-world edge cases, we must examine the high-profile incidents that highlight the stakes involved.

In June 2026, 76-year-old Martha Avila was killed in Katy, Texas, after a Tesla Model 3 crashed directly into her home while she was standing in her front room. Her family filed a lawsuit alleging that the driver, Michael Butler, was operating the vehicle using an automated driving assistance system.

In another notable incident in March 2025, three students in Tongling, Anhui, China, died when their Xiaomi SU7 electric vehicle caught fire after striking a highway cement barrier. Xiaomi confirmed that the vehicle was operating in its "Navigate on Autopilot" (NOA) assisted-driving mode at 116 km/h in a road construction zone. While the car's sensors detected the barrier and initiated braking, it could not stop in time. The incident drew widespread scrutiny, particularly because Xiaomi founder Lei Jun had previously stated that the car's Automatic Emergency Braking (AEB) system could detect stationary obstacles and automatically stop at speeds up to 135 km/h during the day and 120 km/h at night.

AI vs. Human drivers: What the numbers show

Despite high-profile crashes, statistical data indicates that autonomous systems are significantly safer on average than human drivers.

According to data from the Insurance Institute for Highway Safety (IIHS) and autonomous fleet operators, AI-driven vehicles experience a 68% lower overall crash rate per mile travelled. Furthermore, they achieve an 81% to 82% reduction in injury-causing crashes compared to human drivers.

Waymo reported that by late June 2026, its driverless fleet had driven over 270 million autonomous miles across major metropolitan markets, including Atlanta, Austin, Los Angeles, Phoenix, and San Francisco. The company’s safety benchmarks showed that its autonomous driver was involved in 82% fewer injury-causing crashes than human drivers, preventing an estimated 841 injuries.

How fleet learning actually works

Despite these impressive statistics, autonomous vehicles continually encounter unfamiliar situations requiring software and operational updates.

For instance, Waymo previously issued software recalls—including one for nearly 3,800 vehicles after a robotaxi entered a flooded lane in San Antonio during severe weather, and another for nearly 3,900 vehicles after cars entered freeway construction zones after failing to recognise closure signs.

These episodes highlight a central question for Physical AI: when a machine encounters an unfamiliar situation, what happens next?

A common misconception is that an AI vehicle learns dynamically on the spot, immediately rewriting its own behaviour while driving. Teja Gudena points out that real-world safety requires a far strictly controlled process.

"In Physical AI systems, inference typically occurs at the edge, while learning occurs through a structured edge-to-cloud training and validation process," Gudena explains.

In simple terms, the car's onboard computer (the "edge") processes raw sensor data to make real-time driving decisions, but it does not reprogram itself on the fly. Instead, data from unusual events is sent back to central cloud servers, where engineers analyze the scenario, train updated models, and validate safety improvements. Only after rigorous testing is the updated software deployed across the entire fleet via over-the-air updates.

"A vehicle cannot be allowed to experiment with unverified changes while carrying passengers or sharing the road with others," Gudena adds. "The AI does not make those decisions independently."

Moving toward contextual reasoning and world models

"The real world presents an almost infinite number of possibilities," says Gudena. "No training dataset can capture every combination of road conditions, weather, vehicle interactions, human behaviour, animals, or unexpected events."

As a result, modern AI driving architectures are shifting away from simple object recognition toward world models and contextual reasoning.

Beyond Detection: Instead of merely asking "What is this object?", the system must evaluate where it is, how it is moving, and what surrounding road users are likely to do next.

Predictive Awareness: When encountering animals, detecting a dog or deer is only step one. The system must determine if the animal is stationary, moving toward the road, or unpredictable, while simultaneously factoring in surrounding traffic.

Edge Cases & Bias: Physical AI must also account for human diversity. Addressing disparities in pedestrian detection across different lighting conditions, skin tones, clothing, and geographies requires diverse, representative datasets. "It is not enough to evaluate a model only on its overall accuracy," Gudena notes. "Performance must remain consistent across all populations and operating conditions."

Balancing Speed with Safety

While fast updates are critical, Gudena warns against rushing updates and over-sensitising systems. Rushing patches without proper calibration can lead to unnecessary alerts or erratic driving behaviour, creating new risks on the road.

The ultimate goal of Physical AI is not simply to make vehicles learn faster but to ensure that every lesson learned by a single vehicle permanently improves the safety of the entire fleet.