What are AI-Defined Vehicles and Why They Are the Future of Automotive?

While it may seem like a distant vision into the future, we are well on the road to AI-defined vehicles. Today, AI is already completely changing the driving experience and how we interact with our cars.

The next generation of vehicles will be AI-defined, built on the foundation of today’s software-defined vehicles (SDVs), and with AI integrated into every aspect of the vehicle’s functionality, intelligence and user interaction. This provides opportunities for the development of brand-new AI applications that will transform the in-vehicle experience.  

Just like how software updates continuously transform the applications and features of SDVs, the capabilities and intelligence of AI-defined vehicles continue to evolve over time as technology learns from different environments, user behaviors and operational data. This will enable faster real-time decision-making, continuous adaptive learning, highly intelligent and conversational human to vehicle interactions, advanced autonomy, and high levels of personalization for safer, more intuitive driver and passenger experiences.

The AI-defined vehicle use cases

Two key in-vehicle use cases will benefit the most from the ongoing development and deployment of AI-defined vehicles – advanced driver assistance systems (ADAS) and in-vehicle infotainment (IVI).

For ADAS, the AI-defined vehicle will mean:

• Advanced driver behavior monitoring and prediction, with AI models processing data from cameras, radar and lidar in real-time to predict potential collisions through not just reacting to objects but also anticipating driver behavior, road conditions and nearby vehicle trajectories, before enabling features like automatic emergency braking and steering with higher accuracy.
• Enhanced adaptive cruise control, with AI-enhanced systems able to understand complex driving scenes, like merging lanes, construction zones and roadworks or erratic drivers, and then adjusting driver behavior proactively for smoother, safer driving.
• Predictive collision avoidance, with AI-powered driving monitoring systems (DMS) using computer vision to assess driver alertness, gaze and behavior before issuing warnings or taking action, like slowing down the vehicle.

Meanwhile, for IVI, large language models (LLMs) are being incorporated into AI-defined vehicles to enable the following:

• More intelligent natural language understanding, with drivers and passengers able to control climate, media, navigation or vehicle settings using everyday language, improving usability and minimizing driver distraction.
• Greater personalization, with AI algorithms learning driver and passenger preferences over time – from seat positions and lighting to music and navigation routes – and creating a personalized profile that adapts automatically and can be transferred across vehicles.
• In-cabin emotion monitoring, with AI-powered systems detecting the emotional state and attention level of drivers and passengers, and responding with adaptive content suggestions, like calming music, or safety responses, like alerts for drowsy drivers.

The role of SDVs

AI-defined vehicles represent the natural evolution of SDVs through enhancing AI capabilities, which in many cases already exist in today’s vehicles. At the same time, AI-defined vehicles will leverage the underlying software-defined cloud to car infrastructure and standardized software frameworks that the industry has been building for years. This provides the flexible foundation that AI needs to function effectively, safely, and at scale inside a car.

Making AI-defined vehicles a reality

We believe there will be three key pillars to making the transition to this next era of vehicle computing and intelligent capabilities a reality:  

1. Safe, secure, performant and efficient compute platforms;
2. Advanced AI software stacks, models, and tools; and
3. Open industry collaboration.

Safe, secure, performance and efficient compute platforms

All AI workloads require a significant amount of real-time computing power, especially in the vehicle. However – more than any other market – low power, efficient computing is vital, as the majority of compute for AI-defined vehicles runs at the edge – in the vehicle. This leads to reduced latency and faster response times, as well as robust safety and security mechanisms that match the rising AI capabilities in the vehicle.

The varying computing demands from different AI workloads in the car, alongside the ongoing evolution of AI models, require heterogeneous compute platforms that enable different components, including accelerating computing across CPU, GPU, and AI accelerators, to work together seamlessly and efficiently. These platforms need to scale from entry-level to premium vehicles without losing computing capabilities, and integrate advanced safety and security features that comply with international functional safety standards, such as ISO 26262 and ISO/SAE 21434, the cybersecurity framework for automotive systems.

Arm is delivering a broad range of high-performance, low-power automotive enhanced (AE) technologies to automotive markets that cover a variety of AI workloads for different in-vehicle use cases and have in-built functional safety and security features.

Advanced AI software stacks, models, and tools

As part of these safe, secure, performance and efficient compute platforms, there needs to be a full software foundation where AI models and workloads can be continuously developed, deployed and updated. Advanced AI software stacks will act as the bridge between the raw AI data and models and real-world, on-road behavior. While traditional modular architecture for AI development is here to stay, new methods – like end-to-end AI models that are trained in enormous, representative datasets and continuously updated with new data – are emerging.

End-to-end AI models enable AI-defined vehicles to go straight from perception to action through deep learning capabilities that are built into huge datasets in the vehicle based on human driving behavior. They directly map raw inputs, like camera, radar, lidar and other sensor data, to key driving inputs, like steering, acceleration and braking, without needing a long chain of separate, manually coded modules based on different environments and potential hazards. This improves decision-making and control while guiding navigation and avoiding obstacles through interacting with dynamic environments in real-time.

Arm is collaborating with our ecosystem to enable full stack software solutions that run across a broad range of AI applications, and recently introduced Arm Kleidi to automotive markets to deliver automatic performance optimizations for developers’ AI workloads running on the Arm CPU, leading to better in-vehicle application experiences. Moreover, it is important to provide the ecosystem with robust cloud to car parity across AI hardware, models and software. With the Armv9 architecture pervasive from cloud to edge, we are delivering the ISA parity that the ecosystem needs to test and verify AI applications, models and workloads in the cloud before seamlessly deploying in the car.

Open industry collaboration

As with SDVs, the development and deployment of AI-defined vehicles cannot be achieved by one company alone. It will require extensive industry collaborations built on common standards where different companies in the automotive supply chain work together to enable faster innovation, broader interoperability, reduced development costs and a faster time-to-market.

These collaborations are already taking place, with Arm working alongside leading automotive companies, like Mapbox, to accelerate the development and deployment of AI-based applications in vehicles. Another great example of collaboration is with AWS Automotive which is leveraging an Arm KleidiAI integration to develop its in-vehicle chatbot prototype, improving response times of 10x and saving six weeks of development time. Meanwhile, working with Cerence AI, which develops AI assistant software for automotive applications, the KleidiAI integration with llama.cpp led to an around 3x improvement in tokens-per-second for faster response times when running its 1B model on the Arm-based NVIDIA Orin board.

Furthermore, the continuous collaborations taking place through SOAFEE are providing the foundational support to accelerate modern AI-based automotive development, making it easier for developers to port software across platforms for different applications in the AI-defined vehicle.

Turning AI-defined vehicles into a reality

As vehicles evolve into intelligent, adaptive computing systems that respond in real-time, the automotive industry is on the road to AI-defined vehicles now. At Arm, we see our role not just as a technology provider, but as a catalyst for innovation – working closely with our partners to build the foundational platforms that will power and scale this new era of automotive. This will turn the promise of AI-defined vehicles into reality.