What are the Latest Tech Innovations from Arm in October 2024?

As we move further into the era of advanced computing, Arm is continuing to lead the charge with groundbreaking tech innovations. October 2024 has been a month of significant strides in technology, particularly in AI, machine learning (ML), security, and system-on-chip (SoC) architecture.  

The Arm Editorial Team has highlighted the cutting-edge tech innovations that happened at Arm in October 2024 – all to shape the next generation of intelligent, secure, and high-performing compute systems. 

Enhancing AI, ML, and Security for Next-Gen SoCs with Armv9.6-A

Arm’s latest CPU architecture, Armv9.6-A, introduces key enhancements to meet evolving computing needs, focusing on AI, ML, security, and chiplet-based systems-on-chip (SoCs). Martin Weidmann, Director Product Management, discusses the latest features in the Arm A-Profile architecture for 2024. 

The 2024 updates enhance Scalable Matrix Extension (SME) with structured sparsity and quarter-tile operations for efficient matrix processing while improving memory management, resource partitioning, secure data handling, and multi-chip system support. 

Streamlining PyTorch Model Deployment on Edge Devices with ExecuTorch on Arm

Arm’s collaboration with Meta has led to the introduction of ExecuTorch, enhancing support for deploying PyTorch models on edge devices, particularly with the high-performing Arm Ethos-U85 NPU. Robert Elliott, Director of Applied ML, highlights how this collaboration enables developers to significantly reduce model deployment time and utilize advanced AI inference workloads with better scalability. 

With an integrated GitHub repository providing a fully supported development environment, ExecuTorch simplifies compiling and running models, allowing users to create intelligent IoT applications efficiently.  

Accelerating AI with Quantized Llama 3.2 Models on Arm CPUs

Arm and Meta have partnered to empower the AI developer ecosystem by enabling the deployment of quantized Llama 3.2 models on Arm CPUs with ExecuTorch and KleidiAI. Gian Marco Iodice, Principal Software Engineer, details how this integration allows quantized Llama 3.2 models to run up to 20% faster on Arm Cortex-A CPUs, while maintaining model quality and reducing memory usage. 

With the ExecuTorch beta release and support for lightweight quantized Llama 3.2 models, Arm is simplifying the development of AI applications for edge devices, resulting in notable performance gains in prefill and decode phases.  

Optimizing Shader Performance with Arm Performance Studio 2024.4

Arm’s latest Frame Advisor enhancement helps mobile developers identify inefficient shaders, boosting performance, memory usage, and power efficiency. Julie Gaskin, Staff Developer Evangelist, details the new features in Arm Performance Studio 2024.4, including support for new CPUs, improved Vulkan and OpenGL ES integration, and expanded RenderDoc debugging tools.   

This update provides detailed shader metrics – like cycle costs, register usage, and arithmetic precision – enabling developers to optimize performance and lower costs.  

Boosting Performance and Security for Arm Architectures with LLVM 19.1.0 

LLVM 19.1.0, released in September 2024, introduces nearly 1,000 contributions from Arm, including new architecture support for Armv9.2-A cores and performance improvements for data-center CPUs like Neoverse-V3. Volodymyr Turanskyy, Principal Software Engineer, highlights the features of LLVM 19.1.0, which deliver better performance and enhanced security.   
  
The update optimizes shader performance and Fortran intrinsics, adds support for Guarded Control Stack (GCS), security mitigations for Cortex-M Security Extensions (CMSE), enhancements for OpenMP reduction, function multi-versioning, and new command-line options for improved code generation. 

Introducing System Monitoring Control Framework (SMCF) for Neoverse CSS

Arm’s System Monitor Control Framework (SMCF) streamlines sensor and monitor management in complex SoCs with a standardized software interface. Marc Meunier, Director of Ecosystem Development, highlights how it supports seamless integration of third-party sensors, flexible data sampling, and efficient data collection through DMA, reducing processor overhead.   
  
The SMCF enables distributed power management and improves system telemetry, offering insights for profiling, debugging, and remote management while ensuring secure, standards-compliant data handling.   

Achieving Human-Readable Speeds with Llama 3 70B on AWS Graviton4 CPUs  

AWS’s Graviton4 processors, built with Arm Neoverse V2 CPU cores, are designed to boost cloud performance for high-demand AI workloads. Na Li, ML Solutions Architect, explains how deploying the Llama 3 70B model on Graviton4 leverages quantization techniques to achieve token generation rates of 5-10 tokens per second.   

This innovation enhances cloud infrastructure, enabling more powerful AI applications and improving performance for tasks requiring advanced reasoning.   

Superior Performance on Arm CPUs with Pardiso Sparse Linear Solver

Panua Technologies optimized the Pardiso sparse linear solver for Arm CPUs, delivering significant performance gains over Intel’s MKL. David Lecomber, Senior Director Infrastructure Tools, highlights how Pardiso on Arm Neoverse V1 processors outperform MKL, demonstrating superior efficiency and scalability for large-scale scientific and engineering computations.   

This breakthrough positions Pardiso as a top choice for industries like automotive manufacturing and semiconductor design, offering unmatched speed and performance.   

Built on Arm Partner Stories

Vince Hu, Corporate Vice President, MediaTek, talks about the Arm MediaTek partnership, which drives ongoing tech innovation and delivers transformative technologies to enhance everyday life. 

Eben Upton, CEO of Raspberry Pi, shares how the company has evolved from an educational tool to a key player in industrial and embedded applications, all powered by Arm technology. He highlights the development of new tools over the past decade and his personal journey with the BBC Microcomputer.  

Clay Nelson, Industry Solutions Strategy Lead at GitHub, discusses the partnership between GitHub and Arm, which combines GitHub Actions with Arm native hardware to revolutionize software development, leading to faster development times and reduced costs. 

Sy Choudhury from Meta Platforms Inc. explains how the collaboration with Arm is optimizing AI on the Arm Compute Platform, enhancing digital interactions through devices like AR smart glasses, and impacting everyday experiences with advanced AI applications.  

Highlights from PyTorch Conference 2024 

To accelerate the development of custom silicon solutions, Arm partners are tapping into the latest industry expertise and resources. Principal Software Engineer, Gian Marco Iodice discusses this in, “Democratizing AI: Powering the Future with Arm’s Global Compute Ecosystem,” from PyTorch Conference 2024. 

Iodice highlights KleidiAI-accelerated demos, key AI tech innovations from cloud to edge, and the latest Learning Paths for developers.