An Update on Arm’s AI Journey Toward a Trillion Connected Devices
By Dennis Laudick, vice president of marketing, Machine Learning Group, Arm
The Arm ecosystem recently achieved the milestone of having shipped more than 150 billion Arm-based chips to date as we progress toward 1 trillion connected devices by 2035. Given this latest indicator of the pervasiveness of Arm technology, it’s a good opportunity to provide an update on our efforts to enable AI to run on Arm everywhere.
Arm’s product roadmaps have always been defined by input from our ecosystem on future software and applications needs across multiple markets. However, software isn’t just about the CPU, but instead a system-level approach that Arm has always taken when architecting our IP for specific applications. And in this new computing landscape where AI, IoT, and 5G are converging to enable new and increasingly complex use cases, having a system-level approach has never been more critical.
It’s with this background that Arm’s Project Trillium (first announced in February 2018) was conceived several years ago with this simple vision; an all-inclusive solution for connecting all of Arm’s AI/ML innovations across our product portfolio. That vision is manifesting itself today as AI/ML requirements are rapidly evolving from reliance on a single compute processor into a collective heterogenous compute solution, all based on principles which have underpinned Arm’s success in many markets:
- Software always starts with the CPU
- Augment Arm CPUs and GPUs with scalable and flexible dedicated ML processors for a wide range of dynamic ML functions
- Support industry-leading APIs and provide open source software to ensure maximum performance and application portability
- Invest in tools to make developing and deploying ML on Arm easy
- Value, nurture, and support the Arm ML ecosystem
With these principles in mind, here’s a brief review of recently announced Arm IP products designed to play a role in our highly comprehensive, solutions approach to ML:
- A 35x ML performance increase over previous generations for the Cortex-A76 and Cortex-A77 processors designed for premium mobile and large screen compute devices
- Cortex-A55 with new neural network elements including improved prediction, 8-bit integer matrix multiply, and new architecture instructions incorporated in the NEON pipeline specifically benefiting ML workloads
- Mali-G77 GPU; a 60% ML performance increase
- Helium CPU extensions for Cortex-M processors, providing 15x more ML performance and efficiency for IoT and other power-constrained connected devices
- The first in a range of ML processors initially targeted for mobile, home and security devices
Additionally, we continue to make ongoing performance optimizations to our open source ML projects, Arm NN, Arm Compute Library, and CMSIS-NN, all of which have shipped in hundreds of millions of devices a year over the last few years.
The road ahead
Many of the aforementioned products were already in development prior to SoftBank acquiring Arm in 2016. Following that acquisition, Arm became a private company and SoftBank gave us the freedom to increase our investment in AI/ML solutions, which enabled us to accelerate the development of today’s products. In fact, being part of SoftBank has been extremely advantageous and has enabled us to increase our investment in AI/ML features across all our product roadmaps. A recent example of this is our plans to support the Bfloat16 (BF16) data type for ML workloads in the next revision of the Armv8-A architecture and will be implemented in upcoming CPUs from Arm and its partners.
There is much more to come as we continue to accelerate the results of our years of product development with a range of CPU and GPU improvements, new NPU product lines, enhanced SW, tools, and developer support. At the upcoming Arm TechCon conference (Oct. 8-10 in San Jose, CA), Arm Machine Learning Fellow Ian Bratt will provide a glimpse of what’s ahead for Arm in ML as part of his day two keynote.
Talking with our silicon partners, device makers, and a diversity of service providers about their future AI/ML requirements at our annual partner meeting last month was further validation that our roadmaps and ongoing system-level design principles are hitting the mark. I walked away from those three days of partner meetings with our thriving ML ecosystem with the following realization; AI everywhere on Arm is no longer just a vision but an inevitability.
Any re-use permitted for informational and non-commercial or personal use only.
Media Contacts
Latest on Twitter
🗓️ Join us on Wednesday July 31 for our next financial results conference call.
After market close, we'll report our earnings results for the first quarter of fiscal year 2025, followed by a conference call to review our results and business outlook: https://okt.to/koqGT0
The "Write Once, Deploy Everywhere" approach to code is key to seamless deployment for developers.
Through expert talks and demos @WeAreDevs, we demonstrated how this is a game-changer for building and deploying next generation apps at scale.
Thanks to everyone who joined us!🙌
With the escalating need for power-efficient systems, @FujitsuFNC's turned to Arm to build the FUJITSU-MONAKA CPU - a processor designed to help achieve energy savings of 40% in Japanese data centers by 2030.
+ That's without compromising AI innovation: https://okt.to/WM0F3A
Arm KleidiAI brings 30% performance improvements to @Google AI Edge's MediaPipe and XNNPACK running the Gemma 2B LLM. 💪
This is the first integration of many, as we enable more LLMs to run as effectively and efficiently as possible on Arm mobile CPUs: https://okt.to/qLW8He
Flooding can happen anytime, but early warning systems can help reduce its impact on communities and protect lives.
We're proud to work with @UNICEF_uk and support the team in Karonga, Malawi to bring the latest in flood alert technology to communities:
How can developers stay ahead in the age of AI?
Andy Wafaa shares details on the tools required to optimize code, increase accessibility, and shape the future of computing at @WeAreDevs World Congress: https://okt.to/wJakX3
Since 2012, Raspberry Pi's been changing the game for businesses in the ecosystem.🚀
Founder & CEO, Eben Upton shares the origin story of Raspberry Pi and why he chose Arm as the architecture of choice:
Interview recorded in Jan 2023.
Arm Education in Conversation with Eben Upton
In this episode, we interview Eben Upton, founder and CEO of Raspberry Pi (Trading) Ltd. Eben shares with us the...
okt.to
In the age of AI, collaboration is crucial to solve technological and societal challenges.
Together with a group of academics at our Cambridge office, we've developed a "Call for Action," identifying the steps needed to ensure AI adoption in both the public and private sectors.
Ready to upscale your mobile game development without losing performance?
👉 Say hello to Arm Accuracy Super Resolution
It's our open source solution derived from FSR2, designed to increase the resolution of images from low to high with our efficient shader code:
.@AWSCloud Graviton4 is here! 🙌
Announced in GA, Graviton4 uses Arm Neoverse V2 to provide the most powerful, energy efficient AWS processor to date.
It follows 5+ years of collaboration on Graviton to enable CPU silicon innovation in the Amazon Cloud: https://okt.to/UWM0FR
Proud to partner with @AlzheimersBRACE to support dementia research and foster community. 🤝
From running the Bristol Half-Marathon to raising £2K+ for charity and spreading awareness, shoutout to our team for making a real impact. 💙
Not sure where to start with running Llama models? We've got you. Our 🆕 learning path teaches you how to build and run a simple Android Chat app with different Llama models using ExecuTorch and XNNPACK.
Dive into the step-by-step guide in the link below.
⚡ Silicon is defining and revolutionizing technology.
With the help of our partners, we're at the forefront of its evolution, driving innovative new approaches to silicon design with CSS and chiplets to define the future of computing.
More in @WSJ. 👇
Paid Program: The Evolution of Silicon
From Moore’s Law to today’s modern designs for AI, silicon is defining technology markets now and in the future.
okt.to
Arm CSS for Client is unleashing the power of the Armv9 architecture for AI innovation.
CSS for Client delivers physical implementations across Arm CPUs and GPUs, simplifying SoC builds for the fastest path to production silicon.
Learn more👇 https://okt.to/Q3yLjc
The advancements in large multimodal models and GenAI herald a new era in AI robotics.
That’s why tomorrow's robotics are harnessing the power of LLMs to improve energy efficiency, security, and functional safety.
🦾 And it's all happening #onArm: https://okt.to/6nbZ9m
The world's leading AI companies are prioritizing data center efficiency to scale and grow. And they're doing it #onArm.
Arm CMO Ami Badani joined The Artificial Human Podcast to discuss why efficiency is crucial to driving AI innovation.
The Artificial Human - How green is my AI? - BBC Sounds
Aleks Krotoski and Kevin Fong ask, how green is my AI?
okt.to
Here's a glimpse of our #INWED24 celebrations in Kenya, Ghana and Switzerland.
The Arm-Ambassador-led events brought everyone together to celebrate the significant impact of women engineers and to inspire future innovators to help bridge the STEM gap: https://okt.to/9ZAuna
Find $ARM in the Nasdaq-100 Index from today. 🙌
As we go from strength-to-strength as a public company, we're excited to join industry leading companies on the @Nasdaq stock exchange, emphasizing the importance of Arm tech among the global ecosystem: https://okt.to/gpRsAe
Create the best possible AI experiences with KleidiAI. ⚡
KleidiAI provides a collection of highly optimized AI kernels that can be embedded directly into popular AI frameworks as part of Kleidi Libraries.
Ready to reap the benefits?
🥳 Our partners @HPE and @NVIDIA are making turnkey private-cloud AI a reality!
Announced this week, NVIDIA AI Computing by HPE brings the power and efficiency of Grace Hopper Superchips, built #onArm, to the HPE ProLiant Compute DL384 Gen12 server.
HPE ProLiant Compute DL384 Gen12 | HPE Store United Kingdom
okt.to
For #WorldRefugeeDay, we're celebrating our partnership with @Refugees & @UNRefugeesUK to ensure AI tools for humanitarian efforts are used responsibly.
By supporting @UNHCRinnovation, we can ensure diversity is built into tools from the ground up:
We're putting the foundations in place for a thriving, diverse, chiplet ecosystem in Automotive.
From participation in @imec_int's Automotive Chiplet Program, to our role in SOAFEE, here's how we're ensuring interoperability and performance for chiplets: https://okt.to/eBfYgd
Here's to our fantastic, award-winning Internal Comms team. 👏
At the #ICEAwards, the team took home 3️⃣ awards for:
⭐ Best event
⭐ Best Internal Comms from the tech, media and telecommunications sector
⭐ Best Internal Comms Campaign in a large business under 10,000 employees
Rene Haas joined the Samsung Foundry Forum to discuss our almost 30-year partnership and the possibilities of Arm CSS.
By removing the complexity from SoC development, Arm CSS enables our partners to move faster in the age of AI, with the support of partners like Samsung.
