Arm Corstone-320: Accelerating Voice, Audio and Vision IoT Systems

The integration of voice, audio and vision into the Internet of Things (IoT) has revolutionized the way connected devices interact with, interpret and perceive the world around them. The future opportunities for IoT markets are vast, particularly in vision. In the next decade, there will be millions of advanced cameras deployed across a variety of locations, including factories, shops, warehouse, cities, offices and shops. This will drive an exponential increase in IoT use cases not imaginable before. These cover security, people and environmental monitoring, energy, facility and inventory management, access control, safety and user interface, with each use case requiring different levels of deployment and management based on the location.

Introducing Arm Corstone-320

Arm Corstone-320 is our new IoT Reference Design Platform that captures these exciting IoT market opportunities for our ecosystem of partners. It brings together leading-edge embedded IP with software, tools and support, including Arm Virtual Hardware. This accelerates the development and deployment of voice, audio and vision systems, and reduces the complexity for system-on-chip (SoC) designers and software developers.

Alongside reducing complexity and accelerating time to market, Corstone-320 has been designed to provide the following benefits:

• Increased performance, through improving compute throughput, to provide the right compute for a variety of IoT devices and use cases;
• Improved memory bandwidth, such as data width and memory channels, to deliver system wide interconnect bandwidth for pixel traffic;
• Greater energy efficiency for improved sustainability and battery life;
• A range of power modes, including sleep mode, standby mode and power islands;
• Lower design costs and engineering time, as designers and engineers can start their SoC designs straight away with the “out-of-the box” Corstone-320;
• More security features, including boot-time and runtime security mechanisms, that are integrated across the system;
• Observability, with full debug and trace of all components built into Corstone-320;
• Improved flexibility, with the Corstone-320 design being customizable across different segments, devices and use cases; and
• A range of software, including a large amount of open-source software, to reduce overall costs and complexity.

The Corstone-320 hardware

Focusing on the hardware, Corstone-320 integrates the following leading-edge Arm IP into the IoT Reference Design Platform:

• Arm Cortex-M85;
• Arm Mali-C55 ISP;
• Arm Ethos-U85 NPU; and
• Arm CoreLink DMA-350.

Cortex-M85 is the highest performing Cortex-M processor with Arm Helium technology. It is a great choice for vision use cases in particular, as it offers:

• Energy efficiency for cameras that need to operate for long periods of time;
• High performance for cameras that need to capture high-quality video and audio in real-time;
• Security to defend against hacking or other forms of cyber attacks; and
• Flexibility to be used across a wide range of cameras and vision use cases.

Cortex-M85 provides the natural upgrade path for Cortex-M based applications that require significantly higher performance and increased security. In addition to Arm TrustZone technology, it integrates Pointer Authentication and Branch Target Identification (PACBTI) security technologies to mitigate against return-oriented programming (ROP) and jump-oriented programming (JOP) security exploit attacks.

Ethos-U85 is Arm’s third generation NPU, which delivers a 4x performance uplift and 20 percent higher power efficiency compared to its predecessor. It is scalable across IoT applications, supporting configurations from 128 to 2048 MAC units, which is up to 4TOPs of machine learning (ML) acceleration at 1GHz. Ethos-U85 also supports transformer networks, as well as Convolutional Neural Networks (CNNs) for AI inference.

Mali-C55 provides the best combination of high-resolution image processing supporting 48 megapixel image resolution, energy efficiency, flexibility and unparalleled image quality for wide range of IoT vision use cases. Meanwhile, the highly configurable DMA-350 is a direct memory access controller that enables the efficient data movement to improve system performance and power efficiency, with support for Arm TrustZone.

The Corstone-320 package also includes technical reference manuals, configuration and rendering scripts, alongside verification reports. This means ASIC developers can build SoCs to a specific segment requirement or use the package to explore Arm’s compute design intent before starting their custom design.

The Corstone-320 software

The Corstone-320 package of software includes firmware, drivers for all the IP, middleware, real-time operating system (RTOS) and cloud integrations, ML models and reference applications. This means software developers can easily select the components required for their specific segment and build the IoT stack for that device using the development tools of their choice. The open-source applications demonstrate keyword spotting, speech recognition and object recognition use cases. Meanwhile, the prototyping platforms included in Corstone-320 enable software development to take place in parallel to SoC design.

The software also includes Fixed Virtual Platforms (FVPs), which model the peripherals that complete the system for an FPGA. FVPs are used by software developers to accelerate development by removing the need for hardware availability to start the application development.

Finally, Arm has the world’s largest ecosystem of AI partners that supply ML models and software to meet the diversity of IoT device requirements. These vary from low power sensors for embedded systems to high-performance IoT applications.

Unlocking IoT innovation opportunities

The continuous integration of voice, audio and vision into IoT devices represents a significant opportunity for innovation. By allowing these devices to interact with, perceive and interpret their surroundings in a more sophisticated manner, a wide range of new applications and capabilities that were previously unimaginable are now being unlocked. This has the potential to drive significant innovation and evolution across IoT devices and technologies.

Corstone-320 is at the forefront of this evolution, providing the low-cost, low-power, high-performance, intelligent Reference Design Platform for IoT markets. Through the combination of integrated software and hardware, it will dramatically accelerate product timelines and reduce the complexity of SoC design and software development across a variety of use cases and settings. This will help the ecosystem to scale IoT applications at a rate never seen before.

Learn more about Arm Corstone-320 here.