A Smarter TV Experience, Powered by Arm

Arm technology already sits at the heart of over 600 million home consumer devices—smart TVs, set-top boxes (STBs) and games consoles—shipped every year. Over 95 percent of smart TVs are based on the Cortex family of processors. Just like anyone’s favorite box set or series that is being binge-watched, the home technology category is incredibly important to Arm and what we do.

The astronomical growth of content providers and streaming services, such as Netflix, Amazon Prime Video, YouTube and Disney Plus, has helped the continuous evolution of today’s smart TVs, which now account for 75 percent of all TVs sold worldwide.

The smart TV is evolving into a multifunctional device with a far richer user experience, not only including high-quality video streaming but also other new use cases, such as video calling, health and fitness applications and gaming. These are all supported by the availability of high-speed broadband and more capable hardware platforms in the smart TV itself. Leading to more connected and performant smart TVs that can access more services and enable more use-cases.

High-speed broadband and high-performing hardware

The widespread availability of high-speed broadband means that many viewers can enjoy their favourite shows in pixel-perfect 4K through the various streaming services, which are important influencers in the smart TV market, driving new user experiences and capabilities. This is causing the smart TV platform to evolve, with higher-performing hardware solutions needed to ensure that consumers can enjoy these expanding capabilities.

Today’s smart TV processors need to be able to handle a wide variety of use cases including multiple video streams while delivering high-resolution user interfaces and the responsiveness demanded by a connected device. At the same time, the resolutions and frame rates have continued to increase to enable better picture quality, which places increasing demands on the processor.

While most of the smart TVs shipped today are 4k resolution (with 8k also starting to ship), the majority of the content is still 1080p or below. The smart TV processor needs to upconvert this lower resolution to the native resolution of the display. Due to the increased resolution and screen size, traditional scaling methods are not sufficient to provide the desired quality.

This is one area where Arm is seeing a lot of innovation, as smart TV manufacturers reassess the IP required to deliver high-quality pictures using newer methods such as artificial intelligence (AI).

Artificial intelligence powering new TV use cases

AI is rapidly becoming ubiquitous throughout smart TV functionality. It’s being used for picture quality enhancement, with super-resolution emerging as an important technology in the picture quality process. This uses a combination of AI and traditional video processing techniques to achieve a high-quality picture when upconverting the video sources from lower resolution to 4K (and indeed all the way to 8K).

AI is also being used to recommend new content on streaming services through powerful ML algorithms and bring virtual assistants to the big screen. In addition, through the re-appearance of cameras in smart TVs, we are starting to see AI smart camera use-cases such as fitness applications and richer video calling experiences.

The need for highly-capable AI and Machine Learning (ML) technology across all of Arm technology was reflected in Arm’s latest IP launch, across CPUs, GPUs and NPUs.

Cloud gaming enables gaming without the console

Cloud gaming is an important new use case for TV. Various games will be made available through smart TVs from providers like Google Stadia, which began shipping in November 2019. In cloud gaming, rendering of the virtual world is performed in the cloud and streamed to the TV. Cloud gaming services will also require a high-performance user interface, which again creates more demand for performance from the smart TV processor.

In addition, for cloud gaming to work effectively, the latency through the entire network and device is critical. This may be difficult to achieve in all environments, so we could see some hybrid approaches to cloud gaming where the game engine is in the cloud but the final rendering is done on the device.

Cloud gaming is also bringing more attention to smart TVs as an important device for developers. We are seeing the development of gaming apps that are specifically geared towards running on smart TVs and set-top box (STB) devices.

Hardware security is more vital than ever

Premium content is coming directly to TVs earlier than ever before through traditional carrier services and the new streaming services. For TVs, ‘content is king’ with huge competition and differentiation in the smart TV ecosystem over the content that’s being provided. With video being operated on by more processing engines in the smart TV chipset, the big challenge is ensuring that all this content is kept secure across the entire platform.

Just like the content, smart TV picture quality is a big market differentiator. Therefore, TV OEMs also need to protect the AI and picture quality algorithms that make these visual improvements possible.

In addition, there is now more user data on the smart TV, highlighting the need for greater security. New applications, such as for social media accounts, payment and video calling, mean that personal and visual details are being accessed on smart TVs. This is leading to new smart TV security requirements to protect user data, just as we do on smartphones today.

The television as a central hub for the smart home

Traditionally, the smart TV was a closed fixed-function device where users watch specific channels at specific times. Now it’s a multi-functional device for consumers, with more use-cases and experiences. Even allowing third-party developers to create and add more applications and experiences to the smart TV, just like the smartphone. As the ecosystem continues to become more mature, the largest screen in the house could be an ideal central control hub not only for immersive experiences but for device management and home security. Arm’s family of scalable, compatible and performant IP sits at the heart of the more enriched and advanced experiences offered by today’s smart TVs.