The Intersection of AI and IoT on Arm

Geof:  Welcome back to the Arm viewpoints podcast, and it’s going to be a good one. We’re going to talk about the internet of things, or IOT as it’s known more commonly with an old friend of this show, Paul Williamson. Paul is the senior vice president and general manager of the IoT line of business at Arm. This is just the latest in a number of key roles. Paul has played at arm. Welcome back, Paul. And I have to note that Paul is now the most frequent guest on this podcast, with today marking the fourth time we’ve had the good fortune to talk to you. So welcome back, Paul.

Paul: That’s fantastic. Thanks so much, Jeff, and always a pleasure to talk to you.

Geof: It’s always an interesting discussion. So why don’t you start by telling us a bit more about your current role, and then we’ll get into some fun questions.

Paul: Sure. So as the leader of the IoT division at Arm, my team together define the compute that serves so many of the devices around us, everything from the smallest and simplest motor controllers or thermostats on the wall to complex and advanced compute devices for things like robotic vision and automation in industry.

Geof: So we’re going to start with a bit of fun, as I said. We played our lightning round game the last time we had you on the program, in your role as head of the Arm client line of business, and it was highly entertaining. So we’ve got a few new warm up questions for you this time. The first is a variation on Desert Island Discs. In this one, you’re trapped on a desert island, but we want to ask, what one book do you want?

Paul: That’s interesting. So, I’ve always been a crime, well, sort of spy fiction novels person. If anyone out there has been watching the Slough House series on Apple TV, then Mick Herron is the author and his new book is out very shortly. And I’ve been waiting for that one eagerly. So that would be probably the one I’d want most at the moment is to find out what happens in that interesting saga.

Geof: Fascinating. And I actually have been following the same series, so I’m excited that you said that. Okay. So, next question. If you could use technology to solve one problem in the world, what would that problem be?

Paul: I think the area that I’m really interested in at the moment, and it’s one that we’re thankfully being able to be involved in is energy, energy and the way we can continue to evolve and develop our businesses and our lives, but consume energy carefully. So whether it be an incredible technology that just changes the way that we can generate it with less impact on the planet, or the areas we’re working on where we’re helping to. You know, consume it more intelligently at the right times of the day and to use the latest renewable energy more, more frequently rather than relying on gas and coal. I think that that would be really beneficial to the planet and something would be amazing if I could snap my fingers and have it solved and deployed everywhere.

Geof: Yeah, we’ll look forward to that. Okay. Okay. Last one. You can dine with either Alan Turing or Stephen Hawking. Which one and why? And just as importantly, what would you have for dinner?

Paul: That’s a great question. And I think I have to say, much though I have huge respect for Stephen Hawking, and I did once sit next to him in a cinema, which wasn’t very interactive. Alan Turing is the person that I’d be most interested to speak to. And I think that’s partly the innovations in computing and obviously that being a core area of my life and my, and the business here at Arm, but also just thinking about the personal tragedy around him and the discrimination he faced in his life. And to show him or talk to him about the way the world is emerging and whether that’s getting better and the opportunity that provides for the world as well. I think that would be a fascinating discussion. As to what we’d eat, I have no idea what the guy would enjoy, but I, I, I think on a personal level, I recently had the pleasure of one of our customers buying me an incredible Japanese sushi dinner with a chef ironically in Taiwan. And it was just fabulous. So, repeating that dinner with him as a guest in a lovely private dining room would be an exceptional experience.

Geof: I think that sounds amazing. Okay, well, hopefully you’re feeling warmed up now, Paul. Now comes some questions that are specific to the world of IoT. Since your new job represents a return to the world of IoT, I’m going to ask you to cast your mind back to when you first started working in IoT, and how has the industry changed since then?

Paul: The IoT business, I suppose I first started working in it when I was working in a company called Cambridge Silicon Radio around 15 years ago or so. And at Cambridge Silicon Radio, we were in an era where we were moving from audio connectivity to smartphones to starting to think about how can we connect more things to smartphone and the emergence of Bluetooth low energy as a technology, which I was fortunate to lead at CSR really broke the mold and allowed us to create things that connect and we could collect data from in a new way. So we were able to see the emergence and, and deployment of things like, you know, Fitbits. And for me at the time, Nike fuel bands, which were collecting personal data and relaying it to apps, being able to display it, giving you a really powerful user control in your hand. This created a change in the compute needs of these embedded devices. Suddenly they had to have more complex software stacks, more memory to be able to operate. And that really drove a step change in an evolution that continued for the subsequent 10 years. And those devices have got more proliferated around our homes with more smart home technology. But they have become more valuable. They now interact with each other through the cloud and can actually be more intelligent than the individual device. And that’s caused ever greater complexity of standards. So we’ve seen the emergence of newer standards now that are trying to pull together these in home devices and these personal connected devices to share data. And that’s, once again, stepping up the need for more compute, more memory and more power in these devices. So it’s been an evolution and a trend that progressed for quite a long time. But we are, I think on the verge of a new wave and that’s going to be interesting.

Geof: So you mentioned intelligence and intelligence in the devices. And artificial intelligence, or AI, is the buzzword at the moment, mostly around generative AI powered by data centers. But AI and machine learning are having a profound impact on IoT, and much of it is starting to be processed at the edge. So let’s talk a little about what’s behind that.

Paul: Yeah, so AI is increasingly being used for everything. And AI today is happening on Arm. In fact, everywhere AI is happening, Arm is present. In fact, 85 percent of smartphones are running machine learning workloads on the CPU itself rather than using any kind of specialized technology. So it’s already widely present and we see it in IoT in the home with devices like our Alexa for voice control and using AI to recognize our voices and respond to us. So it’s definitely becoming more and more pervasive and that’s changing again, as I said, there is a need for the compute capability in these IoT devices, they’re moving from needing to do simple sensing to more advanced sensing and making more use of machine learning as a key workload to solve the problems they face.

And that means more complex software and also connectivity to the cloud. And I think actually a smart speaker is a good model, but shows how you’re having to increase that compute and software complexity and also the connectivity that comes with it. But it is also going to apply in many other even simpler things. Even those simple motor controllers or thermostats I described are going to be using machine learning for analytics in the future. So we need to bring inference down in a cost and power effective way into all of those devices.

Geof: Yeah, and as you mention those devices, you remind me that over the years, IoT has seen an immense range of application possibilities. Where do you see right now the most interesting growth opportunities in terms of IoT deployment? What types of applications have captured your imagination?

Paul: I think most recently I’ve been on a bit of an adventure into the world of industrial IoT and also smart cities. When we think about a Smart city in the scale of smart city and something like electrification of vehicles and bringing smart charging infrastructure into that broad city environment, initially you think about the post that you’re going to plug your car into and the fact that’s going to need, compute capability to handle charging or payment and user interface to control the charging. But then you go beyond that individual device and you think about the more mundane things that are hidden behind the scenes, something like a substation, the amount of control systems and switching control gear and power management for the network that sits behind that, that needs to be connected to deliver these higher levels of current quickly into new infrastructure across a city. And then you’re coordinating that with the grid, with power production across the whole of a city or across the whole of a nation. There’s a huge amount of connected infrastructure that’s going into that. So, learning more about that has really opened my eyes to the opportunity for us all in improving these systems, but also the complexity and the amount of work we have to do to put them together. Yeah, complexity and a lot of work.

Geof: Designing and developing for the edge means balancing a lot of factors. You have power, space, and connectivity considerations just to start with. So what’s your sense for how the world’s evolving there?

Paul: I think the one common theme I’ve mentioned is the ever continuing demand for more compute in these devices. But I think you raise a very sensible point in addition to that, which is you’ve got to do that across this full scale. You’ve got to do it across this sort of very small little sensor that might be running off a coin cell all the way through to these huge data centers with immense amounts of power, so processing that compute takes a lot of a lot of work. It also means you’ve got to think about other considerations beyond that around the software infrastructure or the frameworks that are going to be in place to allow developers to access the compute that we’re putting into more of these devices. So we’ve been thinking a lot about machine learning and AI in particular, and how to bring that down into deeply embedded devices. So we’ve done a couple of fairly significant things over the last couple of years in our Cortex M line of processors, which are the really small, low power processors we have. We’ve introduced new instruction capabilities that accelerate vector and matrix mathematics, which makes it more efficient to run these machine learning tasks in small, low power devices. And then we’ve also brought in a new line of accelerators. under a brand called Ethos. And what those allow you to do is go even more optimal, um, on power. So they’re even more power efficient when your main operation is machine learning, then they can run at an even more efficient power point. This is something that is allowing us to bring ever more performance, even while the power envelope remains very constrained. But looking more broadly, going back to those industrial segments, we have areas where we’re not as power constrained. And there we can use a mix of different performance IP to bring together a solution that is optimal for that space as well. So, more deployment of Linux software development in those environments. We’re seeing more of our A class higher performance processors be deployed and even GPUs, as I said, to handle things. Not only for machine learning tasks, but also for having user interface on those devices. So, it is a full suite of IP that’s required to solve these complex problems.

Geof: Yeah, and when you talk to designers and developers who are putting all these things together, what else are they asking for or curious

Paul: Yeah, it’s funny you should ask about that because we’ve actually recently completed a survey in this space. We went out and spoke to about 600 developers, managers, and executives, and they’ve been looking at this problem and they’ve been telling us a bit about what it takes for them to compete in their own markets and, and the opportunities they have to grow. And one thing’s clear, they’re tired of fragmentation. It’s really important that when they develop their software and they don’t want to be troubled with individually porting or picking up new environments or moving to a specific platform.

They want to be able to just move faster and innovate faster towards their goal. So, standardization in the software frameworks has been really critical. So building on the same CPU architecture supports that goal, but as they innovate faster to be effective, they’re also looking to, as the survey showed, embrace standards. And there’s a couple of areas that they’re doing that in, but one of them in particular for us is security, making sure that you can secure your IoT device is critical. It’s clear that if it’s going to live in the field in something like that smart city infrastructure, it needs to remain updated and secured for its lifetime. So being able to take common software frameworks and access to security with the initiatives that we’ve been driving around things like the platform security architecture at Arm, really helped them in that goal of achieving security in their device development. So we talk about security and standards overall.

Geof: How critical are standards based initiatives for enabling innovation, particularly in relation to IoT?

Paul: Yeah, I have an analogy I think about when you think about how a network works in IoT, it is a network of devices gathering, collecting information and actuating and making decisions. And when you are doing that, if you’re building out your distribution center for some kind of logistics program for parcel delivery. When you’re controlling that and you’re switching these motors to coordinate all of the devices in that big, big warehouse, you need to be sure that the information you’re receiving, and that you’re acting on, is information from your sensor, not something else in the system. You don’t want to be going and powering things down or causing things to crash into each other or ignore their control because of rogue data that you’re receiving. So you need to be able to validate and trust the data on which you’re going to make decisions. And particularly when you’re using machine learning and there’s not a human in the loop to observe or take care of these systems as they’re getting bigger and more complex. Beyond human scale problems, you’re going to have to be able to trust the data that you’re acting on. So securing the device and being able to authenticate that the data that you’re acting on is coming from the device that you care about is really critical. You can’t innovate and change these systems to more advanced machine learning based systems, unless you have that foundational trust.

Geof: So moving beyond standards and turning to IoT and Arm specifically, what does it mean that IoT runs on Arm? And maybe let’s talk about some cool examples.

Paul: So yeah, IoT runs on ARM is our way of thinking about the interaction we’re having with that. That very vast software development community, over 15 million software developers globally who are writing software to target Arm-based platforms. And we recognize that they need to be able to have that trust in those base platforms and they need to have access to the right tools and software to be able to get the performance they need for their innovations. What we need to do is make sure that we’re giving them that access that they need to be successful. So there’s a range of things that we see as examples of people making use of our technology and the areas we’re seeing that are really interesting are things like smart home. We’ve discussed already a lot about smart city and industry and warehousing and logistics. Another area that’s fascinating is the smart home/ And so innovations like the matter protocol, which I referenced earlier in talking about how smart home devices talk to each other, are showing the ability to bring more of those devices into our home and have them work more seamlessly together with that level of trust across standards. So rather than having an individual app or an individual provider for each one of these different things and having to sort of treat them separately, we can actually bring their intelligence together so that they can be designed to live around us rather than us having to sort of live around controlling them. And I, I think that’s pretty cool thinking that our homes are going to react and be more responsive to us in future, rather than us have to manually interact and individually task each device with what it needs to do for them to understand us as people, our preferences and react to us I think is fantastic, but I think the further layer that excites me that sort of wraps this back to that example of electric charging and energy that we discussed is if they can do that in a way that also helps the world. If your devices can decide to use electricity and sip at it when coal fired power stations are on, but guzzle it down when the wind power is channeling through the wind-storms to give a surplus is something that could change the perspective on solving some of the global problems as well. So making our home smarter will make them more livable, but it will also hopefully help us towards goals like improving their energy efficiency. And I think that’s, that’s really exciting.

Geof: Yeah, it is exciting. So I’m going to wrap up by asking what we can expect from ARM going forward when it comes to AI and IoT?

Paul: Today, as I look at it, the most innovative organizations are leveraging AI to solve problems in new ways. I’ve mentioned a few today, and I think you’re going to see Arm feeding that demand for AI and helping to ensure that it’s available on the broadest range of devices from those high performance down to the most power efficient devices. And we’re going to use a range of tools that are at our fingertips to bring together example reference systems to show how you can achieve. Machine learning in embedded environments and ethos accelerators are probably the best example of really squeezing every power optimization out of this to allow you to deliver ML in even the very smallest device. I think we’re really just scratching the surface of the potential of AI and machine learning applied to all sorts of products. And the more that Arm can solve this problem of putting at the fingertips of innovators everywhere means that we’re going to see it opening up a new future for ML and AI running on Arm.

Geof: Thanks, Paul, for your insight on IoT, for helping us all understand a little bit more about what makes you tick, and for being our first four-time guest. It’s always great to have you here, now and in the future. And speaking of the future, we look forward to bringing you more news in the next episode of Arm Viewpoints, and look forward to connecting with you all again soon. Thanks for listening today.