The Potential of AI and Machine Learning

Geof Wheelwright: We’re back with another Arm Viewpoints. Today, we’re going to talk about something that plays more of a role in everyone’s daily lives than perhaps they realize, artificial intelligence and machine learning. Joining us to talk about these technologies and the role that specialized processing has to play in their growth is an old friend of this podcast, Dennis Laudick.

Dennis is vice-president of marketing at Arm. He has more than two decades of experience in the machine learning, AI, mobile, automotive and consumer electronics industries. His leadership experience at Arm has included roles within the ML group and Arm’s GPU business. Dennis is also known for his broad understanding of customers’ needs in bringing new technology and exciting use cases to market as successful products. Welcome back Dennis.

Dennis Laudick: Great. Thank you very much, Jeff. It’s a pleasure to be here.

Geof Wheelwright: When we last spoke, you talked about Arm’s background and role in machine learning and artificial intelligence, and whether you believe the hype cycle was over. So I suppose a good question to kick us off would be to ask what’s happened in the last year. What does the landscape look like now?

Dennis Laudick: Yeah, that’s quite an interesting question. I mean, it’s fair to say the area of AI continues to be quite an exciting area, quite dynamic. A lot of experimentation and frontiers are still being pushed. I think probably that the hype cycle question is still an interesting one to focus on.

I think a lot of people last few years have been asking, are we riding out a traditional hype cycle in the AI technology? And in many ways we are but the most more interesting things I’ve seen is that we haven’t really gone through what I would refer to as a trough cycle.

To look at that more it might be helpful to kind of split it and turn it to the technology and the applications from a technology perspective. I think what was referred to as a hype cycle initially was actually just the unknown. If you look back a few years ago, the industry was still trying to understand this new capability suddenly had an ability to generate almost a universal function.

We didn’t know what that meant. We didn’t necessarily even know why it worked and we certainly didn’t know what was capable or what it was capable of. So I think that’s kind of settled down. I wouldn’t call it a trough, but I think it it’s calmed down now the industry understands it more.

We can dimension it more. We understand the capabilities and why and how and where we can possibly go with it. But what we haven’t seen in my mind at least is any slowdown in the application. We find more and more capabilities, more and more novel ways of approaching or using this tool.

And you know, in a world where there’s huge amounts of data, we can suddenly bring value and use to all of that as well as do lots of things that we couldn’t do before. So I think that’s quite exciting and that continues just to go from strength to strength.

Geof Wheelwright: So do you see it revolutionizing different industries?

Dennis Laudick: I see AI sort of revolutionizing almost every industry. I mean, some of the industries that I’m really excited about are energy conservation, and medicine, everything from devices that help us do a better medical diagnosis through to macro level predictions through to even vaccine development. On top of medicine, there’s energy, we all know that we’ve got issues with needing to be more energy conservative and the way we approach AI can bring a lot of help to this.

There’s a lot of instances where AI is making things better. A report I read the other day from an IDC mentioned they were talking about the industrial and commercial sectors only and they predicted that when using data and analytics which is essentially AI, they expected that there would be a 3% reduction in emissions from the industrial and commercial sectors by 2024.

Now you might think 3% – it’s not a huge number in its own, but this is at a global level. If you think about it that way, I think it’s something like the equivalent of getting 30 million cars off the road and that’s just in the next couple of years, and that’s just looking at one sliver of the segment.

So that’s huge and the other thing I think is quite interesting too is not only will it allow companies to do their part in terms of bringing down emissions and bringing up conservation, but it will allow them to do it in a way that will actually allow them to make money in many cases. So that’s kind of a win-win for everybody.

And again, we don’t really know where the end of this capability is. I hear every day or every week about a new application for AI, that’s helping with the planet or medicine or healthcare. Data centers is another good example; datacenters are growing and becoming a significant footprint in the world.

Energy costs, electricity costs are some of the highest operating costs and again, we’re seeing AI be used in a multitude of ways to try to make this more efficient. So if you look at companies like Google and others that work at a hyperscale, we see them using AI and others are really relevant software.

Look at distributing workloads both in time and in locations so that they can happen much more efficiently and reduce the overall footprint.

Geof Wheelwright: So what common themes are you seeing?

Dennis Laudick: You know, we’re seeing it go everywhere. I think one of the tell-tale signs for me is what is our partners are doing. What I’ve noticed over the last year in particular is a lot of our partners aren’t necessarily referring to AI so much.

They’re now referring to the use cases, the things you can do. So that’s it. I think for me, that’s a good sign that things are starting to normalize. We’re starting to see this as a tool rather than a mysterious AI ubiquitous term. People are starting to see the utility in the specialization of it.

I think one of the unique aspects of AI versus technology waves we’ve had in the past is it affects everything vertically as well. So it’s not just about a new capability in devices. We are seeing AI go across all the devices but not only that but they can be used at a macro level.

You’re looking across all of these devices, the world’s becoming a massively distributed compute world and you can use AI everywhere from that very macro level down to the micro level. So it’s pretty exciting and it continues to continue to move in.

Geof Wheelwright: Dennis, you referenced distributed computers. Now, can you describe how Arm enables specialized processing and design flexibility to enable teams to put the right compute at the right points from cloud to endpoint?

Dennis Laudick: Yeah, that’s, it’s a good point. Distributed compute is something we’re seeing more and more of and alongside that you need a different range of capabilities.

I think this is something we’re seeing generally in competing. We’ve seen it very acutely in the AI space. When I joined the AI aspects arm, one of the biggest things we needed to deal with was the misnomer that to do any sort of AI, you needed to have a specialist processor, and for any company that’s focused on doing efficient compute like most Arm-based devices are, that’s simply not true.

From our perspective everything can run on a CPU. The CPU is the universal tool. All software runs on that and it only makes sense to go to a kind of dedicated hardware where you need a very high performance where if you know ultra-efficiency is really important. That comes with a certain amount of cost as well.

In terms of silicon and complexity, not only AI for any type of processing, moving to additional dedicated hardware only makes sense where that’s necessary, where it’s really justified by that the needs. As I said from what we see, a lot of AI actually runs on CPUs and that’s kind of the major focus for us in terms of our AI capabilities.

We need to make sure that that CPU is as capable as possible. You can look at it just in terms of process of elimination. So say smartphones, smart phones is a big area for Arm. It’s one of the most popular devices in the market and in most phones today, smartphones have some sort of AI capability.

And if you look at those in terms of the hardware, we estimate less than 20% have a dedicated AI processor. So for that vast majority of the devices, they’re still realizing the possibilities of AI around cameras, voice and so forth. But they’re running that on the CPU. So for about 4 billion people, they’re using AI today on an Arm CPU.

So that’s where we got to need to focus to start with. You’re drawing back a bit more from an Arm perspective. What we’re seeing is a couple of different dynamics. So everyone’s kind of talked about the fact that Moore’s law is slowing down. You can argue about how much, but generally speaking we’re not seeing the same kind of linear progress that we saw in parallel with that.

What we’re seeing is a continued demand for more complex workloads. So we’re seeing people wanting to run more sophisticated capabilities and compute AI is to send things through the roof. That in addition to a vast range of devices that we’re seeing, we’re seeing everything now from a sensor through to supercomputers, which is just a range we’ve never seen in history before.

So given this range and these demands, we’ve really seen what we call specialized processing or specialized computing. And the concept behind this is that you need to more closely tailor what you design to match what you’re trying to do. You can’t come at devices today with a one size fits all solution.

So we’re saying that the part of the reason that the Arm ecosystem has been so popular and continues to move into more and more markets and become popular in those markets is because it allows you to specialize. It allows you to get the maximum capability out of the least amount of cost and power.

And from an Arm perspective, we’ve got a huge range of devices. We need to try to service everything from sensors that run on a coin cell battery for years. Through to devices we have around us, in our homes to phones, cars, and up into supercomputers. We’re constantly looking at how we can improve our CPUs and GPUs.

We also have a range of specialized processors dedicated to ML, and we have a long history in all of these devices. Five years to ten years of improvements in each of those. Wuite significant ones, the 10x type improvements, generational on generation. We work on giving people the tools to develop what they need to all of our processes tend to be configurable.

Some devices, we see a tiny minuscule CPU in it. Others will see eight very large cores. In addition to that, we spend a huge amount of investment on the software and tools. You know one thing at Arm that we’re very aware of is that the importance of ecosystem.

In modern semiconductors it’s very complex. We try to make sure that everybody has all the tools that they need to be able to create a right size type of compute element. From our perspective, one common theme that we’re seeing across every industry is that everything’s getting smarter. There’s more demands for compute and there’s more demands for AI capabilities.

We’re seeing devices that already use AI. More devices that didn’t use AI are now using it. People are putting AI standards into sensors. In addition to that, One of the things I really like about working at Arm is how Arm has always been an open book. We make sure that Arm doesn’t do things exclusively.

We make sure that the Arm platform works with people’s processors. If they want to use a dedicated AI processor, their own, we make sure that that’s really easy and it’s going to work well for you. So from our standpoint, it’s a wide range of processors with a huge amount of flexibility and the software and the tools and the ecosystem to make that easy.

That’s kind of what’s made our team successful. That’s kind of what we’ve been doing since day one and it just becomes more and more important as time goes on.

Geof Wheelwright: So, if you look at the history of computing, the one thing that comes out and you mentioned this earlier is the fact that the full potential of a technology like AI will only be achieved with the support of a strong ecosystem of companies. So maybe you can give a bit more detail on what this means and why it’s essential?

Dennis Laudick: Yeah, ecosystems are definitely the way of the today in the future. I mean, if you look at the technology people produce today it’s incredibly complicated. The hardware is incredibly complicated, the software is incredibly complicated because the capabilities that people are expecting continue to go up.

And so innovation continues to if anything accelerate. It’s increasingly difficult for companies to be able to do the entire thing. They need to be able to specialize what component they can really focus on bringing to that vertical technology stack? That’s going to make us stand out..

I think the days of being able to create a complex device on your own are long past us. AI is really proving that it’s very difficult for a company to be really great at making semiconductors, really great at software, really great at AI algorithms. And not only that, AI is kind of propagating across more and more use cases and those are specializing.

We’re seeing people specialize in particular use case areas for AI. So again from an Arm perspective, this is kind of how we’ve done business since the beginning and why we become successful. We focus on making sure we’re an open platform and we have a huge amount of value and respect for the ecosystem around us.

In many ways we know that a lot of the value we bring to our customers. So we do a lot to nurture the ecosystem and the developer community and make sure that their experiences with arm are going to be the best that they can be. And then everybody feels they’ve got a right to play and then everyone can build value on top of an Arm platform.

So we nurture relationships up and down the technology stack. Not only is that ecosystem necessary to bring devices together, but there’s a huge amount of knowledge and information to be had by understanding the different layers of the technology stack. So this approach has made Arm successful around the Arm ecosystem.

Again, that’s something we brought into the AI space as well. We’ve got a growing range of partners that we’re working with around AI. Looking to understand that the valuable information they can bring but also making sure they can be successful and build value on top of an Arm platform and build a successful business out of it.

Geof Wheelwright: Now, you reminded me when we spoke last time that you gave a great example of AI in action with a company that produced an energy efficient refrigerator system over the last year. There must have been many other exciting use cases of AI that you’ve come across. Are you able to share any areas that you have seen mature with their AI progress since we spoke last?

Dennis Laudick: Yeah, I mean, I mentioned a little bit about this earlier on but the areas of innovation and the improvements that people are seeing with AI, it’s just incredible. And there seems to be no end to it. If anything, it’s accelerating, I’m really excited about some of the benefits that we’re seeing in energy efficiency and healthcare.

One company I find quite interesting is a company called Span. They develop what’s referred to as a smart electrical panel. So this is the grey circuit breaker box that sits in your garage or your attic that’s been pretty much the same for a very, very long time and they are revolutionizing an industry by bringing smart electrical panels to the market – based on Arm processors.

But the way they work is they recognize the different electrical signals of different appliances in your home. So the classic situation where you’re trying to remember if you left the curling iron on or the clothes iron on, it allows you to be able to check and to turn off the circuits in that room.

And in future versions, I’ll even bring that down to the individual appliances. So you can extrapolate the value of this smart capabilities in the panel. If you’ve got a home that has some sort of battery storage which is becoming more common, or if you’ve got solar panels or if you’ve got like an EVP vehicle, electronic vehicle, the insights and efficiency it can bring to the panel and to the user is incredible.

Mostly in many cases it’s using AI. So that’s going to be huge. That’s just one example. And as I said, you know, the, the company referred to Arcelik refrigerators, industrial pumps. There’s some massive predictions in terms of electricity on how we can save the planet.

Personally, healthcare is a big one for me. I see it happening at device level. Being able to make sense of the huge amount of medical data we got and build better predictions and health outcomes for people in the environments. Another big ones is energy conservation and making the environment a better place. One company stands out recently I was reading about was a company called Telesense out of UC Berkeley.

They’ve got I think it’s a Nordic Semiconductor based Arm chip that has a pin-shaped sensor that detects CO2, moisture and other different parameters around grain silos. And they use it to manage the storage and shipments for grain silos. I don’t know if you know but there’s something like 35% of grain is wasted every year and so there’s a huge amount of efficiencies to be made there and again, AI is an ideal tool for being able to approach that from multiple directions. And they’re working with that as some large companies like Cargill and Archer Daniels Midland, who are taking this out to scale and it’s really exciting to see this kind of these applications in the food and agriculture.

The barrier, the regulatory barriers for an area like that are much lower than they are for something like healthcare and you can get to the benefits much faster. Like I said, you can go in every access, public safety is a good example. California wildfires becoming more and more regular in lots of places they are unfortunately and there’s a couple of companies working in this area. We work with a company called Edge Impulse. They have a sensor that uses AI in the sensor itself to detect fires early. Another company called Pano AI uses a camera to do similar to help everyone be able to identify fires earlier and react to them quicker.

So almost every industry is getting impacted by AI. It’s quite incredible. I think the benefits are just starting to be realized.

Geof Wheelwright: So Dennis, that broad range of benefits reminds me of something, a tech futurist, Matt Griffin, another friend of the show has said, and that’s about seeing global innovation accelerated by AI and ML. How do you view an increasingly autonomous computing world and what might the consequences?

Dennis Laudick: That’s a very interesting question. So it’s useful to start off by saying, right, what do we mean by autonomous? And a lot of AI is about autonomy. It’s used for predictions but it’s from a device perspective. There’s a lot of autonomy happening and I tend to think that most devices will have some degree of autonomy.

We’re not talking about robot butlers and flying cars and things like that. What we’re talking about is devices that can make some rudimentary decisions. The quintessential example is there a cat or not in this photograph? But you know, that’s a metaphor for many different types of analysis you can use with AI.

So I think AI is going to revolutionize almost every industry. Again, it’s not going to be robot butlers and flying cars but it’s going to improve every industry one way or another. And I don’t know where the end of that would be. If you look at our world in many ways, it’s generating more and more sensor and sensor information and more date.

AI makes sense of that and it brings value out of that. I mentioned medicine earlier, I’m really excited about what we can achieve. In the future, conservation and power saving agriculture. These are just some examples you can go into almost every industry and it can be improved with AI.

If we look at autonomous devices I think one of the biggest things is the IC. What I call autonomous is the revolution in the UI. Touchscreens were kind of interesting as when touchscreens came out, they kind of changed everything. There’s a massive revolution in the UI space and we look nowadays and touchscreens are ubiquitous.

They’re literally everywhere, industrial, commercial, your cars, your homes, they’ve gone everywhere. I think that was a huge revelation in terms of devices. AI is going to bring the next big revolution in UI, which is the devices are going to learn to interact with us. Do you see this in their vision capabilities and their language capabilities?

When you talk to automated assistants on the phone, you’re seeing this already. And the interesting thing about this is this isn’t us learning about a new UI. This is the UI learning about us. And I think we’re going to see this across more and more sectors, industrial medical in our house, vehicles, I think it’s quite exciting  what we’re going to be able to see.

So I think the functionality is almost endless and I think the benefits are going to be huge and it’s going to be an era when people look back on, in time as being a massive revolution, across many different aspects of technology.

Geof Wheelwright: Fantastic. Thank you, Dennis. You’ve outlined the vast potential of AI and ML. Although I’m a little disappointed by the lack of robot butlers and flying cars. I am excited about where artificial intelligence and machine learning will take us. And I’m sure our listeners have a much better idea of how their lives will be impacted.

Thanks to everyone for listening today. We hope you enjoyed it and look forward to joining you again soon on the next episode of Arm Viewpoints.