How Arm Neoverse is Redefining the Global Computing Infrastructure: Part 1

Geof: Hello and welcome back to the Arm Viewpoints Podcast. Today we want to talk about something that affects not just those of us with a passion for technology. But a topic that affects everyone who uses or relies on any kind of connected digital technology. It’s why we’re calling this episode redefining global computing infrastructure, and it’s about nothing less than looking at the vital underpinnings of our digital future as there’s so much great stuff to cover.

This is a two-part special. In this part one, we’re going to focus on the latest Arm infrastructure roadmap announcement. Joining me to talk about this is Dermot Driscoll, Vice President of Product Solutions from the Arm infrastructure line of business. His job involves the definition, delivery, and successful deployment of Arm-based products across global infrastructure markets. To do that job, he’s become an expert on data center and networking applications and leads the team responsible for Arm ‘s, IP, and software products in those markets. He works with Arm ‘s silicon partners to support the development of their products.

He also engages in customers in the cloud and telecom industries to ensure successful deployment of efficient Arm -based solutions. Welcome Dermot. Thank you

Dermot: Jeff. Very excited to have this conversation with you today.

Geof: So we know a little bit about what you do, but let’s find out a little bit more about who Dermot really is. We’ll start with our lightning round of questions to reveal the inner Dermot. Give us one technology you couldn’t live without.

Dermot: So beyond my phone and my Arm-powered laptop, it would have to be my Garmin watch. I’m a bit of a sports addict. I work out pretty much every day and everything gets tracked in there.

Track there. Downloaded, uploaded Strava. Anybody who wants to know more about me should probably look for me on Strava. It’s my one and only social addiction.

Geof: Now, tell us about one place you’d love to live in for a year.

Dermot: Oh, gosh, that’s a tough one. I love the sea and I love the mountains.

But I guess Colorado would be the place for me. Beautiful mountains great people. Snow sports, summer sports. Yeah I’d go with Colorado.

Geof: All right. Now I’m going to ask you to fill in the blank. I’m happiest when I’ve just eaten …

Dermot: Paneer Tikka Masala.

Geof: There we go. That sounds pretty delicious. One thing about you that most people don’t know, but what surprised them?

Dermot: Depending on which side of the Atlantic I’m on, I either have my original Irish accent or my adopted Texas accent which has come from 20 years here. A lot of people don’t know that I can actually speak the Irish language. I’d say that’s something unusual.

Geof: If price was no object, what would you like to buy?

Dermot: Selfishly, I think I’d love a one wheel. I don’t know if you know what those are, Jeff, but those are those little, they’re almost like skateboards, but they have a wheel in the middle and you can balance them. I see people riding those things all the time and think am I too old to learn how to ride one of those?

I’m not that old, but I do worry about the collateral damage of coming over one. Maybe that’s what I’d go for.

Geof: I don’t know that looks to me like a unicycle territory, but maybe it’s a little easier than that.

Dermot: Probably requires a little more balance than I have.

Geof: Yeah me too. Me too. And a really good helmet. So why and how did you first get into technology?

Dermot: So my older sister I’m one of eight kids. Geof and my older sister tell this great story about how she was electrocuted by me. Rewiring her hair dryer as a, I think I was probably seven or eight years old at the time.

The following story to that is I also subsequently almost electrocuted myself trying to fix my parents’ stove. So I was definitely a tinkerer. I loved taking things apart, trying to figure out how they worked. Usually had to fix them. That was generally my, my mo. But yeah, so that, that was very basic, obviously technology, but it went from there.

Geof: Yeah, no, that, that sounds like a great kid thing of what would happen if I plugged this into that?

Dermot: I was totally, I took my parents my, my dad had this very expensive record player, and I took all the speakers of that, and I wired them around my house with, cables so that we could have music in every.

This was before there was a Sonos. We had, a switching system that was pretty complicated to figure out, okay, do I want music in the kitchen, or do I want it in the living room? Or where do I want it? So that, that was totally my game. I love tinkering and playing with stuff.

Geof: So what do you love most about technology and what do you love most about Arm?

Dermot: I love that technology’s unlimited. It opens so many doors, it open door, it opens doors. Every day it, I’m fascinated by technology. I have two teenage girls and like a lot of teenage kids addicted to their phones and have no idea how it all works. And I just love to walk them through the, everything from the software application they’re playing with, to the operating system that’s underneath it, to the hardware facets that it takes to put all that together to the chip to all.

So that whole like, complexity, like it’s so cool how complex, the stack we’ve been able to build as engineers as technologists, and It’s also very simple. It’s so simple that you can explain it to people, and you can, sometimes it’s analogies you’re using, but I, but the power of technology to enable people to, to do what my kids like to do, which is, watch TikTok, but also to, to change the world, just, from automated driving to, how we manage crops, all of that stuff. So to me, technology is the ultimate equal. It allows us all to, to lift ourselves up in terms of what’s possible. What do I love about Arm? I can tell you I’ve been at Arm over 20 years and the one thing that is fundamental about Arm is the can-do attitude of the people who work here.

And it’s infectious, we have so many partners who come to us and that’s a beauty of the R model is just how many the straight to the relationships we have in the ecosystem.

Geof: You’ve been involved in the infrastructure line of business for a. And we’ve seen increasingly rapid adoption of Arm s technology in that time.

So what’s the latest from your team?

Dermot: Yeah. I have to say it’s almost sometimes I have to pinch myself, Jeff, and the progress we’ve been making over the last five years. I started working and there were people before me looking at the infrastructure market at Arm, but we started the first infrastructure line of business concept about six years ago.

And at that point, we had very limited market share. We had very limited traction outside of some embedded networking type applications. And today, the contrast of that, and we have every major public cloud deploying Arm. We have every one of them looking at significant deployments of r We have a software ecosystem that came from being a.

A constant topic of conversation with partners as a challenge for us to actually being something that we lead on. So we have transitioned from being a very small player with very low likelihood of success to taking on, some of the biggest players in the industry like our competition and winning those deployments.

That’s been fantastic.

Geof: In the spirit of looking forwards, let’s talk a little bit more about your recent roadmap announcement, which all sounds pretty. First of all, why did Arm decide to make this roadmap announcement?

Dermot: Yeah, it’s a great question, Jeff, because in many ways we don’t need to make roadmap announcements, we’re a in many cases a secret technology that lies underneath what people in the world experience, right? My kids who are running their TikTok or their, their Instagram are not thinking about who’s serving those pages, if they even think of it that way in the cloud.

But we have a very. Relationship, an amazing relationship with all of the companies, the major cloud players and beyond major networking companies, major, wireless companies. So we have that relationship, and we can share our roadmaps with them, and we do on a con, consistent and a constant basis.

But what we wanted to do with the announcement was to show the world the traction we’re. And the progress that we’re making. And because there is a side of the infrastructure market, which is the developer space, and developers need to know that it’s good for them to be developing on.

And they don’t necessarily see what’s happening inside the walls of a Google or a Microsoft or an Amazon or an Alibaba or any of these companies. So it’s important for us to show them that, look, this is leading edge performance. This is how we’re challenging, the industry actually to do the right thing.

Performance around efficiency, around building a better class of product for the infrastructure market. And that, that draws people to you, right? Getting software ecosystems to invest, getting those teams to invest, we have to respect the fact that they’re gonna do that if it’s in their interests and it’s in their interests when they see us with the leadership product.

Geof: And, and they see what’s coming. And speaking of what’s coming, I in looking at the recent announcement, it looks like there’s some great products coming out of that. What are the standout highlights for you?

Dermot: Yeah so we announced the Neoverse V2 product.

We had that code named as Demeter, the Neoverse V2 two product. When compared to our V1 class product the performance that we’re bringing to the market is outstanding, right? People are super excited about it. We have a lot of customers pulling for that product. And actually, helping us design that product.

What we did with that product is we worked with the primary customers in the public cloud, and we said, what are the workloads that really matter to you? We don’t want to build. And we say this quite strongly we’re not building general purpose computing solutions. We’re building products targeted.

To what these cloud native solutions need to be, what they need to support them. So we take their workloads, the applications that they care about that take up a big percentage of their use cases, and they share that with us because they know that we will respond to that, and we design our processor.

To target the performance that they need on those applications, right? We respond to what those companies tell us they need. And then we go, and we basically test it with them. And the feedback on the V2 product has been super strong, right? I think that that will play out in the market.

We are very careful not to. Talk about what our customers are doing specifically. But that, so that was the first thing. First thing was let’s make it something that is specific to what those customers want. Let’s make it what they need. Coming back to what I said earlier, I get excited by taking a customer set of requirements, taking a technology we have in house, bringing those two together and building something that meets that customer’s needs.

And that’s what we’ve been doing around the V2 product. The second thing is talking about How HPC is done, right? So HPC is traditionally very wide floating-point unit very therefore heavy silicon costs and power costs associated with that and complexity.

We wanted to rethink that. We believe there’s a better way to do that, a more efficient way to do that and a more efficient way to do that. Still giving the performance that these companies need, the HPC companies need. So that’s what we did with Demeter. We basically said we wanna make sure that we have the right amounts of integer performance for the cloud providers on the applications that they care about, and push that to the limit.

The same with floating point to make sure that the floating-point support that the HPC guys need is there. And beyond that, there, there’s obviously a massive development and evolution of computing and it’s super disruptive actually, which is machine learning, right? The requirement for people to process huge amounts of data in very efficient ways.

That is, increasing at such a hockey stick. Traditional computing methods have had to be reinvented. And that’s some of what we’re looking at with sort of the support we put into the V2 product is making sure that those machine learning algorithms can be run really well on, on that processor.

And it, it’s a, again, it comes from the customers saying what we see in the 2023-2024 workloads and applications that we’re running is a lot of this type of code. Can you make this type of code run much more efficiently and much better? And I use the word efficiently and better a lot, right?

Because that, that is core to what we’re doing, right? Is making sure that it’s an efficient way of doing the people can, you can build a bigger processor and there’s a lot of people who build bigger processors. But we are in a world of constrained. And with, within a world of constrained energy and increasingly expensive energy efficiency really matters.

And it’s the core of what we’re about at

Geof: At the announcement you’d mentioned 16 different partners were talking about their adoption of the Neoverse. Maybe you could talk a bit more about those partners and what they’re doing.

Dermot: Yeah. So we have so when we were talking about the 16 partners.

We were talking about the Neo N2 product. That was the product we, we released a couple of years back. And the way the model works, right? For not all the listeners will understand how this works with Arm because we operate differently than a traditional semiconductor company does. But we release the IP in just say Year N

our partner takes that builds a chip. Tapes it out somewhere at the end of year N or early in year N plus one, and then they go into production at the end of that year, N plus one. So there’s a one-to-two-year gap sometimes between the technology we announce and the technology that, that our partners are delivering now.

That was the traditional model. What we’re seeing actually is the acceleration of that. We’re seeing customers taking our products. And very simple example around the V2 product we announced is how soon after that Nvidia we’re basically talking about their product. So it’s definitely that timeline is shortening.

But the reason I bring up the timeline is We have a lot of customer traction with N2, and a lot of that is data center, but it goes well beyond data center. We have traction in the networking and DPU space. For those of us who are we use the smart NIC space there’s that DPU smart NIC space is every major cloud company is attaching a smart a slash DPU to almost every server. Most of those are now based on Arm-based processing, so that, that allow that traction, that attach rate has allowed people to get very comfortable with the Arm instruction set. And why that’s important to the 16 customers is some of them are building DPU, some of them are building security devices.

Some of them are building cloud-based computing. Some of them are building 5G wireless base stations. That breadth of the application space. For that product line has been incredible, right? It’s been super successful for us, almost surprisingly. Actually how many customers have jumped on but that’s because there’s actually real demand for that efficiency of compute when you’re building.

Give you some simple examples to more directly answer your question. When you’re building a 40 or 60 wat t base station and you have, somewhat around 30 to 50% of the power envelope of that base station is dedicated to accelerators or DSPs. You need a very efficient processor to sit next to it, to manage the control plane, to manage the operating system so that, that’s a very simple example where people are saying, I need this kind of compute, but I need it in this power envelope.

Where can I get it? And they’re coming to us because they see the legacy of Arm with, our power sipping you. Mobile technology coming into this space with just the right level of performance to meet that need. That’s one example. Another example is the DPU space I mentioned where most DPU have about a 75-watt power limit. They’re constrained by P C I E and imagine what we would’ve done with 75 watts, right, at a certain point in time, but they’re constrained by 75 watts, and they’ve got a lot of networking support. They’re building 200, 400, 800 gig. Traffic management into those DPU, but they also need control planes.

So they’re putting out 16, 24 cores of Neoverse N1, N2, into that product. And that allows them to get, almost the performance of a server in, in that type of, power envelope, which is, if you look back. Actually, if you look even today at the other side of the market, quite often people are spending 150 to 200 watts to get that kind of performance that, that we can offer in, in, a much smaller footprint and in a much smaller power envelope.

So those are just a couple of the spaces where we’re seeing traction around the neo verse and two product line. On the v2, like I mentioned earlier, the demand around workload performance and specific workloads is becoming just very interesting in our space.

It’s driving the behavior of these cloud players. They want to get to a certain level of performance that matters and then you need to scale out. What they need, what they don’t wanna do is they don’t want to spend 400, 500 watts on a very single threaded performance because most of these companies and most software developers have transitioned from what we consider monolithic type software applications to microservices.

And microservices. It’s it, it fits very well with the Arm design philosophy of build just the right level of performance. And make it efficient, because at that design point, you can scale out as wide as you want and you don’t have, this very high cost of ownership, which is driven by, very high single thread performance demands that some of the legacy architectures are driving.

Geof: Thank you, Dermot. Rejoin our discussion in part two, where we’ll take a broader look at Arm ‘s role in the infrastructure market, now and in the future.