Intel’s Arrow Lake CPUs keep AI simple to make gamers happy

Intel’s new Arrow Lake architecture, aka the Core Ultra 200S series, brings AI capabilities onto Intel desktops. But the chip doesn’t use the Copilot+ capabilities of Intel’s mobile Lunar Lake chip — its designers used the older NPU found on Meteor Lake instead.

For now, this means that if you buy an Arrow Lake chip, you won’t be able to use it with some of the new AI enhancements found within Windows 11’s 2024 Update, like generative AI and the controversial Recall function. Arrow Lake’s optimized NPU 3 only provides 13 TOPS, while Microsoft set 40 TOPS as the bar for Copilot+ status. Intel still plans to ship more than 40 million AI PCs in 2024, using the vaguer “AI PC” definition that accompanied the launch of Meteor Lake.

Arrow Lake is the not the first desktop PC architecture to include an NPU for AI. Last January, AMD announced the Ryzen 8000 series of desktop processors with an NPU capable of 39 TOPS. However, the blink-or-you’ll-miss-it Ryzen 8000 was quickly replaced by the Ryzen 9000 processor six months later in June — without an NPU, but ushering in Zen 5 with a potent 16 percent performance improvement over the prior generation. Neither the Ryzen 8000 nor Arrow Lake meets the threshold to be called a Copilot+ PC.

Arrow Lake, like Meteor Lake, is a disaggregated architecture — a fancy name for a modular design. Theoretically, couldn’t Intel have just made more of the 45 TOPS NPU 4 found within Lunar Lake, and added it to the Arrow Lake package? Yes…and no, Intel executives said.

Intel

Intel

Intel

Keeping it simple

First off, Arrow Lake’s NPU is the same NPU 3 as Meteor Lake, Robert Hallock, an Intel vice president and general manager of client AI and technical marketing at Intel, told reporters.

“So we’ve had a lot of time to learn it and optimize it, and it made sense that we would fit this in,” he said. (Meteor Lake’s NPU generated 11.5 TOPS, while Arrow Lake’s optimized version produces 13 TOPS.)

However, Intel’s customer base of enthusiasts also indicated that they didn’t want to give up certain features, like a potent GPU, to check the box of AI. And Intel, which faced pressure to deliver Arrow Lake on time, found it easier to just use an established design.

“We actually had a long, a really long chat internally about how to allocate the transistor budget on this part,” Hallock continued. “To be clear, yes, it was fully possible to put a 50 TOPS, 40 TOPS NPU on this product, but to do so would also require shrinking the core count, changing the GPU core count. You start making sacrifices in sort of fundamental performance dimensions that enthusiasts really care about — that didn’t feel like the right mix. And we also talked at length about sort of the enthusiast market’s disposition on AI as a whole. And I think it’s fair to say it’s somewhat reluctant.”

Instead, Intel believes that software developers aren’t always using AI hardware effectively, that a combination of components (CPU, GPU, and NPU) is better than an NPU alone, and various AI models are being effectively compressed small enough that they don’t need a massive NPU. All told, Arrow Lake-S (the Core Ultra 200S series) has a total of 36 platform TOPS. Although Arrow Lake has an integrated GPU, many customers will pair the chip with a discrete GPU, which provides far more AI horsepower than just the NPU alone.

“We’ve proven that Meteor Lake’s 13-TOPS NPU is more than enough,” Hallock said. “We’ve got enthusiast users who are somewhat skeptical, and so we sized an NPU that kind of fits all of those constraints while preserving and protecting the CPU performance that people care about most and still gives enough AI to embrace workloads that are coming down the pipe.”

Interestingly, Intel will debut two mobile chips in the Arrow Lake family during the first quarter of 2024. One, code-named Arrow Lake-HX, will essentially be a mobile copycat of the Core Ultra 200S chips that Intel will begin shipping in a few weeks. A second family of Arrow Lake-H chips for notebooks provides many more TOPS overall than Arrow Lake-S or -HX: 99 platform TOPS in all. However, those will come from a beefed-up GPU that uses XMX extensions and more Xe cores. The NPU will still provide 13 TOPS.

Roger Chandler, vice president and general manager for enthusiast PC and workstation product marketing at Intel, reiterated what Hallock said — that software developers aren’t using the full capabilities of the existing NPUs effectively. Intel’s goal, he said, was to offer a “balanced platform.”

“When I look at AI right now, we’re like, 10 seconds into a 20-hour movie,” Chandler said.