Japanese firms eye GPU alternatives as costs bite, with Korean NPU joining the mix

Japanese companies are increasingly looking for alternatives to graphics processing units (GPUs) as soaring costs make it difficult to keep pace with AI investment demand, the Nikkei reported Wednesday.

Japan has seen a flurry of AI data center construction announcements in recent months, but filling those facilities with Nvidia AI servers — each carrying a price tag running into the hundreds of millions of won — poses steep cost and procurement challenges. Nvidia holds roughly 90 percent of the global market for data center GPUs.

The Nikkei said demand for server products that can substitute for Nvidia hardware is growing in Japan's AI market, and that the trend could affect GPU supply and broader market conditions over the medium to long term.

The Japanese unit of US-based Penguin Solutions plans to launch a server solution in Japan this year that addresses the memory bottleneck problem common in GPU-equipped servers.

AI servers are built around GPUs alongside memory chips that supply and store data for the GPU. The memory bottleneck refers to a drop in GPU efficiency that occurs when the transfer speed or capacity of memory chips cannot keep up with the GPU's processing speed.

Penguin Solutions specializes in solutions that expand memory capacity while reducing GPU usage, and has a strategic partnership with South Korea's SK Telecom.

The Nikkei also reported that Tomen Device, Japan's largest distributor of semiconductors and electronic components, is launching a proof-of-concept trial of servers equipped with neural processing units (NPUs) made by South Korean semiconductor startup Rebellions. Unlike GPUs, which handle both AI training and inference, NPUs are optimized for inference performance and offer advantages in power efficiency and price competitiveness.

Tomen Device President Kiyotaka Nakao said he expects NPU-based alternatives to become "a strong option for building AI infrastructure."

The Nikkei said the growing adoption of NPUs, along with moves away from Nvidia hardware such as Google's in-house tensor processing units (TPUs), could become a significant variable in the GPU market going forward.