Samsung accelerates push for 'next HBM' with CXL memory technology

Samsung Electronics has released evaluation results showing that Compute Express Link (CXL) technology can resolve AI inference bottlenecks. As large language models (LLMs) store information gathered during user interactions in KV cache — a temporary memory space on the GPU that prevents the model from recalculating previously read context — memory demands have surged, and Samsung says CXL offers a practical solution.

Samsung is leading the charge toward market adoption with its CMM-D (CXL Memory Module DRAM), designed for AI data centers. As the technology gains recognition as a next-generation AI semiconductor successor to HBM, the industry is watching closely to see how quickly data centers will move to adopt it.

According to industry sources Sunday, Samsung recently published AI inference evaluation results using CXL memory pooling through its tech blog.

Samsung configured a 1-terabyte CXL memory pool using a server built on Nvidia's RTX Pro 6000 Blackwell GPU and its own CMM-D modules, then assessed the setup's performance in an AI inference environment.

The evaluation found that performance in a single-GPU environment was comparable to DRAM, and that even in a multi-GPU environment running eight GPUs, performance remained at 92 percent of DRAM levels while offering significantly greater memory capacity.

CXL also demonstrated strengths in KV cache-based inference. While a 512-gigabyte DRAM configuration saw performance degrade when KV cache demands exceeded available memory, the 1-terabyte CXL memory pool maintained stable performance throughout.

As the era of AI inference takes hold, LLMs store previously computed information in KV cache to improve efficiency, eliminating the need to repeat identical calculations. However, as model sizes grow and the number of concurrent users rises, the memory required for KV cache can balloon to hundreds of gigabytes.

Samsung's answer is CXL. Using a CXL switch, the company says multiple memory units can be consolidated into a shared memory pool through memory pooling. CXL is a next-generation interconnect technology that enables CPUs, GPUs and memory to communicate seamlessly within a unified architecture.

Under conventional server designs, the amount of DRAM that can be connected to a motherboard is limited, but linking CMM-D modules with CXL support dramatically expands memory capacity. Latency also decreases as different processors — CPUs, GPUs and others — operate in synchronization.

Where HBM attaches ultra-fast memory directly to a GPU, CXL flexibly connects and expands memory across an entire server.

Samsung developed the industry's first CXL-based DRAM technology in 2021 in a bid to establish an early market lead. It commercialized a CXL 2.0 standard DRAM in 2023. Its PCIe 5.0-based MD220 is currently available in 128-gigabyte and 256-gigabyte capacities, and the company is preparing to begin full-scale production of a CMM-D supporting CXL 3.0.

The semiconductor industry hopes CXL will follow the same growth trajectory as HBM, with some observers predicting it will become the "next HBM" in the AI chip market.

As Samsung's own analysis suggests, CXL can deliver performance comparable to DRAM while substantially expanding capacity, which could sharply reduce the cost of building and operating AI data centers. According to market research firm Yole, the CXL market — valued at $2.1 billion this year — is expected to grow to approximately $16 billion by 2028.

Building out the broader ecosystem remains the most critical challenge to wider CXL adoption, which is still in its early stages. The technology cannot be used unless a server's CPU supports CXL.

Samsung acknowledges this limitation. In a white paper accompanying the evaluation results, the company said the constraints identified "are expected to be gradually resolved as CXL 3.0-based platforms such as Intel's DMR (Diamond Rapids) and AMD's Venice are introduced," adding that it plans to "continue pursuing follow-on collaboration to accelerate CXL memory pooling adoption and expand the related ecosystem."