AI inference is running into a memory bottleneck. Huawei wants to tackle it with OceanStor M900

Huawei has unveiled OceanStor M900, a storage platform designed to ease the memory bottleneck in large-scale AI inference. It targets the growing key-value cache, which lets models reuse earlier computations as conversations and reasoning tasks lengthen, and aims to keep accelerators supplied with data more efficiently.

Introduced at HUAWEI CONNECT 2026 in Shanghai, the system adds a shared storage tier for cache across clusters, with claimed capacity of up to 64 petabytes. Huawei says its design can cut access latency to 60 microseconds, double token throughput and halve time to first token in typical AI coding workloads; it also claims endurance of up to 24 drive writes per day. These are company claims, and the platform reflects a broader effort to make storage, networking and compute work together more closely.