Skip to content
Chips and compute

AI’s hidden energy tax: More than 80% of scale-up traffic is just memory moving around.

SiliconANGLE AIIndependent reporting·
AI brief

Memory bottlenecks are becoming one of the costliest problems in AI infrastructure as inference overtakes training, with GPUs sitting idle waiting on data and high-bandwidth memory staying attached to processors that cannot share it.

Why it matters: If memory movement is the dominant cost in scaled AI systems, it constrains how efficiently inference workloads can run.

Written by AI from SiliconANGLE AI's published text.

Source

“Memory bottlenecks are emerging as one of the costliest problems in AI infrastructure as inference overtakes training. GPUs sit idle waiting on data, while high-bandwidth memory, or HBM, stays bolted to processors that cannot share it.”
SiliconANGLE AI, 18 Nov 2026, 14:57.

Reporting from a publisher outside the organisation being covered.

Read original story ↗