Rethinking Memory for AI Inference

Moving compute closer to memory — via processing near memory (PNM) and processing in memory (PIM) — can cut data movement, power consumption and latency for AI inference at the edge. Cadence's Frank Ferro says LLMs moving into physical devices are forcing architects to rethink memory partitioning, while Quadric's Steve Roddy notes inference's known data access patterns suit ahead-of-time compilation. Software support, standardization and thermal management remain barriers to adoption.