Trying to Actually Define Continual Learning for AI Agents

A writer examines continual learning for LLM agents, framing today's models as amnesiac interns whose knowledge resets when the context window ends. LLMs have only two native memories: lossless but costly context and KV cache, and compressed but hard-to-update weights, with little in between. The piece weighs longer context, stateful memory harnesses, and compressing context into learned latent representations, citing Llama 3.1 70B's 141 GB weights and 320 KiB KV cache per token.