- Article
Prompt-Shape- and Prefix-Cache-Aware CPU/NPU Routing for Large Language Model Inference on Edge SoCs
Large language model (LLM) inference on edge systems must exploit heterogeneous central processing unit (CPU) and neural processing unit (NPU) resources under tight memory and latency constraints. Because prefilling and autoregressive decoding have d...