EdgeTalk-MCU: State-Aware Prompt-Constrained Local LLM Control with Runtime Shielding for Low-Latency Microcontroller Interaction

Large language models (LLMs) offer a flexible interface for human–machine interaction, but their direct use in embedded control remains difficult because low-cost microcontrollers cannot host such models locally and unconstrained language generation is not physically grounded. This paper presents EdgeTalk-MCU, a local host–microcontroller framework for low-latency natural-language control of resource-constrained devices. The system couples a locally deployed LLM on the host side with an ESP32-S3 microcontroller through a lightweight serial protocol and closes the loop with real-time state feedback. The reported end-to-end decision latency of ∼0.15 s refers to the host-side inference pipeline; physical platform latency additionally includes UART round-trip and servo actuation overhead. The design combines two complementary mechanisms: a state-aware prompt constraint that injects task progress and physical state into the host-side policy, and a runtime shield that enforces hard execution consistency before actuation. This decomposition separates raw policy quality from executed safety. Across representative obstacle scenarios in simulation, unshielded controllers remain unreliable—LLM-only and Prompt-only exhibit collision rates of 30.6% and 26.5%, respectively, in the Sudden Obstacle setting—whereas both shielded methods reduce collision to 0%. An ablation study confirms that the runtime shield is the decisive safety mechanism; the state-aware prompt constraint contributes primarily at the raw-proposal level by reducing the fraction of unsafe proposals submitted to the shield, rather than by independently guaranteeing safe execution. Hardware-in-the-loop (HIL) validation on a physical ESP32-S3 platform confirms that the same qualitative pattern holds under real sensing and communication conditions.