Separating Execution and Target Selection in Vision–Language–Action Evaluation: A VLABench Case Study

Task failure alone cannot distinguish limited robot execution from incorrect target selection. We evaluate released SmolVLA and ${\pi }_0$ policies in VLABench using family-matched native controls, same-state trusted and null instructions, online symmetric stopping and a separate retention check. The main experiment contains 1860 attempts under three inference seeds; a command-duration follow-up adds 420. Native completion rises from 0/210 and 1/210 at 6 simulated seconds to 73/210 and 101/210 at 20 s, respectively, correcting a short evaluation cutoff. Reviewer-requested secondary analysis of 40 benchmark-derived paired scenes gives a symmetric request-swap contrast of 10.00 percentage points for ${\pi }_0$ (scene-bootstrap 95% CI [3.33, 17.08]) and zero for SmolVLA ([−4.17, 4.58]), yet requested choice under switched instructions is only 20/47 (42.6%) and 9/22 (40.9%) among goal-reaching attempts, respectively. Across all 60 nominal fixtures, the ${\pi }_0$ advantage over blank input decomposes into 11.39 points of coverage and 3.33 points of net-choice contribution ([1.11, 5.69]). Its shuffled comparison warrants caution: the original adjusted $p=0.0481$ is borderline; 10/40 benchmark donors name a candidate, and all 20 constructed donors equal request A. Holding each command for two environment steps rather than one reduces native completion to 39/210 and 55/210 at the same 20 s budget while also changing command count and replanning time. The results demonstrate configuration-sensitive execution and a bounded instruction effect, not reliable target switching, architecture superiority or sim-to-real transfer.