LLMPC: Large Language Model Predictive Control

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for the opportunity to review this article. The topic of linking MPC and LLM is a modern one and needs to be addressed. However, I find the article rather brief. Especially since it is a very specific topic (there are few people who understand MPC and LLM at the same time), I would recommend explaining the different parts in more detail. 

Comments and suggestions:
1. While the study effectively demonstrates improved performance through iterative sampling, it lacks an analysis of the computational overhead associated with generating multiple plans per iteration, which could limit the framework’s scalability in real-time applications.
2. The study compares LLMPC only to traditional few-shot prompting. Including additional baselines, such as other heuristic planners or advanced search algorithms (e.g. Monte Carlo Tree Search), would strengthen the comparative analysis.
3. Please, expand comparative analysis by including additional state-of-the-art planning algorithms as baselines.
4. Please, provide a deeper theoretical explanation for the effectiveness of multiple plan sampling strategies.
5. The theoretical background presented in the manuscript appears to be relatively weak, particularly given the complexity of integrating Large Language Models with Model Predictive Control. These are two highly specialized fields, and few readers are likely to possess in-depth expertise in both areas simultaneously. To strengthen the theoretical foundation, I recommend providing a more comprehensive explanation of the core concepts of MPC and how they intersect with LLM-based planning. A clearer, more accessible discussion (supported by illustrative examples or diagrams) would make the paper more understandable to a broader audience and enhance its overall impact.

After revision, the article can be judged again.

Author Response

Thank you for highlighting both the relevance of this modern topic and the need for more detailed explanations at the intersection of MPC and LLMs. I agree this specialized combination requires careful exposition.
1. **Computational overhead analysis**: I've added an explanation of the computational considerations of LLMPC, particularly discussing how sampling multiple plans through efficient methods like Key-Value caching can be computationally effective compared to alternatives like MCTS.
2. **Additional baselines**: As suggested, I've expanded the comparative analysis to include Monte Carlo Tree Search (MCTS) as a baseline. This strengthens the evaluation by comparing against a strong search-based planning algorithm used in recent LLM planning research.
3. **Theoretical explanation for multiple plan sampling**: I've enhanced the theoretical foundation by more explicitly connecting LLMPC to optimization theory, explaining how LLMs act as approximate cost function optimizers and how sampling multiple plans improves the optimization outcome.
4. **Theoretical background**: I've substantially expanded the theoretical background sections, particularly in Section 2 (Model Predictive Control) and Section 3 (LLM as MPC Plan Sampler), providing more accessible explanations of how these two fields intersect. I've also added illustrative diagrams to make the concepts more understandable.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This paper proposes the LLMPC framework based on Model Predictive Control (MPC) to explicitly integrate Large Language Models (LLMs) into planning tasks. While LLMs implicitly optimize cost functions during plan generation, explicit optimization through the MPC framework further enhances performance. Experimental results demonstrate that LLMPC significantly outperforms single-round few-shot prompting methods in complex scenarios, highlighting the substantial research significance of this work. To further improve the manuscript, the authors are advised to address the following:

(1) The mathematical foundation of LLMs as implicit optimizers remains underexplored, and no comparisons are made with traditional optimizers (e.g., gradient descent). A detailed analysis of differences between LLMPC and classical MPC should be added.

(2) The study only benchmarks against single-round LLM prompting. Broader comparisons with other planning methods (e.g., reinforcement learning [5] or symbolic planners like PDDL [10]) are needed. Validation on real-world robotic tasks (e.g., robotic arm control) would strengthen physical applicability.

(3) Critical LLM hyperparameters (e.g., temperature, top-p sampling strategies) are unspecified. Explicit documentation of these parameters is essential for reproducibility.

(4) The Discussion section should address risks of LLM-generated plans violating physical constraints (e.g., invalid trajectories in robotic systems) and propose mitigation strategies.

(5) Terms like "cost function" (used in Section 3) and "objective function" (used in Sections 2 and 4) are conflated. Consistent terminology aligned with MPC literature (e.g., unified use of "objective function") is recommended.

Author Response

Thank you for recognizing the research significance of this work and providing constructive feedback to improve it.
1. **Mathematical foundation of LLMs as optimizers**: I've expanded the discussion on how LLMs implicitly optimize cost functions, clarifying the differences between LLMPC and classical MPC approaches. This includes better articulating how instruction-tuned LLMs can be viewed as approximate optimizers.
2. **Broader comparisons**: As suggested, I've added MCTS as a planning baseline and conducted more extensive evaluations across all benchmarks. I've also expanded the discussion section to address the applicability to physical systems like robotic control.
3. **LLM hyperparameters**: I've now documented all hyperparameters used in the experiments, including temperature, top-p values, and model specifications for better reproducibility.
4. **Safety considerations**: I've added a new paragraph in the Discussion section addressing the risks of LLM-generated plans potentially violating physical constraints and proposed approaches for mitigating these risks.
5. **Terminology consistency**: I've standardized the use of "cost function" throughout the paper to align with MPC literature and ensure consistency.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

I have one point that requires significant consideration and revision of the manuscript before it is suitable for publication.

1. Both the paper's title and its Abstract state sufficiently clearly the main aims of the paper. However, the use of acronyms should be avoided in the paper's Abstract, as they reduce its readability.

2. How is the proposed method implemented? It is suggested to add the flow chart for control algorithm implementation.

3. In the section of the introduction, a structural summary of the full text is suggested presented in the revision.

4. References are time-sensitive, can closely track the frontiers of subject research. Some recent results on MPC, such as “Review on model predictive control: An engineering perspective; Robust predictive fault-tolerant switching control for discrete linear systems with actuator random failures”, may be helpful for the introduction. A refresh of the state-of-the-art survey is advised.

5. For clarity of the contribution summarized at the end of the Introduction, the references should be given for comparison.

6. In the simulation comparison, model and controller parameters should be included and explain how to get these parameters of model and controller.

7. Which MPC method was it compared to? Was it compared to the most recent MPC method?

Comments on the Quality of English Language

The English could be improved to more clearly express the research.

Author Response

Thank you for your feedback and suggestions for improving the structure and clarity of the manuscript.
1. **Abstract clarity**: I've revised the Abstract to avoid acronyms and enhance readability.
2. **Algorithm implementation**: I've added a flowchart (Figure 2) for the LLMPC control algorithm implementation to provide a clearer visual representation of the approach.
3. **Introduction structure**: I've added a structural summary at the end of the Introduction section to better guide readers through the paper's organization.
4. **Updated MPC references**: I've refreshed the state-of-the-art survey on MPC, including the suggested references on model predictive control engineering perspectives and robust predictive fault-tolerant switching control.
5. **Contribution comparison**: I've added references for comparison in the contribution summary to better contextualize the paper's advancements.
6. **Model and controller parameters**: I've expanded the experimental section to include more details on how model and controller parameters were determined and configured.
7. **MPC method comparison**: I've clarified which MPC methods were used in the comparisons and explained why they were selected as appropriate baselines.
Thank you again for your valuable feedback.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for the revisions. They look good overall. However, as per standard practice, responses to the review should be more detailed. It is common to include line numbers indicating where changes were made, along with a brief description of the modifications.

This additional information would make it easier to track updates and ensure clarity in the review process. I appreciate your efforts and look forward to the next steps.

Author Response

Absolutely we are happy to provide a detailed overview of the revisions:
1. Computational Overhead Analysis of Multiple Plan Sampling On lines 149-163 we add a detailed discussion on the computational considerations of sampling multiple plans. We highlight that sampling multiple plans can be done in an efficient manner by asking the LLM for multiple plans, with further efficiency gains possible by using methods such as Key-Value caching.
2 & 3. MCTS Baseline We added comparisons to MCTS for both the trip planning and meeting planning benchmark. * Lines 201-203: highlighting MCTS addition and parameter choices for trip planning case * Lines 221-227: compares LLMPC against MCTS on trip planning and highlights similar performance, albeit with fewer iterations required by LLMPC * Table 2 and Figure 5: added success rate metrics for MCTS * Lines 251-253: highlighting MCTS addition and parameter choices for meeting planning case * Lines 266-271: Comparison of LLMPC with multiple plans to MCTS highlighting better performance of LLMPC * Table 3 and Figure 6: added success rate metrics for MCTS * Lines 287-298: Overall discussion of MCTS versus LLMPC performance highlighting improved efficiency of LLMPC
4. More Explanation on Multiple Plan Sampling We added discussion and references on how LLM planning in MPC is equivalent to using an LLM to solve the optimization problem and that sampling multiple plans should improve their performance. * Lines 144-151: references on LLM as optimizer performance and discussion of multiple plan sampling * Lines 164-168: additional discussion on sampling multiple plans for constraint satisfaction
5. Expanded Theory on MPC and LLM Integration We substantially expanded the sections on MPC and LLM as MPC Planner, highlighting that MPC relaxes control problems into iterative smaller planning problems and that for similar reasons LLMPC should enhance LLM performance. * Lines 82-95: Added explanation of the exact planning problem MPC is obtained from * Lines 95-109: Discuss how MPC is an iterative relaxation of the exact planning problem, and that reducing the horizon makes the problem tractable. * Lines 112-123: Highlighting that LLM performance degrades as problem complexity increases and that breaking problems down into smaller iterative sub problems can enable LLMs to solve the larger problem, similar to MPC. * Lines 124-144: Added additional details on how control plan is generated and extracted from LLM * Figure 1 and 2: Added diagrams showing generation of a single output from the LLM and the full LLMPC control loop
We hope this overview is helpful and thank you again for the detailed review comments.

Reviewer 2 Report

Comments and Suggestions for Authors

The authors made all the comments. The manuscript is suitable for publication in the journal.

Author Response

We appreciate your prompt response and thank you again for the detailed review comments.

Reviewer 3 Report

Comments and Suggestions for Authors

The authors have answered my suggestions well. I have no further comments.  

Author Response

We appreciate your prompt response and thank you again for your detailed review comments.

Round 3

Reviewer 1 Report

Comments and Suggestions for Authors

The article can be accepted.