MPSR: A Multi-Perspective Self-Reflection Framework for Public Opinion Report Generation

Yu, Jinzheng; Fan, Weijian; Xu, Yang; Feng, Yifan; Luo, Jia; Zhu, Ligu; Shen, Hao

doi:10.3390/electronics15020404

Abstract

Crisis events generate massive information flows from diverse sources, which need to be consolidated into public opinion reports to enable timely response by governments and enterprises. Current LLMs, despite strong generation capabilities, fail to achieve perspective diversity, maintain factual consistency, and perform coherent high-level planning. To address these gaps, we propose MPSR: a multi-perspective self-reflection framework. Our framework first assigns diverse stakeholder personas to agents who independently generate initial writing plans from complementary viewpoints. Subsequently, a three-stage debate mechanism refines these plans by identifying conflicts, formulates resolution strategies, and produces a consensus plan, thereby enhancing factual consistency. Finally, we introduce a Report Fusion mechanism to synthesize reports across temporal batches, ensuring comprehensive event coverage. Extensive experiments demonstrate that MPSR significantly outperforms baselines, achieving Date F1 of 0.67, G-Eval of 4.54, and MiniCheck score of 79.43, which represent improvements of 17.5%, 70.0%, and 25.8% over the strongest baseline, respectively.

Keywords:

public opinion reports; large language models; multi-perspective self-reflection; debate mechanism; report fusion

1. Introduction

In today’s internet era, crisis events generate a massive, rapid flow of information from thousands of daily news articles alongside vast volumes of social media posts containing redundant and noisy content. Public Opinion Reports serve as critical tools to address this, consolidating these diverse sources to enable governments or enterprises to respond timely to rapidly spreading incidents such as earthquakes and floods.

Open-ended text generation, where LLMs produce extended content based on user inputs, has emerged as a promising research direction [1,2]. Recent efforts have explored diverse approaches, from real-world scenario generation [3] and long-form writing with divide-and-conquer strategies [4], to personalized content creation capturing community-specific styles [5].

Despite these advances, current LLM-based report generation approaches exhibit three critical limitations. First, most methods rely on single-perspective generation that may overlook important viewpoints inherent in public opinion analysis [6,7]. Second, while generating fluent summaries, they still make errors on fine-grained qualities such as factual consistency [8,9]. Third, they predominantly generate text autoregressively without explicit planning, which can result in inadequate structured organization that is essential for well-formed analytical reports [10].

To address these limitations, we propose MPSR (multi-perspective self-reflection framework) for generating perspective-diverse, factually consistent, and logically coherent reports (Figure 1). Drawing on multi-agent collaboration research [11,12], our framework assigns distinct stakeholder personas to agents who independently generate initial writing plans, thereby capturing diverse viewpoints from different perspectives. A three-stage debate mechanism then collaboratively refines these plans, where agents identify discrepancies, deliberate resolution strategies, and vote on a final consensus plan. This cross-agent verification process enhances factual consistency by enabling agents to detect and challenge errors from complementary perspectives, while the explicit planning stage ensures structured organization before content generation. Finally, the Report Fusion mechanism synthesizes reports across temporal batches, maintaining comprehensive event coverage. To rigorously evaluate performance on the core challenges of crisis reporting, we adopt three complementary metrics. Date F1 measures temporal accuracy through explicit date mentions and time anchoring, while G-Eval provides multi-dimensional quality assessment covering factual consistency, perspective diversity, and logical coherence. MiniCheck (MCS) verifies sentence-level factual consistency against source documents, complementing G-Eval with a strict entailment check that helps mitigate potential judge bias or hallucinated assessments.

Figure 1. Comparison of (a) standard generation and (b) MPSR framework. (a) Standard approaches fail to achieve perspective diversity, maintain factual consistency, and establish coherent planning (✗). (b) MPSR synthesizes diverse stakeholder viewpoints through collaborative debate, ensuring factual consistency and logical coherence (✓).

This work makes the following key contributions:

We propose MPSR, a multi-perspective self-reflection framework that assigns diverse stakeholder personas to independently formulate writing plans, thereby ensuring comprehensive perspective coverage.
Our framework introduces a three-stage debate mechanism and a Report Fusion mechanism to collaboratively refine plans and synthesize reports across temporal batches, ensuring factual consistency, structured organization, and comprehensive temporal coverage.
Extensive experiments demonstrate that MPSR significantly outperforms baselines in generating high-quality, perspective-diverse, and factually consistent reports.

2. Preliminaries: The Structure of a Public Opinion Report

A Public Opinion Report is structured into five key components, each designed to provide a comprehensive overview of a crisis event.

Event Title

The Event Title serves as a concise identifier that allows readers to immediately grasp the event’s essence. A well-formed title conveys the crisis name, type, and time, facilitating efficient storage and retrieval.

Event Summary

The Event Summary offers a structured overview covering six key aspects: Crisis Name (What), Location (Where), Time (When), Cause (Why/How), Involved Parties (Who), and Impact (the event’s consequences and severity).

Event Timeline

The Event Timeline traces public opinion evolution through three phases: Incubation Period (initial low-activity phase), Peak Period (rapid attention growth with widespread discussion), and Decline Period (diminishing interest phase).

Event Focus

The Event Focus analyzes public opinion from both Netizens and Authoritative Institutions, extracting three dimensions for each: Core Topics, Sentiment Stance, and Key Viewpoints.

Event Suggestions

The Event Suggestions section provides actionable recommendations based on the preceding analysis. These may include communication strategies, public relations actions, or policy adjustments.

3. Method

We introduce the MPSR for public opinion report generation. The framework processes multi-source document streams into comprehensive, perspective-diverse, and temporally coherent reports. Our framework operates in three distinct steps, as illustrated in Figure 2:

Figure 2. Overview of the MPSR framework for crisis report generation. Left: Documents are organized chronologically into temporal batches. Middle: Each batch undergoes persona-based generation, where agents with diverse personas create independent plans and then collaboratively refine them through structured debate. Right: Final plans are written into reports and iteratively fused for temporal coherence.

3.1. Task Definition

The report writing aims to generate a structured, multi-section report R for a given public opinion event

e_{i} = (X_{i}, Y_{i})

. The input for each event consists of a set of news articles

X_{i} = {x_{i, 1}, x_{i, 2}, \dots, x_{i, M}}

and a set of social media posts

Y_{i} = {y_{i, 1}, y_{i, 2}, \dots, y_{i, K}}

.

Formally, the task is defined as

R = LLM (P_{gen}, X_{i}, Y_{i})

(1)

where

P_{gen}

represents the generation prompt.

3.2. Chronological Data Organization

To manage extensive source documents while respecting context window limitations, we organize the input documents chronologically and segment them into n sequential batches (

D_{1}, D_{2}, \dots, D_{n}

), where each batch contains documents from the same day and within context limits. This batching strategy allows the model to process the event’s evolution step-by-step, ensuring temporal coherence.

3.3. Report Generation Process

For each given document batch

D_{i}

containing news articles and social media posts about the crisis event, our framework generates report

R_{i}

through the following process:

3.3.1. Persona Assignment

When analyzing a crisis event, different stakeholder groups focus on distinct perspectives based on their roles and concerns. To ensure fairness and inclusivity, we maintain a diverse persona pool, each representing a distinct viewpoint relevant to crisis events and formalized with a brief description. Appendix A provides a representative subset.

For each event, we first prompt an LLM to propose personas that are likely to write and rely on the report given the event description and source snippets. Because dynamic generation may introduce semantically overlapping personas, we apply a two-stage deduplication process to ensure fairness and inclusivity while maintaining high perspective quality. We first apply MinHash with n gram features to remove personas with high surface level similarity, and then perform embedding-based semantic filtering to remove conceptually similar personas. After deduplication, we select the top-6 most relevant personas for the subsequent debate process.

These personas act as hidden variables guiding the agents’ contributions. After assignment, each agent independently generates an initial writing plan from its assigned perspective. For example, an agent assigned the Government Official persona emphasizes official response measures and verified statistics, while a Netizen persona highlights public sentiment and community concerns. This independent generation ensures that the initial set of plans captures the maximum possible perspective diversity before collaborative refinement occurs. To facilitate effective debate and conflict resolution in subsequent stages, we construct a detailed conflict and error resolution protocol, informed by analyzing common discrepancies in multi-source reporting, as outlined in Appendix B. This protocol includes definitions, examples, and explicit strategies such as overwrite and correct, providing consistent guidance for both debater agents and the judge agent during the self reflection process.

3.3.2. Self-Reflection

To enable effective self-reflection, we employ a structured Self-Reflection Debate mechanism involving debater agents(who propose and refine plans) and a judge agent (who provides structured guidance). In this iterative process, agents update their responses based on feedback from other agents within each debate round. Within each stage, agents engage in iterative discussions until reaching consensus (i.e., majority agreement) or the maximum turn limit. The plan refinement process in each round unfolds through three stages:

Stage 1: Issue Identification

Debaters first compare their independently generated writing plans to identify discrepancies and conflicting interpretations. The judge then introduces specific issue definitions as detailed in the error protocol (Appendix B). When multiple discrepancies arise simultaneously, the judge enforces a prioritization hierarchy where Faithfulness Errors are addressed first as they compromise the report’s factual foundation, followed by Conflict Errors that require reconciling contradictory sources, and finally Integration Errors that involve information consolidation. Debaters discuss discrepancies following this priority order to determine which specific issue types apply, producing brief justifications that justify their assessments. To ensure consistency across rounds, the judge maintains a cumulative issue tracker that records identified problems and their resolution status throughout the debate process.

Stage 2: Solution Strategy

Once issues are confirmed, the focus shifts to formulating solutions. The judge provides relevant resolution guidelines and examples from the protocol (Appendix B) to frame the discussion. Debaters then deliberate on specific strategies to resolve each identified issue.

Stage 3: Plan Revision

Based on the agreed-upon resolution strategies, each debater independently revises their writing plan. The revised plans are then presented, and all debaters vote to select the plan that best incorporates the consensus solutions. Let

P

denote the set of revised plans proposed by all debaters in a round. Each debater r casts one vote

V_{r} \in P

. The consensus plan

\hat{P}

is selected by voting

\hat{P} = \underset{P \in P}{argmax} \sum_{r = 1}^{| R |} 1 (V_{r} = P),

(2)

where

| R |

is the number of debaters,

V_{r}

denotes the vote cast by debater r, and

1 (\cdot)

is the indicator function.

The selected plan becomes the final consensus plan of the current round, representing a logically cohesive and factually consistent framework ready for report generation.

3.3.3. Report Writing

The report writing module transforms the debate-generated plan into the report

R_{i}

for the current round by employing a dedicated generation agent. The writing agent faithfully adheres to the debated plan’s logical structure and factual consensus, organizing the content into the standard report structure.

3.4. Iterative Report Fusion

After each round’s report generation, we perform an additional fusion operation to maintain comprehensive event coverage. This phase primarily addresses Redundancy and Time Updates errors (Appendix B), which require systematic integration rather than perspective negotiation. Therefore, the judge directly handles this task without multi-agent debate. Specifically, in round i (

i \geq 2

), the judge agent executes a structured merger of the current report

R_{i}

with the accumulated historical report

R_{i - 1}

through a three-step process (formally described in Appendix C):

1.: Temporal Conflict Resolution:
The judge first identifies contradictions across temporal batches, particularly Time Updates where later information supersedes earlier reports (e.g., updated casualty figures). We enforce a Temporal Recency Principle: since $R_{i}$ is generated from later data sources than $R_{i - 1}$ , conflicting status updates in $R_{i}$ are treated as the latest ground truth and used to overwrite outdated information in $R_{i - 1}$ , while non-conflicting historical context is preserved with explicit temporal attribution.
2.: Redundancy Elimination: The judge detects redundancy by comparing semantic overlap between sections of $R_{i}$ and $R_{i - 1}$ . It performs semantic matching to identify information in $R_{i}$ that repeats facts already established in $R_{i - 1}$ , discarding duplicate units to maintain conciseness.
3.: Fused Report Generation: Finally, the judge produces an updated fused report ${\tilde{R}}_{i}$ that integrates the new verified information. This step involves rewriting the fused content into a coherent narrative, ensuring the final report evolves dynamically to reflect the latest crisis developments while retaining valuable historical context.

4. Experiments

4.1. Experimental Setups

4.1.1. Dataset

OPOR-Bench [13] is an event-centric dataset specifically designed for Public Opinion Reports, covering 463 crisis events spanning from 2012 to 2025. Each crisis event in the dataset includes a collection of multi-source documents, specifically news articles and social media posts (as shown in Table 1), along with a structured reference summary containing key event information (location, time, cause, impact, etc.).

Table 1. Distribution of event sub-types in OPOR-Bench, detailing the number of events alongside the average count of associated news articles and social media posts per event.

4.1.2. Baseline

OPENASP: A “Filter-then-Summarize” approach that uses Sentence-T5 to select aspect-relevant sentences and fine-tunes BART-large to generate aspect-based summaries [14].
PRIMERA: A pre-trained model developed specifically for multi-document summarization that builds upon the Longformer–Encoder–Decoder (LED) architecture [15].
Single-Turn Generation (STG): This baseline randomly samples documents until reaching the context window limit and generates the entire report in a single pass.
Modular Generation (MG): Following [4], this baseline decomposes the task into five sequential subtasks (title, summary, timeline, focus, and suggestions). Each component is generated independently and then assembled into the final report.

4.1.3. Implementation Details

Different models exhibit distinct hallucination patterns due to diverse training data and methods. Multi-model frameworks can exploit this diversity, as model-specific hallucinations may cancel out through collaborative refinement while complementary reasoning patterns enhance overall performance. Based on these motivations, we evaluate our MPSR framework using two advanced LLMs, GPT-4o and Gemini-2.5 Pro. This choice allows us to test the framework’s performance both within a single model (MA-SM) and in collaboration across different models (MA-MM). Therefore, we design two experimental settings:

(1) Multi-Agent Single-Model (MA-SM): In this setting, all agents in the debate (i.e., Debaters and Judge) are instances of the same model. We run two separate experiments: one where all agents use GPT-4o, and another where all agents use Gemini-2.5 Pro.

(2) Multi-Agent Multi-Model (MA-MM): This setting employs both GPT-4o and Gemini-2.5 Pro as agents within the same framework. To mitigate potential role-assignment bias, we randomly assign personas to models for each run, ensuring a balanced 1:1 ratio between the two model types.

For both settings, all debates run for a maximum of 10 rounds, with the corresponding computational efficiency and cost analysis detailed in Table 2.

Table 2. Efficiency analysis per event on OPOR-Bench. We report total API calls, average inference latency (seconds), and estimated monetary cost (USD). Fine-tuned models (OPENASP, PRIMERA) run locally with negligible inference cost.

4.1.4. Evaluation Metrics

Public opinion report generation requires evaluation across three core dimensions comprising factual consistency, perspective diversity, and logical coherence. Factual consistency refers to accurate representation of source information. Perspective diversity entails the coverage of multiple stakeholder viewpoints. Logical coherence denotes well-structured organization and actionable insights. To collectively assess these dimensions, we adopt three complementary metrics.

Date F1 [16]: Measures temporal accuracy by computing the F1 score of explicit date mentions in generated reports against the reference, encouraging correct event chronology and time anchoring. It is particularly suited for crisis reporting where correct chronology is vital, though we note it focuses on explicit expressions and may not fully capture implicit temporal ordering.
G-Eval [17]: Following this work, we prompt GPT-4o to evaluate report quality on a 1–5 Likert scale using a fine-grained rubric tailored to each report section. This enables assessment across all three dimensions, including factual consistency (e.g., Nature, Time & Location), perspective diversity (e.g., Emotional Analysis, Feasibility), and logical coherence (e.g., Relevance, Guidance). While LLM-based evaluation may exhibit scoring biases, it provides scalable multi-dimensional assessment aligned with human judgment.
MiniCheck (MCS) [18]: A Flan-T5-based binary classifier that verifies sentence-level factual consistency of the generated report against the provided source documents. This provides a strict entailment check to detect hallucinations, complementing the high-level assessment of G-Eval. Its coverage can be limited when evidence is indirect or when correctness depends on multi sentence reasoning.

4.2. Main Results

Table 3 presents the main results. We observe that MPSR consistently outperforms all of the baselines across all evaluation dimensions: temporal accuracy (Date F1), overall quality (G-Eval), and factual consistency (MCS). This significant improvement highlights the fundamental effectiveness of our incremental, multi-perspective self-reflection approach.

Table 3. Main results comparing MPSR (MA-SM, MA-MM) against baselines across Date F1, G-Eval, and MCS metrics. The MA-MM configuration (GPT-4o + Gemini 2.5 Pro) achieves the best results across all dimensions. ^† denotes statistically significant improvement by MPSR (MA-MM) over that entry p < 0.05. The upward arrow (↑) indicates that higher scores are better. Bold values indicate the best performance in each column.

Notably, the MA-MM configuration, which combines GPT-4o and Gemini 2.5 Pro, consistently outperforms both single-model variants (MA-SM) across all metrics. This superior performance demonstrates that cross-model debate is significantly more effective than same-model debate. Stemming from diverse training data and methodologies, different foundational models are less likely to share the same hallucination tendencies or reasoning approaches [19,20]. Consequently, these model-specific biases tend to cancel out during the collaborative refinement process, such as when GPT-4o hallucinates a detail that Gemini 2.5 Pro successfully identifies and corrects, and vice versa. Meanwhile, their complementary reasoning patterns mutually reinforce valid inferences, leading to more robust and factually consistent outputs. Among single-model setups, Gemini 2.5 Pro demonstrates stronger performance, likely due to its superior instruction-following capabilities for complex multi-turn interactions.

4.3. Human Evaluation

To validate the reliability of automatic metrics and given the subjective nature of report quality, we conduct a small-scale human evaluation study. We randomly sample 30 crisis events from the OPOR-Bench test set. Three annotators with backgrounds in crisis communication independently rate each report on a 1–5 Likert scale, using the same fine-grained criteria employed in G-Eval. For correlation analysis, we map the fine-grained dimensions into three high-level aspects. Factual Consistency encompasses Nature, Time & Location, Parties, Causes, Impact, and Date Accuracy. Perspective Diversity covers Controversy Topic, Representative Statement, Emotional Analysis, Feasibility, and Guidance. Logical Coherence includes Title, Relevance, Feasibility, Guidance, and Innovation. To assess human–automatic agreement, we compute the Pearson correlation coefficient r between human aggregated scores and G-Eval aggregated scores across all evaluated reports, as shown in Table 4. We observe a strong correlation for Factual Consistency (

r = 0.82

), indicating G-Eval provides evaluations well-aligned with human judgment. Logical Coherence also shows substantial agreement (

r = 0.76

). Crucially, even for the highly subjective dimension of Perspective Diversity, we achieve a strong correlation (

r = 0.69

).

Table 4. Pearson correlation (r) between human ratings and G-Eval scores over 30 events with three annotators. ^† denotes statistically significant at

p < 0.05

.

5. Further Analysis

To gain more profound insights into our approach, we explore the following research questions:

(1) Are all modules effective for MPSR?

(2) Can Iterative Round Processing mitigate the inherent long-context challenge of LLMs?

(3) Can Multi-Agent Self-Reflection improve factual consistency?

(4) Does Persona Assignment enhance perspective diversity in subjective content?

(5) How does the Plan-then-Write mechanism contribute to the final report quality?

(6) What are the primary conflicts and errors in crisis event report generation?

(7) How does the MPSR process work in real-world scenarios?

5.1. Answer 1: All Modules Are Effective for MPSR

To investigate the contribution of each component, we conduct ablation studies by systematically removing key modules from our best-performing MPSR (MA-MM), as shown in Table 5. We create four variants: (1) w/o Plan bypasses the planning stage, (2) w/o Persona removes persona assignment, (3) w/o Self-Reflection eliminates the three-stage debate mechanism, and (4) w/o Round uses single-round generation instead of iterative processing.

Table 5. Ablation study of MPSR (MA-MM) components using G-Eval scores to provide fine-grained analysis of each component’s impact on different aspects of report quality.

The results reveal a clear performance hierarchy. The w/o Round, w/o Self-Reflection, and w/o Persona cause severe performance degradation across all dimensions. In contrast, the w/o Plan results in moderate but still significant degradation. Notably, this degradation is not uniform: the w/o Persona variant exhibits pronounced drops in subjective analysis sections (e.g., Event Focus and Event Suggestions), while the w/o Plan variant maintains relatively stronger performance in factual sections.

This hierarchy validates the necessity of each component. Multi-round processing, self-reflection, and persona assignment are fundamental because they enable incremental temporal coherence, factual consistency, and perspective diversity, respectively. In contrast, structured planning primarily benefits higher-level reasoning by establishing document structure, but has less impact on straightforward factual reporting.

5.2. Answer 2: Round Processing Effectively Mitigates Information Loss

Processing long-context inherently poses a challenge for LLMs, particularly when crisis events span multiple days and generate extensive information flows. Figure 3 presents a macro-level analysis, plotting average G-Eval scores against temporal batch count. As events grow longer (from 5 to 15 temporal batches), baselines exhibit progressive degradation, while MPSR maintains stable performance. The ablation study (Table 5) provides micro-level confirmation: the w/o Round variant suffers the severest collapse (−1.71), with Event Focus and Suggestions experiencing the largest declines (Controversy Topic: −1.88, Innovation: −1.87). Even factual components show substantial drops, with Nature and Time & Location declining approximately 1.72.

Figure 3. Impact of event temporal complexity on generation quality. Each point represents the average G-Eval score across all events with n temporal batches. Baselines degrade as event length increases, while MPSR maintains consistent quality.

This failure occurs because baselines are limited to the context window, making them incapable of managing multi-day complexity. As event duration increases, this information loss compounds, degrading both factual accuracy and analytical depth. In contrast, MPSR’s multi-round approach processes each temporal batch incrementally and employs iterative report fusion to synthesize information across rounds.

5.3. Answer 3: Self-Reflection Mechanism Substantially Improves Factual Consistency

As shown in Table 5, the w/o Self-Reflection variant causes severe performance degradation (−1.57). This drop is particularly significant across all factual components in Event Summary, such as Nature (−1.78) and Time & Location (−1.81).

This result is expected, as the self-reflection debate is fundamental to ensuring factual consistency. Specifically, the Issue Identification stage enables agents to detect discrepancies and factual errors by comparing their respective plans. Subsequently, the Solution Strategy and Plan Revision stages guide them to deliberate on resolution strategies and execute consensus-based corrections. This structured multi-stage verification acts as a rigorous pre-generation filter, effectively mitigating hallucinations and ensuring factual grounding before the model generates the text.

5.4. Answer 4: Persona Assignment Significantly Enhances Perspective Diversity in Subjective Analysis

As shown in Table 5, the w/o Persona variant causes a significant overall performance degradation (−1.03). This decline is not uniform across all dimensions. It is most pronounced in subjective analysis sections, particularly within Event Focus (e.g., Emotional Analysis, −1.79) and Event Suggestions (e.g., Feasibility, −1.76). Conversely, factual sections like Nature (−0.87) and Time & Location (−0.81) experience more moderate declines.

This pattern confirms that diverse personas are essential for generating multi-faceted analysis and actionable recommendations, while having less impact on objective factual reporting. Subjective sections (e.g., Event Focus, Suggestions) require synthesizing competing viewpoints; without distinct personas, agents generate homogeneous interpretations that fail to capture this essential diversity. Conversely, factual sections primarily involve information extraction and verification, where different personas offer similar assessments.

5.5. Answer 5: The Plan-Then-Write Mechanism Enhances High-Level Logical Coherence

As shown in Table 5, the w/o Plan variant shows the smallest average degradation (−0.56), yet its impact is highly concentrated. The performance drop is most pronounced in dimensions requiring high-level reasoning and coherence, particularly the Event Suggestions section, which suffers a significant and uniform drop across sub-dimensions like Relevance (−0.85) and Guidance (−0.95).

This differential impact occurs because structured planning provides the logical framework for inter-section coherence. Event Suggestions requires synthesizing insights from preceding sections (Summary, Timeline, Focus) to generate actionable recommendations. Without explicit planning, agents produce suggestions that lack holistic integration despite being individually reasonable.

5.6. Answer 6: Perspective Diversity and Factual Faithfulness Are the Primary Challenges

To better understand the primary challenges in this task, we conduct a statistical analysis of the specific error types identified and addressed during the debate process across all experimental runs, as illustrated in Figure 4. The results reveal that Perspective & Standpoint Conflicts account for the largest proportion of errors across all issues. This distribution highlights that the core difficulty is not simple extraction, but the sophisticated synthesis of contradictory, multi-source viewpoints. Perspective conflicts dominate because synthesizing diverse stakeholder viewpoints is complex. Faithfulness errors arise frequently as models navigate source contradictions, generating plausible but inaccurate content through semantic distortion or mis-referencing. Integration errors, though less prevalent, remain important as models must detect subtle redundancies and omissions across multi-source documents. This distribution underscores that report generation demands dual capabilities: embracing perspective diversity while ensuring factual consistency.

Figure 4. Distribution of error types in report generation. The inner ring shows three main categories, while the outer ring displays specific error types.

5.7. Answer 7: A Case Study on Multi-Perspective with Self-Reflection Generation Process

We conduct a case study on the “2024 Singapore Oil Spill” to illustrate how MPSR addresses the dual challenges of perspective diversity and factual consistency in public opinion report generation. As shown in Appendix D, the four personas shape distinct planning focuses. The Official emphasizes response actions and scale, the Journalist prioritizes event narrative and ecological impact, the Rescuer concentrates on actionable guidance and operational readiness, and the Netizen captures public sentiment and community concerns. However, this isolated focus introduces specific error types, such as integration errors or faithfulness errors. During self-reflection, the judge identifies and prioritizes issues by addressing faithfulness problems first, then reconciling time related conflicts, and finally resolving integration issues such as omissions. Agents engage in structured debate turns, proposing resolution strategies such as reconcile for aligning conflicting timelines. After applying these strategies, each debater independently revises their writing plan. The final report generated from this consensus plan resolves the identified conflicts while preserving complementary information from different perspectives.

6. Related Work

6.1. Self-Refinement and Factual Consistency

Large language models suffer from hallucinations and unfaithful generation [21,22]. Such inconsistencies and fabrications significantly undermine LLM reliability in practical applications [23,24]. To address this, researchers have developed post hoc self-refinement techniques, though each has trade-offs. Human feedback approaches [25,26], while effective, face scalability and annotation cost challenges. Automated self-critiquing methods [27] risk inheriting the model’s original knowledge gaps. Trained reward models [28,29] require substantial annotated data, while external verification tools [30] necessitate integration with external resources despite improving accuracy.

6.2. Multi-Agent Debate

One promising direction is multi-agent debate [12,31], where multiple LLMs collaboratively improve their outputs through structured debate. Specifically, research demonstrates that collaborative debate among agents leads to more divergent thinking [10,32] and enhanced factuality and reasoning [11,33]. Building on these advantages, recent studies apply multi-agent frameworks to diverse domains: argumentative essay writing [34,35], summarization evaluation [36,37,38], diagnosis discussions [39,40], and social simulations [41,42]. However, existing multi-agent systems predominantly rely on employing the same model across agents for collaboration [43]. Different models, due to diverse training data and methods, exhibit distinct hallucination patterns [19,20]. Multi-model frameworks can exploit this diversity to achieve higher faithfulness, as model-specific hallucinations cancel out through collaborative refinement [44].

7. Conclusions

We introduce MPSR, a multi-perspective self-reflection framework for generating high-quality public opinion reports. Our framework assigns diverse stakeholder personas to agents who collaboratively refine writing plans through a structured three-stage debate mechanism. This process identifies conflicts, formulates resolution strategies, and produces consensus plans that ensure factual consistency and perspective diversity. To support long-running crises, the Report Fusion mechanism further synthesizes reports across temporal batches, maintaining comprehensive coverage and temporal coherence. Extensive experiments demonstrate that MPSR significantly outperforms baselines in generating factually consistent, perspective-diverse, and logically coherent reports. Further analysis confirms the necessity of each component. Iterative temporal processing is critical for preventing information loss, persona assignment is essential for perspective diversity, self-reflection debate is fundamental to ensuring factual consistency, and structured planning is crucial for high-level logical coherence. While MPSR incurs higher inference cost due to multi-agent and multi-round interactions, this overhead is the justifiable trade-off for improved report quality. Future work will explore more efficient debate mechanisms and investigate the framework’s generalization to other long-form generation tasks beyond Public Opinion Reports.

Author Contributions

Conceptualization, J.Y., W.F. and Y.X.; investigation, J.Y. and W.F.; methodology, J.Y. and Y.X.; analysis, J.Y., W.F. and Y.X.; software, Y.F.; validation, Y.F. and J.Y.; data curation, Y.F.; drafting the manuscript, J.Y.; supervision, J.L., L.Z. and H.S. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Fundamental Research Funds for the Central Universities (Grant No. CUC25SG007), the China Postdoctoral Science Foundation (Grant No. 2025M783493), and the Public Computing Cloud, CUC. This work was also supported by the Beijing Natural Science Foundation (Grant No. 9242003) and the Natural Science Foundation of Chongqing, China (Grant No. CSTB2023NSCQ-MSX0391).

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy and ethical restrictions.

Acknowledgments

During the preparation of this manuscript, the authors used Claude Opus 4.5 for grammar checking and language refinement. The authors have reviewed and take full responsibility for the content of the publication.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A. Persona Pool Details

Table A1 presents a representative subset of the MPSR persona pool. To ensure distinct perspectives, we define each role with a specific functional scope that directly shapes the writing plan’s structure and content prioritization.

Table A1. Representative subset of the Persona Pool. The table details the definition of each role and illustrates how their unique perspectives influence the report’s planning phase.

Role	Description
Refugee	Individuals directly impacted by the crisis, primarily seeking safety and aid information. They prioritize casualty statistics, shelter locations, and aid distribution channels to address immediate survival needs.
Government Officials	Officials responsible for crisis management, policy implementation, and social order. They focus on situational control metrics, policy updates, and social stability risks to support decision-making.
Rescuer/ First Responders	Personnel involved in on-the-ground operations requiring actionable logistical data. They emphasize operational bottlenecks, safety hazards, and real-time hazard assessments to guide field action.
Journalists/Media	Professionals seeking verifiable facts and investigating conflicting narratives. They highlight timeline discrepancies, source verification, and neutrality, ensuring investigative rigor.
Publicist	Experts in public relations assessing the report’s strategic communication value. They direct the plan towards narrative framing, sentiment analysis, and communication advice to manage public perception.
Netizens/ General Public	The general online public expressing sentiments, rumors, or demands for accountability. They focus on trending hashtags, hot-button social media topics, and emotional outbursts to capture the “unofficial” narrative.
Scientific Researchers	Experts looking for data accuracy, root cause analysis, and long-term trends. They demand a plan focused on root cause investigation, data methodology, and technical accuracy.
Corporate Stakeholders/ Local Businesses	Stakeholders representing businesses affected by supply chain disruptions or market volatility. They influence the plan to include economic loss assessments, recovery estimates, and business continuity advice.
International Aid Organizations	Independent bodies focusing on vulnerable groups and gaps in government aid. They prioritize coverage of marginalized communities, human rights protection, and independent aid verification.
Lawyers/ Legal Advisor	Experts focusing on liability, regulatory compliance, and potential litigation. They ensure the plan includes compliance checks, liability risks, and adherence to legal frameworks.

Appendix B. Detailed Error Resolution Strategies

Table A2 presents the detailed error taxonomy and resolution protocol employed in the MPSR framework. It defines the main error categories and provides representative examples and actionable resolution strategies that guide issue identification and plan revision.

Table A2. Error resolution protocol for the MPSR framework. It defines three error categories: Integration Errors and Conflict Errors arising from diverse source documents, and Faithfulness Errors introduced by the generation model. Each error type includes Definition, Example, and Resolution Strategy.

#	Error Type	Definition	Example	Strategy
Integration Errors
1	Redundancy	Same information or facts repeated in both news reports.	A: No injuries have been reported in the explosion. B: No one was hurt when the first bomb exploded remotely near the car.	Remove the same information or facts repeated in both news reports, avoid unnecessary duplication in report.
2	Omission	Additional information not present in the other.	A: A fire broke out at the factory. B: A fire broke out at the factory, causing $50,000 in damage.	Include the additional information that is not present in the other, avoid potentially leading to an incomplete understanding of the event.
Conflict Errors
3	Time Updates	The evolving updates over time between initial news reports and subsequently developing news.	A: …the death of 109 people as a result of landslides…B: …increased to 129, according to information provided by the local authorities.	Overwrite the original information with subsequent information and refrain from mentioning the original information.
4	Perspective & Standpoint Conflicts	Contradictory or incompatible details arising from diverse standpoints, differing viewpoints, or cultural/linguistic discrepancies.	(Perspective): A: …Israeli soldiers…ended in the death of three Palestinian terrorists. B: …Prime Minister…accused the Israeli government of “horrific crimes…” (Cultural): A: …receive about 1.14 billion won…B: …giving 790,000 euros funding…	Coexist with these viewpoints and present them in an appropriate manner, while maintaining neutrality. Reconcile cultural/linguistic discrepancies (e.g., currency) where possible.
Faithfulness Errors
5	Tense/Modality Error	The tense or modal verb (can, may, must, will) in the report misrepresents the certainty or timing conveyed in the source.	Report: The government will implement the policy…Source: The government may implement the policy…	Align the tense and modality in the report with the source to accurately reflect certainty levels and temporal information.
6	Mis-Referencing	An entity, property, or event mentioned in the report is associated with the wrong entity from the source.	Report: President Biden announced new sanctions…Source: The White House spokesperson announced new sanctions…	Correct the entity attribution by identifying the accurate source from the documents and reassigning the property or event accordingly.
7	Semantic Distortion	The report misinterprets or distorts the meaning or implication of information in the source.	Report: …decision to stop composting waste…criticized as “catastrophic”. Source: The composting program itself was a catastrophic failure…	Correct the misinterpretation by accurately reflecting the source’s intended meaning, avoiding semantic shifts.
8	Stating Opinion As Fact	The report presents someone’s opinion, speculation, or subjective statement as an objective fact.	Report: The policy will devastate the economy. Source: Critics argue that the policy will devastate the economy.	Reframe subjective statements by explicitly attributing them to their sources (e.g., “according to X”), distinguishing opinion from fact.

Appendix C. Iterative Report Fusion Algorithm

To ensure the fused report maintains temporal coherence and conciseness, we implement a structured merging logic.

Appendix D. Case Study Details

Table A3 illustrates persona-specific planning, issue prioritization during self-reflection, and the final report written from the consensus plan.

Table A3. Case study on the Singapore Oil Spill. Phase 1 shows persona-specific planning with complementary details in bold, exposing specific errors: Conflict, Integration (Omission), and Faithfulness (Opinion/Tense). Phase 2 details the Self-Reflection process, where the Judge prioritizes these issues and guides agents to apply Resolution Strategies. Phase 3 presents the final report generated from the consensus plan.

Input Sources (Summarized) [News 1-Gov Report] “Over 1500 volunteers participated…containment booms deployed. Report date: June 14.” [News 2-Article] “Oil slick reached Chek Jawa Wetlands. Impact reported on June 15.” [Social Post] “The response was incompetent and far too slow.”
Phase 1: Persona Assignment
Official Agent	“Focus response actions and scale. Authorities deployed containment booms and mobilized 1500+ volunteers with updates on June 14.” ↪ Integration Error (Omission) missed ecological impact context.
Journalist Agent	“Focus event narrative and ecological impact. The spill reached Chek Jawa Wetlands with impact updates on June 15.” ↪ Conflict Error time anchor discrepancy across June 14 and June 15.
Rescuer Agent	“Focus actionable guidance and readiness. Urgent cleanup actions will commence immediately along the shoreline.” ↪ Faithfulness Error (Tense) actions stated with unwarranted certainty.
Netizen Agent	“Focus public sentiment and concerns. Many residents criticized the response as incompetent and far too slow.” ↪ Faithfulness Error (Opinion) subjective critique presented as fact.
Phase 2: Self-Reflection (Expanded Debate Round)
Stage 1: Issue Identification	Judge Prompt: “Identify issues by category: (1) Faithfulness, (2) Conflict, (3) Integration. Prioritize factual accuracy.” Priority 1 [Faithfulness] Opinion & Tense: Attribute Netizen’s critique; soften Rescuer’s certainty. Priority 2 [Conflict] Time Discrepancy: Reconcile June 14 vs. June 15 timeline. Priority 3 [Integration] Omission: Merge Official’s response scale with Journalist’s ecological impact.
Stage 2: Solution Strategy	Judge Guidelines: (1) Reframe/Align: Convert opinions to attribution; adjust modality. (2) Update/Reconcile: Indicate timeline progression. (3) Include: Combine complementary facts from all personas. [Official → Netizen]: “The term ‘incompetent’ is subjective. Strategy: Reframe to ‘Netizens criticized’.” [Rescuer → Official]: “Misses ecological impact. Strategy: Include Chek Jawa location.”
Stage 3: Plan Revision & Voting	Official Revision: Adds attribution and updates timeline logic. Rescuer Revision: Changes “will commence” to “authorities plan to commence”. Voting Result: Official: Plan A Rescuer: Plan B Journalist: Plan A Netizen: Plan A ⇒ Consensus: $\hat{P}$ = Plan A (majority voting)
Phase 3: Report Writing
Generated Report	“An oil spill occurred in Singapore, with reports spanning June 14 to June 15 [Reconciled]. The spill reached the Chek Jawa Wetlands [Included], and authorities mobilized 1500+ volunteers [Included]. Netizens criticized the response pace [Reframed], while cleanup operations are planned [Adjusted Modality] for affected shorelines.” ↪ Strategies Applied: Reconcile (Conflict); Include (Integration); Reframe/Adjust (Faithfulness).

Algorithm A1 Iterative Report Fusion Process

Require: Current Round Report

R_{i}

, Historical Fused Report

R_{i - 1}

Ensure: Updated Fused Report

{\tilde{R}}_{i}

1:: Initialization: Set ${\tilde{R}}_{i} \leftarrow R_{i - 1}$
2:: Segmentation: Parse $R_{i}$ into a set of information units $U = {u_{1}, u_{2}, \dots, u_{n}}$
3:: for each information unit $u_{k}$ in U do
4:: // Step 1: Temporal Conflict Resolution
5:: if $u_{k}$ contradicts an existing claim $c_{o l d}$ in ${\tilde{R}}_{i}$ then
6:: Apply Temporal Recency Principle:
7:: Update ${\tilde{R}}_{i}$ by overwriting $c_{o l d}$ with $u_{k}$ ▹ Prioritize latest status updates
8:: // Step 2: Redundancy Elimination
9:: else if $u_{k}$ is semantically equivalent to any content in ${\tilde{R}}_{i}$ then
10:: Discard $u_{k}$ ▹ Avoid duplication
11:: // Step 3: Information Integration
12:: else
13:: Append $u_{k}$ to ${\tilde{R}}_{i}$ with timestamp ▹ Preserve novel details
14:: end if
15:: end for
16:: Final Synthesis: ${\tilde{R}}_{i} \leftarrow LLM_Rewrite ({\tilde{R}}_{i})$
17:: return ${\tilde{R}}_{i}$

References

Liu, Y.; Yu, J.; Xu, Y.; Li, Z.; Zhu, Q. A survey on transformer context extension: Approaches and evaluation. arXiv 2025, arXiv:2503.13299. [Google Scholar] [CrossRef] [Scilit]
Liang, X.; Tang, Z.; Li, J.; Zhang, M. Open-ended Long Text Generation via Masked Language Modeling. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Rogers, A., Boyd-Graber, J., Okazaki, N., Eds.; Association for Computational Linguistics: Toronto, ON, Canada, 2023; pp. 223–241. [Google Scholar]
Tan, H.; Guo, Z.; Shi, Z.; Xu, L.; Liu, Z.; Feng, Y.; Li, X.; Wang, Y.; Shang, L.; Liu, Q.; et al. ProxyQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Ku, L.W., Martins, A., Srikumar, V., Eds.; Association for Computational Linguistics: Bangkok, Thailand, 2024; pp. 6806–6827. [Google Scholar]
Bai, Y.; Zhang, J.; Lv, X.; Zheng, L.; Zhu, S.; Hou, L.; Dong, Y.; Tang, J.; Li, J. LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs. In Proceedings of the Thirteenth International Conference on Learning Representations, Singapore, 24–28 April 2025. [Google Scholar]
Rafailov, R.; Sharma, A.; Mitchell, E.; Ermon, S.; Manning, C.D.; Finn, C. Direct preference optimization: Your language model is secretly a reward model. In Proceedings of the 37th International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 10–16 December 2023. [Google Scholar]
Muscato, B.; Mala, C.S.; Marchiori Manerba, M.; Gezici, G.; Giannotti, F. An Overview of Recent Approaches to Enable Diversity in Large Language Models through Aligning with Human Perspectives. In Proceedings of the 3rd Workshop on Perspectivist Approaches to NLP (NLPerspectives) @ LREC-COLING 2024; Abercrombie, G., Basile, V., Bernadi, D., Dudy, S., Frenda, S., Havens, L., Tonelli, S., Eds.; ELRA and ICCL: Torino, Italy, 2024; pp. 49–55. [Google Scholar]
Sorensen, T.; Jiang, L.; Hwang, J.; Levine, S.; Pyatkin, V.; West, P.; Dziri, N.; Lu, X.; Rao, K.; Bhagavatula, C.; et al. Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties. In Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada, 26–27 February 2024; Volume 38, pp. 19937–19947. [Google Scholar]
Tang, L.; Goyal, T.; Fabbri, A.; Laban, P.; Xu, J.; Yavuz, S.; Kryscinski, W.; Rousseau, J.; Durrett, G. Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Rogers, A., Boyd-Graber, J., Okazaki, N., Eds.; Association for Computational Linguistics: Toronto, ON, Canada, 2023; pp. 11626–11644. [Google Scholar]
Zhu, R.; Qi, J.; Lau, J.H. Annotating and Detecting Fine-grained Factual Errors for Dialogue Summarization. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Rogers, A., Boyd-Graber, J., Okazaki, N., Eds.; Association for Computational Linguistics: Toronto, ON, Canada, 2023; pp. 6825–6845. [Google Scholar]
Hu, Z.; Chan, H.P.; Li, J.; Yin, Y. Debate-to-Write: A persona-driven multi-agent framework for diverse argument generation. In Proceedings of the 31st International Conference on Computational Linguistics; Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa, H., Eugenio, B.D., Schockaert, S., Eds.; Association for Computational Linguistics: Abu Dhabi, United Arab Emirates, 2025; pp. 4689–4703. [Google Scholar]
Yin, Z.; Sun, Q.; Chang, C.; Guo, Q.; Dai, J.; Huang, X.; Qiu, X. Exchange-of-Thought: Enhancing large language model capabilities through cross-model communication. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing; Bouamor, H., Pino, J., Bali, K., Eds.; Association for Computational Linguistics: Singapore, 2023; pp. 15135–15153. [Google Scholar]
Kaesberg, L.B.; Becker, J.; Wahle, J.P.; Ruas, T.; Gipp, B. Voting or consensus? Decision-making in multi-agent debate. In Proceedings of the Findings of the Association for Computational Linguistics: ACL 2025; Che, W., Nabende, J., Shutova, E., Pilehvar, M.T., Eds.; Association for Computational Linguistics: Vienna, Austria, 2025; pp. 11640–11671. [Google Scholar]
Yu, J.; Xu, Y.; Li, H.; Li, J.; Zhu, L.; Shen, H.; Shi, L. OPOR-Bench: Evaluating Large Language Models on Online Public Opinion Report Generation. Comput. Mater. Contin. 2025, accepted. [Google Scholar] [CrossRef] [Scilit]
Amar, S.; Schiff, L.; Ernst, O.; Shefer, A.; Shapira, O.; Dagan, I. OpenAsp: A Benchmark for Multi-document Open Aspect-based Summarization. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing; Association for Computational Linguistics: Singapore, 2023; pp. 1967–1991. [Google Scholar]
Xiao, W.; Beltagy, I.; Carenini, G.; Cohan, A. PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Association for Computational Linguistics: Dublin, Ireland, 2022; pp. 5245–5263. [Google Scholar]
Wu, W.; Huang, S.; Jiang, Y.; Xie, P.; Huang, F.; Zhao, H. Unfolding the Headline: Iterative Self-Questioning for News Retrieval and Timeline Summarization. In Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2025; Chiruzzo, L., Ritter, A., Wang, L., Eds.; Association for Computational Linguistics: Albuquerque, NM, USA, 2025; pp. 4385–4398. [Google Scholar]
Liu, Y.; Iter, D.; Xu, Y.; Wang, S.; Xu, R.; Zhu, C. G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing; Bouamor, H., Pino, J., Bali, K., Eds.; Association for Computational Linguistics: Singapore, 2023; pp. 2511–2522. [Google Scholar]
Tang, L.; Laban, P.; Durrett, G. MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing; Al-Onaizan, Y., Bansal, M., Chen, Y.N., Eds.; Association for Computational Linguistics: Miami, FL, USA, 2024; pp. 8818–8847. [Google Scholar]
Guerreiro, N.M.; Alves, D.M.; Waldendorf, J.; Haddow, B.; Birch, A.; Colombo, P.; Martins, A.F.T. Hallucinations in Large Multilingual Translation Models. Trans. Assoc. Comput. Linguist. 2023, 11, 1500–1517. [Google Scholar] [CrossRef] [Scilit]
Rawte, V.; Chadha, A.; Sheth, A.; Das, A. Tutorial Proposal: Hallucination in Large Language Models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024): Tutorial Summaries; Klinger, R., Okazaki, N., Calzolari, N., Kan, M.Y., Eds.; ELRA and ICCL: Torino, Italy, 2024; pp. 68–72. [Google Scholar]
Wang, Y.; Wang, M.; Manzoor, M.A.; Liu, F.; Georgiev, G.N.; Das, R.J.; Nakov, P. Factuality of large language models: A survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing; Al-Onaizan, Y., Bansal, M., Chen, Y.N., Eds.; Association for Computational Linguistics: Miami, FL, USA, 2024; pp. 19519–19529. [Google Scholar]
Chen, Q.; Qin, L.; Liu, J.; Peng, D.; Guan, J.; Wang, P.; Hu, M.; Zhou, Y.; Gao, T.; Che, W. Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models. arXiv 2025, arXiv:2503.09567. [Google Scholar] [CrossRef] [Scilit]
Yin, Z.; Sun, Q.; Guo, Q.; Wu, J.; Qiu, X.; Huang, X. Do large language models know what they don’t know? In Proceedings of the Findings of the Association for Computational Linguistics: ACL 2023; Rogers, A., Boyd-Graber, J., Okazaki, N., Eds.; Association for Computational Linguistics: Toronto, ON, Canada, 2023; pp. 8653–8665. [Google Scholar]
Li, B.; Qi, P.; Liu, B.; Di, S.; Liu, J.; Pei, J.; Yi, J.; Zhou, B. Trustworthy AI: From principles to practices. ACM Comput. Surv. 2023, 55, 1–46. [Google Scholar] [CrossRef] [Scilit]
Chen, G.; Zhang, F.; Lin, J.; Lu, C.; Cheng, J. RRHF-V: Ranking responses to mitigate hallucinations in multimodal large language models with human feedback. In Proceedings of the 31st International Conference on Computational Linguistics; Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa, H., Eugenio, B.D., Schockaert, S., Eds.; Association for Computational Linguistics: Abu Dhabi, United Arab Emirates, 2025; pp. 6798–6815. [Google Scholar]
Saunders, J. Improving automated prediction of English lexical blends through the use of observable linguistic features. In Proceedings of the 20th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology; Nicolai, G., Chodroff, E., Mailhot, F., Çöltekin, Ç., Eds.; Association for Computational Linguistics: Toronto, ON, Canada, 2023; pp. 93–97. [Google Scholar]
Wen, X.; Lou, J.; Lu, X.; Ji, Y.; Guan, X.; Lu, Y.; Lin, H.; He, B.; Han, X.; Zhang, D.; et al. On-policy self-alignment with fine-grained knowledge feedback for hallucination mitigation. In Proceedings of the Findings of the Association for Computational Linguistics: ACL 2025; Che, W., Nabende, J., Shutova, E., Pilehvar, M.T., Eds.; Association for Computational Linguistics: Vienna, Austria, 2025; pp. 5215–5231. [Google Scholar]
Xu, W.; Deutsch, D.; Finkelstein, M.; Juraska, J.; Zhang, B.; Liu, Z.; Wang, W.Y.; Li, L.; Freitag, M. LLMRefine: Pinpointing and refining large language models via fine-grained actionable feedback. In Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2024; Duh, K., Gomez, H., Bethard, S., Eds.; Association for Computational Linguistics: Mexico City, Mexico, 2024; pp. 1429–1445. [Google Scholar]
Akyurek, A.F.; Akyurek, E.; Kalyan, A.; Clark, P.; Wijaya, D.T.; Tandon, N. RL4F: Generating natural language feedback with reinforcement learning for repairing model outputs. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Rogers, A., Boyd-Graber, J., Okazaki, N., Eds.; Association for Computational Linguistics: Toronto, ON, Canada, 2023; pp. 7716–7733. [Google Scholar]
Mo, K.; Hu, R. ExpertEase: A multi-agent framework for grade-specific document simplification with large language models. In Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2024; Al-Onaizan, Y., Bansal, M., Chen, Y.N., Eds.; Association for Computational Linguistics: Miami, FL, USA, 2024; pp. 9080–9099. [Google Scholar]
Xiong, K.; Ding, X.; Cao, Y.; Liu, T.; Qin, B. Examining inter-consistency of large language models collaboration: An in-depth analysis via debate. In Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2023; Bouamor, H., Pino, J., Bali, K., Eds.; Association for Computational Linguistics: Singapore, 2023; pp. 7572–7590. [Google Scholar]
Liang, T.; He, Z.; Jiao, W.; Wang, X.; Wang, Y.; Wang, R.; Yang, Y.; Shi, S.; Tu, Z. Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing; Al-Onaizan, Y., Bansal, M., Chen, Y.N., Eds.; Association for Computational Linguistics: Miami, FL, USA, 2024; pp. 17889–17904. [Google Scholar]
Li, Y.; Du, Y.; Zhang, J.; Hou, L.; Grabowski, P.; Li, Y.; Ie, E. Improving multi-agent debate with sparse communication topology. In Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2024; Al-Onaizan, Y., Bansal, M., Chen, Y.N., Eds.; Association for Computational Linguistics: Miami, FL, USA, 2024; pp. 7281–7294. [Google Scholar]
Chu, S.; Kim, J.W.; Wong, B.; Yi, M.Y. Rationale behind essay scores: Enhancing S-LLM’s multi-trait essay scoring with rationale generated by LLMs. In Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2025; Chiruzzo, L., Ritter, A., Wang, L., Eds.; Association for Computational Linguistics: Albuquerque, NM, USA, 2025; pp. 5796–5814. [Google Scholar]
Liu, X.; Song, R.; Wang, X.; Chen, X. Select, Read, and Write: A Multi-Agent Framework of Full-Text-based Related Work Generation. In Proceedings of the Findings of the Association for Computational Linguistics: ACL 2025; Che, W., Nabende, J., Shutova, E., Pilehvar, M.T., Eds.; Association for Computational Linguistics: Vienna, Austria, 2025; pp. 7009–7028. [Google Scholar]
Lee, S.; Cai, Y.; Meng, D.; Wang, Z.; Wu, Y. Unleashing Large Language Models’ Proficiency in Zero-shot Essay Scoring. In Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2024; Al-Onaizan, Y., Bansal, M., Chen, Y.N., Eds.; Association for Computational Linguistics: Miami, FL, USA, 2024; pp. 181–198. [Google Scholar]
Koupaee, M.; Vincent, J.W.; Mansour, S.; Shalyminov, I.; He, H.; Song, H.; Shu, R.; He, J.; Nian, Y.; Wong, A.W.m.; et al. Faithful, Unfaithful or Ambiguous? Multi-Agent Debate with Initial Stance for Summary Evaluation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers); Chiruzzo, L., Ritter, A., Wang, L., Eds.; Association for Computational Linguistics: Albuquerque, NM, USA, 2025; pp. 12209–12246. [Google Scholar]
Min, H.; Lee, Y.; Ban, M.; Deng, J.; Kim, N.H.Y.; Yun, T.; Su, H.; Cai, J.; Song, H. Towards Multi-dimensional Evaluation of LLM Summarization across Domains and Languages. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Che, W., Nabende, J., Shutova, E., Pilehvar, M.T., Eds.; Association for Computational Linguistics: Vienna, Austria, 2025; pp. 14417–14450. [Google Scholar]
Zhou, Y.; Song, L.; Shen, J. MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration. In Proceedings of the Findings of the Association for Computational Linguistics: ACL 2025; Che, W., Nabende, J., Shutova, E., Pilehvar, M.T., Eds.; Association for Computational Linguistics: Vienna, Austria, 2025; pp. 25319–25333. [Google Scholar]
Zhao, Y.; Wang, H.; Zheng, Y.; Wu, X. A Layered Debating Multi-Agent System for Similar Disease Diagnosis. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers); Chiruzzo, L., Ritter, A., Wang, L., Eds.; Association for Computational Linguistics: Albuquerque, NM, USA, 2025; pp. 539–549. [Google Scholar]
Tang, J.; Gao, H.; Pan, X.; Wang, L.; Tan, H.; Gao, D.; Chen, Y.; Chen, X.; Lin, Y.; Li, Y.; et al. GenSim: A General Social Simulation Platform with Large Language Model based Agents. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (System Demonstrations); Dziri, N., Ren, S.X., Diao, S., Eds.; Association for Computational Linguistics: Albuquerque, NM, USA, 2025; pp. 143–150. [Google Scholar]
Wang, Q.; Wang, T.; Tang, Z.; Li, Q.; Chen, N.; Liang, J.; He, B. MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPs. In Proceedings of the Findings of the Association for Computational Linguistics: ACL 2025; Che, W., Nabende, J., Shutova, E., Pilehvar, M.T., Eds.; Association for Computational Linguistics: Vienna, Austria, 2025; pp. 4998–5036. [Google Scholar]
Rawte, V.; Chakraborty, S.; Pathak, A.; Sarkar, A.; Tonmoy, S.T.I.; Chadha, A.; Sheth, A.; Das, A. The Troubling Emergence of Hallucination in Large Language Models–An Extensive Definition, Quantification, and Prescriptive Remediations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing; Bouamor, H., Pino, J., Bali, K., Eds.; Association for Computational Linguistics: Singapore, 2023; pp. 2541–2573. [Google Scholar]
Wan, D.; Chen, J.; Stengel-Eskin, E.; Bansal, M. MAMM-Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaboration. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers); Chiruzzo, L., Ritter, A., Wang, L., Eds.; Association for Computational Linguistics: Albuquerque, NM, USA, 2025; pp. 9882–9901. [Google Scholar]

Figure 1. Comparison of (a) standard generation and (b) MPSR framework. (a) Standard approaches fail to achieve perspective diversity, maintain factual consistency, and establish coherent planning (✗). (b) MPSR synthesizes diverse stakeholder viewpoints through collaborative debate, ensuring factual consistency and logical coherence (✓).

Figure 2. Overview of the MPSR framework for crisis report generation. Left: Documents are organized chronologically into temporal batches. Middle: Each batch undergoes persona-based generation, where agents with diverse personas create independent plans and then collaboratively refine them through structured debate. Right: Final plans are written into reports and iteratively fused for temporal coherence.

Figure 3. Impact of event temporal complexity on generation quality. Each point represents the average G-Eval score across all events with n temporal batches. Baselines degrade as event length increases, while MPSR maintains consistent quality.

Figure 4. Distribution of error types in report generation. The inner ring shows three main categories, while the outer ring displays specific error types.

Table 1. Distribution of event sub-types in OPOR-Bench, detailing the number of events alongside the average count of associated news articles and social media posts per event.

Table 2. Efficiency analysis per event on OPOR-Bench. We report total API calls, average inference latency (seconds), and estimated monetary cost (USD). Fine-tuned models (OPENASP, PRIMERA) run locally with negligible inference cost.

Table 3. Main results comparing MPSR (MA-SM, MA-MM) against baselines across Date F1, G-Eval, and MCS metrics. The MA-MM configuration (GPT-4o + Gemini 2.5 Pro) achieves the best results across all dimensions. ^† denotes statistically significant improvement by MPSR (MA-MM) over that entry p < 0.05. The upward arrow (↑) indicates that higher scores are better. Bold values indicate the best performance in each column.

Table 4. Pearson correlation (r) between human ratings and G-Eval scores over 30 events with three annotators. ^† denotes statistically significant at

p < 0.05

.

Table 4. Pearson correlation (r) between human ratings and G-Eval scores over 30 events with three annotators. ^† denotes statistically significant at

p < 0.05

.

Dimension	Pearson r
Factual Consistency	0.82 ^†
Perspective Diversity	0.69 ^†
Logical Coherence	0.76 ^†

Table 5. Ablation study of MPSR (MA-MM) components using G-Eval scores to provide fine-grained analysis of each component’s impact on different aspects of report quality.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Journal Statistics

Multiple requests from the same IP address are counted as one view.

Event Sub-Type	# Events	# Avg. News	# Avg. Tweets
Natural Disasters
Cyclones	120	21.3	400.9
Floods	26	14.0	551.3
Wildfires	21	16.9	768.2
Earthquakes	17	20.7	761.2
Volcanic	9	24.2	210.3
Others (Natural)	15	25.2	854.5
Human-caused Crises
Traffic	87	16.5	274.8
Industrial	76	15.2	341.5
Health	26	22.0	455.0
Violence	26	22.5	268.3
Political	22	24.6	114.0
Others (Human)	18	17.5	442.3

Model	Total Calls	Time (s)	Cost ($)
OPENASP	-	133.5	-
PRIMERA	-	152.4	-
STG	1	43.2	0.31
MG	5	89.5	0.53
MPSR (MA-SM)	121	382.2	5.01
MPSR (MA-MM)	125	421.3	5.54

Method	Date F1 ↑	G-Eval ↑	MCS ↑
Baselines
OPENASP	0.22 ^†	1.64 ^†	23.10 ^†
PRIMERA	0.16 ^†	1.70 ^†	28.02 ^†
STG (GPT-4o)	0.48 ^†	2.34 ^†	43.03 ^†
STG (Gemini 2.5 Pro)	0.46 ^†	2.40 ^†	46.22 ^†
MG (GPT-4o)	0.53 ^†	2.63 ^†	61.42 ^†
MG (Gemini 2.5 Pro)	0.57 ^†	2.67 ^†	63.12 ^†
MPSR Framework
MA-SM (GPT-4o)	0.58 ^†	3.84 ^†	70.18 ^†
MA-SM (Gemini 2.5 Pro)	0.62 ^†	4.08 ^†	71.48 ^†
MA-MM	0.67	4.54	79.43

Method	Title	Summary					Timeline		Focus			Suggestions				AVG
Method	Tit.	Nat.	T&L.	Par.	Cau.	Imp.	Dat.	Sub.	Top.	Sta.	Emo.	Rel.	Fea.	Gui.	Inn.	AVG
MPSR	4.37	4.62	4.59	4.60	4.56	4.57	3.94	4.40	4.64	4.68	4.63	4.56	4.63	4.64	4.63	4.54
w/o Plan	4.08 ^†	4.19	4.26 ^†	4.21 ^†	4.18 ^†	4.21 ^†	3.86 ^†	3.60	4.16 ^†	4.06 ^†	4.05 ^†	3.71 ^†	3.69	3.69 ^†	3.69 ^†	3.98 ^† (−0.56)
w/o Persona	3.84 ^†	3.75 ^†	3.78 ^†	3.76 ^†	3.84 ^†	3.79 ^†	3.80 ^†	3.97 ^†	3.38 ^†	2.84 ^†	3.45	2.87 ^†	2.94 ^†	3.29 ^†	3.33	3.51 ^† (−1.03)
w/o Self-Ref.	3.36 ^†	2.84 ^†	2.78 ^†	2.79	2.78 ^†	2.79 ^†	3.02 ^†	2.91 ^†	2.84 ^†	3.44 ^†	2.82 ^†	3.27	3.35 ^†	2.81 ^†	2.81 ^†	2.97 ^† (−1.57)
w/o Round	2.79	2.90 ^†	2.87 ^†	2.89 ^†	2.95 ^†	2.92 ^†	2.77 ^†	2.81 ^†	2.76 ^†	2.81 ^†	2.81 ^†	2.78 ^†	2.81	2.80 ^†	2.76 ^†	2.83 ^† (−1.71)