Skip to Content
ElectronicsElectronics
  • Article
  • Open Access

7 November 2026

22 Pages

Hybrid Optimization with Estimation of Distribution and Monte Carlo Tree Search for Cultural-Heritage Fragment Reconstruction

,
and
1
School of Design, Southwest Jiaotong University, Chengdu 611756, China
2
School of Electrical and Electronic Engineering, Chengdu Technological University, Chengdu 611730, China
*
Author to whom correspondence should be addressed.

Abstract

Reconstruction of cultural-heritage fragments requires selecting suitable edge correspondences in the absence of geometric and visual information. LLMFR is a new hybrid framework that integrates large language model-based evaluation with an estimation of distribution algorithm (EDA) and Monte Carlo Tree Search (MCTS). EDA samples candidate fracture edges based on an adaptive probability model, and then MCTS performs a search in the sampled candidate edges set for assemblies. A large language model evaluates the semantic consistency of the reconstructed images. Its global and local feedback updates the EDA probability model for subsequent iterations. We evaluated LLMFR qualitatively on synthetically degraded images and quantitatively on RePAIR under edge damage, pattern staining, fragment loss, and combined damage. The geometry-based GGM achieved the highest mean scores under pattern staining. LLMFR achieved higher mean adjacency F1 and correct placement rate than the geometry-based and texture-based baselines under edge damage, fragment loss, and combined damage.

1. Introduction

Cultural-heritage objects are generally referred to as scrolls, calligraphic works, paintings, ancient documents, ceramics, murals, etc. [1]. Fluctuations in temperature and humidity, microbial decomposition, geological disasters, and human activities can all produce numerous small, irregularly shaped pieces of these objects that damage their text and pictures [2]. Figure 1 shows painted ceiling fragments from Pompeii [3]. For instance, the individual manuscripts from the Library Cave (Cave 17) of the Dunhuang Mogao Grottoes may have been broken into over a hundred pieces [4]. The edges of these are often abraded, carbonized, or creased, and the surface images may have faded, with color bleeding and staining. There is damage to both the shape and the appearance of the fragments.
Figure 1. Painted ceiling fragments from Pompeii. Reproduced from Figure 1 of Zuchtriegel et al. [3] under the Creative Commons Attribution License (CC BY).
Conservation specialists have traditionally reconstructed cultural-heritage objects by visual comparison of fragments [5]. They consider the shape of the edges, the connection of surface patterns, and changes in color and texture according to knowledge of the dynasty and art of the object. This approach is suitable for building a reliable model when there are few damaged areas. The number of fragments and the extent of damage increase, and thus reconstruction becomes more difficult. Reconstruction of a single batch of severely fragmented objects will take the specialists several months or years [6].
Automated reconstruction methods are based on fragment geometry or visual content. Geometry-based methods extract contour feature points or approximate contours as polygons, then rank candidate matches according to geometric agreement [7]. These methods work well on preserved edges, but abrasion, chipping, and curling can cause false matches. Visual content-based methods are based on the texture and color of surfaces [8]. They match handcrafted feature points, analyze grayscale continuity, or extract features with deep learning. The effectiveness of the preserved surface pattern is reduced due to pigment oxidation, staining, and damage to the pattern.
From an optimization perspective, fragment reconstruction requires selecting compatible correspondences among irregular fragment edges and assembling the resulting configuration. Edge degradation reduces the geometric information, and staining and fragment loss increase the uncertainty of appearance-based matching. These conditions motivate a search process that combines geometric information with semantic feedback when evaluating candidate assemblies.
LLMFR (Large Language Model Fragment Reconstruction) is a new hybrid combinatorial search method for fragment image reconstruction. Computer Vision methods extract geometric and appearance features from fragment images. An estimation of distribution algorithm (EDA) samples candidate fracture edges from an adaptive probability model, and Monte Carlo Tree Search (MCTS) performs assembly exploration based on these edges. A large language model assesses the semantic coherence of the assembled pictorial patterns. Its global and local feedback updates the EDA probability matrix to guide candidate-edge sampling in the next iteration.

3. System Model

Fragment reconstruction involves discrete decisions about which edges are candidate fracture edges and which pairs should be matched. The number of possible combinations is directly proportional to the number of fragments and candidate edges. Geometric congruence and the continuity of surface patterns provide support for the above decisions but can also be disrupted. LLMFR solves the combinatorial search problem using probabilistic candidate selection and tree search with semantic evaluation.
Fragment contours contain exterior boundary segments and fracture segments created by fragmentation. In the two-dimensional domain, reconstruction estimates the relative poses of individual fragments; at the combinatorial level, it selects candidate correspondences between fracture segments. Figure 2 illustrates these two types of segments. A fragment adjacency graph records the topological structure induced by the selected correspondences. In the dual edge space, by contrast, the reconstruction process becomes the task of establishing correct one-to-one correspondences between fracture edges to achieve combinatorial restoration. It can be formally proven that, within a topologically connected space, the one-to-one pairing of fracture edges in the edge space is mathematically equivalent to the combinatorial optimization of relative fragment poses in the two-dimensional spatial domain. Accordingly, the fragment reconstruction problem can be reduced to the core task of identifying the correct one-to-one matching between fracture edges.
Figure 2. An example of boundary edges and fracture edges in a fragment.
Theorem 1. 
Let G = ( V , E ) be a connected simple graph whose vertices represent fragments and whose edges represent their adjacency relations induced by correct fracture-edge correspondences. If G is isomorphic to neither K 3 nor K 1 , 3 , its line graph L ( G ) determines G up to graph isomorphism among connected simple graphs.
Proof. 
We use the Whitney Line Graph Isomorphism Theorem [21] to relate the isomorphism of line graphs to that of their underlying graphs.
For an arbitrary simple graph G = ( V , E ) , its line graph L ( G ) (see Figure 3) is defined as follows:
Figure 3. A schematic diagram of G and L ( G ) . (a) The original graph G, with vertices A, B, and C representing fragments. (b) The line graph L ( G ) , with each vertex labeled by the endpoints of the corresponding edge in G.
  • The vertex set of L ( G ) is identical to the edge set of G, i.e., V L ( G ) = E ( G ) ;
  • Two vertices e 1 , e 2 ∈ V L ( G ) are adjacent in L ( G ) if and only if the corresponding edges e 1 and e 2 share a common endpoint in G.
Let G and H be connected simple graphs. Then L ( G ) ≅ L ( H ) if and only if G ≅ H with the unique exception of the graph pair K 3 , K 1 , 3 : L K 3 ≅ L K 1 , 3 ≅ K 3 , but K 3 ≇ K 1 , 3 where K n denotes the complete graph on n vertices, and K 1 , 3 denotes the complete bipartite claw graph with vertex partitions of size 1 and 3. □
Except for the ambiguity between K 3 and K 1 , 3 , a connected simple graph is determined up to isomorphism by its line graph. The Whitney Line Graph Isomorphism Theorem thus confirms that establishing a correct one-to-one correspondence between fracture edges is sufficient to achieve a complete combinatorial reconstruction of the fragments. By reframing the fragment reconstruction problem as the search for a topological one-to-one mapping of fracture edges, this framework circumvents the need for conventional matching criteria such as edge geometric congruence and surface pattern continuity, thereby enabling reliable reconstruction even in scenarios with missing fragments and severe structural degradation.
Finding the correct fracture-edge correspondences remains a difficult combinatorial problem. The reconstruction workflow has three stages: distinguish fracture edges from boundary edges, pair the matching fracture edges, and assemble the fragments from the verified correspondences.

4. Fragment Reconstruction Algorithm

4.1. Fragment Image Extraction

Contour extraction, polygonal approximation, and edge segmentation turn the raw fragment contours into ordered edge segments with geometric attributes [22,23]. Each segment is given a fragment ID and a local edge sequence number for boundary-edge and fracture-edge classification.

4.1.1. Initial Extraction of Closed Contours

A foreground mask is first obtained from the alpha channel of four-channel fragment images or from grayscale conversion followed by fixed-threshold binarization of three-channel images [22]. Extract the outer contour of the mask, and if its area exceeds the lower bound, keep this contour. This retrieval mode excludes internal hole contours. The implementation uses alpha and grayscale thresholds of 128 and 127, respectively, and a minimum contour area of 100 square pixels.
Let the outer contour of the k-th fragment be represented by C k = { c k , 1 , c k , 2 , … , c k , L k } , where L k is the number of contour points and c k , t = x k , t , y k , t gives the pixel coordinates of the t-th point. The points follow the order of the extracted closed contour. Clockwise ordering is imposed on the vertices obtained during subsequent polygonal approximation.

4.1.2. Polygonal Approximation and Edge Segmentation

The continuous contour is discretized using Douglas–Peucker polygonal approximation, with a tolerance expressed as a percentage of the contour perimeter and a default value of 1%. The resulting vertices are ordered clockwise in image coordinates. Each consecutive vertex pair, including the last-to-first pair, defines a polygon edge. The n k segments are indexed sequentially around the contour to form the fragment’s local edge set:
E k = { e k , 1 , e k , 2 , … , e k , n k }
where e k , i is the i-th edge segment of the k-th fragment. Thus, the neighbors of the adjacent areas are still disconnected. It is a local sequence that does not need a spatial arrangement of the edge sets of the fragments.

4.2. EDA-Based Edge Classification

Incremental EDA with an independent Bernoulli probability model selects candidate fracture edges [24]. Each edge has a selection probability. Sampling and coverage repair generate candidate sets, and feedback from the preceding reconstruction cycle updates the probabilities for the next iteration.
Let E = ⋃ k E k denote all fragment edges, including boundary and fracture edges, and let K denote the candidate fracture edge set at the current iteration. For each e ∈ E , define X e ∈ { 0 , 1 } , with X e = 1 indicating inclusion in the initial candidate sample and X e = 0 indicating exclusion. Coverage repair, described below, produces the final candidate set K. Inclusion in this set does not establish that an edge is a true fracture edge.
Each edge is assigned a probability p e ( t ) = P X e = 1 , where t denotes the iteration. For a fixed iteration t, we abbreviate X e ( t ) as X e . Initial sampling assumes independence of edges and obtains the factorized Bernoulli model:
P ( X = x ; p ( t ) ) = ∏ e ∈ E p e ( t ) x e 1 − p e ( t ) 1 − x e
The initial probability is set to p e ( 0 ) = 0.5 for every edge segment, assigning equal prior probabilities to the two classes in the absence of domain-specific prior knowledge.
Each edge is sampled independently according to the current probability model:
X e ( t ) = I U e < p e ( t ) , U e ∼ U ( 0 , 1 )
where I [ · ] is the indicator function and U e is uniformly distributed over ( 0 , 1 ) . The selected edges form the set of candidate fracture edges.
A coverage-repair heuristic is then used. If the fragment does not have selected edges, then the edge with the highest probability will be selected. If the total is still less than 2 ( F − 1 ) , where F is the number of fragments, the unselected edges will be added in descending order of probability until the threshold is reached or no more edges are available. This process generates the set of active candidate fracture edges K.

4.3. MCTS-Based Fragment Reconstruction

MCTS identifies a one-to-one fracture-edge correspondence for optimization of fragment connectivity [25]. These pairs determine the topology of the fragment-level reconstruction, and fragment positions and poses are set in assembly.

4.3.1. Search States and Tree Nodes

EDA supplies the candidate fracture edge set. Each candidate edge is a search unit identified by its parent fragment and local edge index. At each MCTS node, G = V m c t s , E m c t s records the current pairings: V m c t s contains the candidate fracture edges, and E m c t s contains valid edge pairings used for reconstruction. The initial state has an empty E m c t s .
A sequence of fracture-edge pair operations is required for the path from the root to a search node. Further along in the Path, the confirmed pairs and local assemblies are more numerous.

4.3.2. Reconstruction Performance Evaluation System

At each MCTS node, only whether the fracture edges are connected is recorded in G = V m c t s , E m c t s . Overlap errors, rotation angles, displacement, or contour-matching measures are not stored. Evaluation at this time is therefore limited to topology. A fast scoring system is constructed based on four indicators: the number of connected components C, the number of isolated fragments N 0 , the number of over-adjacent fragments deg over , and the number of triangular cycles R 3 .
  • C: The number of connected components, corresponding to disconnected blocks in the final reconstruction graph. The ideal value, C = 1 , represents a single integrated structure. Larger values of C indicate a more fragmented reconstruction.
  • N 0 : The number of degree-zero nodes, representing fragments with no adjacent neighbors. The ideal value is N 0 = 0 ; larger values of N 0 indicate more extensive matching failure.
  • deg over : A higher degree does not necessarily indicate a better reconstruction, since a physical fragment can be adjacent to only a limited number of neighbors, typically four to six. Unusually high degrees suggest excessive false matches. An empirical upper bound is therefore specified according to fragment type. For thin planar fragments, including paper, murals, and pottery shards, the maximum adjacency is d max = 4 ∼ 6 . The number of fragments satisfying deg ( v ) > d max is denoted by deg over .
  • R 3 : Mutual connections among three fragments form a triangular cycle. Such cycles can occur in valid fragment assemblies and are not, by themselves, evidence of a false match. The current scoring function uses their count, R 3 , as a heuristic penalty to discourage premature closed local clusters during search. This heuristic does not explicitly prohibit triangular states, but it may penalize valid clusters.
The comprehensive scoring function S is then defined as:
S = 1 C · 1 − N 0 | V m c t s | · 1 − R 3 | E m c t s | · 1 − deg over | V m c t s |

4.3.3. MCTS

Run MCTS for 1000 tree-search iterations on each pair of fracture-edge decisions and set the exploration constant to 1.414. Each iteration starts at the root and selects the next node from the current reconstruction. Select based on the average reward of past rollouts and explore less frequently visited nodes. Figure 4 shows the MCTS search of consecutive fracture-edge pairs.
Figure 4. Flowchart of MCTS.
If the search finds a non-terminal node that has not been fully explored, a valid pair can be obtained from its unused fracture edges. The resulting state is a child node. Sibling nodes represent alternative pairings from the same state, and tree depth is the number of pairing decisions.
After expansion, a random simulation rollout is initiated from the newly created child node. Valid fracture-edge pairings are successively added until no further pairing is available or the score differs from 1 by no more than 10 − 12 . No maximum rollout length is imposed in the present implementation. Repeated rollouts estimate how a local pairing decision may affect the eventual reconstruction and provide an empirical basis for comparing alternative pairing sequences.
A high-reward pair will not be connected further and will be left alone. A low-reward pair may be suitable for the final assembly. MCTS employs rollouts to approximate the effects on the final reconstruction topology.
The final topological score S of each rollout is backpropagated to the root as the reward, updating the visit count and cumulative reward of every node on the path. These statistics estimate the outcomes of subsequent pairing decisions from each state.
After the prescribed number of tree-search iterations, the root child with the highest average simulation reward determines the pairing selected for the current reconstruction stage. This pairing is applied to the current state, and a new search tree is constructed from the updated state to determine the next pairing. The sequence of pairing decisions continues until no legal pairing remains or the score is within the same tolerance of 1.
When the search terminates, the fracture-edge indices are mapped to their corresponding fragment identifiers and local edge indices, yielding the selected fracture edge correspondences.

4.4. Fragment Reconstruction

Assembly estimates the rotations and translations of a common coordinate system with respect to the one-to-one fracture-edge correspondences [17,22]. Voronoi fragments are created before the reconstruction loop. Contour vertices and edge lengths are extracted once and reused in the assembly. Then, cross-correspondence is performed to extend the rigid-body constraint graph of fragment adjacency via Breadth-First Search (BFS). Alpha blending integrates the aligned fragment layers to construct the reconstructed image.

4.4.1. Voronoi Fragment Generation

Two-dimensional seed points are sampled uniformly over the fragment image. For each pixel position, Euclidean distances to all seeds are computed, and the pixel is assigned to the Voronoi region of the nearest seed. A predefined probability determines whether boundary perturbation is applied to each region, rather than whether individual pixels are deleted. When perturbation is enabled, an eight-neighborhood test identifies the boundary pixels, from which a subset is sampled without replacement for removal. The number of removals is the result of dividing the boundary-pixel count by a random integer from 3 to 8 and taking the floor, with a minimum of 1 for a non-empty boundary. Next, crop the region based on its bounding box and add its mask to the alpha channel to obtain an RGBA image of only the target fragment.

4.4.2. Geometric Feature Extraction

First, read the fragment image and then select the largest outer contour as the representation of the fragment. Douglas–Peucker polygonal approximation reduces this contour to a finite vertex set. The signed area determines the orientation, and the vertex sequence is reversed when necessary to ensure clockwise ordering in image coordinates. Geometry is extracted once before the iterative reconstruction loop, and then assembly reuses the stored vertices and edge lengths.

4.4.3. Edge Correspondence Verification

Three checks precede reconstruction. First, all edge indices are checked against their valid ranges. Second, compare the corresponding edge lengths and, if the difference between the longest and shortest edges is more than 10% of the shortest length, then consider the match invalid. Third, fragment-level BFS verifies that the correspondence graph contains a single connected component. If this graph is disconnected, only the component containing fragment 0 can be placed, and connectivity verification is needed before reconstruction.

4.4.4. BFS Rigid Pose Propagation

Edge Correspondences are converted into Fragment Adjacency Relations. The fixed position of the reference fragment is at ( 0 , 0 ) , and no rotation of BFS is employed. When a constraint connects a placed fragment A to an unplaced fragment B, the rotation and translation of B are determined by the corresponding edge of B, and then B is marked as visited. Pose propagation is a coordinate transformation, using a discrete search over 72 rotation angles with edge midpoint alignment.

4.4.5. Alpha Blending

Based on the determined positions, the axis-aligned extent of each fragment image can be obtained from its half-diagonal length. Increase the Margin of the four-channel canvas on all sides to 100 px. Rotate and position each fragment according to the calculated location in the canvas.

4.5. LLM-Based Reconstruction Scoring

Scores based primarily on contour fitting and texture similarity may provide limited evidence when geometric or appearance cues are degraded. The proposed scoring framework supplements these cues with large language model-based assessments of visual and textual coherence. Archaeological context also provides specific conditions for the application of cultural heritage. The purpose of this assessment is to select candidates, not to verify the historical accuracy of the assembly.
The scoring framework combines physical features, semantic features, and archaeological knowledge through an LLM [16,26]. The reconstruction workflow does not introduce a separate task-specific model training stage; it uses a pretrained language model for evaluation.

4.5.1. Multi-Dimensional Scoring Framework

The scoring framework comprises three levels: basic physical properties, semantic features, and archaeological knowledge. Each Level has two score indices. The above are used to conduct all-around and localized evaluations in the search.
  • Basic Physical Layer: Match the geometric shape, materials, and craftsmanship consistency assessment to ensure the feasibility of reconstruction.
  • Semantic Feature Layer: Continuity of decorative patterns and coherence of text evaluate the semantic consistency of the reconstruction. LLMs contribute most directly at this level.
  • Archaeological Knowledge Layer: Assess the archaeological plausibility of chronological and cultural compatibility and formal-contextual rationality. This level assesses formal compatibility rather than performing rigorous archaeological restoration validation.

4.5.2. Multimodal Scoring Scheme

The LLM assigns scores across the six dimensions and combines them into comprehensive and local assessments. It solves the problem of conflicting geometry and semantics in setting the evaluation score.
The assessment proceeds through the following stages:
  • Preliminary Screening Layer: Information from the basic physical layer is used to rule out obviously incorrect local assessments and obtain an overall judgment of reconstruction performance.
  • Semantic Verification Layer: Determine the continuity of decorative patterns and coherence of text; identify logical errors in pattern logic and inconsistencies in text sequence.
  • Knowledge Validation Layer: Archaeological plausibility is checked with the existing metadata and contextual information of artifacts. Unsupported chronological or cultural judgments are listed as unknown and are not considered verification of historical accuracy.
  • Comprehensive Evaluation Layer: The LLM integrates the preceding assessments with domain knowledge to generate a comprehensive score s t . It also identifies local strengths and prominent errors. s t , m , n denotes the local score assigned to the correspondence between fragment edges m and n at iteration t.
The following are the prompt JSON scoring rules: All assessment criteria scores, the overall score s t , and the valid local correspondence scores s t , m , n are integers ranging from 0 to 10, and a higher score indicates stronger support for the reconstruction or correspondence. Unassessable criteria are shown as null, and unassessable correspondences are not added to the local-score list. Comprehensive and local scores are used directly in the probability-update formulas without normalization.
SYSTEM
You evaluate a candidate reconstruction of cultural-heritage fragments.
Use only the supplied image, identifiers, and metadata. No ground-truth
assembly is provided. Do not invent historical facts or claim access to
an external database. State uncertainty explicitly.
USER
Evaluate the attached assembly.
Fragment IDs: {{fragment_ids}}
Matched pairs [fragment_a, edge_a, fragment_b, edge_b]:
{{matched_pairs}}
Verified artifact metadata: {{metadata_or_none}}
Identifier labels do not imply that a match is correct.
Assess:
1. Visible geometric compatibility.
2. Visible material and craftsmanship consistency.
3. Decorative-pattern continuity.
4. Text continuity where legible text exists.
5. Chronological/cultural compatibility where metadata supports it.
6. Formal/contextual plausibility.
Score each applicable criterion using an integer from 0 to 10:
0 = strong visible contradiction; 5 = ambiguous or mixed evidence;
10 = strong visible support. Use intermediate integers for intermediate
levels of support. Use null for unassessable criteria, not zero.
Return global_score as one integer from 0 to 10 on the same scale,
considering only applicable criteria. Explain how the applicable
criteria support this overall score in their reason fields.
Identify up to three strongest and three weakest assessable supplied
matched pairs. Return exact pair IDs, integer local scores from 0 to 10
on the same support scale, strength/weakness labels, and evidence-based
reasons. Do not invent IDs. Omit unassessable pairs; use an empty
local_scores list if no pair is assessable.
Apply the same scoring scale independently to every reconstruction.
Do not normalize scores or output fractional scores.
Return JSON only with these keys:
{"criteria":[{"name":"...","score":null,"reason":"..."}],
 "global_score":5,
 "local_scores":[{"pair":["f1","e1","f2","e2"],"score":8,
          "type":"strength","reason":"..."}],
 "uncertainty":"...","failure_modes":[],"limitations":[]}
The shown values illustrate the schema, not the evaluation result.
Before saving the feedback, check that the score is a whole number between 0 and 10, and confirm that the local correspondence identifier matches the provided one. The valid global and local scores are added to the update formula in their original 0–10 scale without division by 10 or other normalization.

4.6. Probability Model Matrix Update for EDA

Before iteration t, the probability model is updated using feedback from the reconstruction produced by MCTS at iteration t − 1 . The update applies to the used-edge set U t − 1 . Edges with high-scoring or low-scoring correspondences use their local scores s t − 1 , m , n ; edges without an explicit local assessment use the comprehensive reconstruction score s t − 1 . Candidate edges are then resampled from the updated probability model.
For edges without an explicit local assessment, the update before iteration t ≥ 2 uses Δ s t = s t − 1 − s t − 2 . The baseline is s 0 = 0 , and the first iteration uses Δ s 1 = 0 . For an edge assigned a valid local correspondence assessment from iteration t − 1 , the feedback signal is Δ s t , m , n = s t − 1 , m , n − s t − 1 , comparing the local score with the comprehensive score of the same reconstruction. Both scores use the same integer 0–10 scale, so Δ s t and Δ s t , m , n are computed directly in score points without normalization. The formulas below describe the comprehensive-score update; the corresponding local signal is substituted for edges with a local assessment. Edge-selection probabilities p e ( t ) and correspondence scores s t , m , n describe different quantities.

4.6.1. Adaptive Learning Rate

Feedback is applied only to the edges used in the reconstruction selected at iteration t − 1 , denoted by U t − 1 , rather than to the entire candidate set K. The update computes p e ( t ) from p e ( t − 1 ) before candidate sampling at iteration t. No update is applied if the used-edge set is empty or Δ s t is zero within an absolute tolerance of 10 − 12 .
An increase in the evaluation score provides a positive update signal for the fracture edges used in the preceding reconstruction, without establishing a causal contribution for each edge. Their selection probabilities are therefore increased, whereas a decrease in the score reduces these probabilities. Restricting the update to used edges avoids assigning global feedback to candidates that did not contribute to the final reconstruction.
The adaptive learning rate is defined as:
α t = α min + ( α max − α min ) | Δ s t | 1 + | Δ s t |
where α min = 0.2 and α max = 0.6 . The term Δ s t 1 + Δ s t is a monotonic increasing saturation function. Small changes in the score result in a small adjustment, and only large changes increase the learning rate to 0.6, thereby enhancing the impact of significant feedback on the probability model.

4.6.2. Probability Model Matrix

If U t − 1 is empty, all probabilities remain unchanged. For each used edge, a zero applicable feedback signal leaves that edge’s probability unchanged. For a nonzero comprehensive-score signal, the binary learning target is defined below; for an edge with a local assessment, its local signal replaces Δ s t in both the learning rate and the target. Thus, the learning rate and target are evaluated per edge when local feedback is used.
y t = 1 , Δ s t > 0 , 0 , Δ s t < 0 .
For each used edge e with a nonzero feedback signal from the reconstruction at iteration t − 1 , exponential smoothing updates its selection probability:
p ˜ e ( t ) = p e ( t − 1 ) + α t y t − p e ( t − 1 )
Equivalently,
p ˜ e ( t ) = ( 1 − α t ) p e ( t − 1 ) + α t y t
For Δ s t > 0 , the target is y t = 1 , giving p ˜ e ( t ) = p e ( t − 1 ) + α t 1 − p e ( t − 1 ) , which increases the probability towards 1. For Δ s t < 0 , the target is y t = 0 , giving p ˜ e ( t ) = 1 − α t p e ( t − 1 ) , which decreases the probability towards 0.
To prevent premature convergence to the deterministic values 0 and 1, the updated probabilities are restricted to the interval [ 0.05 , 0.95 ] :
p e ( t ) = min 0.95 , max 0.05 , p ˜ e ( t )
This clipping operation maintains a minimum degree of exploration. An edge that has repeatedly received negative evaluations retains at least a 5% probability of being sampled in a subsequent iteration. Similarly, repeated positive evaluations do not make its selection deterministic.
The probabilities of edges outside U t − 1 are left unchanged:
p e ( t ) = p e ( t − 1 ) , e ∉ U t − 1

4.7. Overall Workflow

Figure 5 shows the LLMFR workflow. Computer vision methods extract contour geometry and surface pictorial information. EDA samples candidate fracture edges, and MCTS searches for assemblies using those candidates. The LLM evaluates the reconstructed visual content and provides global and local feedback to update the EDA probability matrix for the next iteration.
Figure 5. Algorithm flowchart of LLMFR.

5. Experiments

After edge damage, pattern staining, fragment loss, and combined damage were observed in LLMFR. We artificially reduced the original images by adding complex visual information for qualitative comparison with geometry-based and texture-based baselines. Figure 6 shows the untreated reference sample and the sample after artificial pattern staining.
Figure 6. (a) Intact baseline sample. (b) The same sample with simulated pattern staining.
We used greedy geometric matching (GGM) and pattern texture stitching (PTS) as baselines. GGM reconstructs fragments by aligning their contours through iterative nearest-point computation [17]. PTS reconstructs fragments by matching texture features [2].
Experiments used an Intel Core i5-14600KF processor, an NVIDIA RTX 3090 graphics card, and 32 GB of RAM. The software environment comprised Python 3.7.9 and OpenCV 4.13.0. The original language model-based evaluation was described as using ChatGPT 5.6 Sol [27,28,29]. The exact model version and inference settings of the original scorer are given in Table 1 and the alternative model versions are identified in Section 5.5.1.
Table 1. Model version and inference settings of the original ChatGPT scorer.
RePAIR was used for the quantitative evaluation [17]. It contains digitized fresco fragments from Pompeii, including irregular fractures, erosion, and missing pieces. Expert-assembled groups provide reference configurations. We used version 2 of the public dataset (https://doi.org/10.5281/zenodo.15800029), specifically the 2D_SOLVED.zip archive, which provides fragment images and reference transformations for two-dimensional reconstruction. Groups were included if they contained at least two fragments with readable images and valid reference poses. Open-discovery fragments without verified solutions were not counted in the quantitative score. Ground-truth configurations were used only for evaluation and were not provided to the reconstruction method or the LLM scorer.
Table 2 shows the size strata of the evaluated RePAIR subset before the additional removal. Use the same test set for the four damage conditions and repeat each test case five times. The total number of subsets is 100, and among them, 40 have 2–5 fragments, 40 have 6–10 fragments, and 20 have 11–44 fragments. Edge damage and pattern staining do not destroy all the pieces. Repeated damage versions of the same group are not considered new independent source groups.
Table 2. Composition of the evaluated RePAIR subset before additional removal.
The four conditions below were constructed by applying additional controlled perturbations to the selected RePAIR groups rather than using official RePAIR damage categories. The settings were 20% additional boundary-pixel removal, 40% artificial surface staining, and removal of 15–20% of the supplied fragments. The combined-damage condition applied all three perturbations. Fragment-removal counts were rounded to integers, so the realized removal fractions could differ from the target proportions. Damage added was measured relative to the released input, and natural deterioration has not occurred. All the methods used the same perturbed input for all the test cases. The figures in this section are synthetic qualitative examples, and Table 3, Table 4, Table 5 and Table 6 present the quantitative results of RePAIR.
Table 3. Quantitative results on RePAIR under edge damage (20% additional boundary-pixel removal). Values are means ± standard deviations across group averages. An upward arrow indicates that higher values are better.
Table 4. Quantitative results on RePAIR under pattern staining (40% artificial surface staining). Values are means ± standard deviations across group averages. An upward arrow indicates that higher values are better.
Table 5. Quantitative results on RePAIR under fragment loss. Values are means ± standard deviations across group averages. An upward arrow indicates that higher values are better.
Table 6. Quantitative results on RePAIR under combined damage. Values are means ± standard deviations across group averages. An upward arrow indicates that higher values are better.
Adjacency F1 F 1 , adj was computed from sets of unordered fragment pairs. Let P denote the set of predicted fragment pairs and G the set of ground-truth adjacent fragment pairs. Each unordered pair was included only once, regardless of the number of matched edge segments between the two fragments. Predicted pairs were obtained from the final fragment correspondences used for assembly. The numbers of true positives, false positives, and false negatives were defined as TP = P ∩ G , FP = P ∖ G , and FN = G ∖ P , respectively. For a nonzero denominator, adjacency F1 is
F 1 , adj = 2 TP 2 TP + FP + FN .
For the fragment-loss experiments, the ground-truth adjacency graph was restricted to the retained fragments. Fragments without the corresponding pieces were not employed as ground-truth neighbors. Due to the proximity of the fragments, matching of individual edge segments is not required for this index.
The correct placement rate was computed as the percentage of all input fragments whose reference-centroid positions and orientations agreed with their ground-truth poses within the evaluation tolerances. Reference centroids and fragment areas were computed from the released masks before additional degradation so that artificial erosion did not change the evaluation landmarks. The rotation tolerance was 5°, using the smallest angular difference modulo 360°. Unplaced fragments were counted as incorrect.
For each condition, the scores of the five reconstruction runs in a test instance and all instances in a RePAIR group were averaged. Table 3, Table 4, Table 5 and Table 6 report the mean ± standard deviation of the group averages, with equal weight for each group. Adjacency F1 is between 0 and 1, and the proportion of correctly placed items is shown as a percentage.

5.1. Edge Breakage Test Scenario

Edge damage is common in cultural-heritage fragments and arises mainly from burial-environment erosion and mechanical impacts during excavation. Abrasion and exfoliation along fragment edges degrade the geometry of the original fracture surfaces. We simulated edge weathering by randomly deleting edge pixels at an erosion rate of 20%, defined as the ratio of damaged edge pixels to the total number of edge pixels in a fragment.
As shown in the synthetic example of Figure 7, LLMFR appears to keep the fragment arrangement better than both baselines. In panel (c) of Figure 7, Figure 8, Figure 9 and Figure 10, the cyan numbers identify individual fragments, and the smaller numbers along each fragment’s outline identify its edge segments, numbered separately for each fragment. It also had the highest mean adjacency F1 and correct placement rate among the three methods for RePAIR with edge damage (Table 3). Its core advantage lies in its capacity to effectively distinguish original fracture edges from weathered boundary edges via the EDA module, which filters out noise interference from damaged edges and restricts geometric matching exclusively to valid fracture edges. Combined with the global search optimization capability of MCTS, this mechanism significantly mitigates the adverse impact of edge fracture on reconstruction accuracy.
Figure 7. Qualitative reconstruction results for a synthetic edge-damage example. (a) GGM. (b) PTS. (c) LLMFR.
Figure 8. Qualitative reconstruction results for a synthetic pattern-staining example. (a) GGM. (b) PTS. (c) LLMFR.
Figure 9. Qualitative reconstruction results for a synthetic fragment-loss example. (a) GGM. (b) PTS. (c) LLMFR.
Figure 10. Qualitative reconstruction results for a synthetic combined-damage example. (a) GGM. (b) PTS. (c) LLMFR.

5.2. Pattern-Staining Test Scenario

Pigment exfoliation, surface calcification, and staining can hide or change the original surface pattern and texture. Black blocks and white scratch strips are used to create pattern stains. Set the proportion of the stained area to 40% of the surface of the fragment and count the number of stained pixels divided by the total number of surface pixels.
In the synthetic staining example in Figure 8, LLMFR shows better continuity of visible pictorial content than the baselines. On the evaluated RePAIR groups, however, GGM achieved the highest mean adjacency F1 and correct placement rate under pattern staining (Table 4). The added surface staining does not change the fracture-edge shape directly and may be responsible for the enhanced performance of GGM in this case.

5.3. Fragment-Loss Test Scenario

Historical deterioration and the loss of individual fragments leave gaps in an assembly, reducing the available matching context and increasing the risk of false matches. We artificially introduced fragmentation loss by random deletion. The proportion of target removal from the original fragment count was 15–20%, and the number of removals was rounded to an integer.
Figure 9 shows a synthetic fragment-loss example. On RePAIR, LLMFR achieved the highest mean adjacency F1 and correct placement rate among the three evaluated methods under fragment loss (Table 5). Conventional algorithms depend on local adjacent feature matching; missing fragments break the continuity of matching chains, resulting in a substantial number of misaligned pairings. The proposed LLMFR algorithm integrates the MCTS strategy, which effectively narrows the matching search space even in the presence of incomplete fragments.

5.4. Compound Interference Test Scenario

There are many types of damage to cultural-heritage fragments. To investigate the combined effects, edge damage, pattern staining, and fragment loss were introduced simultaneously.
Figure 10 shows a synthetic combined-damage example. On Repair, LLMFR achieved a higher mean adjacency F1 and correct placement rate than both baselines in the presence of damage (Table 6). For LLMFR under combined damage, the adjacency F1 score is 0.48, and the correct placement rate is only 37%. Erosion reduces the geometric constraints; Staining damages semantic continuity; Missing fragments lose intermediate adjacency evidence. An incorrect match may result in a false adjacent constraint and increase the reconstruction error. These reasons accumulate misalignment during the iteration process and lead to failure. In the future, experts will review the uncertain or conflicting assessments and correct the feedback to update the probability.

5.5. Sensitivity Analysis and Limited Ablation

The sensitivity and limited-ablation study used the 40-fragment synthetic edge-damage case in Figure 7, and the set condition of 20% boundary-pixel damage was employed. We studied the scorer substitution and removal of adaptive probability updates.

5.5.1. Scorer Substitution

The original scorer was compared with Gemini 3.1 Pro and DeepSeek-V4.1-Flash using the same visual inputs, prompt, fragment and edge identifiers, and metadata. Table 1 reports the configuration of the original ChatGPT scorer. The JSON prompt and integer 0–10 scoring protocol are specified in Section 4.5.2. Global and local scores are used directly in the update formulas without normalization, as described in Section 4.6.
Table 7 shows the scorer-substitution results on the 40-fragment synthetic edge-damage case by ChatGPT, Gemini, and DeepSeek. It can be seen that different models have little impact.
Table 7. Scorer-substitution results on the 40-fragment synthetic edge-damage case in Figure 7.

5.5.2. Repeated Scoring of Identical Input

The fixed image in Figure 7a, corresponding to the GGM output, was scored ten times without rerunning reconstruction. ChatGPT 5.6 Sol was used for all ten calls, with no conversation history shared between requests. Table 8 summarizes all ten global-score records, with a mean of 3.60 and a sample standard deviation of 0.5. This indicates that the fluctuation of repeated scoring is relatively small.
Table 8. Repeated scoring of the fixed GGM reconstruction in Figure 7a.

5.5.3. Fixed-Probability Ablation

The fixed-probability variant sets p e ( t ) = 0.5 for every edge and every iteration, disabling both global and local feedback updates to the EDA model while retaining the other reconstruction and selection settings. It was run five times using the same ChatGPT scorer and paired reconstruction seeds as the full method. Table 9 shows the fixed-probability results. The mean adjacency F1 and the mean correct placement of this variant were 0.40 and 39%, respectively; those of the full ChatGPT method were 0.85 and 85% (Table 7). The corresponding decreases were 0.45 F1 and 46 percentage points. Under fixed-probability conditions, LLMFR cannot effectively identify fracture edges, causing the reconstruction process to approximate random combinations and resulting in very poor performance.
Table 9. Fixed-probability ablation on the 40-fragment synthetic edge-damage case in Figure 7.

5.6. Computational Cost and Reproducibility

External LLM calls are distinct from the 1000 internal MCTS iterations performed per pairing decision. Table 10 reports approximate resource totals for one complete reconstruction of the 40-fragment synthetic edge-damage case in Figure 7.
Table 10. Approximate resource totals for one complete reconstruction of the 40-fragment synthetic edge-damage case in Figure 7.

6. Conclusions

LLMFR combines EDA-based candidate-edge sampling, MCTS-based assembly search, and language model-based semantic evaluation. Global and local feedback from the language model updates the probability model used to sample edges for subsequent reconstructions. We evaluated the framework on synthetic examples and RePAIR under edge damage, pattern staining, fragment loss, and combined damage. LLMFR achieved the highest mean adjacency F1 and correct placement rate under edge damage, fragment loss, and combined damage, while GGM achieved the highest mean scores under pattern staining. Substantial placement errors remained under combined damage. Further work should test broader fragment collections and damage levels, isolate the contributions of individual components, and assess computational cost and sensitivity to the LLM scorer. Meanwhile, incorporating expert human intervention into the iterative process to promptly correct errors will lead to more effective repairs.

Author Contributions

Methodology, J.Y.; validation, X.S.; investigation, J.Y.; writing—original draft preparation, J.Y.; writing—review and editing, Z.H. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

The RePAIR dataset used for the quantitative evaluation is publicly available at https://doi.org/10.5281/zenodo.15800029 (version 2).

Acknowledgments

The authors thank the editors and reviewers for their contributions to the manuscript.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:
LLMlarge language model
MCTSMonte Carlo Tree Search
EDAEstimation of Distribution Algorithm
LLMFRLarge Language Model Fragment Reconstruction
SURFSpeeded-Up Robust Features
ORBOriented FAST and Rotated BRIEF
RANSACRandom Sample Consensus
GGMgreedy geometric matching
PTSpattern texture stitching

References

  1. Nuri, A.; Ismayil, Z.; Babayeva, M.; Guliyev, A.; Rzayeva, F.; Shiraliyeva, G.; Jahangirli, T. Artistic Expressions as Vehicles of Cultural Memory: Bridging Identity, Heritage, and Intercultural Understanding. J. Ethn. Cult. Stud. 2025, 12, 258–275. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  2. Derech, N.; Tal, A.; Shimshoni, I. Solving archaeological puzzles. Pattern Recognit. 2021, 119, 108065. [Google Scholar] [CrossRef] [Scilit]
  3. Zuchtriegel, G.; Brunetto, M.A.; Gravina, E.; Napolitano, M.C.; Ricciardi, F.S.; Zambrano, A.; Khoroshiltseva, M.; Palmieri, L.; Pelillo, M.; Vascon, S.; et al. The RePAIR Project: Datasets for archaeological and restoration studies in Pompeii. In Proceedings of the Digital Heritage International Congress, Siena, Italy, 8–13 September 2025. [Google Scholar] [CrossRef]
  4. Xu, H. Determining the Original Appearances of the Scrolls from the Dunhuang Library Cave Based on Their Inventory Records. J. Zhejiang Univ. 2023, 53, 86–103. [Google Scholar]
  5. Fang, L.; Han, B.; Bi, M.; Wang, L.; Wang, D. DCAF-GAN: Enhancing historical landscape restoration with dual-branch feature extraction and attention fusion. PLoS ONE 2025, 20, e0334532. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  6. Wang, H.; Zang, Y.; Liang, F.; Dong, Z.; Fan, H.; Yang, B. A Probabilistic Method for Fractured Cultural Relics Automatic Reassembly. J. Comput. Cult. Herit. 2021, 14, 5. [Google Scholar] [CrossRef] [Scilit]
  7. Rasheed, N.A.; Nordin, M.J. Reconstruction algorithm for archaeological fragments using slope features. ETRI J. 2020, 42, 420–432. [Google Scholar] [CrossRef] [Scilit]
  8. Eslami, D.; Di Angelo, L.; Di Stefano, P.; Guardiani, E. A Semi-Automatic Reconstruction of Archaeological Pottery Fragments from 2D Images Using Wavelet Transformation. Heritage 2021, 4, 76–90. [Google Scholar] [CrossRef] [Scilit]
  9. Barbieri, L.; Fuoco, F.; Bruno, F.; Muzzupappa, M. Exhibit supports for sandstone artifacts designed through topology optimization and additive manufacturing techniques. J. Cult. Herit. 2022, 55, 329–338. [Google Scholar] [CrossRef] [Scilit]
  10. Shahar, O.I.; Elkin, G.; Ben-Shahar, O. The Missing GAP: From Solving Square Jigsaw Puzzles to Handling Real World Archaeological Fragments. arXiv 2026, arXiv:2605.12077. [Google Scholar]
  11. de Lima-Hernandez, R.; Vergauwen, M. A Hybrid Approach to Reassemble Ancient Decorated Block Fragments through a 3D Puzzling Engine. Remote Sens. 2020, 12, 2526. [Google Scholar] [CrossRef] [Scilit]
  12. Gigilashvili, D.; Lukesova, H.; Gulbrandsen, C.F.; Harijan, A.; Hardeberg, J.Y. Computational techniques for virtual reconstruction of fragmented archaeological textiles. Herit. Sci. 2023, 11, 259. [Google Scholar] [CrossRef] [Scilit]
  13. Zheng, Y.T.; Li, X.L.; Yin, Z.X.; Gao, G.; Weng, Y. Multi-feature fusion based automatic reconstruction in related to Chinese ancient manuscript fragments of Dunhuang. J. Image Graph. 2023, 28, 2330–2342. [Google Scholar]
  14. Paumard, M.-M.; Picard, D.; Tabia, H. Deepzzle: Solving visual jigsaw puzzles with deep learning and shortest path optimization. IEEE Trans. Image Process. 2020, 29, 3569–3581. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  15. Hu, Q.; Huang, W.; Luo, Y.; Cao, R.; Peng, X.; Peng, J.; Fan, J. Sgrgan: Sketch-guided restoration for traditional Chinese landscape paintings. Herit. Sci. 2024, 12, 163. [Google Scholar] [CrossRef] [Scilit]
  16. Zhang, Y.; Li, H.; Zhang, S.; Wang, R.; He, B.; Dou, H.; Yan, J.; Zhang, Y.; Wu, F. LLMCO4MR: LLMs-aided Neural Combinatorial Optimization for Ancient Manuscript Restoration from Fragments with Case Studies on Dunhuang. In Proceedings of the 18th European Conference on Computer Vision (ECCV 2024); Springer: Cham, Switzerland, 2024; pp. 253–269. [Google Scholar] [CrossRef] [Scilit]
  17. Tsesmelis, T.; Palmieri, L.; Khoroshiltseva, M.; Islam, A.; Elkin, G.; Shahar, O.I.; Scarpellini, G.; Fiorini, S.; Ohayon, Y.; Alali, N.; et al. Re-assembling the past: The RePAIR dataset and benchmark for real world 2D and 3D puzzle solving. In Advances in Neural Information Processing Systems; Curran Associates, Inc.: Red Hook, NY, USA, 2024; Volume 37, pp. 30076–30105. [Google Scholar]
  18. Islam, A.; Fiorini, S.; James, S.; Morerio, P.; Del Bue, A. ReassembleNet: Learnable Keypoints and Diffusion for 2D Fresco Reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Honolulu, HI, USA, 19–23 October 2025; pp. 9048–9057. [Google Scholar]
  19. Gao, Y.; Zhu, H.; Zhang, Z.; Wang, Y.; Ai, Q.; Zhu, J. Resilient hierarchical control strategy for digital twin cloud-edge collaboration in multi-energy microgrids under non-ideal communication conditions. Appl. Energy 2026, 424, 128374. [Google Scholar] [CrossRef] [Scilit]
  20. Liu, C.; Kou, Y.; Wang, S.; Xie, Z.; Su, Y. Research on traffic flow prediction of progressive graph convolutional networks based on spatio-temporal self-attention mechanism. Sci. Rep. 2026, 16, 14112. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  21. Harary, F. Graph Theory; Addison-Wesley: Reading, MA, USA, 1969; pp. 72–75. [Google Scholar]
  22. Zheng, Y.; Li, X.; Weng, Y. Reunion helper: An edge matcher for sibling fragment identification of the Dunhuang manuscript. Herit. Sci. 2024, 12, 52. [Google Scholar] [CrossRef] [Scilit]
  23. Douglas, D.H.; Peucker, T.K. Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica 1973, 10, 112–122. [Google Scholar] [CrossRef] [Scilit]
  24. Mühlenbein, H.; Paass, G. From recombination of genes to the estimation of distributions I. Binary parameters. In Parallel Problem Solving from Nature—PPSN IV; Springer: Berlin, Germany, 1996; pp. 178–187. [Google Scholar]
  25. Browne, C.B.; Powley, E.; Whitehouse, D.; Lucas, S.M.; Cowling, P.I.; Rohlfshagen, P.; Tavener, S.; Perez, D.; Samothrakis, S.; Colton, S. A survey of Monte Carlo tree search methods. IEEE Trans. Comput. Intell. AI Games 2012, 4, 1–43. [Google Scholar] [CrossRef] [Scilit]
  26. Brown, T.B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems (NeurIPS 2020); Curran Associates, Inc.: Red Hook, NY, USA, 2020; Volume 33, pp. 1877–1901. [Google Scholar]
  27. OpenAI. Previewing GPT-5.6 Sol: A Next-Generation Model. Available online: https://openai.com/index/previewing-gpt-5-6-sol/ (accessed on 3 November 2026).
  28. OpenAI. GPT-5.6: Frontier Intelligence that Scales with your Ambition. Available online: https://openai.com/index/gpt-5-6/ (accessed on 3 November 2026).
  29. OpenAI. GPT-5.6 System Card. Available online: https://deploymentsafety.openai.com/gpt-5-6/ (accessed on 3 November 2026).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.