UTTSR: A Novel Non-Structured Text Table Recognition Model Powered by Deep Learning Technology

Abstract

To prevent the compilation of documents, many table documents are formatted with non-editable and non-structured texts such as PDFs or images. Quickly recognizing the contents of tables is still a challenge due to factors such as irregular formats, uneven text quality, and complex and diverse table content. This article proposes the UTTSR table recognition model, which consists of four parts: text region detection, text line detection and recognition, and table sequence recognition. For table detection, the Cascade Faster RCNN with the ResNeXt105 network is implemented, using TPS (Thin Plate Spline) transformation and affine transformation to correct the image and to improve accuracy. For text line detection, DBNET is used with Do-Conv in FPN (Feature Pyramid Networks) to speed up training. Text lines are recognized using CRNN without the CTC module, enhancing recognition performance. Table sequence recognition is based on the transformer combined with post-processing algorithms that fuse table structure sequences and unit grid content. Experimental results show that the UTTSR model outperforms the compared methods. This upgraded model significantly improves the accuracy of the previous state-of-the-art F1 score on complex tables, reaching 97.8%.

1. Introduction

There is currently a high demand from enterprises and organizations regarding non-structured text recognition for academics and the industry. The recognition of three-line, half-line, and complex tables containing special characters and formulas needs to be revised. Many researchers use deep learning methods to solve the problems of table recognition of unstructured text. For example, Tejas et al. [1] proposed an end-to-end table structure recognition in scanned documents. CascadeTabNet [2] generates table structures with cell-level information to stress the complex relationship of merged cells. There is no better solution for blurry tables and curved table borders.

Currently, fast and accurate table content recognition is still a challenge because of irregular forms, uneven text quality, and complex and diverse table content. This paper proposes a novel form recognition algorithm UTTSR that includes table structure sequence recognition, text detection and recognition to extract non-structural text content. Our contributions are summarized as follows:

(1)

Table detection utilizes Cascade Faster R-CNN with the ResNeXt105 to extract image features. Additionally, TPS transformation and radiation are employed to enhance the accuracy of table detection.

(2)

Improved DBNet and attention-based CRNN are utilized for the detection and recognition phase of text lines in order to identify simple formulas and special characters.

(3)

TableMaster is implemented to convert the resulting table structure into an HTML format that can be easily used for future applications.

The remaining part of this article is structured as follows. Section 2 provides an overview of related works on table recognition. Section 3 describes our methodology, while Section 4 presents our experimental results and analysis. Finally, we conclude the article in Section 5.

2. Related Work

2.1. Table Detection

Early methods for table detection have relied on image processing and some text information. For example, Watanabe et al. [3] and Hirayama [4] processed scanned documents to obtain text blocks and horizontal and vertical lines that could be used to locate tables. Ramel et al. [5] found the first horizontal line at the top of the table and searched for other regions by matching 4 “T”-shaped templates in nine cases where frame lines intersected. Watanabe et al. [6] used horizontal and vertical lines, as well as the upper left corner of the cell, as a reference point to determine the table position.

In recent years, numerous deep-learning-based detection algorithms for table detection have emerged. For instance, Schreiber et al. [7] adopted Faster R-CNN [8] for table detection. Li et al. [9] proposed a progressive scale expansion network (PSENet), which accurately detects text of any shape and solves the problem of relatively poor cell detection. Zhang et al. [10] proposed the VSR network, which combines the visual and semantic information of documents.

2.2. Table Structure Recognition

Table structure recognition involves identifying the layout or hierarchical structure of the table’s content, which mainly includes the specific position of the cell’s row and column, as well as the relationship between cells. Early researchers typically designed heuristic algorithms to complete table structure recognition. For example, Rahgozar et al. [11] presented a table structure recognition method based on rows and columns and first identified text blocks in the image. Khan et al. [12] used a sequence model to extract the table structure and to classify rows and columns. Xue et al. [13] proposed a trainable table graph reconstruction network to manage and extract meaningful information for table structure. Ye et al. [14] divided table recognition into four sub-tasks: table structure detection, text line detection, text line recognition, and box assignment. Pascal et al. [15] proposed multi-type TD-TSR for table detection, providing an end-to-end solution for table recognition.

In recent years, numerous deep-learning-based methods have been proposed for table structure recognition. For instance, the DeepDeSRT system [7] uses FCN [16] as the basic architecture to perform semantic segmentation of rows and columns. Paliwal et al. [17] proposed a novel terminal deep learning model, Tablenet, that processes table detection and structure recognition using the semantic segmentation framework. DeepTabStR [18] applies variable convolution to the target detection network, detects rows, columns, and cells simultaneously, and restores the table according to the location characteristics of the cells. PubTabNet [19] provides the HTML code of the table structure and the text content of each cell. In the encoder-dual decoder model, EDD, a single convolutional neural network is used in the encoding stage, while two recurrent neural networks are used in the decoding stage. Qiao et al. [20] proposed the LGPMA network, which focuses on cell detection. The network fully utilizes local and global visual features to obtain more reliable aligned cell regions through the proposed mask re-scoring strategy, and it uses soft labels to subtly solve the interference of blank cells. Long et al. [21] proposed the Cycle-CenterNet for table structure recognition on a natural scene table recognition dataset WTW. Based on CenterNet [22], the center of the cell and the intersection of four cells are detected simultaneously, and the table structure is directly restored after cell detection is completed.

2.3. Table Content Recognition

Table content recognition involves recognizing the text within a cell, while also performing tasks such as table classification, cell classification, and information extraction based on the table content. Zhong et al. [19] introduced a novel measure called TEDS for table recognition, which effectively balances errors in text content recognition and table structure recognition. Li et al. [23] presented Tablebank, a new dataset for image-based table detection and recognition, constructed using weak supervision from Word and LaTeX documents found online. This dataset aims to empower deep learning methods in table-detection and recognition tasks.

Lu et al. [24] proposed a self-attention-based mechanism for scene text recognition, which learns self-attention while encoding input–output attention. Li et al. [25] developed a graph-based convolutional network GFTE and an open-source financial benchmark dataset FinTab. Meanwhile, Yang et al. [26] devised a two-channel network for formula recognition in table cells, although it must be used with other structure recognition models in real-world scenarios.

Wu et al. [27] developed a text-to-table approach. Ly et al. [28] proposed an end-to-end multitask learning model for image-based table recognition, consisting of a shared encoder, a shared decoder, and three independent decoders. Smock et al. [29] solved the issue of misalignment in the benchmark dataset for table structure recognition. Wang et al. [30] introduced a method that is robust to S-TSR sub-scenarios with different challenges, while Huang et al. [31] utilized visual alignment sequential coordinate modeling to enhance table recognition and to produce better cell boundary boxes.

Kazdar et al. [32] conducted table detection and table structure recognition using deep learning and heuristic methods. Zhou et al. [33] were able to identify noisy images, while Nassar et al. and Lee et al. [34] proposed a new graph-based framework for table structure recognition designed to handle complex tables. Furthermore, Nassar et al. [35] presented a new table recognition model that can extract table contents directly from PDF sources, but it needs improvement for complex tables. Smock et al. [36] introduced the PubTable-1M dataset, which contains nearly a million articles from science, supports multiple input forms, and has detailed title and location information for table structures.

3. Methods

Our method utilizes the Cascade Faster R-CNN to detect tables and employs TPS transformation and radiation transformation techniques to correct images and to improve the accuracy of table detection. In addition, we utilize DBNet to detect text lines and CRNN and attention models to identify text within these lines. The table information is encoded using a transformer and is then decoded into two processing paths. One path deals with text position regression for cell text, while the other focuses on extracting features of the table structure. The final table structure recognition result is obtained by logically combining the outputs of both paths, which include three features: cell text position, cell text content, and table structure. The overall structure is shown in Figure 1.

In Figure 1, there are two main branches. One branch consists of the core TableMaster component, which uses the transformer to encode table information and then splits it into two processing lines for regression of cell text position and extraction of table structure features via a decoder. The other branch first goes through Cascade Faster R-CNN and splits into two processing lines: one for regressing the cell text position and fusing it with the corresponding output line from TableMaster to obtain the final cell text position feature and the other for using the CRNN module to obtain the cell text content feature. Finally, the three outputs from both branches, namely the cell text position feature, cell text content feature, and table structure feature, are logically added together and are subjected to post-processing to obtain the final table structure recognition result.

3.1. Regional Detection Based on Cascade Faster R-CNN

Table recognition involves three steps, namely table detection, text recognition, and table structure recognition. In this article, we use an improved Cascade Faster R-CNN to carry out the table-detection task, as shown in Figure 2.

In Figure 2, the cascaded detection method adds $H4$ and $H5$ detectors with higher thresholds to the traditional method. The detector thresholds for $H1$, $H2$, $H3$, $H4$, and $H5$ are 0.5, 0.6, 0.7, 0.8, and 0.9, respectively. Different inputs are used at different stages of the Cascade Faster R-CNN to further improve detection accuracy by utilizing output information from previous stages. In addition, introducing detectors with higher thresholds can filter out more accurate detection results and produce final results with greater accuracy.

To preserve more features, a multi-scale fusion network is incorporated into the feature pyramid network (FPN). Following feature extraction, features are fused with the $P2$, $P3$, $P4$, and $P5$ stages.The FPN and region proposal network (RPN) are fitted with $H4$ and $H5$ detectors, with detector thresholds (intersection over union, IOU) of $H1$, $H2$, $H3$, $H4$ and $H5$ set at 0.5, 0.6, 0.7, 0.8, and 0.9, respectively. The structure of the Cascade Faster R-CNN relies on a sequence of specialized regression factors.

$$f(x,b)=f_T\circ f_{(T-1)}\circ \cdots \circ f_1(x,b)$$

(1)

where T is the total amount of the number of cascades, and ∘ denotes the cascading method. In the fourth class, each regression factor $f_T$ is optimized using the sample distribution of the corresponding stage $b^t$, rather than the initial distribution $b^t$. The optimization process begins with a set of examples $(x_i,b_i)$, and at each level couplet, it returns to the sample distribution of the higher IOUs example $(x_i^t,x_i^t)$. This approach ensures that the number of positive models at each stage remains constant, even if the IOU threshold is increased.

In each T stage, Faster R-CNN incorporates a classifier $h_t$ and a regression module $f_t$ optimized for the IOU threshold, where the threshold size $u^t>u^{(t-1)}$. The loss function is defined as:

$$L(x^t,g)=L_{cls}(h_t\left(x^t\right),y^t)+\lambda [y^t\ge 1]\times L_{loc}(f_t(x^t,b^t),g)$$

(2)

where

$$b^t=f_{\left(t-1\right)}(x^{(t-1)},b^{(t-1)})$$

(3)

and

$$y^t=\left\{\begin{array}{cc}{g}_y,& IOU(x^t,g)>u\\ 0,& otherwise\end{array}\right.$$

(4)

For $x^t$, g is the benchmark object, and $\lambda =1$ is the coefficient to balance two types of losses. $y^t$ is the label of $x^t$, and $[y^t\ge 1]$ is the Averson bracket. If $y^t\ge 1$, then a certain type of target that needs to be detected. Otherwise, it means this item is the background.

If the text in a table is not arranged horizontally after the form test, it is usually necessary to correct the text orientation. This paper employs the TPS transformation with four-corners calibration, as illustrated in Figure 3. Removing the warping and perspective transformation of a table obtains a regular table.

3.2. DBNet Text Detection Based on Do-Conv

DBNet (Figure 4) is a text-detection method based on image segmentation. It utilizes a ResNet-based structure for feature extraction, which helps to reduce time costs and to improve prediction speed. Furthermore, the Do-Conv method is employed to accelerate the training process and to achieve faster results.

The backbone of DBNet utilizes ResNeSt50 to extract multi-scale features from the input image. After applying FPN, the size of the output feature map is reduced to a quarter of the original image. The head module is then responsible for computing the probability map, threshold map, and binary map for the text region.

The standard binarization function is problematic, as it is discontinuous, non-differentiable, and cannot be optimized for semantic segmentation training. To address this issue, a differentiable binarization approach was proposed to approximate the step function in standard binarization. The differentiable binarization technique is defined as follows:

$$B_{(i,j)}=\frac{1}{1+e^{(-k(P_{(i,j)}-T_{(i,j)})}}$$

(5)

where $\widehat{B}$ is the two-value diagram of the output, P is the probability chart obtained, T is the threshold diagram obtained, and K is the gain factor.

During the training process, the probability map, threshold map, and binary map are outputted. To calculate the loss function, these three maps and their corresponding annotations are combined to create three partial loss functions. The total loss function is defined as follows:

$$L=L_s+\alpha \times L_b+\beta \times L_t$$

(6)

where L is the total loss, $L_s$ is the probability diagram loss, $L_b$ is the dual-value diagram loss, $L_t$ is the threshold diagram loss, and $\alpha$ and $\beta$ are the weight coefficients. Both of the loss calculations of $L_s$ and $L_b$ use Bceloss. The wrong question set strategy is used during the loss calculation process to balance the positive samples. The loss function of the probability diagram and the two-value diagram is defined as:

$$L_s=L_b={\sum }_{\left(iϵS_t\right)}^{}{y}_{i}log{x}_i+(1-y_i)log(1-x_i)$$

(7)

where $S_t$ denotes the proportion of positive samples with a ratio of 1:3.

$L_t$ is the sum of the prediction results in the extended polygon $G_d$ (label) and the $L_1$ distance between the predicted threshold map and its corresponding ground truth label. The loss function for the threshold map is defined as follows:

$$L_t={\Sigma }_{\left(iϵR_d\right)}{y}_i^{*}-x_i^{*}$$

(8)

where $R_d$ is an index in the extended polygon $G_d$, and $y_i^{*}$ is a threshold icon tag.

3.3. Attention-Based CRNN Text Recognition

In this paper, CRNN is used as the text recognition network in combination with an attention structure. The attention mechanism calculates weights for the query and is key to achieving better performance. The entire process is illustrated in Figure 5.

In Figure 5, the attention mechanism describes a method where weights are calculated for each value based on its similarity or correlation with the given query and keys. These weights are then used to compute a weighted sum of values, resulting in an output. Thus, the entire process maps the input query and a series of key-value pairs to an output through the use of weight coefficients.

The CTC used in CRNN is decoded based on frame sequences, which can result in the loss of overall information of the formula. To improve recognition performance in scenarios involving special characters and formulas within tables, this paper proposes an observation- and parser-based approach. The observator is a full convolutional encoder that maps the input image to advanced features. The parser is a circular neural decoder that converts these advanced features to the output sequence. For each prediction, the parser’s built-in attention mechanism scans the entire input image and selects the most relevant region to describe the segmentation symbol or hidden space operator. This approach helps to retain the contextual information of each predicted word and to preserve the general knowledge of the formula, as illustrated in Figure 6.

As demonstrated in Figure 6. This attention mechanism is employed on the test image where it concentrates more on the “P” on the left and the area between “P” and “3” on the right. With the aid of contextual information, the model generates the outputs “P” and “_”.

3.4. Sequence Structure Recognition Based on Transformer

After completing text detection and text recognition, the next step is to perform table structure recognition. The TableMaster model for sequence recognition consists of two parts: encoding and decoding, as illustrated in Figure 7.

Each residual connector contains multiple convolutional layers and a normalized layer, which can effectively reduce gradient disappearance and overfitting. After each residual connection block, there is the multi-pass channel attention module Multi-Aspect GCAttention (MAGCA) that is used to enhance the characteristics without increasing the parameters. MAGCA is defined as:

$$\left\{\begin{array}{c}y=x+\delta \left(MAGC\right(x\left)\right),\\ MAGC\left(x\right)=Concat(g{c}_1,g{c}_2,\cdots ,g{c}_h),\\ g{c}_i={\sum }_{j=1}^L{\alpha }_j{\chi }_j,\\ \alpha =softmax(\frac{\left(w_k{x}_1\right)}{\sqrt{d_h}},\frac{\left(w_k{x}_2\right)}{\sqrt{d_h}},\cdots ,\frac{\left(w_k{x}_L\right)}{\sqrt{d_h}},)\end{array}\right.$$

(9)

where h is the number of bulls.

The TableMaster is utilized for encoding and decoding as shown in Figure 8. The encoding phase converts a picture into a sequence to extract the feature representation of the input sequence. The decoding part adds a branch based on the transformer for cell position prediction and table structure sequence prediction.

The decoder in the TableMaster model consists of N = 5 identical layers that are stacked on top of each other, with each layer consisting of three sublayers. The first sublayer performs multiple attention operations on the output of the encoder stack. To prevent positions from focusing on subsequent positions, the self-attention sublayer is modified with masking and position offsets to ensure that the position prediction only relies on the known output. The second sublayer is the multi-head self-attention layer, while the third sublayer is a simple fully connected feedforward network. These sublayers are depicted in Figure 9.

4. Experiment

4.1. Dataset

In our experiment, the ICDAR 2015, Marmot [37], SCITSR [38] and PubTabnet [19] datasets were used. ICDAR 2015 (IC15) is a commonly used dataset for text detection. It contains a total of 1500 pictures, in which 1000 are used for training and the rest for testing. The text region is annotated by four top-sized points of the quadrilateral shape. Marmot DataSet1 contains 2000 pages in PDF format, most of which come from research papers. It consists of Chinese and English pages, storing the tree structure of all document layouts. SCITSR is a comprehensive dataset. It consists of 15,000 images in PDF format, the image region, corresponding structural labels, and the boundary frames of each cell. It is divided into 120,000 training and 3000 tests. At the same time, a complex table is also provided, called scitsr-comp. The PUBTABNET dataset contains 5.68 million images (PNG) including the corresponding HTML of the table representing annotation. The surface structure and characters are provided, but the boundary frame is lacking.

4.2. Experimental Settings

4.2.1. Experimental Environment

In Cascade Faster RCNN training, 8 Tesla V100 GPUs are used with a batch size of 4 per GPU. The initial learning rate is 0.001, and the epoch is 400. In the table structure training, 8 Tesla V100 GPUs are used, and the batch size of each GPU is 6. The input image size is 480 × 480, and the maximum sequence length is 500. The initial learning rate is 0.001. In the training of text line recognition, 8 Tesla V100 GPUs are used with a batch size of 64 per GPU. The input size is 256 × 48, and the maximum length is 100. Simultaneous BN and ranger optimizers are used simultaneously, and the hyperparameter settings are the same as the table structure training.

The experiment uses Python version 3.7, Pytorch framework version 1.8.1, CUDA version 11.1, and cuDNN version 8.0.5, which can run the code in this article. This paper’s model training and inference were performed on NVIDIA GTX 3090 and Intel i9-9900k@5GHz.

4.2.2. Evaluation Index

In our experiments, we utilized accuracy (precision), recall rate (recall), F1 score, and TEDS (tree edit distance similarity) to evaluate the effectiveness of our proposed approach. TEDS measures the similarity between predicted and ground-truth HTML tag tree structures. The formula is as follows:

$$Precision=\frac{TP}{TP+FP}$$

(10)

$$Recall=\frac{TP}{TP+FN}$$

(11)

$$F1=2\ast \frac{Precision\ast Recall}{Precision+Recall}$$

(12)

$$TEDS(T_a,T_b)=1-\frac{EditDist(T_a,T_b)}{max(\left|T_a\right|,\left|T_b\right|)}$$

(13)

where $T_a$ and $T_b$ represent tables in tree-structure HTML format. EditDist denotes the tree-edit distance, and $\left|T\right|$ represents the number of nodes in T.

4.3. Comparative Experiment

This paper evaluated our proposed models using the ICDAR 2015, MARMOT, SCITSR, and PUBTABNET table recognition datasets. To compare with other existing table recognition models such as Splerge and TableNet, we present the experimental results of different models on these four datasets in

Table 1. Performance comparison of UTTSR, SPLETGE and TableNet on multiple datasets.

Dataset	Methods	Precision (%)	Recall (%)	F1 (%)	TEDS (%)
ICDAR2015	SPLETGE	99.2	98.8	98.99	-
	UTTSR (ours)	99.5	99.4	99.45	-
	TableNet	97.4	96.2	96.80	-
Marmote	SPLETGE	98.8	99.0	98.90	-
	UTTSR (ours)	99.2	98.7	98.94	-
	TableNet	93.1	90.1	91.58	-
PubTabnet	SPLETGE	-	-	-	97.1
	UTTSR (ours)	-	-	-	98.5
	TableNet	-	-	-	96.7
SCITSR	SPLETGE	85.4	82.3	83.82	-
	UTTSR (ours)	95.2	94.8	95.00	-
	TableNet	92.2	89.8	90.98	-

.

Table 2. Three network model comparisons of the table-detection method.

Method	Backbone	Precision (%)
Faster RCNN	VGG16	62.70
Cascade Faster RCNN	ResNet101	79.20
	ResNeXt105	80.10
Cascade Faster RCNN+ FPN	ResNet101	80.50
	ResNeXt105	81.40

shows the evaluation results of Faster RCNN, Cascade Faster R-CNN, and Cascade Faster RCNN+FPN on the SCITSR and Marmot datasets for table detection. The accuracy rate of Faster R-CNN is only 62.7%. The other two models use ResNeXt101 and ResNeXt105 as backbones, with ResNeXt105 being more accurate due to having more parameters than ResNeXt101.

The detection accuracy of the improved Cascade Faster RCNN+FPN model is 81.40%, which is a significant improvement compared to Faster R-CNN and Cascade Faster RCNN. This demonstrates that the improved FPN+Cascade-RCNN model is highly effective and robust, with better overall performance when compared to other networks.

For table structure recognition, we conducted experiments on datasets using the traditional algorithm + PDF, the layout parser method, and TableMaster sequence recognition, as shown in

Table 3. Table structure detection method comparison.

Method	Precision (%)	Recall (%)	F1 (%)
Image Algorithm+PDF	90.00	30.36	72.0
Layout Parser	95.25	84.16	89.0
TableMaster	96.95	93.07	95.0

. The F1 score of TableMaster was found to be higher than that of the layout parser algorithm. Additionally, its accuracy was slightly better compared to the layout parser and the combined image with PDF information method, as presented in Table 3.

For text line detection, we conducted experiments using DBNet, DBNet++, and Do-cov-based DBNet on the ICDAR2015 dataset. The performance of these models for text line recognition is presented in

Table 4. The test results of the text recognition method on the ICDAR2015 dataset.

Method	Precision (%)	Recall (%)	F1 (%)
DBNet	88.60	76.50	82.10
DBNet++	87.80	77.50	82.60
DBNet (Do-cov)	90.60	76.00	82.70

.

The DO-Conv-based DBNet method outperformed DBNet and DBNet++ in terms of precision and F1 scores. Specifically, on the ICDAR2015 dataset, the DBNet based on DO-Conv had a precision of 2 percentage points higher than the original DBNet and 2.8 percentage points higher than the DBNet++. Regarding recall, the DO-Conv-based DBNet was 0.5 percentage points lower than the original DBNet and 1.5 percentage points lower than the DBNet++. In terms of F1 score, our proposed DO-Conv-based DBNet achieved a performance that was 0.5 percentage points higher than the original DBNet and 0.3 percentage points higher than the DBNet++. Overall, the DO-Conv-based DBNet method demonstrated better performance compared to the other DBNet methods.

4.4. Ablation Experiment

To further validate the effectiveness of each model in UTTSR, this paper conducted ablation experiments on the PubTabnet dataset, as shown in

Table 5. The ablation experiments. TD: table detection; TSR: table structure recognition; TLD: text line detection; TLR: text line recognition.

TD	TSR	TLD	TLR	F1 (%)
Cascade Faster R-CNN	Transformer	DBNet (Do-cov)	CRNN
✓	×	×	×	82.45
✓	✓	×	×	86.42
✓	✓	✓	×	92.31
✓	✓	✓	✓	97.85

. It is evident that the performance of our proposed models gradually improved with the increase in and fusion of various methods, achieving a high accuracy of 0.9785 when processing tables. This indicates that the UTTSR model can achieve better overall performance for table recognition tasks.

4.5. Implementation Details

There are various types of tables, with the most common being the three-line table and the half-line table. As the complexity of table structures continues to increase, processing and analyzing table information also become more challenging. The dataset sample is shown in Figure 10.

In Figure 11, we show a result example of sequence prediction and box regression. We can see that the structure can predict out-of-the-box coordinates correctly.

The position annotations in the dataset are at the cell level, and cropped text images according to the position annotation in the dataset contain both single-line and multi-line text images. This is shown in Figure 12.

5. Conclusions

This paper focuses on table recognition with complex structures and presents the UTTSR network. The proposed network is designed to identify complex table structures and cell contents from unstructured text by dividing the table model into four parts: table region detection, text line detection, text line recognition, and table sequence recognition. Compared with other methods, our approach is superior to other methods to a great extent and can solve the problem of table structure recognition in most scenarios.Various table recognition methods have been developed prior to this, including those based on image segmentation and end-to-end deep learning, which have achieved varying degrees of success. However, these methods still encounter certain limitations such as their inability to detect complex table structures and handle the problem of spanning cell recovery. To overcome these challenges, the UTTSR network draws inspiration from the transformer model and integrates multiple modalities, including visual features of images, positional relationship features of text, and semantic features of text. By doing so, it significantly enhances the accuracy and robustness of complex table structure recognition.

In the future, as the volume of data in various industries continues to increase, the demand for data-driven analysis of tabular information will also grow. Therefore, we need to further explore optimization methods for data analysis to solve the identification problems of various table structures and data types that arise in different industries and fields. At the same time, we also need to combine multiple technical means such as document technology, information extraction, and complex layout recognition methods to further enhance the processing and analysis capabilities of tabular information, providing a guarantee to better achieve intelligence and digitization.