Learning-Based Image Damage Area Detection for Old Photo Recovery

Most methods for repairing damaged old photos are manual or semi-automatic. With these methods, the damaged region must first be manually marked so that it can be repaired later either by hand or by an algorithm. However, damage marking is a time-consuming and labor-intensive process. Although there are a few fully automatic repair methods, they are in the style of end-to-end repairing, which means they provide no control over damaged area detection, potentially destroying or being unable to completely preserve valuable historical photos to the full degree. Therefore, this paper proposes a deep learning-based architecture for automatically detecting damaged areas of old photos. We designed a damage detection model to automatically and correctly mark damaged areas in photos, and this damage can be subsequently repaired using any existing inpainting methods. Our experimental results show that our proposed damage detection model can detect complex damaged areas in old photos automatically and effectively. The damage marking time is substantially reduced to less than 0.01 s per photo to speed up old photo recovery processing.


Introduction
Old photos can often contain various levels of damage caused by human improper storage or environmental factors that deteriorate the integrity of photos. Fortunately, digital image processing technology can be applied to recover the content of these photos to its original state. The existing recovery methods for damaged old photos can be divided into non-automatic and automatic processes according to whether human intervention is required. The non-automatic methods can be further subdivided into manual and semiautomatic methods. Manual recovery is made through a variety of image editing tools, such as Photoshop or GIMP [1], to recover damaged photos based on user knowledge. The semi-automatic method manually marks the damaged areas on the photos and then applies the inpainting methods [2,3] to recover the contents of these locations. The mentioned works focused on the design of repair methods. For example, Li et al. [2] modified the confidence computation, strategy matching, and filling scheme to improve the inpainting method. Zhao et al. [3] proposed an inpainting model based on the generative adversarial network (GAN) and gated convolution [4]. With their methods, in addition to the damaged photos as the input, additional damage masks should be specified before inputting into the model. Non-automatic methods, while providing good recovery results, require physically marking the damaged areas in the photos, taking a lot of time and effort.
The automatic method does not have the aforementioned problems as it does not require any additional information in the process of restoring damaged old photos. Works [5,6] have used deep learning techniques to develop automatic methods that can be applied to a wider variety of photo content and types of damage. Wan et al. [5] designed a model based on the architecture of variational autoencoder (VAE) [7]. They used an encoder model to for semantic segmentation-like applications, but this function is weak in handling the imbalanced dataset issue. As a consequence, Lau et al. [15] replaced BCE functions with dice coefficients to evaluate the correctness of the detected areas. Liu et al. [16] further combined the BCE functions and the dice coefficients. Cheng et al. [19] applied distance weight to improve the original binary cross entropy. Existing deep learning-based road crack detection algorithms can work on more complex and diverse damage than can traditional algorithms. However, since there are many differences between the content of road images and old photos, it is not possible to use the road crack detection method directly, so we need to develop a method suitable for detecting damage in old photos.
To summarize the main contributions of our work, unlike other literature approaches where the content of some intact areas is changed during repair, our way of recovering damaged old photos ensures no alteration of intact areas during repair to preserve photo integrity and fidelity. Since the existing methods for detecting image damage are not satisfactory, in this paper an automatic damage detection method is proposed for the recovery of old damaged photos to save time and effort. The advantage of our work is that our detection result enables the possibility of combining any subsequent inpainting methods to repair the photo, which is not possible using existing automatic end-to-end repairing methods.

Proposed Method
Our recovery processing of damaged old photos is divided into two parts, as shown in Figure 1. In the detection model (M D ), the model input is an old damaged photo (I damaged ) and the model output is a damaged area mask (Mask). The I damaged and the Mask are then exported to the inpainting method (M R ) to generate the repaired photo (I Repaired ), where the M R can be any existing method.
Sensors 2022, 22, 8580 3 of 13 cracked blocks only as the training set for their crack-patch-only (CPO) supervised adversarial learning. Jenkins et al. [18] set the specific ratio between cracked and non-cracked blocks in the training set to place more weight on cracked blocks. As for the loss function, most works use binary cross-entropy (BCE) as a loss function for semantic segmentationlike applications, but this function is weak in handling the imbalanced dataset issue. As a consequence, Lau et al. [15] replaced BCE functions with dice coefficients to evaluate the correctness of the detected areas. Liu et al. [16] further combined the BCE functions and the dice coefficients. Cheng et al. [19] applied distance weight to improve the original binary cross entropy. Existing deep learning-based road crack detection algorithms can work on more complex and diverse damage than can traditional algorithms. However, since there are many differences between the content of road images and old photos, it is not possible to use the road crack detection method directly, so we need to develop a method suitable for detecting damage in old photos.
To summarize the main contributions of our work, unlike other literature approaches where the content of some intact areas is changed during repair, our way of recovering damaged old photos ensures no alteration of intact areas during repair to preserve photo integrity and fidelity. Since the existing methods for detecting image damage are not satisfactory, in this paper an automatic damage detection method is proposed for the recovery of old damaged photos to save time and effort. The advantage of our work is that our detection result enables the possibility of combining any subsequent inpainting methods to repair the photo, which is not possible using existing automatic end-to-end repairing methods.

Proposed Method
Our recovery processing of damaged old photos is divided into two parts, as shown in Figure 1. In the detection model ( ), the model input is an old damaged photo ( ) and the model output is a damaged area mask ( ). The and the are then exported to the inpainting method ( ) to generate the repaired photo ( ), where the can be any existing method. Figure 1. Flow chart of our architecture to automatically repair damaged old photos. By feeding an old damaged photo into our damage detection network, we can generate a damaged area mask. To restore the photo, the damaged photo and the mask are fed together into an arbitrary inpainting algorithm. Figure 2 shows the architecture of our damaged detection model is derived from U-Net [21]. The first half is an encoder that extracts the image features, while the second half restores the image to its original size by up-sampling and uses the sigmoid function to Figure 1. Flow chart of our architecture to automatically repair damaged old photos. By feeding an old damaged photo into our damage detection network, we can generate a damaged area mask. To restore the photo, the damaged photo and the mask are fed together into an arbitrary inpainting algorithm. Figure 2 shows the architecture of our damaged detection model is derived from U-Net [21]. The first half is an encoder that extracts the image features, while the second half restores the image to its original size by up-sampling and uses the sigmoid function to find out the map of pixel damage probabilities. The advantage of using U-Net is its ability to capture features at different scales, which are important for old photo damage detection and allow the model to more accurately identify damage in different shapes and sizes. Another merit of U-Net is the ability to concatenate features of the encoder into the decoder, allowing the model to train without losing the features obtained in the shallow network.
find out the map of pixel damage probabilities. The advantage of using U-Net is its ability to capture features at different scales, which are important for old photo damage detection and allow the model to more accurately identify damage in different shapes and sizes. Another merit of U-Net is the ability to concatenate features of the encoder into the decoder, allowing the model to train without losing the features obtained in the shallow network. In order to improve the ability to extract features, we replaced the original convolutional layers of U-Net with residual dense blocks (RDBs) [22]. This block is a combination of a residual block [23] and a dense block [24]. The residual block uses a skip connection to combine the input of the block with the output of the block, thus increasing the stability of the model training and the speed of convergence. The dense block continuously passes all the shallow features of the block to the deeper layers, thus making full use of the information from the shallow features. The RDB retains these advantages to improve the performance of the whole model. The original convolution layers at each scale used in U-Net would gradually lose its shallow feature information, but this problem was solved when we adopted RDB. In this way, it is possible to use more information from the area surrounding the damage for damage detection.
Since there is no open dataset of damaged old photos available for use, we collected photos from the Internet and marked the damaged areas in the images by ourselves. These photos consisted mainly of portraits, buildings, and natural scenery, with their sizes ranging from 129 × 317 to 797 × 1131 pixels. To generate ground truths, we manually marked the damaged areas of the collected photos using the image editing tool GIMP [1]. The transparency function of the GIMP layer feature makes marking damaged areas in photos easier and more precise. Figure 3 shows examples of photos from our collected dataset as well as the corresponding marked ground truth. We collected a total of 170 old damaged photos and manually labeled them, 123 of which were for the training set, 18 for the test set, and the remaining 29 for the validation set. On account of the limited number of photos in the data collection, the data augmentation technique was used to increase the dataset size via horizontal flipping and the 90-, 180-, and 270-degree rotation of photos. In order to improve the ability to extract features, we replaced the original convolutional layers of U-Net with residual dense blocks (RDBs) [22]. This block is a combination of a residual block [23] and a dense block [24]. The residual block uses a skip connection to combine the input of the block with the output of the block, thus increasing the stability of the model training and the speed of convergence. The dense block continuously passes all the shallow features of the block to the deeper layers, thus making full use of the information from the shallow features. The RDB retains these advantages to improve the performance of the whole model. The original convolution layers at each scale used in U-Net would gradually lose its shallow feature information, but this problem was solved when we adopted RDB. In this way, it is possible to use more information from the area surrounding the damage for damage detection.
Since there is no open dataset of damaged old photos available for use, we collected photos from the Internet and marked the damaged areas in the images by ourselves. These photos consisted mainly of portraits, buildings, and natural scenery, with their sizes ranging from 129 × 317 to 797 × 1131 pixels. To generate ground truths, we manually marked the damaged areas of the collected photos using the image editing tool GIMP [1]. The transparency function of the GIMP layer feature makes marking damaged areas in photos easier and more precise. Figure 3 shows examples of photos from our collected dataset as well as the corresponding marked ground truth. We collected a total of 170 old damaged photos and manually labeled them, 123 of which were for the training set, 18 for the test set, and the remaining 29 for the validation set. On account of the limited number of photos in the data collection, the data augmentation technique was used to increase the dataset size via horizontal flipping and the 90-, 180-, and 270-degree rotation of photos.
find out the map of pixel damage probabilities. The advantage of using U-Net is its ability to capture features at different scales, which are important for old photo damage detection and allow the model to more accurately identify damage in different shapes and sizes. Another merit of U-Net is the ability to concatenate features of the encoder into the decoder, allowing the model to train without losing the features obtained in the shallow network. In order to improve the ability to extract features, we replaced the original convolutional layers of U-Net with residual dense blocks (RDBs) [22]. This block is a combination of a residual block [23] and a dense block [24]. The residual block uses a skip connection to combine the input of the block with the output of the block, thus increasing the stability of the model training and the speed of convergence. The dense block continuously passes all the shallow features of the block to the deeper layers, thus making full use of the information from the shallow features. The RDB retains these advantages to improve the performance of the whole model. The original convolution layers at each scale used in U-Net would gradually lose its shallow feature information, but this problem was solved when we adopted RDB. In this way, it is possible to use more information from the area surrounding the damage for damage detection.
Since there is no open dataset of damaged old photos available for use, we collected photos from the Internet and marked the damaged areas in the images by ourselves. These photos consisted mainly of portraits, buildings, and natural scenery, with their sizes ranging from 129 × 317 to 797 × 1131 pixels. To generate ground truths, we manually marked the damaged areas of the collected photos using the image editing tool GIMP [1]. The transparency function of the GIMP layer feature makes marking damaged areas in photos easier and more precise. Figure 3 shows examples of photos from our collected dataset as well as the corresponding marked ground truth. We collected a total of 170 old damaged photos and manually labeled them, 123 of which were for the training set, 18 for the test set, and the remaining 29 for the validation set. On account of the limited number of photos in the data collection, the data augmentation technique was used to increase the dataset size via horizontal flipping and the 90-, 180-, and 270-degree rotation of photos. Because there are more undamaged old photos on the Internet, in order to further extend the training dataset we collected and used these undamaged photos, along with a collection of damage-like textures, to synthesize artificial damaged photos. Compared to Figure 4a we can see some differences between the artificially damaged photo and the old real damaged photo. The real damaged area of an old photo is composed of complex multitoned contents, not just simple good or bad, but our synthesized damaged photo only uses a single color to represent damage. We treated this difference as a type of damage to improve the generalizability of the model. Because there are more undamaged old photos on the Internet, in order to further extend the training dataset we collected and used these undamaged photos, along with a collection of damage-like textures, to synthesize artificial damaged photos. Compared to Figure 4a we can see some differences between the artificially damaged photo and the old real damaged photo. The real damaged area of an old photo is composed of complex multitoned contents, not just simple good or bad, but our synthesized damaged photo only uses a single color to represent damage. We treated this difference as a type of damage to improve the generalizability of the model.  The model parameters are initialized using the MSRA initialization method [25] in the experiments, and the optimizer is the Adam optimizer with β = 0.9, β = 0.999. The initial learning rate of the model is set to 0.0001, and every 1000 epochs are multiplied by 0.1 to train a total of 2000 epochs. The training patch size 48 × 48, which is commonly used in pavement crack detection, is less appropriate for our task. The main reason for this is that most cracked pavement images only have a black background and a few white cracks, whereas old cracked photos have more complex content, such as portraits, objects, buildings, and so on. Therefore, we partitioned the photos of the training set into patches of 100 × 100 pixels in size to account for more context to improve the performance, and in our experiment, larger patch sizes than this did not result in any additional performance gain. We also controlled the ratio of patches with damaged areas to patches without damage at 8:2 in training.
The loss function was balanced cross entropy. The main reason for employing balanced cross entropy was to compensate for the imbalance between intact and damaged areas. It modified the original binary cross entropy with the ratio of the two categories, giving more weight to the fewer damaged areas and less weight to the more numerous intact areas as shown in (3) where N is the total number of pixels in training blocks, is the weight of the intact areas, denotes whether the th pixel belongs to the intact category in the ground truth, and ( ) is the model's prediction of the probability that the th pixel belongs to the intact areas. The model parameters are initialized using the MSRA initialization method [25] in the experiments, and the optimizer is the Adam optimizer with β 1 =0.9, β 2 =0.999. The initial learning rate of the model is set to 0.0001, and every 1000 epochs are multiplied by 0.1 to train a total of 2000 epochs. The training patch size 48 × 48, which is commonly used in pavement crack detection, is less appropriate for our task. The main reason for this is that most cracked pavement images only have a black background and a few white cracks, whereas old cracked photos have more complex content, such as portraits, objects, buildings, and so on. Therefore, we partitioned the photos of the training set into patches of 100 × 100 pixels in size to account for more context to improve the performance, and in our experiment, larger patch sizes than this did not result in any additional performance gain. We also controlled the ratio of patches with damaged areas to patches without damage at 8:2 in training.
The loss function was balanced cross entropy. The main reason for employing balanced cross entropy was to compensate for the imbalance between intact and damaged areas. It modified the original binary cross entropy with the ratio of the two categories, giving more weight to the fewer damaged areas and less weight to the more numerous intact areas as shown in (3) where N is the total number of pixels in training blocks, α i is the weight of the intact areas, y i denotes whether the ith pixel belongs to the intact category in the ground truth, and p(i) is the model's prediction of the probability that the ith pixel belongs to the intact areas.

Experiment Result
In our experiment, model training and testing were carried out on a computer equipped with an Intel i5-2400 CPU and an NVIDIA 2070 8GB GPU. To assess the model performance of damage detection, we adopted the evaluation methods commonly used in image segmentation and pavement crack segmentation, including precision, recall, F1measure, and precision-recall curve (PR curve), as our evaluation metrics. Precision is the percentage of the results identified as damaged areas that are actually damaged. The percentage of true damaged areas detected is represented by recall. The F1 measure considers both precision and recall. Since the ground truth is created by manual marking and each person has different damage marking criteria, we adopted the regional precision and recall proposed in [26], which considers the detection result correct as long as it is within five pixels of the manual marking results, to compensate for the ground truth credibility problem caused by manual marking errors.

Comparison of Various Modules
In this section, we first evaluate the performance of our damage detection model on old photos by testing the performance of U-Net barebones combined with various modules. We compared the results of our proposed method with three methods, including the original U-Net architecture, the U-Net architecture with a residual block module, and the U-Net architecture with a dense block module. Figure 5 and Table 1 show the results in terms of the PR curve, precision, recall, and F1-measure, which show that our proposed approach outperformed all other module combinations. Figure 6 depicts the visual outcome of using various modules to detect damage. It can be seen that our proposed method is capable of detecting more subtle damage as well as the damage border. The more complete the detection, particularly along the damage border, the more it can assist us in repairing damage without affecting the repair result.
In our experiment, model training and testing were carried out on a computer equipped with an Intel i5-2400 CPU and an NVIDIA 2070 8GB GPU. To assess the model performance of damage detection, we adopted the evaluation methods commonly used in image segmentation and pavement crack segmentation, including precision, recall, F1measure, and precision-recall curve (PR curve), as our evaluation metrics. Precision is the percentage of the results identified as damaged areas that are actually damaged. The percentage of true damaged areas detected is represented by recall. The F1 measure considers both precision and recall. Since the ground truth is created by manual marking and each person has different damage marking criteria, we adopted the regional precision and recall proposed in [26], which considers the detection result correct as long as it is within five pixels of the manual marking results, to compensate for the ground truth credibility problem caused by manual marking errors.

Comparison of Various Modules
In this section, we first evaluate the performance of our damage detection model on old photos by testing the performance of U-Net barebones combined with various modules. We compared the results of our proposed method with three methods, including the original U-Net architecture, the U-Net architecture with a residual block module, and the U-Net architecture with a dense block module. Figure 5 and Table 1 show the results in terms of the PR curve, precision, recall, and F1-measure, which show that our proposed approach outperformed all other module combinations. Figure 6 depicts the visual outcome of using various modules to detect damage. It can be seen that our proposed method is capable of detecting more subtle damage as well as the damage border. The more complete the detection, particularly along the damage border, the more it can assist us in repairing damage without affecting the repair result.

Comparison of Different Detection Methods
Next, we compare our method with other methods in the literature. We disassembled the damage detection part from the whole end-to-end work [5] and compared it to our method. Since there are so few existing deep learning-based damage detection methods for old photos, we also compared the results of pavement crack detection models [16,18,19] that have been retrained using our dataset to work on old photo damage detection. The results of the PR curve are shown in Figure 7. The best recall, precision, and F1 measure values for each method are shown in Table 2. The comparison results show that our detection effect is the best.

Comparison of Different Detection Methods
Next, we compare our method with other methods in the literature. We disassembled the damage detection part from the whole end-to-end work [5] and compared it to our method. Since there are so few existing deep learning-based damage detection methods for old photos, we also compared the results of pavement crack detection models [16,18,19] that have been retrained using our dataset to work on old photo damage detection. The results of the PR curve are shown in Figure 7. The best recall, precision, and F1 measure values for each method are shown in Table 2. The comparison results show that our detection effect is the best.

Comparison of Different Detection Methods
Next, we compare our method with other methods in the literature. We disassembled the damage detection part from the whole end-to-end work [5] and compared it to our method. Since there are so few existing deep learning-based damage detection methods for old photos, we also compared the results of pavement crack detection models [16,18,19] that have been retrained using our dataset to work on old photo damage detection. The results of the PR curve are shown in Figure 7. The best recall, precision, and F1 measure values for each method are shown in Table 2. The comparison results show that our detection effect is the best.  [5,16,18,19], and our proposed method.  Figure 8 compares the visual results of the proposed method with those detected by other methods. It can be seen that our proposed method of detecting damage in the photo was more accurate, especially in the detection border denoted inside the yellow boxes. By contrast, the methods proposed by [16,18,19] failed to completely detect the damage in the image, and [5] often labeled undamaged areas as damage, such as around the tip of the nose in Figure 8.

Method
Recall Precision F1 Measure Wan et al. [5] 0.845 0.837 0.831 Liu et al. [16] 0.832 0.767 0.785 Jenkins et al. [18] 0.838 0.734 0.763 Cheng et al. [19] 0.839 0.763 0.784 Our proposed method 0.911 0.847 0.873 Figure 8 compares the visual results of the proposed method with those detected by other methods. It can be seen that our proposed method of detecting damage in the photo was more accurate, especially in the detection border denoted inside the yellow boxes. By contrast, the methods proposed by [16,18,19] failed to completely detect the damage in the image, and [5] often labeled undamaged areas as damage, such as around the tip of the nose in Figure 8.  [18]; (e) the result of Cheng et al. [19]; (f) the result of our proposed method.
As shown the Table 3, we also compared the number of parameters and computation speed with these methods [5,16,18,19] where the size of the test photos was 512 × 512. Jenkins et al. [18] and Cheng et al. [19] used the same model framework, but the model was trained using different strategies. Therefore, they have the same number of parameters and running time. Table 3 shows that both our detection models and those of [5] are  [18]; (e) the result of Cheng et al. [19]; (f) the result of our proposed method.
As shown the Table 3, we also compared the number of parameters and computation speed with these methods [5,16,18,19] where the size of the test photos was 512 × 512. Jenkins et al. [18] and Cheng et al. [19] used the same model framework, but the model was trained using different strategies. Therefore, they have the same number of parameters and running time. Table 3 shows that both our detection models and those of [5] are fast as both lower to the scale of 10 −3 s, but our model is much lighter as our number of parameters is only about one-sixteenth of all the other methods. Table 3. Parameter and run time.

Combination with Inpainting Methods
Next, we present our results regarding practical application. We used [4,27,28] as the inpainting method in the subsequent process to repair actually damaged photos. The repair results using actually damaged photos are shown in Figures 9c-e and 10c-e, which demonstrate the results of our damage detection followed by different inpainting methods [4,27,28]. We can see in Figure 9c that Yu [27] failed to repair the cheeks and mouth in our detected area. Repair to damaged areas by gated convolution [4] is generally blurred as shown in Figure 9d. Figure 10c,d shows that deformation of the collar edge occurred after restoration. In general, the results of partial convolution [28] as shown in Figures 9e and 10e are more satisfactory compared to other inpainting methods [4,27]. This demonstrates that our architecture can be combined with any inpainting method, but we suggest that partial convolution [28] will achieve better results. In Figures 9b and 10b, we also compare our method with the end-to-end method [5], which integrates damage detection and repairs in one stage. Although [5] looks to have been effective in repairing the damaged areas, there are some color distortion problems with unfaithful tonal changes and a loss of texture in the image, such as in the cheeks as shown in Figure 9b. We can see that there are unrestored damaged areas and missing window frame details marked in the red box in Figure 10b. Thus, combining our architecture with the inpainting method [4,27,28] in contrast to [5] provides better results without affecting content in the undamaged regions in the recovery results.

Combination with Inpainting Methods
Next, we present our results regarding practical application. We used [4,27,28] as the inpainting method in the subsequent process to repair actually damaged photos. The repair results using actually damaged photos are shown in Figures 9c-e and 10c-e, which demonstrate the results of our damage detection followed by different inpainting methods [4,27,28]. We can see in Figure 9c that Yu [27] failed to repair the cheeks and mouth in our detected area. Repair to damaged areas by gated convolution [4] is generally blurred as shown in Figure 9d. Figure 10c,d shows that deformation of the collar edge occurred after restoration. In general, the results of partial convolution [28] as shown in Figures 9e  and 10e are more satisfactory compared to other inpainting methods [4,27]. This demonstrates that our architecture can be combined with any inpainting method, but we suggest that partial convolution [28] will achieve better results. In Figures 9b and 10b, we also compare our method with the end-to-end method [5], which integrates damage detection and repairs in one stage. Although [5] looks to have been effective in repairing the damaged areas, there are some color distortion problems with unfaithful tonal changes and a loss of texture in the image, such as in the cheeks as shown in Figure 9b. We can see that there are unrestored damaged areas and missing window frame details marked in the red box in Figure 10b. Thus, combining our architecture with the inpainting method [4,27,28] in contrast to [5] provides better results without affecting content in the undamaged regions in the recovery results.  [27]; (d) the result of ours + gated convolution [4]; (e) the result of ours + partial convolution [28].  [27]; (d) the result of ours + gated convolution [4]; (e) the result of ours + partial convolution [28].
There will still be cases where our approach may fail. For example, if the model encounters a mixture of various complex damage, as shown in Figure 11, it becomes difficult to distinguish the damaged areas, resulting in partial detection and incomplete repair results. To deal with such a complex pattern of damage, future studies could investigate and apply the concept of directional clues in damage patterns [29][30][31] to aid in crack damage detection. There will still be cases where our approach may fail. For example, if the model encounters a mixture of various complex damage, as shown in Figure 11, it becomes difficult to distinguish the damaged areas, resulting in partial detection and incomplete repair results. To deal with such a complex pattern of damage, future studies could investigate and apply the concept of directional clues in damage patterns [29][30][31] to aid in crack damage detection.

Conclusions
Most restoration methods for damaged old photos require the manual marking of damaged areas for restoration, which is quite inefficient. Therefore, we proposed a damage detection model for old photos. Our method can detect damaged areas automatically

Conclusions
Most restoration methods for damaged old photos require the manual marking of damaged areas for restoration, which is quite inefficient. Therefore, we proposed a damage detection model for old photos. Our method can detect damaged areas automatically without manual marking, which significantly reduces repair time. The detection results can be optionally screened and flexibly combined with any powerful inpainting method to fully automatically recover the content of the photos. We analyzed various block modules to design the detection model and found that the residual dense block (RDB), which combines the advantages of residual block and dense block, can effectively improve model detection capability. When compared to other detection algorithms, our method can detect damaged areas more accurately. We demonstrated the restoration of damaged old photos by combining our detection results with three different inpainting methods. In our restoration results, both the damaged and undamaged areas of the photos did not suffer from color tone changes, color distortion, or texture loss. Our method can better preserve the integrity of photos than can the existing end-to-end method, which alters the undamaged areas of photos.