Skip to Content
You are currently on the new version of our website. Access the old version .
Applied SciencesApplied Sciences
  • Article
  • Open Access

2 October 2021

21 Pages

A Photo Identification Framework to Prevent Copyright Infringement with Manipulations

,
,
,
and
1
Department of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, Korea
2
DRMinside, Seoul 05179, Korea
3
Department of Radiology, College of Medicine, Yonsei University, Seoul 03722, Korea
*
Author to whom correspondence should be addressed.

Abstract

In recent years, copyright infringement has been one of the most serious problems that hamper the development of the culture and arts industry. Due to the limitations of existing image search services, these infringements have not been properly identified and the number of infringements has been increasing continuously. To uncover these infringements and handle big data extracted from copyright photos, we propose a photo copyright identification framework to accurately handle manipulations of stolen photos. From a collage of cropped photos, regions of interest (RoIs) are detected to reduce the influence of cropping and identify each photo by Image RoI Detection. Binary descriptors for quick database search are generated from the RoIs by Image Hashing robustly to geometric and color manipulations. The matching results of Image Hashing are verified by measuring their similarity using the proposed Image Verification to reduce false positives. Experimental results demonstrate that the proposed framework outperforms other image retrieval methods in identification accuracy and significantly reduces the false positive rate by 2.8%. This framework is expected to identify copyright infringements in practical situations and have a positive effect on the copyright market.

1. Introduction

Copyright is one of the legal protections used to encourage creative activities as the creator’s exclusive right to use or allow others to use their content. Despite the legal protections, numerous copyright violations still occur worldwide by pirates. According to Copytrack [1], approximately 65M images in 115 countries suffer from copyright infringement and the damage from the stolen image is estimated at approximately USD 63M. If an illegal use is detected, corrective recommendations can be taken to compensate copyright holders by public institutions such as the Copyright Commission. In particular, as shown in Figure 1, the number of corrective recommendations for copyright infringement in South Korea is increasing. In 2019, more than 670,000 corrective recommendations were made and the total amount of compensation is approximately USD 28M [2].
Figure 1. The number of corrective recommendations for copyright infringements reported by the Korea Copyright Commission from 2014 to 2019 in Korea [2].
More than 90% of the violations are images and videos, and it takes a lot of cost to manually identify and uncover hundreds of thousands of infringements. Considering the large number of photos and videos that have not yet been detected on the web, it is very inefficient to capture all of the infringements with human labor. An automated copyright photo identification method is required to efficiently capture these copyright infringements [3,4,5,6] by finding a perfectly identical original photo the same as an query image. When a subject is photographed at different angle at the same time, the photos are not identical but very similar, and it is a very challenging task to distinguish these photos considering the trade-off between speed and accuracy.
Web image search services [7,8,9] correspond to automated solutions that can perform the aforementioned functions. They provide results of not only visually similar images but also identical images for image queries. However, existing web image search services such as Google, TinEye, Yandex, and Bing do not accurately find the original photo from manipulations such as image cropping or image collage, which frequently occurs when photos are stolen. For example, as shown in Figure 2, if a shopping website steals photos to promote sunglasses, it only crops the person wearing sunglasses from the background (image cropping) and combines several images of sunglasses (image collage). In addition, image manipulations such as resizing, color distortion, flipping, and compression can occur during image editing, and these can also lead to degradation in identification performance. Image cropping and image collages have problems, loss of information from original photos and misidentification of multiple photo copyrights, respectively. These problems are difficult to solve with existing web image search services.
Figure 2. Example of a photo collage with two cropped photos. In these photos, the sunglasses are the target object for unauthorized use. The source of these photos is https://unsplash.com (accessed on 27 September 2021), which provides copyright-free images.
As shown in Figure 3, since existing image search services do not consider the characteristics of copyright infringements, existing web image search services show the poor responses to the aforementioned manipulations. Figure 3a,b show that Google image search and TinEye matched only one of the match results correctly, and Yandex matched only one photo as shown in Figure 3c. The limitations of existing image search services emphasize the need for a new photo copyright identification and database. In addition to technical limitations, the absence of a service capable of providing copyright information also makes it difficult to handle infringements. The service in Figure 4 can facilitate the legal use of copyrighted photos by connecting users and copyright holders, and helps to find copyright infringements based on the photo copyright identification framework.
Figure 3. Image search results of Figure 2a by Google [7], TinEye [9], and Yandex [8]. We additionally tried with Bing, but did not obtain any results.
Figure 4. Service block diagram for the photo copyright search. The yellow box on the right side is the proposed photo copyright identification framework in this paper.
In this paper, we propose a photo copyright identification framework that can handle the manipulations that appear in Figure 2a. The proposed framework in Figure 4 is composed of three modules, Image RoI Detection, Image Hashing, and Image Verification, where the original photo and the manipulated photo are matched accurately. Image RoI Detection divides a collage of cropped images into RoIs, which represent the characteristics of each image well. This module reduces the influence of geometric manipulations to detect RoIs consistently from original photos and manipulated photos. The image collage dataset is constructed to train optimized detectors for Image RoI Detection. The most efficient detector is applied through benchmarking the state-of-art object detectors for this dataset.
Image Hashing generates a binary descriptor from each detected RoI to search the photo copyright DB. By training the image hashing network through image augmentation with diverse manipulations, the generated binary descriptor is robust to manipulations such as color transformation, blurring, and cropping that ImageRoI Detection did not handle. However, severe manipulations make it very difficult to identify photos correctly with their binary descriptors, leading to occurrences of false positives. Since it is not easy to check whether identification results are correct in the field test, lots of false positives can be identified as incorrect copyrights.
To tackle this problem, we propose Image Verification to confirm whether the identification results of Image Hashing are correct. This module geometrically aligns a query image and its matched photo, and obtains the overlapping region between them for pixel-to-pixel comparison. The image similarity Siamese network calculates the similarity of the overlapping region and verifies whether the images are identical.
This stable framework composed of the series of three modules improves the identification accuracy and minimizes the false positive rate for manipulated photos. Through exhaustive evaluations in experimental results, we verify the benefits of each module in the proposed framework for photo copyright identification. Furthermore, it is demonstrated that the proposed framework significantly improves identification accuracy and decreases false positive rate, outperforming conventional and state-of-the-art retrieval methods [10,11].

Contributions

The contributions of this paper is summarized as follows.
  • We propose Image RoI Detection, a preprocessing technique that can effectively handle geometric manipulations such as image collage and image cropping, which cannot be solved by existing image search services. To train the image detector used in Image RoI Detection, image collage datasets with 300,000 image collages are introduced.
  • We propose Image Hashing that can generate similar binary descriptors from manipulated photos and original photos. This module is designed to handle color manipulations such as color distortion and blur as well as geometric manipulations.
  • To accomplish Image Verification, we introduce an geometric alignment method to accurately align an query image with the matched one in the database obtained by Image Hashing method and propose a similarity measurement network to distinguish whether the aligned image infringes copyright robustly to manipulations.
  • By applying the aforementioned three steps, we improve the identification accuracy and significantly reduce the false positive rate for manipulated photos. The experimental results demonstrate advantages of each module in the proposed framework and show notable performance improvements of the proposed framework over the state-of-the-art image retrieval method [10].

4. Implementation Details

4.1. Photo Copyright Dataset

To develop the photo copyright identification method, we used 35,824 copyrighted photos provided by copyright holders for purpose of research only. To leverage diverse images, images from ImageNet and MS-COCO were also randomly sampled to compose the dataset. In total, the dataset was composed of 289,111 images. We used this dataset in Image RoI Detection and Image Hashing. On the other hand, as mentioned in Section 3.4.2, 200,000 frames were sampled from the Youtube 8M dataset [50] to train the image similarity Siamese network of Image Verfication.

4.2. Image RoI Detection

EfficientDet-D1 [12] was deployed to train the object detector and the image detector with MS-COCO [45] and image collage dataset, respectively. We generated 300,000 image collages by using the image collage generation method in Section 3.2 from the photo copyright dataset as shown in Figure 10. 270,000 and 30,000 image collages were used for training and testing, respectively. In object detection, the objectness score S o b j was used to select reliable objects among object predictions. We set the thresholds to T h o b j = 0.7 and T h i m g = 0.9 for the object detector and the image detector. Note that the larger T h i m g yields tighter bounding box for the image detector. Furthermore, to filter out small objects, we set the threshold for IoU (Intersection over Union) to T h i o u = 0.8 between an image region and an object region.
Figure 10. Samples from the image collage dataset. The collages are generated according to random partition graphs.

4.3. Image Hashing

The SimSiam [35] was employed for the image hashing network to generate binary descriptors. The ResNet-50 [28] pretrained by ImageNet [29] was used as the backbone network. The hidden unit size of Projection MLP and Prediction MLP was set as the same as the binary descriptor size, i.e., 128, 256, and 512.

4.4. Image Verification

For AKAZE [46] feature extraction, the threshold and number of octaves were empirically set to 10 − 4 and 3, respectively, and the reprojection threshold e t h in Algorithm 2 was set to 0.2.

5. Experimental Results

5.1. Image Detector with Image Collage Dataset

For image detection from image collages, we benchmark Yolo v3 [17], Faster R-CNN [14], and EfficientDet [12]. Yolo v3 and EfficientDet are one-stage detectors whereas Faster R-CNN is a two-stage detector.
Table 1 shows the results of image detection by using three detectors, which are trained on the image collage dataset. Avg. IoU and Avg. time indicate an averaged IoU between prediction and ground truth boxes and an average measure of time spent on a box prediction. The results of Yolo v3 and Faster R-CNN show the difference of one-stage and two-stage detectors. Faster R-CNN shows better average IoUs with ResNet-101 without Feature Pyramid Network (FPN) [19] than Yolo v3 while the average time of Faster R-CNN is more than ten times that of Yolo v3. EfficientDet shows much better average IoUs than those of Yolo v3 and Faster R-CNN and its inference speed is fast due to the characteristics of one-stage detectors. EfficientDet-D1 with the saturated average IoU is employed in this framework.

5.2. Image Hashing

To verify the performance of the image hashing network, identification accuracies are measured for geometric manipulation and color manipulation. To construct the photo copyright database, binary descriptors are generated from RoIs of original photos. For testing, random cropping is applied for the geometric manipulation and color distortion in HSV space; brightness change, and Gaussian blur are applied for the color manipulation. A Histogram of Gradients (HoG) [11] and BinaryGAN [10] are used for comparison with various binary descriptor sizes.

5.2.1. Geometric Manipulation

As shown in Table 2, the identification accuracies are measured according to IoUs between the original photos and randomly cropped images with various aspect ratios and sizes. The accuracies of all methods increase as an IoU and a descriptor size increase. The proposed image hashing achieves better accuracies than those of HoG and binaryGAN. These results show that the proposed image hashing network that learns various geometric manipulation is effective. HoG, a low-level handcrafted feature, shows a rapid decrease in accuracy as an IoU decreased, and BinaryGAN based on autoencoder network seems difficult to handle the geometric manipulation compared to the proposed image hashing.
Table 2. Identification results on the copyright image dataset by applying the HoG, the BinaryGAN and the proposed method with geometric manipulation and binary descriptor sizes.

5.2.2. Color Manipulation

Table 3 shows the identification accuracies of three methods. All methods show similar accuracies for color manipulations. HoG which is vulnerable to the geometric manipulations, shows slightly higher accuracies than BinaryGAN and the proposed image hashing for color distortion. Since HoG describes the input image with its graidents in detail, HoG obtains good accuracies if there is no geometric manipulation. BinaryGAN has the lowest accuracies for all color manipulations, and the proposed image hashing shows the highest accuracies for brightness change and Gaussian blur. These results show that the proposed method can handle color manipulations effectively by learning manipulated photos.
Table 3. Identification results on the copyright image dataset by applying the HoG, the BinaryGAN and the proposed method with various color manipulations and descriptor sizes.

5.3. Image Verification

In Image Verification, the performance of the image alignment and the image similarity Siamese network is measured after applying random clipping (10–30%).

5.3.1. Image Alignment

We verify whether AKAZE [46] is precise enough to perform the image alignment, and the results are shown in Table 4. After the image alignment, the significantly high accuracies are shown with small deviations. These average IoUs are not visually distinctive, and AKAZE is sufficient to utilize for Image Verification.
Table 4. Image alignment results according to clipping proportions.

5.3.2. Image Similarity Siamese Network

Figure 11 shows the receiver operating characteristic (ROC) curve of the image similarity measure results. The proposed similarity measure shows better performance than that of MSE and SSIM. Since handcrafted image quality metrics such as MSE and SSIM take an average of local similarities for their final similarity scores, it is insensitive to capture local differences. Specifically, SSIM is robust to linear transforms such as contrast enhancement and mean shift, but its similarity score is greatly reduced when nonlinear transforms are such as JPEG compression or blurring occur. On the other hand, since the proposed network is designed from a cognitive perspective on geometric structures, it is able to handle color manipulations better than the other similarity metrics. Table 5 also provides F1-scores and AUC (Area Under the Curve) of Figure 11, showing the great performance of the proposed similarity measure.
Figure 11. Receiver operating characteristic curves of similarity metrics with geometric coincident image pair or patch pairs. Patch means a similarity metric uses multiple patches sampled from an image as inputs. The pink curve (ROC of mse) is same as the cyan curve (ROC of mse_patch).
Table 5. Similarity metric performance with geometric coincident image pair or patch pairs. Patch means a similarity metric which uses multiple patches sampled from an image as inputs. FN and FP are False Negative and False Positive, respectively.
In addition, to show efficiency of similarity measure based on significantly different patches, the results of patch-based MSE and SSIM are also provided. MSE of sampled patches is measured as the same as MSE of an image due to its consistent similarity statistics. The patch-based SSIM reflects only local patches with large structural differences and seems very sensitive to subtle differences. However, due to the sensitivity to nonlinear transforms, the results of SSIM are worse than those of the proposed similarity measure.

5.4. Overall Framework Test for Photo Copyright Identification

To demonstrate the effectiveness of each module in the proposed framework, as shown in Table 6, Image RoI Detection and Image Verification are progressively applied to Image hashing. There are three identification results, identified, unidentified, and misidentified. unidentified is only decided by Image Verification.
Table 6. Ablation tests of photo copyright identification framework by using HoG [11], BinaryGAN [10], and the proposed image hashing with the binary descriptor size 512. The identification results are divided into identified, unidentified, and misidentified. Identified means the correct identification, and unidentified defers the image identification. Misidentified is the identification error (false positive rate), which should be reduced.
(1) Image hashing only shows the identification results without Image RoI Detection. Due to randomness, most image collages with one image are correctly identified, whereas image collages with more than two images are prone to be misidentified. In (2) Image RoI detection + Image hashing, images are separated well from images collages and this method achieves identified of more than 80%, but there still remains a misidentified rate of 11% at least. The last one, (3) Image RoI detection + Image hashing + Image verification, can distinguish unidentified from misidentified. By applying Image Verification, the proposed framework reduces the number of misidentified collages significantly and the number of misidentified collages which can be categorized as the false positive rate is reduced to 2.8 % as a consequence.
Figure 12 shows identification results of HoG, BinaryGAN, and the proposed framework. The number of results is the same as the number of images in a collage, meaning that the image detector works well to separate images from an image collage. The performance of the proposed framework is especially remarkable in comparison with the other methods in the second results of Figure 12. While the other methods give incorrect images similar to original images in terms of shape or color, the proposed framework correctly matches images by distinguishing image details. Lastly, only one image is determined as misidentified by reducing false positives.
Figure 12. Copyright identification results of image collage queries. (a–c) indicate the results of HoG [11], BinaryGAN [10], and the proposed hashing method, respectively. Green, Red, and blue boxes mean identified, misidentified, and unidentified results, respectively.

6. Conclusions

We proposed the photo identification framework to prevent copyright infringements. To handle manipulations by pirates, we developed preprocessing steps to identify all photo copyrights in images. First, we detected image RoIs from an image collage to distinguish multiple copyright photos. To train the image RoI detector, we collected the dataset for image collage and proposed the recursive partitioning method to build an image collage frame. Subsequently, we identified each RoI by using the proposed image hashing method that handles image manipulations. By augmenting images, we generated similar binary descriptors from an original image and a manipulated image. Consequently, we have verified identification results by measuring image similarities. This process reduced false positives and left them unidentified while reducing false positives. We experimentally showed the effectiveness of each step in the proposed photo copyright identification framework in comparison with the other methods.
The development of the identification framework is expected to automatically monitor the copyrights of infringed photos from websites, vitalize the photo copyright search service, and prevent copyright holders from copyright infringements. In addition to identification, an image descriptor can be developed for image retrieval, which is utilized in various web image search services. Users also will use legally copyrighted photos by paying royalty and be able to access a lot of available copyright photos. This virtuous cycle, creation of copyrighted works → use of creations and payment of royalties→ recreation copyrighted works, will make it possible to build a ecosystem for copyrights.

Author Contributions

Conceptualization, methodology, software, validation, formal analysis, investigation, resources, data curation, and writing—original draft preparation, D.K. and S.H.; writing—review and editing, D.K., S.H., J.K. and H.K.; supervision and funding acquisition, S.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research project was supported by Ministry of Culture, Sports and Tourism (MCST) and from Korea Copyright Commission in 2021.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Copytrack. Available online: https://www.copytrack.com/ (accessed on 27 September 2021).
  2. Korea Copyright Commission. Available online: https://www.copyright.or.kr/eng/main.do (accessed on 27 September 2021).
  3. Oh, T.; Choi, N.; Kim, D.; Lee, S. Low-complexity and robust comic fingerprint method for comic identification. Signal Process. Image Commun. 2015, 39, 1–16. [Google Scholar] [CrossRef] [Scilit]
  4. Lee, S.-H.; Kim, D.; Jadhav, S.; Lee, S. A restoration method for distorted comics to improve comic contents identification. Int. J. Doc. Anal. Recognit. 2017, 20, 223–240. [Google Scholar] [CrossRef] [Scilit]
  5. Lee, S.-H.; Kim, J.; Lee, S. An identification framework for print-scan books in a large database. Inf. Sci. 2017, 396, 33–54. [Google Scholar] [CrossRef] [Scilit]
  6. Kim, D.; Lee, S.-H.; Jadhav, S.; Kwon, H.; Lee, S. Robust fingerprinting method for webtoon identification in large-scale databases. IEEE Access 2018, 6, 37932–37946. [Google Scholar] [CrossRef] [Scilit]
  7. Google Image Search. Available online: https://www.google.co.kr/imghp (accessed on 27 September 2021).
  8. Yandex Images. Available online: https://yandex.com/images/ (accessed on 27 September 2021).
  9. TinEye Reverse Image Search. Available online: https://tineye.com/ (accessed on 27 September 2021).
  10. Song, J.; He, T.; Gao, L.; Xu, X.; Hanjalic, A.; Shen, H.T. Binary generative adversarial networks for image retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA, 2–7 February 2018; Volume 32.1. [Google Scholar]
  11. Mcconnell, R.K. Method of and Apparatus for Pattern Recognition. U.S. Patent 4,567,610, 28 January 1986. [Google Scholar]
  12. Tan, M.; Pang, R.; Le, Q.V. Efficientdet: Scalable and efficient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14–19 June 2020; pp. 10781–10790. Available online: https://openaccess.thecvf.com/content_CVPR_2020/html/Tan_EfficientDet_Scalable_and_Efficient_Object_Detection_CVPR_2020_paper.html (accessed on 20 August 2021).
  13. Girshick, R. Fast r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 7–13 December 2015; pp. 1440–1448. [Google Scholar]
  14. Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 2016, 39, 1137–1149. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  15. He, K.; Gkioxari, G.; Dollar, P.; Girshick, R. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2961–2969. [Google Scholar]
  16. Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788. [Google Scholar]
  17. Redmon, J.; Farhadi, A. YOLO9000: Better, faster, stronger. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 7263–7271. [Google Scholar]
  18. Zoph, B.; Cubuk, E.D.; Ghiasi, G.; Lin, T.-Y.; Shlens, J.; Le, Q.V. Learning data augmentation strategies for object detection. In Proceedings of the European Conference on Computer Vision, Glasgow, UK, 23–28 August 2020; pp. 566–583. [Google Scholar]
  19. Lin, T.-Y.; Dollar, P.; Girshick, R.; He, K.; Hariharan, B.; Belongie, S. Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 2117–2125. [Google Scholar]
  20. Cao, Y.; Long, M.; Liu, B.; Wang, J. Deep cauchy hashing for hamming space retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 1229–1237. [Google Scholar]
  21. Cakir, F.; He, K.; Bargal, S.A.; Sclaroff, S. Hashing with mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 2019, 41, 2424–2437. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  22. Zieba, M.; Semberecki, P.; El-Gaaly, T.; Trzcinski, T. BinGAN: Learning compact binary descriptors with a regularized GAN. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, QC, Canada, 3–8 December 2018; pp. 3612–3622. [Google Scholar]
  23. Deng, C.; Yang, E.; Liu, T.; Li, J.; Liu, W.; Tao, D. Unsupervised semantic-preserving adversarial hashing for image search. IEEE Trans. Image Process. 2019, 28, 4032–4044. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  24. Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial nets. In Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada, 8–13 December 2014; pp. 2672–2680. [Google Scholar]
  25. Krizehevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 2012, 25, 1097–1105. [Google Scholar]
  26. Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. In Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
  27. Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
  28. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
  29. Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M.; et al. ImageNet Large Scale Visual Recognition Challenge. Int. J. Comput. Vis. 2015, 115, 211–252. [Google Scholar] [CrossRef] [Scilit]
  30. Krizhevsky, A.; Hinton, G. Learning Multiple Layers of Features from Tiny Images. 2009. Available online: https://citeseerx.ist.psu.edu/ (accessed on 27 September 2021).
  31. Lecun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 1998, 86, 2278–2324. [Google Scholar] [CrossRef] [Scilit]
  32. Netzer, Y.; Wang, T.; Coates, A.; Bissacco, A.; Wu, B.; Ng, A.Y. Reading digits in natural images with unsupervised feature learning. In Proceedings of the NIPS Workshop on Deep Learning and Unsupervised Feature Learning, Granada, Spain, 10–13 December 2011. [Google Scholar]
  33. He, K.; Fan, H.; Wu, Y.; Xie, S.; Girshick, R. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 16–18 June 2020; pp. 9729–9738. [Google Scholar]
  34. Chen, T.; Kornblith, S.; Norouzi, M.; Hinton, G. A simple framework for contrastive learning of visual representations. In Proceedings of the International Conference on Machine Learning (PMLR), Montréal, QC, Canada, 6–8 July 2020; pp. 1597–1607. [Google Scholar]
  35. Chen, X.; He, K. Exploring Simple Siamese Representation Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 19–25 June 2021; pp. 15750–15758. [Google Scholar]
  36. Richemond, P.H.; Gril, J.-B.; Altche, F.; Tallec, C.; Strub, F.; Brock, A.; Smith, S.; De, S.; Pascanu, R. BYOL works even without batch statistics. arXiv 2020, arXiv:2010.10241. [Google Scholar]
  37. Florian, S.; Dmitry, K.; James, P. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 815–823. [Google Scholar]
  38. Lake, B.M.; Salakhutdinov, R.; Tenenbaum, J.B. Human-level concept learning through probabilistic program induction. Science 2015, 350, 1332–1338. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  39. Koch, G.; Zemel, R.; Salakhutdinov, R. Siamese neural networks for one-shot image recognition. In Proceedings of the ICML Deep Learning Workshop, Lille, France, 10–11 July 2015. [Google Scholar]
  40. Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 2004, 13, 600–612. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  41. Kim, J.; Lee, S. Deep learning of human visual sensitivity in image quality assessment framework. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 1676–1684. [Google Scholar]
  42. Kim, W.; Kim, J.; Ahn, S.; Kim, J.; Lee, S. Deep video quality assessor: From spatio-temporal visual sensitivity to a convolutional neural aggregation network. In Proceedings of the European Conference on Computer Vision, Munich, Germany, 8–14 September 2018; pp. 219–234. [Google Scholar]
  43. Kim, W.; Nguyen, A.-D.; Lee, S.; Bovik, A.C. Dynamic receptive field generation for full-reference image quality assessment. IEEE Trans. Image Process. 2020, 29, 4219–4231. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  44. Lin, T.Y.; Goyal, P.; Girshick, R.; He, K.; Dollár, P. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2980–2988. [Google Scholar]
  45. Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Dollar, P.; Zitnick, C.L. Microsoft coco: Common objects in context. In European Conference on Computer Vision; Springer: Cham, Switzerlnad, 2014; pp. 740–755. [Google Scholar]
  46. Alcantarilla, P.F. Solutions, T. Fast explicit diffusion for accelerated features in nonlinear scale spaces. IEEE Trans. Patten Anal. Mach. Intell. 2011, 34, 1281–1298. [Google Scholar]
  47. Lowe, D.G. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 2004, 60, 91–110. [Google Scholar] [CrossRef] [Scilit]
  48. Bay, H.; Tuytelaars, T.; Van, G.L. Surf: Speeded up robust features. In Proceedings of the European Conference on Computer Vision, Graz, Austria, 7–13 May 2006; pp. 404–417. [Google Scholar]
  49. Huber, P.J. Robust Statistics; Wiley: Hoboken, NJ, USA, 1981; p. 1. [Google Scholar]
  50. Abu-El-Haija, S.; Kothari, N.; Lee, J.; Natsev, P.; Toderici, G.; Varadarajan, B.; Vijayanarasimhan, S. Youtube-8m: A large-scale video classification benchmark. arXiv 2016, arXiv:1609.08675. [Google Scholar]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.