On the Use of Deep Active SemiSupervised Learning for Fast Rendering in Global Illumination
Abstract
:1. Introduction
 We used active semisupervised learning for efficient stochastic noise detection in global illumination rendering algorithms.
 We combined active and semisupervised learning to minimize the effect of imbalanced image classification in order to model the capability of the deep feature distribution for efficient noise detection.
 We extensively evaluated our algorithm on different global illumination scenes with different resolutions containing diffuse and specular surfaces. The proposed algorithm demonstrates outstanding performance compared with stateoftheart algorithms applied for accelerating global illumination rendering.
 We made a comparative study based on memory space and computation time. We showed that our model is computational efficient for real world applications because it yielded a small questioning time on a block of images.
2. Related Work on Active SemiSupervised Learning
3. Design of the Image Quality Database
4. The Proposed Method
4.1. Architecture of the Convolution Neural Network
4.2. Active SemiSupervised Learning of Noise Features
4.2.1. Active Learning
4.2.2. Deep SemiSupervised Learning
Algorithm 1 Deep active semisupervised learning algorithm 
Require:Set of scenes; P: Number of iterations of active learning; S: Number of iterations of semisupervised learning. 

5. Experimental Results
5.1. Experimental Setup
5.2. Parameter FineTuning
5.3. Performance Comparison
5.3.1. Comparative Study with SVM
5.3.2. Comparison Based on Memory Space and Time Complexities
5.3.3. Comparison with Other StateoftheArt Deep Learning Models
5.3.4. Comparison between Active and SemiSupervised Learning
6. Discussion
7. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
Code and Data Availability Policies
References
 Wang, B.; Meng, X.; Xu, Y.; Song, X. Fast Point Based Global Illumination. In Proceedings of the 2011 12th International Conference on ComputerAided Design and Computer Graphics (CAD/Graphics), Jinan, China, 15–17 September 2011; pp. 93–98. [Google Scholar]
 Ikeda, S.; Watanabe, S.; Raytchev, B.; Tamaki, T.; Kaneda, K. Spectral Rendering of Interference Phenomena Caused by Multilayer Films Under Global Illumination Environment. ITE Trans. Media Technol. Appl. 2015, 3, 76–84. [Google Scholar] [CrossRef]
 Popov, S.; Ramamoorthi, R.; Durand, F.; Drettakis, G. Probabilistic connections for bidirectional path tracing. Comput. Graph. Forum 2015, 34, 75–86. [Google Scholar] [CrossRef]
 Takouachet, N.; Delepoulle, S.; Renaud, C. A perceptual stopping condition for global illumination computations. In Proceedings of the 23rd Spring Conference on Computer Graphics, Budmerice, Slovakia, 26–28 April 2007; pp. 61–68. [Google Scholar]
 Lubin, J. A visual discrimination model for imaging system design and evaluation. In Vision Models for Target Detection and Recognition; Peli, E., Ed.; World Scientific: Singapore, 1995; pp. 245–283. [Google Scholar]
 Longhurst, P.; Chalmers, A. User validation of image quality assessment algorithms. In Proceedings of the TPCG 04: Theory and Practice of Computer Graphics, Bournemouth, UK, 8–10 June 2004; IEEE Computer Society: Washington, DC, USA, 2004; pp. 196–202, ISBN 0769521371. [Google Scholar]
 Constantin, J.; Bigand, A.; Constantin, I. Pooling spike neural network for fast rendering in global illumination. Neural Comput. Appl. 2019, 69, 49–58. [Google Scholar] [CrossRef]
 Constantin, J.; Bigand, A.; Constantin, I.; Hamad, D. Image Noise Detection in Global Illumination Methods based on FRVM. NeuroComputing 2015, 64, 82–95. [Google Scholar] [CrossRef]
 Takouachet, N.; Delepoulle, S.; Renaud, C.; Zoghlami, N.; Tavares, J.R. Perception of noise and Global Illumination: Towards an automatic stopping criterion based on SVM. Comput. Graph. 2017, 69, 49–58. [Google Scholar] [CrossRef] [Green Version]
 Lu, Z.; Wu, X.; Bongard, J.C. Active learning through adaptive heterogeneous ensembling. IEEE Trans. Knowl. Data Eng. 2015, 27, 368–381. [Google Scholar] [CrossRef]
 Xu, L.; Li, B.; Chen, E. Ensemble pruning via constrained eigenoptimization. In Proceedings of the 2012 IEEE 12th International Conference on Data Mining, ICDM, Brussels, Belgium, 10–13 December 2012; pp. 715–724. [Google Scholar]
 Maulik, U.; Chakraborty, D. A selftrained ensemble with semisupervised SVM: An application to pixel classification of remote sensing imagery. Pattern Recognit. 2011, 44, 615–623. [Google Scholar] [CrossRef]
 Bianco, S.; Celona, L.; Napoletano, P.; Schettin, R. On the use of Deep Learning for Blind Image quality Assessment. arXiv 2016, arXiv:1602.05531, 1–7, 1–7. [Google Scholar] [CrossRef]
 Guo, H.; Wang, W. An Active Learningbased SVM MultiClass Classification Model. Pattern Recognit. 2015, 48, 130–135. [Google Scholar] [CrossRef]
 Rhee, P.K.; Erdenee, E.; Kyun, S.D.; Ahmed, M.U.; Jin, S. Active and semisupervised learning for object detection with imperfect data. Cogn. Syst. Res. 2017, 45, 109–1231. [Google Scholar] [CrossRef]
 Guangwu, A.; Minggang, S. SemiSupervised Learning Based Social Image Semantic Mining Algorithm. J. Multimed. 2014, 9, 245–252. [Google Scholar]
 Richarz, J.; Vajda, S.; Grzeszick, R.; Fink, G. Semisupervised learning for character recognition in historical archive documents. J. Pattern Recognit. 2014, 47, 1011–1020. [Google Scholar] [CrossRef]
 Settles, B. Active Learning Litterature Survey; University of WisconsinMadison: Madison, WI, USA, 2009. [Google Scholar]
 Huang, T.; Yu, Y.; Guo, G. A classification algorithm based on local clusters with a few labeled training examples. Knowl. Based Syst. 2010, 23, 563–571. [Google Scholar] [CrossRef]
 Dornaika, F.; Traboulsi, Y.E.; Assoum, A. Inductive and Flexible Feature Extraction for SemiSupervised Pattern Categorization. Pattern Recognit. 2016. [Google Scholar] [CrossRef]
 Ruz, G.A. Improving the performance of inductive learning classifiers through the presentation order of the training patterns. Expert Syst. Appl. 2016, 58, 1–9. [Google Scholar] [CrossRef]
 Zhu, R.; Dornaika, F.; Ruichek, Y. Semisupervised elastic manifold embedding with deep learning architecture. Pattern Recognit. 2020, 107, 107425. [Google Scholar] [CrossRef]
 Zhou, Z.H.; Li, M. Tritraining: Exploiting unlabeled data using three classifiers. IEEE Trans. Knowl. Data Eng. 2005, 17, 1529–1541. [Google Scholar] [CrossRef] [Green Version]
 Rasmus, A.; Berglund, M.; Honkala, M.; Valpola, H.; Raiko, T. Semisupervised learning with ladder networks. Adv. Neural Inf. Process 2015, 14, 3546–3554. [Google Scholar]
 Tran, P.V. SemiSupervised Learning with SelfSupervised Networks. Machine learning. CoRR 2019. Available online: https://128.84.21.199/abs/1906.10343v1 (accessed on 2 September 2020).
 Zhai, X.; Oliver, A.; Kolesnikov, A.; Beyer, L. SelfSupervised SemiSupervised Learning. Computer Vision and Pattern Recognition. CoRR 2019. Available online: https://arxiv.org/pdf/1905.03670.pdf (accessed on 2 September 2020).
 Yan, W.; Sun, Q.; Sun, H.; Li, Y. Semisupervised learning framework based on statiscal analysis for image set classification. Pattern Recognit. 2020, 107, 107500. [Google Scholar] [CrossRef]
 Li, C.; Wu, C. A new semisupervised support vector machine learning algorithm based on active learning. In Proceedings of the International Conference on Future Computer and Communication, Wuhan, China, 21–24 May 2010; pp. 638–641. [Google Scholar]
 Leng, Y.; Xu, X.; Qi, G. Combining active learning and semisupervised learning to construct SVM classifier. Knowl. Based Syst. 2013, 44, 121–131. [Google Scholar] [CrossRef]
 He, X.; Ji, M.; Bao, H. A Unified Active and Semisupervised Learning Framework for Image Compression. In Proceedings of the International Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 65–72. [Google Scholar]
 Yu, D.; Varadarajan, B.; Deng, L.; Acero, A. Active learning and semisupervised learning for speech recognition: A unified framework using the global entropy reduction maximization criterion. Comput. Speech Lang. 2010, 24, 433–444. [Google Scholar] [CrossRef]
 Li, H.; Liao, X.; Carin, L. Active learning for semisupervised multitask learning. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing, Taipei, Taiwan, 19–24 April 2009; pp. 1637–1640. [Google Scholar]
 Constantin, J.; Constantin, I.; Bigand, A.; Hamad, D. Perception of noise in Global Illumination based on Inductive Learning. In Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada, 24–29 July 2016; pp. 5021–5028. [Google Scholar]
 Liu, X.; Pedersen, M.; Hardeberg, J. CID:IQ A New Image Quality Database. Image Signal Process. 2014, 8509, 193–202. [Google Scholar]
 Virtanen, T.; Nuutinen, M.; Vaahteranoksa, M.; Oittinen, P.; Häkkinen, J. CID2013: A Database for Evaluating NoReference Image Quality Assessment algorithms. IEEE Trans. Image Process. 2015, 24, 390–402. [Google Scholar] [CrossRef] [PubMed]
 Wald, I.; Kollig, T.; Benthin, C.; Keller, A.; Slusalleki, P. Interactive Global Illumination using Fast Ray Tracing. In Proceedings of the 13th Eurographics Workshop on Rendering, Pisa, Italy, 26–28 June 2002; pp. 15–24. [Google Scholar]
 Kollig, T.; Keller, A. Efficient Bidirectional Path Tracing by Randomized QuasiMonte Carlo Integration. In Monte Carlo and QuasiMonte Carlo Methods 2000; Niederreiter, H., Fang, K., Hickernell, F., Eds.; Springer: Berlin/Heidelberg, Germany, 2002; pp. 290–305. ISBN 9783540427186. [Google Scholar]
 Hedman, P.; Karras, T.; Lehtinen, J. Sequential Monte Carlo instant radiosity. In Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, Redmond, WA, USA, 26–28 February 2016; pp. 121–128. [Google Scholar]
 LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef]
 Razavian, A.; Azizpour, H.; Sullivan, J.; Carlsson, S. CNNfeatures offtheshelf: An astounding baseline for recognition. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition CVPR Workshops, Columbus, OH, USA, 23–28 June 2014; pp. 806–813. [Google Scholar]
 Makandar, A.; Halalli, B. Image Enhancement Techniques using Highpass and Lowpass Filters. Int. J. Comput. Appl. 2015, 109, 21–27. [Google Scholar] [CrossRef]
 Dawood, F.; Rahmat, R.; Kadiman, S.; Abdullah, L.; Zamrin, M. Effect Comparison of Speckle Noise Reduction Filters on 2Dechocardiographic. World Acad. Sci. Eng. Technol. 2012, 6, 425–430. [Google Scholar]
 Biswas, P.; Sarkar, A.; Mynuddin, M. Deblurring Images using a Wiener Filter. Int. J. Comput. Appl. 2015, 109, 36–38. [Google Scholar] [CrossRef]
 Gao, D.; Liao, Z.; Lv, Z.; Lu, Y. Multiscale Statistical Signal Processing of Cutting Force in Cutting Tool Condition Monitoring. Int. J. Adv. Manuf. Technol. 2015, 90, 1843–1853. [Google Scholar] [CrossRef]
 Gou, H.; Swaminathan, A.; Wu, M. Intrinsic Sensor Noise Features for Forensic Analysis on Scanners and Scanned Images. IEEE Trans. Inf. Forensics Secur. 2009, 4, 476–491. [Google Scholar]
 FernandezMaloigne, C.; RobertInacio, F.; Macaire, L. Digital Color: Acquisition, Perception, Coding and Rendering; Chapter 9: Quality Assessment Approaches, 314 Pages; WileyISTE: London, UK, 2012; ISBN 9781848213463. [Google Scholar]
 Abin, A.A.; Beigy, H. Active selection of clustering constraints: A sequential approach. Pattern Recognit. 2014, 47, 443–458. [Google Scholar] [CrossRef]
 Demir, B.; Persello, C.; Bruzzone, L. Batchmode active learning methods for the interactive classification of remote sensing images. IEEE Trans. Geosci. Remote. Sens. 2011, 49, 1014–1031. [Google Scholar] [CrossRef] [Green Version]
 Tong, S.; Koller, D. Support Vector Machine Active Learning with Applications to Text Classification. J. Mach. Learn. Res. 2000, 2, 45–66. [Google Scholar]
 Chapelle, O.; Scholkopf, B.; Zien, A. SemiSupervised Learning; MIT Press: Cambridge, MA, USA, 2006; 528p, ISBN 9780262033589. [Google Scholar]
 Amorin, W.; Rosa, G.; Thomazella, R.; Castanho, J.E.C.; Dotto, F.R.L.; Junior, U.P.R.; Marana, A.N.; Beigy, J.P.P. Semisupervised learning with connectivitydriven convolutional neural networks. Pattern Recognit. Lett. 2019, 128, 16–22. [Google Scholar] [CrossRef]
 Chen, D.G.; Wang, H.Y.; Tsang, E.C. Generalized Mercer theorem and its application to feature space related to indefinite kernels. In Proceedings of the IEEE International Conference on Machine Learning and Cybernetics, Kunming, China, 12–15 July 2008; Volume 2, pp. 774–777. [Google Scholar]
 Lipton, Z.; Elkan, C.; Narayanaswamy, B. Thresholding Classifiers to Maximize F1 Score. arXiv 2014, arXiv:1402.1892. [Google Scholar]
 Chollet, F. Deep Learning with Python; Manning Publications: Greenwich, CT, USA, 2017; 384p, ISBN 9781617294433. [Google Scholar]
 Falk, M.; Grottel, S.; Krone, M.; Reina, G. Interactive GPUBased Visualization of Large Dynamic Particle Data; Morgan & Claypool: San Rafael, CA, USA, 2016; 122p, ISBN 9781627052856. [Google Scholar]
Scenes with $800\times 800$ Resolution  Number of Paths between Two Successive SubImages  Largest Number of Paths 

(g)  5  700 
(h)  50  5000 
(i)  10  950 
(j)  50  5000 
(k)  10  950 
(l)  50  5000 
Layers  C1  C2  C3  C4  C5  C6  C7  C8  C9  C10  C11  C12 

Size  $3\times 3$  $5\times 5$  $3\times 3$  $5\times 5$  $3\times 3$  $5\times 5$  $3\times 3$  $5\times 5$  $3\times 3$  $5\times 5$  $3\times 3$  $5\times 5$ 
Padding  1  2  1  2  1  2  1  2  1  2  1  2 
Learning Model  Penality Factor C  Standard Deviation  Precision  Mean Number of Kernels 

SVM  8  50  96.24%  1750 
DSVM  8  70  90.7%  1970 
Learning Model  Penality Factor C  Standard Deviation  Precision  Mean Number of Kernels 

SVM  8  70  97.50%  1770 
DSVM  8  120  83.22%  5350 
Scenes  SVM  ${\mathrm{SVM}}_{\mathrm{AS}}$  DSVM  ${\mathrm{DSVM}}_{\mathrm{AS}}$ 

(a)  0  [0–0.01]  [0–0.15]  [0–0.13] 
(b)  [0–0.07]  [0–0.02]  [0–0.14]  [0–0.14] 
(c)  [0–0.07]  0  [0–0.13]  [0–0.11] 
(d)  [0–0.06]  0  [0–0.07]  [0–0.05] 
(e)  [0–0.07]  [0–0.01]  [0–0.1]  [00.07] 
(f)  [0.04–0.13]  [0–0.09]  [0–0.09]  [0–0.07] 
Scenes  SVM  ${\mathrm{SVM}}_{\mathrm{AS}}$  DSVM  ${\mathrm{DSVM}}_{\mathrm{AS}}$ 

(a)  1  0.99  0.92  0.93 
(b)  0.95  0.99  0.94  0.95 
(c)  0.95  1  0.91  0.95 
(d)  0.96  1  0.94  0.97 
(e)  0.95  0.99  0.94  0.96 
(f)  0.86  0.97  0.92  0.95 
Scenes  SVM  ${\mathrm{SVM}}_{\mathrm{AS}}$  DSVM  ${\mathrm{DSVM}}_{\mathrm{AS}}$ 

(g)  0  [0–0.03]  [0–0.09]  [0.01–0.15] 
(h)  [0.15–0.27]  [0–0.03]  [0.07–0.19]  [0.03–0.15] 
(i)  [0–0.15]  [0–0.15]  [0–0.13]  [0–0.07] 
(j)  [0.05–0.13]  [0–0.01]  [00.11]  [0–0.11] 
(k)  [0.07–0.35]  [0–0.03]  [00.11]  [0–0.11] 
(l)  [0–0.05]  [0–0.03]  [0–0.09]  [0–0.05] 
Scenes  SVM  ${\mathrm{SVM}}_{\mathrm{AS}}$  DSVM  ${\mathrm{DSVM}}_{\mathrm{AS}}$ 

(g)  1  0.99  0.90  0.90 
(h)  0.63  0.99  0.77  0.85 
(i)  0.91  0.98  0.93  0.97 
(j)  0.90  0.99  0.91  0.95 
(k)  0.67  0.99  0.93  0.96 
(l)  0.97  0.99  0.96  0.97 
Model  No. of Parameters  Learning Time (s)  Testing Time (s) 

SVM  20,038,855  4  0.17 
DSVM  42,120  1  0.008 
${\mathrm{SVM}}_{\mathrm{AS}}$  16,106,455  8053  0.15 
${\mathrm{DSVM}}_{\mathrm{AS}}$  69,768  2095  0.008 
Model  Precision (%) 

InceptionV3  76 
MobileNet  75 
Resnet210  81 
VGG19  86 
${\mathrm{SVM}}_{\mathrm{AS}}$  99 
${\mathrm{DSVM}}_{\mathrm{AS}}$  91 
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Constantin, I.; Constantin, J.; Bigand, A. On the Use of Deep Active SemiSupervised Learning for Fast Rendering in Global Illumination. J. Imaging 2020, 6, 91. https://doi.org/10.3390/jimaging6090091
Constantin I, Constantin J, Bigand A. On the Use of Deep Active SemiSupervised Learning for Fast Rendering in Global Illumination. Journal of Imaging. 2020; 6(9):91. https://doi.org/10.3390/jimaging6090091
Chicago/Turabian StyleConstantin, Ibtissam, Joseph Constantin, and André Bigand. 2020. "On the Use of Deep Active SemiSupervised Learning for Fast Rendering in Global Illumination" Journal of Imaging 6, no. 9: 91. https://doi.org/10.3390/jimaging6090091