# An Approach for Streaming Data Feature Extraction Based on Discrete Cosine Transform and Particle Swarm Optimization

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

- A novel, efficient incremental feature extraction approach-based DCT is developed for the data stream to overcome the computation cost and time complexity problems.
- The proposed approach is based on DCT and PSO. To our knowledge, it is the first time using DCT and PSO for the data stream feature extraction algorithm.

## 2. Related Work

## 3. Metarials and Methods

#### 3.1. Initial Phase

#### 3.1.1. Data Normalization

#### 3.1.2. Discrete Cosine Transform

#### 3.1.3. Feature Selection

#### 3.1.3.1. Experimentally Feature Selection

#### 3.1.3.2. Automatic Feature Selection

Algorithm 1: Objective Function | |

Input: label, k, data | |

Output: score | |

_{1} | Divide data into k parts |

_{2} | for each i in k parts do |

_{3} | Set ith part as test data and initialize score as zero |

_{4} | Set remainder parts as training data |

_{5} | for each x in test data do |

_{6} | Calculate Euclidean Distances between x and train data |

_{7} | Find minimum distance and related index of train data |

_{8} | If label of the related indexed train data not equal to the label of x then |

_{9} | Increment score |

_{10} | end if |

_{11} | end for |

_{12} | Assign score to ith value of score array |

_{13} | end for |

_{14} | Average score array and set score as output of average score array. |

#### 3.2. Sequential Phase

## 4. Results and Discussion

^{®}Core

^{TM}i7-7500 (2.70 GHz) with 8 GB of random-access memory. All algorithms have been implemented as reported in their original papers. The result CIKPCA is used as reported in their original papers. Three main experiments are focused in this study. The first one is to investigate the influence of the proposed feature extraction approach on classification. The accuracy rate (Acc) [%], the number of the data stream that classified correctly (NDSCC), and F-score are evaluation metrics in the first experiment. Another experiment is to examine the influence of variation of the DCT coefficients. The last experiment is to investigate the effect of automatic feature selection in the proposed DCT-based feature selection approach. PSO and APSO algorithms have been implemented in MATLAB (R2016b) to handle automatic feature selection.

#### 4.1. Data Sets

#### 4.2. The Classification Performance

#### 4.3. The Analysis of the Variation of DCT Coefficients

#### 4.4. The Analysis of the Automatic Feature Selection

## 5. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## Abbreviations

DCT | Discrete Cosine Transform |

PSO | Particle Swarm Optimization |

APSO | Accelerated Particle Swarm Optimization |

PCA | Principal Component Analysis |

IPCA | Incremental Feature Extraction |

CCFIPCA | Candid Covariance Free Incremental Principal Component Analysis |

CIKPCA | Chunk Incremental Kernel Principal Component Analysis |

MOA | Massive Online Analysis |

Acc | Accuracy Ratio |

NDSCC | The Number of the Data Stream that Classified Correctly |

## References

- Tso, F.; Cu, L.; Zhang, L. Dragonnet: A robust mo-bile Internet service system for long-distance trains. IEEE Trans. Mob. Comput.
**2013**, 12, 2206–2218. [Google Scholar] - Atzori, L.; Iera, A.; Morabit, G. The internet of things: A survey. Comput. Netw.
**2010**, 54, 2787–2805. [Google Scholar] [CrossRef] - Armbrust, M.; Fox, A.; Griffith, R. A view of cloud computing. Commun. ACM
**2010**, 53, 50–58. [Google Scholar] [CrossRef] [Green Version] - Fu, Z.; Sun, X.; Liu, Q.; Zhou, L.; Shu, J. Achieving efficient cloud search services: Multi-keyword ranked search over encrypted cloud data supporting parallel computing. IEICE Trans. Commun.
**2015**, E98-B, 190–200. [Google Scholar] [CrossRef] - Lall, A.; Sekar, V.; Ogihara, M.; Xu, J.; Zhang, H. Data streaming algorithms for estimating entropy of network traffic. ACM Sigmetrics
**2006**, 34, 145–156. [Google Scholar] [CrossRef] [Green Version] - Aceto, G.; Ciuonzo, D.; Montieri, A.; Pescapé, A. Mobile encrypted traffic classification using deep learning: Experimental evaluation, lessons learned, and challenges. IEEE Trans. Netw. Serv.
**2019**, 16, 445–458. [Google Scholar] [CrossRef] - Gupta, A.; Birkner, R.; Canini, M.; Feamster, N.; Mac-Stoker, C.; Willinger, W. Network monitoring as a streaming analytics problem. In Proceedings of the 15th ACM Workshop on Hot Topics in Networks, Atlanta, GA, USA, 9–10 November 2016; pp. 106–112. [Google Scholar]
- Amini, A.; Saboohi, H.; Ying, W.T.; Herawan, T. A fast density-based clustering algorithm for real-time internet of things stream. Sci. World J.
**2014**, 2014, 926020. [Google Scholar] [CrossRef] - Tan, C.; Ji, G. Semi-supervised incremental feature extraction for large-scale data stream. Concurr. Comp-Pract. E
**2017**, 29, e3914. [Google Scholar] [CrossRef] - Zeng, X.Q.; Li, G.Z. Incremental partial least squares analysis of big streaming data. Pattern Recognit.
**2014**, 47, 3726–3735. [Google Scholar] [CrossRef] [Green Version] - Yan, J.; Zhang, B.; Liu, N.; Yan, S.; Cheng, Q.; Fan, W.; Yang, Q.; Xi, W.; Chen, Z. Effective and efficient dimensionality reduction for large-scale and streaming data preprocessing. TKDE
**2006**, 18, 320–333. [Google Scholar] - Ahmed, N.; Natarajan, T.; Rao, K.R. Discrete cosine transform. IEEE Trans. Comput.
**1974**, C-23, 90–93. [Google Scholar] [CrossRef] - Jolliffe, I.T. Principal component analysis and factor analysis. In Principal Component Analysis; Springer: New York, NY, USA, 1986; pp. 115–128. [Google Scholar]
- Zhao, H.; Yuen, P.C.; Kwok, J.T. A novel incremental principal component analysis and its application for face recognition. IEEE Trans. Syst. Man. Cybern. B Cybern.
**2006**, 36, 873–886. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Hall, P.; Martin, R. Incremental Eigenanalysis for Classification. In Proceedings of the British Machine Vision Conference, Southampton, UK, 14–17 September 1998; pp. 286–295. [Google Scholar]
- Hall, P.; Marshall, D.; Martin, R. Merging and splitting eigenspace model. IEEE Trans. Pattern Anal. Mach. Intell.
**2000**, 22, 1042–1049. [Google Scholar] [CrossRef] [Green Version] - Liu, X.; Chen, T. Shot boundary detection using temporal statistics modeling. In Proceedings of the 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, Orlando, FL, USA, 13–17 May 2002; p. IV-3389. [Google Scholar]
- Li, Y. On incremental and robust subspace learning. Pattern Recognit.
**2004**, 37, 1509–1518. [Google Scholar] [CrossRef] - Ozawa, S.; Pang, S.; Kasabov, N. A modified incremental principal component analysis for online learning of feature space and classifier. In Proceedings of the Pacific Rim International Conference on Artificial Intelligence, Auckland, New Zealand, 9–13 August 2004; pp. 231–240. [Google Scholar]
- Ozawa, S.; Pang, S.; Kasabov, N. Incremental learning of chunk data for online pattern classification systems. IEEE Trans. Neural Netw.
**2008**, 19, 1061–1074. [Google Scholar] [CrossRef] [Green Version] - Rosas-Arias, L.; Portillo-Portillo, J.; Hernandez-Suarez, A.; Olivares-Mercado, J.; Sanchez-Perez, G.; Toscano-Medina, K.; Perez-Meana, H.; Orozco, A.L.S.; García Villalba, L.J. Vehicle Counting in Video Sequences: An Incremental Subspace Learning Approach. Sensors
**2019**, 19, 2848. [Google Scholar] [CrossRef] [Green Version] - Fujiwara, T.; Chou, J.K.; Shilpika, S.; Xu, P.; Ren, L.; Ma, K.L. An incremental dimensionality reduction method for visualizing streaming multidimensional data. IEEE Trans. Vis. Comput. Graph.
**2019**, 26, 418–428. [Google Scholar] [CrossRef] [Green Version] - Jain, P.; Jin, C.; Kakade, S.M.; Netrapalli, P.; Sidford, A. Streaming PCA: Matching Matrix Bernstein and Near-Optimal Finite Sample Guarantees for Oja’s Algorithm. In Proceedings of the 29th Annual Conference on Learning Theory, New-York City, NY, USA, 23–36 June 2016; pp. 1147–1164. [Google Scholar]
- Kuncheva, L.I.; Faithfull, W.J. Pca feature extraction for change detection in multidimensional unlabelled streaming data. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), Tsukuba Science City, Japan, 11–15 November 2012; pp. 1140–1143. [Google Scholar]
- Qahtan, A.A.; Alharbi, B.; Wang, S.; Zhang, X. A pca-based change detection framework for multidimensional data streams: Change detection in multidimensional data streams. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, Australia, 10–13 August 2015; pp. 935–944. [Google Scholar]
- Weng, J.; Zhang, Y.; Hwang, W.S. Candid covariance-free incremental principal component analysis. IEEE Trans. Pattern Anal.
**2003**, 25, 1034–1040. [Google Scholar] [CrossRef] [Green Version] - Wei, Y.; Wang, H.; Wang, S.; Saporta, G. Incremental modelling for compositional data streams. Commun. Stat-Simul. C
**2019**, 48, 2229–2243. [Google Scholar] [CrossRef] - Tokumoto, T.; Ozawa, S. A fast incremental kernel principal component analysis for learning stream of data chunks. In Proceedings of the 2011 International Joint Conference on Neural Networks, San Jose, CA, USA, 31 July–5 August 2011; pp. 2881–2888. [Google Scholar]
- Ghashami, M.; Perry, D.J.; Phillips, J. Streaming kernel principal component analysis. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, Cadiz, Spain, 9–11 May 2016; pp. 1365–1374. [Google Scholar]
- Joseph, A.A.; Tokumoto, T.; Ozawa, S. Online feature extraction based on accelerated kernel principal component analysis for data stream. Evol. Syst.
**2016**, 7, 15–27. [Google Scholar] [CrossRef] - Chin, T.J.; Suter, D. Incremental kernel principal component analysis. IEEE Trans. Image Process.
**2007**, 16, 1662–1674. [Google Scholar] [CrossRef] - Takeuchi, Y.; Ozawa, S.; Abe, S. An efficient incremental kernel principal component analysis for online feature selection. In Proceedings of the International Joint Conference on Neural Networks, Orlando, FL, USA, 12–17 August 2007; pp. 2346–2351. [Google Scholar]
- Fredrik, H.; Paul, N. Incremental kernel PCA and the Nyström method. arXiv
**2018**, arXiv:1802.00043. [Google Scholar] - Liu, J.; Zhang, J.; Lee, S.W.; Zhao, F.; Rekik, I.; Shen, D. Two-Phase Incremental Kernel PCA for Learning Massive or Online Datasets. Complexity
**2019**, 2019, 5937274. [Google Scholar] - Dabbaghchian, S.; Ghaemmaghami, M.P.; Aghag-olzadeh, A. Feature extraction using discrete cosine transform and discrimination power analysis with a face recognition technology. Pattern Recognit.
**2010**, 43, 1431–1440. [Google Scholar] [CrossRef] - Nassih, B.; Amine, A.; Ngadi, M.; Hmina, N. DCT and HOG Feature Sets Combined with BPNN for Efficient Face Classification. Procedia Comput. Sci.
**2019**, 148, 116–125. [Google Scholar] [CrossRef] - Tjahyadi, R.; Liu, W.; Venkatesh, S. Application of the DCT energy histogram for face recognition. In ICITA 2004: Proceedings of the Second International Conference on Information Technology and Applications; IEEE: Sydney, Australia, 2004; pp. 314–319. [Google Scholar]
- Er, M.J.; Chen, W.; Wu, S. High-speed face recognition based on discrete cosine transform and RBF neural networks. IEEE Trans. Neural Netw.
**2005**, 16, 679–691. [Google Scholar] [CrossRef] - Yu, F.; Oyana, D.; Hou, W.C.; Wainer, M. Approximate Clustering on Data Streams Using Discrete Cosine Transform. JIPS
**2010**, 6, 67–78. [Google Scholar] [CrossRef] - Hayat, M.Z.; Hashemi, M.R. A dct based approach for detecting novelty and concept drift in data streams. In Proceedings of the 2010 International Conference of Soft Computing and Pattern Recognition, Paris, France, 7–10 December 2010; pp. 373–378. [Google Scholar]
- Yan, F.; Hou, W.C.; Jiang, Z.; Huan, Y.; Che, D. Selectivity estimation of range queries over data streams using cosine transform. Int. J. Comput. Sci.
**2007**, 1, 422–439. [Google Scholar] - Sharma, V.K.; Mahapatra, K.K.; Acharya, B. Visual object tracking based on discriminant DCT features. DSP
**2019**, 95, 102572. [Google Scholar] [CrossRef] - Cho, H.; Han, S.; Hwang, S.Y. Design of an Efficient Real-Time Algorithm Using Reduced Feature Dimension for Recognition of Speed Limit Signs. Sci. World J.
**2013**, 2013, 135614. [Google Scholar] [CrossRef] - Rashidi, S.; Fallah, A.; Towhidkhah, F. Feature extraction based DCT on dynamic signature verification. Scientia Iranica
**2012**, 19, 1810–1819. [Google Scholar] [CrossRef] [Green Version] - Wijaya, I.G.P.S.; Husodo, A.Y.; Arimbawa, I.W.A. Real time face recognition using DCT coefficients based face descriptor. In Proceedings of the 2016 International Conference on Informatics and Computing (ICIC), Mataram, Indonesia, 28–29 October 2016; pp. 142–147. [Google Scholar]
- Loizou, C.P.; Pantziaris, M.; Pattichis, C.S.; Seimenis, I. Brain MR Image Normalization in Texture Analysis of Multiple Sclerosis. JBGC
**2013**, 3. [Google Scholar] - Kennedy, J.; Eberhart, R. Particle swarm optimization. In Proceedings of the ICNN’95—International Conference on Neural Networks, Perth, WA, Australia, 27 Novenber–1 December 1995; Volume 4, pp. 1942–1948. [Google Scholar]
- Yang, X.S. Nature-Inspired Metaheuristic Algorithms; Luniver Press: Frome, UK, 2010. [Google Scholar]
- Fong, S.; Wong, R.; Vasilakos, A.V. Accelerated PSO swarm search feature selection for data stream mining big data. IEEE Trans. Serv. Comput.
**2015**, 9, 33–45. [Google Scholar] [CrossRef] - Cheng, X.; Ciuonzo, D.; Rossi, P.S. Multi-bit decentralized detection through fusing smart & dumb sensors based on rao test. IEEE Trans. Aerosp. Electr. Syst.
**2019**. [Google Scholar] [CrossRef] - Cheng, X.; Ciuonzo, D.; Rossi, P.S. Multi-bit decentralized detection of a weak signal in wireless sensor networks with a rao test. In Proceedings of the 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP), Shanghai, China, 19–21 November 2018; pp. 1–5. [Google Scholar]
- Fong, S.; Liang, J.; Fister, I.; Mohammed, S. Gesture recognition from data streams of human motion sensor using accelerated PSO swarm search feature selection algorithm. J. Sens.
**2015**, 2015, 205707. [Google Scholar] [CrossRef] - Deshpande, M.; Karypis, G. Evaluation of techniques for classifying biological sequences. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining, Taipei, Taiwan, 6–8 May 2002; pp. 417–431. [Google Scholar]
- Kong, X.; Philip, S.Y. An ensemble-based approach to fast classification of multi-label data streams. In Proceedings of the 7th International Conference on Collaborative Computing: Networking Applications and Worksharing (CollaborateCom), Orlanda, FL, USA, 15–18 October 2011; pp. 95–104. [Google Scholar]
- Khan, M.; Ding, Q.; Perrizo, W. k-nearest neighbor classification on spatial data streams using P-trees. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining, Taipei, Taiwan, 6–8 May 2002; pp. 517–528. [Google Scholar]
- Dua, D.; Graff, C. UCI Machine Learning Repository; University of California, School of Information and Computer Science: Irvine, CA, USA, 2019. [Google Scholar]
- Bifet, A.; Holmes, G.; Kirkby, R.; Pfahringer, B. Moa: Massive online analysis. JMLR
**2010**, 11, 1601–1604. [Google Scholar] - Laohakiat, S.; Phimoltares, S.; Lursinsap, C. A clustering algorithm for stream data with LDA-based unsupervised localized dimension reduction. Inf. Sci.
**2017**, 381, 104–123. [Google Scholar] [CrossRef]

**Figure 3.**The sliding window technique (M1). (

**a**) ($i+1$)th data is the current data stream sample, the initial model consists of N data, and N is the sample number of the initial model. (

**b**) The current sample is added at the end of the initial model. (

**c**) The first sample of the initial model is ejected from the model. (

**d**) Updated initial model with the current sample.

**Figure 4.**The M3 technique. (

**a**) $(N+1)$th data is the current sample, the initial model consists of N data, and N is the sample number of the initial model. (

**b**) The current sample is added to the end of the initial model. (

**c**) $(N+2)$th data comes for processing. (

**d**) $(N+2)$th data is added at the end of the initial model. The sample number of the initial model is increased to $N+2$.

**Figure 5.**Examining the influence of the interval length for five data sets: (

**a**) ElecNormNews (

**b**), Poker (

**c**), and Forest CovType (

**d**) DS1.

Data Dets | Instance | Attribute | Class | Characteristic |
---|---|---|---|---|

ForestCovType | 100,000 | 54 | 7 | real |

DS1 | 26,733 | 10 | 2 | synthetic |

ElecNormNews | 45,312 | 8 | 2 | real |

Poker | 100,000 | 10 | 8 | real |

Waveform | 5000 | 21 | 3 | synthetic |

Optic-digit | 5620 | 64 | 10 | real |

**Table 2.**Accuracy Ratio (Acc), The Number of the Data Stream that Classified Correctly (NDSCC), and F-scores for Principal Component Analysis (PCA) and the proposed approach.

Data Sets | PCA [13] | The Proposed Approach | ||||
---|---|---|---|---|---|---|

Acc [%] | NDSCC | F-Score | Acc [%] | NDSCC | F-Score | |

ForestCovType | 45.67 | 45,216 | 0.15 | 66.87 | 66,209 | 0.25 |

DS1 | 97.007 | 24,963 | 0.50 | 96.07 | 24,724 | 0.52 |

ElecNormNews | 42.76 | 18,950 | 0.50 | 64.42 | 28,548 | 0.63 |

Poker | 48.59 | 48,106 | 0.15 | 48.38 | 47,897 | 0.31 |

Data Sets | IPCA-Li [18] | CCFIPCA [26] | The Proposed Approach | |||
---|---|---|---|---|---|---|

M1 | M3 | M1 | M3 | M1 | M3 | |

(a) Acc[%] | ||||||

ForestCovType | 66.03 | 63.63 | 83.89 | 83.99 | 93.14 | 91.82 |

DS1 | 94.4 | 94.82 | 94.06 | 95.38 | 97.09 | 97.19 |

ElecNormNews | 62.54 | 59.13 | 64.41 | 62.20 | 81.35 | 79.06 |

Poker | 51.64 | 47.59 | 75.10 | 73.23 | 89.11 | 84.25 |

(b) NDSCC | ||||||

ForestCovType | 45,216 | 65,300 | 83,059 | 83,157 | 92,216 | 90,903 |

DS1 | 24,293 | 24,401 | 24,207 | 24,546 | 24,986 | 25,011 |

ElecNormNews | 27,713 | 26,203 | 28,545 | 27,566 | 36,049 | 35,034 |

Poker | 51,131 | 47,122 | 74,354 | 72,505 | 88,220 | 83,412 |

Data Sets | CIKPCA [30] | The Proposed Approach |
---|---|---|

Waveform | 74.8 | 75.4 |

Optical-digit | 88.3 | 97.5 |

Data Sets | APSO-DCT | IPCA-Ozawa [19] | ||
---|---|---|---|---|

Acc [%] | Avg Learning Time (s) | Acc [%] | Avg Learning Time (s) | |

ForestCovType | 93.93 | 1.06 | 18.94 | 760.61 |

DS1 | 97.16 | 1.28 | 46.63 | 702.09 |

ElecNormNews | 81.74 | 1.11 | 23.78 | 702.21 |

Poker | 91.49 | 1.18 | 21.89 | 840.61 |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Aydoğdu, Ö.; Ekinci, M.
An Approach for Streaming Data Feature Extraction Based on Discrete Cosine Transform and Particle Swarm Optimization. *Symmetry* **2020**, *12*, 299.
https://doi.org/10.3390/sym12020299

**AMA Style**

Aydoğdu Ö, Ekinci M.
An Approach for Streaming Data Feature Extraction Based on Discrete Cosine Transform and Particle Swarm Optimization. *Symmetry*. 2020; 12(2):299.
https://doi.org/10.3390/sym12020299

**Chicago/Turabian Style**

Aydoğdu, Özge, and Murat Ekinci.
2020. "An Approach for Streaming Data Feature Extraction Based on Discrete Cosine Transform and Particle Swarm Optimization" *Symmetry* 12, no. 2: 299.
https://doi.org/10.3390/sym12020299