# Application of Relative Entropy and Gradient Boosting Decision Tree to Fault Prognosis in Electronic Circuits

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Processing of Fault Data

#### 2.1. Feature Extraction

#### 2.2. FI

## 3. GBDT

- (1)
- Initializing weak learners$${f}_{0}(\mathrm{x})=\underset{c}{\underbrace{\mathrm{arg}\mathrm{min}}}{\displaystyle \sum _{i=1}^{m}L({\mathrm{y}}_{i},\mathrm{c})}\text{}$$
- (2)
- As for the iterations t = 1, 2, …, T, so
- (a)
- Based on the sample i = 1, 2, …, m, the negative gradient is calculated:$${r}_{ti}=-{\left[\frac{\partial L({\mathrm{y}}_{i},f({\mathrm{x}}_{i}))}{\partial f({\mathrm{x}}_{i})}\right]}_{f(\mathrm{x})={f}_{t-1}(\mathrm{x})}\text{}$$
- (b)
- Using $({\mathrm{x}}_{i},{r}_{ti})(i=1,2,\dots m)$, a CART is fitted to acquire the tth regression tree, whose corresponding leaf node is ${R}_{tj},j=1,2,\dots ,J$. Here, J refers to the number of leaf nodes of the regression tree t.
- (c)
- The best-fit value is calculated according to the leaf region $j=1,2,\dots ,J$.$${c}_{tj}=\underset{c}{\underbrace{\mathrm{arg}\mathrm{min}}}{\displaystyle \sum _{{x}_{i}\in {R}_{tj}}L({\mathrm{y}}_{i},{f}_{t-1}({\mathrm{x}}_{i})}+\mathrm{c})\text{}$$
- (d)
- Updating the strong learners,$${f}_{t}(\mathrm{x})={f}_{t-1}(\mathrm{x})+{\displaystyle \sum _{j=1}^{J}{c}_{tj}}I(\mathrm{x}\in {\mathrm{R}}_{tj})\text{}$$

- (3)
- The expression of the strong learner f(x) is thus attained:$$f(\mathrm{x})={f}_{T}(\mathrm{x})={f}_{0}(\mathrm{x})+{\displaystyle \sum _{t=1}^{T}{\displaystyle \sum _{j=1}^{J}{c}_{tj}}I(\mathrm{x}\in {\mathrm{R}}_{tj})}\text{}$$

## 4. Example Analysis

- (1)
- There are numerous electronic elements in circuit systems. The elements prone to faults are determined according to sensitivity analysis and expert experience. A hard fault is the most extreme form of parameter degradation-induced faults, and is caused by the deterioration of parameter degradation-induced faults. Therefore, predicting parameter degradation-induced faults in electronic elements is key to conducting on-condition maintenance of circuit systems.
- (2)
- The common parameter degradation-induced faults in which capacitance gradually decreases, while resistance gradually increases, are analyzed. The characteristic curve of the amplitude-frequency response of the fault within its parameter-degradation range is extracted.
- (3)
- At different frequencies, some voltage amplitudes undergo no significant change and even remain unchanged. To improve the prediction efficiency, only those frequencies with a large amplitude change in the amplitude-frequency response curve within the parameter-degradation range were extracted for analysis.
- (4)
- By applying the relative entropy distance, the changes in output voltages of specific frequency responses under nominal values were examined with the changes in the parameters of some key electronic elements.
- (5)
- In the curve of relative entropy distance, the first 100 time indices were selected as training samples while the 101st to 150th time indices were taken as testing samples to carry out fault prediction.
- (6)
- By taking the RMSE of predicted and tested values as the objective function, the parameters of GBDT were optimized to determine the best predicted result.
- (7)
- Based on a predicted rational unified process (RUP) and RMSE, the prediction model was evaluated.

#### 4.1. Feature Extraction and FI

#### 4.1.1. The Sallen–Key Band-Pass FILTER Circuit

#### 4.1.2. The Tow–Thomas Filter Circuit

- (1)
- Fault prediction for C2 was carried out using variance-based selection on its amplitude-frequency response: the voltages with a variance greater than 0.1 were selected, whose corresponding frequencies are 8912.509 Hz, 10 KHz, 11,220.18 Hz, 14,125.38 Hz, and 15,848.93 Hz, respectively. The relative entropies between the voltages values and amplitude-frequency voltage under nominal value conditions were calculated (Figure 8).
- (2)
- Fault prediction for R4 was conducted using variance-based selection on its amplitude-frequency response: the voltages with a variance greater than 0.015 were selected, whose corresponding frequencies are 6309.573 Hz, 7079.458 Hz, 7943.282 Hz, 11,220.18 Hz, and 12,589.25 Hz, respectively. The relative entropies between the voltages values and amplitude-frequency voltage under nominal value conditions were calculated (Figure 8). The changes in R4 mainly result in the forward shift of the cut-off frequency of the filter. Although the voltage changes at around 10 kHz, the relative entropy undergoes little change owing to the similar distribution of the model data.

#### 4.2. Fault Prognostic

## 5. Conclusions

- (1)
- To improve the operational efficiency and reduce the amount of redundant data, several specific frequencies with a large change in output voltage were screened within the full-frequency band using a variance-based selection method. The corresponding voltage changes at these frequencies were taken as sample data for measuring parameter change.
- (2)
- Using relative entropy, the distances from changing parameters of elements to those under normal working conditions were measured. Through comparison, it can be seen that the distance obtained using relative entropy shows a larger amplitude change and improves the anti-jamming ability. The two examples of circuits under test is Sallen-key filter circuit and Tow–Thomas filter circuit. The first one, which is a single-amplifier filter circuit, has high sensitivity to component tolerances of the circuit. The second circuit, which is a multi-amplifier circuit, has low-sensitivity to passive component variations. The relative entropy of C1 is greater than C2, and that of R1 is greater than R4, which just proves this. We can also see the entropy distance of capacitor C1 and C2 is greater than resistor R1 and R4, which fits the circuitous design philosophy, the filter circuit is sensitive to the change of capacitor.
- (3)
- The regression prediction was carried out using GBDT, the average prediction accuracy is 97.5%.

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Xie, S.; Zhang, S.; Liu, J. A Simulation Research on Gradual Faults in Analog Circuits for PHM. In Engineering Asset Management—Systems, Professional Practices and Certification; Springer International Publishing: Berlin, Germany, 2015; pp. 1635–1647. [Google Scholar]
- Hou, W.; Fan, X. A testability modeling method for analog circuit fault prediction. In Proceedings of the 2016 Prognostics and System Health Management Conference, Chengdu, China, 19–21 October 2016; pp. 1–6. [Google Scholar]
- Zhang, L.; Qin, Q.; Shang, Y. Application of DE-ELM in analog circuit fault diagnosis. In Proceedings of the Prognostics and System Health Management Conference, Chengdu, China, 19–21 October 2016; pp. 1–6. [Google Scholar]
- Yin, M.; Ye, X.; Tian, Y. Research of fault prognostic based on LSSVM for electronic equipment. In Proceedings of the 2015 Prognostics and System Health Management Conference, Beijing, China, 21–23 October 2015; pp. 1–5. [Google Scholar]
- Sun, J.; Wang, C. Analog circuit fault diagnosis based on mRMR and optimized SVM. Chin. J. Sci. Instrum.
**2013**, 34, 221–226. [Google Scholar] [CrossRef] - Lu, Z.K.; Ye, J. Cosine similarity measure between vague sets and its application of fault diagnosis. Res. J. Appl. Sci. Eng. Technol.
**2013**, 6, 2625–2629. [Google Scholar] [CrossRef] - Zhang, C.; He, Y.; Deng, F. An approach for analog circuit fault prognostics based on QPSO-RVM. Chin. J. Sci. Instrum.
**2014**, 35, 1751–1757. [Google Scholar] - Qi, T.; Zhang, Y.; Di, K. Analog Circuit Fault Prediction Based on LS-SVM Optimized by PSO. Comput. Meas. Control
**2016**, 24, 4–8. [Google Scholar] - Li, M.; Xian, W.; Long, B. Prognostics of Analog Filters Based on Particle Filters Using Frequency Features. J. Electron. Test. Theory Appl.
**2013**, 29, 567–584. [Google Scholar] [CrossRef] - Vasan, A.S.S.; Long, B.; Pecht, M. Diagnostics and prognostics method for analog electroniccircuits. IEEE Trans. Ind. Electron.
**2013**, 60, 5277–5291. [Google Scholar] [CrossRef] - Sun, S.; Yu, H.; Du, T. Vibration and Acoustic Joint Fault Diagnosis of Conventional Circuit Breaker Based on Multi-Feature Fusion and Improved QPSO-RVM. Trans. China Electrotech. Soc.
**2017**, 32, 107–117. [Google Scholar] [CrossRef] - Zhang, C.; He, Y.; Yuan, L. A Novel Approach for Diagnosis of Analog Circuit Fault by Using GMKL-SVM and PSO. J. Electron. Test.
**2016**, 32, 531–540. [Google Scholar] [CrossRef] - Zhang, C.; He, Y.; Yuan, L. A multiple heterogeneous kernel RVM approach for analog circuit fault prognostic. Clust. Comput.
**2018**, 1–13. [Google Scholar] [CrossRef] - Gao, M.; Xu, A.; Zhang, W. Adaptive modeling relevance vector machine and its application in electronic system state prediction. Syst. Eng. Electron.
**2017**, 39, 1898–1904. [Google Scholar] - Ichikawa, D.; Saito, T.; Ujita, W. How can machine-learning methods assist in virtual screening for hyperuricemia? A healthcare machine-learning approach. J. Biomed. Inform.
**2016**, 64, 20–24. [Google Scholar] [CrossRef] [PubMed] - Liu, L.; Ji, M.; Buchroithner, M. Combining Partial Least Squares and the Gradient-Boosting Method for Soil Property Retrieval Using Visible Near-Infrared Shortwave Infrared Spectra. Remote Sens.
**2017**, 9, 1299. [Google Scholar] [CrossRef] - Ding, C.; Wang, D.; Ma, X. Predicting Short-Term Subway Ridership and Prioritizing Its Influential Factors Using Gradient Boosting Decision Trees. Sustainability
**2016**, 8, 1100. [Google Scholar] [CrossRef] - Regadío, A.; Tabero, J.; Sánchezprieto, S. Impact of colored noise in pulse amplitude measurements: A time-domain approach using differintegrals. Nucl. Inst. Methods Phys. Res. A
**2016**, 811, 25–29. [Google Scholar] [CrossRef] - Chandrasekaran, V.; Shah, P. Relative entropy optimization and its applications. Math. Program.
**2016**, 161, 1–32. [Google Scholar] [CrossRef] - Winter, A. Tight Uniform Continuity Bounds for Quantum Entropies: Conditional Entropy, Relative Entropy Distance and Energy Constraints. Commun. Math. Phys.
**2016**, 347, 291–313. [Google Scholar] [CrossRef] [Green Version] - Hao, D.; Li, Q.; Li, C. Digital Image Stabilization Method Based on Variational Mode Decomposition and Relative Entropy. Entropy
**2017**, 19, 623. [Google Scholar] [CrossRef] - Rabiei, E.; Droguett, E.L.; Modarres, M. Fully Adaptive Particle Filtering Algorithm for Damage Diagnosis and Prognosis. Entropy
**2018**, 20, 100. [Google Scholar] [CrossRef] - Kriheli, B.; Levner, E. Entropy-Based Algorithm for Supply-Chain Complexity Assessment. Algorithms
**2018**, 11, 35. [Google Scholar] [CrossRef] - Friedman, J.H. Greedy Function Approximation: A Gradient Boosting Machine. Ann. Stat.
**2001**, 29, 1189–1232. [Google Scholar] [CrossRef] - Zhang, Y.; Haghani, A. A gradient boosting method to improve travel time prediction. Transp. Res. Part C Emerg. Technol.
**2015**, 58, 308–324. [Google Scholar] [CrossRef] - Brillante, L.; Gaiotti, F.; Lovat, L. Investigating the use of gradient boosting machine, random forest and their ensemble to predict skin flavonoid content from berry physical-mechanical characteristics in wine grapes. Comput. Electron. Agric.
**2015**, 117, 186–193. [Google Scholar] [CrossRef] - Zhou, J.; Tian, S.; Yang, C. A novel prediction method about single components of analog circuits based on complex field modeling. Sci. World J.
**2014**, 2014, 530942. [Google Scholar] [CrossRef] [PubMed] - Zhang, C.; He, Y.; Zuo, L.; Wang, J.; He, W. A novel approachto diagnosis of analog circuit incipient faults based on KECA and OAO LSSVM. Metrol. Meas. Syst.
**2015**, 22, 251–262. [Google Scholar] [CrossRef]

**Figure 4.**Flow chart for fault prediction. GBDT—gradient boosting decision tree; RUP—rational unified process; RMSE—root mean square error.

**Table 1.**Optimal parameters of the gradient boosting decision tree (GBDT). RMSE—root mean square error.

Circuit | Electronic Component | n_Estimators | Learning_Rate | Subsample | RMSE |
---|---|---|---|---|---|

Sallen–key | C1 | 80 | 0.2 | 0.7 | 0.1257797 |

R1 | 100 | 0.1 | 0.8 | 0.0585154 | |

Tow–Thomas | C2 | 90 | 0.3 | 0.7 | 0.1026238 |

R4 | 80 | 0.3 | 0.6 | 0.0759023 |

Circuit | Electronic Component | Real RUP | Estimated RUP | Deviation | Accuracy |
---|---|---|---|---|---|

Sallen–key | C1 | 50 | 48 | −2 | 96% |

R1 | 50 | 50 | 0 | 100% | |

Tow–Thomas | C2 | 50 | 52 | +2 | 96% |

R4 | 50 | 51 | +1 | 98% |

Average RMSE | Average Accuracy | |
---|---|---|

Relative entropy | 0.22236 | 97.5% |

cosine distance | 5.0512 × 10^{−7} | 96.5% |

Pearson’s correlation coefficient | 9.0165 × 10^{−8} | 96.0% |

Euclidean distance | 0.25836 | 97.0% |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Wang, L.; Zhou, D.; Zhang, H.; Zhang, W.; Chen, J.
Application of Relative Entropy and Gradient Boosting Decision Tree to Fault Prognosis in Electronic Circuits. *Symmetry* **2018**, *10*, 495.
https://doi.org/10.3390/sym10100495

**AMA Style**

Wang L, Zhou D, Zhang H, Zhang W, Chen J.
Application of Relative Entropy and Gradient Boosting Decision Tree to Fault Prognosis in Electronic Circuits. *Symmetry*. 2018; 10(10):495.
https://doi.org/10.3390/sym10100495

**Chicago/Turabian Style**

Wang, Ling, Dongfang Zhou, Hao Zhang, Wei Zhang, and Jing Chen.
2018. "Application of Relative Entropy and Gradient Boosting Decision Tree to Fault Prognosis in Electronic Circuits" *Symmetry* 10, no. 10: 495.
https://doi.org/10.3390/sym10100495