Quantitative Research on Generalized Linear Modeling of SEU and Test Programs Based on Small Sample Data

Abstract

Complex integrated circuits (ICs) have complex functions and various working modes, which have many factors affecting the performance of a single event effect. The single event effect performance of complex ICs is highly program-dependent and the single event sensitivity of a typical operating mode is generally used to represent the single event performance of the circuits. Traditional evaluation methods fail to consider the cross effects of multiple factors and the comprehensive effects of each factor on the single event soft error cross section. In order to solve this problem, a new quantitative study method of single event error cross section based on a generalized linear model for different test programs is proposed. The laser test data is divided into two groups: a training set and a validation set. The former is used for model construction and parameter estimation based on five methods, such as the generalized linear model and Ensemble, while the latter is used for quantitative evaluation and validation of a single event soft error cross section of the model. In terms of percentage error, the minimum mean estimation error on the validation set is 13.93%. Therefore, it has a high accuracy to evaluate the single event soft error cross section of circuits under different testing programs based on the generalized linear model, which provides a new idea for the evaluation of a single event effect on complex ICs.

1. Introduction

With the rapid development of aerospace technology and deeper exploration in space, the requirements for performance of spacecraft are also increasing. Correspondingly, the reliability and the radiation-hardened performance of complex integrated circuits (ICs) are facing higher requirements [1]. Nowadays, with the reduction in the process size of semiconductor devices, the single event effect (SEE), is seriously affecting the safety of space missions, which is becoming more and more significant [2]. Therefore, it is necessary to evaluate the SEE sensitivity of complex ICs before they are applied to space missions [3]. At present, the SEE evaluation methods of ICs recognized by the industry mainly include a heavy ion test, a proton test, and other radiation tests. The SEE sensitivity of complex ICs has a strong program dependence on which different users have different concerns and applications, so the test programs cannot be traversed. In addition, due to the limited time and high cost of the heavy ion accelerator, it is not suitable for all ICs to carry out radiation tests. Therefore, several simulation methods and fault injection methods have been used to study the SEE sensitivity of complex ICs.

VSenek et al. [4] proposed a Single Event Upset (SEU) simulation prediction method based on the duty cycle to predict the SEU cross section of processors under different test programs. A simple error rate prediction model was preliminarily established. However, the method of analyzing the duty factor was difficult to be applied to complex applications, such as programs with conditional branches. Emmanuel et al. [5] carried out the SEU simulation on processor under different programs based on fault injection, and applied a new fault model of multi-fault injection to dual-fault injection. The model was designed to represent a possible non-concurrent radiation-induced soft error, which was only useful for specific processors in this paper. Gao jie, li qiang et al. [6] studied the relationship between the dynamic and the static SEU rates of satellite microprocessors by using the concept of program duty ratio and fault injection technology, which verified only by fault injection but not by radiation experiments. Zhao Yuanfu et al. [7] proposed a method to predict the SEE of complex ICs, which required a detailed analysis of different test programs and a large amount of work. The prediction method has been verified by radiation experiments, which has a good guiding significance.

The simulation method needs to establish different models for different circuits, which is complicated and time-consuming, and the simulation results have relatively large errors compared with the real test results. The method of fault injection has some problems of precision, accuracy, and speed for the modeling. None of the above methods have taken the cross influence of multiple factors and the comprehensive influence and accurate quantification of SEE soft error cross section into account.

In view of the above shortcomings, laser SEE test data as small sample training set has been used for modeling based on the generalized linear model. Quantitative evaluation on the SEE cross section of the circuit under different test programs has been conducted and the evaluation errors of training set and validation set under different methods are verified, and they have been compared. It provides a new idea for the evaluation of SEE soft error cross section in complex ICs. The SEU cross section of devices under different test programs can be predicted by the new method without carrying out radiation tests on all test modes. It can effectively solve the problem of evaluating the radiation performance of complex ICs under different test modes and obtain high accuracy.

2. Circuit Descriptions and Radiation Experiments

2.1. Circuit Description

The research object is a 32-bit radiation-hardened microprocessor, which has all the typical characteristics of complex ICs, such as large scale, high frequency, multiple modules and complex functions. The system-level error detection and correction are adopted by the radiation-hardened microprocessor. The circuit consists of an integer processing unit (IU), a floating point processing unit (FPU), CACHE, register (REGFILE), a debugging support unit (DSU), a serial port (UART), a storage/interrupt controller, a watchdog, a timer, and other units, which realize data interaction through AMBA bus. The functional block diagram of a microprocessor is shown in Figure 1.

2.2. Experiment Setting

In this paper, a set of function test programs is developed to simulate the typical function state of a user, which makes the CPU instruction coverage reach 100%, and it also covers all the logic units of the circuit. Taking P1–P4 test programs as the training test, they perform a single-precision integer operation and a double-precision floating point operation in the CACHE open and closed states, respectively, which cover eight standard test functions. P1 is a single-precision integer operation in CACHE ON mode, P2 is a single-precision integer operation in CACHE OFF mode. P3 is the double-precision floating-point operation in CACHE ON mode, and P4 is the double-precision floating-point operation in CACHE OFF mode. P5–P8 test programs are designed as validation test programs. In order to better verify the effectiveness of the model, this validation program has randomness, and it does not require instruction coverage and logical unit coverage. The processor will continuously access Refile data when executing different test programs. The statistics of register usage of the target circuit under eight different test programs are shown in

Table 1. Statistics of register usage under eight test programs.

Programs	Register Reads (Times)	Program Execution Cycle T1 (s)	Average Register Access Time T2 (us)
P1	~29,843	0.063	2.11
P2	~29,843	0.714	23.93
P3	~327,660	0.072	0.22
P4	~327,660	0.857	2.62
P5	~74,468	0.330	17.48
P6	~71,015	0.837	2.45
P7	~301,430	0.524	21.20
P8	~193,327	0.247	14.62

.

2.3. Radiation Experiments

The single event upset soft error cross sections (SEU cross sections) of the circuit under different test programs are obtained by pulse laser test [8]. During the test, the working voltage of the target circuit is set as the lowest level, where IO voltage is 2.97V and core voltage is 1.62 V. The backside irradiation laser test is carried out using PL2210A-P17 pulsed laser with 100 Hz frequency. The laser SEE test site is shown in Figure 2. The laser test data of the training set is shown in

Table 2. Laser SEE soft error cross section of Training set (10⁻⁷ errors/cm²).

	Equivalent LET	324pJ = 28.7 (MeV.cm2/mg)	432pJ = 37.5 (MeV.cm2/mg)	621pJ = 53 (MeV.cm2/mg)	950pJ = 80 (MeV.cm2/mg)
Test Programs
P1		11	31.8	51.7	114.3
P2		6.3	22	33	69.9
P3		16.9	46.5	72	147.8
P4		9.3	24.4	35	76.8

, and the laser test data of the validation set is shown in

Table 3. Laser SEE soft error cross section of validation set (10⁻⁷ errors/cm²).

	Equivalent LET	135pJ = 13.1 (MeV.cm2/mg)	241pJ = 21.8 (MeV.cm2/mg)	241pJ = 21.8 (MeV.cm2/mg)	889pJ = 75 (MeV.cm2/mg)
Test Programs
P5		18.1	23.5	33.6	112.2
P6		21.1	16.5	26.0	59.6
P7		21.9	14.7	39.5	131.7
P8		4.8	11.7	15.2	70.8

in which the effective laser energy (i.e., laser energy focused in the active region ${\mathrm{E}}_{\mathrm{eff}}$) has been equivalent to the LET value of heavy ion [9].

The conversion relation between the initial laser energy ${\mathrm{E}}_0$ and the LET value of the heavy ion is shown in Formula (1):

$$\left\{\begin{array}{c}{\mathrm{E}}_{\mathrm{eff}}=\mathrm{f}\left(1-\mathrm{R}\right){\mathrm{e}}^{-\mathsf{\alpha }\mathrm{h}}\left(1+{\mathrm{R}}^{\prime }\right){\mathrm{E}}_0\\ \mathrm{LET}=0.082{\mathrm{E}}_{\mathrm{eff}}+2.07\end{array}\right.$$

(1)

where f is the effect factor of the spot, R is the reflectance of the device surface, ${\mathrm{R}}^{\prime }$ is the metal layer reflectance of the device, $\mathsf{\alpha }$ is the silicon substrate absorption coefficient of the device and its measured value, and h is the Planck constant.

Four validation programs for P5–P8 were designed, and laser tests were carried out under different laser energies, respectively. The obtained laser test data of P5–P8 are shown in Table 3.

3. Modeling and Parameter Estimation

Compared with the data obtained by software simulation, the laser test data is closer to the radiation sensitivity of the circuit in the actual radiation environment, so the model established based on laser test data have a higher accuracy. There are 16 observations of laser test data, which are typical small sample data, and the use of second-order and above polynomial or tree models may lead to overfitting [10,11,12]. The generalized linear model is linear with respect to the unknown parameters, but nonlinear with respect to the known variables. The nonlinear functional relational quantization model between the independent variable and the dependent variable can be established based on the linear parameters and multiple bases. The corresponding model has a good fitting effect and prediction accuracy. Therefore, the pulsed laser SEE test data of P1–P4 as the training set are used to build the model based on the generalized linear model. Four methods of generalized least squares method [10,11], the weighted least squares estimation method [13], the median regression method [12], and the least trimmed squares method [12] are used to obtain the estimation parameter of the generalized linear model. The four methods mentioned above are combined with optimal weights, namely the Ensemble method as the fifth method. Then, a laser test is performed on the target circuit under the P5–P8 validation programs, and the obtained laser test data is used as the validation set to verify the generalized linear model. The flow chart of the method to quantitatively evaluate the SEE soft error cross section of complex ICs based on the generalized linear model is shown in Figure 3.

Firstly, the P1–P4 test program for the training set is written, and 16 groups of small sample test data are obtained by laser test. The test data is used as the training set for the generalized linear model. Then, GLS, WLS, MR, LTS, and Ensemble methods are established and parameters are optimized on the target functions. The P5–P8 test programs as the validation set are written and then the generalized linear model is used to predict the SEE soft error cross section. Meanwhile, the soft error cross section under radiation is obtained by laser test. By evaluating the prediction error and the confidence interval on the test data, the precision of the quantitative prediction model is obtained.

3.1. Model Building

The generalized linear model is established using Formula (2).

$$g\left(SEU\right)={\beta }_0+{{\displaystyle \sum }}_{i=1}^p{\beta }_i{f}_i\left(LET,times,T1,T2\right)+\epsilon$$

(2)

Formula (2) represents the quantitative relationship between SEU and its impact factor, such as $LET,times,T1,T2$. Since the value of the SEU soft error cross section must be positive, and the right side of the equation takes the value of the entire real number domain, we take the logarithm of the SEU and put it into the model, that is, taken as the link function of Poisson regression. The model is linear with respect to unknown parameters, but nonlinear with respect to several known independent variables. Here, the function $f_i\left(LET,times,T1,T2\right)$ can be made of any nonlinear function about independent variables according to your observations and assumptions, such as neural networks, GBDT (gradient boosting decision tree), spline functions, or polynomials about independent variables, etc. [10,11]. In this paper, $f_i\left(LET,times,T1,T2\right)$ is taken as each independent variable itself to minimize the number of parameters to prevent overfitting and to achieve better predictions. The simulation software used for modeling and parameter estimation in this paper is R-3.5.3.

At this point, the above model can be simplified to Formula (3)

$$Y=X\beta +\epsilon$$

(3)

Among them, $Y=\mathit{log}(SEU)={(y_1,y_2,\dots ,y_n)}^T$, $n$ is the number of experiments, which is 16 in this paper, and $SEU$ is a column vector of length n. Each column of $X$ is the number of registers read times (times), program execution cycle (T1), average register access time (T2), LET and an intercept term (I) that is all 1, each row of $X$ is an observation data. The dimension of $X$ matrix is $n\times 5$. $\beta$ is the unknown parameter column vector of length 5 to be solved, and $\epsilon$ is the measurement error column vector of length $n$.

3.2. Variable Selection

There are multiple criteria such as $C_p$, AIC, and BIC for variable selection of the model [11]. In this paper, AIC criterion is chosen for variable selection, which is the sum of the negative average log-likelihood and the penalty term considering the number of parameters. The lower the value, the better the prediction effect of the model. The result of variable selection using the AIC criterion is shown in Figure 4.

As can be seen from Figure 3, the final result of variable selection retains the number of register reads (times), program execution cycle (T1), LET and the intercept term (I) that is all 1, and it excludes the average register access time (T2).

3.3. Parameter Estimation

For different assumptions of measurement error or different loss functions, different parameter estimates $\widehat{\beta }$ and predicted values of SEU cross section $S\widehat{E}U$ can be obtained. In order to further analyze the experimental data and to obtain a more accurate and robust model, the following four methods are firstly used to carry out model building, in which the first two are based on the Gaussian distribution assumption, and the latter two are robust parameter estimation methods, both of them are used to solve Formula (3), and $\widehat{\beta }$ is the estimated value of $\beta$. Methods for parameter estimation are shown in

Table 4. Methods for parameter estimation.

Methods for Parameter Estimation	Formulas	Parameter Description
GLS, generalized least squares [10,11]	${\widehat{\beta }}_{GLS}=\underset{\beta }{argmin}{\displaystyle {\displaystyle \sum }_{i=1}^n}{(Y_i-X_i^T\beta )}^2={(X^{T}X)}^{-1}{X}^{T}Y$	$X_i$ is the column vector formed for the i row of the matrix X.
WLS, weighted least squares [13]	$\begin{array}{c}{\widehat{\beta }}_{WLS}=\underset{\beta }{argmin}{\displaystyle {\displaystyle \sum }_{i=1}^n}{w}_i{(Y_i-X_i^T\beta )}^2\\ ={(X^{T}WX)}^{-1}{X}^{T}WY\end{array}$ $W=diag\left\{w_1,w_2,\dots ,w_n\right\},w_i=\frac{1}{SE{U}_i}$	$X_i$ is the column vector formed for the i row of the matrix X.
MR, median regression [12]	${\widehat{\beta }}_{MR}=\underset{\beta }{argmin}{\displaystyle {\displaystyle \sum }_{i=1}^n}\left|Y_i-X_i^T\beta \right|$	To calculate the ${\widehat{\beta }}_{MR}$, slack variables can be introduced and the simplex method can be used [13].
LTS [12], least trimmed squares	${\widehat{\beta }}_{LTS}=\underset{\beta }{argmin}{\displaystyle {\displaystyle \sum }_{i=1}^h}{\left|\epsilon \right|}_{\left(i\right)}^2$ $h=\left[\frac{n+p+1}{2}\right]$ ${\left|\epsilon \right|}_{\left(1\right)}\le {\left|\epsilon \right|}_{\left(2\right)}\le \dots \le {\left|\epsilon \right|}_{\left(n\right)}$	$p$ is the number of columns of $X, \left[m\right]$ represents the largest integer not greater than $m. {({\left|\epsilon \right|}_{\left(1\right)},{\left|\epsilon \right|}_{\left(2\right)},\dots ,{\left|\epsilon \right|}_{\left(n\right)})}^T$ is the vector sorted by absolute value from smallest to largest for each element in $\epsilon ={({\epsilon }_1,{\epsilon }_2,\dots ,{\epsilon }_n)}^T$.

.

The estimation parameter for the training set obtained by the above four methods are shown in

Table 5. Estimation parameters for training set under different methods.

Column Names of X	GLS	WLS	MR	LTS
(Intercept)	1.71685	1.52232	2.34029	2.64312
LET	0.03934	0.04237	0.03053	0.02697
times	1.0578 × 10−6	1.18 × 10−6	8.26 × 10−7	7.48 × 10−6
T1	−0.78625	−0.79952	−0.78671	−0.83454

. It can be seen that the influence coefficients of LET and times obtained by different methods are positive, while T1 is on the contrary, and the values of the four methods are relatively close, that is, they increase with the growing of LET and times and decrease with the increase of T1.

3.4. Model Optimization

Considering that each of the four methods in Section 3.3 may have advantages and disadvantages, the Ensemble method [10,11], which performs optimized linear weighting on the predicted values of these four methods by minimizing the combined variance, is adopted to reduce the prediction variance, shrink the confidence interval, and make predictions more robust. It can be seen from Formula (4) that after Ensemble, the variance of the forecast value, which is the weighted average of the evaluation from the four methods mentioned above, is reduced to minimize and thus becomes more reliable.

$$\begin{array}{c}\underset{p}{min}{p}^T\Sigma p\\ s.t. {\displaystyle \sum _{i=1}^4}{p}_i=1\\ p_i\ge 0\end{array}$$

(4)

$\Sigma$ is the covariance matrix of the evaluation errors of the four methods.

The covariance matrix of these four methods in Section 3.3 calculated based on the training dataset is shown in

Table 6. Covariance matrix of the evaluation errors of the four methods.

	GLS	WLS	MR	LTS
GLS	122.15	165.03	33.73	10.30
WLS	165.03	230.93	31.04	−2.90
MR	33.73	31.04	38.39	38.97
LTS	10.30	−2.90	38.97	48.53

, where the diagonal is the variance of the evaluation errors of the four methods. It can be seen that the evaluation variances of the two robust parameter estimation methods, MR and LTS, are all less than 50, which means the standard deviation of the evaluation errors of the SEU cross section prediction value is less than $\sqrt{50}=7.07$. The covariance of the WLS method and the LTS method is the smallest, which is −2.9, thus the correlation coefficient of these two is ${\rho }_{WLS,LTS}=\frac{-2.9}{\sqrt{230.93\ast 48.53}}=-0.03$. The negative correlation of these two is used by the Ensemble method to further reduce the evaluation error variance of SEU.

The inner point method of constraint optimization algorithm [12,13] is used to calculate the optimal weights $p=\left\{\begin{array}{c}0.00118461\\ 0.09161878\\ 0.68351429\\ 0.22368231\end{array}\right\}$. It can be seen that MR method has the largest weight, while GLS method has almost zero weight.

3.5. Evaluation Error

In order to verify the evaluation performance of the model on the test data, the above five evaluation methods are used in the training set and the validation set to verify the evaluation effect of the model, respectively. For the evaluation of error, root mean square error (RMSE) [10,11], average error percentage, and other various evaluation indicators are selected. The smaller the value, the smaller the evaluation error and the higher the evaluation accuracy. The evaluation errors of the training set and the validation set under different methods are shown in

Table 7. Evaluation errors of training set and validation set under different methods.

	Evaluation Errors of Training Set under Different Methods					Evaluation Errors of Validation Set under Different Methods
Method	GLS	WLS	MR	LTS	Ensemble	GLS	WLS	MR	LTS	Ensemble
Root mean square error	10.74	14.87	6.22	8.03	6.40	8.92	11.11	6.44	7.57	6.41
Mean absolute error	9.11	11.31	4.38	5.05	4.76	7.58	9.34	5.03	6.03	4.83
Mean absolute error in percent %	22.38	30.98	12.96	16.73	13.33	19.39	24.15	14.0	16.4	13.93

.

Taking RMSE [10,11], the most commonly used evaluation method, as an example, Ensemble and the MR method have relatively low RMSE in the training set and the validation set, while the other three methods have relatively large evaluation errors in validation set. The MR method, as one of the robust parameter estimation methods, shows a lower evaluation error in the training set and the validation set compared with the GLS and the WLS methods based on Gaussian distribution, indicating that the error of SEU data collected in the experiment may be non-Gaussian distribution, or Gaussian characteristics are not obvious due to small samples. It also verified the reliability of the MR and the Ensemble methods in data modeling, that is, the quantitative evaluation model established by us is effective, and it has a certain accuracy.

4. Confidence Interval Analysis

In the above section, the evaluation errors are compared. Considering the quantity of data is small, the performance of five methods can be affected by accidental data, therefore in this section further analysis on the confidence interval is carried out. The shorter confidence interval means a smaller evaluation error variance and a higher reliability. So, in this section, further comparison of confidence interval with five methods is performed.

In this paper, the bootstrap method based on statistical sampling is used to calculate the confidence interval of SEU cross section. Multiple SEU cross sections can be obtained by repeated sampling with replacements. The bootstrap times in this paper is set as B = 300.

Figure 5, Figure 6, Figure 7 and Figure 8 show the evaluation value and a 95% confidence interval of the five methods on the training set and the validation set, respectively. The evaluation error is the difference between the evaluation value and the soft error cross section value observed in the laser experiment. Gray bars representing known experimental values, points and line segments of five colors are the respective evaluation values and confidence intervals of the five methods. The narrower the confidence interval is, the higher the evaluation accuracy is. Figure 7 and Figure 8 show the probability density function of the evaluation error of the five methods on the training set and validation set, which is calculated using kernel method (gaussian kernel is used, bandwidth is determined based on cross-validation). It can be seen that the value of the error corresponding to the highest density of Ensemble and MR methods on the validation set is concentrated near zero, but the former is more concentrated at zero than the latter. The error distribution of the other three methods is relatively flat, and the error corresponding to the highest probability density is far from zero. As can be seen from Figure 5, Figure 6, Figure 7 and Figure 8 and Table 7, the Ensemble method has the best evaluation accuracy RMSE and the narrowest confidence interval in the validation set, which can be used to evaluate the SEE soft error cross section based on different test programs and laser energy.

5. Conclusions

A new method based on the generalized linear model for quantitative evaluation of a SEE soft error cross section under different test programs is presented. Different test programs are designed, and the data sets of a laser test under several test programs and register calls of different test programs are divided into training and validation groups. The training set is used for modeling and parameter estimation of the five methods, and the validation set is used to evaluate the model accuracy. The evaluation value, evaluation error, 95% confidence interval, and the probability density function of the five methods on the training set and the validation set are computed. The results show that the quantitative evaluation method of complex ICs based on generalized linear models can achieve a high accuracy of 13.93%. Various factors of the SEE sensitivity of comprehensive effect and quantitative evaluation are considered in the evaluation method. Quantitative evaluation is suitable for small sample experiment data under different test programs, which is an applicable innovation.