Decision tree is a classification model where the target function is represented by a sequence of queries based on the attributes of input test patterns. The sequence of queries is reflected in a tree structure with finite depth. The classification of a particular pattern begins at the root decision node and terminates in the leaf node where a single class label is specified as the classification result. Each decision node contains different tests involving one or more attributes of the test patterns. Depending on the output of the decision node, only one branch of the decision node is selected and the child (descendent) node is visited. This process is repeated until a leaf is reached.

Figure 1(a) depicts a binary decision tree diagram with 4 decision nodes A, B, C, D and three classes F1, F2, F3. In a given node, a function can be represented in the following general format: (1) f ( A 1 , A 2 , A 3 , … , A n ) = 0where f is a function of n attributes (A₁, A₂, A₃,…,A_n). Depending on the value of function f, either the left or right branch of the node is activated. If only one attribute A_i is considered in one node, we can simplify the function as (2) f = A i + w iwhere w_i is the threshold value of the attribute A_i. This function corresponds to the case of axis-parallel DTs (see e.g., ID3 [9] and C4.5 [10] or J48 in Weka [11]). On the other hand, oblique DTs (OC1 [12]) are represented by the following function (3) f = ∑ i = 1 n w i × A i + w n + 1where w_i represents the weight value of the attribute of A_i and w_n+1 is the threshold value of the function f.

While quadratic polynomial nonlinear DTs correspond to the function (4) f 2 n d = ( ∑ i = 1 n w i × A i + w n + 1 ) × ( ∑ j = 1 n w j ′ × A j + w n + 1 ′ ) = ∑ j = 1 n ∑ j = i n W i , j × A i × A j + ∑ i = 1 n W i × A i + Wwhere either A_i or A_j refers to a single attribute and the new weight W i , j = w i × w j ′ + w i ′ × w j ; W i = w n + 1 × w i + w n + 1 ′ × w n + 1 and the new threshold W = w n + 1 × w n + 1 ′.

Figure 1 depicts the three commonly used decision tree classification methods for the three-category problem. The two coordinate axes represent the two attributes A₁ and A₂ of test patterns. The decision boundaries of different decision tree methods have their own properties. For axis-parallel DTs, partition lines are parallel with the attribute axes. For oblique DTs, partition lines are oblique straight lines and the decision boundaries are curves for nonlinear DTs.

The three aforementioned DT classifiers are actually intimately related. For example, Equation (4) can be embedded into Equation (3) by introducing a new set of attributes corresponding to the products A i m i × A j m j with m_i < 2 and m_j < 2. Nonlinear DTs with polynomial hypersurfaces can be regarded as oblique DTs in the higher-dimension attribute space. In addition, axis-parallel DT can be treated as a special oblique tree where the weights for all attributes are zeros except for a single attribute whose weight is 1.

Nonlinear DTs are more complex than the other two types of DTs. Oblique DTs are normally much smaller than axis-parallel DTs due to their flexibility at each decision node. However, finding the best oblique DT is an NP-complete problem [12].

A straightforward approach to the hardware implementation of DTs would be to implement every DT node as a separate module as proposed in [13]. The idea is illustrated [Figure 2(a)] for the case of the DT diagram of Figure 1(a). The disadvantage of this approach is large latency, due to the fact that a new test pattern cannot be fed into the module before the classification output has propagated. To overcome the large latency problem, a universal node DT architecture [Figure 2(b)], using registers to pipeline each stage, was proposed in [14]. Although it reduces hardware cost by folding each level, it exhibits a relatively low throughput as a result of serial processing [13,14].

A second approach [6] to the hardware implementation of DTs is based on the equivalence between DTs and threshold networks [15]. It exhibits a relatively high throughput since the signals need only to propagate through the threshold unit and the combinational logic layers. In the threshold network architecture [Figure 2(d)], each decision node is implemented as a processing element in the threshold unit layer. Each element of the output layer is adopted to obtain the class-belonging according to the Boolean function. Obviously, this architecture requires more hardware resources to process the decision node, in parallel, and achieves higher classification speed.

In this paper, we propose to explore axis-parallel DT based classifiers, to remove the need for multiplication operations and enable a low cost compact implementation with a classification accuracy comparable to prior works [6]. OC1 tool is used to find the possibly optimal weight values by using a randomized algorithm with simulated annealing or heuristics methods [12].

For an oblique DT architecture, each node requires to realize the function given by Equation (3). The sum of the weighted feature is compared with a stored threshold value w_n+1 to make the decision on the node. With the increase of the number of attributes n, hardware resources increase linearly. By using pipelining and folding techniques, hardware resources can be minimized but throughput decreases linearly.

In the proposed axis-parallel DT architecture (Figure 4), each node generates the function given by Equation (2). The latter only requires a single digital comparator in place of the multiplication and accumulation (MAC) circuits. The processing element associated to each DT node is thus simplified significantly.

However due to the same classification problem, the built tree's node number and tree's depth are typically much larger than that of the oblique DT structure.

In prior works [5], a full custom chip was designed to implement the TLU layer for the oblique decision tree algorithm while AND-OR array-based combinational logic in CPLD was used to implement the output layer of the classifier. In this paper, we propose to design a tree-structure combinational logic circuit to replace the AND-OR array-based combinational logic circuit.

As shown in Figure 5, our proposed circuit for tree-based programmable logic circuit includes four parts: (i) a dynamic current mode tree-based decoder, (ii) two lines of transistor switches to enable programmability, (iii) a differential dynamic comparator to amplify the low-swing signal and (iv) latches in the input and output stages to synchronize the data with the TLU layer.

Figure 6 shows the stacked transistor tree circuit. This tree-based structure is the same as the DT diagram whereby only one branch will be turned on from the root to the leaf. There are two lines of switches that group all the stacked paths to differential outputs similarly to current based logic circuits [16]. To avoid static current consumption, the current source is controlled and the resistive load [16] is replaced by the parasitic capacitance. The operational principle is as follows. Firstly, the two lines of switches are configured according to the values stored in the registers. The stable input data come from the input latches. When reset is high, the differential outputs of the stacked transistor are pulled down to 0 (low). When reset is low and tree_enable is high, one of the differential outputs is conditionally charged.

The comparator used in our proposed circuit is illustrated in Figure 7. Because the accuracy of the comparator plays a critical role in the proposed circuit, transistors M_s1 and M_s2 are included to avoid hysteresis or delayed response in the resetting phase. The operational principle is as follows. When the clock is high, the comparator is operated in the resetting mode and both outputs (out and ōut) are pulled to 0 (low) and the bias transistor M_b is turned off to avoid the static current. On the other hand, when the clock is low, the circuit will execute the comparison of differential inputs. The outputs of the comparator will be buffered and latched to the next stage.

Figure 8 shows the timing waveform of our proposed circuit. The circuit is synchronous with the front-end TLU, whose data is serially processed and the valid signals appearing once every 11 clock cycles. The latches are used between the TLU and the programmable circuit to maintain the signals. There are four control signals in our circuit, which are generated by a synchronous control block. Latch_en signal is used to activate the latches when the output signals from TLU are valid. Next, the reset signal turns on the two reset transistors to discharge both differential buses of stacked tree and make them equal. After that, the root transistor turns on when Tree_enable = 1, the current flows from one specified path and charges one bus. Next, S A_enable activates the sense amplifier to amplify the low-swing signal. The latches in the output stage store the correct data for the next stage.

Figure 9 shows the experimental equipment used to acquire the data from our in-house fabricated tin-oxide gas sensor array. The gas delivery system includes four mass flow controllers (MFC), three of which are for the target gases and one for dry air. The gas sensor array was placed into a gas chamber, in which the gas concentration is adjusted by selecting the correct flow rate. Voltages across the sensors are measured and stored in the data acquisition board. This data was used to evaluate the performance of the proposed VLSI friendly DT classifier.

Figure 10 shows an example of measured voltage response across a gas sensor. A test gas with a fixed concentration is injected periodically and the injection phase is followed by a cleaning phase by injecting fresh dry air. After the cleaning phase, the signal will settle down at the baseline value which is the reference value before gas injection. When the test gas is injected into the chamber, the voltage across the sensor changes as a result of variations in its conductivity. After several minutes of gas injection, the sensor output is sampled for concentrations ranging from 20 ppm to 200 ppm. CO, H₂ and Ethanol are chosen as the test gases. Figure 10 shows the response of a gas sensor array comprising 12 individual tin oxide gas sensors fabricated in our design house. The overlapping curves of different gas sensors illustrate the low selectivity and non-linearity of the gas sensor array response, which adds to the complexity of the gas classification problem.

Principal component analysis (PCA) is often used to pre-process the initial data and decrease the dimensions of the feature vector [8]. Figure 11 shows the PCA pre-processing of the gas sensor array multivariate response for the case with two principal components—PC₁ and PC₂. In this section, we compare the performance of the proposed binary decision tree classifier with and without PCA pre-processing. 12 gas sensors are selected to build the sensor test vector. The dimensions of test vectors are reduced to 2, 3 and 4 by using PCA program implemented in MATLAB. Both original and PCA pre-processed data were fed into the OC1 software tool to build the axis-parallel decision tree and the oblique decision tree for gas classification. Table 1 shows classification performance and hardware complexity using the axis-parallel decision tree and the oblique decision tree classifiers with different number of principal components (PC) and without PCA, respectively.

According to our simulation results, both decision tree based classifiers benefit from the PCA algorithm. In particular, oblique decision tree hardware complexity can be significantly decreased by reducing the number of attributes in the test vector by using the PCA. However, PCA requires more hardware resources such as matrix multiplications. For example, if we want to select m principal components from n attributes, the PCA computation includes m × n multiplications and m × (n – 1) additions. From Table 1, one can note that with the increase of the number of principal components, the accuracy of the classifier increases. However, the requirements for hardware resources also increase. The axis-parallel decision tree classifier requires less computational units and memory bits compared to the oblique decision tree classifier. The trade-off between computational units and memory bits suggests that axis-parallel decision tree is a better choice for gas identification applications.

The proposed classifier was designed using Charter 0.18 μm CMOS process. The TLUs and the control unit were modeled using verilog HDL and synthesized using Design Compiler. The generated netlist was fed into SOC Encounter and the layout was generated through auto placement and routing using the standard digital cell library. The tree-structure programmable unit was designed in full-custom using Cadence tools. Figure 12 shows the layout of the complete classifier.

Table 2 compares the synthesized performance of oblique DT's Front-end TLU blocks together with our designed TLU blocks. Compared with prior art, this work significantly improves the hardware implementation cost in terms of power consumption, delay and silicon area as illustrated in Table 2.

Table 3 summarizes the overall performance in terms of hardware implementation cost and detection rate. To compare with previously reported FPGA implementation of gas classifiers [8], we also synthesized our classifier model using the same Xilinx Virtex II FPGA chip. According to our simulation results, our proposed axis-parallel DT classifier saves more than 90% FPGA resources compared with the committee machine method reported in [8]. It also saves up to almost 80% resources compared with the oblique DT classifier. Our proposed classifier only consumes 3.1 mW at a clock frequency of 100 MHz.