Feedforward Factorial Hidden Markov Model

Abstract

This paper introduces a novel kind of factorial hidden Markov model (FHMM), specifically the feedforward FHMM (FFHMM). In contrast to traditional FHMMs, the FFHMM is capable of directly utilizing supplementary information from observations through predefined states, which are derived using an automatic feature filter (AFF). We investigate two variations of FFHMM models that integrate predefined states with the FHMM: the direct FFHMM and the embedded FFHMM. In the direct FFHMM, alterations to one sub-hidden Markov model (HMM) do not affect the others, enabling individual improvements in HMM estimation. On the other hand, the sub-HMM chains within the embedded FFHMM are interconnected, suggesting that adjustments to one HMM chain may enhance the estimations of other HMM chains. Consequently, we propose two algorithms for these FFHMM models to estimate their respective hidden states. Ultimately, experiments conducted on two real-world datasets validate the efficacy of the proposed models and algorithms.

1. Introduction

The factorial hidden Markov model (FHMM) represents a variant of the Hidden Markov Model (HMM) that incorporates distributed state representations [1]. The latent state space encompasses multiple chains that evolve concurrently, collectively influencing the single observation at any given moment [2]. Due to its ability to model outcomes influenced by several independent, time-varying processes, the FHMM is suitable for a wide range of applications. For instance, the FHMM has been successfully utilized in various fields, including visual tracking [3], non-intrusive load monitoring (NILM) [4], consumers’ choice models [5], natural language processing [6], estimating tumor clonal composition [7], fault diagnosis [8], online skill rating [9], random telegraph noise [10], forecasting financial portfolios with value at risk [11], and pitch and formant tracking [12].

According to [13], an additive FHMM model is defined as

Model 1. 

$$\begin{array}{ccc}\hfill x_1^{\left(i\right)}& \sim & \mathit{Cat}\left({\varphi }^{\left(i\right)}\right),\hfill \end{array}$$

(1)

$$\begin{array}{ccc}\hfill x_t^{\left(i\right)}|x_{t-1}^{\left(i\right)}& \sim & \mathit{Cat}(P_{x_{t-1}^{\left(i\right)}{x}_t^{\left(i\right)}}^{\left(i\right)}),\hfill \end{array}$$

(2)

$$\begin{array}{ccc}\hfill {\overline{y}}_t|x_t^{(1:N)}& \sim & \mathcal{N}\left(\sum _{i=1}^N{\mu }_{x_t^{\left(i\right)}}^{\left(i\right)},\Sigma \right),\hfill \end{array}$$

(3)

In this model, $N\in {\mathbb{Z}}_{+}$ is the number of HMM chains; $m_i\in {\mathbb{Z}}_{+}$ is the number of states in the i-th HMM; $x_t^{\left(i\right)}\in \{1,\dots ,m_i\}$ denotes the state of the i-th HMM at time t; ${\varphi }^{\left(i\right)}\in {[0,1]}^{m_i}$ is the initial state distribution for the i-th HMM, while ${\varphi }_{j_i}^{\left(i\right)}\in [0,1]$ is the initial state distribution for the i-th HMM at state j; $P^{\left(i\right)}\in {[0,1]}^{m_i\times m_i}$ is the transition matrix for the i-th HMM, and its element $P_{x_{t-1}^{\left(i\right)}{x}_t^{\left(i\right)}}^{\left(i\right)}\in [0,1]$ represents the probability of the i-th HMM being transferred from state $x_{t-1}^{\left(i\right)}$ to state $x_t^{\left(i\right)}$; ${\overline{y}}_t\in {\mathbb{R}}^N$ is the aggregated observation; ${\mu }_{j_i}^{\left(i\right)}\in {\mathbb{R}}^N$ is the mean of the i-th HMM for state $j_i$; and $\Sigma \in {\mathbb{R}}^{N\times N}$ is the covariance matrix. More specifically, Equation (3) can be explicitly written as

$$\begin{array}{c}\hfill Pr\left({\overline{y}}_t\right|x_t)={\displaystyle \frac{1}{\sqrt{{\left(2\pi \right)}^{N}det(\Sigma )}}}exp\left\{-{\displaystyle \frac{1}{2}}{({\overline{y}}_t-{\mu }_t)}^T{\Sigma }^{-1}({\overline{y}}_t-{\mu }_t)\right\},\end{array}$$

(4)

where ${\mu }_t={\sum }_{i=1}^N{\mu }_{x_t^{\left(i\right)}}^{\left(i\right)}$ and $x_t={[x_t^{\left(1\right)},\dots ,x_t^{\left(N\right)}]}^T$. A scheme of the FHMM model is depicted in Figure 1a. Then, the task of the FHMM is to estimate the states of all HMMs, which is illustrated in Figure 1b.

To achieve optimal estimates, the Viterbi algorithm as described in [14] for Hidden Markov Models (HMMs) can be adapted for use with FHMMs, resulting in the EXACT algorithm. Nonetheless, the EXACT algorithm necessitates the enumeration of an exponential number of states, leading to significant computational demands and highlighting the need for approximate algorithms. Even particle approximations via particle filters [15] suffer from the curse of dimensionality in high-dimensional FHMMs. As the dimensionality increases, the state space becomes increasingly sparse, and the number of particles required to accurately represent the posterior distribution grows exponentially. A possible solution can be found in localised particle filters [16] and localised forward–backward algorithms [17], which trade-off accuracy to overcome the curse of dimensionality. As an alternative,  Ref. [13] introduces an approximate approach termed the additive factorial approximate MAP (AFAMAP), designed to globally optimize state estimates within FHMMs. AFAMAP offers three advantages over other approximation techniques: (1) it accounts for the variations in observations between successive time points, (2) it employs robust component accounting to manage unmodeled observations effectively, and (3) it transforms the problem into a convex optimization task with a guaranteed global optimum.

Nevertheless, in numerous instances, the FHMM struggles to detect hidden state variables that exhibit notable patterns but possess small magnitudes. In such scenarios, during specific time intervals, state variables with distinct patterns can be readily identified using alternative methods. In this paper, we define a pattern as significant if it is readily discernible from observations, such as the square-wave electricity consumption of a refrigerator, illustrated in Figure 2a between 1000 and 1200 min. To address these limitations of the FHMM, this paper proposes a feedforward FHMM (FFHMM) that employs predefined states via an automatic feature filter (AFF). The term “feedforward” is derived from feedforward control systems [18], referring to the predefined states in our context. In this study, we only utilize the AFF to extract information from testing data. Nonetheless, other forms of prior knowledge can also be employed to determine the predefined states. Consequently, our FFHMM framework can improve the performance of the FHMM by incorporating various additional pieces of information.

The remainder of this paper is structured as follows. In Section 2, we first introduce an AFF designed to identify significant patterns and treat them as predefined states. Subsequently, we present our key contributions: the direct FFHMM and embedded FFHMM. We also outline the exact and approximate algorithms associated with these models. The efficacy of the FFHMM models is evaluated in Section 3 through rigorous experimentation on two distinct datasets. Lastly, Section 4 summarizes the key points and findings of this paper.

2. Feedforward FHMM

In this section, we first introduce an automatic feature filter (AFF) aimed at extracting notable patterns from the aggregated observations—patterns that are discernible yet not accurately identified by the FHMM. The AFF subsequently facilitates the determination of certain predefined states. We then propose a novel model, the feedforward FHMM (FFHMM), which integrates the FHMM with these predefined states. Depending on the method of integration, we discuss two variations: the direct FFHMM and the embedded FFHMM.

2.1. Automatic Feature Filter

As previously discussed, the FHMM tends to overlook numerous significant patterns present in the observations, which could otherwise be utilized to refine its estimates for individual HMM chains. To address this issue, we have developed the following automatic feature filter:

• If for all $j_1,j_2=1,\dots ,k_i$ such that $j_1\ne j_2$, we have $g_{j_1}^{\left(i\right)}\left(\overline{y}\left(t\right)\right)=g_{j_2}^{\left(i\right)}\left(\overline{y}\left(t\right)\right)$, then

  $$\begin{array}{c}\hfill {\tilde{x}}_t^{\left(i\right)}=g_{j_1}^{\left(i\right)}\left(\overline{y}\left(t\right)\right),\end{array}$$

  (5)
• Otherwise,

  $$\begin{array}{c}\hfill {\tilde{x}}_t^{\left(i\right)}=0,\end{array}$$

  (6)

where ${\tilde{x}}_t^{\left(i\right)}$ is the estimated status from the AFF for the i-th HMM chain at time t. $g_j^{\left(i\right)}\left(\overline{y}\left(t\right)\right)$ is a selection function, whose output is the corresponding state of the i-th HMM chain if its j-th feature has been detected from the aggregated observation at time t. $k_i$ is the number of features we used for detecting the i-th HMM chain.

Figure 3 presents an illustration of the AFF method. Here, we employ green-colored operational windows with specified width w and height h as the features for detection purposes. The filter automatically retrieves the most similar pairs of w and h values. Following the clustering process, the selection function $g_j^{\left(i\right)}\left(\overline{y}\left(t\right)\right)$ is responsible for determining the predefined state at time t.

2.2. Direct FFHMM

Now, we examine two distinct approaches to integrating predefined states with the FHMM: first, a simple concatenation method, followed by a more intricate technique. Alongside these, we also introduce the EXACT algorithms and their derivatives, the AFAMAP algorithms.

Firstly, the straightforward method involves the direct fusion of estimates from the FHMM and AFF. Specifically, this entails substituting the non-zero estimates from the AFF into the corresponding positions within the FHMM. This approach gives rise to what we term the direct FFHMM model:

Model 2. 

During $t\in [1,T]$, every individual HMM chain $i\in \{1,\dots ,N\}$ satisfies

$$\begin{array}{ccc}\hfill {\widehat{x}}_t^{\left(i\right)}& =& \left\{\begin{array}{cc}{\tilde{x}}_t^{\left(i\right)},\hfill & \mathit{if} {\tilde{x}}_t^{\left(i\right)}>0,\hfill \\ x_t^{\left(i\right)},\hfill & \mathit{otherwise},\hfill \end{array}\right.\hfill \end{array}$$

(7)

$$\begin{array}{ccc}\hfill x_1^{\left(i\right)}& \sim & \mathit{Cat}\left({\varphi }^{\left(i\right)}\right),\hfill \end{array}$$

(8)

$$\begin{array}{ccc}\hfill x_t^{\left(i\right)}|x_{t-1}^{\left(i\right)}& \sim & \mathit{Cat}(P_{x_{t-1}^{\left(i\right)}{x}_t^{\left(i\right)}}^{\left(i\right)}),\hfill \end{array}$$

(9)

$$\begin{array}{ccc}\hfill {\overline{y}}_t|x_t^{(1:N)}& \sim & \mathcal{N}\left(\sum _{i=1}^N{\mu }_{x_t^{\left(i\right)}}^{\left(i\right)},\Sigma \right),\hfill \end{array}$$

(10)

where ${\widehat{x}}_t^{\left(i\right)}\in \{1,\dots ,m_i\}$ denotes the output estimation provided by the direct FFHMM for the i-th HMM at time t; ${\tilde{x}}_t^{\left(i\right)}\in \{0,1,\dots ,m_i\}$ represents the output from the AFF for the i-th HMM at time t, with ${\tilde{x}}_t^{\left(i\right)}=0$ indicating that no state has been retrieved by the AFF from the aggregated observation for the i-th HMM at time t; and finally, $x_t^{\left(i\right)}\in \{1,\dots ,m_i\}$ signifies the output of the FHMM as described in Model 1, which corresponds to the optimal estimation for the i-th HMM at time t.

Figure 4 presents a schematic representation of the direct FFHMM approach. According to this figure, one can readily discern that the estimation for the individual HMM, denoted as ${\widehat{x}}_t^{\left(i\right)}$, is chiefly reliant on the FHMM’s corresponding estimate $x_t^{*\left(i\right)}$. Nevertheless, there are instances at particular time points where the AFF yields non-zero estimations; in such cases, these AFF-derived estimates supersede those originating from the FHMM. Then, an EXACT algorithm for the direct FFHMM is proposed in Algorithm 1.

Algorithm 1: EXACT algorithm for direct FFHMM
1	Initialization: $$\begin{array}{c}\hfill {\delta }_1\left(j\right)=\left(\sum _{i=1}^N{\varphi }_{j_i}^{\left(i\right)}\right)Pr\left({\overline{y}}_1\right|x_1\left(j\right)),\end{array}$$ (11) where $j={[j_1,\dots ,j_N]}^T$, $j_i\in \{1,\dots ,m_i\}$, $1⩽i⩽N$, $x_1\left(j\right)={[x_1^{\left(1\right)}=j_1,\dots ,x_1^{\left(N\right)}=j_N]}^T$.
2	Recursion: $$\begin{array}{c}\hfill {\delta }_t\left(j\right)=\underset{k}{max}\left(\sum _{i=1}^N{\delta }_{t-1}\left(k\right)P_{k_i{j}_i}^{\left(i\right)}\right)Pr\left({\overline{y}}_t\right|x_t\left(j\right)),\end{array}$$ (12) $$\begin{array}{c}\hfill {\psi }_t\left(j\right)=arg\underset{k}{max}\left(\sum _{i=1}^N{\delta }_{t-1}\left(k\right)P_{k_i{j}_i}^{\left(i\right)}\right),\end{array}$$ (13) where $2⩽t⩽T$, $k={[k_1,\dots ,k_N]}^T$, $k_i\in \{1,\dots ,m_i\}$, $1⩽i⩽N$, $x_t\left(j\right)={[x_t^{\left(1\right)}=j_1,\dots ,x_t^{\left(N\right)}=j_N]}^T$.
3	Termination: $$\begin{array}{ccc}\hfill q_T^{*}& =& arg\underset{k}{max}\left(\sum _{i=1}^N{\delta }_{t-1}\left(k\right)P_{k_i{j}_i}^{\left(i\right)}\right).\hfill \end{array}$$ (14)
4	Path backtracking: $$\begin{array}{c}\hfill q_t^{*}={\psi }_{t+1}\left(q_{t+1}^{*}\right),t=T-1,\dots ,1.\end{array}$$ (15)
5	Merging: $${\widehat{q}}_t^{\left(i\right)}=\left\{\begin{array}{cc}{\tilde{q}}_t^{\left(i\right)},\hfill & \mathrm{if} {\tilde{q}}_t^{\left(i\right)}>0,\hfill \\ q_t^{*\left(i\right)},\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$ (16)

In Algorithm 1, ${\widehat{q}}_t^{\left(i\right)}\in \{1,\dots ,m_i\}$ is the index of the estimation of the FFHMM for the i-th HMM at time t; ${\tilde{q}}_t^{\left(i\right)}\in \{0,1,\dots ,m_i\}$ is the index of the estimation of the AFF for the i-th HMM at time t, where ${\tilde{q}}_t^{\left(i\right)}=0$ means no state has been retrieved by the AFF from the aggregated observation for the i-th HMM at time t; and $q_t^{*\left(i\right)}\in \{1,\dots ,m_i\}$ is the index of the optimal estimation for the i-th HMM at time t by the FHMM.

Evidently, Algorithm 1 employs the Viterbi algorithm within the proposed direct FFHMM. In this algorithm, all possible states are traced through the application of the dynamic programming algorithm approach. However, this method necessitates the enumeration of an exponentially large number of states, which in turn gives rise to substantial computational demands. In light of these computational challenges, we introduce an approximation, specifically Algorithm 2, with the aim of mitigating the heavy computational burden imposed by Algorithm 1.

Algorithm 2: AFAMAP algorithm for direct FFHMM
1	Input: ${\tilde{x}}_{1:T}^{(1:N)}$, ${\overline{y}}_{1:T}$, ${\mu }_{1:m_i}^{(1:N)}$, ${\Sigma }_1$, ${\Sigma }_2$, ${\lambda }_1$ and ${\lambda }_2$.
2	Minimization: $$\begin{array}{c}\hfill \underset{\mathcal{S}\in \mathcal{L}\cap \mathcal{O},z_{1:T}}{min}=f(\mathcal{S},z_{1:T})\end{array}$$ (17)
3	Output: ${\widehat{y}}_{1:T}^{(1:N)}$, predicted the output of individual HMM $${\widehat{y}}_t^{\left(i\right)}=\left\{\begin{array}{cc}{\mu }_{{\tilde{x}}_t^{\left(i\right)}}^{\left(i\right)},\hfill & \mathrm{if} {\tilde{x}}_t^{\left(i\right)}>0,\hfill \\ \sum _{l=1}^{m_i}{\mu }_l^{\left(i\right)}Q{\left(x_t^{*\left(i\right)}\right)}_l,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$ (18)

In Algorithm 2, ${\overline{y}}_{1:T}$ is the aggregated observation; ${\mu }_{1:m_i}^{(1:N)}$ represents state means for N HMMs; ${\Sigma }_1$ and ${\Sigma }_2$ are the covariance matrices for the additive FHMM and difference FHMM, respectively; and ${\lambda }_1$ and ${\lambda }_2$ are regularization parameters. The cost function f in (17) is defined as

$$\begin{array}{c}f(\mathcal{S},z_{1:T})={\displaystyle \frac{1}{2}}\sum _t\left|\right|{\overline{y}}_t-{\Sigma }^{{\displaystyle \frac{1}{2}}}{z}_t-\sum _{i,j}{\mu }_j^{\left(i\right)}Q{\left(x_t^{\left(i\right)}\right)}_j|{|}_{{\Sigma }^{-1}}^2\hfill \\ +{\displaystyle \frac{1}{2}}\sum _{t,i,j,k\ne j}\left|\right|\Delta {\overline{y}}_t-\Delta {\mu }_{k,j}^{\left(i\right)}|{|}_{{\Sigma }^{-1}}^{2}Q{(x_{t-1}^{\left(i\right)},x_t^{\left(i\right)})}_{j,k}\hfill \\ +{\displaystyle \frac{1}{2}}\sum _{t}D({\Sigma }^{-{\displaystyle \frac{1}{2}}}\Delta {\overline{y}}_t,{\lambda }_2)\left(1-\sum _{i,j}Q{(x_{t-1}^{\left(i\right)},x_t^{\left(i\right)})}_{j,j}\right)\hfill \\ +\sum _{t,i,j,k}Q{(x_{t-1}^{\left(i\right)},x_t^{\left(i\right)})}_{j,k}(-log{P}_{k,j}^{\left(i\right)})\hfill \\ +{\lambda }_1\sum _t{\Vert z_t-z_{t-1}\Vert }_1,\hfill \end{array}$$

(19)

where $\Delta {\overline{y}}_t$ and $\Delta {\mu }_{j,k}^{\left(i\right)}$ represents the changes in observations and states, given by

$$\begin{array}{cc}\hfill \Delta {\overline{y}}_t|x_t^{(1:N)},x_{t-1}^{(1:N)}& \sim \mathcal{N}\left(\sum _{i=1}^N\left({\mu }_{x_t^{\left(i\right)}}-{\mu }_{x_{t-1}^{\left(i\right)}}\right),\Sigma \right),\hfill \end{array}$$

(20)

and

$$\begin{array}{c}\hfill \Delta {\mu }_{j,k}^{\left(i\right)}={\mu }_j^{\left(i\right)}-{\mu }_k^{\left(i\right)}.\end{array}$$

(21)

Meanwhile, the set $\mathcal{S}$ of indicators is defined as

$$\begin{array}{c}\hfill \mathcal{S}=\bigcup _{\begin{array}{c}i\in \{1,\dots ,N\}\\ t\in \{1,\dots ,T\}\end{array}}\left(\left\{Q\left(x_t^{\left(i\right)}\right)\in {\{0,1\}}^{m_i}\right\}\cup \left\{Q(x_{t-1}^{\left(i\right)},x_t^{\left(i\right)})\in {\{0,1\}}^{m_i\times m_i}\right\}\right)\end{array}$$

(22)

in which $Q{\left(x_t^{\left(i\right)}\right)}_j=1\iff x_t^{\left(i\right)}=j$, $Q{(x_{t-1}^{\left(i\right)},x_t^{\left(i\right)})}_{j,k}=1\iff x_{t-1}^{\left(i\right)}=j,x_t^{\left(i\right)}=k$, and $t\in \{1,\dots ,T\}$; $i\in \{1,\dots ,N\}$; $j,k\in \{1,\dots ,m_i\}$. The minimization constraints are

$$\begin{array}{c}\hfill \mathcal{L}=\left\{\mathcal{S}:\begin{array}{c}{\displaystyle \sum _j}Q{\left(x_t^{\left(i\right)}\right)}_j=1\hfill \\ {\displaystyle \sum _k}Q{(x_{t-1}^{\left(i\right)},x_t^{\left(i\right)})}_{j,k}=Q{\left(x_{t-1}^{\left(i\right)}\right)}_j\hfill \\ {\displaystyle \sum _k}Q{(x_{t-1}^{\left(i\right)},x_t^{\left(i\right)})}_{k,j}=Q{\left(x_t^{\left(i\right)}\right)}_j\hfill \\ Q{\left(x_t^{\left(i\right)}\right)}_j,Q{(x_{t-1}^{\left(i\right)},x_t^{\left(i\right)})}_{k,j}⩾0\hfill \end{array}\right\}.\end{array}$$

(23)

In addition, the one-at-a-time condition leads to an extra constraint as

$$\begin{array}{c}\hfill \mathcal{O}=\left\{\mathcal{S}:\sum _{i,j,k\ne j}Q{(x_{t-1}^{\left(i\right)},x_t^{\left(i\right)})}_{j,k}⩽1\right\},\end{array}$$

(24)

which means the off-diagonal elements of $Q(x_{t-1}^{(i:N)},x_t^{(i:N)})$ sum to at most one. The last term $\Vert z_t-z_{t-1}{\Vert }_1$ in the cost function is the robust component. It is used to handle unmodeled observations.

Clearly, the direct FFHMM possesses two notable advantages:

• Firstly, it simplifies the control over outcomes for a designated HMM chain. Specifically, as the AFF’s estimations directly supersede those of the FHMM during particular timeframes, the accuracy of the FFHMM for the target HMM chain is unlikely to decline, assuming the AFF’s accuracy is maintained with sufficient conservatism. Secondly, an aggressive AFF can be employed to enhance the performance of the target HMM chain, though this approach carries a risk of potentially exacerbating outcomes.
• Secondly, alterations in accuracy for the estimation of one HMM chain are independent of changes in others. This attribute facilitates the independent improvement in accuracy for each HMM chain. Additionally, customized AFFs can be applied to specific HMM chains, thereby achieving superior results.

2.3. Embedded FFHMM

In this subsection, we employ an alternative method to integrate predefined states with the FHMM, whereby the non-zero estimations from the AFF take the place of the hidden states within the embedded HMMs of the FHMM model. The structure of the embedded FFHMM can be described as follows:

Model 3. 

1. 

The initial state:

(a) 

If ${\tilde{x}}_1^{\left(i\right)}>0$, then

$$\begin{array}{c}\hfill x_1^{\left(i\right)}={\tilde{x}}_1^{\left(i\right)},\end{array}$$

(25)

(b) 

Otherwise,

$$\begin{array}{c}\hfill x_1^{\left(i\right)}\sim \mathit{Cat}\left({\varphi }^{\left(i\right)}\right)\end{array}$$

(26)

2. 

The state transition:

(a) 

If ${\tilde{x}}_{t-1}^{\left(i\right)}>0$, then

$$\begin{array}{c}\hfill x_t^{\left(i\right)}={\tilde{x}}_t^{\left(i\right)},\end{array}$$

(27)

(b) 

Otherwise,

$$\begin{array}{c}\hfill x_t^{\left(i\right)}|x_{t-1}^{\left(i\right)}\sim \mathit{Cat}(P_{x_{t-1}^{\left(i\right)}{x}_t^{\left(i\right)}}^{\left(i\right)})\end{array}$$

(28)

3. 

The observation probability distribution:

$$\begin{array}{c}\hfill {\overline{y}}_t|x_t^{(1:N)}\sim \mathcal{N}\left(\sum _{i=1}^N{\mu }_{x_t^{\left(i\right)}}^{\left(i\right)},\Sigma \right),\end{array}$$

(29)

The notations used in this model can be understood in reference to the preceding discussions. Figure 5 presents a diagram of the embedded FFHMM, where a hidden state $x_t^{\left(i\right)}$ from the FHMM is substituted by the non-zero estimation ${\tilde{x}}_t^i$ from the AFF for the i-th HMM at time t. The FFHMM’s output ${\widehat{x}}_t^{\left(i\right)}$ is the optimal estimation, determined by the observation probability distribution, as described in (29). Subsequently, we introduce the EXACT and AFAMAP algorithms designed for the embedded FFHMM, as outlined in Algorithms 3 and Algorithm 4, respectively.

Algorithm 3: EXACT algorithm for embedded FFHMM
1	Initialization: $j={[j_1,\dots ,j_N]}^T$, $1⩽i⩽N$, if ${\tilde{x}}_1^{\left(i\right)}>0$, then $$\begin{array}{c}\hfill j_i={\tilde{x}}_1^{\left(i\right)},\end{array}$$ (30) otherwise, $$\begin{array}{c}\hfill j_i\in \{1,\dots ,m_i\}.\end{array}$$ (31) Then, we initialize ${\delta }_1\left(j\right)$ by (11), which is the same as Algorithm 1,
2	Recursion: $2⩽t⩽T$, $k={[k_1,\dots ,k_N]}^T$, $k_i\in \{1,\dots ,m_i\}$, $1⩽i⩽N$, if ${\tilde{x}}_t^{\left(i\right)}>0$, then $$\begin{array}{c}\hfill j_i={\tilde{x}}_t^{\left(i\right)},\end{array}$$ (32) $$\begin{array}{c}\hfill P_{k_i{j}_i}^{\left(i\right)}=1,\end{array}$$ (33) if ${\tilde{x}}_t^{\left(i\right)}=0$ but ${\tilde{x}}_{t-1}^{\left(i\right)}>0$, then $$\begin{array}{c}\hfill k_i={\tilde{x}}_{t-1}^{\left(i\right)},\end{array}$$ (34) otherwise, $$\begin{array}{c}\hfill j_i\in \{1,\dots ,m_i\},k_i\in \{1,\dots ,m_i\}.\end{array}$$ Then, we recur ${\delta }_t\left(j\right)$ and ${\psi }_t\left(j\right)$ using (12) and (13), which is the same as Algorithm 1.
3	Termination: $$\begin{array}{ccc}\hfill q_T^{*}& =& arg\underset{k}{max}\left(\sum _{i=1}^N{\delta }_{t-1}\left(k\right)P_{k_i{j}_i}^{\left(i\right)}\right).\hfill \end{array}$$ (35)
4	Path backtracking: $$q_t^{*}={\psi }_{t+1}\left(q_{t+1}^{*}\right),t=T-1,\dots ,1.$$ (36)

In Algorithm 4, constraint $\widehat{\mathcal{L}}$ is defined as

$$\begin{array}{c}\hfill \widehat{\mathcal{L}}=\left\{\mathcal{S}:\begin{array}{c}Q{\left(x_t^{\left(i\right)}\right)}_{{\tilde{x}}_t^{\left(i\right)}}=1 \mathrm{for} \mathrm{all} {\tilde{x}}_t^{\left(i\right)}>0\hfill \\ {\displaystyle \sum _j}Q{\left(x_t^{\left(i\right)}\right)}_j=1\hfill \\ {\displaystyle \sum _k}Q{(x_{t-1}^{\left(i\right)},x_t^{\left(i\right)})}_{j,k}=Q{\left(x_{t-1}^{\left(i\right)}\right)}_j\hfill \\ {\displaystyle \sum _k}Q{(x_{t-1}^{\left(i\right)},x_t^{\left(i\right)})}_{k,j}=Q{\left(x_t^{\left(i\right)}\right)}_j\hfill \\ Q{\left(x_t^{\left(i\right)}\right)}_j,Q{(x_{t-1}^{\left(i\right)},x_t^{\left(i\right)})}_{k,j}⩾0\hfill \end{array}\right\}.\end{array}$$

(37)

Algorithm 4: AFAMAP algorithm for embedded FFHMM
1	Input: ${\tilde{x}}_{1:T}^{(1:N)}$, ${\overline{y}}_{1:T}$, ${\mu }_{1:m_i}^{(1:N)}$, ${\Sigma }_1$, ${\Sigma }_2$, ${\lambda }_1$ and ${\lambda }_2$.
2	Minimization: $$\begin{array}{c}\hfill \underset{\mathcal{S}\in \widehat{\mathcal{L}}\cap \mathcal{O},z_{1:T}}{min}=f(\mathcal{S},z_{1:T})\end{array}$$ (38) Output: $y_{1:T}^{*(1:N)}$, predicted individual HMM output $$y_t^{*\left(i\right)}=\sum _{l=1}^{m_i}{\mu }_l^{\left(i\right)}Q{\left(x_t^{\left(i\right)}\right)}_l$$ (39)

As previously discussed in the earlier subsection, the estimations of the direct FFHMM for the target HMM chain are independent of other HMM chains. In contrast, the estimations of the embedded FFHMM for the target HMM chain are interconnected with other HMM chains via the observation probability distribution described in (29). Due to these interdependencies, enhancing the accuracy of the AFF for one HMM can indirectly influence the estimation accuracies of other HMMs. This represents a significant advantage of the embedded FFHMM approach. However, this also necessitates greater care in managing the embedded FFHMM, as these correlations can lead to unforeseen changes in the estimations of other HMM chains.

In summary, given their distinct characteristics, the following guidelines can be followed when selecting between the direct FFHMM and embedded FFHMM: Firstly, opt for the direct FFHMM if the goal is to independently improve the estimation of a single HMM chain. Secondly, choose the embedded FFHMM if the aim is to enhance not only the estimation of an individual HMM chain but also the estimations across multiple HMM chains simultaneously.

The computational complexity of the Viterbi algorithm applied to the FHMM is

$$\begin{array}{c}\hfill O\left({\left(\prod _{i=1}^N{m}_i\right)}^{2}T\right),\end{array}$$

(40)

where N is the number of HMMs, $m_i$ is the number of states in the i-th HMM, and T is the number of time steps. This exponential complexity makes the Viterbi algorithm intractable for large FHMMs. When contrasted with the exponential complexity, the cost associated with implementing the AFF is so minimal that it can be considered negligible. As a result, the computational complexities of the EXACT algorithms in Algorithms 1 and 3 are identical to those expressed in Equation (40), both of which exhibit an exponential-in-dimension computational cost.

Similarly, as per the findings in [13], the computational complexity of AFAMAP for the additive FHMM is approximately linear with respect to the number of HMMs. Consequently, the AFAMAP algorithms for the FFHMM presented in Algorithms 2 and 4 share the same computational complexity.

3. Results

In this section, we implement our models and algorithms to address the non-intrusive load monitoring (NILM) problem, showcasing their advantages over the traditional FHMM. The evaluations are conducted using the Pecan Street dataset [19] and the REDD [20] dataset.

The Pecan dataset comprises daily electricity consumption data from households located in the green-built Mueller community in Austin, Texas. This dataset encompasses both aggregated and appliance-level consumption data, collected at a sampling rate of $1/60$ Hz, meaning one sample is recorded per minute. Specifically, the Pecan dataset used in our experiment contains energy consumption data from 10 households over a one-week period, with one sample recorded every minute. The entire dataset consists of $60\min /\mathrm{h}\times 24\mathrm{h}/\mathrm{day}\times 7\mathrm{days}=\mathrm{10,080}$ sampled points. The first half of these sampled points, i.e., 5040 points, are employed for training, while the remaining 5040 points are reserved for testing purposes.

Meanwhile, the power consumption data of house 1 from the REDD dataset are also utilized to validate our algorithms. The sampling rate of these data is $1/3$ Hz. We selected house 1 because it contains the most comprehensive information and features 18 devices [21].

3.1. Automatic Pattern Recognition Using AFF

In this subsection, we employ the AFF to identify similar patterns of refrigerator usage from aggregated observations. The effectiveness of the AFF is assessed using the metrics “Rate of Detection” and “precision”, which are defined as follows:

$$\begin{array}{c}\hfill \mathrm{Rate} \mathrm{of} \mathrm{Detection}={\displaystyle \frac{tp+fp}{tp+fp+fn}},\end{array}$$

(41)

$$\begin{array}{c}\hfill \mathrm{Precision}={\displaystyle \frac{tp}{tp+fp}},\end{array}$$

(42)

where $tp$, $fp$, and $fn$ denote “true positive”, “false positive”, and “false negative”, respectively. These terms represent correct identifications, erroneous detections, and missed identifications. It is important to note that our definition of “Rate of Detection” differs from the conventional definition of “recall”, as outlined in [22]:

$$\begin{array}{c}\hfill \mathrm{Recall}={\displaystyle \frac{tp}{tp+fn}}.\end{array}$$

(43)

In the definition of “Rate of Detection”, we use “tp+fp+fn” to signify the total number of time points under evaluation, and “tp+fp” to denote the number of time points that have been identified by the AFF.

We then apply the AFF to the Pecan dataset, with the results presented in

Table 1. Performance of AFF on Pecan dataset.

No. of Home	Rate of Detection	Precision
1	30.52%	76.01%
2	27.54%	86.02%
3	17.94%	98.78%
4	20.87%	75.29%
5	37.92%	95.36%
6	36.09%	93.95%
7	26.81%	79.94%
8	36.85%	88.53%
9	36.79%	85.22%
10	17.92%	74.97%

, evaluated based on the metrics of “Rate of Detection” and “precision”. These findings are derived from a moderate configuration of the AFF parameters. As discussed in the previous section, relaxing the variances of the filter window’s width w and height h can enhance the “Rate of Detection” but may lead to a decrease in “precision”. Conversely, tightening these variances can boost “precision” at the cost of a reduced “Rate of Detection”.

Assuming the refrigerator operates in two states, “turn on” (cooling) or “turn off” (standby), the AFF’s task is to identify the time periods during which these states can be classified as “on” or “off” by the AFF. To illustrate the detailed outcomes of our AFF, Figure 6 displays the estimated operational status of the refrigerator in the Pecan dataset. In this figure, higher red lines represent estimations of the “turn on” state, while lower red lines indicate the “turn off” state. When comparing these estimated states with the aggregated observations (depicted by the blue lines), we observe that the AFF has successfully captured most of the significant patterns of the refrigerator’s operation.

In the subsequent subsection, the retrieved statuses will serve as the predefined states within the models of both the direct FFHMM and the embedded FFHMM.

3.2. Experiments Using Direct FFHMM and Embedded FFHMM

In order to assess the performance of various models and algorithms, we utilize the metric known as “total energy correctly assigned” [20], which is defined as follows:

$$\begin{array}{c}\hfill \mathrm{acc}=1-{\displaystyle \frac{{\sum }_{t=1}^T{\sum }_{i=1}^n|{\widehat{y}}_t^{\left(i\right)}-y_t^{\left(i\right)}|}{2{\sum }_{t=1}^T{\overline{y}}_t}},\end{array}$$

(44)

where ${\overline{y}}_t$ represents the observed total power consumption at time t, while $y_t^{\left(i\right)}$ denotes the actual power consumed by the i-th appliance at time t, with its corresponding estimate being ${\widehat{y}}_t^{\left(i\right)}$. In our experimental setup, we apply both the EXACT and AFAMAP algorithms to both the direct FFHMM model and the embedded FFHMM model.

Based on the design presented in Figure 4, the direct FFHMM model can enhance—and is specifically designed to enhance—the performance of appliances with predefined states. In this experiment, the refrigerator serves as the target appliance. When compared to the FHMM, the direct FFHMM shows this enhancement. Therefore, we only need to compare the performances of the direct FFHMM and FHMM on the refrigerator, as illustrated in Figure 7. It is evident that, regardless of whether the EXACT algorithm or the AFAMAP algorithm is employed, the direct FFHMM invariably attains a higher level of accuracy compared to the FHMM. This is attributable to the fact that the AFF within the direct FFHMM model takes into account additional information that the FHMM overlooks.

When using the EXACT algorithm, the mean of the difference in accuracy between the direct FFHMM and FHMM is $0.0788$, with a standard deviation of $0.0246$. The calculated T-statistic is $10.1158$. These statistical results clearly indicate that there is a highly significant difference in the performance of these two models. Additionally, the $95\%$ confidence interval for this difference is $[0.0612,0.0964]$, further validating the advantage of the proposed direct FFHMM over the FHMM.

When the AFAMAP algorithm is utilized, the mean of the difference in accuracy between the direct FFHMM and the FHMM amounts to $0.0759$. The standard deviation associated with this difference is $0.0407$. A T-statistic of $5.8959$ is computed. Such a T-statistic value strongly suggests that there exists a statistically significant difference in the performance of the two models. Moreover, the $95\%$ confidence interval for this performance difference is precisely determined to be in the range of $[0.0468,0.1050]$, which further bolsters the conclusion that the performance of the direct FFHMM is better than that of the FHMM.

The estimation results by the EXACT and AFAMAP algorithms for the Pecan dataset using the embedded FFHMM model are presented in

Table 2. Accuracy of energy disaggregation on Pecan dataset employing the FHMM and embedded FFHMM models with the EXACT and the AFAMAP algorithms.

Homes	Appliances	Ratio	FHMM		Embedded FFHMM
			AFAMAP	EXACT	AFAMAP	EXACT
1	Refrigerator	5.31%	0.6262	0.6336	0.6749	0.7012
	Air conditioner 1	70.99%	0.8417	0.9476	0.9387	0.9475
	Furnace	13.51%	0.7580	0.7372	0.7645	0.7542
2	Refrigerator	3.42%	0.5315	0.5429	0.6344	0.5864
	Air conditioner 1	50.35%	0.8096	0.9208	0.8106	0.9158
	Furnace 1	27.29%	0.9694	0.9589	0.9698	0.9608
	Electrical dryer 1	6.74%	0.5251	0.4955	0.5243	0.4008
	Living room 1	5.47%	0.2959	0.6970	0.3094	0.6491
3	Refrigerator	8.84%	0.5511	0.6706	0.6003	0.6831
	Air conditioner 1	45.60%	0.8923	0.8975	0.8941	0.8988
	Furnace	21.22%	0.8298	0.8360	0.8415	0.8368
	Electrical Car	17.30%	0.7065	0.8152	0.6994	0.8152
4	Refrigerator	4.78%	0.5986	0.5304	0.6048	0.5716
	Air conditioner 1	63.63%	0.9437	0.9432	0.9436	0.9433
	Furnace	22.10%	0.7089	0.8083	0.6996	0.8102
5	Refrigerator	4.85%	0.9065	0.8507	0.9309	0.9313
	Air conditioner 1	49.13%	0.8355	0.9292	0.8363	0.9287
	Furnace	7.79%	0.7169	0.7134	0.7171	0.7297
	Electrical car	34.30%	0.8564	0.9568	0.8548	0.9591
6	Refrigerator	5.16%	0.5201	0.5992	0.6163	0.6963
	Air conditioner 1	69.27%	0.8198	0.9237	0.8150	0.9213
	Air conditioner 2	13.67%	0.5384	0.8579	0.5205	0.8526
7	Refrigerator	4.15%	0.5774	0.5524	0.6645	0.6638
	Air conditioner 1	57.32%	0.8869	0.8978	0.8883	0.9003
	Furnace	16.10%	0.8183	0.8454	0.8221	0.8451
	Electrical dryer 1	8.23%	0.7771	0.8225	0.7780	0.8219
	Family room 1	7.34%	0.7301	0.7647	0.7208	0.7626
8	Refrigerator	2.39%	0.5795	0.5510	0.6065	0.6109
	Air conditioner 1	40.61%	0.8129	0.8740	0.8156	0.8622
	Air conditioner 2	18.02%	0.6402	0.7609	0.6500	0.7084
	Furnace	17.89%	0.8168	0.8789	0.8171	0.8662
	Electrical dryer 1	5.87%	0.7677	0.8248	0.7697	0.8173
	Living room 1	7.03%	0.6510	0.6463	0.6438	0.6812
9	Refrigerator	4.33%	0.6193	0.6282	0.6839	0.6992
	Air conditioner 1	59.86%	0.9437	0.9458	0.9422	0.9437
	Furnace	15.65%	0.7921	0.8027	0.7928	0.8168
	Theater	13.50%	0.6693	0.8222	0.6707	0.8580
10	Refrigerator	4.03%	0.4136	0.4661	0.4953	0.5845
	Air conditioner 1	66.92%	0.9281	0.9514	0.9327	0.9502
	Furnace	15.75%	0.7910	0.8173	0.8080	0.8266
	Living room 1	5.08%	0.6525	0.6541	0.6461	0.7529

. The third column, labeled as “Ratio”, indicates the proportion of the cumulative consumption of each appliance relative to the total consumption of the entire household.

The bolded figures in Table 2 emphasize the best accuracies, which were all obtained by either the EXACT or AFAMAP algorithm. It is evident that the embedded FFHMM model has the potential to boost the performance of several additional appliances, albeit it may also lead to a decrease in performance for some.

These findings substantiate the assertions made in the preceding section, indicating that improvements for appliances are independent in the direct FFHMM model but become interdependent in the embedded model.

For ease of comparison, Figure 8 illustrates the enhancements in disaggregation accuracy for the refrigerator after substituting the FFHMM models for the FHMM model. Within the figure, the term “Improvement of AFAMAP by direct FFHMM” denotes the disparity between the accuracy achieved by the AFAMAP algorithm when applied to the direct FFHMM model and that of the FHMM model. A positive difference indicates that the accuracy derived from the FFHMM-based algorithm surpasses that of the FHMM-based algorithm, while a negative difference signifies the converse. The remaining three lines are derived in an analogous manner. Consequently, as all lines in the figure are above zero, it can be inferred that the FFHMM models yield superior disaggregation results across all households compared to the FHMM models.

Figure 9 presents a detailed disaggregation analysis of the refrigerator. Upon examining the figure, it becomes evident that both the EXACT and AFAMAP algorithms significantly enhance the disaggregation results when applied to either the direct FFHMM or embedded FFHMM models. For instance, between 3600 and 4000 min, the operational status of the refrigerator is accurately estimated by the algorithms employing FFHMM models. In contrast, both the EXACT and AFAMAP algorithms based on the FHMM model fail to detect such significant patterns.

Meanwhile, we apply FFHMM models to the REDD dataset. To demonstrate that the efficacy of our models is not confined to a single metric, we employ “precision”, “recall”, and “F-measure” to evaluate the performance of FFHMM models on the REDD dataset. The F-measure is defined as per [22]:

$$\begin{array}{c}\hfill F=2·{\displaystyle \frac{\mathrm{precision}·\mathrm{recall}}{\mathrm{precision}+\mathrm{recall}}}\end{array}$$

(45)

where the precision and recall are defined in (42) and (43), respectively.

On the REDD dataset, when the direct FFHMM model replaces the FHMM model, the enhancements in precision, recall, and F-measure of the AFAMAP algorithm for the refrigerator are $0.0149$, $0.2668$, and $0.2020$ respectively. Meanwhile, the findings for the embedded FFHMM are presented in

Table 3. REDD dataset using the FHMM and embedded FFHMM models with the AFAMAP algorithm.

Appliances	No. of Status	FHMM			Embedded FFHMM
		Precision	Recall	F-Measure	Precision	Recall	F-Measure
refrigerator	2	0.9745	0.5043	0.6647	0.9839	0.7743	0.8666
dishwasher	5	0.9871	0.8798	0.9304	0.9873	0.8840	0.9328
microwave	2	0.9860	0.6365	0.7736	0.9983	0.6875	0.8143
bathroomfi	2	0.9838	0.9998	0.9917	0.9921	0.9999	0.9960
kOutlet 2	2	0.9906	0.3008	0.4615	0.9945	0.3557	0.5240
kOutlets 3	3	0.4546	0.6933	0.5491	0.4780	0.6782	0.5607
light 3	3	0.7103	0.3944	0.5072	0.7803	0.3826	0.5135
light 1	2	0.9997	0.1455	0.2541	0.9994	0.1684	0.2883
light 2	2	0.9689	0.2451	0.3913	0.9368	0.3340	0.4924
washdryer 2	2	1.0000	0.9538	0.9764	0.9999	0.9816	0.9907
washdryer 1	3	0.9735	0.7783	0.8650	0.9814	0.7477	0.8488
kOutlet 1	2	0.9966	0.7788	0.8743	0.9952	0.7226	0.8373
oven 1& 2	2	1.0000	0.9728	0.9862	0.9925	0.8104	0.8923

. Further detailed illustrations about the direct FFHMM and embedded FFHMM are provided in Figure 10. These outcomes bear resemblance to those obtained from experiments conducted on the Pecan dataset, thereby validating the effectiveness of the proposed FFHMM models.

In this subsection, the proposed direct FFHMM and embedded FFHMM are demonstrated by their application to NILM. However, it is evident that these two models are not limited to NILM. As stated in the introduction, similar to the FHMM model, they can also be applied to a wide range of other domains. These include consumers’ choice models, estimating tumor clonal composition, fault diagnosis, online skill rating, and forecasting financial portfolios with value at risk.

4. Discussion

In order to leverage the information embedded within the aggregated observations, which is often significant yet overlooked by the FHMM, we employ an empirical automatic feature filter (AFF) to extract pertinent patterns designated as predefined states. The AFF utilized in this study is sufficient to showcase the efficacy of both the direct FFHMM and embedded FFHMM; however, more specialized approaches could be explored in future research to refine the identification of these predefined states with enhanced precision. Subsequently, we introduce the FFHMM model, which integrates the FHMM with these predefined states. The FFHMM can be classified into two distinct variants: the direct FFHMM and embedded FFHMM. In the former, the predefined states directly substitute the estimated outcomes of the FHMM, allowing for the independent enhancement of each individual HMM chain’s estimation. Conversely, in the embedded FFHMM, the hidden state variables of each HMM chain are replaced by the predefined states, which, in conjunction with the states of other HMM chains, determine the final outcome of the embedded FFHMM. As a result, within the embedded FFHMM framework, the estimation of multiple HMM chains can potentially be improved by merely defining some states for a single HMM chain. To validate the effectiveness of the FFHMM models, we conducted experiments on two real-world datasets, employing both the EXACT and AFAMAP algorithms.