A Simplified Matrix Formulation for Sensitivity Analysis of Hidden Markov Models

Abstract

In this paper, a new algorithm for sensitivity analysis of discrete hidden Markov models (HMMs) is proposed. Sensitivity analysis is a general technique for investigating the robustness of the output of a system model. Sensitivity analysis of probabilistic networks has recently been studied extensively. This has resulted in the development of mathematical relations between a parameter and an output probability of interest and also methods for establishing the effects of parameter variations on decisions. Sensitivity analysis in HMMs has usually been performed by taking small perturbations in parameter values and re-computing the output probability of interest. As recent studies show, the sensitivity analysis of an HMM can be performed using a functional relationship that describes how an output probability varies as the network’s parameters of interest change. To derive this sensitivity function, existing Bayesian network algorithms have been employed for HMMs. These algorithms are computationally inefficient as the length of the observation sequence and the number of parameters increases. In this study, a simplified efficient matrix-based algorithm for computing the coefficients of the sensitivity function for all hidden states and all time steps is proposed and an example is presented.

1. Introduction

A Hidden Markov Model (HMM) is a stochastic model of the dynamic process of two related random processes that evolve over time. A set of stochastic processes that produces the sequence of observed symbols is used to infer an underlying stochastic process that is not observable (hidden states). HMMs have been widely utilized in many application areas including speech recognition [1], bioinformatics [2], finance [3], computer vision [4], and driver behavior modeling [5,6]. A comprehensive survey on the applications of HMMs is presented in [7]. A simple dynamic Bayesian network can be used to represent a stationary HMM with discrete variables [8,9]. Consequently, algorithms and theoretical results developed for dynamic Bayesian networks can be applied to HMMs as well.

Sensitivity analysis in HMMs has been done by perturbation analysis in which a brute-force computation of the effect of small changes on the output probability of interest is done [10,11]. Sensitivity analysis research has shown that a functional relationship can be established between a parameter variation and the output probability of interest. This function can represent the effect of any change in the parameter under consideration as compared to the perturbation analysis. Consequently, the goal of sensitivity analysis becomes developing techniques to create this function. This general function will also enable us to compute bounds on the output probabilities without sensitivity analysis [12,13,14,15].

The objective of this work is to develop an algorithm for sensitivity analysis in HMMs directly from the model’s representation. Our focus is on parameter sensitivity analysis where the variation of the output is studied as the model parameters vary. In this case, the parameters are the (conditional) probabilities of the HMM model (initial, transition, and observation parameters) and the output is some posterior probability of interest (filtering, smoothing, predicted future observations, and most probable path).

Parametric sensitivity analysis can be used for multiple purposes. The sensitivity analysis can be used to identify those parameters which have significant impact on the output probability of interest. Consequently, the result can be used as a basis for parameter tuning and studying the robustness of the model output to changes in the parameters [16,17,18].

Many researchers study the problem of sensitivity analysis in Bayesian Networks and HMMs. There are two approaches used to compute the constants of the sensitivity function in the standard Bayesian network inference algorithms. The first method is solving systems of linear equations [19] by evaluating the sensitivity function for different values of the varying parameter $\theta$. The second is based on a differential approach [20] by which the coefficients of a multivariate sensitivity function can be computed from partial derivatives. The tailored version of the junction tree algorithm has been used to establish the sensitivity functions in Bayesian networks more efficiently [21,22]. In [23], sensitivity analysis of Bayesian networks for multiple parameter changes is presented. In [24], credal sets that are known to represent the probabilities of interest are used for sensitivity analysis of Markov chains. In [10,11], sensitivity analysis of HMMs based on small perturbations in the parameter values is presented. A tight perturbation bound is derived and it is shown that the distribution of the HMM tends to be more sensitive to perturbations in the emission probabilities than to the transition and initial distributions. A tailored approach for HMM sensitivity analysis, the Coefficient-Matrix-Fill procedure, is presented in [25,26]. It directly utilizes the recursive probability expressions in an HMM. Consequently, it improves the computational complexity of applying the existing approaches for sensitivity analysis in Bayesian networks to HMMs. In [27], imprecise HMMs are presented as a tool for performing sensitivity analysis of HMMs.

In this paper, we propose a sensitivity analysis algorithm for HMMs using a simplified matrix formulation directly from the model representation based on a recently proposed technique called the Coefficient-Matrix-Fill procedure [26]. Until recently, sensitivity analysis in HMMs has been performed using Bayesian network sensitivity analysis techniques. The HMM is represented as a dynamic Bayesian network unrolled for a fixed number of time slices, and the Bayesian sensitivity algorithms are used. However, these algorithms do not utilize the HMMs’ recursive probability formulations. In this work, a simple algorithm based on a simplified matrix formulation is proposed. In this algorithm, to calculate the coefficients of the sensitivity functions, a series of Forward matrices $F^k,k=1,...,t$ are used, where k represents the time slice in the observation sequence. The matrices (Initial, Transition, and Observation) where the corresponding model parameter $\theta$ varies are decomposed into the parts independent of and dependent on $\theta$ for mathematical convenience. This enables us to compute the coefficients of the sensitivity function at each iteration in the recursive probability expression and implement the algorithm in a computer program. These matrices are computed based on a proportional co-variation assumption. The Observation Matrix O is represented as $n\times n$ diagonal matrices $O^t$ whose vth diagonal entry is $P(y_e^t|x_v^t)$ for each state v and whose other entries are 0 at the time t of the observation sequence. The proposed algorithm is computationally efficient as the length of the observation sequence increases. It computes the coefficients of a polynomial sensitivity function by filling coefficient matrices for all hidden states and all time steps.

The paper is organized as follows. In Section 2, the background on HMM and Sensitivity Analysis in HMM is presented. The details of the proposed algorithm for the filtering probability sensitivity function are explained in Section 3. In Section 4, the sensitivity analysis of the smoothing probability using the proposed algorithm is discussed. Finally, the paper is concluded with a summary of the results achieved and recommendations for future research works.

2. Background

In this section, we present the background on HMMs and Sensitivity Analysis in HMMs. First, we will discuss HMMs, HMM inference tasks and their corresponding recursive probability expressions. Then sensitivity analysis in HMMs is explained based on the assumption of proportional co-variation, and a univariate polynomial sensitivity function whose coefficients the proposed algorithm computes is defined. Here, the variables are denoted by capital letters and their values by lower case letters.

2.1. Hidden Markov Models

An HMM is a stochastic statistical model of a discrete Markov chain of a finite number of hidden variables X that can be observed by a sequence of a set of output variables Y from a sensor or other sources. The probability of transitioning from one state to another in this Markov chain is time-invariant, which makes the model stationary. The observed variables Y are stochastic with the observation (emission) probabilities, which are also time invariant. The overall HMM consists of n distinct hidden states of the Markov chain and m corresponding observable symbols. In general, the observations can be discrete or continuous, however, in this work, we focus on the discrete observations. The variable X has $n\ge 2$ hidden states, denoted by $x_i,1\le i\le n$, and the variable Y has $m\ge 2$ observable symbols, denoted by $y_j,1\le j\le m$. Formally, an HMM $\Lambda$ is specified by a set of parameters $(A,O,\Gamma )$ that are defined as follows:

• The prior probability distribution (initial vector) $\Gamma$ has entries ${\gamma }_i=p(x_i),1\le i\le n$, which are the probabilities of state $x_i$ being the first state in the Markov chain.
• The transition matrix A has entries $a_{i,j}=p(x_j|x_i),1\le i,j\le n$, which are the probabilities of transit from state $x_i$ to state $x_j$ in the Markov chain.
• The observation matrix O has entries $o_{i,j}=p(y_j|x_i),1\le i\le n,1\le j\le m$, which are the probabilities to observe $y_j$ if the current state is $x_i$.

An example of an HMM where X has two states and Y three symbols is shown in Figure 1a. The two states are $x_1$ and $x_2$ and based on them three symbols $y_1$, $y_2$ or $y_3$ are observed. The parameters of the HMM (including initial vector, transition matrix and observation matrix) are also shown in Figure 1a.

Figure 1. (a) An example of an HMM representation. (b) Its dynamic Bayesian network representation, unrolled for T time slices.

The dynamic Bayesian network representation of the HMM in Figure 1a is shown in Figure 1b, unrolled for T time slices [8,9]. The initial vector, transition matrix and observation matrix of the HMM are represented by the labels $\Gamma$, A and O respectively, attached to the nodes in the Bayesian network graph. The superscript for the variables X and Y and their values indicate the time slice under consideration.

In this paper, ${\mathbf{y}}_{\mathbf{e}}^t$ denotes the actual evidence for the variable Y in time slice t and ${\mathbf{y}}_{\mathbf{e}}^{t_i:t_j}$ represents a sequence of observations ${\mathbf{y}}_{\mathbf{e}}^{t_i},...,{\mathbf{y}}_{\mathbf{e}}^{t_j}$ where $\mathbf{e}$ is a sequence of observations from the set $y_j,1\le j\le m$. The representation $x_i^t$ denotes the actual state $x_i,1\le i\le n$, for the variable X in time slice t.

2.1.1. Inference in HMMs

In temporal models, inference means computing the conditional probability distribution of X at time t, given the evidence up to and including time T, that is $p(X^t|{\mathbf{y}}_{\mathbf{e}}^{1:T})$. The main inference tasks include filtering for $T=t$, prediction of a future state for $t>T$ and smoothing, that, is inferring the past for $t<T$. In an HMM, the Forward-Backward algorithms are used for the inference tasks and training of the model using an iterative procedure called the Baum-Welch method (Expectation Maximization) [28,29,30]. The Forward-Backward algorithms computes the following probabilities for all hidden states i at time $t\le T$:

• forward probability $F(i,t)=p(x_i^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})$, and
• backward probability $B(i,t)=p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_i^t)$

resulting in

$$\begin{array}{cc}\hfill p(x_i^t|{\mathbf{y}}_{\mathbf{e}}^{1:T})& =\frac{p(x_i^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})}{p({\mathbf{y}}_{\mathbf{e}}^{1:T})}=\frac{p(x_i^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})}{{\sum }_{j=1}^{n}p(x_j^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})}=\frac{p(x_i^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})·p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_i^t)}{{\sum }_{j=1}^{n}p(x_j^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})·p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_j^t)}\hfill \\ & =\frac{F(i,t)·B(i,t)}{{\sum }_{j=1}^{n}F(j,t)·B(j,t)}\hfill \end{array}$$

(1)

For $T<t$, the algorithm can be applied by taking $B(i,t)=1$ and adopting the convention that the configuration of an empty set of observations is TRUE, i.e., ${\mathbf{y}}_{\mathbf{e}}^{T+1:t}\equiv TRUE$, giving

$$F(i,t)=p(x_i^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})=p(x_i^t,{\mathbf{y}}_{\mathbf{e}}^{1:T},TRUE)=p(x_i^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})$$

As shown in Equation (1), any conditional probability $p(x_i^t|{\mathbf{y}}_{\mathbf{e}}^{1:T})$ can be calculated from the joint probabilities $p(x_i^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})$ for all hidden states i using Bayes’ theorem. Since our objective is sensitivity analysis of the HMM output probability of interest for parameter variations, in the remainder of this paper our focus will be on the joint probabilities $p(x_i^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})$ for all hidden states i.

The other important inference tasks are the prediction of future observations, i.e., $p({\mathbf{y}}_{\mathbf{e}}^t|{\mathbf{y}}_{\mathbf{e}}^{1:T})$ for $t>T$, finding the most probable explanation, i.e., $argma{x}_{x_i^{1:t}}p(x_i^{1:t}|{\mathbf{y}}_{\mathbf{e}}^{1:t})$ using the Viterbi Algorithm and learning the HMM parameters as new observations come in using the Baum-Welch algorithm, which can be achieved by the Forward-Backward algorithms. The prediction of future observations can be computed as the fraction of the two probabilities $p({\mathbf{y}}_{\mathbf{e}}^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})$, $t>T$, and $p({\mathbf{y}}_{\mathbf{e}}^{1:T})$, which are computed using forward probabilities.

$$p({\mathbf{y}}_{\mathbf{e}}^{1:t})=\sum _{i=1}^{n}p(x_i^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})=\sum _{i=1}^{n}F(i,t)$$

Note that if $t>T+1$, then $p({\mathbf{y}}_{\mathbf{e}}^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})$ can be computed by setting all in-between observations ${\mathbf{y}}_{\mathbf{e}}^k$, $t>k>T$, to $TRUE$ as above.

2.1.2. Recursive Probability Expressions

In order to establish the sensitivity functions for HMMs directly from the model’s representation, the recursive probability expression for the Forward-Backward algorithm should be investigated. The repetitive character of the model parameters in the Forward-Backward algorithm is used to drive the sensitivity functions. Consequently, in this section the recursive expressions of the Forward-Backward algorithm [29,31] are reviewed briefly.

Filtering. The filter probability is $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^t)$ for a specific state v of X. This probability is the same as the forward probability $F(v,t)$ in the Forward-Backward algorithm. For time slice $t=1$, we simply have that

$$p(x_v^1,{\mathbf{y}}_{\mathbf{e}}^1)=p({\mathbf{y}}_{\mathbf{e}}^1|x_v^1)·p(x_v^1)=o_{v,{\mathbf{e}}^1}·{\gamma }_v,$$

where ${\mathbf{e}}^1$ corresponds to the symbol of Y that is observed at time 1. For time slices $t>1$, we exploit the fact that $Y^t$ is independent of $Y^1,...,Y^{t-1}$, given $X^t$, written $Y^t\perp Y^{1:t-1}|X^t$. Then,

$$p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})=p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t-1},{\mathbf{y}}_{\mathbf{e}}^t)=p({\mathbf{y}}_{\mathbf{e}}^t|x_v^t)·p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t-1})$$

The first factor in the above product corresponds to $o_{v,{\mathbf{e}}^t}$; conditioning the second factor on the n states of $X^{t-1}$, we find with $X^t\perp Y^{1:t-1}|X^{t-1}$ that

$$\begin{array}{cc}\hfill p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t-1})& =\sum _{z=1}^{n}p(x_v^t|x_z^{t-1}).p(x_z^{t-1},{\mathbf{y}}_{\mathbf{e}}^{1:t-1})=\sum _{z=1}^n{a}_{z,v}.p(x_z^{t-1},{\mathbf{y}}_{\mathbf{e}}^{1:t-1})\hfill \end{array}$$

Taken together, we find for $F(v,t)=p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})$ the recursive expression

$$F(v,t)=\left\{\begin{array}{cc}{o}_{v,{\mathbf{e}}^1}·{\gamma }_v\hfill & \mathrm{if}t=1\hfill \\ o_{v,{\mathbf{e}}^t}·\sum _{z=1}^n{a}_{z,v}·F(z,t-1)\hfill & \mathrm{if}t>1\hfill \end{array}\right.$$

(2)

Prediction. The prediction probability $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})$ for $t>T$ can be handled by the the Forward-Backward algorithm by computing $F(v,t)$ with an empty set of evidence for $Y^{T+1:t}$. It can be implemented by replacing the term $o_{v,{\mathbf{e}}^t}$ by 1 for $t>T$ in Equation (2). As a result, for $T=0$, the prediction probability $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})$ becomes the prior probability $p(x_v^t)$. The prediction task can be seen as a special case of the filtering task.

Smoothing. Finally, we consider a smoothing probability $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})$ with $t<T$. Using $Y^{t+1:T}\perp Y^{1:t}|X^t$ we find that

$$\begin{array}{cc}\hfill p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})& =p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t},{\mathbf{y}}_{\mathbf{e}}^{t+1:T})=p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)·p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})\hfill \end{array}$$

The second term in this product is a filter probability; the first term is the same as the backward probability $B(v,t)$ in the Forward-Backward algorithm. By conditioning the first term on $X^{t+1}$ and exploiting the independences $X^t\perp Y^{t+1:T}|X^{t+1}$ and $Y^{t+1}\perp Y^{t+2:T}|X^{t+1}$ for $T>t+1$, we find that

$$\begin{array}{cc}\hfill p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)& =\sum _{z=1}^{n}p({\mathbf{y}}_{\mathbf{e}}^{t+1}|x_z^{t+1})·p({\mathbf{y}}_{\mathbf{e}}^{t+2:T}|x_z^{t+1})·p(x_z^{t+1}|x_v^t)\hfill \\ & =\sum _{z=1}^n{o}_{z,{\mathbf{e}}^{\mathbf{t}+\mathbf{1}}}·a_{v,z}·p({\mathbf{y}}_{\mathbf{e}}^{t+2:T}|x_z^{t+1})\hfill \end{array}$$

For $t+1=T$, this results in

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill p({\mathbf{y}}_{\mathbf{e}}^{T:T}|x_v^{T-1})& =\sum _{z=1}^{n}p({\mathbf{y}}_{\mathbf{e}}^T|x_z^T)·p(x_z^T|x_v^{T-1})=\sum _{z=1}^n{o}_{z,{\mathbf{e}}^{\mathbf{T}}}·a_{v,z}\hfill \end{array}\end{array}$$

Taken together, we find for $B(v,t)=p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)$ the recursive expression

$$B(v,t)=\left\{\begin{array}{cc}{\sum }_{z=1}^n{o}_{z,{\mathbf{e}}^{\mathbf{T}}}·a_{v,z}\hfill & \mathrm{if}t=1\hfill \\ {\sum }_{z=1}^n{o}_{z,{\mathbf{e}}^{\mathbf{t}+\mathbf{1}}}·a_{v,z}·B(z,t+1)\hfill & \mathrm{if}t>1\hfill \end{array}\right.$$

(3)

In this work, we present the sensitivity of the HMM for filtering and smoothing probabilities for transition, initial and observation parameter variation. Therefore, the above discussion on the recursive probability expression for the filtering probability, Equation (2), and smoothing probability Equation (3), will be used to formulate our algorithm.

2.2. Sensitivity Analysis in HMM

As shown in recent studies [13,25,32,33], sensitivity analysis is establishing a functional relationship that describes how output probability varies as the network’s parameters change. A posterior marginal probabilities, $y=p(v|e)$, where v is value of a variable V and e is the evidence available, is considered to represent the output probabilities of interest. The network parameter under consideration is represented as $\theta =p(v_j|\pi )$ for a value $v_j$ of a variable V, and $\pi$ is an arbitrary combination of the values of the set of evidences of V. So $p(v|e)(\theta )$ represents the posterior marginal probabilities as a function of $\theta$.

Consequently, when the parameter $\theta =p(v_j|\pi )$ varies, each of the other conditional probabilities $\theta =p(v_i|\pi )$ from the same distribution co-vary accordingly by the ratio between the the remaining probabilities. That is, if a parameter $\theta =p(v_j|\pi )$ for a variable V is varied, then

$$p(v_i|\pi )(\theta )=\left\{\begin{array}{cc}\theta \hfill & \hfill \mathrm{if}i=j\\ p(v_i|\pi )·\frac{1-\theta }{1-p(v_j|\pi )}\hfill & \hfill i\ne j\end{array}\right.$$

(4)

The co-variation simplifies to $p(v_i|\pi )(\theta )=1-\theta$ for binary-valued V. The sensitivity function is defined based on the stated proportional co-variation assumption. The constants in the general form of the sensitivity function are generated from the network’s parameters that are neither varied nor co-varied, and their values depend on the output probability of interest. A sensitivity function for a posterior probability of interest is a quotient of two polynomial functions, since $p(v|e)=p(ve)/p(e)$, and hence a rational function.

We have the following univariate polynomial sensitivity function to define the joint probability of a hidden state and a sequence of observations as a function of a model parameter $\theta$.

$$p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})=\sum _{i=0}^N{c}_{v,i}^t·{\theta }^i$$

(5)

The coefficients $c_{v,i}^t$ are derived from the network parameters that are not co-varied by assuming proportional co-variation of the parameters from the same conditional distribution. The coefficients $c_{v,i}^t$ depend on the hidden state v and time slice t under consideration (see for details [25,26]).

3. Sensitivity Analysis of Filtering Probability

In this section, the simplified matrix formulation for sensitivity of the filtering probability to the variation of the initial, transition and observation parameters is proposed. Let us consider the sensitivity functions $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})(\theta )$ for a filtering probability and the variation of one of the model parameters, viz., $\theta$. From Equation (2), the forward recursive filtering probability expression, it follows that

$$p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})(\theta )=\left\{\begin{array}{cc}{o}_{v,{\mathbf{e}}^{\mathbf{1}}}(\theta )·{\gamma }_v(\theta )\hfill & \mathrm{if}t=1\hfill \\ o_{v,{\mathbf{e}}^{\mathbf{t}}}(\theta )·{\sum }_{z=1}^n{a}_{z,v}(\theta )·p(x_v^{t-1},{\mathbf{y}}_{\mathbf{e}}^{1:t-1})(\theta )\hfill & \mathrm{if}t>1\hfill \end{array}\right.$$

(6)

where

$${\gamma }_v(\theta )=\left\{\begin{array}{cc}{\gamma }_v({\theta }_{\gamma })\hfill & \mathrm{for}initialparametervariation\theta ={\theta }_{\gamma }\hfill \\ {\gamma }_v\hfill & otherwise\hfill \end{array}\right.$$

$$a_{z,v}(\theta )=\left\{\begin{array}{cc}{a}_{z,v}({\theta }_a)\hfill & \mathrm{for}transitionparametervariation\theta ={\theta }_a\hfill \\ a_{z,v}\hfill & otherwise\hfill \end{array}\right.$$

and

$$o_{v,{\mathbf{e}}^{\mathbf{t}}}(\theta )=\left\{\begin{array}{cc}{o}_{v,{\mathbf{e}}^{\mathbf{t}}}({\theta }_o)\hfill & \mathrm{for}observationparametervariation\theta ={\theta }_o\hfill \\ o_{v,{\mathbf{e}}^{\mathbf{t}}}\hfill & otherwise\hfill \end{array}\right.$$

The sensitivity function $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})(\theta )$ coefficients are computed by the proposed method using a series of Forward matrices $F^k,k=1,...,t$. The sensitivity of the filtering probability for transition, initial and observation parameter variations is formulated in the next subsections.

3.1. Transition Parameter Variation

In this subsection, the simplified matrix formulation (SMF) for sensitivity of the filtering probability to variation of the transition parameter is proposed. Let us consider the sensitivity functions $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_a)$ for a filtering probability and transition parameter ${\theta }_a=a_{r,s}$. From Equation (6), the recursive filtering probability sensitivity function, it follows that for $t=1$ we have the constant $p(x_v^1,{\mathbf{y}}_{\mathbf{e}}^1)({\theta }_a)=o_{v,{\mathbf{e}}^{\mathbf{1}}}·{\gamma }_v$, and for $t>1$,

$$p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_a)=o_{v,{\mathbf{e}}^{\mathbf{t}}}·\sum _{z=1}^n{a}_{z,v}({\theta }_a)·p(x_v^{t-1},{\mathbf{y}}_{\mathbf{e}}^{1:t-1})({\theta }_a)$$

(7)

In the next sub-subsections, the formulation of the proposed method to compute the coefficients of the sensitivity function in Equation (5) is presented in detail. First, the decomposition of the transition matrix into the parts of A independent of and dependent on ${\theta }_a$ and the observation matrix representation for the simplified matrix formulation is presented. Then the SMF representation for the sensitivity of the filtering probability to transition parameter variation and its algorithm implementation are discussed.

3.1.1. Transition Matrix Decomposition and Observation Matrix Representation

In the proposed algorithm, to calculate the coefficients in a series of Forward matrices $F^k$, $k=1,...,t,$ the Transition Matrix A is divided into $\overline{A}$ and $\widehat{A}$ that are independent of and dependent on ${\theta }_a$, respectively, for mathematical convenience. The matrices $\overline{A}$ and $\widehat{A}$ are computed based on the proportional co-variation assumption shown in Equation (4). Let us explain the decomposition considering an HMM with an $n\times n$ Transition Matrix A with the transition parameter ${\theta }_a=a_{r,s}=a_{n,1}=p(x_1^t|x_n^{t-1})$. This makes the transition matrix A a function of ${\theta }_a$, and, with the proportional co-variation assumption shown in Equation (4), it becomes

$$A({\theta }_a)=\left[\begin{array}{cccc}{a}_{1,1}& a_{1,2}& \cdots & a_{1,n}\\ a_{2,1}& a_{2,2}& \cdots & a_{2,n}\\ ⋮& ⋮& \ddots & ⋮\\ {\theta }_a& a_{n,2}.\frac{1-{\theta }_a}{1-a_{n,1}}& \cdots & a_{n,n}.\frac{1-{\theta }_a}{1-a_{n,1}}\end{array}\right]$$

This matrix is divided into $\overline{A}$ and $\widehat{A}$ that are independent of and dependent on ${\theta }_a$, respectively, as follows, and such that $A({\theta }_a)$ = $\overline{A}$ + $\widehat{A}({\theta }_a)$.

$$\overline{A}=\left[\begin{array}{cccc}{a}_{1,1}& a_{1,2}& \cdots & a_{1,n}\\ a_{2,1}& a_{2,2}& \cdots & a_{2,n}\\ ⋮& ⋮& \ddots & ⋮\\ 0& \frac{a_{n,2}}{1-a_{n,1}}& \cdots & \frac{a_{n,n}}{1-a_{n,1}}\end{array}\right]\mathrm{and}\widehat{A}({\theta }_a)=\left[\begin{array}{cccc}0& 0& \cdots & 0\\ 0& 0& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 1& \frac{-a_{n,2}}{1-a_{n,1}}& \cdots & \frac{-a_{n,n}}{1-a_{n,1}}\end{array}\right]·{\theta }_a$$

In short, this decomposition can be represented in the algorithm simply as $\overline{A}$ and $\widehat{A}$ that are independent of and dependent on ${\theta }_a$, respectively.

In the simplified matrix representation, the Observation Matrix O is represented as an $n\times n$ diagonal matrices $O^t$ whose $v^{th}$ diagonal entry is $P(y_e^t|x_v^t)$ for each state v and whose other entries are 0 at the time t of the observation sequence [31]. Let us explain this representation by considering the HMM with an $n\times m$ Observation Matrix O. Given the following sequence of observations: $y_{\mathbf{2}}^1,y_{\mathbf{1}}^2$ and $y_{\mathbf{3}}^3$, the corresponding diagonal matrices become

$$O^1=\left[\begin{array}{cccc}{o}_{1,2}& 0& \cdots & 0\\ 0& o_{2,2}& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & o_{n,2}\end{array}\right]=diag(O_{:,2}),O^2=diag(O_{:,1})\mathrm{and}{O}^3=diag(O_{:,3})$$

3.1.2. SMF for Sensitivity Analysis of the Filtering Probability to the Transition Parameter Variation

In the Simplified Matrix Representation (SMF), the sensitivity function $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_a)$ shown in Equation (7) can be written as a simple matrix-vector multiplication. It follows that for $t=1$ we have the constant $p(x_v^1,{\mathbf{y}}_{\mathbf{e}}^1)({\theta }_a)=o_{v,{\mathbf{e}}^{\mathbf{1}}}·{\gamma }_v$, which becomes $p(X^1,{\mathbf{y}}_{\mathbf{e}}^1)({\theta }_a)=O^1{\Gamma }^{⊺}$ in the SMF. For $t>1$, we have

$$p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_a)=o_{v,{\mathbf{e}}^{\mathbf{t}}}·\sum _{z=1}^n{a}_{z,v}({\theta }_a)·p(x_v^{t-1},{\mathbf{y}}_{\mathbf{e}}^{1:t-1})({\theta }_a)$$

which is represented in SMF as

$$p(X^t,{\mathbf{y}}_{\mathbf{e}}^t)({\theta }_a)=O^{t}A{({\theta }_a)}^{⊺}p(X^{t-1},{\mathbf{y}}_{\mathbf{e}}^{1:t-1})({\theta }_a)$$

The overall procedure of computing the sensitivity coefficients of $p(X^t,{\mathbf{y}}_{\mathbf{e}}^t)({\theta }_a)$ using the SMF is described in the following algorithm.

3.1.3. Algorithm Implementation

This algorithm summarizes the proposed technique, where e is the sequence of observations and $\theta =a_{r,s}$, and solves the recursive Equation (7). Once the Transition and Observation matrices are represented in SMF as explained above, the sensitivity coefficients are computed by filling the forward matrices $F^k$ for $k=1\cdots t$. The details of filling contents of the forward matrices in Algorithm 1 are explained as follows. $F^1$ is initialized by the matrix multiplications of $O^1$, which is a diagonal matrix whose $v^{th}$ diagonal entry is $P(y_e^1|x_v^1)$, and the initial distribution vector ${\Gamma }^{⊺}$,

$$F^1=O^1*{\Gamma }^{⊺}$$

The remaining matrices $F^k$ of size $n\times k,2\le k\le t,$ are initialized by filling them with zeros. Two temporary matrices $Ftm{p}_1$ and $Ftm{p}_2$ are used for computing the coefficients as explained previously. $Ftm{p}_1$ is computed as the matrix multiplication of $O^k$, ${\overline{A}}^{⊺}$ (which is independent of ${\theta }_a$) and $F^{k-1}$, and a zero vector of size n (= the number of states) is appended at its end. $F^{k-1}$ represents the recursion in the filtering probability. In the same way $Ftm{p}_2$ is computed as the matrix multiplication of $O^k$, ${\widehat{A}}^{⊺}$ (which is dependent on ${\theta }_a$) and $F^{k-1}$, and a zero vector of size n is appended to the front. Then $F^k$ is set as the sum of $Ftm{p}_1$ and $Ftm{p}_2$:

$$F^k=Ftm{p}_1+Ftm{p}_2$$

Algorithm 1 Computes the coefficients of the forward (filtering) probability sensitivity function $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_a)$ with ${\theta }_a=a_{r,s}$ in forward matrices $F^k$ for $k=1,...,t$ using SMF.

Input: $A,O,\Gamma ,\mathbf{e},\mathrm{and}{\theta }_a$

Output: $F^1,\cdots ,F^t$

1: $\overline{A}\leftarrow A$ 2: ${\overline{A}}_{r,s}\leftarrow 0$ 3: ${\overline{A}}_{r,:}\leftarrow {\overline{A}}_{r,:}./(1-a_{r,s})$ 4: $\widehat{A}\leftarrow A-\overline{A}$ 5: ${\widehat{A}}_{r,:}\leftarrow {\widehat{A}}_{r,:}./a_{r,s}$ 6: $O^1\leftarrow diag(O_{:,{\mathbf{e}}^{\mathbf{1}}})$ 7: $F^1\leftarrow O^1*{\Gamma }^{⊺}$ 8: for $k=2\mathrm{to}t$ do 9:     $O^k\leftarrow diag(O_{:,{\mathbf{e}}^{\mathbf{k}}})$ 10:     $Ftm{p}_1\leftarrow [O^k*{\overline{A}}^{⊺}*F^{k-1},zeros(n,1)]$ 11:     $Ftm{p}_2\leftarrow [zeros(n,1),O^k*{\widehat{A}}^{⊺}*F^{k-1}]$ 12:     $F^k\leftarrow Ftm{p}_1+Ftm{p}_2$ 13: Return $F^1,\cdots ,F^t$

3.2. Initial Parameter Variation

In this subsection, the sensitivity of the filtering probability to the initial parameter variation is derived based on SMF. Let us consider the sensitivity functions $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_{\gamma })$ for a filtering probability and variation in the initial parameter ${\theta }_{\gamma }={\gamma }_r$. From Equation (6), the recursive filtering probability sensitivity expression, for $t=1$, we have $p(x_v^1,{\mathbf{y}}_{\mathbf{e}}^1)({\theta }_{\gamma })=o_{v,{\mathbf{e}}^{\mathbf{1}}}·{\gamma }_v({\theta }_{\gamma })$, and for $t>1$,

$$p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_{\gamma })=o_{v,{\mathbf{e}}^{\mathbf{t}}}·\sum _{z=1}^n{a}_{z,v}·p(x_v^{t-1},{\mathbf{y}}_{\mathbf{e}}^{1:t-1})({\theta }_{\gamma })$$

(8)

3.2.1. Initial Vector Decomposition

In the proposed algorithm, to calculate the coefficients in a series of forward matrices $F^k,k=1,...,t,$ the Initial Vector $\Gamma$ is divided into $\overline{\Gamma }$ and $\widehat{\Gamma }$ that are independent of and dependent on ${\theta }_{\gamma }$, respectively, for mathematical convenience. The matrices $\overline{\Gamma }$ and $\widehat{\Gamma }$ are computed based on the proportional co-variation assumption shown in Equation (4).

Let us explain the Initial Vector $\Gamma$ decomposition considering the HMM presented in Section 3.1 where the Initial Vector $\Gamma$ is $[{\gamma }_1,{\gamma }_2,\cdots ,{\gamma }_n]$ and the initial parameter variation ${\theta }_{\gamma }$ is ${\gamma }_2=p(x_2^1)$. This makes the Initial Vector $\Gamma$ a function of ${\theta }_{\gamma }$, and, with the proportional co-variation assumption shown in Equation (4), it becomes

$$\Gamma ({\theta }_{\gamma })=\left[\begin{array}{cccc}\frac{{\gamma }_1(1-{\theta }_{\gamma })}{1-{\gamma }_2}& {\theta }_{\gamma }& \cdots & \frac{{\gamma }_n(1-{\theta }_{\gamma })}{1-{\gamma }_2}\end{array}\right]$$

This matrix is divided into $\overline{\Gamma }$ and $\widehat{\Gamma }$ that are independent of and dependent on ${\theta }_{\gamma }$, respectively, as follows, and such that $\Gamma ({\theta }_{\gamma })$ = $\overline{\Gamma }$ + $\widehat{\Gamma }({\theta }_{\gamma })$.

$$\overline{\Gamma }=\left[\begin{array}{cccc}\frac{{\gamma }_1}{1-{\gamma }_2}& 0& \cdots & \frac{{\gamma }_n}{1-{\gamma }_2}\end{array}\right]\mathrm{and}\widehat{\Gamma }({\theta }_{\gamma })=\left[\begin{array}{cccc}\frac{-{\gamma }_1}{1-{\gamma }_2}& 1& \cdots & \frac{-{\gamma }_n}{1-{\gamma }_2}\end{array}\right]·{\theta }_{\gamma }$$

In short, this decomposition can be represented simply as $\overline{\Gamma }$ and $\widehat{\Gamma }$ that are independent of and dependent on ${\theta }_{\gamma }$, respectively.

Once the Initial Vector $\Gamma$ is decomposed as shown above, to compute the coefficients of the sensitivity function $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_{\gamma })$ using the recursive Equation (8), the Observation Matrix O is represented as an $n\times n$ diagonal matrices $O^t$.

3.2.2. SMF for Sensitivity Analysis of the Filtering Probability to the Initial Parameter Variation

In the simplified matrix representation, the sensitivity function $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_{\gamma })$ shown in Equation (8) can be written as a simple matrix-vector multiplication. It follows that for $t=1$ we have the constant $p(x_v^1,{\mathbf{y}}_{\mathbf{e}}^1)({\theta }_{\gamma })=o_{v,{\mathbf{e}}^{\mathbf{1}}}·{\gamma }_v({\theta }_{\gamma })$, which becomes $p(X^1,{\mathbf{y}}_{\mathbf{e}}^1)({\theta }_{\gamma })=O^1{\Gamma }^{⊺}({\theta }_{\gamma })$ in the simplified matrix representation. For $t>1$, we have

$$p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_{\gamma })=o_{v,{\mathbf{e}}^{\mathbf{t}}}·\sum _{z=1}^n{a}_{z,v}·p(x_v^{t-1},{\mathbf{y}}_{\mathbf{e}}^{1:t-1})({\theta }_{\gamma })$$

which is represented as

$$p(X^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_{\gamma })=O^t{A}^{⊺}p(X^{t-1},{\mathbf{y}}_{\mathbf{e}}^{1:t-1})({\theta }_{\gamma })$$

To solve the forward (filtering) probability sensitivity function $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_{\gamma })$, the following algorithm implementation is presented.

3.2.3. Algorithm Implementation

This procedure summarizes the proposed method, where e is the sequence of observations and $\theta ={\theta }_{\gamma }$, and solves the recursive Equation (8).

As shown in Algorithm 2, once the Initial Vector is decomposed into the parts, $\overline{\Gamma }$ independent of ${\theta }_{\gamma }$ and $\widehat{\Gamma }$ dependent on ${\theta }_{\gamma }$ and the Observation Matrix is represented in the diagonal matrices, the sensitivity coefficients are computed by filling the forward matrices $F^k$ for $k=1\cdots t$. The details of filling the forward matrices in Algorithm 2 are discussed as follows. $F^1$ is initialized as a sum of two temporary matrices $Ftm{p}_1$ and $Ftm{p}_2$ which are used to compute the coefficients for the parts independent of and dependent on ${\theta }_{\gamma }$, respectively. $Ftm{p}_1$ is computed as the matrix multiplication of $O^1$ and ${\overline{\Gamma }}^{\prime }$, and a zero vector of size n is appended at its end to represent the zero coefficients of degree 1. In the same way, $Ftm{p}_2$ is computed as the matrix multiplication of $O^1$ and ${\widehat{\Gamma }}^{\prime }$, and a zero vector of size n is appended in front of it to represent the zero coefficients of degree 0. Then

Algorithm 2 This computes the coefficients of the forward (filtering) probability sensitivity function $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_{\gamma })$ with ${\theta }_{\gamma }={\gamma }_r$ in forward matrices $F^k$ for $k=1,\cdots ,t$ using SMF.

Input: $A,O,\Gamma ,\mathbf{e},\mathrm{and}{\theta }_{\gamma }$

Output: $F^1,\cdots ,F^t$

1: $\overline{\Gamma }\leftarrow \Gamma$ 2: ${\overline{\Gamma }}_r\leftarrow 0$ 3: ${\overline{\Gamma }}_{:}\leftarrow {\overline{\Gamma }}_{:}./(1-{\gamma }_r)$ 4: $\widehat{\Gamma }\leftarrow \Gamma -\overline{\Gamma }$ 5: ${\widehat{\Gamma }}_{:}\leftarrow {\widehat{\Gamma }}_{:}./{\gamma }_r$ 6: $O^1\leftarrow diag(O_{:,{\mathbf{e}}^{\mathbf{1}}})$ 7: $Ftm{p}_1\leftarrow [O^1*{\overline{\Gamma }}^{\prime },zeros(n,1)]$ 8: $Ftm{p}_2\leftarrow [zeros(n,1),O^1*{\widehat{\Gamma }}^{\prime }]$ 9: $F^1\leftarrow Ftm{p}_1+Ftm{p}_2$ 10: for $k=2\mathrm{to}t$ do 11:     $O^k\leftarrow diag(O_{:,{\mathbf{e}}^{\mathbf{k}}})$ 12:     $F^k\leftarrow [O^k*A^{\prime }*F^{k-1}]$ 13: Return $F^1,\cdots ,F^t$

$$F^1=Ftm{p}_1+Ftm{p}_2$$

The remaining matrices $F^k$ of size $n\times k,2\le k\le t,$ are computed using recursion. They are computed as the matrix multiplication of $O^k$, $A^{\prime }$ and $F^{k-1}$.

3.3. Observation Parameter Variation

In this subsection, the sensitivity of the filtering probability $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})$ to the observation parameter variation is derived based on SMF. Let us consider the sensitivity functions $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_o)$ for a filtering probability and variation in the observation parameter ${\theta }_o=o_{r,s}$. From Equation (6), the recursive filtering probability sensitivity function, it follows that for $t=1$ we have $p(x_v^1,{\mathbf{y}}_{\mathbf{e}}^1)({\theta }_0)=o_{v,{\mathbf{e}}^{\mathbf{1}}}({\theta }_o)·{\gamma }_v$, and for $t>1$,

$$p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_o)=o_{v,{\mathbf{e}}^{\mathbf{t}}}({\theta }_o)·\sum _{z=1}^n{a}_{z,v}·p(x_v^{t-1},{\mathbf{y}}_{\mathbf{e}}^{1:t-1})({\theta }_o)$$

(9)

3.3.1. Observation Matrix Decomposition

In the proposed algorithm, to calculate the coefficients in a series of forward matrices $F^k,k=1,...,t,$ the Observation Matrix O is divided into $\overline{O}$ and $\widehat{O}$ that are independent of and dependent on ${\theta }_o$, respectively, for mathematical convenience. The matrices $\overline{O}$ and $\widehat{O}$ are computed based on the proportional co-variation assumed as shown in Equation (4). Let us explain the Observation Matrix O decomposition by considering the HMM presented in the previous section and let the observation parameter ${\theta }_o$ be $o_{2,1}=p(y_1^t|x_2^t)$.

The observation matrix O becomes a function of ${\theta }_o$ based on the proportional co-variation assumption shown in Equation (4) as follows.

$$O({\theta }_o)=\left[\begin{array}{cccc}{o}_{1,1}& o_{1,2}& \cdots & o_{1,m}\\ {\theta }_o& \frac{o_{2,2}(1-{\theta }_o)}{1-o_{2,1}}& \cdots & \frac{o_{2,m}(1-{\theta }_o)}{1-o_{2,1}}\\ ⋮& ⋮& \ddots & ⋮\\ o_{n,1}& o_{n,2}& \cdots & o_{n,m}\end{array}\right]$$

This matrix is decomposed into $\overline{O}$ and $\widehat{O}$ that are independent of and dependent on ${\theta }_o$, respectively, such that $O({\theta }_o)=\overline{O}+\widehat{O}({\theta }_o)$, just as we decomposed $A({\theta }_a)=\overline{A}+\widehat{A}({\theta }_a)$ in Section 3.1.1.

Once the Observation Matrix is decomposed into $\overline{O}$ and $\widehat{O}$, in the SMF, they are represented as $n\times n$ diagonal matrices. For example, the sequence of observations $y_{\mathbf{2}}^1,y_{\mathbf{1}}^2$ and $y_{\mathbf{3}}^3$, the corresponding diagonal matrices become

$${\overline{O}}^1=diag({\overline{O}}_{:,2}),{\overline{O}}^2=diag({\overline{O}}_{:,1})\mathrm{and}{\overline{O}}^3=diag({\overline{O}}_{:,3})$$

$${\widehat{O}}^1=diag({\widehat{O}}_{:,2}),{\widehat{O}}^2=diag({\widehat{O}}_{:,1})\mathrm{and}{\widehat{O}}^3=diag({\widehat{O}}_{:,3})$$

3.3.2. SMF for Sensitivity Analysis of the Filtering Probability to the Observation Parameter Variation

In the simplified matrix representation, the sensitivity function $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_o)$ can be written as a simple matrix-vector multiplication. From Equation (9), it follows that for $t=1$ we have the constant $p(x_v^1,{\mathbf{y}}_{\mathbf{e}}^1)({\theta }_o)=o_{v,{\mathbf{e}}^{\mathbf{1}}}({\theta }_o)·{\gamma }_v$, which becomes $p(x_v^1,{\mathbf{y}}_{\mathbf{e}}^1)({\theta }_o)=O^1({\theta }_o){\Gamma }^{⊺}$ in the SMF. For $t>1$, we have

$$p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_o)=o_{v,{\mathbf{e}}^{\mathbf{t}}}({\theta }_o)·\sum _{z=1}^n{a}_{z,v}·p(x_v^{t-1},{\mathbf{y}}_{\mathbf{e}}^{1:t-1})({\theta }_o),$$

which is represented as

$$p(X^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_o)=O^t({\theta }_o)A^{⊺}p(X^{t-1},{\mathbf{y}}_{\mathbf{e}}^{1:t-1})({\theta }_o)$$

The derivation of the proposed technique for the sensitivity analysis of the forward (filtering) probability $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_o)$ with observation parameter variation ${\theta }_o=o_{r,s}$ is formulated in Algorithm 3.

Algorithm 3 This computes the coefficients of the forward (filtering) probability sensitivity function $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})({\theta }_o)$ with ${\theta }_o=o_{r,s}$ in forward matrices $F^k$ for $k=1,\cdots ,t$ using SMF.

Input: $A,O,\Gamma ,\mathbf{e},\mathrm{and}{\theta }_o$

Output: $F^1,\cdots ,F^t$

1: $\overline{O}\leftarrow O$ 2: ${\overline{O}}_{r,s}\leftarrow 0$ 3: ${\overline{O}}_{r,:}\leftarrow {\overline{O}}_{r,:}./(1-o_{r,s})$ 4: $\widehat{O}\leftarrow O-\overline{O}$ 5: ${\widehat{O}}_{r,:}\leftarrow {\widehat{O}}_{r,:}./o_{r,s}$ 6: ${\overline{O}}^1\leftarrow diag({\overline{O}}_{:,{\mathbf{e}}^{\mathbf{1}}})$ 7: ${\widehat{O}}^1\leftarrow diag({\widehat{O}}_{:,{\mathbf{e}}^{\mathbf{1}}})$ 8: $Ftm{p}_1\leftarrow [{\overline{O}}^1*{\Gamma }^{\prime },zeros(n,1)]$ 9: $Ftm{p}_2\leftarrow [zeros(n,1),{\widehat{O}}^1*{\Gamma }^{\prime }]$ 10: $F^1\leftarrow Ftm{p}_1+Ftm{p}_2$ 11: for $k=2\mathrm{to}t$ do 12:     ${\overline{O}}^k\leftarrow diag({\overline{O}}_{:,{\mathbf{e}}^{\mathbf{k}}})$ 13:     ${\widehat{O}}^k\leftarrow diag({\widehat{O}}_{:,{\mathbf{e}}^{\mathbf{k}}})$ 14:     $Ftm{p}_1\leftarrow [{\overline{O}}^k*A^{\prime }*F^{k-1},zeros(n,1)]$ 15:     $Ftm{p}_2\leftarrow [zeros(n,1),{\widehat{O}}^k*A^{\prime }*F^{k-1}]$ 16:     $F^k\leftarrow Ftm{p}_1+Ftm{p}_2$ 17: Return $F^1,\cdots ,F^t$

3.3.3. Algorithm Implementation

In this algorithm, e is the sequence of observation and $\theta ={\theta }_o$. It solves the recursive sensitivity function Equation (9) in forward matrices $F^k$ for $k=1,\cdots ,t$. The observation matrix is decomposed into the independent $\overline{O}$ and dependent $\widehat{O}$ observation matrices on ${\theta }_o$. Then, these observation matrices are represented as diagonal matrices in the SMF computation. The detailed steps of filling the contents of the forward matrices $F^k$ are explained as follows. $F^1$ is initialized as a sum of two temporary matrices $Ftm{p}_1$ and $Ftm{p}_2$, which are used to compute the coefficients for the independent and dependent parts on ${\theta }_o$. $Ftm{p}_1$ is computed as the matrix multiplication of ${\overline{O}}^1$ and ${\Gamma }^{\prime }$, and a zero vector of size n states is appended at its end to represent the zero coefficients of degree 1. In the same way, $Ftm{p}_2$ is computed as the matrix multiplication of ${\widehat{O}}^1$ and ${\Gamma }^{\prime }$, and a zero vector of size n states is appended in front of it to represent the zero coefficients of degree 0.

$$F^1=Ftm{p}_1+Ftm{p}_2$$

The remaining matrices $F^k$ of size $n\times k,2\le k\le t,$ are computed using recursion. Again, two temporary matrices $Ftm{p}_1$ and $Ftm{p}_2$ are used for computing the coefficients as it is explained above. Here, $Ftm{p}_1$ is computed as the matrix multiplication of the diagonal matrix ${\overline{O}}^k$ that is independent of ${\theta }_o$, $A^{\prime }$ and $F^{k-1}$, and a zero vector of size n states is appended at its end. $F^{k-1}$ represents the recursion in the filtering probability. Likewise, $Ftm{p}_2$ is computed as the matrix multiplication of the diagonal matrix $O^k$ that is dependent on ${\theta }_o$, $A^{\prime }$ and $F^{k-1}$, and a zero vector of size n states is appended in front of it. Then, $F^k$ is set as the sum of $Ftm{p}_1$ and $Ftm{p}_2$:

$$F^k=Ftm{p}_1+Ftm{p}_2$$

3.4. Filtering Sensitivity Function Example

The proposed algorithm is illustrated by the following example. The example is also used to illustrate the computations in the Coefficient-Matrix-Fill procedure in [26]. The computed coefficients using the proposed algorithm are the same as those computed using the Coefficient-Matrix-Fill procedure. However, the proposed algorithm is simplified and works for any number of hidden states and evidence (observation) variables. It is also efficient in terms of computational time as the length of the observation sequence increases.

Example 1.

Consider an HMM with binary-valued hidden state X and binary-valued evidence variable Y. Let $\Gamma =[0.20,0.80]$ be the initial vector for $X^1$. The transition matrix A and observation matrix O be as follows:

$$A=\left[\begin{array}{cc}0.95& 0.05\\ 0.15& 0.85\end{array}\right]andO=\left[\begin{array}{cc}0.75& 0.25\\ 0.90& 0.10\end{array}\right]$$

Let us compute the sensitivity functions for the filtering probability using Algorithm 1 for the two states of $X^t$ as a function of transition parameter ${\theta }_a=a_{2,1}=p(x_1^t|x_2^{t-1})=0.15$ given the following sequence of observations: $y_2^1,y_1^2$ and $y_1^3$.

The simplified matrix formulation procedure divides the Transition Matrix A into $\overline{A}$ and $\widehat{A}$:

$$A=\left[\begin{array}{cc}0.95& 0.05\\ {\theta }_a& 1-{\theta }_a\end{array}\right],\overline{A}=\left[\begin{array}{cc}0.95& 0.05\\ 0& 1\end{array}\right]and\widehat{A}=\left[\begin{array}{cc}0& 0\\ 1& -1\end{array}\right]$$

For the observation sequence given, $y_2^1,y_1^2$ and $y_1^3$, the diagonal matrices become

$$O^1=\left[\begin{array}{cc}0.25& 0\\ 0& 0.10\end{array}\right],O^2=\left[\begin{array}{cc}0.75& 0\\ 0& 0.90\end{array}\right]and{O}^3=\left[\begin{array}{cc}0.75& 0\\ 0& 0.90\end{array}\right]$$

The coefficients for the sensitivity functions are computed by the proposed procedure using the forward matrices that are constructed as follows:

$$F^1=O^1*{\Gamma }^{\prime }=\left[\begin{array}{cc}0.25& 0\\ 0& 0.10\end{array}\right]\times \left[\begin{array}{c}0.20\\ 0.80\end{array}\right]=\left[\begin{array}{c}0.05\\ 0.08\end{array}\right]$$

Then, $F^2=Ftm{p}_1+Ftm{p}_2$, where the temporary matrices $Ftm{p}_1$ and $Ftm{p}_2$ are computed as follows:

$$\begin{array}{ll}Ftm{p}_1& =[O^2*{\overline{A}}^{\prime }*F^1,zeros(2,1)]\\ & =\left[\begin{array}{c}\left[\begin{array}{cc}0.75& 0\\ 0& 0.90\end{array}\right]\times \left[\begin{array}{cc}0.95& 0\\ 0.05& 1\end{array}\right]\times \left[\begin{array}{c}0.05\\ 0.08\end{array}\right],\left[\begin{array}{c}0\\ 0\end{array}\right]\end{array}\right]\\ & =\left[\begin{array}{cc}0.0356& 0\\ 0.0743& 0\end{array}\right]\end{array}$$

$$\begin{array}{ll}Ftm{p}_2& =[zeros(2,1),O^2*{\widehat{A}}^{\prime }*F^1]\\ & =\left[\begin{array}{c}\left[\begin{array}{c}0\\ 0\end{array}\right],\left[\begin{array}{cc}0.75& 0\\ 0& 0.90\end{array}\right]\times \left[\begin{array}{cc}0& 1\\ 0& -1\end{array}\right]\times \left[\begin{array}{c}0.05\\ 0.08\end{array}\right]\end{array}\right]\\ & =\left[\begin{array}{cc}0& 0.060\\ 0& -0.072\end{array}\right]\end{array}$$

$$F^2=Ftm{p}_1+Ftm{p}_2=\left[\begin{array}{cc}0.0356& 0.060\\ 0.0743& -0.072\end{array}\right],$$

and finally, for $F^3$:

$$\begin{array}{ll}Ftm{p}_1& =[O^3*{\overline{A}}^{\prime }*F^2,zeros(2,1)]\\ & =\left[\begin{array}{c}\left[\begin{array}{cc}0.75& 0\\ 0& 0.90\end{array}\right]\times \left[\begin{array}{cc}0.95& 0\\ 0.05& 1\end{array}\right]\times \left[\begin{array}{cc}0.0356& 0.060\\ 0.0743& -0.072\end{array}\right],\left[\begin{array}{c}0\\ 0\end{array}\right]\end{array}\right]\\ & =\left[\begin{array}{ccc}0.0254& 0.0428& 0\\ 0.0684& -0.0621& 0\end{array}\right]\end{array}$$

$$\begin{array}{ll}Ftm{p}_2& =[zeros(2,1),O^3*{\widehat{A}}^{\prime }*F^2]\\ & =\left[\begin{array}{c}\left[\begin{array}{c}0\\ 0\end{array}\right],\left[\begin{array}{cc}0.75& 0\\ 0& 0.90\end{array}\right]\times \left[\begin{array}{cc}0& 1\\ 0& -1\end{array}\right]\times \left[\begin{array}{cc}0.0356& 0.060\\ 0.0743& -0.072\end{array}\right]\end{array}\right]\\ & =\left[\begin{array}{ccc}0& 0.0557& -0.0540\\ 0& -0.0668& 0.0648\end{array}\right]\end{array}$$

$$F^3=Ftm{p}_1+Ftm{p}_2=\left[\begin{array}{ccc}0.0254& 0.0984& -0.0540\\ 0.0684& -0.1289& 0.0648\end{array}\right]$$

From $F^3$, we have the sensitivity function

$$p(x_1^3,{\mathbf{y}}_{\mathbf{e}}^{1:3})({\theta }_a)=0.0254+0.0984·{\theta }_a-0.054·{\theta }_a^2,$$

and from $F^2$,

$$p(x_2^2,{\mathbf{y}}_{\mathbf{e}}^{1:2})({\theta }_a)=0.0743-0.072·{\theta }_a$$

The sensitivity functions for probability of evidence by summing column entries of the forward matrices become:

$$p({\mathbf{y}}_{\mathbf{e}}^{1:3})({\theta }_a)=(F_{1,1}^3+F_{2,1}^3)+(F_{1,2}^3+F_{2,2}^3)·{\theta }_a+(F_{1,3}^3+F_{2,3}^3)·{\theta }_a^2,$$

and

$$p({\mathbf{y}}_{\mathbf{e}}^{1:2})({\theta }_a)=(F_{1,1}^2+F_{2,1}^2)+(F_{1,2}^2+F_{2,2}^2)·{\theta }_a$$

The sensitivity functions for the two filtering tasks become:

$$p(x_1^3|{\mathbf{y}}_{\mathbf{e}}^{1:3})({\theta }_a)=\frac{-0.054·{\theta }_a^2+0.0984·{\theta }_a+0.0254}{0.0108·{\theta }_a^2-0.0305·{\theta }_a+0.0938},$$

and

$$p(x_2^2|{\mathbf{y}}_{\mathbf{e}}^{1:2})({\theta }_a)=\frac{-0.072·{\theta }_a+0.0743}{-0.012·{\theta }_a+0.1099},$$

these sensitivity functions are displayed in Figure 2.

Figure 2. Sensitivity functions: (a) $p(X^2|{\mathbf{y}}_{\mathbf{e}}^{1:2})({\theta }_a)$ for both states of $X^2$. (b) $p(X^3|{\mathbf{y}}_{\mathbf{e}}^{1:3})({\theta }_a)$ for both states of $X^3$.

The derivative of these sensitivity functions can be used to analyze the sensitive of the filtering tasks for the change in parameter ${\theta }_a$.

$$\frac{dp(x_1^3|{\mathbf{y}}_{\mathbf{e}}^{1:3})({\theta }_a)}{d{\theta }_a}=\frac{-0.006·{\theta }_a^2-0.0107·{\theta }_a-0.0094}{{(0.0108·{\theta }_a^2-0.0305·{\theta }_a+0.0938)}^2},$$

and

$$\frac{dp(x_2^2|{\mathbf{y}}_{\mathbf{e}}^{1:2})({\theta }_a)}{d{\theta }_a}=\frac{-0.0070}{{(-0.012·{\theta }_a+0.1099)}^2}$$

3.5. Run-time Analysis

The time for computing the coefficients of the filtering probabilities for transition parameter variation is recorded for the Coefficient-Matrix-Fill procedure and our method (SMF) using the HMM shown in Example (1). It is shown in Figure 3, where the Coefficient-Matrix-Fill procedure is labled as CMFP. In [26], the algorithm for the Coefficient-Matrix-Fill procedure is presented. It is shown for an HMM model with $n=2$ hidden states and $m=2$ observable symbols. The proposed SMF algorithm has shown a significant improvement in computational time compared to the Coefficient-Matrix-Fill procedure as the length of the observation sequence increases. For example, for an observation sequence length of 1000, the proposed technique takes a mean time of 0.048 s. In comparison, it takes the Coefficient-Matrix-Fill procedure 2.183 seconds. This is an improvement of 98%. The algorithms are implemented in MATLAB on a 64-bit Windows 7 Professional Workstation (Dell Precision T7600 with Intel(R) Xeon CPU E5-2680 0 @ 2.70 GHz dual core processors and 64GB RAM).

Figure 3. Time in seconds to compute the sensitivity coefficients for an observation sequence length from 1 to 1000 with a step size of 10.

The computational complexity of our method is linear time, $O(l)$, where l is the length of the observation sequence, whereas the Coefficient-Matrix-Fill procedure is quadratic time, $O(l^2)$. In our method, the sensitivity function coefficients are computed in one for loop that runs for the length of the observation sequence, as shown in Algorithms 1–6, whereas in the Coefficient-Matrix-Fill procedure two nested for loops are used. It computes the sensitivity coefficients in the forward matrices element-by-element in the inner loop, which runs in increasing time up to the maximum of the length of the observation sequence, and the outer loop runs for the length of the observation sequence.

4. Sensitivity Analysis of Smoothing Probability

In this section, the sensitivity analysis of smoothing probability for variation of the transition, initial and observation parameters are presented using simplified matrix formulation. The sensitivity function $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})(\theta )$ where $t<T$ and $\theta$ is one of the model parameters, can be computed using the recursive probability expression shown in Equation (2) and (3).

$$\begin{array}{cc}\hfill p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})(\theta )& =p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t},{\mathbf{y}}_{\mathbf{e}}^{t+1:T})(\theta )=p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)(\theta )·p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})(\theta )\hfill \end{array}$$

The first term in this product can be computed using the backward procedure as shown in Equation (3); the second term is the filtering probability that is computed using the forward procedure. Consequently, the sensitivity function for the smoothing probability $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})(\theta )$ is the product of the polynomial sensitivity functions computed using the forward and backward procedures. The sensitivity function for the filtering probability $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:t})(\theta )$ is discussed in Section 3. From Equation (3), the backward recursive probability expression, it follows that

$$p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)(\theta )=\left\{\begin{array}{ll}{\sum }_{z=1}^n{o}_{z,{\mathbf{e}}^{\mathbf{T}}}(\theta )·a_{v,z}(\theta )& \mathrm{if}t=T-1\\ {\sum }_{z=1}^n{o}_{z,{\mathbf{e}}^{\mathbf{t}+\mathbf{1}}}(\theta )·a_{v,z}(\theta )·p({\mathbf{y}}_{\mathbf{e}}^{t+2:T}|x_z^{t+1})(\theta )\hfill & \mathrm{if}t<T-1,\end{array}\right.$$

(10)

where

$$a_{v,z}(\theta )=\left\{\begin{array}{cc}{a}_{v,z}({\theta }_a)\hfill & \mathrm{for}transitionparametervariation\theta ={\theta }_a\hfill \\ a_{v,z}\hfill & otherwise,\hfill \end{array}\right.$$

and

$$o_{z,{\mathbf{e}}^{\mathbf{t}}}(\theta )=\left\{\begin{array}{cc}{o}_{z,{\mathbf{e}}^{\mathbf{t}}}({\theta }_o)\hfill & \mathrm{for}observationparametervariation\theta ={\theta }_o\hfill \\ o_{z,{\mathbf{e}}^{\mathbf{t}}}\hfill & otherwise\hfill \end{array}\right.$$

To compute the coefficients of the sensitivity function $p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)(\theta )$, a series of Backward matrices $B^k,k=t,...,T-1,$ based on the simplified matrix formulation as presented in Section 3 are used. In the next subsections, the sensitivity functions for the smoothing probability $p(x_v^t,{\mathbf{y}}_{\mathbf{e}}^{1:T})(\theta )$ for transition, initial and observation parameter variations are discussed.

4.1. Transition Parameter Variation

In the simplified matrix formulation, the sensitivity function $p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_a)$ for transition parameter ${\theta }_a=a_{r,s}$ can be represented as follows:

$$p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_a)=A({\theta }_a)O^{t+1}p({\mathbf{y}}_{\mathbf{e}}^{t+2:T}|x_z^{t+1})({\theta }_a)$$

For $t=T-1$, $p({\mathbf{y}}_{\mathbf{e}}^{T+1:T}|x_z^T)({\theta }_a)=1$, the recursive probability expression reduces to the following:

$$p({\mathbf{y}}_{\mathbf{e}}^{T:T}|x_v^{T-1})({\theta }_a)=A({\theta }_a)O^{t+1}$$

The proposed technique for the sensitivity analysis of the backward probability sensitivity function $p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_a)$ for the transition parameter is formulated in Algorithm 4.

Algorithm 4 This computes the coefficients of the backward probability sensitivity function $p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_a)$ in backward matrices $B^k$ for $k=T,\cdots ,t$ using SMF.

Input: $A,O,\Gamma ,\mathbf{e},\mathrm{and}{\theta }_a$

Output: $B^T,\cdots ,B^t$

1: $\overline{A}\leftarrow A$ 2: ${\overline{A}}_{r,s}\leftarrow 0$ 3: ${\overline{A}}_{r,:}\leftarrow {\overline{A}}_{r,:}./(1-a_{r,s})$ 4: $\widehat{A}\leftarrow A-\overline{A}$ 5: ${\widehat{A}}_{r,:}\leftarrow {\widehat{A}}_{r,:}./a_{r,s}$ 6: $B^T\leftarrow ones(n,1)$ 7: for $k=T-1\mathrm{to}t$do 8:     $O^{k+1}\leftarrow diag(O_{:,{\mathbf{e}}^{\mathbf{k}+\mathbf{1}}})$ 9:     $Btm{p}_1\leftarrow [\overline{A}*O^{k+1}*B^{k+1},zeros(n,1)]$ 10:     $Btm{p}_2\leftarrow [zeros(n,1),\widehat{A}*O^{k+1}*B^{k+1}]$ 11:     $B^k\leftarrow Btm{p}_1+Btm{p}_2$ 12: Return $B^T,\cdots ,B^t$

4.1.1. Algorithm Implementation

In this algorithm, e is the sequence of observation and $\theta ={\theta }_a$. It solves the recursive backward probability sensitivity function Equation (10) using the backward matrices $B^k$ for $k=T,\cdots ,t$. Once the transition matrix is decomposed into the transition matrices $\overline{A}$ independent of ${\theta }_a$ and $\widehat{A}$ dependent on ${\theta }_a$, the coefficients of the sensitivity function are computed by filling the backward matrices $B^k$ as follows. First, $B^T$ is initialized as a vector of ones for n hidden states.

$$B^T=ones(n,1)$$

The remaining matrices $B^k$ of size $n\times k,k=T-1\le k\le t$ are computed by filling them as follows: Once the observation matrix $O^{k+1}$ is represented as a diagonal matrix, two temporary matrices $Btm{p}_1$ and $Btm{p}_2$ are used for computing the coefficients as previously explained. $Btm{p}_1$ is computed as the matrix multiplication of $\overline{A}$ that is independent of ${\theta }_a$, $O^{k+1}$ and $B^{k+1}$ and a zero vector of size n is appended at its end. $B^{k+1}$ represents the recursion in the backward probability. The same way $Btm{p}_2$ is computed as the matrix multiplication of $\widehat{A}$ that is dependent on ${\theta }_a$, $O^{k+1}$ and $B^{k+1}$ and a zero vector of size n is appended to the front. Then $B^k$ is set as the sum of $Btm{p}_1$ and $Btm{p}_2$,

$$B^k=Btm{p}_1+Btm{p}_2$$

4.2. Initial Parameter Variation

In this subsection, the sensitivity of the backward probability to initial parameter variation is derived based on SMF. The sensitivity function $p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_{\gamma })$ for initial parameter variation ${\theta }_{\gamma }={\gamma }_r$, as shown in the recursive backward probability sensitivity function in Equation (10), can be represented in SMF as

$$p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_{\gamma })=A{O}^{t+1}p({\mathbf{y}}_{\mathbf{e}}^{t+2:T}|x_z^{t+1})({\theta }_{\gamma })$$

For $t=T-1$, $p({\mathbf{y}}_{\mathbf{e}}^{T+1:T}|x_z^T)({\theta }_{\gamma })=1$ and the recursive probability expression reduces to

$$p({\mathbf{y}}_{\mathbf{e}}^{T:T}|x_v^{T-1})({\theta }_{\gamma })=A{O}^{t+1}$$

The procedure to compute the coefficients of the function $p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_{\gamma })$ in backward matrices $B^k$ for $k=T,\cdots ,t$ using SMF is summarized in Algorithm 5.

Algorithm 5 This computes the coefficients of the backward probability sensitivity function $p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_{\gamma })$ with ${\theta }_{\gamma }={\gamma }_r$ in backward matrices $B^k$ for $k=T,\cdots ,t$ using SMF.

Input: $A,O,\Gamma ,\mathbf{e},\mathrm{and}{\theta }_{\gamma }$

Output: $B^T,\cdots ,B^t$

1: $B^T\leftarrow ones(n,1)$ 2: for $k=T-1\mathrm{to}t$ do 3:     $O^{k+1}\leftarrow diag(O_{:,{\mathbf{e}}^{\mathbf{k}+\mathbf{1}}})$ 4:     $B^k\leftarrow A*O^{k+1}*B^{k+1}$ 5: Return $B^T,\cdots ,B^t$

4.2.1. Algorithm Implementation

As discussed above, the sensitivity function for the backward probability $p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_{\gamma })$ with ${\theta }_{\gamma }={\gamma }_r$ is a polynomial of degree zero. The contents of the backward matrices $B^k$ are filled as follows. $B^T$ is initialized as a vector of ones for n hidden states.

$$B^T=ones(n,1)$$

The remaining matrices $B^k$ of size $n\times k,k=T-1\le k\le t,$ are computed by filling them as follows. Once the observation matrix $O^{k+1}$ is represented as a diagonal matrix, $B^k$ is set as the matrix multiplication of A, $O^{k+1}$ and $B^{k+1}$. $B^{k+1}$ represents the recursion in the backward probability.

4.3. Observation Parameter Variation

The sensitivity function for the backward probability $p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_o)$ for the observation parameter variation ${\theta }_o=o_{r,s}$ can be represented as follows:

$$p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_o)=A{O}^{t+1}({\theta }_o)p({\mathbf{y}}_{\mathbf{e}}^{t+2:T}|x_z^{t+1})({\theta }_o)$$

For $t=T-1$, $p({\mathbf{y}}_{\mathbf{e}}^{T+1:T}|x_z^T)({\theta }_o)=1$, the recursive probability expression reduces to the following

$$p({\mathbf{y}}_{\mathbf{e}}^{T:T}|x_v^{T-1})({\theta }_o)=A{O}^{t+1}({\theta }_o)$$

The sensitivity analysis of the backward probability sensitivity function $p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_o)$ for observation parameter variation ${\theta }_o=o_{r,s}$ is summarized in Algorithm 6.

Algorithm 6 This computes the coefficients of the backward probability sensitivity function $p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_o)$ with ${\theta }_o=o_{r,s}$ in backward matrices $B^k$ for $k=T,\cdots ,t$ using SMF.

Input: $A,O,\Gamma ,\mathbf{e},\mathrm{and}{\theta }_o$

Output: $B^T,\cdots ,B^t$

1: $\overline{O}\leftarrow O$ 2: ${\overline{O}}_{r,s}\leftarrow 0$ 3: ${\overline{O}}_{r,:}\leftarrow {\overline{O}}_{r,:}./(1-o_{r,s})$ 4: $\widehat{O}\leftarrow O-\overline{O}$ 5: ${\widehat{O}}_{r,:}\leftarrow {\widehat{O}}_{r,:}./o_{r,s}$ 6: $B^T\leftarrow ones(n,1)$ 7: for $k=T-1\mathrm{to}t$ do 8:     ${\overline{O}}^{k+1}\leftarrow diag({\overline{O}}_{:,{\mathbf{e}}^{\mathbf{k}+\mathbf{1}}})$ 9:     ${\widehat{O}}^{k+1}\leftarrow diag({\widehat{O}}_{:,{\mathbf{e}}^{\mathbf{k}+\mathbf{1}}})$ 10:     $Btm{p}_1\leftarrow [A*{\overline{O}}^{k+1}*B^{k+1},zeros(n,1)]$ 11:     $Btm{p}_2\leftarrow [zeros(n,1),A*{\widehat{O}}^{k+1}*B^{k+1}]$ 12:     $B^k\leftarrow Btm{p}_1+Btm{p}_2$ 13: Return $B^T,\cdots ,B^t$

4.3.1. Algorithm Implementation

In this algorithm, e is the sequence of observation and $\theta ={\theta }_o$ and it solves the recursive backward probability sensitivity function Equation (10) in the backward matrices $B^k$ for $k=T,\cdots ,t$ using SMF. Once the observation matrix is decomposed into the independent $\overline{O}$ and dependent $\widehat{O}$ observation matrices on ${\theta }_o$, the coefficients of the sensitivity function $p({\mathbf{y}}_{\mathbf{e}}^{t+1:T}|x_v^t)({\theta }_o)$ are computed by filling the backward matrices $B^k$ as follows: $B^T$ is initialized as a vector of ones for n hidden states.

$$B^T=ones(n,1)$$

The remaining matrices $B^k$ of size $n\times k,k=T-1\le k\le t,$ are computed by filling them as follows: Once the independent ${\overline{O}}^{k+1}$ and dependent ${\widehat{O}}^{k+1}$ observation matrices on ${\theta }_o$ is represented as a diagonal matrix, two temporary matrices $Btm{p}_1$ and $Btm{p}_2$ are used for computing the coefficients of the sensitivity function as previously explained. $Btm{p}_1$ is computed as the matrix multiplication of A, ${\overline{O}}^{k+1}$ and $B^{k+1}$. A zero vector of size n is appended at its end. $B^{k+1}$ represents the recursion in the backward probability. In the same way $Btm{p}_2$ is computed as the matrix multiplication of A, ${\widehat{O}}^{k+1}$ and $B^{k+1}$ and a zero vector of size n is appended to the front. Then $B^k$ is set as the sum of $Btm{p}_1$ and $Btm{p}_2$,

$$B^k=Btm{p}_1+Btm{p}_2$$

5. Conclusions and Future Research

This research has shown that it is more efficient to compute the coefficients for the HMM sensitivity function directly from the HMM representation. The proposed method exploits the simplified matrix formulation for HMMs. A simple algorithm is presented which computes the coefficients for the sensitivity function of filtering and smoothing probabilities for transition, initial and observation parameter variation for all hidden states, as well as all time steps. This method differs from the other approaches in that it neither depends on a specific computational architecture nor requires a Bayesian network representation of the HMM.

The future extension of this work will include sensitivity analysis of predicted future observations $p(y_e^t|y_e^{1:T})(\theta ),t>T$, and the most probable explanation for the corresponding parameter variations. Future research on the sensitivity analysis of HMM, where different types of model parameters are varied simultaneously, as well as the case of continuous observations will be considered.