Amortized Parameter Inference for the Arbitrary-Order Hidden Markov Model

Abstract

The arbitrary-order hidden Markov model ($\alpha$-HMM) is a nontrivial generalization of the standard HMM, designed to model stochastic processes with higher-order dependences among arbitrarily distant random events. The $\alpha$-HMM admits an efficient Viterbi-style optimal decoding algorithm, making it feasible to discover higher-order dependences among data objects in observed sequential data. Because the $\alpha$-HMM exceeds the expressive power of standard HMMs, fixed $k\mathrm{th}$-order HMMs, and stochastic context-free grammars, effective probabilistic parameter estimation approaches are required to translate this theoretical expressiveness of the $\alpha$-HMM into practical utility. This paper introduces a principled methodology for effective estimation of probabilistic parameters of the $\alpha$-HMM from observed data. In large-scale sequential datasets, higher-order dependencies can vary widely across instances, so a single global parameter set may be inadequate. Instead, an amortized parameter inference approach is proposed for the $\alpha$-HMM, in which an input-conditioned parameter estimator is learned from data and used to infer instance-specific parameters for each input instance to the decoding algorithm. Specifically, the neural parameter estimator is trained using a composite learning objective that is partially enabled by the optimal decoding algorithm. The effectiveness of the proposed parameter estimation method is demonstrated through empirical results of the application of the $\alpha$-HMM in biomolecular structure modeling and prediction.

1. Introduction

The hidden Markov model (HMM) is a probabilistic graphical model well-suited for stochastic processes and has achieved broad success in applications involving inference over time-series and sequential data [1,2,3]. While efficient inference via the linear-time Viterbi’s algorithm enables tractable decoding of latent state sequences from observed data [4], the practical effectiveness of the HMM depends critically on the availability of reliable methods for parameter estimation. In particular, well-established learning techniques, in particular, maximum likelihood estimation for fully observed training data and the expectation–maximization-based Baum–Welch algorithm for data with latent or missing structures, make it possible for HMM parameters to be robustly learned from data [5,6]. It is this combination, and especially the maturity of parameter estimation methodologies, that has translated the HMM’s theoretical formulation into a widely adopted modeling framework.

While the standard HMM is effective at modeling local dependencies, applications of sequential data may require inference of richer structural information that goes beyond simple stochastic processes. Such structures may arise in diverse settings, such as syntactic and semantic dependencies in linguistic sequences [7,8], bio-residue interactions that determine biomolecular structure and function [9,10], and latent patterns in time-series data such as weather forecasting and market prediction [11,12,13,14]. While such higher-order dependences between random events have been modeled with extensions to the standard HMM [15,16], they rarely achieve the same level of practical effectiveness as the standard HMM. Due to the increased model complexity, parameter estimation becomes substantially harder than in the standard HMM; the richer structural dependencies dramatically increase the number of parameters and complicate latent inference. In particular, learning in $k\mathrm{th}$-order HMMs suffers from exponential parameter growth and data sparsity [17,18], while stochastic context-free grammar (SCFG) [19] training is computationally expensive and statistically ineffective [20,21].

More recently, the arbitrary-order hidden Markov model ($\alpha$-HMM) [22] has been introduced as a generalization of the standard HMM with the expressiveness to overcome the shortcomings of previous extensions to the HMM, e.g., the $k\mathrm{th}$-order HMM and SCFG. The $\alpha$-HMM  is equipped with a mechanism to express dependences between events in arbitrary distances with a well-defined joint probability distribution over an observable sequence of data and hidden events that generate the data. Such dependences may be nested, parallel, and crossing patterns, some of which are beyond what SCFG is capable of characterizing. One significant utility of the $\alpha$-HMM, as with the standard HMM, is its ability to uncover higher-order dependencies among latent events through decoding. Though the task of decoding with the full strength of the $\alpha$-HMM  can be computationally intractable, it has been shown that prediction of an optimal set of dependences induced by exclusive pairs of hidden events can be accomplished much more efficiently, in $\mathcal{O}\left(n^3\right)$ time complexity, on observed sequences of n data items [22].

To translate the theoretical potential of the $\alpha$-HMM into practical utility, its decoding algorithm must be paired with an effective method for probabilistic parameter estimation. However, because the $\alpha$-HMM is strictly more expressive than $k\mathrm{th}$-order HMMs and stochastic context-free grammars, parameter estimation methods developed for those models are no longer adequate. Moreover, with large-scale datasets, higher-order dependency structures can vary substantially across instances, making a single global parameter set insufficient. To address these challenges, we propose an amortized learning method [23,24] that enables scalable, instance-specific parameter inference and fully exploits the modeling capacity of the $\alpha$-HMM. Rather than estimating a single set of global parameters shared by all sequences, we learn an input-conditioned parameter estimator from data which generates instance-specific $\alpha$-HMM parameters for each input sequence of data whose dependence structures are to be predicted.

Specifically, the amortized parameter estimator is trained to infer parameters by optimizing a composite objective that integrates global and local structural constraints on the latent-event sequence. One term encourages the reference hidden-event path to receive a high score under the inferred parameters. A second term applies supervision to the reference hidden-state sequence, helping stabilize locally consistent state-transition behavior. A third term introduces a structured margin that separates the reference path from competing paths produced by Viterbi-style decoding under the same parameters. A fourth, task-specific term further guides the model toward the desired structured output. Together, these terms favor parameter settings that support correct global decoding, locally consistent latent states, and task-relevant structural predictions. Evaluated on RNA secondary structure prediction, the proposed framework achieves strong empirical performance competitive with state-of-the-art methods.

The rest of the paper is organized as follows. Section 2 reviews the hidden Markov model and introduces some fundamentals of machine learning. Section 3 gives a thorough description of the $\alpha$-HMM ’s optimal decoding algorithm, and its expressiveness. Section 4 presents the amortized parameter inference framework in detail. Section 5 demonstrates the performance of the proposed work through comparisons with the-state-of-art methods on RNA secondary structure prediction. We conclude in Section 6 with some discussions.

2. Fundamentals

In this section, we review some fundamentals of the hidden Markov model (HMM) and in machine learning that are necessary for further reading of the $\alpha$-HMM ’s decoding algorithm, and the parameter estimation method to be introduced in later sections. We refer the reader to [2,25] for more detailed information.

2.1. Hidden Markov Model

A stochastic process is a family of random variables $\{X_k:k\in \mathbb{N}\}$, where $\mathbb{N}$ is the natural number set and random variables $X_k$ are defined over the same probability space.

Definition 1.

A discrete hidden Markov model (HMM) [5] is a bivariate stochastic process $\{(X_k,Y_k):k\in \mathbb{N}\}$, where $X_k$ draws values from a finite set S of states and $Y_k$ draws values from a finite set Σ of symbols, such that, for every finite $n\ge 1$, the joint probability $P\left(\{(X_k,Y_k):k\le n\}\right)$ is defined as

$$P\left(\{(X_k,Y_k):k\le n\}\right)=P\left(X_1\right)\prod _{k=1}^{n}P\left(Y_k\right|X_k)\prod _{k=1}^{n-1}P\left(X_{k+1}\right|X_k)$$

(1)

Note that Equation (1) is derived based on the local Markov property as well as the chain rule of probability theory. Graphically, an HMM is represented as a probabilistic finite state machine $(S,\Sigma ,ϵ,\tau )$ in which each state $r\in S$ can emit a symbol $a\in \Sigma$ with probability ${ϵ}_r\left(a\right)$ of distribution ${\sum }_{a\in \Sigma }{ϵ}_r\left(a\right)=1$. Every state r can transit to another state s with probability ${\tau }_r\left(s\right)$ of distribution ${\sum }_{s\in S}{\tau }_r\left(s\right)=1$. Specifically, for any symbol $a\in \Sigma$ and any state $r\in S$, the conditional probability $P(Y_k=a|X_k=r)$ in (1) is defined as ${ϵ}_r\left(a\right)$, and conditional probability $P(X_{k+1}=s|X_k=r)$ is defined as ${\tau }_r\left(s\right)$ for any state $s\in S$. The distributions $ϵ$ and $\tau$ are the parameters that completely determine the HMM model.

A typical task associated with the HMM is the decoding of hidden states $\{X_k^{\ast }:k\le n\}$ from an observed finite sequences of symbols $\{Y_k:k\le n\}$ with the following optimization

$$\{X_k^{\ast }:k\le n\}=arg\underset{\{X_k:k\le n\}}{max}P\left(\{Y_k:k\le n\}|\{X_k:k\le n\}\right)$$

which can be computed with the $O\left(n\right)$-time dynamic programming Viterbi algorithm [4] by taking the advantage probability factorization in Equation (1).

To deploy the decoding algorithm for practical applications, the parameter $\theta =(ϵ,\tau )$ of an HMM model needs to be known. While the accurate values of the the parameters may not always be known, estimation of parameter values from training data can be achieved. Depending on availability of data, there are broadly two different approaches for parameter estimation associated with the HMM. When training data contain both observed sequences of symbols and the hidden states that have generated them, distributions $ϵ$ and $\tau$ can be computed with the maximum likelihood method through counting of all the emissions and transitions labeled in the training data and assigning the frequencies to the probabilistic parameters.

A difficult situation arises when the states emitting observable symbols are not known or not included in the training data. To deal with this scenario of missing data, typically, the expectation maximization (EM) method is used to compute expected frequencies of event occurrences. Technically, EM is an iterative process consisting of alternate steps to compute expectation and to maximize it. Given the current parameter ${\theta }^{\left(t\right)}$, the expected log-likelihood $P(\mathbf{X},\mathbf{Y}|\theta )$ of $\theta$ over all possible values of hidden variables in $\mathbf{X}$ (drawn from the training data) is computed and maximized for new parameters as

$$\begin{array}{cc}\hfill {\theta }^{(t+1)}& =arg\underset{\theta }{max}{E}_{\mathbf{X}\sim P(·|\mathbf{Y},{\theta }^{\left(t\right)})}\left[logP(\mathbf{X},\mathbf{Y}|\theta )\right]\hfill \\ & =arg\underset{\theta }{max}\sum _{\mathbf{X}}logP\left(\mathbf{X},\mathbf{Y}\left|\theta \right)P\left(\mathbf{X}\right|\mathbf{Y},{\theta }^{\left(t\right)}\right)\hfill \end{array}$$

For the HMM, the iterative process is efficiently realized by Baum–Welch algorithm [6] that coordinates two processes forward and backward, computing the total probability ${\sum }_{\mathbf{X}}P(\mathbf{X},\mathbf{Y})$. The algorithm guarantees converging to a local optima.

2.2. Deep-Learning Preliminaries

Deep learning is a state-of-the-art technique to approximate functions of complex relationships between inputs and outputs via neural networks [26]. A neural network consists of stacked layers of computational units, each computing the weighted sum of its inputs followed by a nonlinear transformation. In practice, earlier layers often capture simple or local patterns of abstract, while deeper layers combine them to encode more global and structured information.

A neural network is learned through training on available data by settling those parameters associated with the computation units in the network. They are determined by minimizing an empirical objective function that quantifies the discrepancy between the network’s predicted outputs and observed data. Arising from probabilistic modeling, the most common choice of objective, among others, is the negative log-likelihood of the data under a parameterized conditional distribution. Due to the typical non-convexity and high-dimensionality of the optimization problem, the network parameters are computed with iterative gradient-based methods via backpropagation that is reasonably efficient to achieve gradients [25].

The following are some basic functions which will also be used in our proposed $\alpha$-HMM  neural parameter estimator to be introduced in Section 4.

• Unit-level functions
  sigmoid: The sigmoid function transforms a real number (e.g., output of a neural gate) to a probability in interval $(0,1)$ for every $t\in \mathbb{R}$,

  $$\sigma \left(t\right)={\displaystyle \frac{1}{1+exp(-t)}}\in (0,1)$$
  softplus: The softplus function maps a real number to a nonnegative value for every $t\in \mathbb{R}$,

  $$\mathrm{softplus}\left(t\right)=log(1+exp\left(t\right))\in {\mathbb{R}}_{\ge 0}$$
• Layer-level functions
  MLP [27]: An MLP is multilayer perceptron that defines mapping: ${\mathbb{R}}^d\to {\mathbb{R}}^m$ from an input layer to the output layer with one or more hidden layers in between, where neurons in two consecutive layers are fully connected.
  softmax: The softmax function maps a vector $u\in {\mathbb{R}}^K$ to a probability vector $p\in {[0,1]}^K$, such that, for $k=1,\dots ,K$,

  $$p_k=\mathrm{softmax}\left(u_k\right)={\displaystyle \frac{exp\left(u_k\right)}{{\sum }_{j=1}^{K}exp\left(u_j\right)}}.$$

  satisfying ${\sum }_{k=1}^K{p}_k=1$. It is typically used as a normalization operator and applied at the output layer only.  
• Functions on sequences with embeddings
  Transformer encoder [28]: A Transformer encoder is a mapping function: $T_{\theta }:{\mathbb{R}}^{L\times d}\to {\mathbb{R}}^{L\times d}$ from an input length-L sequence of vectors to a sequence of contextualized representations of the same length. Each input vector (of d entries) is an embedding (semantics) of the corresponding input position, while each output vector is a weighted combination of all input positions (computed using self-attention).
  Attention pooling: An attention pooling function, ${\mathbb{R}}^{L\times d}\to {\mathbb{R}}^d$, computes a weighted summary of a sequence of vectors, where the weights are learned to emphasize the most relevant elements. It assigns higher weight to positions that are more informative for the task, represented with aggregated values that focus on what matters most.

3. Arbitrary-Order Hidden Markov Model

3.1. $\alpha$-HMM

The HMM is extended to encompass a framework for modeling higher-order dependences that go beyond linear relationships between consecutive events in stochastic processes. Technically, the dependency between two states can be specified via a distribution of the joint probability of data emissions by the two states [22].

Definition 2.

An influence is an ordered pair of states $\langle r,s\rangle$, $r,s\in S$ with probability distribution ${\eta }_{r,s}^a$ such that for every fixed $a\in \Sigma$,

$$\sum _{b\in \Sigma }{\eta }_{r,s}^a\left(b\right)=1$$

(2)

Let $\left\{(X_k,Y_k|k\in \mathbb{N})\right\}$ be a stochastic process where $X_i$ and $X_j$, $i<j$, are two variables. Then $X_i$ is a parent of $X_j$ if there is an influence $\langle r,s\rangle$ such that $X_j=s$, $X_i=r$, and i is the largest integer.

In addition, we denote with $\pi \left(X_j\right)=\{X_i:i<j,X_i \mathrm{is} \mathrm{a} \mathrm{parent} \mathrm{of} X_j\}$ the set of parents of $X_j$. Also let $\delta \left(X_i\right)=\{X_k:X_i\in \pi \left(X_k\right)\}$ represent the set of decedents of $X_i$.

Definition 3.

An arbitrary-order hidden Markov model (α-HMM) is a bivariate stochastic process $\{(X_k,Y_k):k\in \mathbb{N}\}$, where $X_k$ draws values from a finite set S of states and $Y_k$ draws values from a finite set Σ of symbols, such that, for finite $n\ge 1$, the joint probability $P\left(\{(X_k,Y_k):k\le n\}\right)$ is defined as

$$P\left(\{(X_k,Y_k):k\le n\}\right)=P\left(X_1\right)\prod _{k=1}^{n-1}P\left(X_{k+1}|X_k\right)\prod _{k=1}^{n}P\left(Y_k|X_k,Y_i,X_i\in \pi \left(X_k\right)\right)$$

(3)

We point out that Equation (3) does not specify how the last conditional probability term $P\left(Y_k|X_k,Y_i,X_i\in \pi \left(X_k\right)\right)$ is computed. This resembles the Bayesian network setting, where different computation methods may be adopted. Since variables $Xi\in \pi \left(X_k\right)$ may be interdependent, one viable method is the Noisy-OR [29]. While it is a heuristic method, normalization with the partition function can be applied to ensure a well-defined probability distribution, as used in clique factorization in Markov random fields [30,31], assuming that influences on $X_k$ from its parent variables $X_i$ are independent. In the present work, however, we focus on the ${\alpha }_1$-HMM case. Accordingly,

$$P\left(Y_k|X_k,Y_i,X_i\in \pi \left(X_k\right)\right)=\prod _{X_i\in \pi \left(X_k\right)}P\left(Y_k\right|X_k,X_i,Y_i)$$

(4)

which is the assumption we adopt for this work.

By Definition 3, the conditional probability in (4) is defined as,

$$P(Y_k=b | X_k=s,X_i=r,Y_i=a)={\eta }_{r,s}^a\left(b\right)$$

Definition 4.

For fixed integer $h\ge 0$, an α-HMM is called an ${\alpha }_h$-HMM if for every state variable $X_k$ in the stochastic process, both $\left|\pi \right(X_k\left)\right|\le h$ and $\left|\delta \right(X_k\left)\right|\le h$.

In an ${\alpha }_h$-HMM, no state variable can influence or be influenced by more than h variables. Thus, the ${\alpha }_0$-HMM is exactly the standard HMM, where symbol $Y_k$ is completely determined by the its emission state $X_k$. The parameter h defines a family of ${\alpha }_h$-HMM models with different expressive levels. In this paper, we focus on the ${\alpha }_1$-HMM case, which is the specific model used in our decoding algorithm and experiments.

3.2. Expressiveness of the $\alpha$-HMM

The extension to the HMM with the new notion of influence has endowed modeling capability to the $\alpha$-HMM. First, due to Definition 2 (for influence from $X_i$ to $X_j$), where i, $i<j$, needs to be the largest, it becomes possible to model, with the ${\alpha }_1$-HMM , the influence of nested as well as parallel patterns that underline the essentials of SCFG. In addition, the ${\alpha }_1$-HMM  can also model influences of crossing patterns, exceeding SCFG in expressiveness [19]. Figure 1 (1) shows an example of the ${\alpha }_1$-HMM’s ability to model nested, parallel, and crossing patterns of influences.

Moreover, the ${\alpha }_1$-HMM  has the modeling power of the $k\mathrm{th}$-order HMMs, for any fixed $k\ge 1$. Figure 1 (2) shows an example of an ${\alpha }_1$-HMM  that is a canonical simulation of second-order HMMs. It is not difficult to see that similar examples of the ${\alpha }_1$-HMM  can be constructed to simulate the work of $k\mathrm{th}$-order HMMs, for any fixed $k\ge 3$.

3.3. Optimal Decoding with ${\alpha }_1$-HMM

One of the central tasks of modeling with the $\alpha$-HMM  is to decode the hidden states $\mathbf{X}$ that have generated an observed sequence $\mathbf{Y}$ of symbols over the alphabet $\Sigma$, such that the decoded hidden variables achieve the maximum likelihood ${\mathbf{X}}^{\ast }=arg{max}_{\mathbf{X}}P\left(\mathbf{Y}\right|\mathbf{X})$.

Definition 5.

For an α-HMM  of state set S and alphabet set Σ, the decoding problem is, given observation $\mathbf{a}=a_1\dots a_n\in {\Sigma }^n$, to find a sequence of states ${\rho }^{\ast }\in S^n$, such that

$${\rho }^{\ast }=arg\underset{\rho \in S^n}{max}P(\mathbf{X}=\rho ,\mathbf{Y}=\mathbf{a})$$

(5)

We call ${\rho }^{\ast }$ the predicted optimal path for observation $\mathbf{a}$. To solve Equation (5), we resort to Formulas (3) and (4), which make it possible to develop a dynamic programming algorithm for the optimal decoding.

Definition 6.

Let $1\le k\le n$, $r\in S$, and $0\le i<k$. Define $p_{k,r,i}$ as the maximum probability of a state-sequence $\rho =s_1\dots s_k$ in which $s_j$ emits symbol $a_j$, $j=1,\dots ,k$, $s_k=r$, $\langle s_i,s_k\rangle$ is an influence, and i is the largest.

By (3) the joint probability can be factorized; we derive a recurrence for $p_{k,r,i}$ as, in an observed sequence of symbols $\mathbf{a}=a_1\dots a_n$,

$$p_{k,r,i}={\eta }_{s_i,r}^{a_i}\left(a_k\right)·\underset{t\in S, 1\le h<k-1}{max}\{{\tau }_t\left(r\right)·p_{k-1,t,h}\}$$

(6)

The optimally decoded hidden state-sequence from the observed sequence $\mathbf{a}$ is

$${\rho }^{\ast }=arg\underset{r,i}{max}\left\{p_{n,r,i}\right\}$$

It is clear that a table of dimensions $n\times n\times \left|S\right|$ can be used to store values $p_{k,r,i}$, which can be computed by a dynamic programming algorithm with the worst case time complexity of $O\left(n^3\right|S{|}^2)$, where $\left|S\right|$ is the number of states in the $\alpha$-HMM and is a constant.

4. Amortized Parameter Inference for ${\mathbf{\alpha }}_{\mathbf{1}}$-HMM

Higher-order dependency structures modeled by the $\alpha$-HMM  can vary substantially across instances, especially in the present large dataset, making a single global parameter set insufficient. To address these challenges, we propose an amortized learning approach that enables scalable, instance-specific parameter inference and fully exploits the modeling capacity of the $\alpha$-HMM. Rather than estimating a single set of global parameters shared by all sequences, we learn an input-conditioned neural parameter estimator $f_{\theta }$ from data which generates instance-specific $\alpha$-HMM parameters for each input sequence of data whose dependence structures are to be predicted.

The input to an $\alpha$-HMM is a variable-length sequence $\mathbf{a}=a_1\dots a_n$ whose symbols are drawn from a fixed vocabulary (alphabet) $\Sigma$. The vocabulary size $|\Sigma |$ and the number of states $\left|S\right|$ determine the dimensionality of key model components: emission parameters are defined over $\Sigma$ (distributions of size $|\Sigma |$ per emitting state), and both pairwise influence and transition parameters form square tables in space ${\mathbb{R}}^{\left|S\right|\times \left|S\right|}$. Specifically, every influence is defined over an ordered pair of states $\langle r,s\rangle$, encoding a left-to-right influence that accounts for dependence between their symbol emissions.

For amortized learning, we first map each input sequence into a vectorized pre-encoder representation before feeding it into the estimator $f_{\theta }$. This vectorization can be implemented either by one-hot encoding, ${\mathbf{x}}_k=\mathbf{e}\left(a_k\right)\in {\{0,1\}}^{|\Sigma |}$, or by a learned embedding table that maps each symbol to a d-dimensional vector ${\mathbf{x}}_k\in {\mathbb{R}}^d$, $1\le k\le n$, where d is a user-specified hyperparameter. To preserve order information, we optionally add a positional embedding ${\mathbf{p}}_k$ so that the same symbol occurring at different positions can receive different representations. Positional embeddings can be implemented either as fixed sinusoidal features or as learned d-dimensional vectors. This yields a pre-encoder representation sequence $({\tilde{\mathbf{x}}}_1,\dots ,{\tilde{\mathbf{x}}}_n)$, where ${\tilde{\mathbf{x}}}_k={\mathbf{x}}_k+{\mathbf{p}}_k$. Overall, the above steps define a deterministic encoding from the input sequence to its pre-encoder representation.

However, both one-hot vectors and token embeddings are context-independent: the representation of a symbol depends only on its identity (and position, if positional encodings are used), but not on the surrounding symbols in the sequence. As a result, the same symbol occurring at the same position receives the same representation across different sequences, regardless of context. To incorporate contextual information, we apply a lightweight Transformer encoder. Its self-attention mechanism allows each position to aggregate information from the entire sequence, yielding context-dependent representations that are subsequently used by the parameter estimator. We denote the resulting contextual representation sequence by $\mathbf{H}={[{\mathbf{h}}_1,\dots ,{\mathbf{h}}_n]}^{\top }\in {\mathbb{R}}^{n\times d}$, where ${\mathbf{h}}_k\in {\mathbb{R}}^d$ is the contextual embedding at position k.

As discussed in Section 3, any specific $\alpha$-HMM is parameterized by the parameters $\theta =\{ϵ,\tau ,\eta \}$ for emission, transition, and influence parameters, respectively. We perform amortized inference with an input-conditioned estimator $f_{\theta }$ that maps an input sequence $\mathbf{a}$ to instance-specific parameters:

$$f_{\theta }\left(\mathbf{a}\right)=\left(ϵ\left(\mathbf{a}\right), \tau \left(\mathbf{a}\right), \eta \left(\mathbf{a}\right)\right).$$

We point out that designing transitions $\tau$ and influences $\eta$ is particularly important because they govern the complexity of the latent states and the overall modeling capacity. An overly simple model structure may lack the specificity in expressiveness required by the task, while an overly complex one can make inference and training unstable or data-inefficient. In the simplest setting, the total number of unconstrained outputs can be viewed as the sum of three parts: $|\Sigma |$ for emissions (of the involved state) per position, and ${|\Sigma |}^2$ for every pairwise influence, and an additional number of parameters for transitions (depending on the chosen transition structure and $\left|S\right|$, the number of states).

To map from the encoder space $\mathbf{H}$, we first apply an attention-based pooling layer to obtain a fixed-length summary vector $\overline{\mathbf{h}}$, which highlights informative positions in variable-length sequences. A multi-layer perceptron (MLP) then maps $\overline{\mathbf{h}}$ to three groups of unconstrained logits,

$${\mathrm{MLP}}_{\theta }\left(\overline{\mathbf{h}}\right)=\left({\mathbf{o}}^{\left(ϵ\right)}, {\mathbf{o}}^{\left(\tau \right)}, {\mathbf{o}}^{\left(\eta \right)}\right).$$

where $\mathbf{o}$ is the output of MLP.

Since $ϵ$ and $\eta$ represent (conditional) probability distributions, we apply softmax along the appropriate dimensions to project unconstrained logits into the corresponding probability space. For $\tau$, whose transition structure is user-specified and may vary across settings, we recommend a sigmoid-based constrained construction so that the resulting transition probabilities respect the prescribed structure and each row sums to one.

We train the amortized estimator $f_{\theta }$ end-to-end (including the encoder) on a dataset

$$\mathcal{D}={\left\{({\mathbf{a}}^{\left(i\right)},{\mathbf{y}}^{\left(i\right)},{\mathit{\rho }}^{\left(i\right)})\right\}}_{i=1}^N,$$

where ${\mathbf{a}}^{\left(i\right)}=a_1^{\left(i\right)}\dots a_{n_i}^{\left(i\right)}$ is the observed sequence, ${\mathbf{y}}^{\left(i\right)}$ is the task annotation, and ${\mathit{\rho }}^{\left(i\right)}={\rho }_1^{\left(i\right)}\dots {\rho }_{n_i}^{\left(i\right)}$ is a reference hidden-state sequence consistent with ${\mathbf{y}}^{\left(i\right)}$ under the task constraints.

We optimize $f_{\theta }$ with a composite objective consisting of four terms: (i) a path-level loss ${\mathcal{L}}_{\mathrm{path}}$ that encourages the reference hidden-event path $\rho$ to receive a high score under the inferred parameters; (ii) a state-sequence loss ${\mathcal{L}}_{\mathrm{state}}$ that applies supervision to the reference hidden-state sequence through its local state transitions, thereby helping stabilize the learning of locally consistent state-transition behavior; (iii) a structured adversarial loss ${\mathcal{L}}_{\mathrm{adv}}$ that enforces separation between the reference path $\rho$ and a competing decoded path ${\rho }^{\prime }$ under the current predicted parameters; and (iv) a task-specific structured loss ${\mathcal{L}}_{\mathrm{task}}$ that further guides the model toward the desired structured output.

We combine these losses with learnable uncertainty-style weights:

$$\mathcal{L}=\sum _{c\in \{\mathrm{path},\mathrm{state},\mathrm{adv},\mathrm{task}\}}exp(-s_c) {\mathcal{L}}_c+s_c,$$

where $\left\{s_c\right\}$ are scalar parameters learned jointly with $\theta$. Concrete task-specific instantiations of these four terms are given in the following section.

5. Performance Evaluation

We evaluate the proposed amortized parameter inference learning method for the $\alpha$-HMM on RNA secondary structure prediction, a well-studied problem in computational biology with a rich history of modeling approaches. Existing approaches to RNA secondary structure prediction often involve a trade-off between interpretability and predictive accuracy. In this section, we show that RNA secondary structure prediction can be formulated in a simple, accurate, and interpretable manner with the ${\alpha }_1$-HMM, thereby demonstrating the effectiveness of the proposed parameter inference method.

5.1. ${\alpha }_1$-HMM  for RNA Secondary Structure

A ribonucleic acid (RNA) consists of nucleotides adenine (A), guanine (G), cytosine (C), and uracil (U) that are chained together by covalent bonds to form the primary sequence. An RNA sequence can fold back itself via canonical Watson–Crick base pairs A-U and C-G, and the wobble base pair G-U via hydrogen bonds between the nucleotides. Consecutive, stacked base pairs, called stems, connected by unpaired single strands, called loops, constitute the secondary structure acting as a scaffold for its three-dimensional structure (Figure 2). Therefore, determination or prediction of RNA secondary structure is important to understanding the three-dimensional structures and thus functions of RNAs.

The problem of RNA secondary structure prediction is computationally discovering the most plausible secondary structure given only a primary sequence. In essence, this requires determining which (remote) pairs of nucleotides form exclusively canonical base pairs. Such a phenomenon can be well-captured by the expressive power of the ${\alpha }_1$-HMM. Specifically, with the ${\alpha }_1$-HMM, the nucleotides of an RNA molecule are modeled by emissions of states, the primary sequence of nucleotides connected by covalent bonds is modeled by the sequence of state transitions, and most importantly, the base pairs are modeled by influences between states and their emissions of nucleotides.

Figure 3B shows a general ${\alpha }_1$-HMM  graph for modeling RNA secondary structures. The model can capture all patterns of higher-level stem relationships in the RNA secondary structure, including nested, parallel, as in tRNA (Figure 2), and crossing (pseudoknot) as in Figure 3A. In addition, every stem, a stack of base pairs, is modeled with three influences ${\eta }_1, {\eta }_2, \mathrm{and} {\eta }_3$ from state L to states P, Q, and R, respectively, where the inner most base pair is influenced by ${\eta }_1$, the second inner most by ${\eta }_2$, and the third and beyond by ${\eta }_3$.

It is clear to see that hidden states in the ${\alpha }_1$-HMM  emitting an observable nucleotide sequence contain all the needed information for the secondary structure of the sequence. Thus, the structure prediction task is translated to a problem of decoding with the ${\alpha }_1$-HMM.

5.2. RNA-Specific Parameterization of the ${\alpha }_1$-HMM Estimator

To decode hidden states with the ${\alpha }_1$-HMM on an input RNA sequence $\mathbf{a}=a_1\dots a_n$ of length n, the transition, emission, and influence parameters $(\tau ,ϵ,\eta )$ are predicted by the learned parameter estimator $f_{\theta }$, such that

$$f_{\theta }\left(\mathbf{a}\right)=(\tau ,ϵ,\eta ).$$

From Figure 3, we formulate the following matrices:

$$\begin{array}{cc}\hfill \tau & =\left(\begin{array}{cccc}1-\Delta & \Delta & 0& 0\\ p_{\mathrm{leave}}& 0& 1-p_{\mathrm{leave}}& 0\\ q_{\mathrm{leave}}& 0& 0& 1-q_{\mathrm{leave}}\\ \alpha & \beta & 0& 1-\alpha -\beta \end{array}\right),\hspace{1em}ϵ=\left(\begin{array}{cccc}{ϵ}_A& {ϵ}_C& {ϵ}_G& {ϵ}_U\end{array}\right),\hspace{1em}\eta =\left(\begin{array}{cccc}{\eta }_{AA}& {\eta }_{AC}& {\eta }_{AG}& {\eta }_{AU}\\ {\eta }_{CA}& {\eta }_{CC}& {\eta }_{CG}& {\eta }_{CU}\\ {\eta }_{GA}& {\eta }_{GC}& {\eta }_{GG}& {\eta }_{GU}\\ {\eta }_{UA}& {\eta }_{UC}& {\eta }_{UG}& {\eta }_{UU}\end{array}\right).\hfill \end{array}$$

(7)

Here, $\tau$ is the transition probability matrix over states $L,P,Q,R$. In particular, ${\tau }_L\left(P\right)=\Delta$ and ${\tau }_L\left(L\right)=1-\Delta$, while the R-row of $\tau$ is parameterized by $\alpha ,\beta \ge 0$ with $\alpha +\beta \le 1$. In the RNA implementation used in this work, we further introduce two leave probabilities, $p_{\mathrm{leave}}$ and $q_{\mathrm{leave}}$, to control transitions from states P and Q, respectively.

Estimating $(\tau ,ϵ,\eta )$ by $f_{\theta }$ for the ${\alpha }_1$-HMM on each input RNA sequence $\mathbf{a}$ requires five scalars for $(\Delta ,\alpha ,\beta ,p_{\mathrm{leave}},q_{\mathrm{leave}})$, four logits for $ϵ$, and $4\times 4=16$ logits for $\eta$, for a total of $5+4+16=25$ sequence-specific parameters. Other quantities used during decoding, such as the start distribution and several state-dependent costs, are global trainable parameters shared across sequences.

To infer these 25 parameters, we first pass the RNA sequence through a neural encoder, which maps each nucleotide (A, C, G, U) to a learnable 640-dimensional embedding vector. The resulting embedding sequence is then processed by a three-layer Transformer encoder with rotary positional encoding, a hidden dimension of 640, 10 attention heads, a feed-forward dimension of $2\times 640=1280$ with GELU activations, and a dropout rate of $0.05$. The contextualized representation at each sequence position is then passed through a token-level trunk comprising a linear projection from 640 to 320 dimensions, a GELU activation, a layer-normalization layer, and one residual feed-forward block with hidden dimensions of 320 and a dropout rate of $0.05$. These token features are then aggregated by an attention-pooling layer to obtain a sequence-level representation, which is mapped to 25 sequence-specific parameters by a layer-normalization layer followed by a linear projection. For the tRNA experiments, we use a simplified instantiation of the model in which only $\Delta$, $\alpha$, and $\beta$ are predicted as sequence-specific transition parameters, while $p_{\mathrm{leave}}$ and $q_{\mathrm{leave}}$ are not inferred and are instead fixed to zero. Consequently, the tRNA variant predicts $3+4+16=23$ sequence-specific parameters rather than 25. The tRNA encoder is also smaller, using a two-layer Transformer with hidden dimensions of 128, feed-forward dimensions of 192, 4 attention heads, learned absolute positional embeddings, and a dropout rate of $0.1$. In both model variants, the three influence roles are parameter-tied through a single shared $4\times 4$ nucleotide-pair matrix $\eta$.

5.3. Training Objective for RNA Secondary Structure Prediction

We train the model using AdamW [32] with a learning rate of ${10}^{-4}$ and a weight decay of ${10}^{-2}$. Following the general loss formulation introduced in Section 4, we instantiate the RNA secondary structure objective as

$$\mathcal{L}=exp(-s_{\mathrm{path}}) {\mathcal{L}}_{\mathrm{path}}+s_{\mathrm{path}}+exp(-s_{\mathrm{state}}) {\mathcal{L}}_{\mathrm{state}}+s_{\mathrm{state}}$$

$$+exp(-s_{\mathrm{adv}}) {\mathcal{L}}_{\mathrm{adv}}+s_{\mathrm{adv}}+exp(-s_{\mathrm{task}}) {\mathcal{L}}_{\mathrm{task}}+s_{\mathrm{task}},$$

where $s_{\mathrm{path}}$, $s_{\mathrm{state}}$, $s_{\mathrm{adv}}$, and $s_{\mathrm{task}}$ are learnable scalar weights.

Let $\mathbf{a}$ denote an input RNA sequence, $\mathit{\rho }$ the reference hidden-state sequence, and $\mathbf{y}$ the reference dot-bracket secondary structure. Let $S_{\theta }(\mathbf{a},\mathit{\rho },\mathbf{y})$ denote the score of the reference state path under the sequence-specific parameters predicted by $f_{\theta }\left(\mathbf{a}\right)$. Then the path-level loss is defined as

$${\mathcal{L}}_{\mathrm{path}}=softplus(m_{\mathrm{path}}-S_{\theta }(\mathbf{a},\rho ,\mathbf{y})),$$

which encourages the reference path to receive a sufficiently high score.

The structured adversarial loss is defined as

$${\mathcal{L}}_{\mathrm{adv}}={\displaystyle \frac{1}{K}}\sum _{k=1}^{K}softplus(S_{\theta }(\mathbf{a},{\mathit{\rho }}_k^{\prime },{\mathbf{y}}_k^{\prime })-S_{\theta }(\mathbf{a},\mathit{\rho },\mathbf{y})+m_{\mathrm{adv}}),$$

where ${\left\{({\mathit{\rho }}_k^{\prime },{\mathbf{y}}_k^{\prime })\right\}}_{k=1}^K$ are decoded hard negative candidates and $m_{\mathrm{path}},m_{\mathrm{adv}}\ge 0$ are margin constants.

The state-sequence loss is defined as the negative log-likelihood of the reference hidden-state chain under the predicted start and transition distributions:

$${\mathcal{L}}_{\mathrm{state}}=-log{P}_{\theta }\left({\rho }_1\right)-\sum _{i=1}^{n-1}log{P}_{\theta }({\rho }_{i+1}\mid {\rho }_i).$$

In the RNA secondary structure setting, the task-specific term ${\mathcal{L}}_{\mathrm{task}}$ is instantiated as a pairwise base-pair supervision loss:

$${\mathcal{L}}_{\mathrm{task}}={\displaystyle \frac{1}{|{\mathcal{P}}^{+}|}}\sum _{(i,j)\in {\mathcal{P}}^{+}}softplus\left(-s_{\theta }(i,j)\right)+{\displaystyle \frac{1}{|{\mathcal{P}}^{-}|}}\sum _{(i,j)\in {\mathcal{P}}^{-}}softplus\left(s_{\theta }(i,j)\right),$$

where ${\mathcal{P}}^{+}$ and ${\mathcal{P}}^{-}$ denote positive and sampled negative base-pair candidates, respectively.

Decoding is discrete and non-differentiable in our implementation. In particular, the Viterbi-style dynamic program is used only to generate hard negative candidates for ${\mathcal{L}}_{\mathrm{adv}}$, and gradients are not propagated through the discrete backtracking step. Thus, training does not rely on REINFORCE-style gradient estimation; instead, the amortized estimator is optimized through the differentiable surrogate losses above.

5.4. Datasets and Evaluation Settings

We evaluate the ${\alpha }_1$-HMM on RNA secondary structure prediction under four complementary settings designed to assess performance within the tRNA family, on temporally held-out tRNA sequences, on broader RNA families within the training distribution, and on out-of-domain RNA families.

First, we evaluate the model on the curated T-psi-C dataset [33], which contains annotated tRNA sequences. On this dataset, we conduct two evaluations: (i) an in-domain evaluation using 10-fold cross-validation on tRNA sequences released before 2024; and (ii) a temporally held-out evaluation in which the model is trained on all pre-2024 sequences and evaluated on 14 tRNA sequences released in 2024.

Second, to assess generalization beyond the relative tRNA family, we train the broader-family model on RNAStralign and evaluate it on both an in-domain test split and the out-of-domain ArchiveII benchmark. RNAStralign is a widely used RNA secondary structure dataset containing 37,149 sequences from eight RNA families: 16S rRNA, tRNA, 5S rRNA, group I intron (grp1), SRP, tmRNA, RNase P, and telomerase RNA. ArchiveII contains 3975 sequences from ten RNA families, including two additional families, 23S rRNAs and group II introns (grp2), beyond those represented in RNAStralign.

In the broader-family experiments, we follow the general data-processing protocol adopted for learning-based RNA secondary structure prediction benchmarks and exclude pseudoknotted structures and sequences longer than 600 nucleotides, since these are beyond the scope of the present work. After preprocessing, the resulting dataset sizes are 13,950 sequences for training, 2073 for validation, 1439 for the in-domain RNAStralign test split, and 1119 for the out-of-domain ArchiveII test set.

5.5. Baselines and Evaluation Protocol

We compare against CONTRAfold [34], ViennaRNA (RNAfold MFE and centroid) [35], LinearFold [36], SPOT-RNA [37], RNA-FM [38], E2Efold [39], MXfold2 [40], and KnotFold [41].

For the pre-2024 T-psi-C benchmark, all methods are evaluated under the same 10-fold cross-validation protocol, and we report mean ± std across folds (

Table 1. Performance comparison across methods on the tRNA benchmark. Pair-level precision (P), recall (R), and F1 are reported as mean ± std across 10 folds on the pre-2024 T-psi-C dataset.

Method	P	R	F1
ViennaRNA-MFE (2011) [35]	$0.668\pm 0.044$	$0.720\pm 0.046$	$0.692\pm 0.045$
ViennaRNA-Centroid (2011) [35]	$0.711\pm 0.049$	$0.672\pm 0.041$	$0.682\pm 0.043$
LinearFold (2019) [36]	$0.668\pm 0.044$	$0.719\pm 0.047$	$0.691\pm 0.045$
CONTRAfold (2006) [34]	$0.695\pm 0.015$	$0.765\pm 0.016$	$0.724\pm 0.015$
SPOT-RNA (2019) [37]	$0.830\pm 0.009$	$0.988\pm 0.008$	$0.901\pm 0.009$
RNA-FM (2022) [38]	$0.739\pm 0.007$	$0.962\pm 0.009$	$0.835\pm 0.007$
E2Efold (2020) [39]	$0.950\pm 0.022$	$0.940\pm 0.023$	$0.944\pm 0.022$
MXfold2 (2021) [40]	$0.951\pm 0.011$	$0.953\pm 0.012$	$0.951\pm 0.011$
KnotFold (2024) [41]	$0.909\pm 0.007$	$0.993\pm 0.006$	$0.948\pm 0.006$
Our method	$0.986\pm 0.008$	$0.989\pm 0.008$	$0.987\pm 0.008$

). For broader-family evaluation, we additionally report results on ArchiveII (

Table 2. Baseline performance on ArchiveII. We report pair-level precision (P), recall (R), and F1.

Method	P	R	F1
ViennaRNA-MFE	0.5880	0.6690	0.6260
ViennaRNA-Centroid	0.6430	0.6440	0.6430
LinearFold	0.5860	0.6660	0.6230
CONTRAfold	0.6158	0.6895	0.6506
MXfold2	0.7630	0.8200	0.7900
SPOT-RNA	0.7340	0.8260	0.7770
KnotFold	0.7830	0.8720	0.8250
RNA-FM	0.6710	0.8120	0.7350
E2Efold	0.7380	0.5810	0.6500
Our method	0.9013	0.8084	0.8523

).

5.6. Results on tRNA Benchmarks

As summarized in Table 1, our neural ${\alpha }_1$-HMM achieves strong pair-level accuracy on tRNA, with particularly strong precision and F1 compared with the baselines considered here. These results indicate that the proposed amortized parameter inference framework is highly effective in the homogeneous tRNA setting.

• Negative controls. To discourage memorization of common tRNA secondary-structure patterns, we construct two negative sets: (i) random negatives, where for each tRNA length L we generate a sequence by sampling each nucleotide uniformly from $\{A,C,G,U\}$; and (ii) shuffled negatives, where each tRNA sequence is randomly permuted, preserving the mononucleotide composition but disrupting the biological structure. These negatives are used only for evaluation. Our method yields paired-nucleotide ratios (PNRs) of $0.223\pm 0.005$ on shuffled negatives and $0.219\pm 0.006$ on random negatives, indicating that the model does not simply predict high pairing rates on arbitrary sequences.
• Temporal evaluation. To assess temporal generalization, we evaluate the model on 14 tRNA sequences released in 2024, all of which are excluded from training and validation. ${\alpha }_1$-HMM achieves precision = 1.000, recall = 1.000, and F1 = 1.000 on this held-out set. Although this result is encouraging, it should be interpreted cautiously because the temporal test set is small and drawn from a structurally homogeneous RNA family. We visualize one such prediction in Figure 4, showing high-quality structure recovery on entry tdbR00000239 compared with several baseline methods.

5.7. Results on Broader RNA Families

We further evaluate the proposed method beyond the relatively homogeneous tRNA family using the broader-family RNAStralign and ArchiveII benchmarks. On the in-domain RNAStralign test split, our method achieves precision $=0.9820$, recall $=0.9759$ and F1 $=0.9789$. On the out-of-domain ArchiveII benchmark, Table 2 shows that our method achieves the strongest F1 among the compared methods while also maintaining very high precision. Together, these results suggest that the proposed amortized ${\alpha }_1$-HMM framework remains competitive not only within the broader training distribution but also on structurally diverse RNA families outside it. Figure 5 shows a long 16S rRNA example (484 nt) with high base-pair accuracy (F1 $=0.9682$), indicating that our model captures long-range base-pair interactions.

6. Conclusions

We proposed a novel amortized inference pipeline for sequence modeling, centered on the degree-1 arbitrary-order hidden Markov model (${\alpha }_1$-HMM). By learning input-conditioned parameters via a neural estimator, our method enables instance-specific probabilistic modeling in a single forward pass, avoiding costly per-sequence iterative parameter estimation while preserving the interpretability of structured latent-variable models.

In the context of RNA secondary structure prediction, our framework demonstrates high accuracy and robustness, even under challenging negative controls, while offering explainable predictions. Although the current model only captures first-degree influence (${\alpha }_1$), future work could explore higher-order dependencies (${\alpha }_k$ for $k>1$) to model richer long-range interactions. Importantly, amortized inference makes such extensions practical by providing efficient instance-conditioned parameterization; complementary directions include constrained or approximate decoding strategies to further improve scalability beyond the $O\left(N^3\right)$ worst-case complexity of exact exclusive-pair decoding.