Categorical HMM

The Categorical HMM is a variant of HMM that uses a discrete probability distribution over a finite set of symbols as the emission distribution for each state.

This HMM variant can be used to recognize categorical univariate sequences.

Emissions

The emission distribution \(b_m\) of an observation \(o^{(t)}\) at time \(t\) for state \(m\) is given by a probability vector:

\[\bigg[\underbrace{\mathbb{P}\big(o^{(t)}=s_0\ |\ q^{(t)}=m\big)}_{p_{m,0}}, \ldots, \underbrace{\mathbb{P}\big(o^{(t)}=s_K\ |\ q^{(t)}=m\big)}_{p_{m,K}}\bigg]\]

Where:

  • \(\mathcal{S}=\{s_0,s_1,\ldots,s_K\}\) is a finite set of observation symbols.
  • \(o^{(t)}\in\mathcal{S}\) is a single observation at time \(t\).
  • \(q^{(t)}\) is a discrete random variable representing the hidden state at time \(t\).
  • \(p_{m,k}=\mathbb{P}\big(o^{(t)}=s_k\ |\ q^{(t)}=m\big)\) is the probability of observing \(s_k\) while in state \(m\).

The emission distributions for all states can be represented by a single \(M\times K\) emission matrix:

\[\begin{split}\begin{bmatrix} p_{0,0} & \cdots & p_{0,K} \\ \vdots & \ddots & \vdots \\ p_{M,0} & \cdots & p_{M,K} \end{bmatrix}\end{split}\]

Note

Observation symbols must be encoded as integers. Consider performing label encoding using sklearn.preprocessing.LabelEncoder.

API reference

Class

CategoricalHMM

A hidden Markov model with univariate categorical emissions.

Methods

__init__(*[, n_states, topology, ...])

Initializes the CategoricalHMM.

aic(X, *[, lengths])

The Akaike information criterion of the model, evaluated with the maximum likelihood of X.

bic(X, *[, lengths])

The Bayesian information criterion of the model, evaluated with the maximum likelihood of X.

fit(X, *[, lengths])

Fit the HMM to the sequences in X, using the Baum—Welch algorithm.

freeze([params])

Freeze the trainable parameters of the HMM, preventing them from being updated during the Baum—Welch algorithm.

score(x, /)

Calculate the log-likelihood of the HMM generating a single observation sequence.

set_state_emission_probs(probs, /)

Set the state emission distribution of the HMM's emission model.

set_state_start_probs([probs])

Set the initial state probabilities.

set_state_transition_probs([probs])

Set the transition probability matrix.

unfreeze([params])

Unfreeze the trainable parameters of the HMM, allowing them to be updated during the Baum—Welch algorithm.

n_params

Number of trainable parameters — requires fit().

class sequentia.models.hmm.variants.CategoricalHMM

A hidden Markov model with univariate categorical emissions.

Examples

Using a CategoricalHMM to learn how to recognize DNA sequences from the synthetase gene family.

See load_gene_families() for more information on the sample dataset used in this example.

import numpy as np
from sequentia.datasets import load_gene_families
from sequentia.models.hmm import CategoricalHMM

# Seed for reproducible pseudo-randomness
random_state = np.random.RandomState(1)

# Fetch DNA sequences for the synthetase gene family (no. 4)
data, enc = load_gene_families(families=[4])
train_data, test_data = data.split(test_size=0.2, random_state=random_state)

# Create and train a CategoricalHMM to recognize the synthetase DNA sequences
model = CategoricalHMM(random_state=random_state)
model.fit(train_data.X, lengths=train_data.lengths)

# Calculate the log-likelihood of the first test sample being generated by this model
x, y = test_data[0]
model.score(x)
__init__(*, n_states=5, topology=TopologyMode.LEFT_RIGHT, random_state=None, hmmlearn_kwargs=None)

Initializes the CategoricalHMM.

Parameters:
Return type:

CategoricalHMM

aic(X, *, lengths=None)

The Akaike information criterion of the model, evaluated with the maximum likelihood of X.

Parameters:

  • self (BaseHMM) –

  • X (ndarray[Any, dtype[float64]] | ndarray[Any, dtype[int64]]) – Sequence(s).

  • lengths (ndarray[Any, dtype[int64]] | None) –

    Lengths of the sequence(s) provided in X.

    • If None, then X is assumed to be a single sequence.

    • len(X) should be equal to sum(lengths).

Returns:

The Akaike information criterion.

Return type:

float

Notes

This method requires a trained model — see fit().

bic(X, *, lengths=None)

The Bayesian information criterion of the model, evaluated with the maximum likelihood of X.

Parameters:

  • self (BaseHMM) –

  • X (ndarray[Any, dtype[float64]] | ndarray[Any, dtype[int64]]) – Sequence(s).

  • lengths (ndarray[Any, dtype[int64]] | None) –

    Lengths of the sequence(s) provided in X.

    • If None, then X is assumed to be a single sequence.

    • len(X) should be equal to sum(lengths).

Returns:

The Bayesian information criterion.

Return type:

float

Notes

This method requires a trained model — see fit().

fit(X, *, lengths=None)

Fit the HMM to the sequences in X, using the Baum—Welch algorithm.

Parameters:

  • self (BaseHMM) –

  • X (ndarray[Any, dtype[float64]] | ndarray[Any, dtype[int64]]) – Sequence(s).

  • lengths (ndarray[Any, dtype[int64]] | None) –

    Lengths of the sequence(s) provided in X.

    • If None, then X is assumed to be a single sequence.

    • len(X) should be equal to sum(lengths).

Returns:

The fitted HMM.

Return type:

BaseHMM

freeze(params=None, /)

Freeze the trainable parameters of the HMM, preventing them from being updated during the Baum—Welch algorithm.

Parameters:

  • self (CategoricalHMM) –

  • params (str | None) –

    A string specifying which parameters to freeze. Can contain a combination of:

    • 's' for initial state probabilities,

    • 't' for transition probabilities,

    • 'e' for emission probailities.

Return type:

None

Notes

If used, this method should normally be called before fit().

See also

unfreeze

Unfreeze the trainable parameters of the HMM, allowing them to be updated during the Baum—Welch algorithm.

score(x, /)

Calculate the log-likelihood of the HMM generating a single observation sequence.

Parameters:
Returns:

The log-likelihood.

Return type:

float

Notes

This method requires a trained model — see fit().

set_state_emission_probs(probs, /)

Set the state emission distribution of the HMM’s emission model.

If this method is not called, emission probabilities will be initialized by hmmlearn.

Parameters:
Return type:

None

Notes

If used, this method should normally be called before fit().

set_state_start_probs(probs=TransitionMode.RANDOM, /)

Set the initial state probabilities.

If this method is not called, initial state probabilities are initialized depending on the value of topology provided to __init__().

  • If topology was set to 'ergodic', 'left-right' or 'linear', then random probabilities will be assigned according to the topology by calling set_state_start_probs() with probs='random'.

  • If topology was set to None, then initial state probabilities will be initialized by hmmlearn.

Parameters:

  • self (BaseHMM) –

  • probs (ndarray[Any, dtype[float64]] | TransitionMode) –

    Probabilities or probability type to assign as initial state probabilities.

    • If an Array, should be a vector of starting probabilities for each state.

    • If 'uniform', there is an equal probability of starting in any state.

    • If 'random', the vector of initial state probabilities is sampled from a Dirichlet distribution with unit concentration parameters.

Return type:

None

Notes

If used, this method should normally be called before fit().

set_state_transition_probs(probs=TransitionMode.RANDOM, /)

Set the transition probability matrix.

If this method is not called, transition probabilities are initialized depending on the value of topology provided to __init__():

  • If topology was set to 'ergodic', 'left-right' or 'linear', then random probabilities will be assigned according to the topology by calling set_state_transition_probs() with value='random'.

  • If topology was set to None, then initial state probabilities will be initialized by hmmlearn.

Parameters:

  • self (BaseHMM) –

  • probs (ndarray[Any, dtype[float64]] | TransitionMode) –

    Probabilities or probability type to assign as state transition probabilities.

    • If an Array, should be a matrix of probabilities where each row must some to one and represents the probabilities of transitioning out of a state.

    • If 'uniform', for each state there is an equal probability of transitioning to any state permitted by the topology.

    • If 'random', the vector of transition probabilities for each row is sampled from a Dirichlet distribution with unit concentration parameters, according to the shape of the topology.

Return type:

None

Notes

If used, this method should normally be called before fit().

unfreeze(params=None, /)

Unfreeze the trainable parameters of the HMM, allowing them to be updated during the Baum—Welch algorithm.

Parameters:

  • self (CategoricalHMM) –

  • params (str | None) –

    A string specifying which parameters to unfreeze. Can contain a combination of:

    • 's' for initial state probabilities,

    • 't' for transition probabilities,

    • 'e' for emission probailities.

Return type:

None

See also

freeze

Freeze the trainable parameters of the HMM, preventing them from being updated during the Baum—Welch algorithm.

hmmlearn_kwargs: dict[str, Any]

Additional key-word arguments provided to the hmmlearn HMM constructor.

model: BaseHMM

Underlying HMM object from hmmlearn — only set after fit().

property n_params: int

Number of trainable parameters — requires fit().

n_states: int

Number of states in the Markov chain.

random_state: int | RandomState | None

Seed or numpy.random.RandomState object for reproducible pseudo-randomness.

topology: TopologyMode

Transition topology of the Markov chain — see Topologies.