Categorical HMM¶

The Categorical HMM is a variant of HMM that uses a discrete probability distribution over a finite set of symbols as the emission distribution for each state.

This HMM variant can be used to recognize categorical univariate sequences.

Emissions¶

The emission distribution $$b_m$$ of an observation $$o^{(t)}$$ at time $$t$$ for state $$m$$ is given by a probability vector:

$\bigg[\underbrace{\mathbb{P}\big(o^{(t)}=s_0\ |\ q^{(t)}=m\big)}_{p_{m,0}}, \ldots, \underbrace{\mathbb{P}\big(o^{(t)}=s_K\ |\ q^{(t)}=m\big)}_{p_{m,K}}\bigg]$

Where:

• $$\mathcal{S}=\{s_0,s_1,\ldots,s_K\}$$ is a finite set of observation symbols.
• $$o^{(t)}\in\mathcal{S}$$ is a single observation at time $$t$$.
• $$q^{(t)}$$ is a discrete random variable representing the hidden state at time $$t$$.
• $$p_{m,k}=\mathbb{P}\big(o^{(t)}=s_k\ |\ q^{(t)}=m\big)$$ is the probability of observing $$s_k$$ while in state $$m$$.

The emission distributions for all states can be represented by a single $$M\times K$$ emission matrix:

$\begin{split}\begin{bmatrix} p_{0,0} & \cdots & p_{0,K} \\ \vdots & \ddots & \vdots \\ p_{M,0} & \cdots & p_{M,K} \end{bmatrix}\end{split}$

Note

Observation symbols must be encoded as integers. Consider performing label encoding using sklearn.preprocessing.LabelEncoder.

API reference¶

Class¶

 CategoricalHMM A hidden Markov model with univariate categorical emissions.

Methods¶

 __init__(*[, n_states, topology, ...]) Initializes the CategoricalHMM. aic(X, *[, lengths]) The Akaike information criterion of the model, evaluated with the maximum likelihood of X. bic(X, *[, lengths]) The Bayesian information criterion of the model, evaluated with the maximum likelihood of X. fit(X, *[, lengths]) Fit the HMM to the sequences in X, using the Baum—Welch algorithm. freeze([params]) Freeze the trainable parameters of the HMM, preventing them from being updated during the Baum—Welch algorithm. score(x, /) Calculate the log-likelihood of the HMM generating a single observation sequence. set_state_emission_probs(probs, /) Set the state emission distribution of the HMM's emission model. set_state_start_probs([probs]) Set the initial state probabilities. set_state_transition_probs([probs]) Set the transition probability matrix. unfreeze([params]) Unfreeze the trainable parameters of the HMM, allowing them to be updated during the Baum—Welch algorithm. n_params Number of trainable parameters — requires fit().

class sequentia.models.hmm.variants.CategoricalHMM

A hidden Markov model with univariate categorical emissions.

Examples

Using a CategoricalHMM to learn how to recognize DNA sequences from the synthetase gene family.

See load_gene_families() for more information on the sample dataset used in this example.

import numpy as np
from sequentia.models.hmm import CategoricalHMM

# Seed for reproducible pseudo-randomness
random_state = np.random.RandomState(1)

# Fetch DNA sequences for the synthetase gene family (no. 4)
train_data, test_data = data.split(test_size=0.2, random_state=random_state)

# Create and train a CategoricalHMM to recognize the synthetase DNA sequences
model = CategoricalHMM(random_state=random_state)
model.fit(train_data.X, lengths=train_data.lengths)

# Calculate the log-likelihood of the first test sample being generated by this model
x, y = test_data[0]
model.score(x)

__init__(*, n_states=5, topology=TopologyMode.LEFT_RIGHT, random_state=None, hmmlearn_kwargs=None)

Initializes the CategoricalHMM.

Parameters:
Return type:

CategoricalHMM

aic(X, *, lengths=None)

The Akaike information criterion of the model, evaluated with the maximum likelihood of X.

Parameters:
• self (BaseHMM) –

• X (ndarray[Any, dtype[float64]] | ndarray[Any, dtype[int64]]) – Sequence(s).

• lengths (ndarray[Any, dtype[int64]] | None) –

Lengths of the sequence(s) provided in X.

• If None, then X is assumed to be a single sequence.

• len(X) should be equal to sum(lengths).

Returns:

The Akaike information criterion.

Return type:

float

Notes

This method requires a trained model — see fit().

bic(X, *, lengths=None)

The Bayesian information criterion of the model, evaluated with the maximum likelihood of X.

Parameters:
• self (BaseHMM) –

• X (ndarray[Any, dtype[float64]] | ndarray[Any, dtype[int64]]) – Sequence(s).

• lengths (ndarray[Any, dtype[int64]] | None) –

Lengths of the sequence(s) provided in X.

• If None, then X is assumed to be a single sequence.

• len(X) should be equal to sum(lengths).

Returns:

The Bayesian information criterion.

Return type:

float

Notes

This method requires a trained model — see fit().

fit(X, *, lengths=None)

Fit the HMM to the sequences in X, using the Baum—Welch algorithm.

Parameters:
• self (BaseHMM) –

• X (ndarray[Any, dtype[float64]] | ndarray[Any, dtype[int64]]) – Sequence(s).

• lengths (ndarray[Any, dtype[int64]] | None) –

Lengths of the sequence(s) provided in X.

• If None, then X is assumed to be a single sequence.

• len(X) should be equal to sum(lengths).

Returns:

The fitted HMM.

Return type:

BaseHMM

freeze(params=None, /)

Freeze the trainable parameters of the HMM, preventing them from being updated during the Baum—Welch algorithm.

Parameters:
• self (CategoricalHMM) –

• params (str | None) –

A string specifying which parameters to freeze. Can contain a combination of:

• 's' for initial state probabilities,

• 't' for transition probabilities,

• 'e' for emission probailities.

Return type:

None

Notes

If used, this method should normally be called before fit().

unfreeze

Unfreeze the trainable parameters of the HMM, allowing them to be updated during the Baum—Welch algorithm.

score(x, /)

Calculate the log-likelihood of the HMM generating a single observation sequence.

Parameters:
Returns:

The log-likelihood.

Return type:

float

Notes

This method requires a trained model — see fit().

set_state_emission_probs(probs, /)

Set the state emission distribution of the HMM’s emission model.

If this method is not called, emission probabilities will be initialized by hmmlearn.

Parameters:
Return type:

None

Notes

If used, this method should normally be called before fit().

set_state_start_probs(probs=TransitionMode.RANDOM, /)

Set the initial state probabilities.

If this method is not called, initial state probabilities are initialized depending on the value of topology provided to __init__().

• If topology was set to 'ergodic', 'left-right' or 'linear', then random probabilities will be assigned according to the topology by calling set_state_start_probs() with probs='random'.

• If topology was set to None, then initial state probabilities will be initialized by hmmlearn.

Parameters:
• self (BaseHMM) –

• probs (ndarray[Any, dtype[float64]] | TransitionMode) –

Probabilities or probability type to assign as initial state probabilities.

• If an Array, should be a vector of starting probabilities for each state.

• If 'uniform', there is an equal probability of starting in any state.

• If 'random', the vector of initial state probabilities is sampled from a Dirichlet distribution with unit concentration parameters.

Return type:

None

Notes

If used, this method should normally be called before fit().

set_state_transition_probs(probs=TransitionMode.RANDOM, /)

Set the transition probability matrix.

If this method is not called, transition probabilities are initialized depending on the value of topology provided to __init__():

• If topology was set to 'ergodic', 'left-right' or 'linear', then random probabilities will be assigned according to the topology by calling set_state_transition_probs() with value='random'.

• If topology was set to None, then initial state probabilities will be initialized by hmmlearn.

Parameters:
• self (BaseHMM) –

• probs (ndarray[Any, dtype[float64]] | TransitionMode) –

Probabilities or probability type to assign as state transition probabilities.

• If an Array, should be a matrix of probabilities where each row must some to one and represents the probabilities of transitioning out of a state.

• If 'uniform', for each state there is an equal probability of transitioning to any state permitted by the topology.

• If 'random', the vector of transition probabilities for each row is sampled from a Dirichlet distribution with unit concentration parameters, according to the shape of the topology.

Return type:

None

Notes

If used, this method should normally be called before fit().

unfreeze(params=None, /)

Unfreeze the trainable parameters of the HMM, allowing them to be updated during the Baum—Welch algorithm.

Parameters:
• self (CategoricalHMM) –

• params (str | None) –

A string specifying which parameters to unfreeze. Can contain a combination of:

• 's' for initial state probabilities,

• 't' for transition probabilities,

• 'e' for emission probailities.

Return type:

None

freeze

Freeze the trainable parameters of the HMM, preventing them from being updated during the Baum—Welch algorithm.

hmmlearn_kwargs: dict[str, Any]

Additional key-word arguments provided to the hmmlearn HMM constructor.

model: BaseHMM

Underlying HMM object from hmmlearn — only set after fit().

property n_params: int

Number of trainable parameters — requires fit().

n_states: int

Number of states in the Markov chain.

random_state: int | RandomState | None

Seed or numpy.random.RandomState object for reproducible pseudo-randomness.

topology: TopologyMode

Transition topology of the Markov chain — see Topologies.