HMM Classifier¶
Multiple HMMs can be combined to form a multi-class classifier.
The HMMClassifier
can be used to classify:
Univariate/multivariate numerical observation sequences, by using
GaussianMixtureHMM
models.Univariate categorical observation sequences, by using
CategoricalHMM
models.
To classify a new observation sequence \(O'\), the HMMClassifier
works by:
- Creating HMMs \(\lambda_1, \lambda_2, \ldots, \lambda_C\) and training each model using the Baum—Welch algorithm on the subset of training observation sequences with the same class label as the model.
- Calculating the likelihoods \(\mathbb{P}(O'\ |\ \lambda_1), \mathbb{P}(O'\ |\ \lambda_2), \ldots, \mathbb{P}(O'\ |\ \lambda_C)\) of each model generating \(O'\) using the Forward algorithm.
- Scaling the likelihoods by priors \(p(\lambda_1), p(\lambda_2), \ldots, p(\lambda_C)\), producing un-normalized posteriors\(\mathbb{P}(O'\ |\ \lambda_c)p(\lambda_c)\) for each class.
- Choosing the class represented by the HMM with the highest posterior probability for \(O'\).\[\begin{split}c' = \operatorname*{\arg\max}_{c\in\{1,\ldots,C\}}\ p(\lambda_c\ |\ O') = \operatorname*{\arg\\max}_{c\in\{1,\ldots,C\}}\ \mathbb{P}(O'\ |\ \lambda_c)p(\lambda_c)\end{split}\]
These steps are summarized in the diagram below.
API reference¶
Class¶
A classifier consisting of HMMs, each trained independently to recognize sequences of a single class. |
Methods¶
|
Initializes a |
|
Adds a single HMM to the classifier. |
|
Adds HMMs to the classifier. |
|
Fits the HMMs to the sequence(s) in |
|
Predicts classes for the sequence(s) in |
|
Predicts class probabilities for the sequence(s) in |
|
Predicts class scores for the sequence(s) in |
|
Calculates accuracy for the sequence(s) in |
- class sequentia.models.hmm.classifier.HMMClassifier[source]¶
A classifier consisting of HMMs, each trained independently to recognize sequences of a single class.
The predicted class for a given observation sequence is the class represented by the HMM which produces the maximum posterior probability for the observation sequence.
Examples
Using a
HMMClassifier
(withGaussianMixtureHMM
models) to classify spoken digits.import numpy as np from sequentia.datasets import load_digits from sequentia.models.hmm import GaussianMixtureHMM, HMMClassifier # Seed for reproducible pseudo-randomness random_state = np.random.RandomState(1) # Fetch MFCCs of spoken digits data = load_digits() train_data, test_data = data.split(test_size=0.2, random_state=random_state) # Create a HMMClassifier using a class frequency prior clf = HMMClassifier(prior='frequency') # Add an untrained HMM for each class for label in data.classes: model = GaussianMixtureHMM(random_state=random_state) clf.add_model(model, label) # Fit the HMMs by providing training observation sequences for all classes X_train, y_train, lengths_train = train_data.X_y_lengths clf.fit(X_train, y_train, lengths_train) # Predict classes for the test observation sequences X_test, lengths_test = test_data.X_lengths y_pred = clf.predict(X_test, lengths_test)
As done in the above example, we can provide unfitted HMMs using
add_model()
oradd_models()
, then provide training observation sequences for all classes tofit()
, which will automatically train each HMM on the appropriate subset of data.Alternatively, we may provide pre-fitted HMMs and call
fit()
with no arguments.# Create a HMMClassifier using a class frequency prior clf = HMMClassifier(prior='frequency') # Manually fit each HMM on its own subset of data for X_train, lengths_train, label for train_data.iter_by_class(): model = GaussianMixtureHMM(random_state=random_state) model.fit(X_train, lengths_train) clf.add_model(model, label) # Fit the classifier clf.fit()
- __init__(*, prior=None, classes=None, n_jobs=1)[source]¶
Initializes a
HMMClassifier
.- Parameters:
prior (Optional[Union[Literal['frequency'], dict]]) –
Type of prior probability to assign to each HMM.
If
None
, a uniform prior will be used, making each HMM equally likely.If
"frequency"
, the prior probability of each HMM is equal to the fraction of total observation sequences that the HMM was fitted with.If a
dict
, custom prior probabilities can be assigned to each HMM. The keys should be the label of the class represented by the HMM, and the value should be the prior probability for the HMM.
classes (Optional[Array]) –
Set of possible class labels.
If not provided, these will be determined from the training data labels.
If provided, output from methods such as
predict_proba()
andpredict_scores()
will follow the ordering of the class labels provided here.
n_jobs (Union[NegativeInt, PositiveInt]) –
Maximum number of concurrently running workers.
If 1, no parallelism is used at all (useful for debugging).
If -1, all CPUs are used.
If < -1,
(n_cpus + 1 + n_jobs)
are used — e.g.n_jobs=-2
uses all but one.
- Return type:
- add_model(model, label)[source]¶
Adds a single HMM to the classifier.
- Parameters:
model (_HMM) – HMM to add to the classifier.
label (int) – Class represented by the HMM.
- Note:
All models added to the classifier must be of the same type — either
GaussianMixtureHMM
orCategoricalHMM
.
- add_models(models)[source]¶
Adds HMMs to the classifier.
- Parameters:
models (Dict[int, _HMM]) – HMMs to add to the classifier. The key for each HMM should be the label of the class represented by the HMM.
- Note:
All models added to the classifier must be of the same type — either
GaussianMixtureHMM
orCategoricalHMM
.
- fit(X=None, y=None, lengths=None)[source]¶
Fits the HMMs to the sequence(s) in
X
.If fitted models were provided with
add_model()
oradd_models()
, no arguments should be passed tofit()
.If unfitted models were provided with
add_model()
oradd_models()
, training dataX
,y
andlengths
must be provided tofit()
.
- Parameters:
X (Optional[Array]) –
Univariate or multivariate observation sequence(s).
Should be a single 1D array if
CategoricalHMM
is being used, or either a 1D or 2D array ifGaussianMixtureHMM
is being used.Should have length as the 1st dimension and features as the 2nd dimension.
Should be a concatenated sequence if multiple sequences are provided, with respective sequence lengths being provided in the
lengths
argument for decoding the original sequences.
y (Optional[Array]) – Classes corresponding to sequence(s) provided in
X
.lengths (Optional[Array]) –
Lengths of the observation sequence(s) provided in
X
.If
None
, thenX
is assumed to be a single observation sequence.len(X)
should be equal tosum(lengths)
.
- Returns:
The fitted classifier.
- Return type:
- classmethod load(path)[source]¶
Loads and deserializes a fitted HMM classifier.
- Parameters:
path (Union[str, Path, IO]) – Location to load the serialized classifier from.
- Returns:
Fitted HMM classifier.
- Return type:
See also
save
Serializes and saves a fitted HMM classifier.
- predict(X, lengths=None)[source]¶
Predicts classes for the sequence(s) in
X
.- Parameters:
X (Array) –
Univariate or multivariate observation sequence(s).
Should be a single 1D array if
CategoricalHMM
is being used, or either a 1D or 2D array ifGaussianMixtureHMM
is being used.Should have length as the 1st dimension and features as the 2nd dimension.
Should be a concatenated sequence if multiple sequences are provided, with respective sequence lengths being provided in the
lengths
argument for decoding the original sequences.
lengths (Optional[Array]) –
Lengths of the observation sequence(s) provided in
X
.If
None
, thenX
is assumed to be a single observation sequence.len(X)
should be equal tosum(lengths)
.
- Note:
This method requires a trained classifier — see
fit()
.- Returns:
Class predictions.
- Return type:
Array
- predict_proba(X, lengths=None)[source]¶
Predicts class probabilities for the sequence(s) in
X
.Probabilities are calculated as the posterior probability of each HMM generating the sequence.
- Parameters:
X (Array) –
Univariate or multivariate observation sequence(s).
Should be a single 1D array if
CategoricalHMM
is being used, or either a 1D or 2D array ifGaussianMixtureHMM
is being used.Should have length as the 1st dimension and features as the 2nd dimension.
Should be a concatenated sequence if multiple sequences are provided, with respective sequence lengths being provided in the
lengths
argument for decoding the original sequences.
lengths (Optional[Array]) –
Lengths of the observation sequence(s) provided in
X
.If
None
, thenX
is assumed to be a single observation sequence.len(X)
should be equal tosum(lengths)
.
- Note:
This method requires a trained classifier — see
fit()
.- Returns:
Class membership probabilities.
- Return type:
Array
- predict_scores(X, lengths=None)[source]¶
Predicts class scores for the sequence(s) in
X
.Scores are calculated as the log posterior probability of each HMM generating the sequence.
- Parameters:
X (Array) –
Univariate or multivariate observation sequence(s).
Should be a single 1D array if
CategoricalHMM
is being used, or either a 1D or 2D array ifGaussianMixtureHMM
is being used.Should have length as the 1st dimension and features as the 2nd dimension.
Should be a concatenated sequence if multiple sequences are provided, with respective sequence lengths being provided in the
lengths
argument for decoding the original sequences.
lengths (Optional[Array]) –
Lengths of the observation sequence(s) provided in
X
.If
None
, thenX
is assumed to be a single observation sequence.len(X)
should be equal tosum(lengths)
.
- Note:
This method requires a trained classifier — see
fit()
.- Returns:
Class scores.
- Return type:
Array
- save(path)[source]¶
Serializes and saves a fitted HMM classifier.
- Parameters:
path (Union[str, Path, IO]) – Location to save the serialized classifier.
- Note:
This method requires a trained classifier — see
fit()
.
See also
load
Loads and deserializes a fitted HMM classifier.
- score(X, y, lengths, normalize=True, sample_weight=None)[source]¶
Calculates accuracy for the sequence(s) in
X
.- Parameters:
X (Array) –
Univariate or multivariate observation sequence(s).
Should be a single 1D array if
CategoricalHMM
is being used, or either a 1D or 2D array ifGaussianMixtureHMM
is being used.Should have length as the 1st dimension and features as the 2nd dimension.
Should be a concatenated sequence if multiple sequences are provided, with respective sequence lengths being provided in the
lengths
argument for decoding the original sequences.
y (Array) – Classes corresponding to the observation sequence(s) in
X
.lengths (Optional[Array]) –
Lengths of the observation sequence(s) provided in
X
.If
None
, thenX
is assumed to be a single observation sequence.len(X)
should be equal tosum(lengths)
.
normalize (bool) – See
sklearn.metrics.accuracy_score()
.sample_weight (Optional[Array]) – See
sklearn.metrics.accuracy_score()
.
- Note:
This method requires a trained classifier — see
fit()
.- Returns:
Classification accuracy.
- Return type:
float
- classes¶
Set of possible class labels.
- models¶
HMMs constituting the
HMMClassifier
.
- n_jobs¶
Maximum number of concurrently running workers.
- prior¶
Type of prior probability to assign to each HMM.