Download Understanding Hidden Markov Models: Time-Dependent States and Observations - Prof. David J and more Study notes Computer Science in PDF only on Docsity!
Hidden Markov Models
- Generative, rather than descriptive model.
- Objects produced by random process.
- Dependencies in process, some random events influence others. - Time is most natural metaphor here.
- Simplest, most tractable model of dependencies is Markov.
- Lecture based on: Rabiner , “ A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition.”
Markov Chain
- States: S 1 , … SN
- Discrete time steps, 1, 2, …
- State at time t is qt.
- Initial state, q 1. pii = P(q 1 = Si).
- P(qt = Sj | qt-1= Si, qt-2=Sk, … ) = P(qt = Sj | qt-1 = Si). This is what makes it Markov.
- Time independence: aij = P(qt = Sj | qt-1 = Si).
Examples
- 1D random walk in finite space.
- 1D curve generated by random walk in
orientation.
States of Markov Chain
- Represent state at time t as vector:
w(t) = (P(qt=S 1 ), P(qt=S 2 ), … P(qt=SN)
- Put transitions, aij into matrix A.
- A is Stochastic, meaning columns sum to 1.
- Then w(t) = AT*w(t-1).
Three problems
- Probability of observations given model.
- Use to select model given observations (eg, speech recognition).
- To refine estimate of HMM.
- Given model and observations, what were likely states? - States may have semantics (rough/smooth contour). - May provide intuitions about model. - However, this is often least useful problem.
- Find model to optimize probability of observations.
Probability of Observations
- Solved with dynamic programming.
- Whiteboard (see Rabiner for notes).
Problem 1: Probability of observations given model: P(O | lambda). Basic idea: we can do this with dynamic programming. This is basically inductive. Suppose we know the probability of producing the first t symbols and winding up in state i at time t, for all values of i. Then we want to use that to compute the same thing for t+1. The key thing is that to figure this out for time t+1 we just need to know if for time t. In particular, it won’t matter what states we were in for time < t, just what states we were in at t. Specifically, (abbreviate \alpha = \a) we define: \a_t(i) = P(O_1, … O_t, q_t = S_i | \lambda)
- Initialize: \a_1(i) = \pi_i b_i(O_1)
- Recurse: \a_{t+1}(j) = [sum_{i=1}^N \a_t(i) a_{ij}] b_j(O_{t+1})
- Termination: P(O|lambda) = sum_{i=1}^N \a_T(i)
Problem 2: Maximum likelihood sequence of internal states given model and observations. This is the same as (1), except we use maximum instead of sum, and keep backward pointers.
Problem 3: Estimating a model, given a sequence (or many of them). Formally: argmax_lambda P(O | lambda). Note that we are assuming we know the number of states (more states would always allow a better fit). We approach this with an iterative algorithm. We assume some starting point, then improve it. Given a model, we estimate the sample probability of every parameter. Then we adjust the model to use each sample probability as the true one. For example, we estimate the probability that we begin in state 1, given the model AND the observations. Then we use this as the new prior probability that we will start in state 1. We do this for every aspect of the model. The key to computing these sample probabilities is to figure out the probability that we will be in state i at time t, and the probability that we will move from state i at time t to state j at time t+1. It is important to note that we can’t just use \a to determine this. This is because the probability that we will be in state i at time t doesn’t just depend on the probability of emitting the first t symbols and winding up in state i after time t. It also depends on the chances that we will continue on from state i to emit all the rest of the symbols. For example, it is possible that from state i you almost never go to a state that will emit the next symbol you need.
Practical Applications
- Continuous to discrete
- Discretize observations with codebook.
Graphical Models
- Represent conditional dependencies with a graph.
- Each variable is a node
- Two nodes are connected if they are directly dependent.
- Two variables, X1 and X2, are conditionally independent, given knowledge of other variables, iff removing the nodes of the other variables disconnects X1 and X2.
Example
X1 X2 X
X
This graph represents the relationship that given V2, V1 is conditionally independent of V3 and V4. E.g., P(V1|V2,V3,V4) = P(V1|V2)
HMM
q1 q2 q3 q
V1 V2 V3 V
We perform inference with knowledge of conditional probabilities (the model) and of some variables (V1, V2…)
Belief Propagation
- Like inference in an HMM, but can handle
directed edges, and general DAGs.
- Still the basic idea is the same, to combine
information from forward and backward directions (or maybe more than two independent directions).
- In general, lack of cycles allows us to use dynamic programming.
General Graphical models
- Model must include joint distribution of all variables that form cliques.
- We can compress a clique into a single variable, with states that give the cross-product of all states of individual variables. - If doing this produces a DAG, we can perform inference using dynamic programming. - Kfan is a graphical model with a single clique, and all other variables directly connected to this.
- Inference in general is NP-hard, but there is much work on effective algorithms for this problem.
