EM Algorithm

目录

• EM Algorithm
  • Jensen's inequality
  • EM Algorithm

Jensen's inequality

convex function: \(f''(x) \ge 0\) or \(H \ge 0\) (Hessian matrix when x is a vector)

\[E[f(x)] \ge f(EX) \]

EM Algorithm

EM can be proved that it make the likelihood function increase monotonically.

maximize the lower-bound on the likelihood \(\ell\), \(\log\) is a concave function. the process is

\[\theta_{t-1} \rightarrow^{assign} Q_{z,t-1} \rightarrow^{\arg\max} \theta_{t} \rightarrow Q_{z,t} \]

define:

\[J(Q,\theta)=\sum_{i} \sum_{z^{(i)}}Q_i(z^{(i)})\log \frac{p(x^{(i)},z^{(i)};\theta)}{Q_i(z^{(i)})} \]

the EM algorithm can also be defined as coordinate ascent on \(J\).

标签：EM,function,Algorithm,ge,算法,theta,rightarrow
来源： https://www.cnblogs.com/BUAA-Stargazer/p/16613979.html