mjhjmtu

From the book "Machine Learning a probabilistic Perspective",
I'm reading about finite/infinite mixture models. Particularly at paragraph 25.2.1 it's stated:

The usual representation (of a finite mixture model) is as follows:

$p(x_i|z_i = k, boldsymboltheta) = p(x_i|boldsymboltheta_k)$

$p(z_i = k| boldsymbolpi = pi_k) = pi_k$

$p(boldsymbolpi|alpha) =textDir(boldsymbolpi|(alpha/K)boldsymbol1_K)$

The form of $p(boldsymboltheta_k|lambda)$ is chosen to be be
conjugate to $p(x_i|boldsymboltheta_k)$. We can write
$p(x_i|boldsymboltheta_k)$ as $boldsymbolx_i sim F(boldsymboltheta_z_i)$ where F is the observation distribution.
Similarly, we can write $boldsymboltheta_k sim H(lambda)$, where H
is the prior.

Now this modelling is quite confusing to me. What is the difference between $boldsymboltheta_k$ and $boldsymboltheta_z_i$? What is meant by "Observation distribution"? Can we apply EM algorithm to this model, how?

If you consider a mixture model,$$Xsimsum_k pi_k p(x|theta_k)$$it can be expressed as the marginal of the joint model$$(X,Z)sim underbracepi_z_p_pi(z)timesunderbracetheta_z)_p_theta(x$$where $Z$ is an integer valued random variable. This implies that$$X|Z=ksim p(x|theta_k)$$which can also be written as $$X|Zsim p(x|theta_Z)$$The random variable $Z$ is also latent in that (a) it is not observed and (b) it does not necessarily "exist" in the original experiment modelled by the mixture. But the EM algorithm that returns the MLE of the parameters $pi_k$ and $theta_k$ takes advantage of this joint representation by iteratively

1. calculating $mathbbE[Z_i|X_i,theta,pi]$
   


2. maximising the expected log-likelihood in $(theta,pi)$
   

What is the difference between $theta_k$ and $theta_z_i$?

The only difference is the subscript. The value $theta_z_i$ refers to the value $theta_k$ where $k = z_i$. So the bit which says $boldsymbolx sim F(boldsymboltheta_z_i)$ simply means that $boldsymbolx sim F(boldsymboltheta_k)$ where the subscript $k$ is the latent random variable $z_i$.