mjhjmtu

I want to use MH to get samples from $p(theta mid y) approx p(y mid theta) p(theta)$. Let's assume $theta$ is heavily constrained and I transform $theta$ to $f(theta)$ so I can sample from an unconstrained space.

The new posterior becomes $p(f(theta) mid y) approx p(y mid f(theta) ) p(f(theta)) ,times, |det(J_f^-1(y)) |$. Note that I only changed the prior term (Pushforward measure) and left the likelihood term unchanged as it is a probability distribution on $y$, not on $theta$.

(1) My question now is: can I - in the Metropolis Hastings acceptance ratio - just evaluate

$$fracp(y mid theta^star) p(y mid theta) ,times, fracp(f(theta^star)) mid det J_f^-1( theta^star) mid p(f(theta)) mid det J_f^-1( theta ) mid $$

? This term makes me nervous, because I transformed theta, evaluate the pdf of the transformed prior, but then transform it back and evaluate the likelihood of the parameter in the original space. However, I cannot evaluate the first term of this equation:

$$fracp(y mid f(theta^star)) p(y mid f(theta)) ,times, fracp(f(theta^star)) leftlvert det J_f^-1( theta^star) rightrvert p(f(theta)) leftlvert det J_f^-1( theta ) rightrvert .$$

I could somehow reverse engineer this problem, i.e. define priors on $f(theta)$ and then map $f(theta)$ to $theta$. The Jacobian of the inverse transform then becomes the Jacobian of the transform of my original problem. That way I could evaluate all terms. However, I originally wanted to give some meaning to my priors for $theta$, not for some unconstrained $f(theta)$.

EDIT: Problem solved and clarified - thank you, I should have seen this myself! Please also see
linked stackexchange thread to this post for further
clarification.

You should notice that what you denote $p(y|f(theta))$ is actually the same as $p(y|theta)$ [if you overlook the terrible abuse of notations]. As you mention, changing the parameterisation does not modify the density of the random variable at the observed value $y$ and there is no Jacobian associated with that part.

With proper notations, if
beginalign*
theta &sim pi(theta)qquadqquad&textprior\
y|theta &sim f(y|theta)qquadqquad&textsampling\
xi &= h(theta) qquadqquad&textreparameterisation\
dfractextdthetatextdxi(xi) &= J(xi)qquadqquad&textJacobian\
y|xi &sim g(y|xi)qquadqquad&textreparameterised density\
xi^(t+1)|xi^(t) &sim q(xi^(t+1)|xi^(t)) qquadqquad&textproposal
endalign*
the Metropolis-Hastings ratio associated with the proposal $xi'sim q(xi'|xi)$ in the $xi$ parameterisation is
$$
underbracedfracpi(theta(xi'))J(xi')pi(theta(xi))J(xi)_textratio of priorstimes
underbracedfracf(yf(y_textlikelihood ratiotimes
underbracedfracxi')xi)_textproposal ratio
$$
which also writes as
$$dfracpi(h^-1(xi'))J(xi')pi(h^-1(xi))J(xi)times
dfracg(yg(ytimes
dfracxi')xi)
$$