If both prior and likelihood are Gaussian what can we say about the posterior?
Clash Royale CLAN TAG#URR8PPP
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;
up vote
2
down vote
favorite
If X is a random variable that has Gaussian prior and Gaussian likelihood. What can be inferred about the posterior?
As posterior is proporional to prior*likelihood which are Gaussians, the posterior should also be Gaussians. But I have struggles deriving it.
self-study normal-distribution maximum-likelihood likelihood conjugate-prior
asked 5 hours ago
Alex
133
New contributor
Alex is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
up vote
2
down vote
favorite
If X is a random variable that has Gaussian prior and Gaussian likelihood. What can be inferred about the posterior?
As posterior is proporional to prior*likelihood which are Gaussians, the posterior should also be Gaussians. But I have struggles deriving it.
self-study normal-distribution maximum-likelihood likelihood conjugate-prior
asked 5 hours ago
Alex
133
New contributor
Alex is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1
Welcome to CV. Since you’re new here, you may want to take our tour, which has information for new users. Please add the [self-study] tag and read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. Please make these changes as just posting your homework & hoping someone will do it for you is grounds for closing. If this is self-study rather than homework, let us know, and... it's still a good idea to show us what you've tried or explain specifically what you don't understand.
– jbowman
5 hours ago
@Alex, do you mean to say "As posterior is proportional to prior*likelihood" ?
– curious_dan
2 hours ago
@curious_dan, yes thanks
– Alex
1 hour ago
add a comment |
up vote
2
down vote
favorite
up vote
2
down vote
favorite
If X is a random variable that has Gaussian prior and Gaussian likelihood. What can be inferred about the posterior?
As posterior is proporional to prior*likelihood which are Gaussians, the posterior should also be Gaussians. But I have struggles deriving it.
self-study normal-distribution maximum-likelihood likelihood conjugate-prior
asked 5 hours ago
Alex
133
New contributor
Alex is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
If X is a random variable that has Gaussian prior and Gaussian likelihood. What can be inferred about the posterior?
As posterior is proporional to prior*likelihood which are Gaussians, the posterior should also be Gaussians. But I have struggles deriving it.
self-study normal-distribution maximum-likelihood likelihood conjugate-prior
self-study normal-distribution maximum-likelihood likelihood conjugate-prior
asked 5 hours ago
Alex
133
New contributor
Alex is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 5 hours ago
Alex
133
New contributor
Alex is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 1 hour ago
asked 5 hours ago
Alex
133
New contributor
Alex is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 5 hours ago
Alex
133
asked 5 hours ago
Alex
133
133
New contributor
Alex is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Alex is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Alex is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1
Welcome to CV. Since you’re new here, you may want to take our tour, which has information for new users. Please add the [self-study] tag and read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. Please make these changes as just posting your homework & hoping someone will do it for you is grounds for closing. If this is self-study rather than homework, let us know, and... it's still a good idea to show us what you've tried or explain specifically what you don't understand.
– jbowman
5 hours ago
@Alex, do you mean to say "As posterior is proportional to prior*likelihood" ?
– curious_dan
2 hours ago
@curious_dan, yes thanks
– Alex
1 hour ago
add a comment |
1
Welcome to CV. Since you’re new here, you may want to take our tour, which has information for new users. Please add the [self-study] tag and read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. Please make these changes as just posting your homework & hoping someone will do it for you is grounds for closing. If this is self-study rather than homework, let us know, and... it's still a good idea to show us what you've tried or explain specifically what you don't understand.
– jbowman
5 hours ago
@Alex, do you mean to say "As posterior is proportional to prior*likelihood" ?
– curious_dan
2 hours ago
@curious_dan, yes thanks
– Alex
1 hour ago
1
1
Welcome to CV. Since you’re new here, you may want to take our tour, which has information for new users. Please add the [self-study] tag and read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. Please make these changes as just posting your homework & hoping someone will do it for you is grounds for closing. If this is self-study rather than homework, let us know, and... it's still a good idea to show us what you've tried or explain specifically what you don't understand.
– jbowman
5 hours ago
Welcome to CV. Since you’re new here, you may want to take our tour, which has information for new users. Please add the [self-study] tag and read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. Please make these changes as just posting your homework & hoping someone will do it for you is grounds for closing. If this is self-study rather than homework, let us know, and... it's still a good idea to show us what you've tried or explain specifically what you don't understand.
– jbowman
5 hours ago
@Alex, do you mean to say "As posterior is proportional to prior*likelihood" ?
– curious_dan
2 hours ago
@Alex, do you mean to say "As posterior is proportional to prior*likelihood" ?
– curious_dan
2 hours ago
@curious_dan, yes thanks
– Alex
1 hour ago
@curious_dan, yes thanks
– Alex
1 hour ago
add a comment |
1 Answer
1
active
oldest
votes
up vote
4
down vote
accepted
The exponents in the prior density and the likelihood are added to each other
$$
frac(mu-mu_0)^2tau^2 + frac(overline x - mu)^2sigma^2/n
quad = quad fracsigma^2(mu-mu_0)^2 + tau^2(overline x - mu)^2sigma^2tau^2/n tag 1
$$
Now let's work on the numerator:
$$
((sigma^2/n)+tau^2) Big(mu^2 - 2(mu_0sigma^2 + overline x tau^2) + text“constant'' Big) tag 2
$$
where “constant” means not depending on $mu$.
Now complete the square:
$$
Big(mu - (mu_0 sigma^2 + overline x tau^2)Big)^2 + text“constant''
$$
(where this “constant” will differ from the earlier one, but it just becomes part of the normalizing constant).
So the posterior density is $$ textconstant times expBig(
textnegative constant times (mu-textsomething)^2 Big).$$
In other words, the posterior is Gaussian.
The posterior mean is a weighted average of the prior mean $mu_0$ and the sample mean $overline x,$ with weights proportional to the reciprocals variances $tau^2$ (for the prior) and $sigma^2/n$ (for the sample mean).
answered 4 hours ago
Michael Hardy
3,1051330
However, just to note that this works when $mu$ enters linearly in the quadratic form in the Gaussian density of $[X mid mu]$. If we have $x_i - f(mu)$, with $f(cdot)$ nonlinear, then we don’t get a Gaussian posterior.
– Dimitris Rizopoulos
3 hours ago
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
4
down vote
accepted
The exponents in the prior density and the likelihood are added to each other
$$
frac(mu-mu_0)^2tau^2 + frac(overline x - mu)^2sigma^2/n
quad = quad fracsigma^2(mu-mu_0)^2 + tau^2(overline x - mu)^2sigma^2tau^2/n tag 1
$$
Now let's work on the numerator:
$$
((sigma^2/n)+tau^2) Big(mu^2 - 2(mu_0sigma^2 + overline x tau^2) + text“constant'' Big) tag 2
$$
where “constant” means not depending on $mu$.
Now complete the square:
$$
Big(mu - (mu_0 sigma^2 + overline x tau^2)Big)^2 + text“constant''
$$
(where this “constant” will differ from the earlier one, but it just becomes part of the normalizing constant).
So the posterior density is $$ textconstant times expBig(
textnegative constant times (mu-textsomething)^2 Big).$$
In other words, the posterior is Gaussian.
The posterior mean is a weighted average of the prior mean $mu_0$ and the sample mean $overline x,$ with weights proportional to the reciprocals variances $tau^2$ (for the prior) and $sigma^2/n$ (for the sample mean).
answered 4 hours ago
Michael Hardy
3,1051330
However, just to note that this works when $mu$ enters linearly in the quadratic form in the Gaussian density of $[X mid mu]$. If we have $x_i - f(mu)$, with $f(cdot)$ nonlinear, then we don’t get a Gaussian posterior.
– Dimitris Rizopoulos
3 hours ago
add a comment |
up vote
4
down vote
accepted
The exponents in the prior density and the likelihood are added to each other
$$
frac(mu-mu_0)^2tau^2 + frac(overline x - mu)^2sigma^2/n
quad = quad fracsigma^2(mu-mu_0)^2 + tau^2(overline x - mu)^2sigma^2tau^2/n tag 1
$$
Now let's work on the numerator:
$$
((sigma^2/n)+tau^2) Big(mu^2 - 2(mu_0sigma^2 + overline x tau^2) + text“constant'' Big) tag 2
$$
where “constant” means not depending on $mu$.
Now complete the square:
$$
Big(mu - (mu_0 sigma^2 + overline x tau^2)Big)^2 + text“constant''
$$
(where this “constant” will differ from the earlier one, but it just becomes part of the normalizing constant).
So the posterior density is $$ textconstant times expBig(
textnegative constant times (mu-textsomething)^2 Big).$$
In other words, the posterior is Gaussian.
The posterior mean is a weighted average of the prior mean $mu_0$ and the sample mean $overline x,$ with weights proportional to the reciprocals variances $tau^2$ (for the prior) and $sigma^2/n$ (for the sample mean).
answered 4 hours ago
Michael Hardy
3,1051330
However, just to note that this works when $mu$ enters linearly in the quadratic form in the Gaussian density of $[X mid mu]$. If we have $x_i - f(mu)$, with $f(cdot)$ nonlinear, then we don’t get a Gaussian posterior.
– Dimitris Rizopoulos
3 hours ago
add a comment |
up vote
4
down vote
accepted
up vote
4
down vote
accepted
The exponents in the prior density and the likelihood are added to each other
$$
frac(mu-mu_0)^2tau^2 + frac(overline x - mu)^2sigma^2/n
quad = quad fracsigma^2(mu-mu_0)^2 + tau^2(overline x - mu)^2sigma^2tau^2/n tag 1
$$
Now let's work on the numerator:
$$
((sigma^2/n)+tau^2) Big(mu^2 - 2(mu_0sigma^2 + overline x tau^2) + text“constant'' Big) tag 2
$$
where “constant” means not depending on $mu$.
Now complete the square:
$$
Big(mu - (mu_0 sigma^2 + overline x tau^2)Big)^2 + text“constant''
$$
(where this “constant” will differ from the earlier one, but it just becomes part of the normalizing constant).
So the posterior density is $$ textconstant times expBig(
textnegative constant times (mu-textsomething)^2 Big).$$
In other words, the posterior is Gaussian.
The posterior mean is a weighted average of the prior mean $mu_0$ and the sample mean $overline x,$ with weights proportional to the reciprocals variances $tau^2$ (for the prior) and $sigma^2/n$ (for the sample mean).
answered 4 hours ago
Michael Hardy
3,1051330
The exponents in the prior density and the likelihood are added to each other
$$
frac(mu-mu_0)^2tau^2 + frac(overline x - mu)^2sigma^2/n
quad = quad fracsigma^2(mu-mu_0)^2 + tau^2(overline x - mu)^2sigma^2tau^2/n tag 1
$$
Now let's work on the numerator:
$$
((sigma^2/n)+tau^2) Big(mu^2 - 2(mu_0sigma^2 + overline x tau^2) + text“constant'' Big) tag 2
$$
where “constant” means not depending on $mu$.
Now complete the square:
$$
Big(mu - (mu_0 sigma^2 + overline x tau^2)Big)^2 + text“constant''
$$
(where this “constant” will differ from the earlier one, but it just becomes part of the normalizing constant).
So the posterior density is $$ textconstant times expBig(
textnegative constant times (mu-textsomething)^2 Big).$$
In other words, the posterior is Gaussian.
The posterior mean is a weighted average of the prior mean $mu_0$ and the sample mean $overline x,$ with weights proportional to the reciprocals variances $tau^2$ (for the prior) and $sigma^2/n$ (for the sample mean).
answered 4 hours ago
Michael Hardy
3,1051330
answered 4 hours ago
Michael Hardy
3,1051330
answered 4 hours ago
Michael Hardy
3,1051330
answered 4 hours ago
Michael Hardy
3,1051330
3,1051330
However, just to note that this works when $mu$ enters linearly in the quadratic form in the Gaussian density of $[X mid mu]$. If we have $x_i - f(mu)$, with $f(cdot)$ nonlinear, then we don’t get a Gaussian posterior.
– Dimitris Rizopoulos
3 hours ago
add a comment |
However, just to note that this works when $mu$ enters linearly in the quadratic form in the Gaussian density of $[X mid mu]$. If we have $x_i - f(mu)$, with $f(cdot)$ nonlinear, then we don’t get a Gaussian posterior.
– Dimitris Rizopoulos
3 hours ago
However, just to note that this works when $mu$ enters linearly in the quadratic form in the Gaussian density of $[X mid mu]$. If we have $x_i - f(mu)$, with $f(cdot)$ nonlinear, then we don’t get a Gaussian posterior.
– Dimitris Rizopoulos
3 hours ago
However, just to note that this works when $mu$ enters linearly in the quadratic form in the Gaussian density of $[X mid mu]$. If we have $x_i - f(mu)$, with $f(cdot)$ nonlinear, then we don’t get a Gaussian posterior.
– Dimitris Rizopoulos
3 hours ago
add a comment |
Alex is a new contributor. Be nice, and check out our Code of Conduct.
Alex is a new contributor. Be nice, and check out our Code of Conduct.
Alex is a new contributor. Be nice, and check out our Code of Conduct.
Alex is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f376340%2fif-both-prior-and-likelihood-are-gaussian-what-can-we-say-about-the-posterior%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
1
Welcome to CV. Since you’re new here, you may want to take our tour, which has information for new users. Please add the [self-study] tag and read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. Please make these changes as just posting your homework & hoping someone will do it for you is grounds for closing. If this is self-study rather than homework, let us know, and... it's still a good idea to show us what you've tried or explain specifically what you don't understand.
– jbowman
5 hours ago
@Alex, do you mean to say "As posterior is proportional to prior*likelihood" ?
– curious_dan
2 hours ago
@curious_dan, yes thanks
– Alex
1 hour ago