Can a proper prior and exponentiated likelihood lead to an improper posterior?

Clash Royale CLAN TAG#URR8PPP
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;
up vote
6
down vote
favorite
(This question is inspired by this comment from Xi'an.)
It is well known that if the prior distribution $pi(theta)$ is proper and the likelihood $L(theta | x)$ is well-defined, then the posterior distribution $pi(theta|x)propto pi(theta) L(theta|x)$ is proper almost surely.
In some cases, we use instead a tempered or exponentiated likelihood, leading to a pseudo-posterior
$$tildepi(theta|x)propto pi(theta) L(theta|x)^alpha$$
for some $alpha>0$ (for example, this can have computational advantages).
In this setting, is it possible to have a proper prior but an improper pseudo-posterior?
bayesian prior posterior
add a comment |Â
up vote
6
down vote
favorite
(This question is inspired by this comment from Xi'an.)
It is well known that if the prior distribution $pi(theta)$ is proper and the likelihood $L(theta | x)$ is well-defined, then the posterior distribution $pi(theta|x)propto pi(theta) L(theta|x)$ is proper almost surely.
In some cases, we use instead a tempered or exponentiated likelihood, leading to a pseudo-posterior
$$tildepi(theta|x)propto pi(theta) L(theta|x)^alpha$$
for some $alpha>0$ (for example, this can have computational advantages).
In this setting, is it possible to have a proper prior but an improper pseudo-posterior?
bayesian prior posterior
1
Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ ñ product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
â Xi'an
14 hours ago
add a comment |Â
up vote
6
down vote
favorite
up vote
6
down vote
favorite
(This question is inspired by this comment from Xi'an.)
It is well known that if the prior distribution $pi(theta)$ is proper and the likelihood $L(theta | x)$ is well-defined, then the posterior distribution $pi(theta|x)propto pi(theta) L(theta|x)$ is proper almost surely.
In some cases, we use instead a tempered or exponentiated likelihood, leading to a pseudo-posterior
$$tildepi(theta|x)propto pi(theta) L(theta|x)^alpha$$
for some $alpha>0$ (for example, this can have computational advantages).
In this setting, is it possible to have a proper prior but an improper pseudo-posterior?
bayesian prior posterior
(This question is inspired by this comment from Xi'an.)
It is well known that if the prior distribution $pi(theta)$ is proper and the likelihood $L(theta | x)$ is well-defined, then the posterior distribution $pi(theta|x)propto pi(theta) L(theta|x)$ is proper almost surely.
In some cases, we use instead a tempered or exponentiated likelihood, leading to a pseudo-posterior
$$tildepi(theta|x)propto pi(theta) L(theta|x)^alpha$$
for some $alpha>0$ (for example, this can have computational advantages).
In this setting, is it possible to have a proper prior but an improper pseudo-posterior?
bayesian prior posterior
bayesian prior posterior
edited 14 hours ago
asked 14 hours ago
Robin Ryder
1,010513
1,010513
1
Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ ñ product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
â Xi'an
14 hours ago
add a comment |Â
1
Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ ñ product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
â Xi'an
14 hours ago
1
1
Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ ñ product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
â Xi'an
14 hours ago
Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ ñ product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
â Xi'an
14 hours ago
add a comment |Â
2 Answers
2
active
oldest
votes
up vote
4
down vote
accepted
For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?
We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.
On the RHS:
$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$
If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:
$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$
... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
â Robin Ryder
12 hours ago
add a comment |Â
up vote
1
down vote
Not sure if this is super useful, but since I can't comment I will leave this in an answer. In addition to @InfProbSciX's excellent remark about $alpha leq 1$, if one makes the further assumption that $L(theta mid x) in L^p$, then it is impossible to have a proper prior but an improper pseudo-posterior for $ 1 < alpha leq p$. For instance, if we know that the second ($p$-th) moment of $L(theta mid x)$ exists, we know it is in $L^2$ ($L^p$) and hence the pseudo-posterior will proper for $0 leq alpha leq 2$. Section 1 in these notes goes into a bit more detail, but unfortunately it is not clear how broad the class of, say, $L^10$ pdfs is.
I apologise if I'm speaking out of turn here, I really wanted to leave this as a comment.
New contributor
Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1
You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
â InfProbSciX
7 hours ago
add a comment |Â
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
4
down vote
accepted
For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?
We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.
On the RHS:
$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$
If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:
$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$
... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
â Robin Ryder
12 hours ago
add a comment |Â
up vote
4
down vote
accepted
For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?
We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.
On the RHS:
$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$
If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:
$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$
... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
â Robin Ryder
12 hours ago
add a comment |Â
up vote
4
down vote
accepted
up vote
4
down vote
accepted
For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?
We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.
On the RHS:
$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$
If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:
$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$
... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).
For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?
We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.
On the RHS:
$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$
If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:
$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$
... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).
edited 12 hours ago
Xi'an
50.4k686335
50.4k686335
answered 13 hours ago
InfProbSciX
1447
1447
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
â Robin Ryder
12 hours ago
add a comment |Â
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
â Robin Ryder
12 hours ago
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
â Robin Ryder
12 hours ago
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
â Robin Ryder
12 hours ago
add a comment |Â
up vote
1
down vote
Not sure if this is super useful, but since I can't comment I will leave this in an answer. In addition to @InfProbSciX's excellent remark about $alpha leq 1$, if one makes the further assumption that $L(theta mid x) in L^p$, then it is impossible to have a proper prior but an improper pseudo-posterior for $ 1 < alpha leq p$. For instance, if we know that the second ($p$-th) moment of $L(theta mid x)$ exists, we know it is in $L^2$ ($L^p$) and hence the pseudo-posterior will proper for $0 leq alpha leq 2$. Section 1 in these notes goes into a bit more detail, but unfortunately it is not clear how broad the class of, say, $L^10$ pdfs is.
I apologise if I'm speaking out of turn here, I really wanted to leave this as a comment.
New contributor
Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1
You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
â InfProbSciX
7 hours ago
add a comment |Â
up vote
1
down vote
Not sure if this is super useful, but since I can't comment I will leave this in an answer. In addition to @InfProbSciX's excellent remark about $alpha leq 1$, if one makes the further assumption that $L(theta mid x) in L^p$, then it is impossible to have a proper prior but an improper pseudo-posterior for $ 1 < alpha leq p$. For instance, if we know that the second ($p$-th) moment of $L(theta mid x)$ exists, we know it is in $L^2$ ($L^p$) and hence the pseudo-posterior will proper for $0 leq alpha leq 2$. Section 1 in these notes goes into a bit more detail, but unfortunately it is not clear how broad the class of, say, $L^10$ pdfs is.
I apologise if I'm speaking out of turn here, I really wanted to leave this as a comment.
New contributor
Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1
You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
â InfProbSciX
7 hours ago
add a comment |Â
up vote
1
down vote
up vote
1
down vote
Not sure if this is super useful, but since I can't comment I will leave this in an answer. In addition to @InfProbSciX's excellent remark about $alpha leq 1$, if one makes the further assumption that $L(theta mid x) in L^p$, then it is impossible to have a proper prior but an improper pseudo-posterior for $ 1 < alpha leq p$. For instance, if we know that the second ($p$-th) moment of $L(theta mid x)$ exists, we know it is in $L^2$ ($L^p$) and hence the pseudo-posterior will proper for $0 leq alpha leq 2$. Section 1 in these notes goes into a bit more detail, but unfortunately it is not clear how broad the class of, say, $L^10$ pdfs is.
I apologise if I'm speaking out of turn here, I really wanted to leave this as a comment.
New contributor
Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Not sure if this is super useful, but since I can't comment I will leave this in an answer. In addition to @InfProbSciX's excellent remark about $alpha leq 1$, if one makes the further assumption that $L(theta mid x) in L^p$, then it is impossible to have a proper prior but an improper pseudo-posterior for $ 1 < alpha leq p$. For instance, if we know that the second ($p$-th) moment of $L(theta mid x)$ exists, we know it is in $L^2$ ($L^p$) and hence the pseudo-posterior will proper for $0 leq alpha leq 2$. Section 1 in these notes goes into a bit more detail, but unfortunately it is not clear how broad the class of, say, $L^10$ pdfs is.
I apologise if I'm speaking out of turn here, I really wanted to leave this as a comment.
New contributor
Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 8 hours ago
New contributor
Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered 10 hours ago
Luiz Max Carvalho
234
234
New contributor
Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1
You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
â InfProbSciX
7 hours ago
add a comment |Â
1
You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
â InfProbSciX
7 hours ago
1
1
You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
â InfProbSciX
7 hours ago
You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
â InfProbSciX
7 hours ago
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f370919%2fcan-a-proper-prior-and-exponentiated-likelihood-lead-to-an-improper-posterior%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
1
Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ ñ product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
â Xi'an
14 hours ago