Can a proper prior and exponentiated likelihood lead to an improper posterior?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;







up vote
6
down vote

favorite
3












(This question is inspired by this comment from Xi'an.)



It is well known that if the prior distribution $pi(theta)$ is proper and the likelihood $L(theta | x)$ is well-defined, then the posterior distribution $pi(theta|x)propto pi(theta) L(theta|x)$ is proper almost surely.



In some cases, we use instead a tempered or exponentiated likelihood, leading to a pseudo-posterior



$$tildepi(theta|x)propto pi(theta) L(theta|x)^alpha$$
for some $alpha>0$ (for example, this can have computational advantages).



In this setting, is it possible to have a proper prior but an improper pseudo-posterior?










share|cite|improve this question



















  • 1




    Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ α product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
    – Xi'an
    14 hours ago
















up vote
6
down vote

favorite
3












(This question is inspired by this comment from Xi'an.)



It is well known that if the prior distribution $pi(theta)$ is proper and the likelihood $L(theta | x)$ is well-defined, then the posterior distribution $pi(theta|x)propto pi(theta) L(theta|x)$ is proper almost surely.



In some cases, we use instead a tempered or exponentiated likelihood, leading to a pseudo-posterior



$$tildepi(theta|x)propto pi(theta) L(theta|x)^alpha$$
for some $alpha>0$ (for example, this can have computational advantages).



In this setting, is it possible to have a proper prior but an improper pseudo-posterior?










share|cite|improve this question



















  • 1




    Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ α product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
    – Xi'an
    14 hours ago












up vote
6
down vote

favorite
3









up vote
6
down vote

favorite
3






3





(This question is inspired by this comment from Xi'an.)



It is well known that if the prior distribution $pi(theta)$ is proper and the likelihood $L(theta | x)$ is well-defined, then the posterior distribution $pi(theta|x)propto pi(theta) L(theta|x)$ is proper almost surely.



In some cases, we use instead a tempered or exponentiated likelihood, leading to a pseudo-posterior



$$tildepi(theta|x)propto pi(theta) L(theta|x)^alpha$$
for some $alpha>0$ (for example, this can have computational advantages).



In this setting, is it possible to have a proper prior but an improper pseudo-posterior?










share|cite|improve this question















(This question is inspired by this comment from Xi'an.)



It is well known that if the prior distribution $pi(theta)$ is proper and the likelihood $L(theta | x)$ is well-defined, then the posterior distribution $pi(theta|x)propto pi(theta) L(theta|x)$ is proper almost surely.



In some cases, we use instead a tempered or exponentiated likelihood, leading to a pseudo-posterior



$$tildepi(theta|x)propto pi(theta) L(theta|x)^alpha$$
for some $alpha>0$ (for example, this can have computational advantages).



In this setting, is it possible to have a proper prior but an improper pseudo-posterior?







bayesian prior posterior






share|cite|improve this question















share|cite|improve this question













share|cite|improve this question




share|cite|improve this question








edited 14 hours ago

























asked 14 hours ago









Robin Ryder

1,010513




1,010513







  • 1




    Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ α product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
    – Xi'an
    14 hours ago












  • 1




    Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ α product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
    – Xi'an
    14 hours ago







1




1




Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ α product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
– Xi'an
14 hours ago




Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ α product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
– Xi'an
14 hours ago










2 Answers
2






active

oldest

votes

















up vote
4
down vote



accepted










For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?



We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.



On the RHS:



$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$



If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:



$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$



... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).






share|cite|improve this answer






















  • Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
    – Robin Ryder
    12 hours ago

















up vote
1
down vote













Not sure if this is super useful, but since I can't comment I will leave this in an answer. In addition to @InfProbSciX's excellent remark about $alpha leq 1$, if one makes the further assumption that $L(theta mid x) in L^p$, then it is impossible to have a proper prior but an improper pseudo-posterior for $ 1 < alpha leq p$. For instance, if we know that the second ($p$-th) moment of $L(theta mid x)$ exists, we know it is in $L^2$ ($L^p$) and hence the pseudo-posterior will proper for $0 leq alpha leq 2$. Section 1 in these notes goes into a bit more detail, but unfortunately it is not clear how broad the class of, say, $L^10$ pdfs is.
I apologise if I'm speaking out of turn here, I really wanted to leave this as a comment.






share|cite|improve this answer










New contributor




Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.













  • 1




    You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
    – InfProbSciX
    7 hours ago











Your Answer




StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "65"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













 

draft saved


draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f370919%2fcan-a-proper-prior-and-exponentiated-likelihood-lead-to-an-improper-posterior%23new-answer', 'question_page');

);

Post as a guest






























2 Answers
2






active

oldest

votes








2 Answers
2






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
4
down vote



accepted










For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?



We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.



On the RHS:



$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$



If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:



$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$



... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).






share|cite|improve this answer






















  • Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
    – Robin Ryder
    12 hours ago














up vote
4
down vote



accepted










For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?



We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.



On the RHS:



$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$



If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:



$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$



... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).






share|cite|improve this answer






















  • Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
    – Robin Ryder
    12 hours ago












up vote
4
down vote



accepted







up vote
4
down vote



accepted






For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?



We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.



On the RHS:



$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$



If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:



$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$



... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).






share|cite|improve this answer














For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?



We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.



On the RHS:



$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$



If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:



$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$



... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).







share|cite|improve this answer














share|cite|improve this answer



share|cite|improve this answer








edited 12 hours ago









Xi'an

50.4k686335




50.4k686335










answered 13 hours ago









InfProbSciX

1447




1447











  • Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
    – Robin Ryder
    12 hours ago
















  • Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
    – Robin Ryder
    12 hours ago















Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
– Robin Ryder
12 hours ago




Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
– Robin Ryder
12 hours ago












up vote
1
down vote













Not sure if this is super useful, but since I can't comment I will leave this in an answer. In addition to @InfProbSciX's excellent remark about $alpha leq 1$, if one makes the further assumption that $L(theta mid x) in L^p$, then it is impossible to have a proper prior but an improper pseudo-posterior for $ 1 < alpha leq p$. For instance, if we know that the second ($p$-th) moment of $L(theta mid x)$ exists, we know it is in $L^2$ ($L^p$) and hence the pseudo-posterior will proper for $0 leq alpha leq 2$. Section 1 in these notes goes into a bit more detail, but unfortunately it is not clear how broad the class of, say, $L^10$ pdfs is.
I apologise if I'm speaking out of turn here, I really wanted to leave this as a comment.






share|cite|improve this answer










New contributor




Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.













  • 1




    You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
    – InfProbSciX
    7 hours ago















up vote
1
down vote













Not sure if this is super useful, but since I can't comment I will leave this in an answer. In addition to @InfProbSciX's excellent remark about $alpha leq 1$, if one makes the further assumption that $L(theta mid x) in L^p$, then it is impossible to have a proper prior but an improper pseudo-posterior for $ 1 < alpha leq p$. For instance, if we know that the second ($p$-th) moment of $L(theta mid x)$ exists, we know it is in $L^2$ ($L^p$) and hence the pseudo-posterior will proper for $0 leq alpha leq 2$. Section 1 in these notes goes into a bit more detail, but unfortunately it is not clear how broad the class of, say, $L^10$ pdfs is.
I apologise if I'm speaking out of turn here, I really wanted to leave this as a comment.






share|cite|improve this answer










New contributor




Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.













  • 1




    You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
    – InfProbSciX
    7 hours ago













up vote
1
down vote










up vote
1
down vote









Not sure if this is super useful, but since I can't comment I will leave this in an answer. In addition to @InfProbSciX's excellent remark about $alpha leq 1$, if one makes the further assumption that $L(theta mid x) in L^p$, then it is impossible to have a proper prior but an improper pseudo-posterior for $ 1 < alpha leq p$. For instance, if we know that the second ($p$-th) moment of $L(theta mid x)$ exists, we know it is in $L^2$ ($L^p$) and hence the pseudo-posterior will proper for $0 leq alpha leq 2$. Section 1 in these notes goes into a bit more detail, but unfortunately it is not clear how broad the class of, say, $L^10$ pdfs is.
I apologise if I'm speaking out of turn here, I really wanted to leave this as a comment.






share|cite|improve this answer










New contributor




Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









Not sure if this is super useful, but since I can't comment I will leave this in an answer. In addition to @InfProbSciX's excellent remark about $alpha leq 1$, if one makes the further assumption that $L(theta mid x) in L^p$, then it is impossible to have a proper prior but an improper pseudo-posterior for $ 1 < alpha leq p$. For instance, if we know that the second ($p$-th) moment of $L(theta mid x)$ exists, we know it is in $L^2$ ($L^p$) and hence the pseudo-posterior will proper for $0 leq alpha leq 2$. Section 1 in these notes goes into a bit more detail, but unfortunately it is not clear how broad the class of, say, $L^10$ pdfs is.
I apologise if I'm speaking out of turn here, I really wanted to leave this as a comment.







share|cite|improve this answer










New contributor




Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|cite|improve this answer



share|cite|improve this answer








edited 8 hours ago





















New contributor




Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









answered 10 hours ago









Luiz Max Carvalho

234




234




New contributor




Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






Luiz Max Carvalho is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







  • 1




    You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
    – InfProbSciX
    7 hours ago













  • 1




    You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
    – InfProbSciX
    7 hours ago








1




1




You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
– InfProbSciX
7 hours ago





You're right, if the likelihood function $L(theta|x)$ is within the space $L^p(bf pi_theta)$ - i.e. the $L^p$ space w.r.t. the measure induced by the prior, then the posterior will be proper for $1 leq alpha leq p$. I'm totally guessing here, but I think that the space would encompass most likelihoods we can think of - I think I might have read a proof ages ago that says that if $f$ is Riemann integrable, then its positive powers are as well. $f^n, n in mathbb Z^+$ is integrable though. Theorem 1.26 for reference
– InfProbSciX
7 hours ago


















 

draft saved


draft discarded















































 


draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f370919%2fcan-a-proper-prior-and-exponentiated-likelihood-lead-to-an-improper-posterior%23new-answer', 'question_page');

);

Post as a guest













































































Popular posts from this blog

Peggy Mitchell

Palaiologos

The Forum (Inglewood, California)