mjhjmtu

Suppose we have a point mass prior,

$$theta sim begincases I(theta=1) ,& prob=frac12 \
Gamma(c,c), & prob=frac12 endcases$$

Then if we are asked

$lim_c to infty P(theta=1)$

Now here is the issue, since Gamma is a continuous distribution, it seems that in the case of gamma,we will never have $theta=1$.

To me it thus seems that regardless of the value of c, the $p(theta=1)=frac12$

However, we also have that since expected value of a $gamma(a,b)=fracab$ so that the expected value of the gamma is 1 when we have $a=b=c$

But, by Markov, for $X sim Gamma(c,c)$

$lim_c to infty Pr(|X-mu| lt epsilon) to 1$ for any $epsilon gt 0$

So is $lim_c to inftyP(theta=1) =1$ , or is $lim_c to inftyP(theta=1)=frac12$

As even though the markov inequality holds, it is a continous distirbution, so we will never have it exactly equal to 1.

Thanks all

Your specified prior distribution is a mixture of a continuous and discrete part. Let $I sim textBern(1/2)$ be the indicator that the parameter is taken from the gamma distribution. Then using the law of total probability you have:

$$beginequation beginaligned
mathbbP(theta = 1|c)
&= mathbbP(theta = 1|c, I=0) cdot mathbbP(I=0) + mathbbP(theta = 1|c, I=1) cdot mathbbP(I=1) \[6pt]
&= frac12 cdot mathbbP(theta = 1|c, I=0) + frac12 cdot mathbbP(theta = 1|c, I=1) \[6pt]
&= frac12 + frac12 cdot mathbbP(theta = 1| theta sim textGa(c,c)) \[6pt]
&= frac12. \[6pt]
endaligned endequation$$

(Note that the last step comes from recognising that the gamma is a continuous distribution, so the probability of a specific point is zero under this distribution.) This result holds for all values of $c$, so you are correct that it also holds in the limit:

$$lim_c rightarrow infty mathbbP(theta = 1|c) = lim_c rightarrow infty frac12 = frac12.$$

Your later use of Chebychev's inequality shows that as $c rightarrow infty$ you get $theta rightarrow 1$ (convergence in probability), but this does not change the fact that $mathbbP(theta = 1|c) = 1/2$ for all $c > 0$. (To understand the reason for this more fully, have a read about the distinction between convergence in probability and almost-sure convergence.)