Inverse of the covariance matrix of a multivariate normal distribution
Clash Royale CLAN TAG#URR8PPP
$begingroup$
Is the covariance matrix of a multivariate normal distribution always invertible?
normal-distribution covariance-matrix matrix linear-algebra
edited Mar 5 at 13:51
kjetil b halvorsen
31.8k984235
asked Mar 5 at 13:43
DadooDadoo
375
$endgroup$
add a comment |
$begingroup$
Is the covariance matrix of a multivariate normal distribution always invertible?
normal-distribution covariance-matrix matrix linear-algebra
edited Mar 5 at 13:51
kjetil b halvorsen
31.8k984235
asked Mar 5 at 13:43
DadooDadoo
375
$endgroup$
1
$begingroup$
No, consider for instance $(X_1,X_2=2X_1)$ when $X_1simmathcal N(0,1).$
$endgroup$
– Xi'an
Mar 5 at 13:49
add a comment |
$begingroup$
Is the covariance matrix of a multivariate normal distribution always invertible?
normal-distribution covariance-matrix matrix linear-algebra
edited Mar 5 at 13:51
kjetil b halvorsen
31.8k984235
asked Mar 5 at 13:43
DadooDadoo
375
$endgroup$
Is the covariance matrix of a multivariate normal distribution always invertible?
normal-distribution covariance-matrix matrix linear-algebra
normal-distribution covariance-matrix matrix linear-algebra
edited Mar 5 at 13:51
kjetil b halvorsen
31.8k984235
asked Mar 5 at 13:43
DadooDadoo
375
edited Mar 5 at 13:51
kjetil b halvorsen
31.8k984235
asked Mar 5 at 13:43
DadooDadoo
375
edited Mar 5 at 13:51
kjetil b halvorsen
31.8k984235
edited Mar 5 at 13:51
kjetil b halvorsen
31.8k984235
edited Mar 5 at 13:51
kjetil b halvorsen
31.8k984235
31.8k984235
asked Mar 5 at 13:43
DadooDadoo
375
asked Mar 5 at 13:43
DadooDadoo
375
asked Mar 5 at 13:43
DadooDadoo
375
375
1
$begingroup$
No, consider for instance $(X_1,X_2=2X_1)$ when $X_1simmathcal N(0,1).$
$endgroup$
– Xi'an
Mar 5 at 13:49
add a comment |
1
$begingroup$
No, consider for instance $(X_1,X_2=2X_1)$ when $X_1simmathcal N(0,1).$
$endgroup$
– Xi'an
Mar 5 at 13:49
1
1
$begingroup$
No, consider for instance $(X_1,X_2=2X_1)$ when $X_1simmathcal N(0,1).$
$endgroup$
– Xi'an
Mar 5 at 13:49
$begingroup$
No, consider for instance $(X_1,X_2=2X_1)$ when $X_1simmathcal N(0,1).$
$endgroup$
– Xi'an
Mar 5 at 13:49
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
If the variables are perfectly correlated, i.e. $rho=1$, then covariance matrix becomes:
$$Sigma=beginbmatrixsigma_1^2 & sigma_1sigma_2 \ sigma_1sigma_2 & sigma_2^2 endbmatrix$$
and its determinant is $Delta=sigma_1^2sigma_2^2-sigma_1sigma_2sigma_1sigma_2=0$, which means the matrix is not invertible. A possible case this occurs is $X_1=alpha X_2$ as in @Xian's comment. Here $alpha>0$, but for $alpha<0$ $rho=-1$ which still doesn't save the $Sigma$.
It is only invertible when $|rho|<1$ since the covariance matrix is actually
$$Sigma=beginbmatrixsigma_1^2 & rhosigma_1sigma_2 \ rhosigma_1sigma_2 & sigma_2^2 endbmatrix$$
And, the determinant is $Delta=sigma_1^2sigma_2^2(1-rho^2)$, which is $>0$ when $|rho|<1$.
edited Mar 5 at 14:40
whuber♦
206k33453821
answered Mar 5 at 13:52
gunesgunes
6,7401215
$endgroup$
$begingroup$
Thank you very much. Therefore, if I assume that all the variables are not correlated with one another, is my covariance matrix invertible?
$endgroup$
– Dadoo
Mar 5 at 13:55
$begingroup$
Not correlated means $rho=0$, and it is perfectly invertible. And, if some correlation exists such that $|rho|<1$, still it is invertible since $Delta>0$.
$endgroup$
– gunes
Mar 5 at 13:58
$begingroup$
Thank you very much for the clarification.
$endgroup$
– Dadoo
Mar 5 at 14:02
6
$begingroup$
This comment thread seems to contain at least one pitfall for the unwary reader who might suppose it refers to anything more general than a bivariate normal distribution. For multivariate normal distributions with more than two variables, it is possible for all correlation coefficients to be close to zero, yet the matrix can still be singular.
$endgroup$
– whuber♦
Mar 5 at 14:43
$begingroup$
@whuber good point. Also, I think one should mention the cases (for bivariate normal distributions) where at least one of the two variance parameters is zero or extremely close to zero.
$endgroup$
– statmerkur
Mar 5 at 19:17
add a comment |
$begingroup$
No.
The covariance matrix of two perfectly correlated standard normal random variables is given by $Sigma = pmatrix1 & 1 \1 & 1$, which is not invertible.
answered Mar 5 at 13:51
MKRMKR
1655
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "65"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f395766%2finverse-of-the-covariance-matrix-of-a-multivariate-normal-distribution%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
If the variables are perfectly correlated, i.e. $rho=1$, then covariance matrix becomes:
$$Sigma=beginbmatrixsigma_1^2 & sigma_1sigma_2 \ sigma_1sigma_2 & sigma_2^2 endbmatrix$$
and its determinant is $Delta=sigma_1^2sigma_2^2-sigma_1sigma_2sigma_1sigma_2=0$, which means the matrix is not invertible. A possible case this occurs is $X_1=alpha X_2$ as in @Xian's comment. Here $alpha>0$, but for $alpha<0$ $rho=-1$ which still doesn't save the $Sigma$.
It is only invertible when $|rho|<1$ since the covariance matrix is actually
$$Sigma=beginbmatrixsigma_1^2 & rhosigma_1sigma_2 \ rhosigma_1sigma_2 & sigma_2^2 endbmatrix$$
And, the determinant is $Delta=sigma_1^2sigma_2^2(1-rho^2)$, which is $>0$ when $|rho|<1$.
edited Mar 5 at 14:40
whuber♦
206k33453821
answered Mar 5 at 13:52
gunesgunes
6,7401215
$endgroup$
$begingroup$
Thank you very much. Therefore, if I assume that all the variables are not correlated with one another, is my covariance matrix invertible?
$endgroup$
– Dadoo
Mar 5 at 13:55
$begingroup$
Not correlated means $rho=0$, and it is perfectly invertible. And, if some correlation exists such that $|rho|<1$, still it is invertible since $Delta>0$.
$endgroup$
– gunes
Mar 5 at 13:58
$begingroup$
Thank you very much for the clarification.
$endgroup$
– Dadoo
Mar 5 at 14:02
6
$begingroup$
This comment thread seems to contain at least one pitfall for the unwary reader who might suppose it refers to anything more general than a bivariate normal distribution. For multivariate normal distributions with more than two variables, it is possible for all correlation coefficients to be close to zero, yet the matrix can still be singular.
$endgroup$
– whuber♦
Mar 5 at 14:43
$begingroup$
@whuber good point. Also, I think one should mention the cases (for bivariate normal distributions) where at least one of the two variance parameters is zero or extremely close to zero.
$endgroup$
– statmerkur
Mar 5 at 19:17
add a comment |
$begingroup$
If the variables are perfectly correlated, i.e. $rho=1$, then covariance matrix becomes:
$$Sigma=beginbmatrixsigma_1^2 & sigma_1sigma_2 \ sigma_1sigma_2 & sigma_2^2 endbmatrix$$
and its determinant is $Delta=sigma_1^2sigma_2^2-sigma_1sigma_2sigma_1sigma_2=0$, which means the matrix is not invertible. A possible case this occurs is $X_1=alpha X_2$ as in @Xian's comment. Here $alpha>0$, but for $alpha<0$ $rho=-1$ which still doesn't save the $Sigma$.
It is only invertible when $|rho|<1$ since the covariance matrix is actually
$$Sigma=beginbmatrixsigma_1^2 & rhosigma_1sigma_2 \ rhosigma_1sigma_2 & sigma_2^2 endbmatrix$$
And, the determinant is $Delta=sigma_1^2sigma_2^2(1-rho^2)$, which is $>0$ when $|rho|<1$.
edited Mar 5 at 14:40
whuber♦
206k33453821
answered Mar 5 at 13:52
gunesgunes
6,7401215
$endgroup$
$begingroup$
Thank you very much. Therefore, if I assume that all the variables are not correlated with one another, is my covariance matrix invertible?
$endgroup$
– Dadoo
Mar 5 at 13:55
$begingroup$
Not correlated means $rho=0$, and it is perfectly invertible. And, if some correlation exists such that $|rho|<1$, still it is invertible since $Delta>0$.
$endgroup$
– gunes
Mar 5 at 13:58
$begingroup$
Thank you very much for the clarification.
$endgroup$
– Dadoo
Mar 5 at 14:02
6
$begingroup$
This comment thread seems to contain at least one pitfall for the unwary reader who might suppose it refers to anything more general than a bivariate normal distribution. For multivariate normal distributions with more than two variables, it is possible for all correlation coefficients to be close to zero, yet the matrix can still be singular.
$endgroup$
– whuber♦
Mar 5 at 14:43
$begingroup$
@whuber good point. Also, I think one should mention the cases (for bivariate normal distributions) where at least one of the two variance parameters is zero or extremely close to zero.
$endgroup$
– statmerkur
Mar 5 at 19:17
add a comment |
$begingroup$
If the variables are perfectly correlated, i.e. $rho=1$, then covariance matrix becomes:
$$Sigma=beginbmatrixsigma_1^2 & sigma_1sigma_2 \ sigma_1sigma_2 & sigma_2^2 endbmatrix$$
and its determinant is $Delta=sigma_1^2sigma_2^2-sigma_1sigma_2sigma_1sigma_2=0$, which means the matrix is not invertible. A possible case this occurs is $X_1=alpha X_2$ as in @Xian's comment. Here $alpha>0$, but for $alpha<0$ $rho=-1$ which still doesn't save the $Sigma$.
It is only invertible when $|rho|<1$ since the covariance matrix is actually
$$Sigma=beginbmatrixsigma_1^2 & rhosigma_1sigma_2 \ rhosigma_1sigma_2 & sigma_2^2 endbmatrix$$
And, the determinant is $Delta=sigma_1^2sigma_2^2(1-rho^2)$, which is $>0$ when $|rho|<1$.
edited Mar 5 at 14:40
whuber♦
206k33453821
answered Mar 5 at 13:52
gunesgunes
6,7401215
$endgroup$
If the variables are perfectly correlated, i.e. $rho=1$, then covariance matrix becomes:
$$Sigma=beginbmatrixsigma_1^2 & sigma_1sigma_2 \ sigma_1sigma_2 & sigma_2^2 endbmatrix$$
and its determinant is $Delta=sigma_1^2sigma_2^2-sigma_1sigma_2sigma_1sigma_2=0$, which means the matrix is not invertible. A possible case this occurs is $X_1=alpha X_2$ as in @Xian's comment. Here $alpha>0$, but for $alpha<0$ $rho=-1$ which still doesn't save the $Sigma$.
It is only invertible when $|rho|<1$ since the covariance matrix is actually
$$Sigma=beginbmatrixsigma_1^2 & rhosigma_1sigma_2 \ rhosigma_1sigma_2 & sigma_2^2 endbmatrix$$
And, the determinant is $Delta=sigma_1^2sigma_2^2(1-rho^2)$, which is $>0$ when $|rho|<1$.
edited Mar 5 at 14:40
whuber♦
206k33453821
answered Mar 5 at 13:52
gunesgunes
6,7401215
edited Mar 5 at 14:40
whuber♦
206k33453821
edited Mar 5 at 14:40
whuber♦
206k33453821
edited Mar 5 at 14:40
whuber♦
206k33453821
206k33453821
answered Mar 5 at 13:52
gunesgunes
6,7401215
answered Mar 5 at 13:52
gunesgunes
6,7401215
answered Mar 5 at 13:52
gunesgunes
6,7401215
6,7401215
$begingroup$
Thank you very much. Therefore, if I assume that all the variables are not correlated with one another, is my covariance matrix invertible?
$endgroup$
– Dadoo
Mar 5 at 13:55
$begingroup$
Not correlated means $rho=0$, and it is perfectly invertible. And, if some correlation exists such that $|rho|<1$, still it is invertible since $Delta>0$.
$endgroup$
– gunes
Mar 5 at 13:58
$begingroup$
Thank you very much for the clarification.
$endgroup$
– Dadoo
Mar 5 at 14:02
6
$begingroup$
This comment thread seems to contain at least one pitfall for the unwary reader who might suppose it refers to anything more general than a bivariate normal distribution. For multivariate normal distributions with more than two variables, it is possible for all correlation coefficients to be close to zero, yet the matrix can still be singular.
$endgroup$
– whuber♦
Mar 5 at 14:43
$begingroup$
@whuber good point. Also, I think one should mention the cases (for bivariate normal distributions) where at least one of the two variance parameters is zero or extremely close to zero.
$endgroup$
– statmerkur
Mar 5 at 19:17
add a comment |
$begingroup$
Thank you very much. Therefore, if I assume that all the variables are not correlated with one another, is my covariance matrix invertible?
$endgroup$
– Dadoo
Mar 5 at 13:55
$begingroup$
Not correlated means $rho=0$, and it is perfectly invertible. And, if some correlation exists such that $|rho|<1$, still it is invertible since $Delta>0$.
$endgroup$
– gunes
Mar 5 at 13:58
$begingroup$
Thank you very much for the clarification.
$endgroup$
– Dadoo
Mar 5 at 14:02
6
$begingroup$
This comment thread seems to contain at least one pitfall for the unwary reader who might suppose it refers to anything more general than a bivariate normal distribution. For multivariate normal distributions with more than two variables, it is possible for all correlation coefficients to be close to zero, yet the matrix can still be singular.
$endgroup$
– whuber♦
Mar 5 at 14:43
$begingroup$
@whuber good point. Also, I think one should mention the cases (for bivariate normal distributions) where at least one of the two variance parameters is zero or extremely close to zero.
$endgroup$
– statmerkur
Mar 5 at 19:17
$begingroup$
Thank you very much. Therefore, if I assume that all the variables are not correlated with one another, is my covariance matrix invertible?
$endgroup$
– Dadoo
Mar 5 at 13:55
$begingroup$
Thank you very much. Therefore, if I assume that all the variables are not correlated with one another, is my covariance matrix invertible?
$endgroup$
– Dadoo
Mar 5 at 13:55
$begingroup$
Not correlated means $rho=0$, and it is perfectly invertible. And, if some correlation exists such that $|rho|<1$, still it is invertible since $Delta>0$.
$endgroup$
– gunes
Mar 5 at 13:58
$begingroup$
Not correlated means $rho=0$, and it is perfectly invertible. And, if some correlation exists such that $|rho|<1$, still it is invertible since $Delta>0$.
$endgroup$
– gunes
Mar 5 at 13:58
$begingroup$
Thank you very much for the clarification.
$endgroup$
– Dadoo
Mar 5 at 14:02
$begingroup$
Thank you very much for the clarification.
$endgroup$
– Dadoo
Mar 5 at 14:02
6
6
$begingroup$
This comment thread seems to contain at least one pitfall for the unwary reader who might suppose it refers to anything more general than a bivariate normal distribution. For multivariate normal distributions with more than two variables, it is possible for all correlation coefficients to be close to zero, yet the matrix can still be singular.
$endgroup$
– whuber♦
Mar 5 at 14:43
$begingroup$
This comment thread seems to contain at least one pitfall for the unwary reader who might suppose it refers to anything more general than a bivariate normal distribution. For multivariate normal distributions with more than two variables, it is possible for all correlation coefficients to be close to zero, yet the matrix can still be singular.
$endgroup$
– whuber♦
Mar 5 at 14:43
$begingroup$
@whuber good point. Also, I think one should mention the cases (for bivariate normal distributions) where at least one of the two variance parameters is zero or extremely close to zero.
$endgroup$
– statmerkur
Mar 5 at 19:17
$begingroup$
@whuber good point. Also, I think one should mention the cases (for bivariate normal distributions) where at least one of the two variance parameters is zero or extremely close to zero.
$endgroup$
– statmerkur
Mar 5 at 19:17
add a comment |
$begingroup$
No.
The covariance matrix of two perfectly correlated standard normal random variables is given by $Sigma = pmatrix1 & 1 \1 & 1$, which is not invertible.
answered Mar 5 at 13:51
MKRMKR
1655
$endgroup$
add a comment |
$begingroup$
No.
The covariance matrix of two perfectly correlated standard normal random variables is given by $Sigma = pmatrix1 & 1 \1 & 1$, which is not invertible.
answered Mar 5 at 13:51
MKRMKR
1655
$endgroup$
add a comment |
$begingroup$
No.
The covariance matrix of two perfectly correlated standard normal random variables is given by $Sigma = pmatrix1 & 1 \1 & 1$, which is not invertible.
answered Mar 5 at 13:51
MKRMKR
1655
$endgroup$
No.
The covariance matrix of two perfectly correlated standard normal random variables is given by $Sigma = pmatrix1 & 1 \1 & 1$, which is not invertible.
answered Mar 5 at 13:51
MKRMKR
1655
answered Mar 5 at 13:51
MKRMKR
1655
answered Mar 5 at 13:51
MKRMKR
1655
answered Mar 5 at 13:51
MKRMKR
1655
1655
add a comment |
add a comment |
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f395766%2finverse-of-the-covariance-matrix-of-a-multivariate-normal-distribution%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
$begingroup$
No, consider for instance $(X_1,X_2=2X_1)$ when $X_1simmathcal N(0,1).$
$endgroup$
– Xi'an
Mar 5 at 13:49