A Gaussian Mixture Model Is a Universal Approximator of Densities
Clash Royale CLAN TAG#URR8PPP
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;
up vote
2
down vote
favorite
When discussing the concept of mixtures of distributions in my machine learning textbook, the authors state the following:
A Gaussian mixture model is a universal approximator of densities, in the sense that any smooth density can be approximated with any specific nonzero amount of error by a Gaussian mixture model with enough components.
I only have a basic background in probability and statistics, and I have absolutely no idea what this section is saying.
I would greatly appreciate it if people could please take the time to explain this in a way that is more understandable to someone of my level.
machine-learning distributions gaussian-mixture density-estimation
asked Sep 3 at 8:15
The Pointer
28817
add a comment |Â
up vote
2
down vote
favorite
When discussing the concept of mixtures of distributions in my machine learning textbook, the authors state the following:
A Gaussian mixture model is a universal approximator of densities, in the sense that any smooth density can be approximated with any specific nonzero amount of error by a Gaussian mixture model with enough components.
I only have a basic background in probability and statistics, and I have absolutely no idea what this section is saying.
I would greatly appreciate it if people could please take the time to explain this in a way that is more understandable to someone of my level.
machine-learning distributions gaussian-mixture density-estimation
asked Sep 3 at 8:15
The Pointer
28817
I applied this idea once, very early in my career, to carry out quantum-mechanical calculations by using a basis of Gaussians to model bonds in organic molecules. After weeks of running into problems I realized, to my chagrin, that the densities I was trying to approximate were so sharply peaked that it was hopeless to develop a good approximation without using a ridiculously large number of Gaussians, which are too smooth for the job. Thus, one should always be cognizant of the distinction between a purely mathematical theorem and the practical needs of computation.
– whuber♦
Sep 3 at 13:33
add a comment |Â
up vote
2
down vote
favorite
up vote
2
down vote
favorite
When discussing the concept of mixtures of distributions in my machine learning textbook, the authors state the following:
A Gaussian mixture model is a universal approximator of densities, in the sense that any smooth density can be approximated with any specific nonzero amount of error by a Gaussian mixture model with enough components.
I only have a basic background in probability and statistics, and I have absolutely no idea what this section is saying.
I would greatly appreciate it if people could please take the time to explain this in a way that is more understandable to someone of my level.
machine-learning distributions gaussian-mixture density-estimation
asked Sep 3 at 8:15
The Pointer
28817
When discussing the concept of mixtures of distributions in my machine learning textbook, the authors state the following:
A Gaussian mixture model is a universal approximator of densities, in the sense that any smooth density can be approximated with any specific nonzero amount of error by a Gaussian mixture model with enough components.
I only have a basic background in probability and statistics, and I have absolutely no idea what this section is saying.
I would greatly appreciate it if people could please take the time to explain this in a way that is more understandable to someone of my level.
machine-learning distributions gaussian-mixture density-estimation
machine-learning distributions gaussian-mixture density-estimation
asked Sep 3 at 8:15
The Pointer
28817
asked Sep 3 at 8:15
The Pointer
28817
asked Sep 3 at 8:15
The Pointer
28817
asked Sep 3 at 8:15
The Pointer
28817
asked Sep 3 at 8:15
The Pointer
28817
28817
I applied this idea once, very early in my career, to carry out quantum-mechanical calculations by using a basis of Gaussians to model bonds in organic molecules. After weeks of running into problems I realized, to my chagrin, that the densities I was trying to approximate were so sharply peaked that it was hopeless to develop a good approximation without using a ridiculously large number of Gaussians, which are too smooth for the job. Thus, one should always be cognizant of the distinction between a purely mathematical theorem and the practical needs of computation.
– whuber♦
Sep 3 at 13:33
add a comment |Â
I applied this idea once, very early in my career, to carry out quantum-mechanical calculations by using a basis of Gaussians to model bonds in organic molecules. After weeks of running into problems I realized, to my chagrin, that the densities I was trying to approximate were so sharply peaked that it was hopeless to develop a good approximation without using a ridiculously large number of Gaussians, which are too smooth for the job. Thus, one should always be cognizant of the distinction between a purely mathematical theorem and the practical needs of computation.
– whuber♦
Sep 3 at 13:33
I applied this idea once, very early in my career, to carry out quantum-mechanical calculations by using a basis of Gaussians to model bonds in organic molecules. After weeks of running into problems I realized, to my chagrin, that the densities I was trying to approximate were so sharply peaked that it was hopeless to develop a good approximation without using a ridiculously large number of Gaussians, which are too smooth for the job. Thus, one should always be cognizant of the distinction between a purely mathematical theorem and the practical needs of computation.
– whuber♦
Sep 3 at 13:33
I applied this idea once, very early in my career, to carry out quantum-mechanical calculations by using a basis of Gaussians to model bonds in organic molecules. After weeks of running into problems I realized, to my chagrin, that the densities I was trying to approximate were so sharply peaked that it was hopeless to develop a good approximation without using a ridiculously large number of Gaussians, which are too smooth for the job. Thus, one should always be cognizant of the distinction between a purely mathematical theorem and the practical needs of computation.
– whuber♦
Sep 3 at 13:33
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
6
down vote
accepted
The idea is that an arbitrary density on $mathbbR$, $f(cdot)$, can be approximated by a Gaussian mixture model
$$g_k(cdot;boldsymbolomega,mu,sigma)=sum_i=1^k omega_i varphi(cdot;mu_i,sigma_i)$$
in the sense that
$$mathfrakD(f(cdot),g_k(cdot;boldsymbolomega,mu,sigma)stackrelktoinftylongrightarrow0$$
for some specific functional distance $mathfrakD(cdot,cdot)$. This result only applies for weaker types of distance.
answered Sep 3 at 8:30
Xi'an
49.9k685333
1
Thanks. Your answer is clear.
– The Pointer
Sep 3 at 8:57
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
6
down vote
accepted
The idea is that an arbitrary density on $mathbbR$, $f(cdot)$, can be approximated by a Gaussian mixture model
$$g_k(cdot;boldsymbolomega,mu,sigma)=sum_i=1^k omega_i varphi(cdot;mu_i,sigma_i)$$
in the sense that
$$mathfrakD(f(cdot),g_k(cdot;boldsymbolomega,mu,sigma)stackrelktoinftylongrightarrow0$$
for some specific functional distance $mathfrakD(cdot,cdot)$. This result only applies for weaker types of distance.
answered Sep 3 at 8:30
Xi'an
49.9k685333
1
Thanks. Your answer is clear.
– The Pointer
Sep 3 at 8:57
add a comment |Â
up vote
6
down vote
accepted
The idea is that an arbitrary density on $mathbbR$, $f(cdot)$, can be approximated by a Gaussian mixture model
$$g_k(cdot;boldsymbolomega,mu,sigma)=sum_i=1^k omega_i varphi(cdot;mu_i,sigma_i)$$
in the sense that
$$mathfrakD(f(cdot),g_k(cdot;boldsymbolomega,mu,sigma)stackrelktoinftylongrightarrow0$$
for some specific functional distance $mathfrakD(cdot,cdot)$. This result only applies for weaker types of distance.
answered Sep 3 at 8:30
Xi'an
49.9k685333
1
Thanks. Your answer is clear.
– The Pointer
Sep 3 at 8:57
add a comment |Â
up vote
6
down vote
accepted
up vote
6
down vote
accepted
The idea is that an arbitrary density on $mathbbR$, $f(cdot)$, can be approximated by a Gaussian mixture model
$$g_k(cdot;boldsymbolomega,mu,sigma)=sum_i=1^k omega_i varphi(cdot;mu_i,sigma_i)$$
in the sense that
$$mathfrakD(f(cdot),g_k(cdot;boldsymbolomega,mu,sigma)stackrelktoinftylongrightarrow0$$
for some specific functional distance $mathfrakD(cdot,cdot)$. This result only applies for weaker types of distance.
answered Sep 3 at 8:30
Xi'an
49.9k685333
The idea is that an arbitrary density on $mathbbR$, $f(cdot)$, can be approximated by a Gaussian mixture model
$$g_k(cdot;boldsymbolomega,mu,sigma)=sum_i=1^k omega_i varphi(cdot;mu_i,sigma_i)$$
in the sense that
$$mathfrakD(f(cdot),g_k(cdot;boldsymbolomega,mu,sigma)stackrelktoinftylongrightarrow0$$
for some specific functional distance $mathfrakD(cdot,cdot)$. This result only applies for weaker types of distance.
answered Sep 3 at 8:30
Xi'an
49.9k685333
answered Sep 3 at 8:30
Xi'an
49.9k685333
answered Sep 3 at 8:30
Xi'an
49.9k685333
answered Sep 3 at 8:30
Xi'an
49.9k685333
49.9k685333
1
Thanks. Your answer is clear.
– The Pointer
Sep 3 at 8:57
add a comment |Â
1
Thanks. Your answer is clear.
– The Pointer
Sep 3 at 8:57
1
1
Thanks. Your answer is clear.
– The Pointer
Sep 3 at 8:57
Thanks. Your answer is clear.
– The Pointer
Sep 3 at 8:57
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f365155%2fa-gaussian-mixture-model-is-a-universal-approximator-of-densities%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
I applied this idea once, very early in my career, to carry out quantum-mechanical calculations by using a basis of Gaussians to model bonds in organic molecules. After weeks of running into problems I realized, to my chagrin, that the densities I was trying to approximate were so sharply peaked that it was hopeless to develop a good approximation without using a ridiculously large number of Gaussians, which are too smooth for the job. Thus, one should always be cognizant of the distinction between a purely mathematical theorem and the practical needs of computation.
– whuber♦
Sep 3 at 13:33