Memory behavior during huge mysql import
Clash Royale CLAN TAG#URR8PPP
up vote
-1
down vote
favorite
I am importing a humongous sql file (~900GB) into mysql 5.5
The machine has 110G of RAM.
This is my vmstat output.
Why is the cache so huge? Is it because the contents are being cached before dumped into the db?
Would it help if I either periodically flushed it using /proc/sys/vm/drop_caches or altered the dirty pages allowed via the same directory tunables?
Any elaboration why the cache is behaving in such a way?
p.s: After starting the import via mysql> source /path/to/dump I reniced -20 the process in case this would speed up the import.
memory mysql cache-memory mysql-import
edited Nov 18 at 21:20
Rui F Ribeiro
38.2k1475123
asked Nov 18 at 21:14
pkaramol
430216
add a comment |
up vote
-1
down vote
favorite
I am importing a humongous sql file (~900GB) into mysql 5.5
The machine has 110G of RAM.
This is my vmstat output.
Why is the cache so huge? Is it because the contents are being cached before dumped into the db?
Would it help if I either periodically flushed it using /proc/sys/vm/drop_caches or altered the dirty pages allowed via the same directory tunables?
Any elaboration why the cache is behaving in such a way?
p.s: After starting the import via mysql> source /path/to/dump I reniced -20 the process in case this would speed up the import.
memory mysql cache-memory mysql-import
edited Nov 18 at 21:20
Rui F Ribeiro
38.2k1475123
asked Nov 18 at 21:14
pkaramol
430216
2
linuxatemyram.com
– Ipor Sircer
Nov 18 at 21:17
add a comment |
up vote
-1
down vote
favorite
up vote
-1
down vote
favorite
I am importing a humongous sql file (~900GB) into mysql 5.5
The machine has 110G of RAM.
This is my vmstat output.
Why is the cache so huge? Is it because the contents are being cached before dumped into the db?
Would it help if I either periodically flushed it using /proc/sys/vm/drop_caches or altered the dirty pages allowed via the same directory tunables?
Any elaboration why the cache is behaving in such a way?
p.s: After starting the import via mysql> source /path/to/dump I reniced -20 the process in case this would speed up the import.
memory mysql cache-memory mysql-import
edited Nov 18 at 21:20
Rui F Ribeiro
38.2k1475123
asked Nov 18 at 21:14
pkaramol
430216
I am importing a humongous sql file (~900GB) into mysql 5.5
The machine has 110G of RAM.
This is my vmstat output.
Why is the cache so huge? Is it because the contents are being cached before dumped into the db?
Would it help if I either periodically flushed it using /proc/sys/vm/drop_caches or altered the dirty pages allowed via the same directory tunables?
Any elaboration why the cache is behaving in such a way?
p.s: After starting the import via mysql> source /path/to/dump I reniced -20 the process in case this would speed up the import.
memory mysql cache-memory mysql-import
memory mysql cache-memory mysql-import
edited Nov 18 at 21:20
Rui F Ribeiro
38.2k1475123
asked Nov 18 at 21:14
pkaramol
430216
edited Nov 18 at 21:20
Rui F Ribeiro
38.2k1475123
asked Nov 18 at 21:14
pkaramol
430216
edited Nov 18 at 21:20
Rui F Ribeiro
38.2k1475123
edited Nov 18 at 21:20
Rui F Ribeiro
38.2k1475123
edited Nov 18 at 21:20
Rui F Ribeiro
38.2k1475123
38.2k1475123
asked Nov 18 at 21:14
pkaramol
430216
asked Nov 18 at 21:14
pkaramol
430216
asked Nov 18 at 21:14
pkaramol
430216
430216
2
linuxatemyram.com
– Ipor Sircer
Nov 18 at 21:17
add a comment |
2
linuxatemyram.com
– Ipor Sircer
Nov 18 at 21:17
2
2
linuxatemyram.com
– Ipor Sircer
Nov 18 at 21:17
linuxatemyram.com
– Ipor Sircer
Nov 18 at 21:17
add a comment |
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f482607%2fmemory-behavior-during-huge-mysql-import%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
linuxatemyram.com
– Ipor Sircer
Nov 18 at 21:17