How many CPU do I have and how many jobs should I submit?

up vote
0
down vote

favorite

I do some scientific calculation on a PC(in fact several PCs) and I want to know how many jobs should I submit one time. lscpu shows:

CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1

The ambiguity is 'Thread'. I searched the net and learned something about it. But I still felt confused (it is said that how many jobs should I submit is depend). I do not care about the details of machines. For example, now I have an executable file. If I run it directly, it spends about 10 min. Assume now I have 800 of them need to be run. Should I run 4 of them a time or 8 to reduce the total time cost?

asked Jan 15 at 2:58

hengyue li

Is your PC dedicated to this one task, or must some resources be retained for other programs?
â€“Â RonJohn
Jan 15 at 3:58

I use this PC to run jobs only. By the way, I use bsub (openlava).
â€“Â hengyue li
Jan 15 at 4:48

add a commentÂ |Â

up vote
0
down vote

favorite

I do some scientific calculation on a PC(in fact several PCs) and I want to know how many jobs should I submit one time. lscpu shows:

CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1

asked Jan 15 at 2:58

hengyue li

Is your PC dedicated to this one task, or must some resources be retained for other programs?
â€“Â RonJohn
Jan 15 at 3:58

I use this PC to run jobs only. By the way, I use bsub (openlava).
â€“Â hengyue li
Jan 15 at 4:48

add a commentÂ |Â

up vote
0
down vote

favorite

I do some scientific calculation on a PC(in fact several PCs) and I want to know how many jobs should I submit one time. lscpu shows:

CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1

asked Jan 15 at 2:58

hengyue li

I do some scientific calculation on a PC(in fact several PCs) and I want to know how many jobs should I submit one time. lscpu shows:

CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1

asked Jan 15 at 2:58

hengyue li

asked Jan 15 at 2:58

hengyue li

asked Jan 15 at 2:58

hengyue li

asked Jan 15 at 2:58

hengyue li

Is your PC dedicated to this one task, or must some resources be retained for other programs?
â€“Â RonJohn
Jan 15 at 3:58

I use this PC to run jobs only. By the way, I use bsub (openlava).
â€“Â hengyue li
Jan 15 at 4:48

add a commentÂ |Â

Is your PC dedicated to this one task, or must some resources be retained for other programs?
â€“Â RonJohn
Jan 15 at 3:58

I use this PC to run jobs only. By the way, I use bsub (openlava).
â€“Â hengyue li
Jan 15 at 4:48

Is your PC dedicated to this one task, or must some resources be retained for other programs?
â€“Â RonJohn
Jan 15 at 3:58

I use this PC to run jobs only. By the way, I use bsub (openlava).
â€“Â hengyue li
Jan 15 at 4:48

add a commentÂ |Â

1 Answer
1

active

oldest

votes

up vote
1
down vote

accepted

If this PC has an Intel CPU, then the Thread(s) per core most certainly indicates hyper-threading.

https://en.wikipedia.org/wiki/Hyper-threading

For each processor core that is physically present, the operating system addresses two virtual (logical) cores and shares the workload between them when possible. The main function of hyper-threading is to increase the number of independent instructions in the pipeline; it takes advantage of superscalar architecture, in which multiple instructions operate on separate data in parallel.

Should I run 4 of them a time or 8 to reduce the total time cost?

It depends. Some tasks run faster under hyper-threading, and some don't. You'll have to test that yourself.

Assume now I have 800 of them need to be run.

I'd use GNU Parallel to handle this problem.

https://www.gnu.org/software/parallel/

GNU parallel is a shell tool for executing jobs in parallel using one or more computers.

If you've got a list of of files in . which need to be processed, this will work:

find . | parallel -j4 yourprogram

If your earlier tests show that it runs faster with hyper-threading, then change the "4" to an "8".

EDIT: forgot to mention that sometimes programs run faster when you disable HT in the BIOS.

edited Jan 15 at 5:42

answered Jan 15 at 4:13

RonJohn

471213

I made the program myself. Is there any guideline I can follow to write the code so that I can know if I use 4 or 8?
â€“Â hengyue li
Jan 15 at 4:50

I don't know. Here's one link, though: software.intel.com/en-us/articles/â€¦
â€“Â RonJohn
Jan 15 at 5:40

You may have to use a number less than 4 if the memory requirements are large (you don't want to have thrashing) or there's a lot of file i/o required relative to the disk throughput that you have. Or you could use a number higher than 8 if each job alternates between periods of pure CPU usage and i/o wait. Best thing is to test.
â€“Â Mark Plotnick
Jan 15 at 7:58

add a commentÂ |Â

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);

);

draft saved

draft discarded

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f417153%2fhow-many-cpu-do-i-have-and-how-many-jobs-should-i-submit%23new-answer', 'question_page');

);

Post as a guest

Name

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

up vote
1
down vote

accepted

If this PC has an Intel CPU, then the Thread(s) per core most certainly indicates hyper-threading.

https://en.wikipedia.org/wiki/Hyper-threading

For each processor core that is physically present, the operating system addresses two virtual (logical) cores and shares the workload between them when possible. The main function of hyper-threading is to increase the number of independent instructions in the pipeline; it takes advantage of superscalar architecture, in which multiple instructions operate on separate data in parallel.

Should I run 4 of them a time or 8 to reduce the total time cost?

It depends. Some tasks run faster under hyper-threading, and some don't. You'll have to test that yourself.

Assume now I have 800 of them need to be run.

I'd use GNU Parallel to handle this problem.

https://www.gnu.org/software/parallel/

GNU parallel is a shell tool for executing jobs in parallel using one or more computers.

If you've got a list of of files in . which need to be processed, this will work:

find . | parallel -j4 yourprogram

If your earlier tests show that it runs faster with hyper-threading, then change the "4" to an "8".

EDIT: forgot to mention that sometimes programs run faster when you disable HT in the BIOS.

edited Jan 15 at 5:42

answered Jan 15 at 4:13

RonJohn

471213

I made the program myself. Is there any guideline I can follow to write the code so that I can know if I use 4 or 8?
â€“Â hengyue li
Jan 15 at 4:50

I don't know. Here's one link, though: software.intel.com/en-us/articles/â€¦
â€“Â RonJohn
Jan 15 at 5:40

You may have to use a number less than 4 if the memory requirements are large (you don't want to have thrashing) or there's a lot of file i/o required relative to the disk throughput that you have. Or you could use a number higher than 8 if each job alternates between periods of pure CPU usage and i/o wait. Best thing is to test.
â€“Â Mark Plotnick
Jan 15 at 7:58

add a commentÂ |Â

up vote
1
down vote

accepted

If this PC has an Intel CPU, then the Thread(s) per core most certainly indicates hyper-threading.

https://en.wikipedia.org/wiki/Hyper-threading

For each processor core that is physically present, the operating system addresses two virtual (logical) cores and shares the workload between them when possible. The main function of hyper-threading is to increase the number of independent instructions in the pipeline; it takes advantage of superscalar architecture, in which multiple instructions operate on separate data in parallel.

Should I run 4 of them a time or 8 to reduce the total time cost?

It depends. Some tasks run faster under hyper-threading, and some don't. You'll have to test that yourself.

Assume now I have 800 of them need to be run.

I'd use GNU Parallel to handle this problem.

https://www.gnu.org/software/parallel/

GNU parallel is a shell tool for executing jobs in parallel using one or more computers.

If you've got a list of of files in . which need to be processed, this will work:

find . | parallel -j4 yourprogram

If your earlier tests show that it runs faster with hyper-threading, then change the "4" to an "8".

EDIT: forgot to mention that sometimes programs run faster when you disable HT in the BIOS.

edited Jan 15 at 5:42

answered Jan 15 at 4:13

RonJohn

471213

I made the program myself. Is there any guideline I can follow to write the code so that I can know if I use 4 or 8?
â€“Â hengyue li
Jan 15 at 4:50

I don't know. Here's one link, though: software.intel.com/en-us/articles/â€¦
â€“Â RonJohn
Jan 15 at 5:40

You may have to use a number less than 4 if the memory requirements are large (you don't want to have thrashing) or there's a lot of file i/o required relative to the disk throughput that you have. Or you could use a number higher than 8 if each job alternates between periods of pure CPU usage and i/o wait. Best thing is to test.
â€“Â Mark Plotnick
Jan 15 at 7:58

add a commentÂ |Â

up vote
1
down vote

accepted

If this PC has an Intel CPU, then the Thread(s) per core most certainly indicates hyper-threading.

https://en.wikipedia.org/wiki/Hyper-threading

For each processor core that is physically present, the operating system addresses two virtual (logical) cores and shares the workload between them when possible. The main function of hyper-threading is to increase the number of independent instructions in the pipeline; it takes advantage of superscalar architecture, in which multiple instructions operate on separate data in parallel.

Should I run 4 of them a time or 8 to reduce the total time cost?

It depends. Some tasks run faster under hyper-threading, and some don't. You'll have to test that yourself.

Assume now I have 800 of them need to be run.

I'd use GNU Parallel to handle this problem.

https://www.gnu.org/software/parallel/

GNU parallel is a shell tool for executing jobs in parallel using one or more computers.

If you've got a list of of files in . which need to be processed, this will work:

find . | parallel -j4 yourprogram

If your earlier tests show that it runs faster with hyper-threading, then change the "4" to an "8".

EDIT: forgot to mention that sometimes programs run faster when you disable HT in the BIOS.

edited Jan 15 at 5:42

answered Jan 15 at 4:13

RonJohn

471213

If this PC has an Intel CPU, then the Thread(s) per core most certainly indicates hyper-threading.

https://en.wikipedia.org/wiki/Hyper-threading

For each processor core that is physically present, the operating system addresses two virtual (logical) cores and shares the workload between them when possible. The main function of hyper-threading is to increase the number of independent instructions in the pipeline; it takes advantage of superscalar architecture, in which multiple instructions operate on separate data in parallel.

Should I run 4 of them a time or 8 to reduce the total time cost?

It depends. Some tasks run faster under hyper-threading, and some don't. You'll have to test that yourself.

Assume now I have 800 of them need to be run.

I'd use GNU Parallel to handle this problem.

https://www.gnu.org/software/parallel/

GNU parallel is a shell tool for executing jobs in parallel using one or more computers.

If you've got a list of of files in . which need to be processed, this will work:

find . | parallel -j4 yourprogram

If your earlier tests show that it runs faster with hyper-threading, then change the "4" to an "8".

EDIT: forgot to mention that sometimes programs run faster when you disable HT in the BIOS.

edited Jan 15 at 5:42

answered Jan 15 at 4:13

RonJohn

471213

edited Jan 15 at 5:42

answered Jan 15 at 4:13

RonJohn

471213

answered Jan 15 at 4:13

RonJohn

471213

answered Jan 15 at 4:13

RonJohn

471213

I made the program myself. Is there any guideline I can follow to write the code so that I can know if I use 4 or 8?
â€“Â hengyue li
Jan 15 at 4:50

I don't know. Here's one link, though: software.intel.com/en-us/articles/â€¦
â€“Â RonJohn
Jan 15 at 5:40

You may have to use a number less than 4 if the memory requirements are large (you don't want to have thrashing) or there's a lot of file i/o required relative to the disk throughput that you have. Or you could use a number higher than 8 if each job alternates between periods of pure CPU usage and i/o wait. Best thing is to test.
â€“Â Mark Plotnick
Jan 15 at 7:58

add a commentÂ |Â

I made the program myself. Is there any guideline I can follow to write the code so that I can know if I use 4 or 8?
â€“Â hengyue li
Jan 15 at 4:50

I don't know. Here's one link, though: software.intel.com/en-us/articles/â€¦
â€“Â RonJohn
Jan 15 at 5:40

You may have to use a number less than 4 if the memory requirements are large (you don't want to have thrashing) or there's a lot of file i/o required relative to the disk throughput that you have. Or you could use a number higher than 8 if each job alternates between periods of pure CPU usage and i/o wait. Best thing is to test.
â€“Â Mark Plotnick
Jan 15 at 7:58

I made the program myself. Is there any guideline I can follow to write the code so that I can know if I use 4 or 8?
â€“Â hengyue li
Jan 15 at 4:50

I don't know. Here's one link, though: software.intel.com/en-us/articles/â€¦
â€“Â RonJohn
Jan 15 at 5:40

You may have to use a number less than 4 if the memory requirements are large (you don't want to have thrashing) or there's a lot of file i/o required relative to the disk throughput that you have. Or you could use a number higher than 8 if each job alternates between periods of pure CPU usage and i/o wait. Best thing is to test.
â€“Â Mark Plotnick
Jan 15 at 7:58

add a commentÂ |Â

draft saved

draft discarded

draft saved

draft discarded

Post as a guest

Name

搜尋此網誌

mjhjmtu