Does Linux not distinguish between processes and threads?
Clash Royale CLAN TAG#URR8PPP
As far as I know in Linux kernel,
the structure
task_structrepresents threads i.e. light weight processes, but not processes.processes are not represented by any structure, but by groups of threads sharing the same thread group id.
So is the following from Operating System Concepts correct?
Linux also provides the ability
to create threads using theclone()system call. However, Linux does not
distinguish between processes and threads. In fact, Linux uses the term task
—rather than process or thread— when referring to a flow of control within a
program.
What does it mean?
Thanks.
Related How does Linux tell threads apart from child processes?
linux linux-kernel process thread lightweight
asked Dec 30 '18 at 22:02
TimTim
26.3k75247457
|
show 1 more comment
As far as I know in Linux kernel,
the structure
task_structrepresents threads i.e. light weight processes, but not processes.processes are not represented by any structure, but by groups of threads sharing the same thread group id.
So is the following from Operating System Concepts correct?
Linux also provides the ability
to create threads using theclone()system call. However, Linux does not
distinguish between processes and threads. In fact, Linux uses the term task
—rather than process or thread— when referring to a flow of control within a
program.
What does it mean?
Thanks.
Related How does Linux tell threads apart from child processes?
linux linux-kernel process thread lightweight
asked Dec 30 '18 at 22:02
TimTim
26.3k75247457
It is a long fight in the Linux world, more exactly:clone()syscall is a Linux invention. It is essentially afork()while some data structures of the child process (memory map, file descriptors, signal stack and so on) can remain the same (not copy! Same by address!) as the parent. It is a very different thing than the Posix standard says (essentially, there is a separate threads api). Now, after a decade the smoke disappeared from the battlefield of the mailing lists, this is the state what you can see. It is not so bad - what is bad, that threads are still not so lightweight on Linux.
– peterh
Dec 30 '18 at 22:14
Essentially, we still haveclone(), the posix threads library was included into the glibc, but there is a minimal thread support in the kernel. As far I know, it is still not N:M threading, which is still bad, bad, bad. Although real N:M threads are probably incompatible with the essence of the Unix API (where kernel calls are still synchronous).
– peterh
Dec 30 '18 at 22:26
I think, there is some... "committee" behind the Posix API, they should embrace the asynchronous kernel calls ASAP. Now they won't do it, saying "we have threads", which is a crap reason :-( I think it is the largest structural danger for the future of the Linux now.
– peterh
Dec 30 '18 at 22:28
1
meta.stackexchange.com/questions/230676
– JdeBP
Dec 30 '18 at 22:46
@JdeBP I didn't post it as an answer, because I am not sure if it answers the question. The comments contain also a lot of my opinions, and I won't face the downs of others having an opposite opinion. My goal was only to give some background to the OP if no useful answer arrives.
– peterh
Dec 31 '18 at 1:54
|
show 1 more comment
As far as I know in Linux kernel,
the structure
task_structrepresents threads i.e. light weight processes, but not processes.processes are not represented by any structure, but by groups of threads sharing the same thread group id.
So is the following from Operating System Concepts correct?
Linux also provides the ability
to create threads using theclone()system call. However, Linux does not
distinguish between processes and threads. In fact, Linux uses the term task
—rather than process or thread— when referring to a flow of control within a
program.
What does it mean?
Thanks.
Related How does Linux tell threads apart from child processes?
linux linux-kernel process thread lightweight
asked Dec 30 '18 at 22:02
TimTim
26.3k75247457
As far as I know in Linux kernel,
the structure
task_structrepresents threads i.e. light weight processes, but not processes.processes are not represented by any structure, but by groups of threads sharing the same thread group id.
So is the following from Operating System Concepts correct?
Linux also provides the ability
to create threads using theclone()system call. However, Linux does not
distinguish between processes and threads. In fact, Linux uses the term task
—rather than process or thread— when referring to a flow of control within a
program.
What does it mean?
Thanks.
Related How does Linux tell threads apart from child processes?
linux linux-kernel process thread lightweight
linux linux-kernel process thread lightweight
asked Dec 30 '18 at 22:02
TimTim
26.3k75247457
asked Dec 30 '18 at 22:02
TimTim
26.3k75247457
edited Dec 30 '18 at 23:10
Tim
asked Dec 30 '18 at 22:02
TimTim
26.3k75247457
asked Dec 30 '18 at 22:02
TimTim
26.3k75247457
asked Dec 30 '18 at 22:02
TimTim
26.3k75247457
26.3k75247457
It is a long fight in the Linux world, more exactly:clone()syscall is a Linux invention. It is essentially afork()while some data structures of the child process (memory map, file descriptors, signal stack and so on) can remain the same (not copy! Same by address!) as the parent. It is a very different thing than the Posix standard says (essentially, there is a separate threads api). Now, after a decade the smoke disappeared from the battlefield of the mailing lists, this is the state what you can see. It is not so bad - what is bad, that threads are still not so lightweight on Linux.
– peterh
Dec 30 '18 at 22:14
Essentially, we still haveclone(), the posix threads library was included into the glibc, but there is a minimal thread support in the kernel. As far I know, it is still not N:M threading, which is still bad, bad, bad. Although real N:M threads are probably incompatible with the essence of the Unix API (where kernel calls are still synchronous).
– peterh
Dec 30 '18 at 22:26
I think, there is some... "committee" behind the Posix API, they should embrace the asynchronous kernel calls ASAP. Now they won't do it, saying "we have threads", which is a crap reason :-( I think it is the largest structural danger for the future of the Linux now.
– peterh
Dec 30 '18 at 22:28
1
meta.stackexchange.com/questions/230676
– JdeBP
Dec 30 '18 at 22:46
@JdeBP I didn't post it as an answer, because I am not sure if it answers the question. The comments contain also a lot of my opinions, and I won't face the downs of others having an opposite opinion. My goal was only to give some background to the OP if no useful answer arrives.
– peterh
Dec 31 '18 at 1:54
|
show 1 more comment
It is a long fight in the Linux world, more exactly:clone()syscall is a Linux invention. It is essentially afork()while some data structures of the child process (memory map, file descriptors, signal stack and so on) can remain the same (not copy! Same by address!) as the parent. It is a very different thing than the Posix standard says (essentially, there is a separate threads api). Now, after a decade the smoke disappeared from the battlefield of the mailing lists, this is the state what you can see. It is not so bad - what is bad, that threads are still not so lightweight on Linux.
– peterh
Dec 30 '18 at 22:14
Essentially, we still haveclone(), the posix threads library was included into the glibc, but there is a minimal thread support in the kernel. As far I know, it is still not N:M threading, which is still bad, bad, bad. Although real N:M threads are probably incompatible with the essence of the Unix API (where kernel calls are still synchronous).
– peterh
Dec 30 '18 at 22:26
I think, there is some... "committee" behind the Posix API, they should embrace the asynchronous kernel calls ASAP. Now they won't do it, saying "we have threads", which is a crap reason :-( I think it is the largest structural danger for the future of the Linux now.
– peterh
Dec 30 '18 at 22:28
1
meta.stackexchange.com/questions/230676
– JdeBP
Dec 30 '18 at 22:46
@JdeBP I didn't post it as an answer, because I am not sure if it answers the question. The comments contain also a lot of my opinions, and I won't face the downs of others having an opposite opinion. My goal was only to give some background to the OP if no useful answer arrives.
– peterh
Dec 31 '18 at 1:54
It is a long fight in the Linux world, more exactly:
clone() syscall is a Linux invention. It is essentially a fork() while some data structures of the child process (memory map, file descriptors, signal stack and so on) can remain the same (not copy! Same by address!) as the parent. It is a very different thing than the Posix standard says (essentially, there is a separate threads api). Now, after a decade the smoke disappeared from the battlefield of the mailing lists, this is the state what you can see. It is not so bad - what is bad, that threads are still not so lightweight on Linux.– peterh
Dec 30 '18 at 22:14
It is a long fight in the Linux world, more exactly:
clone() syscall is a Linux invention. It is essentially a fork() while some data structures of the child process (memory map, file descriptors, signal stack and so on) can remain the same (not copy! Same by address!) as the parent. It is a very different thing than the Posix standard says (essentially, there is a separate threads api). Now, after a decade the smoke disappeared from the battlefield of the mailing lists, this is the state what you can see. It is not so bad - what is bad, that threads are still not so lightweight on Linux.– peterh
Dec 30 '18 at 22:14
Essentially, we still have
clone(), the posix threads library was included into the glibc, but there is a minimal thread support in the kernel. As far I know, it is still not N:M threading, which is still bad, bad, bad. Although real N:M threads are probably incompatible with the essence of the Unix API (where kernel calls are still synchronous).– peterh
Dec 30 '18 at 22:26
Essentially, we still have
clone(), the posix threads library was included into the glibc, but there is a minimal thread support in the kernel. As far I know, it is still not N:M threading, which is still bad, bad, bad. Although real N:M threads are probably incompatible with the essence of the Unix API (where kernel calls are still synchronous).– peterh
Dec 30 '18 at 22:26
I think, there is some... "committee" behind the Posix API, they should embrace the asynchronous kernel calls ASAP. Now they won't do it, saying "we have threads", which is a crap reason :-( I think it is the largest structural danger for the future of the Linux now.
– peterh
Dec 30 '18 at 22:28
I think, there is some... "committee" behind the Posix API, they should embrace the asynchronous kernel calls ASAP. Now they won't do it, saying "we have threads", which is a crap reason :-( I think it is the largest structural danger for the future of the Linux now.
– peterh
Dec 30 '18 at 22:28
1
1
meta.stackexchange.com/questions/230676
– JdeBP
Dec 30 '18 at 22:46
meta.stackexchange.com/questions/230676
– JdeBP
Dec 30 '18 at 22:46
@JdeBP I didn't post it as an answer, because I am not sure if it answers the question. The comments contain also a lot of my opinions, and I won't face the downs of others having an opposite opinion. My goal was only to give some background to the OP if no useful answer arrives.
– peterh
Dec 31 '18 at 1:54
@JdeBP I didn't post it as an answer, because I am not sure if it answers the question. The comments contain also a lot of my opinions, and I won't face the downs of others having an opposite opinion. My goal was only to give some background to the OP if no useful answer arrives.
– peterh
Dec 31 '18 at 1:54
|
show 1 more comment
2 Answers
2
active
oldest
votes
Linux also provides the ability to create threads using the clone() system call. However, Linux does not distinguish between processes and threads. In fact, Linux uses the term task —rather than process or thread— when referring to a flow of control within a program.
We need to distinguish between the actual implementation and the surface you see.
From user (system software developer) point of view there is a big difference: threads share a lot of common resources (e.g. memory mappings - apart from stack, of course - file descriptors).
Internally (warning: imprecise handwaving arguments) the Linux kernel1) is using what it has at hand, i.e. the same structure for processes and for threads, where for threads of a single process it doesn't duplicate some things rather it references a single instance thereof (memory map description).
Thus on the level of directly representing a thread or a process there is not much difference in the basic structure, the devil lies in how the information is handled.
You may as well be interested in reading Are threads implemented as processes on Linux?
1) Remember that "Linux" these days stands mostly for the whole OS, while in fact it only is the kernel itself.
answered Dec 31 '18 at 16:43
peterphpeterph
23.3k24457
Remember that the word “Linux” is confusing, it is a kernel, but sometimes refers to the whole system, and some times to the whole system minus Linux. see WSL for an example.
– ctrl-alt-delor
Jan 1 at 16:28
add a comment |
Linux threads are implemented as a separate process but sharing the same address space as other threads. By default they are hidden in the ps command, but can be seen with the -L flag.
For example:
% ps -fp 2642
UID PID PPID C STIME TTY TIME CMD
polkitd 2642 1 0 Dec09 ? 00:00:48 /usr/lib/polkit-1/polkitd --no-d
% ps -fLp 2642
UID PID PPID LWP C NLWP STIME TTY TIME CMD
polkitd 2642 1 2642 0 7 Dec09 ? 00:00:18 /usr/lib/polkit-1/pol
polkitd 2642 1 2680 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2683 0 7 Dec09 ? 00:00:30 /usr/lib/polkit-1/pol
polkitd 2642 1 2685 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2687 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2688 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2692 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
We can see that polkitd really consists of 7 threads. They all have the same process ID, but different thread IDs (LWP) but show up as different processes in the ps listing, because they are different processes in the kernel.
This can have impact, e.g. on ulimit constraints. A common issue on RedHat and derivatives is that the default PAM configuration limits your processes
% cat /etc/security/limits.d/20-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.
* soft nproc 4096
root soft nproc unlimited
On heavy java web apps the number of processes can exceed this and cause application failures.
answered Dec 30 '18 at 22:46
Stephen HarrisStephen Harris
25.4k24477
The question is not about howpsmassages information for user consumption. It is about what the kernel deals in. None of this answers the actual question of whether the book explanation is correct.
– JdeBP
Dec 30 '18 at 22:51
There are potentially three questions in the post; "Does Linux not distinguish between processes and threads?", "So is the following from Operating System Concepts correct?", "what does it mean?". I answered the first question, and usedpsto show it, and pointed out a consequence.
– Stephen Harris
Dec 30 '18 at 23:38
I thought they don't have to share the memory space, just sharing the same tgid and namespace. Correct me if I'm wrong.
– 炸鱼薯条德里克
Dec 31 '18 at 2:26
@炸鱼薯条德里克 They don't have to share anything.clone(2)has a lot of options.CLONE_VMallows parent and child to run in the same memory space. POSIX threads (pthreads(7)) says threads share same global memory space, but their own stack.
– Stephen Harris
Dec 31 '18 at 2:35
Yeah, I know, I'm just talking about Linux, not POSIX. But I vaguely remember that threads in one process can't stay in different cgroups, or different pid_namespace
– 炸鱼薯条德里克
Dec 31 '18 at 2:41
|
show 3 more comments
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f491664%2fdoes-linux-not-distinguish-between-processes-and-threads%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Linux also provides the ability to create threads using the clone() system call. However, Linux does not distinguish between processes and threads. In fact, Linux uses the term task —rather than process or thread— when referring to a flow of control within a program.
We need to distinguish between the actual implementation and the surface you see.
From user (system software developer) point of view there is a big difference: threads share a lot of common resources (e.g. memory mappings - apart from stack, of course - file descriptors).
Internally (warning: imprecise handwaving arguments) the Linux kernel1) is using what it has at hand, i.e. the same structure for processes and for threads, where for threads of a single process it doesn't duplicate some things rather it references a single instance thereof (memory map description).
Thus on the level of directly representing a thread or a process there is not much difference in the basic structure, the devil lies in how the information is handled.
You may as well be interested in reading Are threads implemented as processes on Linux?
1) Remember that "Linux" these days stands mostly for the whole OS, while in fact it only is the kernel itself.
answered Dec 31 '18 at 16:43
peterphpeterph
23.3k24457
Remember that the word “Linux” is confusing, it is a kernel, but sometimes refers to the whole system, and some times to the whole system minus Linux. see WSL for an example.
– ctrl-alt-delor
Jan 1 at 16:28
add a comment |
Linux also provides the ability to create threads using the clone() system call. However, Linux does not distinguish between processes and threads. In fact, Linux uses the term task —rather than process or thread— when referring to a flow of control within a program.
We need to distinguish between the actual implementation and the surface you see.
From user (system software developer) point of view there is a big difference: threads share a lot of common resources (e.g. memory mappings - apart from stack, of course - file descriptors).
Internally (warning: imprecise handwaving arguments) the Linux kernel1) is using what it has at hand, i.e. the same structure for processes and for threads, where for threads of a single process it doesn't duplicate some things rather it references a single instance thereof (memory map description).
Thus on the level of directly representing a thread or a process there is not much difference in the basic structure, the devil lies in how the information is handled.
You may as well be interested in reading Are threads implemented as processes on Linux?
1) Remember that "Linux" these days stands mostly for the whole OS, while in fact it only is the kernel itself.
answered Dec 31 '18 at 16:43
peterphpeterph
23.3k24457
Remember that the word “Linux” is confusing, it is a kernel, but sometimes refers to the whole system, and some times to the whole system minus Linux. see WSL for an example.
– ctrl-alt-delor
Jan 1 at 16:28
add a comment |
Linux also provides the ability to create threads using the clone() system call. However, Linux does not distinguish between processes and threads. In fact, Linux uses the term task —rather than process or thread— when referring to a flow of control within a program.
We need to distinguish between the actual implementation and the surface you see.
From user (system software developer) point of view there is a big difference: threads share a lot of common resources (e.g. memory mappings - apart from stack, of course - file descriptors).
Internally (warning: imprecise handwaving arguments) the Linux kernel1) is using what it has at hand, i.e. the same structure for processes and for threads, where for threads of a single process it doesn't duplicate some things rather it references a single instance thereof (memory map description).
Thus on the level of directly representing a thread or a process there is not much difference in the basic structure, the devil lies in how the information is handled.
You may as well be interested in reading Are threads implemented as processes on Linux?
1) Remember that "Linux" these days stands mostly for the whole OS, while in fact it only is the kernel itself.
answered Dec 31 '18 at 16:43
peterphpeterph
23.3k24457
Linux also provides the ability to create threads using the clone() system call. However, Linux does not distinguish between processes and threads. In fact, Linux uses the term task —rather than process or thread— when referring to a flow of control within a program.
We need to distinguish between the actual implementation and the surface you see.
From user (system software developer) point of view there is a big difference: threads share a lot of common resources (e.g. memory mappings - apart from stack, of course - file descriptors).
Internally (warning: imprecise handwaving arguments) the Linux kernel1) is using what it has at hand, i.e. the same structure for processes and for threads, where for threads of a single process it doesn't duplicate some things rather it references a single instance thereof (memory map description).
Thus on the level of directly representing a thread or a process there is not much difference in the basic structure, the devil lies in how the information is handled.
You may as well be interested in reading Are threads implemented as processes on Linux?
1) Remember that "Linux" these days stands mostly for the whole OS, while in fact it only is the kernel itself.
answered Dec 31 '18 at 16:43
peterphpeterph
23.3k24457
answered Dec 31 '18 at 16:43
peterphpeterph
23.3k24457
answered Dec 31 '18 at 16:43
peterphpeterph
23.3k24457
answered Dec 31 '18 at 16:43
peterphpeterph
23.3k24457
23.3k24457
Remember that the word “Linux” is confusing, it is a kernel, but sometimes refers to the whole system, and some times to the whole system minus Linux. see WSL for an example.
– ctrl-alt-delor
Jan 1 at 16:28
add a comment |
Remember that the word “Linux” is confusing, it is a kernel, but sometimes refers to the whole system, and some times to the whole system minus Linux. see WSL for an example.
– ctrl-alt-delor
Jan 1 at 16:28
Remember that the word “Linux” is confusing, it is a kernel, but sometimes refers to the whole system, and some times to the whole system minus Linux. see WSL for an example.
– ctrl-alt-delor
Jan 1 at 16:28
Remember that the word “Linux” is confusing, it is a kernel, but sometimes refers to the whole system, and some times to the whole system minus Linux. see WSL for an example.
– ctrl-alt-delor
Jan 1 at 16:28
add a comment |
Linux threads are implemented as a separate process but sharing the same address space as other threads. By default they are hidden in the ps command, but can be seen with the -L flag.
For example:
% ps -fp 2642
UID PID PPID C STIME TTY TIME CMD
polkitd 2642 1 0 Dec09 ? 00:00:48 /usr/lib/polkit-1/polkitd --no-d
% ps -fLp 2642
UID PID PPID LWP C NLWP STIME TTY TIME CMD
polkitd 2642 1 2642 0 7 Dec09 ? 00:00:18 /usr/lib/polkit-1/pol
polkitd 2642 1 2680 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2683 0 7 Dec09 ? 00:00:30 /usr/lib/polkit-1/pol
polkitd 2642 1 2685 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2687 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2688 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2692 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
We can see that polkitd really consists of 7 threads. They all have the same process ID, but different thread IDs (LWP) but show up as different processes in the ps listing, because they are different processes in the kernel.
This can have impact, e.g. on ulimit constraints. A common issue on RedHat and derivatives is that the default PAM configuration limits your processes
% cat /etc/security/limits.d/20-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.
* soft nproc 4096
root soft nproc unlimited
On heavy java web apps the number of processes can exceed this and cause application failures.
answered Dec 30 '18 at 22:46
Stephen HarrisStephen Harris
25.4k24477
The question is not about howpsmassages information for user consumption. It is about what the kernel deals in. None of this answers the actual question of whether the book explanation is correct.
– JdeBP
Dec 30 '18 at 22:51
There are potentially three questions in the post; "Does Linux not distinguish between processes and threads?", "So is the following from Operating System Concepts correct?", "what does it mean?". I answered the first question, and usedpsto show it, and pointed out a consequence.
– Stephen Harris
Dec 30 '18 at 23:38
I thought they don't have to share the memory space, just sharing the same tgid and namespace. Correct me if I'm wrong.
– 炸鱼薯条德里克
Dec 31 '18 at 2:26
@炸鱼薯条德里克 They don't have to share anything.clone(2)has a lot of options.CLONE_VMallows parent and child to run in the same memory space. POSIX threads (pthreads(7)) says threads share same global memory space, but their own stack.
– Stephen Harris
Dec 31 '18 at 2:35
Yeah, I know, I'm just talking about Linux, not POSIX. But I vaguely remember that threads in one process can't stay in different cgroups, or different pid_namespace
– 炸鱼薯条德里克
Dec 31 '18 at 2:41
|
show 3 more comments
Linux threads are implemented as a separate process but sharing the same address space as other threads. By default they are hidden in the ps command, but can be seen with the -L flag.
For example:
% ps -fp 2642
UID PID PPID C STIME TTY TIME CMD
polkitd 2642 1 0 Dec09 ? 00:00:48 /usr/lib/polkit-1/polkitd --no-d
% ps -fLp 2642
UID PID PPID LWP C NLWP STIME TTY TIME CMD
polkitd 2642 1 2642 0 7 Dec09 ? 00:00:18 /usr/lib/polkit-1/pol
polkitd 2642 1 2680 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2683 0 7 Dec09 ? 00:00:30 /usr/lib/polkit-1/pol
polkitd 2642 1 2685 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2687 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2688 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2692 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
We can see that polkitd really consists of 7 threads. They all have the same process ID, but different thread IDs (LWP) but show up as different processes in the ps listing, because they are different processes in the kernel.
This can have impact, e.g. on ulimit constraints. A common issue on RedHat and derivatives is that the default PAM configuration limits your processes
% cat /etc/security/limits.d/20-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.
* soft nproc 4096
root soft nproc unlimited
On heavy java web apps the number of processes can exceed this and cause application failures.
answered Dec 30 '18 at 22:46
Stephen HarrisStephen Harris
25.4k24477
The question is not about howpsmassages information for user consumption. It is about what the kernel deals in. None of this answers the actual question of whether the book explanation is correct.
– JdeBP
Dec 30 '18 at 22:51
There are potentially three questions in the post; "Does Linux not distinguish between processes and threads?", "So is the following from Operating System Concepts correct?", "what does it mean?". I answered the first question, and usedpsto show it, and pointed out a consequence.
– Stephen Harris
Dec 30 '18 at 23:38
I thought they don't have to share the memory space, just sharing the same tgid and namespace. Correct me if I'm wrong.
– 炸鱼薯条德里克
Dec 31 '18 at 2:26
@炸鱼薯条德里克 They don't have to share anything.clone(2)has a lot of options.CLONE_VMallows parent and child to run in the same memory space. POSIX threads (pthreads(7)) says threads share same global memory space, but their own stack.
– Stephen Harris
Dec 31 '18 at 2:35
Yeah, I know, I'm just talking about Linux, not POSIX. But I vaguely remember that threads in one process can't stay in different cgroups, or different pid_namespace
– 炸鱼薯条德里克
Dec 31 '18 at 2:41
|
show 3 more comments
Linux threads are implemented as a separate process but sharing the same address space as other threads. By default they are hidden in the ps command, but can be seen with the -L flag.
For example:
% ps -fp 2642
UID PID PPID C STIME TTY TIME CMD
polkitd 2642 1 0 Dec09 ? 00:00:48 /usr/lib/polkit-1/polkitd --no-d
% ps -fLp 2642
UID PID PPID LWP C NLWP STIME TTY TIME CMD
polkitd 2642 1 2642 0 7 Dec09 ? 00:00:18 /usr/lib/polkit-1/pol
polkitd 2642 1 2680 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2683 0 7 Dec09 ? 00:00:30 /usr/lib/polkit-1/pol
polkitd 2642 1 2685 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2687 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2688 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2692 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
We can see that polkitd really consists of 7 threads. They all have the same process ID, but different thread IDs (LWP) but show up as different processes in the ps listing, because they are different processes in the kernel.
This can have impact, e.g. on ulimit constraints. A common issue on RedHat and derivatives is that the default PAM configuration limits your processes
% cat /etc/security/limits.d/20-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.
* soft nproc 4096
root soft nproc unlimited
On heavy java web apps the number of processes can exceed this and cause application failures.
answered Dec 30 '18 at 22:46
Stephen HarrisStephen Harris
25.4k24477
Linux threads are implemented as a separate process but sharing the same address space as other threads. By default they are hidden in the ps command, but can be seen with the -L flag.
For example:
% ps -fp 2642
UID PID PPID C STIME TTY TIME CMD
polkitd 2642 1 0 Dec09 ? 00:00:48 /usr/lib/polkit-1/polkitd --no-d
% ps -fLp 2642
UID PID PPID LWP C NLWP STIME TTY TIME CMD
polkitd 2642 1 2642 0 7 Dec09 ? 00:00:18 /usr/lib/polkit-1/pol
polkitd 2642 1 2680 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2683 0 7 Dec09 ? 00:00:30 /usr/lib/polkit-1/pol
polkitd 2642 1 2685 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2687 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2688 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
polkitd 2642 1 2692 0 7 Dec09 ? 00:00:00 /usr/lib/polkit-1/pol
We can see that polkitd really consists of 7 threads. They all have the same process ID, but different thread IDs (LWP) but show up as different processes in the ps listing, because they are different processes in the kernel.
This can have impact, e.g. on ulimit constraints. A common issue on RedHat and derivatives is that the default PAM configuration limits your processes
% cat /etc/security/limits.d/20-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.
* soft nproc 4096
root soft nproc unlimited
On heavy java web apps the number of processes can exceed this and cause application failures.
answered Dec 30 '18 at 22:46
Stephen HarrisStephen Harris
25.4k24477
answered Dec 30 '18 at 22:46
Stephen HarrisStephen Harris
25.4k24477
answered Dec 30 '18 at 22:46
Stephen HarrisStephen Harris
25.4k24477
answered Dec 30 '18 at 22:46
Stephen HarrisStephen Harris
25.4k24477
25.4k24477
The question is not about howpsmassages information for user consumption. It is about what the kernel deals in. None of this answers the actual question of whether the book explanation is correct.
– JdeBP
Dec 30 '18 at 22:51
There are potentially three questions in the post; "Does Linux not distinguish between processes and threads?", "So is the following from Operating System Concepts correct?", "what does it mean?". I answered the first question, and usedpsto show it, and pointed out a consequence.
– Stephen Harris
Dec 30 '18 at 23:38
I thought they don't have to share the memory space, just sharing the same tgid and namespace. Correct me if I'm wrong.
– 炸鱼薯条德里克
Dec 31 '18 at 2:26
@炸鱼薯条德里克 They don't have to share anything.clone(2)has a lot of options.CLONE_VMallows parent and child to run in the same memory space. POSIX threads (pthreads(7)) says threads share same global memory space, but their own stack.
– Stephen Harris
Dec 31 '18 at 2:35
Yeah, I know, I'm just talking about Linux, not POSIX. But I vaguely remember that threads in one process can't stay in different cgroups, or different pid_namespace
– 炸鱼薯条德里克
Dec 31 '18 at 2:41
|
show 3 more comments
The question is not about howpsmassages information for user consumption. It is about what the kernel deals in. None of this answers the actual question of whether the book explanation is correct.
– JdeBP
Dec 30 '18 at 22:51
There are potentially three questions in the post; "Does Linux not distinguish between processes and threads?", "So is the following from Operating System Concepts correct?", "what does it mean?". I answered the first question, and usedpsto show it, and pointed out a consequence.
– Stephen Harris
Dec 30 '18 at 23:38
I thought they don't have to share the memory space, just sharing the same tgid and namespace. Correct me if I'm wrong.
– 炸鱼薯条德里克
Dec 31 '18 at 2:26
@炸鱼薯条德里克 They don't have to share anything.clone(2)has a lot of options.CLONE_VMallows parent and child to run in the same memory space. POSIX threads (pthreads(7)) says threads share same global memory space, but their own stack.
– Stephen Harris
Dec 31 '18 at 2:35
Yeah, I know, I'm just talking about Linux, not POSIX. But I vaguely remember that threads in one process can't stay in different cgroups, or different pid_namespace
– 炸鱼薯条德里克
Dec 31 '18 at 2:41
The question is not about how
ps massages information for user consumption. It is about what the kernel deals in. None of this answers the actual question of whether the book explanation is correct.– JdeBP
Dec 30 '18 at 22:51
The question is not about how
ps massages information for user consumption. It is about what the kernel deals in. None of this answers the actual question of whether the book explanation is correct.– JdeBP
Dec 30 '18 at 22:51
There are potentially three questions in the post; "Does Linux not distinguish between processes and threads?", "So is the following from Operating System Concepts correct?", "what does it mean?". I answered the first question, and used
ps to show it, and pointed out a consequence.– Stephen Harris
Dec 30 '18 at 23:38
There are potentially three questions in the post; "Does Linux not distinguish between processes and threads?", "So is the following from Operating System Concepts correct?", "what does it mean?". I answered the first question, and used
ps to show it, and pointed out a consequence.– Stephen Harris
Dec 30 '18 at 23:38
I thought they don't have to share the memory space, just sharing the same tgid and namespace. Correct me if I'm wrong.
– 炸鱼薯条德里克
Dec 31 '18 at 2:26
I thought they don't have to share the memory space, just sharing the same tgid and namespace. Correct me if I'm wrong.
– 炸鱼薯条德里克
Dec 31 '18 at 2:26
@炸鱼薯条德里克 They don't have to share anything.
clone(2) has a lot of options. CLONE_VM allows parent and child to run in the same memory space. POSIX threads (pthreads(7)) says threads share same global memory space, but their own stack.– Stephen Harris
Dec 31 '18 at 2:35
@炸鱼薯条德里克 They don't have to share anything.
clone(2) has a lot of options. CLONE_VM allows parent and child to run in the same memory space. POSIX threads (pthreads(7)) says threads share same global memory space, but their own stack.– Stephen Harris
Dec 31 '18 at 2:35
Yeah, I know, I'm just talking about Linux, not POSIX. But I vaguely remember that threads in one process can't stay in different cgroups, or different pid_namespace
– 炸鱼薯条德里克
Dec 31 '18 at 2:41
Yeah, I know, I'm just talking about Linux, not POSIX. But I vaguely remember that threads in one process can't stay in different cgroups, or different pid_namespace
– 炸鱼薯条德里克
Dec 31 '18 at 2:41
|
show 3 more comments
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f491664%2fdoes-linux-not-distinguish-between-processes-and-threads%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
It is a long fight in the Linux world, more exactly:
clone()syscall is a Linux invention. It is essentially afork()while some data structures of the child process (memory map, file descriptors, signal stack and so on) can remain the same (not copy! Same by address!) as the parent. It is a very different thing than the Posix standard says (essentially, there is a separate threads api). Now, after a decade the smoke disappeared from the battlefield of the mailing lists, this is the state what you can see. It is not so bad - what is bad, that threads are still not so lightweight on Linux.– peterh
Dec 30 '18 at 22:14
Essentially, we still have
clone(), the posix threads library was included into the glibc, but there is a minimal thread support in the kernel. As far I know, it is still not N:M threading, which is still bad, bad, bad. Although real N:M threads are probably incompatible with the essence of the Unix API (where kernel calls are still synchronous).– peterh
Dec 30 '18 at 22:26
I think, there is some... "committee" behind the Posix API, they should embrace the asynchronous kernel calls ASAP. Now they won't do it, saying "we have threads", which is a crap reason :-( I think it is the largest structural danger for the future of the Linux now.
– peterh
Dec 30 '18 at 22:28
1
meta.stackexchange.com/questions/230676
– JdeBP
Dec 30 '18 at 22:46
@JdeBP I didn't post it as an answer, because I am not sure if it answers the question. The comments contain also a lot of my opinions, and I won't face the downs of others having an opposite opinion. My goal was only to give some background to the OP if no useful answer arrives.
– peterh
Dec 31 '18 at 1:54