How does a shell execute a program?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
11
down vote

favorite
5












If I compile a program using gcc, and try to execute it from the bash shell, what is the exact sequence of steps followed by bash to execute it ?



I know fork(), execve(), loader, dynamic linker (and other things) are involved, but can someone give an exact sequence of steps and some suitable reading reference ?



Edit:



From the answers, it seems the question could imply many possibilities. I want to narrow down to a simple case:



(test.c just prints hello world)



$ gcc test.c -o test
$ ./test


What will be the steps in the above case (./test), specifically relating bash starting program in some child process, doing loading, linking etc. ?







share|improve this question

















  • 4




    I invite you to read lwn.net/Articles/630727
    – cuonglm
    Aug 27 '15 at 1:45






  • 3




    Why not try `strace bash -c 'test' ?
    – Sergiy Kolodyazhnyy
    Aug 27 '15 at 7:30






  • 1




    It seems like a decent Operating Systems textbook would be a good resource for the OP. Trying to learn how operating systems work by asking individual questions like this is not likely to be a productive process.
    – Barmar
    Sep 2 '15 at 19:36














up vote
11
down vote

favorite
5












If I compile a program using gcc, and try to execute it from the bash shell, what is the exact sequence of steps followed by bash to execute it ?



I know fork(), execve(), loader, dynamic linker (and other things) are involved, but can someone give an exact sequence of steps and some suitable reading reference ?



Edit:



From the answers, it seems the question could imply many possibilities. I want to narrow down to a simple case:



(test.c just prints hello world)



$ gcc test.c -o test
$ ./test


What will be the steps in the above case (./test), specifically relating bash starting program in some child process, doing loading, linking etc. ?







share|improve this question

















  • 4




    I invite you to read lwn.net/Articles/630727
    – cuonglm
    Aug 27 '15 at 1:45






  • 3




    Why not try `strace bash -c 'test' ?
    – Sergiy Kolodyazhnyy
    Aug 27 '15 at 7:30






  • 1




    It seems like a decent Operating Systems textbook would be a good resource for the OP. Trying to learn how operating systems work by asking individual questions like this is not likely to be a productive process.
    – Barmar
    Sep 2 '15 at 19:36












up vote
11
down vote

favorite
5









up vote
11
down vote

favorite
5






5





If I compile a program using gcc, and try to execute it from the bash shell, what is the exact sequence of steps followed by bash to execute it ?



I know fork(), execve(), loader, dynamic linker (and other things) are involved, but can someone give an exact sequence of steps and some suitable reading reference ?



Edit:



From the answers, it seems the question could imply many possibilities. I want to narrow down to a simple case:



(test.c just prints hello world)



$ gcc test.c -o test
$ ./test


What will be the steps in the above case (./test), specifically relating bash starting program in some child process, doing loading, linking etc. ?







share|improve this question













If I compile a program using gcc, and try to execute it from the bash shell, what is the exact sequence of steps followed by bash to execute it ?



I know fork(), execve(), loader, dynamic linker (and other things) are involved, but can someone give an exact sequence of steps and some suitable reading reference ?



Edit:



From the answers, it seems the question could imply many possibilities. I want to narrow down to a simple case:



(test.c just prints hello world)



$ gcc test.c -o test
$ ./test


What will be the steps in the above case (./test), specifically relating bash starting program in some child process, doing loading, linking etc. ?









share|improve this question












share|improve this question




share|improve this question








edited Aug 27 '15 at 4:39
























asked Aug 26 '15 at 23:22









Jake

504522




504522







  • 4




    I invite you to read lwn.net/Articles/630727
    – cuonglm
    Aug 27 '15 at 1:45






  • 3




    Why not try `strace bash -c 'test' ?
    – Sergiy Kolodyazhnyy
    Aug 27 '15 at 7:30






  • 1




    It seems like a decent Operating Systems textbook would be a good resource for the OP. Trying to learn how operating systems work by asking individual questions like this is not likely to be a productive process.
    – Barmar
    Sep 2 '15 at 19:36












  • 4




    I invite you to read lwn.net/Articles/630727
    – cuonglm
    Aug 27 '15 at 1:45






  • 3




    Why not try `strace bash -c 'test' ?
    – Sergiy Kolodyazhnyy
    Aug 27 '15 at 7:30






  • 1




    It seems like a decent Operating Systems textbook would be a good resource for the OP. Trying to learn how operating systems work by asking individual questions like this is not likely to be a productive process.
    – Barmar
    Sep 2 '15 at 19:36







4




4




I invite you to read lwn.net/Articles/630727
– cuonglm
Aug 27 '15 at 1:45




I invite you to read lwn.net/Articles/630727
– cuonglm
Aug 27 '15 at 1:45




3




3




Why not try `strace bash -c 'test' ?
– Sergiy Kolodyazhnyy
Aug 27 '15 at 7:30




Why not try `strace bash -c 'test' ?
– Sergiy Kolodyazhnyy
Aug 27 '15 at 7:30




1




1




It seems like a decent Operating Systems textbook would be a good resource for the OP. Trying to learn how operating systems work by asking individual questions like this is not likely to be a productive process.
– Barmar
Sep 2 '15 at 19:36




It seems like a decent Operating Systems textbook would be a good resource for the OP. Trying to learn how operating systems work by asking individual questions like this is not likely to be a productive process.
– Barmar
Sep 2 '15 at 19:36










3 Answers
3






active

oldest

votes

















up vote
5
down vote



accepted










Well, the exact sequence may vary, as there might be a shell alias or function that first gets expanded/interpreted before the actual program gets executed, and then differences for a qualified filename (/usr/libexec/foo) versus something that will be looked for through all the directories of the PATH environment variable (just foo). Also, the details of the execution may complicate matters, as foo | bar | zot requires more work for the shell (some number of fork(2), dup(2), and, of course, pipe(2), among other system calls), while something like exec foo is much less work as the shell merely replaces itself with the new program (i.e., it doesn't fork). Also important are process groups (especially the foreground process group, all PIDs of which eat SIGINT when someone starts mashing on Ctrl+C, sessions, and whether the job is going to be run in the background, monitored (foo &) or background, ignored (foo & disown). I/O redirection details will also change things, e.g., if standard input is closed by the shell (foo <&-), or whether a file is opened as stdin (foo < blah).



strace or similar will be informative about the specific system calls made along this process, and there should be man pages for each of those calls. Suitable system level reading would be any number of chapters from Stevens's "Advanced Programming in the UNIX Environment" while a shell book (e.g., "From Bash to Z Shell") will cover the shell side of things in more detail.






share|improve this answer























  • I edited the question to narrow down to a simple case
    – Jake
    Aug 27 '15 at 1:17

















up vote
1
down vote













Assuming a textbook example shell (for code clarity) that is already running (so the dynamic linker is done), the commands you mention will require the shell to make the following system calls:



  • read: gets the next command in this case gcc

  • fork: two process are needed, we assume the parent has pid 500 and the child for illustration.

  • the parent will call wait(501), meanwhile the child will call exec. At this point the shell is no longer running on pid 501. gcc makes lots of system calls including at a minimum open, close, read, write, chmod, fork, exec, wait and exit.

  • when gcc calls exit, wait will return, write is called to display the prompt and the process will repeat.

More complicated commands of course add more complication to this basic sequence. Two simpler examples of basic complications are basic io redirection where a open, close, dup sequence is inserted between the fork and the exec and background processes where the wait is skipped (and another wait is added to a sigchld handler).






share|improve this answer





















  • Small addition: the question asks about loading and dynamic linking. All the code that is statically linked, i.e. actually included in the program file, is done by the kernel before the program is started. Dynamically loaded libraries, i.e. separate files, are handled by the program itself before starting main(). The code for this is automatically added by gcc.
    – Stig Hemmer
    Aug 27 '15 at 8:55


















up vote
0
down vote













I suggest reading Section 8.4.6 Using fork and execve to Run Programs



on
http://www.groupes.polymtl.ca/inf2610/documentation/ComputerSystemBook.pdf






share|improve this answer





















    Your Answer







    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "106"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    convertImagesToLinks: false,
    noModals: false,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );








     

    draft saved


    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f225736%2fhow-does-a-shell-execute-a-program%23new-answer', 'question_page');

    );

    Post as a guest






























    3 Answers
    3






    active

    oldest

    votes








    3 Answers
    3






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    5
    down vote



    accepted










    Well, the exact sequence may vary, as there might be a shell alias or function that first gets expanded/interpreted before the actual program gets executed, and then differences for a qualified filename (/usr/libexec/foo) versus something that will be looked for through all the directories of the PATH environment variable (just foo). Also, the details of the execution may complicate matters, as foo | bar | zot requires more work for the shell (some number of fork(2), dup(2), and, of course, pipe(2), among other system calls), while something like exec foo is much less work as the shell merely replaces itself with the new program (i.e., it doesn't fork). Also important are process groups (especially the foreground process group, all PIDs of which eat SIGINT when someone starts mashing on Ctrl+C, sessions, and whether the job is going to be run in the background, monitored (foo &) or background, ignored (foo & disown). I/O redirection details will also change things, e.g., if standard input is closed by the shell (foo <&-), or whether a file is opened as stdin (foo < blah).



    strace or similar will be informative about the specific system calls made along this process, and there should be man pages for each of those calls. Suitable system level reading would be any number of chapters from Stevens's "Advanced Programming in the UNIX Environment" while a shell book (e.g., "From Bash to Z Shell") will cover the shell side of things in more detail.






    share|improve this answer























    • I edited the question to narrow down to a simple case
      – Jake
      Aug 27 '15 at 1:17














    up vote
    5
    down vote



    accepted










    Well, the exact sequence may vary, as there might be a shell alias or function that first gets expanded/interpreted before the actual program gets executed, and then differences for a qualified filename (/usr/libexec/foo) versus something that will be looked for through all the directories of the PATH environment variable (just foo). Also, the details of the execution may complicate matters, as foo | bar | zot requires more work for the shell (some number of fork(2), dup(2), and, of course, pipe(2), among other system calls), while something like exec foo is much less work as the shell merely replaces itself with the new program (i.e., it doesn't fork). Also important are process groups (especially the foreground process group, all PIDs of which eat SIGINT when someone starts mashing on Ctrl+C, sessions, and whether the job is going to be run in the background, monitored (foo &) or background, ignored (foo & disown). I/O redirection details will also change things, e.g., if standard input is closed by the shell (foo <&-), or whether a file is opened as stdin (foo < blah).



    strace or similar will be informative about the specific system calls made along this process, and there should be man pages for each of those calls. Suitable system level reading would be any number of chapters from Stevens's "Advanced Programming in the UNIX Environment" while a shell book (e.g., "From Bash to Z Shell") will cover the shell side of things in more detail.






    share|improve this answer























    • I edited the question to narrow down to a simple case
      – Jake
      Aug 27 '15 at 1:17












    up vote
    5
    down vote



    accepted







    up vote
    5
    down vote



    accepted






    Well, the exact sequence may vary, as there might be a shell alias or function that first gets expanded/interpreted before the actual program gets executed, and then differences for a qualified filename (/usr/libexec/foo) versus something that will be looked for through all the directories of the PATH environment variable (just foo). Also, the details of the execution may complicate matters, as foo | bar | zot requires more work for the shell (some number of fork(2), dup(2), and, of course, pipe(2), among other system calls), while something like exec foo is much less work as the shell merely replaces itself with the new program (i.e., it doesn't fork). Also important are process groups (especially the foreground process group, all PIDs of which eat SIGINT when someone starts mashing on Ctrl+C, sessions, and whether the job is going to be run in the background, monitored (foo &) or background, ignored (foo & disown). I/O redirection details will also change things, e.g., if standard input is closed by the shell (foo <&-), or whether a file is opened as stdin (foo < blah).



    strace or similar will be informative about the specific system calls made along this process, and there should be man pages for each of those calls. Suitable system level reading would be any number of chapters from Stevens's "Advanced Programming in the UNIX Environment" while a shell book (e.g., "From Bash to Z Shell") will cover the shell side of things in more detail.






    share|improve this answer















    Well, the exact sequence may vary, as there might be a shell alias or function that first gets expanded/interpreted before the actual program gets executed, and then differences for a qualified filename (/usr/libexec/foo) versus something that will be looked for through all the directories of the PATH environment variable (just foo). Also, the details of the execution may complicate matters, as foo | bar | zot requires more work for the shell (some number of fork(2), dup(2), and, of course, pipe(2), among other system calls), while something like exec foo is much less work as the shell merely replaces itself with the new program (i.e., it doesn't fork). Also important are process groups (especially the foreground process group, all PIDs of which eat SIGINT when someone starts mashing on Ctrl+C, sessions, and whether the job is going to be run in the background, monitored (foo &) or background, ignored (foo & disown). I/O redirection details will also change things, e.g., if standard input is closed by the shell (foo <&-), or whether a file is opened as stdin (foo < blah).



    strace or similar will be informative about the specific system calls made along this process, and there should be man pages for each of those calls. Suitable system level reading would be any number of chapters from Stevens's "Advanced Programming in the UNIX Environment" while a shell book (e.g., "From Bash to Z Shell") will cover the shell side of things in more detail.







    share|improve this answer















    share|improve this answer



    share|improve this answer








    edited Aug 27 '15 at 0:57









    Scott

    6,21332347




    6,21332347











    answered Aug 27 '15 at 0:07









    thrig

    21.8k12751




    21.8k12751











    • I edited the question to narrow down to a simple case
      – Jake
      Aug 27 '15 at 1:17
















    • I edited the question to narrow down to a simple case
      – Jake
      Aug 27 '15 at 1:17















    I edited the question to narrow down to a simple case
    – Jake
    Aug 27 '15 at 1:17




    I edited the question to narrow down to a simple case
    – Jake
    Aug 27 '15 at 1:17












    up vote
    1
    down vote













    Assuming a textbook example shell (for code clarity) that is already running (so the dynamic linker is done), the commands you mention will require the shell to make the following system calls:



    • read: gets the next command in this case gcc

    • fork: two process are needed, we assume the parent has pid 500 and the child for illustration.

    • the parent will call wait(501), meanwhile the child will call exec. At this point the shell is no longer running on pid 501. gcc makes lots of system calls including at a minimum open, close, read, write, chmod, fork, exec, wait and exit.

    • when gcc calls exit, wait will return, write is called to display the prompt and the process will repeat.

    More complicated commands of course add more complication to this basic sequence. Two simpler examples of basic complications are basic io redirection where a open, close, dup sequence is inserted between the fork and the exec and background processes where the wait is skipped (and another wait is added to a sigchld handler).






    share|improve this answer





















    • Small addition: the question asks about loading and dynamic linking. All the code that is statically linked, i.e. actually included in the program file, is done by the kernel before the program is started. Dynamically loaded libraries, i.e. separate files, are handled by the program itself before starting main(). The code for this is automatically added by gcc.
      – Stig Hemmer
      Aug 27 '15 at 8:55















    up vote
    1
    down vote













    Assuming a textbook example shell (for code clarity) that is already running (so the dynamic linker is done), the commands you mention will require the shell to make the following system calls:



    • read: gets the next command in this case gcc

    • fork: two process are needed, we assume the parent has pid 500 and the child for illustration.

    • the parent will call wait(501), meanwhile the child will call exec. At this point the shell is no longer running on pid 501. gcc makes lots of system calls including at a minimum open, close, read, write, chmod, fork, exec, wait and exit.

    • when gcc calls exit, wait will return, write is called to display the prompt and the process will repeat.

    More complicated commands of course add more complication to this basic sequence. Two simpler examples of basic complications are basic io redirection where a open, close, dup sequence is inserted between the fork and the exec and background processes where the wait is skipped (and another wait is added to a sigchld handler).






    share|improve this answer





















    • Small addition: the question asks about loading and dynamic linking. All the code that is statically linked, i.e. actually included in the program file, is done by the kernel before the program is started. Dynamically loaded libraries, i.e. separate files, are handled by the program itself before starting main(). The code for this is automatically added by gcc.
      – Stig Hemmer
      Aug 27 '15 at 8:55













    up vote
    1
    down vote










    up vote
    1
    down vote









    Assuming a textbook example shell (for code clarity) that is already running (so the dynamic linker is done), the commands you mention will require the shell to make the following system calls:



    • read: gets the next command in this case gcc

    • fork: two process are needed, we assume the parent has pid 500 and the child for illustration.

    • the parent will call wait(501), meanwhile the child will call exec. At this point the shell is no longer running on pid 501. gcc makes lots of system calls including at a minimum open, close, read, write, chmod, fork, exec, wait and exit.

    • when gcc calls exit, wait will return, write is called to display the prompt and the process will repeat.

    More complicated commands of course add more complication to this basic sequence. Two simpler examples of basic complications are basic io redirection where a open, close, dup sequence is inserted between the fork and the exec and background processes where the wait is skipped (and another wait is added to a sigchld handler).






    share|improve this answer













    Assuming a textbook example shell (for code clarity) that is already running (so the dynamic linker is done), the commands you mention will require the shell to make the following system calls:



    • read: gets the next command in this case gcc

    • fork: two process are needed, we assume the parent has pid 500 and the child for illustration.

    • the parent will call wait(501), meanwhile the child will call exec. At this point the shell is no longer running on pid 501. gcc makes lots of system calls including at a minimum open, close, read, write, chmod, fork, exec, wait and exit.

    • when gcc calls exit, wait will return, write is called to display the prompt and the process will repeat.

    More complicated commands of course add more complication to this basic sequence. Two simpler examples of basic complications are basic io redirection where a open, close, dup sequence is inserted between the fork and the exec and background processes where the wait is skipped (and another wait is added to a sigchld handler).







    share|improve this answer













    share|improve this answer



    share|improve this answer











    answered Aug 27 '15 at 3:27









    hildred

    4,59722037




    4,59722037











    • Small addition: the question asks about loading and dynamic linking. All the code that is statically linked, i.e. actually included in the program file, is done by the kernel before the program is started. Dynamically loaded libraries, i.e. separate files, are handled by the program itself before starting main(). The code for this is automatically added by gcc.
      – Stig Hemmer
      Aug 27 '15 at 8:55

















    • Small addition: the question asks about loading and dynamic linking. All the code that is statically linked, i.e. actually included in the program file, is done by the kernel before the program is started. Dynamically loaded libraries, i.e. separate files, are handled by the program itself before starting main(). The code for this is automatically added by gcc.
      – Stig Hemmer
      Aug 27 '15 at 8:55
















    Small addition: the question asks about loading and dynamic linking. All the code that is statically linked, i.e. actually included in the program file, is done by the kernel before the program is started. Dynamically loaded libraries, i.e. separate files, are handled by the program itself before starting main(). The code for this is automatically added by gcc.
    – Stig Hemmer
    Aug 27 '15 at 8:55





    Small addition: the question asks about loading and dynamic linking. All the code that is statically linked, i.e. actually included in the program file, is done by the kernel before the program is started. Dynamically loaded libraries, i.e. separate files, are handled by the program itself before starting main(). The code for this is automatically added by gcc.
    – Stig Hemmer
    Aug 27 '15 at 8:55











    up vote
    0
    down vote













    I suggest reading Section 8.4.6 Using fork and execve to Run Programs



    on
    http://www.groupes.polymtl.ca/inf2610/documentation/ComputerSystemBook.pdf






    share|improve this answer

























      up vote
      0
      down vote













      I suggest reading Section 8.4.6 Using fork and execve to Run Programs



      on
      http://www.groupes.polymtl.ca/inf2610/documentation/ComputerSystemBook.pdf






      share|improve this answer























        up vote
        0
        down vote










        up vote
        0
        down vote









        I suggest reading Section 8.4.6 Using fork and execve to Run Programs



        on
        http://www.groupes.polymtl.ca/inf2610/documentation/ComputerSystemBook.pdf






        share|improve this answer













        I suggest reading Section 8.4.6 Using fork and execve to Run Programs



        on
        http://www.groupes.polymtl.ca/inf2610/documentation/ComputerSystemBook.pdf







        share|improve this answer













        share|improve this answer



        share|improve this answer











        answered May 24 '16 at 8:46









        xiaokaoy

        1063




        1063






















             

            draft saved


            draft discarded


























             


            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f225736%2fhow-does-a-shell-execute-a-program%23new-answer', 'question_page');

            );

            Post as a guest













































































            Popular posts from this blog

            How to check contact read email or not when send email to Individual?

            Displaying single band from multi-band raster using QGIS

            How many registers does an x86_64 CPU actually have?