Why stdout won't flush when redirecting input from a fifo?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
2
down vote

favorite












I have a program that displays a prompt message and waits for the user to input some text.



$ program
Input a line of text: <aaaa>
Some more output.
$


Now, I also want to be able to provide this input, programmatically, via a FIFO, like so:



$ program < fifo


But what's happening is, the prompt on stdout does not appear until I have supplied the input to fifo.



I next tried a very simple equivalent of my program, like so:



$ echo aaaa < fifo


Even here, aaaa doesn't appear until fifo has been supplied its input.



Question: How do I make my output till the point I wait for input from stdin appear on stdout when using FIFO?







share|improve this question



















  • The script/program doesn't even start to execute until there is data in the pipe. Also, unrelated to your issue: Prompting should always be done on the standard error stream, not and standard output.
    – Kusalananda
    Jun 14 at 14:15










  • That's (shocking) news to me! Why should the program not even start? If the input were coming in from a regular disk file (and, not from a fifo), would the same semantics of program execution hold? Input should get read ONLY when it's needed, not before. So, a reading from fifo should block only on reaching the point of input in the program, no?
    – Harry
    Jun 14 at 14:23














up vote
2
down vote

favorite












I have a program that displays a prompt message and waits for the user to input some text.



$ program
Input a line of text: <aaaa>
Some more output.
$


Now, I also want to be able to provide this input, programmatically, via a FIFO, like so:



$ program < fifo


But what's happening is, the prompt on stdout does not appear until I have supplied the input to fifo.



I next tried a very simple equivalent of my program, like so:



$ echo aaaa < fifo


Even here, aaaa doesn't appear until fifo has been supplied its input.



Question: How do I make my output till the point I wait for input from stdin appear on stdout when using FIFO?







share|improve this question



















  • The script/program doesn't even start to execute until there is data in the pipe. Also, unrelated to your issue: Prompting should always be done on the standard error stream, not and standard output.
    – Kusalananda
    Jun 14 at 14:15










  • That's (shocking) news to me! Why should the program not even start? If the input were coming in from a regular disk file (and, not from a fifo), would the same semantics of program execution hold? Input should get read ONLY when it's needed, not before. So, a reading from fifo should block only on reaching the point of input in the program, no?
    – Harry
    Jun 14 at 14:23












up vote
2
down vote

favorite









up vote
2
down vote

favorite











I have a program that displays a prompt message and waits for the user to input some text.



$ program
Input a line of text: <aaaa>
Some more output.
$


Now, I also want to be able to provide this input, programmatically, via a FIFO, like so:



$ program < fifo


But what's happening is, the prompt on stdout does not appear until I have supplied the input to fifo.



I next tried a very simple equivalent of my program, like so:



$ echo aaaa < fifo


Even here, aaaa doesn't appear until fifo has been supplied its input.



Question: How do I make my output till the point I wait for input from stdin appear on stdout when using FIFO?







share|improve this question











I have a program that displays a prompt message and waits for the user to input some text.



$ program
Input a line of text: <aaaa>
Some more output.
$


Now, I also want to be able to provide this input, programmatically, via a FIFO, like so:



$ program < fifo


But what's happening is, the prompt on stdout does not appear until I have supplied the input to fifo.



I next tried a very simple equivalent of my program, like so:



$ echo aaaa < fifo


Even here, aaaa doesn't appear until fifo has been supplied its input.



Question: How do I make my output till the point I wait for input from stdin appear on stdout when using FIFO?









share|improve this question










share|improve this question




share|improve this question









asked Jun 14 at 13:39









Harry

3701313




3701313











  • The script/program doesn't even start to execute until there is data in the pipe. Also, unrelated to your issue: Prompting should always be done on the standard error stream, not and standard output.
    – Kusalananda
    Jun 14 at 14:15










  • That's (shocking) news to me! Why should the program not even start? If the input were coming in from a regular disk file (and, not from a fifo), would the same semantics of program execution hold? Input should get read ONLY when it's needed, not before. So, a reading from fifo should block only on reaching the point of input in the program, no?
    – Harry
    Jun 14 at 14:23
















  • The script/program doesn't even start to execute until there is data in the pipe. Also, unrelated to your issue: Prompting should always be done on the standard error stream, not and standard output.
    – Kusalananda
    Jun 14 at 14:15










  • That's (shocking) news to me! Why should the program not even start? If the input were coming in from a regular disk file (and, not from a fifo), would the same semantics of program execution hold? Input should get read ONLY when it's needed, not before. So, a reading from fifo should block only on reaching the point of input in the program, no?
    – Harry
    Jun 14 at 14:23















The script/program doesn't even start to execute until there is data in the pipe. Also, unrelated to your issue: Prompting should always be done on the standard error stream, not and standard output.
– Kusalananda
Jun 14 at 14:15




The script/program doesn't even start to execute until there is data in the pipe. Also, unrelated to your issue: Prompting should always be done on the standard error stream, not and standard output.
– Kusalananda
Jun 14 at 14:15












That's (shocking) news to me! Why should the program not even start? If the input were coming in from a regular disk file (and, not from a fifo), would the same semantics of program execution hold? Input should get read ONLY when it's needed, not before. So, a reading from fifo should block only on reaching the point of input in the program, no?
– Harry
Jun 14 at 14:23




That's (shocking) news to me! Why should the program not even start? If the input were coming in from a regular disk file (and, not from a fifo), would the same semantics of program execution hold? Input should get read ONLY when it's needed, not before. So, a reading from fifo should block only on reaching the point of input in the program, no?
– Harry
Jun 14 at 14:23










1 Answer
1






active

oldest

votes

















up vote
4
down vote



accepted










The FIFO is opened by the shell, but the opening of it is blocking until the FIFO is opened for writing by some other process. Since redirections are processed before the execution of the command, the program is not even executed until the shell's open() call returns. This is documented behaviour for the open() C library function (used by the shell):




O_NONBLOCK



When opening a FIFO with O_RDONLY or O_WRONLY set:



  • If O_NONBLOCK is set, an open() for reading-only
    shall return without delay. An open() for writing-only shall return an error if no process currently
    has the file open for reading.


  • If O_NONBLOCK is clear, an open() for reading-only
    shall block the calling thread until a thread opens
    the file for writing
    . An open() for writing-only
    shall block the calling thread until a thread opens
    the file for reading.




So the shell is obviously not using O_NONBLOCK when it's opening the FIFO.



Solution:




  1. cat fifo | program



    cat reads from the FIFO until end-of-file, then exits.




  2. tail -f fifo | program



    tail reads from the FIFO until end-of-file, then hangs around to wait for more until terminated.



Which of these is best suited to your situation depends a bit on what's writing to the FIFO at the other end, and how it does the writing.



Third solution involves having the program opening the FIFO when needed.




According to the Rationale section on sh, a shell must set standard input to blocking because...




If the shell did not reset this flag, it would immediately terminate because no input data would be available yet and that would be considered the same as end-of-file.







share|improve this answer



















  • 1




    FIFOs on Linux and FreeBSD can be opened in O_RDWR mode, yielding a third choice.
    – JdeBP
    Jun 14 at 15:50










  • Hmh, as for the last quote, wouldn't the read() on the pipe just block until a writer comes along. Or should I say "couldn't it". That's what seems to happen on Linux anyway, if you open in read-write mode... except then it never gives an EOF condition. Logically enough, the same process is a writer too.
    – ilkkachu
    Jun 14 at 16:24











Your Answer







StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);








 

draft saved


draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f449816%2fwhy-stdout-wont-flush-when-redirecting-input-from-a-fifo%23new-answer', 'question_page');

);

Post as a guest






























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
4
down vote



accepted










The FIFO is opened by the shell, but the opening of it is blocking until the FIFO is opened for writing by some other process. Since redirections are processed before the execution of the command, the program is not even executed until the shell's open() call returns. This is documented behaviour for the open() C library function (used by the shell):




O_NONBLOCK



When opening a FIFO with O_RDONLY or O_WRONLY set:



  • If O_NONBLOCK is set, an open() for reading-only
    shall return without delay. An open() for writing-only shall return an error if no process currently
    has the file open for reading.


  • If O_NONBLOCK is clear, an open() for reading-only
    shall block the calling thread until a thread opens
    the file for writing
    . An open() for writing-only
    shall block the calling thread until a thread opens
    the file for reading.




So the shell is obviously not using O_NONBLOCK when it's opening the FIFO.



Solution:




  1. cat fifo | program



    cat reads from the FIFO until end-of-file, then exits.




  2. tail -f fifo | program



    tail reads from the FIFO until end-of-file, then hangs around to wait for more until terminated.



Which of these is best suited to your situation depends a bit on what's writing to the FIFO at the other end, and how it does the writing.



Third solution involves having the program opening the FIFO when needed.




According to the Rationale section on sh, a shell must set standard input to blocking because...




If the shell did not reset this flag, it would immediately terminate because no input data would be available yet and that would be considered the same as end-of-file.







share|improve this answer



















  • 1




    FIFOs on Linux and FreeBSD can be opened in O_RDWR mode, yielding a third choice.
    – JdeBP
    Jun 14 at 15:50










  • Hmh, as for the last quote, wouldn't the read() on the pipe just block until a writer comes along. Or should I say "couldn't it". That's what seems to happen on Linux anyway, if you open in read-write mode... except then it never gives an EOF condition. Logically enough, the same process is a writer too.
    – ilkkachu
    Jun 14 at 16:24















up vote
4
down vote



accepted










The FIFO is opened by the shell, but the opening of it is blocking until the FIFO is opened for writing by some other process. Since redirections are processed before the execution of the command, the program is not even executed until the shell's open() call returns. This is documented behaviour for the open() C library function (used by the shell):




O_NONBLOCK



When opening a FIFO with O_RDONLY or O_WRONLY set:



  • If O_NONBLOCK is set, an open() for reading-only
    shall return without delay. An open() for writing-only shall return an error if no process currently
    has the file open for reading.


  • If O_NONBLOCK is clear, an open() for reading-only
    shall block the calling thread until a thread opens
    the file for writing
    . An open() for writing-only
    shall block the calling thread until a thread opens
    the file for reading.




So the shell is obviously not using O_NONBLOCK when it's opening the FIFO.



Solution:




  1. cat fifo | program



    cat reads from the FIFO until end-of-file, then exits.




  2. tail -f fifo | program



    tail reads from the FIFO until end-of-file, then hangs around to wait for more until terminated.



Which of these is best suited to your situation depends a bit on what's writing to the FIFO at the other end, and how it does the writing.



Third solution involves having the program opening the FIFO when needed.




According to the Rationale section on sh, a shell must set standard input to blocking because...




If the shell did not reset this flag, it would immediately terminate because no input data would be available yet and that would be considered the same as end-of-file.







share|improve this answer



















  • 1




    FIFOs on Linux and FreeBSD can be opened in O_RDWR mode, yielding a third choice.
    – JdeBP
    Jun 14 at 15:50










  • Hmh, as for the last quote, wouldn't the read() on the pipe just block until a writer comes along. Or should I say "couldn't it". That's what seems to happen on Linux anyway, if you open in read-write mode... except then it never gives an EOF condition. Logically enough, the same process is a writer too.
    – ilkkachu
    Jun 14 at 16:24













up vote
4
down vote



accepted







up vote
4
down vote



accepted






The FIFO is opened by the shell, but the opening of it is blocking until the FIFO is opened for writing by some other process. Since redirections are processed before the execution of the command, the program is not even executed until the shell's open() call returns. This is documented behaviour for the open() C library function (used by the shell):




O_NONBLOCK



When opening a FIFO with O_RDONLY or O_WRONLY set:



  • If O_NONBLOCK is set, an open() for reading-only
    shall return without delay. An open() for writing-only shall return an error if no process currently
    has the file open for reading.


  • If O_NONBLOCK is clear, an open() for reading-only
    shall block the calling thread until a thread opens
    the file for writing
    . An open() for writing-only
    shall block the calling thread until a thread opens
    the file for reading.




So the shell is obviously not using O_NONBLOCK when it's opening the FIFO.



Solution:




  1. cat fifo | program



    cat reads from the FIFO until end-of-file, then exits.




  2. tail -f fifo | program



    tail reads from the FIFO until end-of-file, then hangs around to wait for more until terminated.



Which of these is best suited to your situation depends a bit on what's writing to the FIFO at the other end, and how it does the writing.



Third solution involves having the program opening the FIFO when needed.




According to the Rationale section on sh, a shell must set standard input to blocking because...




If the shell did not reset this flag, it would immediately terminate because no input data would be available yet and that would be considered the same as end-of-file.







share|improve this answer















The FIFO is opened by the shell, but the opening of it is blocking until the FIFO is opened for writing by some other process. Since redirections are processed before the execution of the command, the program is not even executed until the shell's open() call returns. This is documented behaviour for the open() C library function (used by the shell):




O_NONBLOCK



When opening a FIFO with O_RDONLY or O_WRONLY set:



  • If O_NONBLOCK is set, an open() for reading-only
    shall return without delay. An open() for writing-only shall return an error if no process currently
    has the file open for reading.


  • If O_NONBLOCK is clear, an open() for reading-only
    shall block the calling thread until a thread opens
    the file for writing
    . An open() for writing-only
    shall block the calling thread until a thread opens
    the file for reading.




So the shell is obviously not using O_NONBLOCK when it's opening the FIFO.



Solution:




  1. cat fifo | program



    cat reads from the FIFO until end-of-file, then exits.




  2. tail -f fifo | program



    tail reads from the FIFO until end-of-file, then hangs around to wait for more until terminated.



Which of these is best suited to your situation depends a bit on what's writing to the FIFO at the other end, and how it does the writing.



Third solution involves having the program opening the FIFO when needed.




According to the Rationale section on sh, a shell must set standard input to blocking because...




If the shell did not reset this flag, it would immediately terminate because no input data would be available yet and that would be considered the same as end-of-file.








share|improve this answer















share|improve this answer



share|improve this answer








edited Jun 14 at 15:04


























answered Jun 14 at 14:43









Kusalananda

101k13199312




101k13199312







  • 1




    FIFOs on Linux and FreeBSD can be opened in O_RDWR mode, yielding a third choice.
    – JdeBP
    Jun 14 at 15:50










  • Hmh, as for the last quote, wouldn't the read() on the pipe just block until a writer comes along. Or should I say "couldn't it". That's what seems to happen on Linux anyway, if you open in read-write mode... except then it never gives an EOF condition. Logically enough, the same process is a writer too.
    – ilkkachu
    Jun 14 at 16:24













  • 1




    FIFOs on Linux and FreeBSD can be opened in O_RDWR mode, yielding a third choice.
    – JdeBP
    Jun 14 at 15:50










  • Hmh, as for the last quote, wouldn't the read() on the pipe just block until a writer comes along. Or should I say "couldn't it". That's what seems to happen on Linux anyway, if you open in read-write mode... except then it never gives an EOF condition. Logically enough, the same process is a writer too.
    – ilkkachu
    Jun 14 at 16:24








1




1




FIFOs on Linux and FreeBSD can be opened in O_RDWR mode, yielding a third choice.
– JdeBP
Jun 14 at 15:50




FIFOs on Linux and FreeBSD can be opened in O_RDWR mode, yielding a third choice.
– JdeBP
Jun 14 at 15:50












Hmh, as for the last quote, wouldn't the read() on the pipe just block until a writer comes along. Or should I say "couldn't it". That's what seems to happen on Linux anyway, if you open in read-write mode... except then it never gives an EOF condition. Logically enough, the same process is a writer too.
– ilkkachu
Jun 14 at 16:24





Hmh, as for the last quote, wouldn't the read() on the pipe just block until a writer comes along. Or should I say "couldn't it". That's what seems to happen on Linux anyway, if you open in read-write mode... except then it never gives an EOF condition. Logically enough, the same process is a writer too.
– ilkkachu
Jun 14 at 16:24













 

draft saved


draft discarded


























 


draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f449816%2fwhy-stdout-wont-flush-when-redirecting-input-from-a-fifo%23new-answer', 'question_page');

);

Post as a guest













































































Popular posts from this blog

How to check contact read email or not when send email to Individual?

Displaying single band from multi-band raster using QGIS

How many registers does an x86_64 CPU actually have?