sed not working correctly when piped
Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
For some reason, I will not have live output when piping the command output of e.g. SSH to sed:
ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee logging
I suspected that the output dit not have newlines, but when i remove the sed command and run:
wc -l logging
It returns 6, which is the correct number of returns. Does anybody have an idea what i am missing here?
[EDIT]
I completely forgot to mention the following: running
ssh someuser@somehost 2>&1 | sed -e "s/[32//g"
returns all values no problem, however, as soon as i add tee into the mix, i won't get outputs until I hit ^C... Not even standard output redirection works (ssh someuser@somehost 2>&1 | sed -e "s/[32//g" > file). Just ssh with tee also works fine.
shell-script sed pipe
asked Nov 17 at 2:40
Matthias Cotting
11
New contributor
Matthias Cotting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
up vote
0
down vote
favorite
For some reason, I will not have live output when piping the command output of e.g. SSH to sed:
ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee logging
I suspected that the output dit not have newlines, but when i remove the sed command and run:
wc -l logging
It returns 6, which is the correct number of returns. Does anybody have an idea what i am missing here?
[EDIT]
I completely forgot to mention the following: running
ssh someuser@somehost 2>&1 | sed -e "s/[32//g"
returns all values no problem, however, as soon as i add tee into the mix, i won't get outputs until I hit ^C... Not even standard output redirection works (ssh someuser@somehost 2>&1 | sed -e "s/[32//g" > file). Just ssh with tee also works fine.
shell-script sed pipe
asked Nov 17 at 2:40
Matthias Cotting
11
New contributor
Matthias Cotting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Possibly related: Turn off buffering in pipe
– steeldriver
Nov 17 at 3:03
It's the same old buffering problem -- on linux, you can work it around by replacing thesed ...withstdbuf -oL sed .... Though notice that you won't have echoing and prompting in the interactivesshif you usesedat all (with or withouttee). If your case, I wonder why you aren't simply usingscript(1).
– mosvy
2 days ago
or ifsedis GNUsed, usesed -u.
– mosvy
2 days ago
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
For some reason, I will not have live output when piping the command output of e.g. SSH to sed:
ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee logging
I suspected that the output dit not have newlines, but when i remove the sed command and run:
wc -l logging
It returns 6, which is the correct number of returns. Does anybody have an idea what i am missing here?
[EDIT]
I completely forgot to mention the following: running
ssh someuser@somehost 2>&1 | sed -e "s/[32//g"
returns all values no problem, however, as soon as i add tee into the mix, i won't get outputs until I hit ^C... Not even standard output redirection works (ssh someuser@somehost 2>&1 | sed -e "s/[32//g" > file). Just ssh with tee also works fine.
shell-script sed pipe
asked Nov 17 at 2:40
Matthias Cotting
11
New contributor
Matthias Cotting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
For some reason, I will not have live output when piping the command output of e.g. SSH to sed:
ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee logging
I suspected that the output dit not have newlines, but when i remove the sed command and run:
wc -l logging
It returns 6, which is the correct number of returns. Does anybody have an idea what i am missing here?
[EDIT]
I completely forgot to mention the following: running
ssh someuser@somehost 2>&1 | sed -e "s/[32//g"
returns all values no problem, however, as soon as i add tee into the mix, i won't get outputs until I hit ^C... Not even standard output redirection works (ssh someuser@somehost 2>&1 | sed -e "s/[32//g" > file). Just ssh with tee also works fine.
shell-script sed pipe
shell-script sed pipe
asked Nov 17 at 2:40
Matthias Cotting
11
New contributor
Matthias Cotting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Nov 17 at 2:40
Matthias Cotting
11
New contributor
Matthias Cotting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 2 days ago
asked Nov 17 at 2:40
Matthias Cotting
11
New contributor
Matthias Cotting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Nov 17 at 2:40
Matthias Cotting
11
asked Nov 17 at 2:40
Matthias Cotting
11
11
New contributor
Matthias Cotting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Matthias Cotting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Matthias Cotting is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Possibly related: Turn off buffering in pipe
– steeldriver
Nov 17 at 3:03
It's the same old buffering problem -- on linux, you can work it around by replacing thesed ...withstdbuf -oL sed .... Though notice that you won't have echoing and prompting in the interactivesshif you usesedat all (with or withouttee). If your case, I wonder why you aren't simply usingscript(1).
– mosvy
2 days ago
or ifsedis GNUsed, usesed -u.
– mosvy
2 days ago
add a comment |
Possibly related: Turn off buffering in pipe
– steeldriver
Nov 17 at 3:03
It's the same old buffering problem -- on linux, you can work it around by replacing thesed ...withstdbuf -oL sed .... Though notice that you won't have echoing and prompting in the interactivesshif you usesedat all (with or withouttee). If your case, I wonder why you aren't simply usingscript(1).
– mosvy
2 days ago
or ifsedis GNUsed, usesed -u.
– mosvy
2 days ago
Possibly related: Turn off buffering in pipe
– steeldriver
Nov 17 at 3:03
Possibly related: Turn off buffering in pipe
– steeldriver
Nov 17 at 3:03
It's the same old buffering problem -- on linux, you can work it around by replacing the
sed ... with stdbuf -oL sed .... Though notice that you won't have echoing and prompting in the interactive ssh if you use sed at all (with or without tee). If your case, I wonder why you aren't simply using script(1).– mosvy
2 days ago
It's the same old buffering problem -- on linux, you can work it around by replacing the
sed ... with stdbuf -oL sed .... Though notice that you won't have echoing and prompting in the interactive ssh if you use sed at all (with or without tee). If your case, I wonder why you aren't simply using script(1).– mosvy
2 days ago
or if
sed is GNU sed, use sed -u.– mosvy
2 days ago
or if
sed is GNU sed, use sed -u.– mosvy
2 days ago
add a comment |
2 Answers
2
active
oldest
votes
up vote
2
down vote
When you run ssh user@somehost without an explicit command, you're requesting ssh to start an interactive shell on the remote machine.
An interactive shell (such as bash) typically wants to have a terminal available, since it will use terminal commands to implement improved experience when editing the command line and browsing the history. (Terminal commands allow full-screen control.)
But ssh will only allocate a terminal by default (also called a pseudo-terminal) if its standard output is connected to a terminal.
If you simply run ssh user@somehost from a terminal program (such as gnome-terminal, rxvt, xterm, etc.), then its standard output will be a terminal, so ssh will create a pseudo-terminal and the remote interactive shell will behave nicely.
If you pipe ssh through something (anything), like ssh user@somehost | cat, then its standard output will be a pipe (and not the terminal), so ssh will not create a terminal and that might make the remote interactive shell misbehave.
A possible solution is to force ssh to create a pseudo-terminal, by passing it the -t option, like ssh -t user@somehost | cat, which might help. (Also, you might need a double option -tt to force allocation of the pseudo-terminal.)
Another possibility, if you're running ssh mainly because you're interested on a specific command, is to run the specific command from the ssh command line, such as ssh user@somehost mycommand | cat. If you run a specific command, an interactive shell does not get involved, in which case having a terminal available is likely not going to cause any issues.
answered Nov 17 at 5:33
Filipe Brandenburger
6,1041625
Nope, sadly the -t option does not help...
– Matthias Cotting
2 days ago
add a comment |
up vote
0
down vote
I would try the same command with the following addition, ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee --output-error=warn logging. With that you should be able to see if their is an error in writing to the stdout. Let me know if that helps.
You could install expect
Then try: unbuffer ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee logging
answered Nov 17 at 3:30
Michael Prokopec
947
New contributor
Michael Prokopec is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Nope, no errors...
– Matthias Cotting
2 days ago
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
When you run ssh user@somehost without an explicit command, you're requesting ssh to start an interactive shell on the remote machine.
An interactive shell (such as bash) typically wants to have a terminal available, since it will use terminal commands to implement improved experience when editing the command line and browsing the history. (Terminal commands allow full-screen control.)
But ssh will only allocate a terminal by default (also called a pseudo-terminal) if its standard output is connected to a terminal.
If you simply run ssh user@somehost from a terminal program (such as gnome-terminal, rxvt, xterm, etc.), then its standard output will be a terminal, so ssh will create a pseudo-terminal and the remote interactive shell will behave nicely.
If you pipe ssh through something (anything), like ssh user@somehost | cat, then its standard output will be a pipe (and not the terminal), so ssh will not create a terminal and that might make the remote interactive shell misbehave.
A possible solution is to force ssh to create a pseudo-terminal, by passing it the -t option, like ssh -t user@somehost | cat, which might help. (Also, you might need a double option -tt to force allocation of the pseudo-terminal.)
Another possibility, if you're running ssh mainly because you're interested on a specific command, is to run the specific command from the ssh command line, such as ssh user@somehost mycommand | cat. If you run a specific command, an interactive shell does not get involved, in which case having a terminal available is likely not going to cause any issues.
answered Nov 17 at 5:33
Filipe Brandenburger
6,1041625
Nope, sadly the -t option does not help...
– Matthias Cotting
2 days ago
add a comment |
up vote
2
down vote
When you run ssh user@somehost without an explicit command, you're requesting ssh to start an interactive shell on the remote machine.
An interactive shell (such as bash) typically wants to have a terminal available, since it will use terminal commands to implement improved experience when editing the command line and browsing the history. (Terminal commands allow full-screen control.)
But ssh will only allocate a terminal by default (also called a pseudo-terminal) if its standard output is connected to a terminal.
If you simply run ssh user@somehost from a terminal program (such as gnome-terminal, rxvt, xterm, etc.), then its standard output will be a terminal, so ssh will create a pseudo-terminal and the remote interactive shell will behave nicely.
If you pipe ssh through something (anything), like ssh user@somehost | cat, then its standard output will be a pipe (and not the terminal), so ssh will not create a terminal and that might make the remote interactive shell misbehave.
A possible solution is to force ssh to create a pseudo-terminal, by passing it the -t option, like ssh -t user@somehost | cat, which might help. (Also, you might need a double option -tt to force allocation of the pseudo-terminal.)
Another possibility, if you're running ssh mainly because you're interested on a specific command, is to run the specific command from the ssh command line, such as ssh user@somehost mycommand | cat. If you run a specific command, an interactive shell does not get involved, in which case having a terminal available is likely not going to cause any issues.
answered Nov 17 at 5:33
Filipe Brandenburger
6,1041625
Nope, sadly the -t option does not help...
– Matthias Cotting
2 days ago
add a comment |
up vote
2
down vote
up vote
2
down vote
When you run ssh user@somehost without an explicit command, you're requesting ssh to start an interactive shell on the remote machine.
An interactive shell (such as bash) typically wants to have a terminal available, since it will use terminal commands to implement improved experience when editing the command line and browsing the history. (Terminal commands allow full-screen control.)
But ssh will only allocate a terminal by default (also called a pseudo-terminal) if its standard output is connected to a terminal.
If you simply run ssh user@somehost from a terminal program (such as gnome-terminal, rxvt, xterm, etc.), then its standard output will be a terminal, so ssh will create a pseudo-terminal and the remote interactive shell will behave nicely.
If you pipe ssh through something (anything), like ssh user@somehost | cat, then its standard output will be a pipe (and not the terminal), so ssh will not create a terminal and that might make the remote interactive shell misbehave.
A possible solution is to force ssh to create a pseudo-terminal, by passing it the -t option, like ssh -t user@somehost | cat, which might help. (Also, you might need a double option -tt to force allocation of the pseudo-terminal.)
Another possibility, if you're running ssh mainly because you're interested on a specific command, is to run the specific command from the ssh command line, such as ssh user@somehost mycommand | cat. If you run a specific command, an interactive shell does not get involved, in which case having a terminal available is likely not going to cause any issues.
answered Nov 17 at 5:33
Filipe Brandenburger
6,1041625
When you run ssh user@somehost without an explicit command, you're requesting ssh to start an interactive shell on the remote machine.
An interactive shell (such as bash) typically wants to have a terminal available, since it will use terminal commands to implement improved experience when editing the command line and browsing the history. (Terminal commands allow full-screen control.)
But ssh will only allocate a terminal by default (also called a pseudo-terminal) if its standard output is connected to a terminal.
If you simply run ssh user@somehost from a terminal program (such as gnome-terminal, rxvt, xterm, etc.), then its standard output will be a terminal, so ssh will create a pseudo-terminal and the remote interactive shell will behave nicely.
If you pipe ssh through something (anything), like ssh user@somehost | cat, then its standard output will be a pipe (and not the terminal), so ssh will not create a terminal and that might make the remote interactive shell misbehave.
A possible solution is to force ssh to create a pseudo-terminal, by passing it the -t option, like ssh -t user@somehost | cat, which might help. (Also, you might need a double option -tt to force allocation of the pseudo-terminal.)
Another possibility, if you're running ssh mainly because you're interested on a specific command, is to run the specific command from the ssh command line, such as ssh user@somehost mycommand | cat. If you run a specific command, an interactive shell does not get involved, in which case having a terminal available is likely not going to cause any issues.
answered Nov 17 at 5:33
Filipe Brandenburger
6,1041625
answered Nov 17 at 5:33
Filipe Brandenburger
6,1041625
answered Nov 17 at 5:33
Filipe Brandenburger
6,1041625
answered Nov 17 at 5:33
Filipe Brandenburger
6,1041625
6,1041625
Nope, sadly the -t option does not help...
– Matthias Cotting
2 days ago
add a comment |
Nope, sadly the -t option does not help...
– Matthias Cotting
2 days ago
Nope, sadly the -t option does not help...
– Matthias Cotting
2 days ago
Nope, sadly the -t option does not help...
– Matthias Cotting
2 days ago
add a comment |
up vote
0
down vote
I would try the same command with the following addition, ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee --output-error=warn logging. With that you should be able to see if their is an error in writing to the stdout. Let me know if that helps.
You could install expect
Then try: unbuffer ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee logging
answered Nov 17 at 3:30
Michael Prokopec
947
New contributor
Michael Prokopec is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Nope, no errors...
– Matthias Cotting
2 days ago
add a comment |
up vote
0
down vote
I would try the same command with the following addition, ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee --output-error=warn logging. With that you should be able to see if their is an error in writing to the stdout. Let me know if that helps.
You could install expect
Then try: unbuffer ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee logging
answered Nov 17 at 3:30
Michael Prokopec
947
New contributor
Michael Prokopec is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Nope, no errors...
– Matthias Cotting
2 days ago
add a comment |
up vote
0
down vote
up vote
0
down vote
I would try the same command with the following addition, ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee --output-error=warn logging. With that you should be able to see if their is an error in writing to the stdout. Let me know if that helps.
You could install expect
Then try: unbuffer ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee logging
answered Nov 17 at 3:30
Michael Prokopec
947
New contributor
Michael Prokopec is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
I would try the same command with the following addition, ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee --output-error=warn logging. With that you should be able to see if their is an error in writing to the stdout. Let me know if that helps.
You could install expect
Then try: unbuffer ssh someuser@somehost 2>&1 | sed -e "s/[32//g" | tee logging
answered Nov 17 at 3:30
Michael Prokopec
947
New contributor
Michael Prokopec is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 2 days ago
answered Nov 17 at 3:30
Michael Prokopec
947
New contributor
Michael Prokopec is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered Nov 17 at 3:30
Michael Prokopec
947
answered Nov 17 at 3:30
Michael Prokopec
947
947
New contributor
Michael Prokopec is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Michael Prokopec is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Michael Prokopec is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Nope, no errors...
– Matthias Cotting
2 days ago
add a comment |
Nope, no errors...
– Matthias Cotting
2 days ago
Nope, no errors...
– Matthias Cotting
2 days ago
Nope, no errors...
– Matthias Cotting
2 days ago
add a comment |
Matthias Cotting is a new contributor. Be nice, and check out our Code of Conduct.
Matthias Cotting is a new contributor. Be nice, and check out our Code of Conduct.
Matthias Cotting is a new contributor. Be nice, and check out our Code of Conduct.
Matthias Cotting is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f482272%2fsed-not-working-correctly-when-piped%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Possibly related: Turn off buffering in pipe
– steeldriver
Nov 17 at 3:03
It's the same old buffering problem -- on linux, you can work it around by replacing the
sed ...withstdbuf -oL sed .... Though notice that you won't have echoing and prompting in the interactivesshif you usesedat all (with or withouttee). If your case, I wonder why you aren't simply usingscript(1).– mosvy
2 days ago
or if
sedis GNUsed, usesed -u.– mosvy
2 days ago