Expect, Command, Pipes and Gzip
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
I am trying to work with the expect command, everything works well until I try to pipe the results of a command into a Gzip file. Here is the snippet of the bash file:
XYZ=$(expect -c "
spawn python log-connector.py -s $strt -e $end -i 600 -a https://server:9000 -u someaccount -f ./conf/firewall-fields2.txt -q ./conf/query.txt -z | gzip >> /data/sources/results-$strt2-$end2.json.gz
expect "Password:"
send "$passr"
interact
")
echo "$XYZ"
$start
- variable (start date)$end
- variable (end date)$strt2
- variable (start date with 00:00)$end2
- variable (end date with 23:59)$pass
- Password entered by the user previously.
If I remove
| gzip >> /data/sources/results-$strt2-$end2.json.gz
from the above it works as expected. When I add it I get the following error:
send: spawn id exp7 not open
while executing
usage: API [-h] [-s START] [-e END] [-d DAY] [-i INTERVAL]
[-a LOGGER] [-l LOGFILE] [-f FIELDS] [-q QUERY]
[-o OUTPUT] [-t FORMAT] [-u USER] [-p PASS] [-z]
API: error: unrecognized arguments: | gzip >> /data/sources/results-20180604-20180604.json.gz
expect
add a comment |Â
up vote
1
down vote
favorite
I am trying to work with the expect command, everything works well until I try to pipe the results of a command into a Gzip file. Here is the snippet of the bash file:
XYZ=$(expect -c "
spawn python log-connector.py -s $strt -e $end -i 600 -a https://server:9000 -u someaccount -f ./conf/firewall-fields2.txt -q ./conf/query.txt -z | gzip >> /data/sources/results-$strt2-$end2.json.gz
expect "Password:"
send "$passr"
interact
")
echo "$XYZ"
$start
- variable (start date)$end
- variable (end date)$strt2
- variable (start date with 00:00)$end2
- variable (end date with 23:59)$pass
- Password entered by the user previously.
If I remove
| gzip >> /data/sources/results-$strt2-$end2.json.gz
from the above it works as expected. When I add it I get the following error:
send: spawn id exp7 not open
while executing
usage: API [-h] [-s START] [-e END] [-d DAY] [-i INTERVAL]
[-a LOGGER] [-l LOGFILE] [-f FIELDS] [-q QUERY]
[-o OUTPUT] [-t FORMAT] [-u USER] [-p PASS] [-z]
API: error: unrecognized arguments: | gzip >> /data/sources/results-20180604-20180604.json.gz
expect
add a comment |Â
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I am trying to work with the expect command, everything works well until I try to pipe the results of a command into a Gzip file. Here is the snippet of the bash file:
XYZ=$(expect -c "
spawn python log-connector.py -s $strt -e $end -i 600 -a https://server:9000 -u someaccount -f ./conf/firewall-fields2.txt -q ./conf/query.txt -z | gzip >> /data/sources/results-$strt2-$end2.json.gz
expect "Password:"
send "$passr"
interact
")
echo "$XYZ"
$start
- variable (start date)$end
- variable (end date)$strt2
- variable (start date with 00:00)$end2
- variable (end date with 23:59)$pass
- Password entered by the user previously.
If I remove
| gzip >> /data/sources/results-$strt2-$end2.json.gz
from the above it works as expected. When I add it I get the following error:
send: spawn id exp7 not open
while executing
usage: API [-h] [-s START] [-e END] [-d DAY] [-i INTERVAL]
[-a LOGGER] [-l LOGFILE] [-f FIELDS] [-q QUERY]
[-o OUTPUT] [-t FORMAT] [-u USER] [-p PASS] [-z]
API: error: unrecognized arguments: | gzip >> /data/sources/results-20180604-20180604.json.gz
expect
I am trying to work with the expect command, everything works well until I try to pipe the results of a command into a Gzip file. Here is the snippet of the bash file:
XYZ=$(expect -c "
spawn python log-connector.py -s $strt -e $end -i 600 -a https://server:9000 -u someaccount -f ./conf/firewall-fields2.txt -q ./conf/query.txt -z | gzip >> /data/sources/results-$strt2-$end2.json.gz
expect "Password:"
send "$passr"
interact
")
echo "$XYZ"
$start
- variable (start date)$end
- variable (end date)$strt2
- variable (start date with 00:00)$end2
- variable (end date with 23:59)$pass
- Password entered by the user previously.
If I remove
| gzip >> /data/sources/results-$strt2-$end2.json.gz
from the above it works as expected. When I add it I get the following error:
send: spawn id exp7 not open
while executing
usage: API [-h] [-s START] [-e END] [-d DAY] [-i INTERVAL]
[-a LOGGER] [-l LOGFILE] [-f FIELDS] [-q QUERY]
[-o OUTPUT] [-t FORMAT] [-u USER] [-p PASS] [-z]
API: error: unrecognized arguments: | gzip >> /data/sources/results-20180604-20180604.json.gz
expect
edited Jun 6 at 12:48
asked Jun 5 at 18:51
ximian
84
84
add a comment |Â
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
2
down vote
accepted
You might want to use a shell to interpret the pipe and redirection: I'll use a heredoc to make quoting easier
XYZ=$(expect <<END_EXPECT
set timeout -1
spawn sh -c python log-connector.py -s $strt -e $end -i 600 -a https://server:9000 -u someaccount -f ./conf/firewall-fields2.txt -q ./conf/query.txt -z
expect "Password:"
send "$passr"
expect eof
END_EXPECT
)
echo "$XYZ"
Well, I thought this did the trick and it sort of does, It runs the the python script once, gets the first chunk of a 10 min log, but does not continue getting the others. So its not really executing to completeness
â ximian
Jun 6 at 12:18
It should go until I get to the $end2 - variable (end date with 23:59), is there a way to check this in the expect ?
â ximian
Jun 6 at 12:48
If it takes longer than 10 seconds thenset timeout -1
at the top of the expect body
â glenn jackman
Jun 6 at 12:56
It can take 10 minutes like it could take a few hours for the logs to be parsed, so will the set timeout -1 handle this ? Also at the top of the expect body, you mean before XYZ=$.
â ximian
Jun 6 at 13:08
-1 means "do not time out". Put it before spawn
â glenn jackman
Jun 6 at 13:43
 |Â
show 1 more comment
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
You might want to use a shell to interpret the pipe and redirection: I'll use a heredoc to make quoting easier
XYZ=$(expect <<END_EXPECT
set timeout -1
spawn sh -c python log-connector.py -s $strt -e $end -i 600 -a https://server:9000 -u someaccount -f ./conf/firewall-fields2.txt -q ./conf/query.txt -z
expect "Password:"
send "$passr"
expect eof
END_EXPECT
)
echo "$XYZ"
Well, I thought this did the trick and it sort of does, It runs the the python script once, gets the first chunk of a 10 min log, but does not continue getting the others. So its not really executing to completeness
â ximian
Jun 6 at 12:18
It should go until I get to the $end2 - variable (end date with 23:59), is there a way to check this in the expect ?
â ximian
Jun 6 at 12:48
If it takes longer than 10 seconds thenset timeout -1
at the top of the expect body
â glenn jackman
Jun 6 at 12:56
It can take 10 minutes like it could take a few hours for the logs to be parsed, so will the set timeout -1 handle this ? Also at the top of the expect body, you mean before XYZ=$.
â ximian
Jun 6 at 13:08
-1 means "do not time out". Put it before spawn
â glenn jackman
Jun 6 at 13:43
 |Â
show 1 more comment
up vote
2
down vote
accepted
You might want to use a shell to interpret the pipe and redirection: I'll use a heredoc to make quoting easier
XYZ=$(expect <<END_EXPECT
set timeout -1
spawn sh -c python log-connector.py -s $strt -e $end -i 600 -a https://server:9000 -u someaccount -f ./conf/firewall-fields2.txt -q ./conf/query.txt -z
expect "Password:"
send "$passr"
expect eof
END_EXPECT
)
echo "$XYZ"
Well, I thought this did the trick and it sort of does, It runs the the python script once, gets the first chunk of a 10 min log, but does not continue getting the others. So its not really executing to completeness
â ximian
Jun 6 at 12:18
It should go until I get to the $end2 - variable (end date with 23:59), is there a way to check this in the expect ?
â ximian
Jun 6 at 12:48
If it takes longer than 10 seconds thenset timeout -1
at the top of the expect body
â glenn jackman
Jun 6 at 12:56
It can take 10 minutes like it could take a few hours for the logs to be parsed, so will the set timeout -1 handle this ? Also at the top of the expect body, you mean before XYZ=$.
â ximian
Jun 6 at 13:08
-1 means "do not time out". Put it before spawn
â glenn jackman
Jun 6 at 13:43
 |Â
show 1 more comment
up vote
2
down vote
accepted
up vote
2
down vote
accepted
You might want to use a shell to interpret the pipe and redirection: I'll use a heredoc to make quoting easier
XYZ=$(expect <<END_EXPECT
set timeout -1
spawn sh -c python log-connector.py -s $strt -e $end -i 600 -a https://server:9000 -u someaccount -f ./conf/firewall-fields2.txt -q ./conf/query.txt -z
expect "Password:"
send "$passr"
expect eof
END_EXPECT
)
echo "$XYZ"
You might want to use a shell to interpret the pipe and redirection: I'll use a heredoc to make quoting easier
XYZ=$(expect <<END_EXPECT
set timeout -1
spawn sh -c python log-connector.py -s $strt -e $end -i 600 -a https://server:9000 -u someaccount -f ./conf/firewall-fields2.txt -q ./conf/query.txt -z
expect "Password:"
send "$passr"
expect eof
END_EXPECT
)
echo "$XYZ"
edited Jun 6 at 21:21
answered Jun 5 at 19:04
glenn jackman
45.6k265100
45.6k265100
Well, I thought this did the trick and it sort of does, It runs the the python script once, gets the first chunk of a 10 min log, but does not continue getting the others. So its not really executing to completeness
â ximian
Jun 6 at 12:18
It should go until I get to the $end2 - variable (end date with 23:59), is there a way to check this in the expect ?
â ximian
Jun 6 at 12:48
If it takes longer than 10 seconds thenset timeout -1
at the top of the expect body
â glenn jackman
Jun 6 at 12:56
It can take 10 minutes like it could take a few hours for the logs to be parsed, so will the set timeout -1 handle this ? Also at the top of the expect body, you mean before XYZ=$.
â ximian
Jun 6 at 13:08
-1 means "do not time out". Put it before spawn
â glenn jackman
Jun 6 at 13:43
 |Â
show 1 more comment
Well, I thought this did the trick and it sort of does, It runs the the python script once, gets the first chunk of a 10 min log, but does not continue getting the others. So its not really executing to completeness
â ximian
Jun 6 at 12:18
It should go until I get to the $end2 - variable (end date with 23:59), is there a way to check this in the expect ?
â ximian
Jun 6 at 12:48
If it takes longer than 10 seconds thenset timeout -1
at the top of the expect body
â glenn jackman
Jun 6 at 12:56
It can take 10 minutes like it could take a few hours for the logs to be parsed, so will the set timeout -1 handle this ? Also at the top of the expect body, you mean before XYZ=$.
â ximian
Jun 6 at 13:08
-1 means "do not time out". Put it before spawn
â glenn jackman
Jun 6 at 13:43
Well, I thought this did the trick and it sort of does, It runs the the python script once, gets the first chunk of a 10 min log, but does not continue getting the others. So its not really executing to completeness
â ximian
Jun 6 at 12:18
Well, I thought this did the trick and it sort of does, It runs the the python script once, gets the first chunk of a 10 min log, but does not continue getting the others. So its not really executing to completeness
â ximian
Jun 6 at 12:18
It should go until I get to the $end2 - variable (end date with 23:59), is there a way to check this in the expect ?
â ximian
Jun 6 at 12:48
It should go until I get to the $end2 - variable (end date with 23:59), is there a way to check this in the expect ?
â ximian
Jun 6 at 12:48
If it takes longer than 10 seconds then
set timeout -1
at the top of the expect bodyâ glenn jackman
Jun 6 at 12:56
If it takes longer than 10 seconds then
set timeout -1
at the top of the expect bodyâ glenn jackman
Jun 6 at 12:56
It can take 10 minutes like it could take a few hours for the logs to be parsed, so will the set timeout -1 handle this ? Also at the top of the expect body, you mean before XYZ=$.
â ximian
Jun 6 at 13:08
It can take 10 minutes like it could take a few hours for the logs to be parsed, so will the set timeout -1 handle this ? Also at the top of the expect body, you mean before XYZ=$.
â ximian
Jun 6 at 13:08
-1 means "do not time out". Put it before spawn
â glenn jackman
Jun 6 at 13:43
-1 means "do not time out". Put it before spawn
â glenn jackman
Jun 6 at 13:43
 |Â
show 1 more comment
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f448053%2fexpect-command-pipes-and-gzip%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password