Redirect output to specific column in file
Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
Daily I am running approx 100 queries in a script that each produce a single value output (a number). I'm currently outputting each query's value to multiple files, appending each days' output to the bottom of the query's respective file:
query1 goes to file1
date value
Jan-01 35
Jan-02 74
query2 goes to file2
date value
Jan-01 834
Jan-02 342
etc
What I would like is to build a single file that contains a column for each query and append the results accordingly:
Date Query1 Query2 etc
Jan01 35 834 etc
Jan02 74 342 etc
My current query:
query `yesterday` direct | awk '$1=="FA"count++ENDprint count' > /x/2018/files/"$file1"
query `yesterday` direct | awk '$1=="FM"count++ENDprint count' > /x/2018/files/"$file2"
query `yesterday` direct | awk '$1=="FT"count++ENDprint count' > /x/2018/files/"$file3"
bash awk
asked Apr 3 at 13:48
Steve
103
add a comment |Â
up vote
0
down vote
favorite
Daily I am running approx 100 queries in a script that each produce a single value output (a number). I'm currently outputting each query's value to multiple files, appending each days' output to the bottom of the query's respective file:
query1 goes to file1
date value
Jan-01 35
Jan-02 74
query2 goes to file2
date value
Jan-01 834
Jan-02 342
etc
What I would like is to build a single file that contains a column for each query and append the results accordingly:
Date Query1 Query2 etc
Jan01 35 834 etc
Jan02 74 342 etc
My current query:
query `yesterday` direct | awk '$1=="FA"count++ENDprint count' > /x/2018/files/"$file1"
query `yesterday` direct | awk '$1=="FM"count++ENDprint count' > /x/2018/files/"$file2"
query `yesterday` direct | awk '$1=="FT"count++ENDprint count' > /x/2018/files/"$file3"
bash awk
asked Apr 3 at 13:48
Steve
103
In each input file, is the first column containing only unique dates? Also, is the date column identical to all input files? I assume yes to both, but please confirm.
– seshoumara
Apr 4 at 3:23
Actually the query output only returns a count. So in query 1 I'd like it's value to go to column 2 of the 'master' file (column 1 would be the date, which I'll get elsewhere). Query 2 goes to column 3, query 3 to column 4, etc. I was really hoping there is a simple way to direct the output at the end of each query:
– Steve
Apr 4 at 13:06
I see. I thought you needed to merge all output files into the master file. Hmm. Can you collect all query returned values into an array and then append at the end of the master file?
– seshoumara
Apr 4 at 13:26
Or simply(queryDate; query1; query2; query3 ...) | xargs >> master_fileperhaps?
– seshoumara
Apr 4 at 13:39
As you can probably tell, I'm a rookie. I don't think I can bundle my queries then output them but instead I have to run them individually, like in my example. So I'd like this query1 > masterfile.txt...field 2, query2 > masterfile.txt..field3, etc
– Steve
Apr 4 at 14:06
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
Daily I am running approx 100 queries in a script that each produce a single value output (a number). I'm currently outputting each query's value to multiple files, appending each days' output to the bottom of the query's respective file:
query1 goes to file1
date value
Jan-01 35
Jan-02 74
query2 goes to file2
date value
Jan-01 834
Jan-02 342
etc
What I would like is to build a single file that contains a column for each query and append the results accordingly:
Date Query1 Query2 etc
Jan01 35 834 etc
Jan02 74 342 etc
My current query:
query `yesterday` direct | awk '$1=="FA"count++ENDprint count' > /x/2018/files/"$file1"
query `yesterday` direct | awk '$1=="FM"count++ENDprint count' > /x/2018/files/"$file2"
query `yesterday` direct | awk '$1=="FT"count++ENDprint count' > /x/2018/files/"$file3"
bash awk
asked Apr 3 at 13:48
Steve
103
Daily I am running approx 100 queries in a script that each produce a single value output (a number). I'm currently outputting each query's value to multiple files, appending each days' output to the bottom of the query's respective file:
query1 goes to file1
date value
Jan-01 35
Jan-02 74
query2 goes to file2
date value
Jan-01 834
Jan-02 342
etc
What I would like is to build a single file that contains a column for each query and append the results accordingly:
Date Query1 Query2 etc
Jan01 35 834 etc
Jan02 74 342 etc
My current query:
query `yesterday` direct | awk '$1=="FA"count++ENDprint count' > /x/2018/files/"$file1"
query `yesterday` direct | awk '$1=="FM"count++ENDprint count' > /x/2018/files/"$file2"
query `yesterday` direct | awk '$1=="FT"count++ENDprint count' > /x/2018/files/"$file3"
bash awk
asked Apr 3 at 13:48
Steve
103
edited Apr 3 at 21:16
asked Apr 3 at 13:48
Steve
103
asked Apr 3 at 13:48
Steve
103
asked Apr 3 at 13:48
Steve
103
103
In each input file, is the first column containing only unique dates? Also, is the date column identical to all input files? I assume yes to both, but please confirm.
– seshoumara
Apr 4 at 3:23
Actually the query output only returns a count. So in query 1 I'd like it's value to go to column 2 of the 'master' file (column 1 would be the date, which I'll get elsewhere). Query 2 goes to column 3, query 3 to column 4, etc. I was really hoping there is a simple way to direct the output at the end of each query:
– Steve
Apr 4 at 13:06
I see. I thought you needed to merge all output files into the master file. Hmm. Can you collect all query returned values into an array and then append at the end of the master file?
– seshoumara
Apr 4 at 13:26
Or simply(queryDate; query1; query2; query3 ...) | xargs >> master_fileperhaps?
– seshoumara
Apr 4 at 13:39
As you can probably tell, I'm a rookie. I don't think I can bundle my queries then output them but instead I have to run them individually, like in my example. So I'd like this query1 > masterfile.txt...field 2, query2 > masterfile.txt..field3, etc
– Steve
Apr 4 at 14:06
add a comment |Â
In each input file, is the first column containing only unique dates? Also, is the date column identical to all input files? I assume yes to both, but please confirm.
– seshoumara
Apr 4 at 3:23
Actually the query output only returns a count. So in query 1 I'd like it's value to go to column 2 of the 'master' file (column 1 would be the date, which I'll get elsewhere). Query 2 goes to column 3, query 3 to column 4, etc. I was really hoping there is a simple way to direct the output at the end of each query:
– Steve
Apr 4 at 13:06
I see. I thought you needed to merge all output files into the master file. Hmm. Can you collect all query returned values into an array and then append at the end of the master file?
– seshoumara
Apr 4 at 13:26
Or simply(queryDate; query1; query2; query3 ...) | xargs >> master_fileperhaps?
– seshoumara
Apr 4 at 13:39
As you can probably tell, I'm a rookie. I don't think I can bundle my queries then output them but instead I have to run them individually, like in my example. So I'd like this query1 > masterfile.txt...field 2, query2 > masterfile.txt..field3, etc
– Steve
Apr 4 at 14:06
In each input file, is the first column containing only unique dates? Also, is the date column identical to all input files? I assume yes to both, but please confirm.
– seshoumara
Apr 4 at 3:23
In each input file, is the first column containing only unique dates? Also, is the date column identical to all input files? I assume yes to both, but please confirm.
– seshoumara
Apr 4 at 3:23
Actually the query output only returns a count. So in query 1 I'd like it's value to go to column 2 of the 'master' file (column 1 would be the date, which I'll get elsewhere). Query 2 goes to column 3, query 3 to column 4, etc. I was really hoping there is a simple way to direct the output at the end of each query:
– Steve
Apr 4 at 13:06
Actually the query output only returns a count. So in query 1 I'd like it's value to go to column 2 of the 'master' file (column 1 would be the date, which I'll get elsewhere). Query 2 goes to column 3, query 3 to column 4, etc. I was really hoping there is a simple way to direct the output at the end of each query:
– Steve
Apr 4 at 13:06
I see. I thought you needed to merge all output files into the master file. Hmm. Can you collect all query returned values into an array and then append at the end of the master file?
– seshoumara
Apr 4 at 13:26
I see. I thought you needed to merge all output files into the master file. Hmm. Can you collect all query returned values into an array and then append at the end of the master file?
– seshoumara
Apr 4 at 13:26
Or simply
(queryDate; query1; query2; query3 ...) | xargs >> master_file perhaps?– seshoumara
Apr 4 at 13:39
Or simply
(queryDate; query1; query2; query3 ...) | xargs >> master_file perhaps?– seshoumara
Apr 4 at 13:39
As you can probably tell, I'm a rookie. I don't think I can bundle my queries then output them but instead I have to run them individually, like in my example. So I'd like this query1 > masterfile.txt...field 2, query2 > masterfile.txt..field3, etc
– Steve
Apr 4 at 14:06
As you can probably tell, I'm a rookie. I don't think I can bundle my queries then output them but instead I have to run them individually, like in my example. So I'd like this query1 > masterfile.txt...field 2, query2 > masterfile.txt..field3, etc
– Steve
Apr 4 at 14:06
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
0
down vote
Assuming the OP confirms the behavior I mentioned in the comments, this script should work:
awk '
FNR != 1 sub("-", "", $1); dates[$1] = dates[$1] FS $2
END
for(i in ARGV) h = h FS "Query"i; sub(" Query0", "Date", h); print h
for(d in dates) print d dates[d]
' $(cat "list.txt")
I thought it would be easier to list all the input file names into a separate file "list.txt", one file per line.
answered Apr 4 at 3:59
seshoumara
25127
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
Assuming the OP confirms the behavior I mentioned in the comments, this script should work:
awk '
FNR != 1 sub("-", "", $1); dates[$1] = dates[$1] FS $2
END
for(i in ARGV) h = h FS "Query"i; sub(" Query0", "Date", h); print h
for(d in dates) print d dates[d]
' $(cat "list.txt")
I thought it would be easier to list all the input file names into a separate file "list.txt", one file per line.
answered Apr 4 at 3:59
seshoumara
25127
add a comment |Â
up vote
0
down vote
Assuming the OP confirms the behavior I mentioned in the comments, this script should work:
awk '
FNR != 1 sub("-", "", $1); dates[$1] = dates[$1] FS $2
END
for(i in ARGV) h = h FS "Query"i; sub(" Query0", "Date", h); print h
for(d in dates) print d dates[d]
' $(cat "list.txt")
I thought it would be easier to list all the input file names into a separate file "list.txt", one file per line.
answered Apr 4 at 3:59
seshoumara
25127
add a comment |Â
up vote
0
down vote
up vote
0
down vote
Assuming the OP confirms the behavior I mentioned in the comments, this script should work:
awk '
FNR != 1 sub("-", "", $1); dates[$1] = dates[$1] FS $2
END
for(i in ARGV) h = h FS "Query"i; sub(" Query0", "Date", h); print h
for(d in dates) print d dates[d]
' $(cat "list.txt")
I thought it would be easier to list all the input file names into a separate file "list.txt", one file per line.
answered Apr 4 at 3:59
seshoumara
25127
Assuming the OP confirms the behavior I mentioned in the comments, this script should work:
awk '
FNR != 1 sub("-", "", $1); dates[$1] = dates[$1] FS $2
END
for(i in ARGV) h = h FS "Query"i; sub(" Query0", "Date", h); print h
for(d in dates) print d dates[d]
' $(cat "list.txt")
I thought it would be easier to list all the input file names into a separate file "list.txt", one file per line.
answered Apr 4 at 3:59
seshoumara
25127
answered Apr 4 at 3:59
seshoumara
25127
answered Apr 4 at 3:59
seshoumara
25127
answered Apr 4 at 3:59
seshoumara
25127
25127
add a comment |Â
add a comment |Â
Â
draft saved
draft discarded
Â
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f435283%2fredirect-output-to-specific-column-in-file%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
In each input file, is the first column containing only unique dates? Also, is the date column identical to all input files? I assume yes to both, but please confirm.
– seshoumara
Apr 4 at 3:23
Actually the query output only returns a count. So in query 1 I'd like it's value to go to column 2 of the 'master' file (column 1 would be the date, which I'll get elsewhere). Query 2 goes to column 3, query 3 to column 4, etc. I was really hoping there is a simple way to direct the output at the end of each query:
– Steve
Apr 4 at 13:06
I see. I thought you needed to merge all output files into the master file. Hmm. Can you collect all query returned values into an array and then append at the end of the master file?
– seshoumara
Apr 4 at 13:26
Or simply
(queryDate; query1; query2; query3 ...) | xargs >> master_fileperhaps?– seshoumara
Apr 4 at 13:39
As you can probably tell, I'm a rookie. I don't think I can bundle my queries then output them but instead I have to run them individually, like in my example. So I'd like this query1 > masterfile.txt...field 2, query2 > masterfile.txt..field3, etc
– Steve
Apr 4 at 14:06