Extract part of the logs to other file

Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
1) We have a process.log file in which we have lot of text data and in between we have some XML data published.
2) There of thousands of different XML published in the logs along with other text data.
3) Now i need to select only the XML files which are published after Outgoing XML: value
4) Also the XML file which must be selected and copied to a new file should be the one which matches the value in the ALERTID tag.
5) The ALERTID value will be provided in the script input. So in our case mGMjhgHgffHhhFdH1u4 will be provided in the input and we need to select the full XML file published for this alertid. Starting tag is from <xml version..> and ending tag is </Alert>
5) So i need to select the relevant Outgoing XML file in a new file based on a particular ALERTID so it can be replayed in different environments.
Format of the log file is like below:
Info Jan 11 17:30:26.12122 The process is not responding to heartbeats
Debug Jan 11 17:30:26.12123 Incoming XML :<xml version "1.0" encoding ="UTF-8"?>
<Alert trigger = "true" >
<Alerttype>orderReject</Alerttype>
<AlertID>ghghfsjUtYuu78T1</AlertID>
<Order>uusingas</Order>
<Quantity>1254</Quanity>
</Alert> (CreateInitEventHandler. C:356)
Debug Jan 11 17:30:26.12199 The process is going down with warnings
Debug Jan 11 17:30:26.148199 Outgoing XML: <xml version "1.0" encoding ="UTF-8"?>
<Alert trigger = "true" >
<Alerttype>orderheld</Alerttype>
<AlertID>mGMjhgHgffHhhFdH1u4</AlertID>
<Order>uwiofhdf</Order>
<Quantity>7651</Quanity>
</Alert>(CreateEventHandler. C:723)
Debug Jan 11 17:30:26.13214 The process has restarted and thread opened
Debug Jan 11 17:30:26.13215 The heartbeat is recieved from alertlistener process
Now the requirement is to take AlertID in the input, scan the process log and extract the matching outgoing XML in a separate file.
Using awk i am able to extract all the outgoing xml files but not sure how to extract the one related to a particular AlertID.
Also i cannot install/use any new XML parser as per the company policy.
This needs to be achieved using shell/perl/awk/sed
Eg:
awk '/Outgoing/p=1; s=$0 P & & /</Alert>/ print $0 FS s; s="" ;p=0p' 1.log>2.log
awk sed perl xml
add a comment |Â
up vote
0
down vote
favorite
1) We have a process.log file in which we have lot of text data and in between we have some XML data published.
2) There of thousands of different XML published in the logs along with other text data.
3) Now i need to select only the XML files which are published after Outgoing XML: value
4) Also the XML file which must be selected and copied to a new file should be the one which matches the value in the ALERTID tag.
5) The ALERTID value will be provided in the script input. So in our case mGMjhgHgffHhhFdH1u4 will be provided in the input and we need to select the full XML file published for this alertid. Starting tag is from <xml version..> and ending tag is </Alert>
5) So i need to select the relevant Outgoing XML file in a new file based on a particular ALERTID so it can be replayed in different environments.
Format of the log file is like below:
Info Jan 11 17:30:26.12122 The process is not responding to heartbeats
Debug Jan 11 17:30:26.12123 Incoming XML :<xml version "1.0" encoding ="UTF-8"?>
<Alert trigger = "true" >
<Alerttype>orderReject</Alerttype>
<AlertID>ghghfsjUtYuu78T1</AlertID>
<Order>uusingas</Order>
<Quantity>1254</Quanity>
</Alert> (CreateInitEventHandler. C:356)
Debug Jan 11 17:30:26.12199 The process is going down with warnings
Debug Jan 11 17:30:26.148199 Outgoing XML: <xml version "1.0" encoding ="UTF-8"?>
<Alert trigger = "true" >
<Alerttype>orderheld</Alerttype>
<AlertID>mGMjhgHgffHhhFdH1u4</AlertID>
<Order>uwiofhdf</Order>
<Quantity>7651</Quanity>
</Alert>(CreateEventHandler. C:723)
Debug Jan 11 17:30:26.13214 The process has restarted and thread opened
Debug Jan 11 17:30:26.13215 The heartbeat is recieved from alertlistener process
Now the requirement is to take AlertID in the input, scan the process log and extract the matching outgoing XML in a separate file.
Using awk i am able to extract all the outgoing xml files but not sure how to extract the one related to a particular AlertID.
Also i cannot install/use any new XML parser as per the company policy.
This needs to be achieved using shell/perl/awk/sed
Eg:
awk '/Outgoing/p=1; s=$0 P & & /</Alert>/ print $0 FS s; s="" ;p=0p' 1.log>2.log
awk sed perl xml
1
there's noOutgoingword in your input - extend your input and post the expected result file contents
â RomanPerekhrest
Jan 15 at 16:34
1
wasted my time answering, this is a duplicate of unix.stackexchange.com/questions/416658/â¦
â Gerard H. Pille
Jan 15 at 17:06
Thanks Gerard for answering... Your script works and that's the answer I was looking for.. It wasn't answered in the other thread and was marked as duplicate of some other issue.. So posted again... But thanks a lot
â abhishek chaudhry
Jan 15 at 20:07
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
1) We have a process.log file in which we have lot of text data and in between we have some XML data published.
2) There of thousands of different XML published in the logs along with other text data.
3) Now i need to select only the XML files which are published after Outgoing XML: value
4) Also the XML file which must be selected and copied to a new file should be the one which matches the value in the ALERTID tag.
5) The ALERTID value will be provided in the script input. So in our case mGMjhgHgffHhhFdH1u4 will be provided in the input and we need to select the full XML file published for this alertid. Starting tag is from <xml version..> and ending tag is </Alert>
5) So i need to select the relevant Outgoing XML file in a new file based on a particular ALERTID so it can be replayed in different environments.
Format of the log file is like below:
Info Jan 11 17:30:26.12122 The process is not responding to heartbeats
Debug Jan 11 17:30:26.12123 Incoming XML :<xml version "1.0" encoding ="UTF-8"?>
<Alert trigger = "true" >
<Alerttype>orderReject</Alerttype>
<AlertID>ghghfsjUtYuu78T1</AlertID>
<Order>uusingas</Order>
<Quantity>1254</Quanity>
</Alert> (CreateInitEventHandler. C:356)
Debug Jan 11 17:30:26.12199 The process is going down with warnings
Debug Jan 11 17:30:26.148199 Outgoing XML: <xml version "1.0" encoding ="UTF-8"?>
<Alert trigger = "true" >
<Alerttype>orderheld</Alerttype>
<AlertID>mGMjhgHgffHhhFdH1u4</AlertID>
<Order>uwiofhdf</Order>
<Quantity>7651</Quanity>
</Alert>(CreateEventHandler. C:723)
Debug Jan 11 17:30:26.13214 The process has restarted and thread opened
Debug Jan 11 17:30:26.13215 The heartbeat is recieved from alertlistener process
Now the requirement is to take AlertID in the input, scan the process log and extract the matching outgoing XML in a separate file.
Using awk i am able to extract all the outgoing xml files but not sure how to extract the one related to a particular AlertID.
Also i cannot install/use any new XML parser as per the company policy.
This needs to be achieved using shell/perl/awk/sed
Eg:
awk '/Outgoing/p=1; s=$0 P & & /</Alert>/ print $0 FS s; s="" ;p=0p' 1.log>2.log
awk sed perl xml
1) We have a process.log file in which we have lot of text data and in between we have some XML data published.
2) There of thousands of different XML published in the logs along with other text data.
3) Now i need to select only the XML files which are published after Outgoing XML: value
4) Also the XML file which must be selected and copied to a new file should be the one which matches the value in the ALERTID tag.
5) The ALERTID value will be provided in the script input. So in our case mGMjhgHgffHhhFdH1u4 will be provided in the input and we need to select the full XML file published for this alertid. Starting tag is from <xml version..> and ending tag is </Alert>
5) So i need to select the relevant Outgoing XML file in a new file based on a particular ALERTID so it can be replayed in different environments.
Format of the log file is like below:
Info Jan 11 17:30:26.12122 The process is not responding to heartbeats
Debug Jan 11 17:30:26.12123 Incoming XML :<xml version "1.0" encoding ="UTF-8"?>
<Alert trigger = "true" >
<Alerttype>orderReject</Alerttype>
<AlertID>ghghfsjUtYuu78T1</AlertID>
<Order>uusingas</Order>
<Quantity>1254</Quanity>
</Alert> (CreateInitEventHandler. C:356)
Debug Jan 11 17:30:26.12199 The process is going down with warnings
Debug Jan 11 17:30:26.148199 Outgoing XML: <xml version "1.0" encoding ="UTF-8"?>
<Alert trigger = "true" >
<Alerttype>orderheld</Alerttype>
<AlertID>mGMjhgHgffHhhFdH1u4</AlertID>
<Order>uwiofhdf</Order>
<Quantity>7651</Quanity>
</Alert>(CreateEventHandler. C:723)
Debug Jan 11 17:30:26.13214 The process has restarted and thread opened
Debug Jan 11 17:30:26.13215 The heartbeat is recieved from alertlistener process
Now the requirement is to take AlertID in the input, scan the process log and extract the matching outgoing XML in a separate file.
Using awk i am able to extract all the outgoing xml files but not sure how to extract the one related to a particular AlertID.
Also i cannot install/use any new XML parser as per the company policy.
This needs to be achieved using shell/perl/awk/sed
Eg:
awk '/Outgoing/p=1; s=$0 P & & /</Alert>/ print $0 FS s; s="" ;p=0p' 1.log>2.log
awk sed perl xml
edited Jan 15 at 16:59
Philippos
5,90211545
5,90211545
asked Jan 15 at 16:01
abhishek chaudhry
35
35
1
there's noOutgoingword in your input - extend your input and post the expected result file contents
â RomanPerekhrest
Jan 15 at 16:34
1
wasted my time answering, this is a duplicate of unix.stackexchange.com/questions/416658/â¦
â Gerard H. Pille
Jan 15 at 17:06
Thanks Gerard for answering... Your script works and that's the answer I was looking for.. It wasn't answered in the other thread and was marked as duplicate of some other issue.. So posted again... But thanks a lot
â abhishek chaudhry
Jan 15 at 20:07
add a comment |Â
1
there's noOutgoingword in your input - extend your input and post the expected result file contents
â RomanPerekhrest
Jan 15 at 16:34
1
wasted my time answering, this is a duplicate of unix.stackexchange.com/questions/416658/â¦
â Gerard H. Pille
Jan 15 at 17:06
Thanks Gerard for answering... Your script works and that's the answer I was looking for.. It wasn't answered in the other thread and was marked as duplicate of some other issue.. So posted again... But thanks a lot
â abhishek chaudhry
Jan 15 at 20:07
1
1
there's no
Outgoing word in your input - extend your input and post the expected result file contentsâ RomanPerekhrest
Jan 15 at 16:34
there's no
Outgoing word in your input - extend your input and post the expected result file contentsâ RomanPerekhrest
Jan 15 at 16:34
1
1
wasted my time answering, this is a duplicate of unix.stackexchange.com/questions/416658/â¦
â Gerard H. Pille
Jan 15 at 17:06
wasted my time answering, this is a duplicate of unix.stackexchange.com/questions/416658/â¦
â Gerard H. Pille
Jan 15 at 17:06
Thanks Gerard for answering... Your script works and that's the answer I was looking for.. It wasn't answered in the other thread and was marked as duplicate of some other issue.. So posted again... But thanks a lot
â abhishek chaudhry
Jan 15 at 20:07
Thanks Gerard for answering... Your script works and that's the answer I was looking for.. It wasn't answered in the other thread and was marked as duplicate of some other issue.. So posted again... But thanks a lot
â abhishek chaudhry
Jan 15 at 20:07
add a comment |Â
2 Answers
2
active
oldest
votes
up vote
0
down vote
accepted
Supposing your ID is given in a variable called ALERTID:
sed -e '/Outgoing XML/!d;:a' -e '$d;N;s/.*(<xml version.*</Alert>).*/1/;Ta' -e "/$ALERTID/!d" yourfile.log
Explanation:
/Outgoing XML/!d;:adelete stuff until theOutgoing XMLline and start a loop then$dto delete an unfinished record at the end of the fileN;s/.*(<xml version.*</Alert>).*/1/;Taappends lines until the</Alert>tag is found and removes everything before and after the desired block"/$ALERTID/!ddeletes blocks without the$ALERTID`
Maybe better to read:
sed '/Outgoing XML/!d;:a
$d;N
s/.*(<xml version.*</Alert>).*/1/;Ta
/'$ALERTID'/!d' yourfile.log
Thanks a lot Phillippos... This solution works really well for me.. Have a great day ahead
â abhishek chaudhry
Jan 15 at 20:10
add a comment |Â
up vote
0
down vote
create a shell script, getalert.sh, with following contents:
awk '
/^Debug .* Outgoing XML/
sub(/^.* Outgoing XML: /,"")
H=$0
LC=0
next
/</Alert>/
sub(/Alert>.*$/,"Alert>")
if (LC>0) print
LC=0
next
/<AlertID>'$1'</AlertID>/
print H
print
LC=1
next
/<AlertID>.*</AlertID>/
H=""
LC=0
next
if (LC > 0)
print
else
H = H $0
' $2
run it as
getalert.sh mGMjhgHgffHhhFdH1u4 process.log
add a comment |Â
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
accepted
Supposing your ID is given in a variable called ALERTID:
sed -e '/Outgoing XML/!d;:a' -e '$d;N;s/.*(<xml version.*</Alert>).*/1/;Ta' -e "/$ALERTID/!d" yourfile.log
Explanation:
/Outgoing XML/!d;:adelete stuff until theOutgoing XMLline and start a loop then$dto delete an unfinished record at the end of the fileN;s/.*(<xml version.*</Alert>).*/1/;Taappends lines until the</Alert>tag is found and removes everything before and after the desired block"/$ALERTID/!ddeletes blocks without the$ALERTID`
Maybe better to read:
sed '/Outgoing XML/!d;:a
$d;N
s/.*(<xml version.*</Alert>).*/1/;Ta
/'$ALERTID'/!d' yourfile.log
Thanks a lot Phillippos... This solution works really well for me.. Have a great day ahead
â abhishek chaudhry
Jan 15 at 20:10
add a comment |Â
up vote
0
down vote
accepted
Supposing your ID is given in a variable called ALERTID:
sed -e '/Outgoing XML/!d;:a' -e '$d;N;s/.*(<xml version.*</Alert>).*/1/;Ta' -e "/$ALERTID/!d" yourfile.log
Explanation:
/Outgoing XML/!d;:adelete stuff until theOutgoing XMLline and start a loop then$dto delete an unfinished record at the end of the fileN;s/.*(<xml version.*</Alert>).*/1/;Taappends lines until the</Alert>tag is found and removes everything before and after the desired block"/$ALERTID/!ddeletes blocks without the$ALERTID`
Maybe better to read:
sed '/Outgoing XML/!d;:a
$d;N
s/.*(<xml version.*</Alert>).*/1/;Ta
/'$ALERTID'/!d' yourfile.log
Thanks a lot Phillippos... This solution works really well for me.. Have a great day ahead
â abhishek chaudhry
Jan 15 at 20:10
add a comment |Â
up vote
0
down vote
accepted
up vote
0
down vote
accepted
Supposing your ID is given in a variable called ALERTID:
sed -e '/Outgoing XML/!d;:a' -e '$d;N;s/.*(<xml version.*</Alert>).*/1/;Ta' -e "/$ALERTID/!d" yourfile.log
Explanation:
/Outgoing XML/!d;:adelete stuff until theOutgoing XMLline and start a loop then$dto delete an unfinished record at the end of the fileN;s/.*(<xml version.*</Alert>).*/1/;Taappends lines until the</Alert>tag is found and removes everything before and after the desired block"/$ALERTID/!ddeletes blocks without the$ALERTID`
Maybe better to read:
sed '/Outgoing XML/!d;:a
$d;N
s/.*(<xml version.*</Alert>).*/1/;Ta
/'$ALERTID'/!d' yourfile.log
Supposing your ID is given in a variable called ALERTID:
sed -e '/Outgoing XML/!d;:a' -e '$d;N;s/.*(<xml version.*</Alert>).*/1/;Ta' -e "/$ALERTID/!d" yourfile.log
Explanation:
/Outgoing XML/!d;:adelete stuff until theOutgoing XMLline and start a loop then$dto delete an unfinished record at the end of the fileN;s/.*(<xml version.*</Alert>).*/1/;Taappends lines until the</Alert>tag is found and removes everything before and after the desired block"/$ALERTID/!ddeletes blocks without the$ALERTID`
Maybe better to read:
sed '/Outgoing XML/!d;:a
$d;N
s/.*(<xml version.*</Alert>).*/1/;Ta
/'$ALERTID'/!d' yourfile.log
edited Jan 15 at 17:20
answered Jan 15 at 17:13
Philippos
5,90211545
5,90211545
Thanks a lot Phillippos... This solution works really well for me.. Have a great day ahead
â abhishek chaudhry
Jan 15 at 20:10
add a comment |Â
Thanks a lot Phillippos... This solution works really well for me.. Have a great day ahead
â abhishek chaudhry
Jan 15 at 20:10
Thanks a lot Phillippos... This solution works really well for me.. Have a great day ahead
â abhishek chaudhry
Jan 15 at 20:10
Thanks a lot Phillippos... This solution works really well for me.. Have a great day ahead
â abhishek chaudhry
Jan 15 at 20:10
add a comment |Â
up vote
0
down vote
create a shell script, getalert.sh, with following contents:
awk '
/^Debug .* Outgoing XML/
sub(/^.* Outgoing XML: /,"")
H=$0
LC=0
next
/</Alert>/
sub(/Alert>.*$/,"Alert>")
if (LC>0) print
LC=0
next
/<AlertID>'$1'</AlertID>/
print H
print
LC=1
next
/<AlertID>.*</AlertID>/
H=""
LC=0
next
if (LC > 0)
print
else
H = H $0
' $2
run it as
getalert.sh mGMjhgHgffHhhFdH1u4 process.log
add a comment |Â
up vote
0
down vote
create a shell script, getalert.sh, with following contents:
awk '
/^Debug .* Outgoing XML/
sub(/^.* Outgoing XML: /,"")
H=$0
LC=0
next
/</Alert>/
sub(/Alert>.*$/,"Alert>")
if (LC>0) print
LC=0
next
/<AlertID>'$1'</AlertID>/
print H
print
LC=1
next
/<AlertID>.*</AlertID>/
H=""
LC=0
next
if (LC > 0)
print
else
H = H $0
' $2
run it as
getalert.sh mGMjhgHgffHhhFdH1u4 process.log
add a comment |Â
up vote
0
down vote
up vote
0
down vote
create a shell script, getalert.sh, with following contents:
awk '
/^Debug .* Outgoing XML/
sub(/^.* Outgoing XML: /,"")
H=$0
LC=0
next
/</Alert>/
sub(/Alert>.*$/,"Alert>")
if (LC>0) print
LC=0
next
/<AlertID>'$1'</AlertID>/
print H
print
LC=1
next
/<AlertID>.*</AlertID>/
H=""
LC=0
next
if (LC > 0)
print
else
H = H $0
' $2
run it as
getalert.sh mGMjhgHgffHhhFdH1u4 process.log
create a shell script, getalert.sh, with following contents:
awk '
/^Debug .* Outgoing XML/
sub(/^.* Outgoing XML: /,"")
H=$0
LC=0
next
/</Alert>/
sub(/Alert>.*$/,"Alert>")
if (LC>0) print
LC=0
next
/<AlertID>'$1'</AlertID>/
print H
print
LC=1
next
/<AlertID>.*</AlertID>/
H=""
LC=0
next
if (LC > 0)
print
else
H = H $0
' $2
run it as
getalert.sh mGMjhgHgffHhhFdH1u4 process.log
answered Jan 15 at 16:51
Gerard H. Pille
1,219212
1,219212
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f417277%2fextract-part-of-the-logs-to-other-file%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
1
there's no
Outgoingword in your input - extend your input and post the expected result file contentsâ RomanPerekhrest
Jan 15 at 16:34
1
wasted my time answering, this is a duplicate of unix.stackexchange.com/questions/416658/â¦
â Gerard H. Pille
Jan 15 at 17:06
Thanks Gerard for answering... Your script works and that's the answer I was looking for.. It wasn't answered in the other thread and was marked as duplicate of some other issue.. So posted again... But thanks a lot
â abhishek chaudhry
Jan 15 at 20:07