rsyslog: send logs to different file by filtering syslogs
Clash Royale CLAN TAG#URR8PPP
0
I have a CentOS based rsyslog server which have the below settings, where it's getting all the remote NATing events from multiple hosts.
$template TmplcpFW, "/var/log/%HOSTNAME%.log"
if ($hostname == ["CP1CGNAT","CP2CGNAT"]) then ?TmplcpFW
& ~
Sample log output in /var/log/CP1CGNAT
<150>1 2018-12-20T16:07:00.482369+01:00 2018-12-20 15 - - - 06:59: CP1CGNAT_O_PolandOR_NAT[jservices-nat]: JSERVICES_NAT_RULE_MATCH: proto 6 (TCP) application: any, xe-1/0/1.1718:100.70.0.2:59794 -> 109.32.8.15:80, Match NAT rule-set: (null), rule: OR_NAT_Poland, term: t1
<150>1 2018-12-20T16:07:00.482369+01:00 2018-12-20 15 - - - 06:59: CP1CGNAT_O_FranceOR_NATJSERVICES_SESSION_OPEN: application:none, xe-1/0/1.1718 100.70.0.2:59794 [55.93.69.53:26620] -> 109.32.8.15:80 (TCP)
<150>1 2018-12-20T16:07:11.091313+01:00 2018-12-20 15 - - - 07:10: CP1CGNAT_O_UKOR_NATJSERVICES_SESSION_CLOSE: application:none, xe-1/0/1.1718 100.70.0.2:59778 [55.93.69.60:40136] -> 109.32.8.15:80 (TCP)
I have configured NAT device service with prefix as 'syslog host 10.10.10.10 log-prefix CP1CGNAT_O_Poland'
Now I would like to separate above logs into different directories/files based on prefix values with timestamps like -
/Poland/Logs_Poland_2018-12-20, /France/Logs_France_2018-12-20, /UK/Logs_UK_2018-12-20
Can you suggest how to achieve this in rsyslog ?
Update taken from comments:
I have used the different log-prefixs (configured in NAT device) as filter conditions and used in templates as (total 10 similar templates):
$template TmplcpFW_P, "/var/log/NIPFW/Poland/Poland_%HOSTNAME%_%$year%.%$month%.%$day%.log"
if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_Poland' then ?TmplcpFW_P
$template TmplcpFW_F, "/var/log/NIPFW/France/France_%HOSTNAME%_%$year%.%$month%.%$day%.log"
if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_France' then ?TmplcpFW_F
Now I want all the matching logs from above conditions to go into corresponding directories/files and remaining logs in /var/log/messages.
logs rsyslog syslog
edited Jan 13 at 16:31
meuh
31.9k11954
asked Jan 10 at 9:55
SarjitSSarjitS
13
add a comment |
0
I have a CentOS based rsyslog server which have the below settings, where it's getting all the remote NATing events from multiple hosts.
$template TmplcpFW, "/var/log/%HOSTNAME%.log"
if ($hostname == ["CP1CGNAT","CP2CGNAT"]) then ?TmplcpFW
& ~
Sample log output in /var/log/CP1CGNAT
<150>1 2018-12-20T16:07:00.482369+01:00 2018-12-20 15 - - - 06:59: CP1CGNAT_O_PolandOR_NAT[jservices-nat]: JSERVICES_NAT_RULE_MATCH: proto 6 (TCP) application: any, xe-1/0/1.1718:100.70.0.2:59794 -> 109.32.8.15:80, Match NAT rule-set: (null), rule: OR_NAT_Poland, term: t1
<150>1 2018-12-20T16:07:00.482369+01:00 2018-12-20 15 - - - 06:59: CP1CGNAT_O_FranceOR_NATJSERVICES_SESSION_OPEN: application:none, xe-1/0/1.1718 100.70.0.2:59794 [55.93.69.53:26620] -> 109.32.8.15:80 (TCP)
<150>1 2018-12-20T16:07:11.091313+01:00 2018-12-20 15 - - - 07:10: CP1CGNAT_O_UKOR_NATJSERVICES_SESSION_CLOSE: application:none, xe-1/0/1.1718 100.70.0.2:59778 [55.93.69.60:40136] -> 109.32.8.15:80 (TCP)
I have configured NAT device service with prefix as 'syslog host 10.10.10.10 log-prefix CP1CGNAT_O_Poland'
Now I would like to separate above logs into different directories/files based on prefix values with timestamps like -
/Poland/Logs_Poland_2018-12-20, /France/Logs_France_2018-12-20, /UK/Logs_UK_2018-12-20
Can you suggest how to achieve this in rsyslog ?
Update taken from comments:
I have used the different log-prefixs (configured in NAT device) as filter conditions and used in templates as (total 10 similar templates):
$template TmplcpFW_P, "/var/log/NIPFW/Poland/Poland_%HOSTNAME%_%$year%.%$month%.%$day%.log"
if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_Poland' then ?TmplcpFW_P
$template TmplcpFW_F, "/var/log/NIPFW/France/France_%HOSTNAME%_%$year%.%$month%.%$day%.log"
if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_France' then ?TmplcpFW_F
Now I want all the matching logs from above conditions to go into corresponding directories/files and remaining logs in /var/log/messages.
logs rsyslog syslog
edited Jan 13 at 16:31
meuh
31.9k11954
asked Jan 10 at 9:55
SarjitSSarjitS
13
You can extract any part of a message if you can write a regex pattern for it. see here
– meuh
Jan 10 at 18:57
Thanks @meuh, I have used the different log-prefixs (configured in NAT device) as filter conditions and used in templates as (total 10 similar templates) - $template TmplcpFW_P, "/var/log/NIPFW/Poland/Poland_%HOSTNAME%_%$year%.%$month%.%$day%.log" if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_Poland' then ?TmplcpFW_P $template TmplcpFW_F, "/var/log/NIPFW/France/France_%HOSTNAME%_%$year%.%$month%.%$day%.log" if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_France' then ?TmplcpFW_F
– SarjitS
Jan 11 at 6:05
Now I want all the matching logs from above conditions to go into corresponding directories/files and remaining logs in /var/log/messages. Please suggest how to do this. Other suggestions/recommendation are most welcome.
– SarjitS
Jan 11 at 6:09
I have copied your comments into your question, so they are easier to read (you can edit your question at any time, and just refer to the change in a short comment). If I understand you, you now have 10 templates and 10if ...statements that do exactly what you want. The only thing I see missing is the line& ~after eachif ...statement. That would stop the matched message from any further handling. Did you want something more than that?
– meuh
Jan 13 at 16:36
add a comment |
0
0
0
I have a CentOS based rsyslog server which have the below settings, where it's getting all the remote NATing events from multiple hosts.
$template TmplcpFW, "/var/log/%HOSTNAME%.log"
if ($hostname == ["CP1CGNAT","CP2CGNAT"]) then ?TmplcpFW
& ~
Sample log output in /var/log/CP1CGNAT
<150>1 2018-12-20T16:07:00.482369+01:00 2018-12-20 15 - - - 06:59: CP1CGNAT_O_PolandOR_NAT[jservices-nat]: JSERVICES_NAT_RULE_MATCH: proto 6 (TCP) application: any, xe-1/0/1.1718:100.70.0.2:59794 -> 109.32.8.15:80, Match NAT rule-set: (null), rule: OR_NAT_Poland, term: t1
<150>1 2018-12-20T16:07:00.482369+01:00 2018-12-20 15 - - - 06:59: CP1CGNAT_O_FranceOR_NATJSERVICES_SESSION_OPEN: application:none, xe-1/0/1.1718 100.70.0.2:59794 [55.93.69.53:26620] -> 109.32.8.15:80 (TCP)
<150>1 2018-12-20T16:07:11.091313+01:00 2018-12-20 15 - - - 07:10: CP1CGNAT_O_UKOR_NATJSERVICES_SESSION_CLOSE: application:none, xe-1/0/1.1718 100.70.0.2:59778 [55.93.69.60:40136] -> 109.32.8.15:80 (TCP)
I have configured NAT device service with prefix as 'syslog host 10.10.10.10 log-prefix CP1CGNAT_O_Poland'
Now I would like to separate above logs into different directories/files based on prefix values with timestamps like -
/Poland/Logs_Poland_2018-12-20, /France/Logs_France_2018-12-20, /UK/Logs_UK_2018-12-20
Can you suggest how to achieve this in rsyslog ?
Update taken from comments:
I have used the different log-prefixs (configured in NAT device) as filter conditions and used in templates as (total 10 similar templates):
$template TmplcpFW_P, "/var/log/NIPFW/Poland/Poland_%HOSTNAME%_%$year%.%$month%.%$day%.log"
if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_Poland' then ?TmplcpFW_P
$template TmplcpFW_F, "/var/log/NIPFW/France/France_%HOSTNAME%_%$year%.%$month%.%$day%.log"
if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_France' then ?TmplcpFW_F
Now I want all the matching logs from above conditions to go into corresponding directories/files and remaining logs in /var/log/messages.
logs rsyslog syslog
edited Jan 13 at 16:31
meuh
31.9k11954
asked Jan 10 at 9:55
SarjitSSarjitS
13
I have a CentOS based rsyslog server which have the below settings, where it's getting all the remote NATing events from multiple hosts.
$template TmplcpFW, "/var/log/%HOSTNAME%.log"
if ($hostname == ["CP1CGNAT","CP2CGNAT"]) then ?TmplcpFW
& ~
Sample log output in /var/log/CP1CGNAT
<150>1 2018-12-20T16:07:00.482369+01:00 2018-12-20 15 - - - 06:59: CP1CGNAT_O_PolandOR_NAT[jservices-nat]: JSERVICES_NAT_RULE_MATCH: proto 6 (TCP) application: any, xe-1/0/1.1718:100.70.0.2:59794 -> 109.32.8.15:80, Match NAT rule-set: (null), rule: OR_NAT_Poland, term: t1
<150>1 2018-12-20T16:07:00.482369+01:00 2018-12-20 15 - - - 06:59: CP1CGNAT_O_FranceOR_NATJSERVICES_SESSION_OPEN: application:none, xe-1/0/1.1718 100.70.0.2:59794 [55.93.69.53:26620] -> 109.32.8.15:80 (TCP)
<150>1 2018-12-20T16:07:11.091313+01:00 2018-12-20 15 - - - 07:10: CP1CGNAT_O_UKOR_NATJSERVICES_SESSION_CLOSE: application:none, xe-1/0/1.1718 100.70.0.2:59778 [55.93.69.60:40136] -> 109.32.8.15:80 (TCP)
I have configured NAT device service with prefix as 'syslog host 10.10.10.10 log-prefix CP1CGNAT_O_Poland'
Now I would like to separate above logs into different directories/files based on prefix values with timestamps like -
/Poland/Logs_Poland_2018-12-20, /France/Logs_France_2018-12-20, /UK/Logs_UK_2018-12-20
Can you suggest how to achieve this in rsyslog ?
Update taken from comments:
I have used the different log-prefixs (configured in NAT device) as filter conditions and used in templates as (total 10 similar templates):
$template TmplcpFW_P, "/var/log/NIPFW/Poland/Poland_%HOSTNAME%_%$year%.%$month%.%$day%.log"
if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_Poland' then ?TmplcpFW_P
$template TmplcpFW_F, "/var/log/NIPFW/France/France_%HOSTNAME%_%$year%.%$month%.%$day%.log"
if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_France' then ?TmplcpFW_F
Now I want all the matching logs from above conditions to go into corresponding directories/files and remaining logs in /var/log/messages.
logs rsyslog syslog
logs rsyslog syslog
edited Jan 13 at 16:31
meuh
31.9k11954
asked Jan 10 at 9:55
SarjitSSarjitS
13
edited Jan 13 at 16:31
meuh
31.9k11954
asked Jan 10 at 9:55
SarjitSSarjitS
13
edited Jan 13 at 16:31
meuh
31.9k11954
edited Jan 13 at 16:31
meuh
31.9k11954
edited Jan 13 at 16:31
meuh
31.9k11954
31.9k11954
asked Jan 10 at 9:55
SarjitSSarjitS
13
asked Jan 10 at 9:55
SarjitSSarjitS
13
asked Jan 10 at 9:55
SarjitSSarjitS
13
13
You can extract any part of a message if you can write a regex pattern for it. see here
– meuh
Jan 10 at 18:57
Thanks @meuh, I have used the different log-prefixs (configured in NAT device) as filter conditions and used in templates as (total 10 similar templates) - $template TmplcpFW_P, "/var/log/NIPFW/Poland/Poland_%HOSTNAME%_%$year%.%$month%.%$day%.log" if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_Poland' then ?TmplcpFW_P $template TmplcpFW_F, "/var/log/NIPFW/France/France_%HOSTNAME%_%$year%.%$month%.%$day%.log" if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_France' then ?TmplcpFW_F
– SarjitS
Jan 11 at 6:05
Now I want all the matching logs from above conditions to go into corresponding directories/files and remaining logs in /var/log/messages. Please suggest how to do this. Other suggestions/recommendation are most welcome.
– SarjitS
Jan 11 at 6:09
I have copied your comments into your question, so they are easier to read (you can edit your question at any time, and just refer to the change in a short comment). If I understand you, you now have 10 templates and 10if ...statements that do exactly what you want. The only thing I see missing is the line& ~after eachif ...statement. That would stop the matched message from any further handling. Did you want something more than that?
– meuh
Jan 13 at 16:36
add a comment |
You can extract any part of a message if you can write a regex pattern for it. see here
– meuh
Jan 10 at 18:57
Thanks @meuh, I have used the different log-prefixs (configured in NAT device) as filter conditions and used in templates as (total 10 similar templates) - $template TmplcpFW_P, "/var/log/NIPFW/Poland/Poland_%HOSTNAME%_%$year%.%$month%.%$day%.log" if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_Poland' then ?TmplcpFW_P $template TmplcpFW_F, "/var/log/NIPFW/France/France_%HOSTNAME%_%$year%.%$month%.%$day%.log" if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_France' then ?TmplcpFW_F
– SarjitS
Jan 11 at 6:05
Now I want all the matching logs from above conditions to go into corresponding directories/files and remaining logs in /var/log/messages. Please suggest how to do this. Other suggestions/recommendation are most welcome.
– SarjitS
Jan 11 at 6:09
I have copied your comments into your question, so they are easier to read (you can edit your question at any time, and just refer to the change in a short comment). If I understand you, you now have 10 templates and 10if ...statements that do exactly what you want. The only thing I see missing is the line& ~after eachif ...statement. That would stop the matched message from any further handling. Did you want something more than that?
– meuh
Jan 13 at 16:36
You can extract any part of a message if you can write a regex pattern for it. see here
– meuh
Jan 10 at 18:57
You can extract any part of a message if you can write a regex pattern for it. see here
– meuh
Jan 10 at 18:57
Thanks @meuh, I have used the different log-prefixs (configured in NAT device) as filter conditions and used in templates as (total 10 similar templates) - $template TmplcpFW_P, "/var/log/NIPFW/Poland/Poland_%HOSTNAME%_%$year%.%$month%.%$day%.log" if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_Poland' then ?TmplcpFW_P $template TmplcpFW_F, "/var/log/NIPFW/France/France_%HOSTNAME%_%$year%.%$month%.%$day%.log" if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_France' then ?TmplcpFW_F
– SarjitS
Jan 11 at 6:05
Thanks @meuh, I have used the different log-prefixs (configured in NAT device) as filter conditions and used in templates as (total 10 similar templates) - $template TmplcpFW_P, "/var/log/NIPFW/Poland/Poland_%HOSTNAME%_%$year%.%$month%.%$day%.log" if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_Poland' then ?TmplcpFW_P $template TmplcpFW_F, "/var/log/NIPFW/France/France_%HOSTNAME%_%$year%.%$month%.%$day%.log" if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_France' then ?TmplcpFW_F
– SarjitS
Jan 11 at 6:05
Now I want all the matching logs from above conditions to go into corresponding directories/files and remaining logs in /var/log/messages. Please suggest how to do this. Other suggestions/recommendation are most welcome.
– SarjitS
Jan 11 at 6:09
Now I want all the matching logs from above conditions to go into corresponding directories/files and remaining logs in /var/log/messages. Please suggest how to do this. Other suggestions/recommendation are most welcome.
– SarjitS
Jan 11 at 6:09
I have copied your comments into your question, so they are easier to read (you can edit your question at any time, and just refer to the change in a short comment). If I understand you, you now have 10 templates and 10
if ... statements that do exactly what you want. The only thing I see missing is the line & ~ after each if ... statement. That would stop the matched message from any further handling. Did you want something more than that?– meuh
Jan 13 at 16:36
I have copied your comments into your question, so they are easier to read (you can edit your question at any time, and just refer to the change in a short comment). If I understand you, you now have 10 templates and 10
if ... statements that do exactly what you want. The only thing I see missing is the line & ~ after each if ... statement. That would stop the matched message from any further handling. Did you want something more than that?– meuh
Jan 13 at 16:36
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f493649%2frsyslog-send-logs-to-different-file-by-filtering-syslogs%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f493649%2frsyslog-send-logs-to-different-file-by-filtering-syslogs%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
You can extract any part of a message if you can write a regex pattern for it. see here
– meuh
Jan 10 at 18:57
Thanks @meuh, I have used the different log-prefixs (configured in NAT device) as filter conditions and used in templates as (total 10 similar templates) - $template TmplcpFW_P, "/var/log/NIPFW/Poland/Poland_%HOSTNAME%_%$year%.%$month%.%$day%.log" if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_Poland' then ?TmplcpFW_P $template TmplcpFW_F, "/var/log/NIPFW/France/France_%HOSTNAME%_%$year%.%$month%.%$day%.log" if ($hostname == ["CP1CGNAT","CP2CGNAT"]) and $msg contains 'CP1CGNAT_O_France' then ?TmplcpFW_F
– SarjitS
Jan 11 at 6:05
Now I want all the matching logs from above conditions to go into corresponding directories/files and remaining logs in /var/log/messages. Please suggest how to do this. Other suggestions/recommendation are most welcome.
– SarjitS
Jan 11 at 6:09
I have copied your comments into your question, so they are easier to read (you can edit your question at any time, and just refer to the change in a short comment). If I understand you, you now have 10 templates and 10
if ...statements that do exactly what you want. The only thing I see missing is the line& ~after eachif ...statement. That would stop the matched message from any further handling. Did you want something more than that?– meuh
Jan 13 at 16:36