mjhjmtu

We are using linux logrotate for rotating our log files,

Example :

/location/tomcat/logs/* /location/jboss/log/* 
 copytruncate
 daily
 rotate 10
 compress
 size 20M
 olddir rotated
 create 0644 test test

As per the copy truncate definition given in LINUX,

copytruncate
 Truncate the original log file in place after creating a copy,
 instead of moving the old log file and optionally creating a new
 one, It can be used when some program can not be told to close
 its logfile and thus might continue writing (appending) to the
 previous log file forever. Note that there is a very small time
 slice between copying the file and truncating it, so some log-
 ging data might be lost. When this option is used, the create
 option will have no effect, as the old log file stays in place.

So there will be a data loss in log file during rotate process. I noticed around 5 to 20 seconds of log loss, Is there a way / configuration to do the same process without data loss?

I know that in syslog-ng, you can set the output filename to use variables, including things like the date, so it would automatically start writing to a new file at 12:00am on the new day, with zero loss.

It requires changing your syslog program to syslog-ng, though. But the flexibility sounds like exactly what you need.

You could move from logfile logging and rotating with logrotate to journald logging using systemd-cat -t indentifier cmdline. Setup log/entry sizes in /etc/systemd/journald.conf or use journalctl --vacuum-size=, --vacuum-time=, --vacuum-files=

Another way is to logrotate once a day by using the prerotate/postrotate feature to stop the servers, rotate the log and start the servers again. The logrotation time is depending on your systemd logrotate.timer or (ana)cron job, which could be timed to 4am or whenever is low to zero traffic.

prerotate
 # stop jboss/tomcat server 
endscript

daily
rotate 10
compress
size 20M

postrotate
 #start servers
endscript

This is how I do it for Catalina logs:

/var/log/tomcat/catalina.out xargs kill -9
 cat /var/log/tomcat/tmp.log >> /var/log/tomcat/catalina.out
 rm /var/log/tomcat/tmp.log -f
 endscript

You may optionally specify -s with tail to decrease refresh interval of tail command itself. And because you use -c option, it's fast.