mjhjmtu

I'm using the following statement (simplified version):

tail -f -c+1 <filename>

in order to pipe a file to a process.

What I've found, though, is that there is a number of lines at the end which are not piped.

A specific example is piping a mysql file, and stopping when the end is reached:

tail -f -c+1 mysqdump.sql | sed '/^-- Dump completed/ q0'

This doesn't work - the last line of the dump -- Dump completed [...] is not piped to sed.

My guess is that the tail -f buffer, in this case, is flushed only when it's full.

Does anybody know how can I workaround this?

=================

I've found the cause - the description is not complete (and the code doesn't exhibit the behavior).

The problem happens when piping from a compressed (lzma) file:

tail -f -c+1 <filename.lzma> | lzma -d | sed '/^-- Dump completed/ q0'

Most likely, tail is not sending the last compressed block, because it doesn't detect any new line, as the input is binary.

tail -f flushes after every input line. You can confirm this with strace (or truss or whatever your unix variant provides to trace processes' system calls).

If there is an incomplete line, tail -f keeps waiting for the next newline. It's a tool designed for text files. If you need to tail a binary file (e.g. if -- Dump completed is not followed by a newline), you'll have to use a custom tool.

If you've redirected sed's output away from the terminal, it will be doing its own buffering. Try stdbuf or unbuffer.

Example using stdbuf -o0. Reference: https://superuser.com/a/1123674/126847

tail -F cpu.log | stdbuf -o0 tr '=' ' ' | stdbuf -o0 awk '$6 > 1'