mjhjmtu

I have a big log file with non text chars. I used grep to search and I got this result:

Binary file (standard input) matches

I can use grep -a to skip these line with non text chars.

Now, how can I find out all lines which contain non-text chars ?

What GNU grep considers non-text varies with the version and the locale.

In first approximation, you can try:

grep -anPe '^((?!.*$)|.*)' < file.log

That is look for the lines that contain a NUL character, 0 byte (likely to be the cause of that Binary file message if your log file was truncated while open for writing by some process without O_APPEND), or non-characters (possible if you're in a locale with a multibyte charset like UTF-8 and some lines were output in another charset).

That assumes your GNU grep was built with PCRE support (for -P).

You may want to pipe that output to something like sed -n l or hexdump -C or od -vtc -tx1 (and maybe omit the -n option to grep) to try and identify those byte sequences that cause the binary message.

Note that grep -a does not skip those lines, it just tells GNU grep to not treat files it considers as binary specially. Lines with those 0 bytes or non-characters will still be reported if they match the pattern.

On Linux at least and most native filesystems, you can tell if a file is sparse, that is has unallocated parts (holes) that would appear full of zero bytes with:

perl -le '
 seek STDIN,0,4 or die; $hole = tell STDIN;
 seek STDIN, $hole, 3 and $data = tell STDIN;
 seek STDIN, 0, 2; $end = tell STDIN;
 if ($hole != $end) ' < file.log

Holes would be created whenever the gap would otherwise include at least one full filesystem block (typically 4KiB). There would probably be more NUL bytes on either side of those hole.