mjhjmtu

I have a unix csv file as pipeline "|" separator . But while I am opening in vi editor there are some extra characters are coming as ~G .
But while I am doing cat , I could not see any ~G characters .

453136~G|OORAHASS0343136~G|Generic Box Access~G|NMBLDD~G|/shelf=0/slot=1/port=7~G|20Mbit/s~G|80Mbit/s~G|IS

How to remove ~G characters .

I have already tried below steps but no luck .

sed -e 's/[^ -~]//g' file_in > file_out

or

grep -c '[^ -~]' file_in 

or

sed -i 's/~H//g;s/~G//g' file_in

cat -e rendering them as M-^G suggests they are 0x87 bytes (0207 in octal). As its documentation¹ says, vim renders byte 0x87 as ~G when in locales using single-byte charsets or when the encoding is Unicode and the ESA character is encoded as a valid UTF-8 multibyte sequence, and renders the byte as <87> when the encoding option is Unicode and the character does not form part of a valid UTF-8 sequence. (It renders ^G for 0x7, the ASCII BEL character.)

That's G (0x47 in ASCII) with bit 7 (meta) set to 1 and bit 6 set to 0 (control). That byte doesn't form a valid character in UTF-8 and is typically the code for a control character (ESA) in the C1 set in ISO8859-x charsets.

To get rid of it, you can do:

tr -d '207' < file > file.new

With GNU sed and a shell like ksh93/zsh/bash with support for $'...':

sed -i $'s/207//g' file

Your

sed 's/[^ -~]//g'

would have done it, but only in the C locale. What character ranges match in other locales is pretty random. So:

LC_ALL=C sed 's/[^ -~]//g' < file > file.new

(note that it would delete all other control characters including tabulation and CR (but not LF) and non-ASCII characters).

0x87 is ‡ in the windows-1252 character set (sometimes improperly refereed to as latin1 or iso8859-1).

If you wanted those 0x87 to be converted to ‡ (because for instance those files come from the Windows world and that's what those 0x87 were intended to be) in your locale's charset (assuming it has such a character), you could use:

iconv -f windows-1252 < file > file.new

¹ Bram Moolenaar (2011-03-22). 'isprint'. "options". VIM Reference Manual.

Using tools from coreutils only:

# Generate a test file
printf 'head207nsome text207nnew line' > /tmp/test.cchar

# And filter with tr
tr -d "207" < /tmp/test.cchar > /tmp/test.filtered 

The ~Gis a bell character which is ASCII 007. An easy way to remove it and update your file, inplace is:

perl -pi -e 's/07//' file_in

See also the ASCII table

A more convoluted sed solution is to use shell substitution:

sed -i 's/'`echo "07"`'//' file_in

When using cat, add the -e option to show non-printing characters.