mjhjmtu

The script below currently removes the ^M character (Ctrl+V+M). I feel it's a bit long winded but I also need to add ^I and any other characters I might see in the future.

Is there an easier way to add ^I (Ctrl+V+I)? This is the first script I wrote for myself about 6 months ago after attending a 2 day shell programming class. I'm not sure if I made it longer than it needs to be, so any general tips would also be appreciated.

#!/bin/bash 

 echo "$# item(s) to review."
 question='Do you want to remove the ^M characters?'

 for file
 do
 if grep "^M" "$file" >> /dev/null 2> /dev/null
 then
 echo "$file contains special characters"
 echo $question
 read answer
 if [[ "$answer" == [yY] ]]
 then
 cat "$file" | sed "s/^M//" > "$file.safe"
 echo "Special characters have been removed and $file.safe has been created."
 elif [[ "$answer" == [yY][eE][sSaA]* ]]
 then
 cat "$file" | sed "s/^M//" > "$file.safe"
 echo "Special characters have been removed and $file.safe has been created."
 else
 echo "Special characters have NOT been removed."
 fi
 elif [[ -d $file ]]
 then
 echo "$file is a directory"
 else
 echo "No special characters in $file"
 fi
 done

You can put a loop around your script. So:

 for c in "^I" "^M"; do
 for file; do
 if grep "$c" "$file"; then
 ...
 etc.
 ...
 fi
 done
 done

I prefer this perl one liner. The 'cM' is the control-M character.
The original file(s) will be backed up with the extension '.bak' This extension can be your choice.

perl -i.bak -pe 's/cM//g;' file(s)

Example using a class of characters to remove. In the brackets perl will find control-I and control-M and remove them. I've not tested this exactly though.

perl -i.bak -pe 's/[cMcI]//g;' files(s)

This is certainly much, much longer than it needs to be. All you need is the tr utility, plus a loop and redirections to act on the files that are passed as arguments to the script.

#!/bin/sh
for file do
 tr -d 'rt' <"$file" >"$file.safe"
done

With the option -d, tr removes the specified characters. The characters to remove are passed together as the first non-option argument. You can use backslash escapes to represent special characters: n for a newline (^J), r for a carriage return (^M), t for a tab (^I), etc.

I haven't reproduced the code for asking the user because it's pointless. Directories will cause an error with redirection anyway, and it's really the job of the caller not to request a nonsensical action such as treating a directory as a regular file, so I also skipped that part.

If you want to replace the original file, write to a temporary file then move the result in place.

#!/bin/sh
for file do
 tmp="$(TMPDIR=$(dirname -- "$file") mktemp)"
 tr -d 'rt' <"$file" >"$tmp" && mv -f -- "$tmp" "$file"
done

The temporary file name is constructed using mktemp so that the script is robust. It will work as long as you have write permission to the directory containing the file, without risking overwriting an existing file. It's secure even if that directory is writable by other users who might try to inject other data (a potential problem in /tmp).

The mv command is only invoked if the call to tr succeeded, so there's no risk of losing data if tr fails, e.g. because the disk becomes full midway through.

If you want to avoid replacing the file by a new, identical file if it doesn't contain any special characters, there are two ways:

• 
  

  You can check for the special characters first. There are several ways to do it. One way is to remove everything except those special characters and count the number of resulting characters. As an optimization, pipe through head -c 1 so that you don't need to go through the whole file if a special character is found close to the top: that way the count is 0 if there's nothing to do and 1 otherwise.

  
  
  

  if [ "$(tr -dc 'rt' <"$file" | head -c 1 | wc -c)" -ne 0 ]; then
   tr -d 'rt' <"$file" >"$tmp" && mv -f -- "$tmp" "$file"
  fi
  

  
  


• 
  

  You can do the transformation, then check if it's identical to the original. This can be slower if the files are often already in the desired state. On the other hand, this technique generalizes to cases where it isn't easy to determine whether the file is in the desired state.

  
  
  

  tr -d 'rt' <"$file" >"$tmp" &&
  if cmp -s "$tmp" "$file"; then
   rm -- "$tmp"
  else
   mv -f -- "$tmp" "$file"
  fi
  

  
  

Have you thought of using

 tr -d .....<characterlist>....

For example, get rid of any non-printable characters and put into another file:

 cat filename | tr -cd '[:print:]' >/tmp/x.out

Modify the characterlist to suit your application....see the tr man apage for more information.

Also it is nice because regex ranges are allowed:

 echo '01020304' | tr -d '[01-03]' | od -c