mjhjmtu

I have a text file with lines (each of 631 characters) consisting of 16 hexadecimal numbers 0,1,2,...,9,A,B,...,F. I would like to replace each number with the corresponding 4 bits of 0 and 1, i.e. 0 with 0000, 1 with 0001, ..., F with 1111 and then save the file as .bin. How can I do so in Linux terminal?

Some lines of my text file are:

85868D6B0FD5F3186DE92A7FC023CC848110FE61E3045B24BD905C8FDF34698794718CB2BAFF2AFC488C07B01510646141290A8FE55BF41F9D5033B13B31473FB4F7E199624370DC9C7587E5FB84B209B104B2BC194243EFC77C479BEA6778C73FB55939CB8654B1EB0B597CEA9507FE5A9636229BE0E7ECA27844D3A4AE16096D9DDC125728A1BF0C3623518311CF7CE629C15EA767E1F6C992106E86D962D47E307AE9F71FE5C6B664820E3D7521D0F783913E8A116BFD42C65FD2E28C9114CF4BFCB8A2AFD9979E7B5EBC7B59C7A31A7C12931D2C794EF5AED991B32560BBEFCC82D3A6AC6D371C7AC6C06DED6BCEFFC3D9E11A17B4738EA2ECF8CDD14805E57C50F973C6832DB94517A71C06E18329694B8CB7F7028730E54F74E27EF35B3D776A7E46A3F187691D460CBFBCD14BAC4AED32E5592AFDAEC5206
6321EB4F15D5F01D2A5F7474AC6FFCD1A8B52E680AC52CA9B500C956FD269DB8107EFCF6EC0B23B2C54B3A727CC53F6793019BF0D95A037485413576E4A0172A597027ED71D6E785430FA4748E164E5290214B01740736C0EC79D286692170B6EFBA21592EC485350F0DC6C635CD3FA7DAED258405C4247BFA68613B5F7CBEC903E16EA2E780D0934F0A2918F837CF0115B3D76AAFD3F37D905F9572D334D959DDD22C37EBA7707A76333A491F79B9C53A7912AEA38ED400949EB9AD9FCEE71765F4F4A2E21C481A509AB6807DDF5374D9A1A4D0C589E628CAA3C6DC9E4A5C1A7EF3C36A4D2231E2C1E856A19B6F7F4DE713D5BE8A4EFA750FB29A34F00754D875C22C2FC981FAECCC7113D1B0D6FF3BCC747852789A662D104DF1A8C90EB4DDFA48502A3BA9AAE70E7229E8A3D6FBBD00C19D99765BAFC9B09D547
2F75AC10A9E20C07DD4686C5706F66EB1AB33ED9DEDAD69079B6EC0C24C8DF24CB1855825D80CD484E5623BB246041D29240720500EA87586CC4CE5A00990EF8B5B7E2A3CF3E997323870499451E7BFA7F23A2208523B84A98DDC0763701C1AB292D854ADA8FB2E060088D7B3436571745DF25182D7F5A646A6F7DD830B9CCC12C37D416882DD7AAA0E8CB2CE50CE33AD1674C0E78E0F171DEC2D087C6CF2E711FAC2548CCB8C0312D59055A0A3A145B402A749F768F637549E0749215B5542BBC9D07EF9416C1BAB1FBEAC4709C458E7C266342908421BC8F67B7E21ED228D13A0F2750C290EFF8CAF7D2AD6D6C2CB12B81570190E1BC555E0CFBA4951BEAD3740D3BEBF2C7BF2AB64C7F046E5B865D85A1411F854906C05785A0A3684EC0F69D9BAB047A9EC17F973E78A92838C73270F97E54CEF4FABAA82876F

Posting my comment as an answer to be marked as "solved"

for n in $(cat myfile | tr 'a-f' 'A-F');do echo "obase=2; ibase=16; $n" | bc ;done

Okay, in the light of what you have said and what I understood, I have a couple of suggestions to you. The first one, converts each hex line into a new binary line and puts a line break at the end of each line. here is that solution:

cat myfile | while read line
do
 len=$#line
 binline=""
 index=1
 while [ $index -le $len ]
 do
 hex=$(echo $line | cut -c $index | tr 'a-f' 'A-F')
 bin=$(echo "obase=2; ibase=16; $hex"|bc)
 padbin=$(printf "%04d" $bin)
 binline="$binline$padbin"
 (( index++ ))
 done
 echo $binline >> program.bin
done

And, if I am not mistaken, you are trying to use this binary file as a program to an embedded system or something similar. If that is the case the line breaks may present a problem. If this is the case, i.e., you want the whole file as one single humongous line, you can use the following snippet:

cat myfile | while read line
do
 len=$#line
 index=1
 while [ $index -le $len ]
 do
 hex=$(echo $line | cut -c $index | tr 'a-f' 'A-F')
 bin=$(echo "obase=2; ibase=16; $hex"|bc)
 printf "%04d" $bin >> p2.bin
 (( index++ ))
 done
done

Having said all of this, if your hexadecimal values file is in gigabytes size or even few hundred megabytes size, bash will complain and may throw a cryptic error. So, it is in your best interest to chop this input file to more manageable chunks and process them piece-by-piece.

Please note that, I wrote these scripts in the simplest for to understand. Hence it will have some performance penalties when you run them. 3 lines takes about 6 seconds of processing time. It is up to you to optimize it, if you find the wait intolerable.

xxd -r -p

Example:

printf 12345678 | xxd -r -p | od -A n -t x1 -v

gives:

 12 34 56 78

od dumped the bytes, so xxd -r -p did what we wanted.

You can use printf:

$ printf '%04dn' 0
0000

$ printf '%04dn' 1
0001

Or the POSIX Awk stdlib library:

$ awklib 'BEGIN print base_conv("F", 16, 2)'
1111