mjhjmtu

You don't need to call out to any external tools: ksh can do arithmetic. I'm also using an array to store the remainders

#!/usr/bin/ksh
div=62
while IFS='|' read -r n x; do
 rem=$(( x % div ))
 quo=$(( x / div ))
 echo "reminder is $rem" >&2
 echo "quotiont is $quo" >&2

 remainders=( $rem )
 while (( quo >= div )); do
 sub_rem=$(( quo % 62 ))
 quo=$(( quo / 62 ))
 echo "reminder is $sub_rem" >&2
 echo "quotiont is $quo" >&2
 remainders=( $sub_rem "$remainders[@]" )
 done
 echo "$n|$x|$quo $remainders[*]"

 x=$quo
 for r in "$remainders[@]"; do
 x=$(( x * div + r ))
 done
 echo Verification: $x
done <<END
1|5147634738948389685
END

There are several things that could be done (and speed gained):

• original on 1000 numbers
  
  35.023 sec


• replace all the expr commands with arithmetic expansions $((x%62))
  
  14.473


• convert grp_rem=`echo $sub_rem" "$grp_rem` to grp_rem="$sub_rem $grp_rem"
  
  3.131


• avoid the use of cut (set IFS='|'; set -f; and use shell split with set -- $1)
  
  
  • or use IFS='|' read a x <<<"$i" (though <<< creates a temp file)
  
  
  • and as one read is already being used, replace that read.
    
    0.454
  
  


• reduce to only one loop (remove the if) and remove trailing space at the end
  
  0.207


• Make the loop tighter Join both $((...))
  
  0.113
  
  ---- shell: a change of ~300 times faster than 35.023 seconds.
  
  ++++ This is probably the best that can be done with a shell script.


• change to awk
  0.123
  
  ---- awk: a total change of ~280 times faster

Resulting script:

#!/usr/bin/ksh
while IFS='|' read a b # read both values split on '|'
do
 x=$b # set value of x (quotient)
 grp_rem="" # clear value of group
 while (( rem=x%62 , x/=62 )) # do both math expressions.
 do
 grp_rem="$rem $grp_rem" # concatenate resulting values
 done
 grp_rem=$grp_rem%? # remove one character (an space)
 echo "$a|$b|$rem $grp_rem" 
done < base62_while.txt >> base62_while.out

An awk script equivalent. I don't know if this is the faster awk script possible, but works fine. Faster than the shell for more than 10k lines.
Note: This is using GNU awk with the option of -M (arbitrary precision) which is a must to process numbers in the order of 19 digits that you presented. It could process even longer numbers, I did not check how long, but I am pretty sure that the limit is pretty high. :-) Note that awk must have been compiled with that option included (check with awk 'BEGIN print( PROCINFO["gmp_version"], PROCINFO["prec_max"]) ')

awk -MF'|' ' x=$2; grp_rem="";
 while(x>0)
 rem=x%62;
 x=int(x/62);
 grp_rem=rem" "grp_rem
 
 printf("%-22s
 ' <base62_while.txt >>base62_while.out

After playing for a bit with the Math::Base::Convert perl module I came up with

perl -F'|' -MMath::Base::Convert -lne '
 BEGIN
 $bc = new Math::Base::Convert(dec,b62); 
 # create a mapping from internal symbol set to desired decimal representation
 $syms = $bc->b62; 
 @h@$syms = (0..61);
 
 print join "|", @F[0..1], (join " ", map $h$_, split //, $bc->cnv($F[1]))
' base62_while.txt

There may be faster perl alternatives as discussed here Base conversion although I'm not sure if they have the same flexibility to manipulate the output mapping.

With dc:

sed 's/.*|(.*)/[&|]P1p/;1s/^/62o/' base62_while.txt | dc > base62_while.out

Or bc (note that historical implementations of bc are actually wrappers around dc):

sed 's/.*|(.*)/"&|";1/;1s/^/obase=62;/' base62_while.txt | bc > base62_while.out

Note that dc and bc wrap long lines of output. With the GNU implementations, you can set the DC_LINE_LENGTH and BC_LINE_LENGTH environment variables to 0 to avoid it.

$ echo '1|167883826163764944817996215305490271305728' | sed 's/.*|(.*)/[&|]P1p/;1s/^/62o/' | dc
1|167883826163764944817996215305490271305728| 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00
$ echo '1|167883826163764944817996215305490271305728' | sed 's/.*|(.*)/[&|]P1p/;1s/^/62o/' | DC_LINE_LENGTH=0 dc
1|167883826163764944817996215305490271305728| 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

You can do some optimizations.

Change

grp_rem=`echo $sub_rem" "$grp_rem`

to

grp_rem="$sub_rem $grp_rem"

Change

x=`echo $i |cut -d"|" -f2`

to

x="$"

You probably also want to change

if [[ $#quo -ge 2 ]]

to

if [[ $quo -ge 62 ]]

Reducing the number of subshells a little will help. If you want more speed, use a language like C.

The shell is slow: use a different language. If we compare the original KSH script (modified to used stdin and stdout), something very similar to steeldriver's Perl code (a script instead of a one-liner that shows similar speeds to glenn jackman's native KSH version), and a LISP implementation with 10,000 lines of input on a Centos 7 test system:

base62.ksh 93.29s user 143.48s system 109% cpu 3:36.73 total
base62.perl 1.32s user 0.00s system 99% cpu 1.326 total
base62.sbcl 0.22s user 0.03s system 99% cpu 0.243 total

Obviously the original code quickly becomes impractical as the input lines increase, as will scripting languages compared to LISP with significant amounts of input. The base62.sbcl time is from a tail call recursive implementation:

#|
eval 'exec sbcl --script "$0" $1+"$@"'
|#
(defun divvy-r (n b l)
 (if (< n b) (cons (truncate n) l)
 (let ((rem (truncate (mod n b))) (quo (/ n b)))
 (divvy-r quo b (cons rem l)))))
(defun divvy (n b)
 (let ((rem (mod n b)) (quo (/ n b)))
 (if (< quo 2)
 (list (truncate quo) (truncate rem))
 (divvy-r n b nil))))
(loop for line = (read-line *standard-input* nil) while line do
 (let ((n (parse-integer (subseq line (1+ (position #| line))))))
 (let ((out (divvy n 62)))
 (format t "~a|~~a~^ ~~&" line out))))

Reading "Common Lisp: A Gentle Introduction to Symbolic Computation" and doing all the exercises therein is how I learned this. Slightly faster (and ever so succinct) is a do* implementation based on glenn jackman's KSH code:

#|
eval 'exec sbcl --script "$0" $1+"$@"'
|#
(defun remainders (n base)
 (do* ((rem (mod n base) (mod quo base))
 (quo (/ n base) (/ quo base))
 (out (cons (truncate rem) nil) (cons (truncate rem) out)))
 ((< quo base) (cons (truncate quo) out))))
(loop for line = (read-line *standard-input* nil) while line do
 (let ((n (parse-integer (subseq line (1+ (position #| line))))))
 (format t "~a|~~a~^ ~~&" line (remainders n 62))))