mjhjmtu

I have a growing file. while will write the log of my application process. In that file, there is a log for "speed" as below

(some text)
speed= 1x
(some text)
speed= 1x
(some text)
speed= 1x
(some text)
speed= 0.9x
..
..
(some text)
speed= 0.8x
(some text)

I have tried the below code to get the average value of speed.

 awk '/speed/ gsub("x","");print $2' $PROCESS_LOG_FILE | awk -F : 'sum+=$1 END print sum/NR'

is it possible to get the average of the last 120 entries? I have tried with grep and tail, but it took a long time since its a growing file.

A bit far fetched, maybe, but as tac will lseek the momentary file end, you'd define the necessary point in time, and from there go backwards until 120 speeds encountered:

tac file | awk '/speed/ SUM += $2; if (++C == 120) print SUM/C; exit'

Or to handle inputs that might have fewer than 120 occurrences of the pattern:

tac file | awk '/speed/ SUM += $2; if (++C == 120) exit
 END if (C) print SUM/C'

I'd use perl rather than awk for this: quite straightforward to remember just the last 120 speeds:

perl -MList::Util=sum -nE '
 if (/speed= ([d.]+)/) @speeds = ($1, @speeds)[0..119] 
 # could also write:
 # if (/speed= ([d.]+)/) push @speeds, $1; shift @speeds if @speeds > 120

 END say @speeds == 0 ? "No matches" : sum(@speeds)/@speeds
' speed.log

You'll have to use a circular table, here is a sample code with last 5 values

BEGIN maxi=5 ; c=0 ; nb=0 ; 
/^speed/ list[nb++]=$2 ; nb=nb % maxi ;
 c++ ; if (c> maxi) c=maxi ;
 s=0 ;
 for(i=0;i<=c;i++) s+=list[i] ;
 printf "NR: %d, c=%d, s=%d AVG : %3.2fn",NR,c,s,s/c ;

you can test on a sample file (or on command line, values to be entered on command line).

awk -f avg.awk sample.txt

then replace 5 by 120, and printf line by what suit you.

note that +=$2 will ignore trailling x.

Does this work for you?

grep speed test | tail -n 120 | cut -d " " -f 2 | cut -d "x" -f 1 | awk -F : 'sum+=$1 END print sum/NR'

I just put your input into a file called test and ran that. Output:

0.94

Can't tell what performance will be on a huge file, though.