mjhjmtu

I currently use wget or curl to download a continuous AAC-stream. I would like to limit what's stored on disk to the most recent N MB. In other words, some kind of size limited FIFO-buffer (I guess?). Any ideas how to accomplish this? This is OS X/BSD.

The purpose is to be able to stop the stream when something interesting has happened and then extract the last few minutes from it to a permanent storage.

Update: an alternative solution would be to interrupt every N MB and start a new local file and rotate out the previous file (that is, rename it with a sequence number, time stamp or similar). However, if doing this there need to be a substantial overlap between the files.

Here's a simple python script that may help. It just reads from stdin and writes to stdout, but keeps the last N bytes in memory. If you interrupt it with control-C (SIGINT) it dumps the memory into file /tmp/sample001 and continues.

#!/usr/bin/python3
# circular buffer in memory without recopy using bytearray
# https://unix.stackexchange.com/a/401875/119298
import sys, os

def copy():
 def dump(fd,v):
 fd.write(v)

 space = 10000000
 buffer = bytearray(space) # all zero bytes
 view = memoryview(buffer)
 head = 0; wrapped = False
 sys.stdin = os.fdopen(sys.stdin.fileno(), 'rb', 0)
 sys.stdout = os.fdopen(sys.stdout.fileno(), 'wb', 0)
 fileno = 1
 while True:
 try:
 nbytes = sys.stdin.readinto(view[head:])
 if nbytes==0: 
 break # eof
 sys.stdout.write(view[head:head+nbytes].tobytes())
 head += nbytes
 if head>=space:
 head = 0; wrapped = True
 except KeyboardInterrupt:
 filename = "/tmp/sample%03d" % fileno
 fileno += 1
 with open(filename,"wb") as fd:
 if wrapped:
 dump(fd, view[head:])
 if head:
 dump(fd, view[0:head])

copy()

If you don't have python3 it will need a few changes for python 2.7.
You might need to worry about how to preserve a legal AAC framing format, but perhaps if you try first you may find whatever you are using manages to self-sync from arbitrary offset data.