mjhjmtu

Ubuntu linux: ls -s disagrees with du & stat for the number of blocks used by a small file.

ls -s ../nc2/.git/logs/refs/heads/ 
total 4

du ../nc2/.git/logs/refs/heads/ 
8 ../nc2/.git/logs/refs/heads/

stat ../nc2/.git/logs/refs/heads/ 
File: ‘../nc2/.git/logs/refs/heads/’ 
Size: 4096 Blocks: 8 IO Block: 4096 ......

sudo blockdev --getbsz /dev/sda
4096

ls -s shows the file using 4 blocks. du & size say it uses 8 blocks.

Why is ls -s seemingly wrong? Can it not detect the correct block size? I can make it say the file uses 8 blocks by running 'ls -s --block-size 512'.

This is NOT a size of file vs number of blocks question. All commands above are listing block size not file size.

Edit: More info requested:

ls --version 
ls (GNU coreutils) 8.21

type ls 
ls is aliased to `ls --color=auto'

LS_BLOCK_SIZE=512 ls -s ../nc2/.git/logs/refs/heads/ 
total 8

ls -s reports the st_blocks member of the structure returned by the stat()/lstat() system calls. That's a number 512-byte blocks. 512 bytes is usually the minimum storage granularity as that corresponds to early disk sectors.

Or at least that's what most ls implementations do including the original Unix implementation, and what POSIX requires.

The GNU implementation of ls (also busybox which mimics it, both found on Ubuntu) however changed that to 1024-byte blocks, but goes back to 512-byte blocks if the $POSIXLY_CORRECT (formerly $POSIX_ME_HARDER) variable is in the environment (not for busybox). I suppose that's to make it more readable to a human but that means we lose precision on filesystems that use 512-byte storage granularity and doesn't help with portability.

From the ChangeLog:

Wed Aug 21 13:03:14 1991 David J. MacKenzie (djm at wookumz.gnu.ai.mit.edu)

• Version 3.0.




• du.c, ls.c: Make 1K blocks the default size, and -k a no-op.
  Down with dumb standards!

With GNU ls (not busybox), the block size can also be specified with the --block-size option or the $LS_BLOCK_SIZE environment variable. So you can use ls --block-size=1 -s or LS_BLOCK_SIZE=1 ls -s to get the disk usage in bytes. Other ls implementations like on BSDs use $BLOCKSIZE for that¹ (also recognised as well as $BLOCK_SIZE by GNU ls as shown by @yahol).

POSIXly, you can use -k to get the count in kibibytes (which thankfully with GNU or BSD ls takes precedence over the $BLOCKSIZE environment variables).

Portably (if you want to take into account busybox ls where, the report in kibibytes is hard coded), to get back to the st_blocks (or at least an approximation thereof), you'd need something like:

blocks=$(ls -skd -- "$file" | awk 'print $1*2; exit')

With GNU find, -printf %b reports a number of 512-byte blocks, and -printf %k 1024-byte blocks, and it's not affected by the environment. -printf is GNU specific.

In any case, nowadays, that has nothing to do with the filesystem block size.

^{¹ On BSDs, $BLOCKSIZE is rounded to a multiple of 512 (BLOCKSIZE=1023 is the same BLOCKSIZE=512) and values below 512 are not allowed}

This is explained in info coreutils Block size (to which I found my way from info ls):

2.3 Block size

(...)The block size used for display is independent of
any file system block size.(...)

‘DF_BLOCK_SIZE’
 This specifies the default block size for the ‘df’ command.
 Similarly, ‘DU_BLOCK_SIZE’ specifies the default for ‘du’ and
 ‘LS_BLOCK_SIZE’ for ‘ls’.

Seems, that each of these programs can use different block size defined in environmental variables.

The following two env variables can help unify the output:

‘BLOCK_SIZE’
 This specifies the default block size for all three commands, if
 the above command-specific environment variables are not set.

‘BLOCKSIZE’
 This specifies the default block size for all values that are
 normally printed as blocks, if neither ‘BLOCK_SIZE’ nor the above
 command-specific environment variables are set. Unlike the other
 environment variables, ‘BLOCKSIZE’ does not affect values that are
 normally printed as byte counts, e.g., the file sizes contained in
 ‘ls -l’ output.

And this explains where it all defaults to 512 blocks:

‘POSIXLY_CORRECT’
 If neither ‘COMMAND_BLOCK_SIZE’, nor ‘BLOCK_SIZE’, nor ‘BLOCKSIZE’
 is set, but this variable is set, the block size defaults to 512.