mjhjmtu

The size of your tar file will be 937MB plus the size of the metadata needed for each file or directory (512 bytes per object), and padding added to align files to a 512-byte boundary.

A very rough calculation tells us that another copy of your data will leave you with 3.4GB free. In 3.4GB we have room for about 7 million metadata records, assuming no padding, or fewer if you assume an average of 256 bytes' padding per file. So if you have millions of files and directories to tar, you might run into problems.

You could mitigate the problem by

• compressing on the fly by using the z or j options to tar
  


• doing the tar as a normal user so that the reserved space on the / partition won't be touched if you run out of space.

tar itself can report on the size of its archives with the --test option:

tar -cf - ./* | tar --totals -tvf -

The above command writes nothing to disk and has the added benefit of listing the individual filesizes of each file contained in the tarball. Adding the various z/j/xz operands to either side of the |pipe will handle compression as you will.

OUTPUT:

...
-rwxr-xr-x mikeserv/mikeserv 8 2014-03-13 20:58 ./somefile.sh
-rwxr-xr-x mikeserv/mikeserv 62 2014-03-13 20:53 ./somefile.txt
-rw-r--r-- mikeserv/mikeserv 574 2014-02-19 16:57 ./squash.sh
-rwxr-xr-x mikeserv/mikeserv 35 2014-01-28 17:25 ./ssh.shortcut
-rw-r--r-- mikeserv/mikeserv 51 2014-01-04 08:43 ./tab1.link
-rw-r--r-- mikeserv/mikeserv 0 2014-03-16 05:40 ./tee
-rw-r--r-- mikeserv/mikeserv 0 2014-04-08 10:00 ./typescript
-rw-r--r-- mikeserv/mikeserv 159 2014-02-26 18:32 ./vlc_out.sh
Total bytes read: 4300943360 (4.1GiB, 475MiB/s)

Not entirely sure of your purpose, but if it is to download the tarball, this might be more to the point:

ssh you@host 'tar -cf - ./* | cat' | cat >./path/to/saved/local/tarball.tar

Or to simply copy with tar:

ssh you@host 'tar -cf - ./* | cat' | tar -C/path/to/download/tree/destination -vxf -

I have done a lot of research on this. You can do a test on the file with a word count but it will not give you the same number number as a du -sb adir.

tar -tvOf afile.tar | wc -c

du counts every directory as 4096 bytes, and tar counts directories as 0 bytes. You have to add 4096 to each directory:

$(( $(tar -tvOf afile.tar 2>&1 | grep '^d' | wc -l) * 4096)))

then you have have to add all of the characters. For something that looks like this:

$(( $(tar -tvOf afile.tar 2>&1 | grep '^d' | wc -l) * 4096 + $(tar -xOf afile.tar | wc -c) ))

I am not sure if this is perfect since I didn't try files that have been touched (files of 0 bytes) or files that have 1 character. This should get you closer.

-cvf does not include any compression, so doing that on a ~1 GB folder will result in a ~1 GB tar file (Flub's answer has more details about the additional size in the tar file, but note even if there are 10,000 files this is only 5 MB). Since you have 4+ GB free, no you will not fill the partition.

an easily downloadable copy

Most people would consider "easier" synonymous with "smaller" in terms of downloading, so you should use some compression here. bzip2 should now-a-days be available on any system w/ tar, I think, so including j in your switches is probably the best choice. z (gzip) is perhaps even more common, and there are other (less ubiquitous) possibilities with more squash.

If you mean, does tar use additional disk space temporarily in performing the task, I am pretty sure it does not for a few reasons, one being it dates back to a time when tape drives were a form of primary storage, and two being it has had decades to evolve (and I am certain it is not necessary to use temporary intermediate space, even if compression is involved).

If speed is important and compression is not needed, you can hook the syscall wrappers used by tar using LD_PRELOAD, to change tar to calculate it for us. By reimplementing a few of these functions to suit our needs (calculating the size of potential output tar data), we are able eliminate a lot of read and write that is performed in normal operation of tar. This makes tar much faster as it doesn't need to context switch back and forth into the kernel anywhere near as much and only the stat of the requested input file/folder(s) needs to be read from disk instead of the actual file data.

The code below includes implementations of the close, read, and write POSIX functions. The macro OUT_FD controls which file descriptor we expect tar to use as the output file. Currently it is set to stdout.

read was changed to just return the success value of count bytes instead of filling buf with the data, given that the actual data wasn't read buf would not contain valid data for passing on to compression, and thus if compression was used we would calculate an incorrect size.

write was changed to sum the input count bytes into the global variable total and return the success value of count bytes only if file descriptor matches OUT_FD, otherwise it calls the original wrapper acquired via dlsym to perform the syscall of the same name.

close still preforms all of its original functionality, but if the file descriptor matches OUT_FD, it knows that tar is done attempting to write a tar file, so the total number is final and it prints it to stdout.

#define _GNU_SOURCE
#include <unistd.h>
#include <stdio.h>
#include <stdint.h>
#include <inttypes.h>
#include <stdlib.h>
#include <errno.h>
#include <dlfcn.h>
#include <string.h>

#define OUT_FD 1
uint64_t total = 0;
ssize_t (*original_write)(int, const void *, size_t) = NULL;
int (*original_close)(int) = NULL;
void print_total(void)

 printf("%" PRIu64 "n", total);


int close(int fd)

 if(! original_close)
 
 original_close = dlsym(RTLD_NEXT, "close");
 
 if(fd == OUT_FD)
 
 print_total();
 
 return original_close(fd);


ssize_t read(int fd, void *buf, size_t count)

 return count;


ssize_t write(int fd, const void *buf, size_t count)

 if(!original_write)
 
 original_write = dlsym(RTLD_NEXT, "write");
 
 if(fd == OUT_FD)
 
 total += count;
 return count;
 
 return original_write(fd, buf, count);

Benchmark comparing a solution where the read disk access and all the syscalls of normal tar operation is performed against the LD_PRELOAD solution.

$ time tar -c /media/storage/music/Macintosh Plus- Floral Shoppe (2011) [Flac]/ | wc -c
332308480
real 0m0.457s
user 0m0.064s
sys 0m0.772s

tarsize$ time ./tarsize.sh -c /media/storage/music/Macintosh Plus- Floral Shoppe (2011) [Flac]/
332308480
real 0m0.016s
user 0m0.004s
sys 0m0.008s

The code above, a basic build script to build the above as a shared library, and a script with the "LD_PRELOAD technique" using it is provided in the repo:
https://github.com/G4Vi/tarsize

Some info on using LD_PRELOAD: https://rafalcieslak.wordpress.com/2013/04/02/dynamic-linker-tricks-using-ld_preload-to-cheat-inject-features-and-investigate-programs/