Archive size > original files?

[jhg]

I created an archive of a large photo directory (mostly consisting of ~20-30MB raw files) and find that, while the on-disk size is just under 1.2TB (where 1TB = 1024⁴) the resulting backup is about 1.7TB. 

 

Is this to be expected?

 

I have software compression enabled based on the fact that gzip can still compress the raw files a few percent. 

 

Is it possible Retrospect's compression is causing that much expansion?

[Lennart_T]

If your calculations are correct, that is not to be expected.

 

I would turn off software compression. It isn't worth the CPU time to save just a few percent. It will take longer time to backup and longer time to restore.

[jhg]

FYI, I deleted the backup set and reran with compression turned off.  Now the backup set is the same size as the source files.

 

Thus the 35% increase in size was due entirely to Retrospect's poor performance on compressing incompressible data. 

 

I'm running Retrospect 9.5.3.103, and this is a rather serious problem that should be corrected if it hasn't been already addresses in version 10.5.

[Scillonian]

The problem is how do you know if the data is incompressible until you try to compress it. For each file you would have to run a test for compressibility then decide whether or not it was worth sending a compressed version to the backup set. In the case of your backup the run time would be considerably increased for no reduction in storage space.

 

And using the file extension to decide would not be reliable. TIFF image files can for example be uncompressed and lossless compressed with two different compression formats. FLAC audio files can also be either.