Large config80.dat file? (225MB)?

[peternlewis]

So I checked what files were being backed up in an incremental backup of my server to see what might need to be excluded from the backups, and one of the biggest files, 225MB, was /Library/Application Support/Retrospect/Config80.dat

 

What is this file, why is it so big, and does it need to be backed up?

[Lennart_T]

It contains Retrospect information such as serial number, clients, scripts and the log. It's most likely the log that is the largest part of the file.

 

225MB seems quite large as a Config80.dat file, but not very large in general terms. You can limit the log file size in the Retrospect Preferences.

[peternlewis]

The Log size limit is 10MB according to the preferences, so I don't see how this would be the issue, unless it is logging something else.

 

Any other causes of this?

[peternlewis]

I took a look inside the file, and it is full of zeros. Like really full. Of the 225MB, 195MB of the file are 0 bytes. The file contains huge swathes of zeros, interspersed with patches of data. Just for fun I gziped the file, it came out as 2MB. Even a simple run length encoding of the zeros would trim off 85% of the file!

 

So why all the empty space?

[Lennart_T]

I have a faint memory of someone in the forum with the same problem as you. I don't remember if there was any solution (other that deleting the file and set up Retrospect from scratch).

 

A search in the forum might reveal something.

[twickland]

It's most likely the log that is the largest part of the file.

Config80.dat does not contain the operations log, but it does contain the data about past backups, activities, and reports. How does the size compare to Config80.bak?

[prl]

Config80.dat does not contain the operations log, but it does contain the data about past backups, activities, and reports. How does the size compare to Config80.bak?

Curiously (suspiciously, perhaps), on my system, Config80.bak is much smaller than Config80.dat (0.76MB vs 56.9MB). Gzip compresses both of them by a large amount (87.5% for .bak and 98.6% for .dat), indicating that there is a large amount of fairly simply-structured repetition in the files (like lots of zeroed-out areas), but the compressed sizes are still quite different (94.8kB for .bak and 777.3kB for .bak).

 

The "file" utility doesn't recognise either as being any particular kind of file:

prl$ file Config80.*

Config80.bak: data

Config80.dat: data

prl$

So the .bak file isn't compressed with a well-known compression tool (otherwise it would be identified as such). Anyway, the fact that it compresses so much tends to suggest that it isn't compressed already.

 

I have a pretty simple setup with a server and single client: 6 sources, 5 media sets and 2 (optical) storage devices. I recycle media sets every few months - I'm backing up for disaster recovery rather than for archive.

 

Oh, for a text-based config file where the answers to such things would be immediately clear.

[peternlewis]

I have a faint memory of someone in the forum with the same problem as you. I don't remember if there was any solution (other that deleting the file and set up Retrospect from scratch).

 

A search in the forum might reveal something.

 

Yes, that would be here - no solution. The OP replaced it was the small .bak file, but it grew back to its large size (curiously similar size of 223 MB).

 

And as with @prl and the OP above, the .bak file is small, the .dat file is very large:

 

 

-rwxr-xr-x 1 root admin 639580 Mar 16 02:00 Config80.bak

-rwxr-xr-x 1 root admin 240108184 Mar 16 08:34 Config80.dat

 

02:00 is when my nightly backup runs, which backs up the server and fails to back up my Mac (see here for details). 8:34 was when the last backup finished (a proactive backup). Curious that the ,bak was only modified this morning and is small, given the .dat file was large previously.

[David W Lee]

Retrospect's Config*.* files have a proprietary format. One of the situations where .bak is created is when Retrospect Engine exits, to preserve the small amount information most important to users, such as sources, scripts and configurations. It is created from the .dat but without execution history and events, therefore much smaller. When Retrospect can't find the .dat, it automatically copy the .bak to .dat.

[peternlewis]

OK, that makes sense as to why the two files exist, and why the .bak is much much smaller than the .dat, but it doesn't really explain why you have a 200+MB file filled almost entirely of zeros that needs to be backed up every time - I suggest you investigate a better "proprietary format" for the file!

[peternlewis]

So does this mean that the .dat file can be safely excluded from the backup, and if lost, the .bak file will be sufficient for the most part?

[prl]

I back up both. Since the .bak is so much smaller than the .dat, it adds relatively little to the backup time.

[David W Lee]

So does this mean that the .dat file can be safely excluded from the backup, and if lost, the .bak file will be sufficient for the most part?

 

Yes if it is acceptable to no longer see in Console past Activities and associated logs. Those information do stay in Server's operations log until that gets too big. Also, Past Backups, Scripts, Sources, Media Sets, Storage Devices and Console Preferences are unaffected.