Retrospect App Not Responding Periodically

[PGalati]

I am experiencing a situation where the Retrospect app on my laptop tends to go unresponsive for sometimes up to a minute before becoming available again for a few moments.  It seems each time it writes to the 36GB catalog file on a 16 Intel Zeon core Mac Pro with SSD, the remote app on my MacBookPro gets the spinning beach ball for a considerable amount of time.  I can continue working in other apps without delay but simply have to wait for Retrospect to return.  The engine does continue to function because when the app returns to life, it may be involved in another workstation backup already.

I am not sure why the catalog file is so large.  I am backing up roughly 30 workstations and 1 Mac OS X server once a day to a single catalog to avoid duplication between server and workstation.  This catalog file was created on Sept. 19th 2017.  Obviously with an empty catalog file, the response times for the app were quick.  As time passes and the catalog takes on some data, the not responding starts to get longer and longer, probably due to the growing size of the catalog.  Currently the catalog does get compressed and no grooming is configured.  All data is saved on hard drives, no tape or optical.

Is there something that could be done to reduce the size of the catalog file?  Do most admins use separate catalogs for each client?  Are there articles that describe best case scenerios based on computer environment config?

Thanks for your time.

[twickland]

Let's begin with clearing up the terminology. Retrospect stores backup data in Media Sets. The catalog file simply keeps track of the data that is in the Media Set. The greater the number of files, folders, and backups a given Media Set contains, the larger its catalog file will be.

It sounds like you are using a Disk Media Set. It's usually best for the catalog for a Disk Media Set to be stored on a different volume than the data. Grooming enables you to recover space on the data volume by deleting older backups. Keeping the Media Set's catalog file compressed enables you to save space on the disk volume where the catalog file is stored, at the price of whatever performance hit is caused by the need to repeatedly compress and expand the catalog.

I'm curious how large your Media Set is, both in the number of files and in the data itself. 36 GB seems like a pretty big catalog file We use both disk and tape Media Sets to back up 38 source volumes, and for a tape set that has around 25 million files, the catalog file is around 11 GB; for a disk set containing 6 million files, the catalog file is around 9 GB. (We don't keep any of our catalogs compressed.)

As always, the more information you can give us about your setup, including OS and software versions, the better we can help.

[PGalati]

Thanks for the response and sorry for the delay.  I don't think I remember seeing the notification of a reply to my post.  Anyway...

The catalog file at this point is reaching 54GB in size.  The media set size is 6TB containing 1.4 million files across 15 members.

Each time I start over and essentially create a new media set, everything works very well initially.  As time goes by the beach ball starts to reveal itself as the catalog file, media set size, and file count start to increase.  My guess is it is the sheer fact of writing a 56GB catalog file after each workstation backup.  Even thought the catalog is on an SSD, it still takes quite awhile.  I just timed it and I get the spinning beach ball for just about 10 minutes before the app becomes responsive again, only to go unresponsive moments later as it connects to the next workstation to look for changes to back up.  My past backups currently show 4066 since Sept 19, 2017.  Each workstation is backed up M-F, once a day.