Jump to content

Backup drive is maxed out, yet master drive is no where near capacity.


Recommended Posts

Hi all,

I'm very new to using retrospect so I apologize if I don't make much sense. Awhile back my local back up was interrupted and I got an option saying "choose media". I tried doing a fast catalog rebuild with no luck. After talking on the phone with support, I was advised to do a thorough rebuild. I had no luck with that either. 

It appears that my back up drive is maxed out on capacity... yet my master drive is no where near capacity. I have no idea what is taking up all the extra storage and why this started happening. Rebuilds and grooming have done nothing to clear up space. 

I'm also noticing an "error 1,017 (insufficient permissions) in the log

Any help is appreciated.

Link to comment
Share on other sites

How much data are there on your source drive?

How large is the backup drive?

Have you repaired the backup drive with Apple's Disk utility?

What version of Retrospect?

What version of Mac OS?

To see what has been backed up:
Click on "Past Backups" and then on "Browse" for the backup in question.
1715297930_Skrmavbild2021-12-16kl_21_43_02.thumb.png.4af2f6a94399bb485ed5dc103279728d.png

Click on the checkbox at lower left and then on the disclosure triangle to see what was backed up. 1122912776_Skrmavbild2021-12-16kl_21_42_38.thumb.png.0f6d50be5ad674b0f7c71dc82685615f.png

 

Link to comment
Share on other sites

Hi Lennart, thanks for getting back to me.

I have roughly 50TB on my source drive.

Back up drive is maxed out at 85TB

I have not repaired the backup drive via the Apple Utilities Disk; how would I go about doing so and what does this do?

I have the latest version of Retrospect, though I did start encountering these problems with their previous update. I'm running MacOS Catalina currently.

I keep getting the "choose media" notice when trying to backup, even after doing a thorough rebuild.
298865379_ScreenShot2021-12-16at3_25_52PM.png.15c56189b4a9dcb61dca86669e8f4196.png

Link to comment
Share on other sites

How big is your backup set? Retrospect has a maximum size for the "virtual media" used when doing a disk backup -- around 8.5TB IIRC. Once your backup set reaches that limit you'll be asked to "Choose Media" and RS will create a new folder for the next piece of "virtual media".

The tricky bit is choosing the right place in the "Choose Media" dialog. On the volume you are storing your backups there will be a directory called "Retrospect" containing a directory for each of your disk backup sets. You need to choose the volume (or directory) that contains the "Retrospect" directory to add the next piece of "virtual media".

I've a mounted volume called "Backups" containing the folder "Retrospect" which contains "2017" (the disk backup set) which, in turn, contains "1-2017", "2-2017", "3-2017"... So in the "Choose Media" dialog I need to select "Backups" -- RS does the rest.

Another problem might be permissions. You say "local backup" -- is this directly attached storage or are one or more of the volumes concerned remote (eg mounted NAS volumes)?

Link to comment
Share on other sites

On 12/17/2021 at 12:38 AM, phalmos said:

Hi Lennart, thanks for getting back to me.

I have roughly 50TB on my source drive.

Back up drive is maxed out at 85TB

I have not repaired the backup drive via the Apple Utilities Disk; how would I go about doing so and what does this do?

I have the latest version of Retrospect, though I did start encountering these problems with their previous update. I'm running MacOS Catalina currently.

I keep getting the "choose media" notice when trying to backup, even after doing a thorough rebuild.
298865379_ScreenShot2021-12-16at3_25_52PM.png.15c56189b4a9dcb61dca86669e8f4196.png

Let's see here.

You mean 50 GB on your source drive (not TB), right? Otherwise you must have a small stack of external disks to store 50 TB.

As for your backup drive: I asked the size of the drive and 85 TB is a slightly larger stack of disks. And if you mean 85 GB? Well, you can't buy disks with that size. So what size is the disk really?

Repairing the backup disk means fixing any errors in the disk directory. Think of it as the index of a book. If the index gets corrupted it would be difficult to find the page you are looking for.
How to repair: https://support.apple.com/en-us/HT210898

Link to comment
Share on other sites

On 12/18/2021 at 7:05 AM, Lennart_T said:

Let's see here.

You mean 50 GB on your source drive (not TB), right? Otherwise you must have a small stack of external disks to store 50 TB.

As for your backup drive: I asked the size of the drive and 85 TB is a slightly larger stack of disks. And if you mean 85 GB? Well, you can't buy disks with that size. So what size is the disk really?

 

Thanks for getting back to me, Lennart. My office has two 24 drive RAID 10's (one for master, one for backup) made up of 8 TB hard drives. So we do indeed have 50 TBs of material on our master drive with a max capacity of 85 TB. 

Our backup drive somehow got maxed out at 85 TB even though our master drive only has 50 TB. I've done multiple rebuilds (fast and thorough) and groomed my media set with no luck of clearing out any space. Every rebuild takes between 1-2 weeks to complete which makes this all the more frustrating. 

I've also tried repairing the backup disk via the link you sent and everything seemed fine. 

I've been reading about recycling a media set in the retrospect manual. I'm hesitant to try this because it seems that it will delete the entire catalog file but at the same time I feel like I don't have much of a choice. I'm very knew to all this so I don't want to make any rash decisions without knowing what I'm getting myself into. Is there a way to save my catalog elsewhere on my computer so that I have a copy of it in case I try to recycle my current media set?

Hope this makes sense. Any help is much appreciated.

Link to comment
Share on other sites

I see. Getting a RAID (or two) into the equation makes it more complicated. I have never used Retrospect with so much data.

How many backups do you have stored on the 85 TB disk? How much data on the 50 TB source gets changed daily (or weekly)? What kind of files are they? Database files that gets updated by the minute? Or video files that "never" changes? 

What groom settings did you use when you groomed?

Link to comment
Share on other sites

It looks like we have 38 past backups on our backup disk. 

Hard to say how much material changes on our master drive. We were running local back ups every 2 hours, 7 days a week, and 1 cloud back up per week before running into this issue. The files are mostly jpgs, tiffs, raw photos, psd, with little to no video files. There hasn't been much new material being put on our computer with everything winding down for the holidays, plus our current backup issues.

I'll attach a screenshot with my groom settings:

1557916502_ScreenShot2021-12-20at11_40_05AM.png.4f3fca4a1610e1f6d42b37f0168b9fd6.png

These we're pretty much the settings I was left with when I was hired on and our old employee left. The only thing different is that we're now set to Performance-optimized grooming that RS support instructed us to do. Before that it was set to storage-optimized grooming.



 

Link to comment
Share on other sites

Retrospect creates backup files that are around 650MB in size.

With performance-optimized grooming, they are all kept intact until all its contents can be groomed out. This often means that hardly any data is groomed out and that can be the source of your problem.

With storage-optimized grooming these files are shrunk as old backups are removed from them, one by one. 

Link to comment
Share on other sites

19 hours ago, phalmos said:

Hard to say how much material changes on our master drive.

You can guesstimate it if you've logged the used capacity of your source volume over the last few weeks -- that'll give you the amount of new data. Then the sum of all incremental backups over that time minus the amount of new will give an idea of your churn.

But if you've got 40TB on the source, even only 0.5% churn a day plus some new stuff will fill your 80TB target -- especially since, as Lennart says, performance optimised grooming can leave a lot ungroomed.

General recommendation for target volume capacity is 2-3 times the source capacity, assuming "regular" usage -- obviously you can get away with less if your data is mainly archival in nature and so doesn't change much. It might be a good idea to go through what was backed up each time (see Lennart's screenshots above) to see what's happening -- keep an eye out for things like database files (sounds like you have a lot of images, perhaps you are cataloguing them somehow?) where a small change will mean a multi-GB whole file backup.

Link to comment
Share on other sites

  • 1 month later...

I figured out a solution so I figured I'd let everyone on here know what I did incase anyone is faced with the same problem as me.

All I did was go into my Retrospect catalog and deleted a whole bunch of old .rdb files. We had files going all the way back to 2019. I deleted all .rdb files from 2019 and 2020, making sure that there was a decent gap in time between my last .rdb file I deleted and the first .rdb file that I left in the catalog (hope that makes sense). This cleared up almost 30 TBs of space. Now we are able to backup no problem. I know that this is probably not the go-to solution but we felt like we needed to take some more drastic measures to get things solved.

Thank you Lennart and Nigel for all the help and advice. I appreciate it!

Link to comment
Share on other sites

The question is how the old .rdb-files got there.

One theory is that you had a backup set with the same name that you deleted (forgot) in Retrospect, did NOT delete the .rdb-files and then created a new backup set with the name and storing the member files in the same location. Does that sound feasible?
In that case it would have been better to recycle the backup set and Retrospect would have deleted the old files for you.

 

Link to comment
Share on other sites

On 2/1/2022 at 10:58 AM, Lennart_T said:

The question is how the old .rdb-files got there.

One theory is that you had a backup set with the same name that you deleted (forgot) in Retrospect, did NOT delete the .rdb-files and then created a new backup set with the name and storing the member files in the same location. Does that sound feasible?
In that case it would have been better to recycle the backup set and Retrospect would have deleted the old files for you.

 

That could be a possibility. I'm still fairly new to the job, only been here for 6 months and Retrospect was handed off to me by my predecessor. I guess this could have happened before I got hired and our backup drive was just slowly filling up without anyone noticing (I personally didn't even know what to look out for at the time because I was so new and inexperienced). I basically had to learn Retrospect as I went while trying to solve this problem.

Everything seems to be working fine now; our backup drive matches our master drive and all backups have been running smoothly for the past week.

  • Thanks 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

×
×
  • Create New...