Jump to content

Nigel Smith

  • Posts

  • Joined

  • Last visited

  • Days Won


Nigel Smith last won the day on June 28 2021

Nigel Smith had the most liked content!

Profile Information

  • Location

Recent Profile Visitors

1,107 profile views

Nigel Smith's Achievements


Newbie (1/14)

  • Dedicated Rare
  • First Post Rare
  • Collaborator Rare
  • Week One Done Rare
  • One Month Later Rare

Recent Badges



  1. Are you using "Locate" to try and add the catalog, and then RS is hanging? Is there anything in the log? As Lennart says, if all else fails you can rebuild the catalog from your tapes -- at least you'll then have access to the data.
  2. What account is Retrospect running in (Preferences->Execution->Security)? What account are you logging in to Windows with? I'm no Windows guru, but I think you only see the icon when logged in with the same account Retrospect is running under.
  3. You can guesstimate it if you've logged the used capacity of your source volume over the last few weeks -- that'll give you the amount of new data. Then the sum of all incremental backups over that time minus the amount of new will give an idea of your churn. But if you've got 40TB on the source, even only 0.5% churn a day plus some new stuff will fill your 80TB target -- especially since, as Lennart says, performance optimised grooming can leave a lot ungroomed. General recommendation for target volume capacity is 2-3 times the source capacity, assuming "regular" usage -- obviously you can get away with less if your data is mainly archival in nature and so doesn't change much. It might be a good idea to go through what was backed up each time (see Lennart's screenshots above) to see what's happening -- keep an eye out for things like database files (sounds like you have a lot of images, perhaps you are cataloguing them somehow?) where a small change will mean a multi-GB whole file backup.
  4. How big is your backup set? Retrospect has a maximum size for the "virtual media" used when doing a disk backup -- around 8.5TB IIRC. Once your backup set reaches that limit you'll be asked to "Choose Media" and RS will create a new folder for the next piece of "virtual media". The tricky bit is choosing the right place in the "Choose Media" dialog. On the volume you are storing your backups there will be a directory called "Retrospect" containing a directory for each of your disk backup sets. You need to choose the volume (or directory) that contains the "Retrospect" directory to add the next piece of "virtual media". I've a mounted volume called "Backups" containing the folder "Retrospect" which contains "2017" (the disk backup set) which, in turn, contains "1-2017", "2-2017", "3-2017"... So in the "Choose Media" dialog I need to select "Backups" -- RS does the rest. Another problem might be permissions. You say "local backup" -- is this directly attached storage or are one or more of the volumes concerned remote (eg mounted NAS volumes)?
  5. A late, and perhaps silly, question -- but why are you recycling weekly rather than using incrementals? Sounds like you are generating a huge amount of extra work by doing that, and I'm not sure the gain is worth the pain.
  6. Trying not to quote Ralph Waldo Emerson, but you can't have it both ways -- either rhwalker is right and "All of the backup data (files, metadata, etc.) is in the .rdb files which, taken together, comprise Retrospect's database", or you are and they don't. I'll let you decide for yourself, and leave well alone until twickland reports back.
  7. No, you are. Yes, the .rdb files are the database and yes, the catalog file is the index. But a component of the Retrospect Engine is a database engine because that's what manipulates the database. Is it the same on Mac and Windows versions of RS? We don't know, and even if the inputs and outputs are the same we still don't know -- it's a black box. To (rather sadly) quote myself: You can see evidence of that last line for yourself by comparing Mac and Win release notes -- most fixes are in both, but a few are specific to one or the other.
  8. Don't know why -- since a database engine is what CRUDs the data in a database, Retrospect has a database, and records in that database can be created, read, etc, there's some kind of DB engine in there. I'm not saying it's MySQL or similar, but there's something. I'd hope that, with the push to a "common engine", it's the same on all platforms. But that's hope, not certainty. And anyway, it's only an example of the possible differences between nominally common code compiled to different platforms. I'm sure the meat of the RS engine is more-or-less the same, but it's the gristle round the edges that always gets stuck in your teeth...
  9. Apologies, I was less than clear (rushed, after my first attempt got eaten). I can see a lot of space in "same code but compiled for different platforms" for quite a lot of variation between the final products. The example was supposed to suggest they might have used different DB engines -- maybe for performance, licensing, or compatibility reasons. There could be lots of these variations, or very few -- I simply don't know. Your catalogs are as I'd expect, the meta holding "summary data" for the set and each client/volume (including links to their component catalogs) and the component containing the meat of that client/volume's backup history (metadata for every file backed up in every session, etc).
  10. I think the "meta catalog" just stores the "higher level" info -- client/volume pairs in the SG, when each was last backed up -- and the "component catalogs" store the session info data. In Mac RS the search "chains" -- in the GUI you search the meta cat, but RS searches each of the component cats then aggregates the results for presentation. I'd hope that Win RS does the same and that the manual's "To restore from a specific source, you need to select the specific source within the Storage Group" is an "and you can restore from a specific source..." rather than "you can only...", but I can't test that at the moment. Assuming that's the "universal Retrospect engine", that still leaves an awful lot of wiggle room! Same design and pseudo-code? Same actual code but compiled per platform? Even if it's the latter ( as I suspect, comparing version fixes across platforms) that still leave space for things like "if macOS then mac_db_code else win_db_code". Not saying they have, but we can't discount the possibility!
  11. Agree about the Transfer and Verify (although Rules are hardly irksome), but I don't understand about Restore -- sessions are listed by Machine + Volume + Media Set, so it's trivial to do a point-in-time (or Browse and select particular files) for any client/volume pairing in any Storage Group. In Windows it's actually more clicking to get to previous snapshots/sessions. I've not tried it, but in Windows do you have to restore from a particular source? Can you easily search for every occurrence of "Important.docx" across a Storage Group, or would you have to search (and restore from) each "component set" individually? As you rightly say, it's potential in improving Rebuilds is massive, if it can be done. I'm not sure how similar Mac and Win RS actually are! There may be more to it than just how their respective consoles allow you to view what's under the hood. Creating Storage Groups by splitting a single over-arching catalog into a "meta catalog" plus a bunch of client/volume "component catalogs" was a very pragmatic solution to a pressing problem -- whether that could be done the same way on both platforms is something way outside my understanding. What we really need is a preference so we can switch both Mac and Win consoles to our preferred method of display 😉
  12. I know what RS does at the filesystem level. But in the UI, which is where I suspect most of us interact with RS most of the time, it is presented as "just another media set" on the Mac and as something different in Windows. Again, I prefer the Mac software's consistent approach -- you obviously don't. And I can't disagree with your preference since Storage Groups aren't "just another media set" (eg lacking cross-client dedup within the set). I'd prefer it even more if there was a consistent approach, one way or the other, across platforms -- but that's probably asking too much. It may have changed in v18, but in v17 you had to rebuild the entire Storage Group on both platforms, even if only a single "component catalog" was damaged. A pain, but at least the rebuild is also parallelised across multiple threads. See this old post for my workarounds, either of which could possibly be adapted to twickland's particular problem.
  13. Trust me, it isn't! Agreed. But whether Mac or PC handling of them is correct (c'mon -- Mac wins every time!) is a matter of perspective. Looking top-down, a media set only has one catalog. A Storage Group only has one catalog. It looks like any other media set in the RS UI, and the manual even states it should be treated the same as any other media set. So it *shouldn't* show the component directories, since no other media sets do. When it comes to UX and abstraction, consistency trumps absolute accuracy every time. And why display something that shows no usable purpose? I don't know, and I don't really care, why Mac and Win do this differently. I know which I prefer, so do you, but those are only opinions and neither of us is "right". While I would hope that you are right about this, I'd recommend that anyone who wants to use this in production thoroughly tests it first. There are limits on destination volume size -- my "use as much space as you want" Disk Media set splits every 8TB, for example -- and I'd want to be sure that the above allowed you to separate across "proper" logical/physical volumes rather create more RS "media set" volumes in the same place. Nice work on the password protection checking!
  14. Which just goes to show how tricky UI design can be because, IMHO, a Storage Group should be treated like any other media set -- those don't show internal client/volume pairs, so why should a Storage Group? Mac consistency, FTW! I think the generic use-case for parallelising backup clients is that "the server processes data faster than a single client supplies it". You can parallelise with multiple scripts, each with its own destination set, or with one or more scripts writing to one (or, less often, more) Storage Groups. My personal experience suggests that for "local" clients (those your RS Server can direct-IP or multi/subnet broadcast to wherever they are in the world), multiple scripts/sets is best. The benefits to me are organisational, de-dup within sets, faster catalog rebuilds when things go wrong (because you only have a subset of clients per media set), etc. But if you have "remote" or hybrid-working clients then Storage Groups are the way to go, mainly because (as at my last set of tests, anyway) there is no way to back up different remote clients to different media sets -- it's "one big bucket", hence the slower catalog rebuilds, although there's no de-dup (even across volumes of the same client). The only way to parallelise write operations to a single set is by using a Storage Group. And having multiple remote clients, which often have restricted upload speeds, are when parallelising on the server can really help you get all your backups done. As always, YMMV.
  15. Try the newest version of Retrospect. If you can, try v17 as well, on that same machine. The more relevant information you can send to Mayoff & team the better (your star sign probably isn't important at this point 🙂 )! You could also try a simple Finder Copy between the two images, just to check that works as expected. Certainly a Copy script should maintain the copied files' metadata, and it does on my tests with macOS 10.15.7, RSv17, and mounted encrypted disk images. But there's always a chance that something's been changed on the OS side, and it's just that you're seeing the effects in RS...
  • Create New...