philrichardson Posted January 3, 2008 Report Share Posted January 3, 2008 Hello I have a few questions and need some advice on whether things are possible with Retrospect. I run a small post-production house in London and we have been using Retrospect Workgroup for some time to lay off to tape projects that have been completed. We are entirely Mac based and have one workstation connected via wide SCSI to a Sony Storstation and AIT-4 drive. All works lovely. As our company is expanding a little and we deal with larger quantities of data, we need to implement a weekly backup schedule that will create a backup set once a week and take nightly snapshots of our storage. As we deal with HD Video footage the file sizes can be immense. If we were to capture several hours of footage, the snapshot could be upwards of several Terra Bytes. If this was written to tape, we would be hanging around all week for it to complete. So i thought, can Retrospect write that amount of data to a desktop mounted volume? A volume that is dedicated to backup sets and snapshots. A Volume which will be made up of a dedicated Raid. Surely this would be much quicker? We are going to implement a SAN soon so i would want it to be able to operate across mounted SAN Volumes too. At present, the server would co-mount (!?) the storage for our suites and run backups of their various storage. Does that sound possible or do i have spend £27,000 on Netvault software that has limitless virtual drive capacity - That aint happening on our turnover! Any help or advice would truly be gratefully received. Thank you. Phil Quote Link to comment Share on other sites More sharing options...
Mayoff Posted January 3, 2008 Report Share Posted January 3, 2008 When working with very large amounts of data, I would recommend using Retrospect for Windows to backup your Mac systems. The Windows version can handle more files, bigger backup sets and is faster. Quote Link to comment Share on other sites More sharing options...
CallMeDave Posted January 3, 2008 Report Share Posted January 3, 2008 Quote: If we were to capture several hours of footage, the snapshot could be upwards of several Terra Bytes... So the gist of your question is, how to backup multi-TB of data to hard drive storage? In Retrospect for Macintosh there are two Backup Sets that can be written to hard drives; File and Removable Disk. A File Backup Set stores all data in a single file (with an accompanying catalog file) on a single logical volume. It works with locally mounted volumes, and on volumes shared via AFP and (new with 6.1.138) CIF/SMB. I do not know how well it works with other, more modern storage choices, but my guess is if you can see it in a standard dialgog box (SFPut) you'll be able to create and use a File Backup Set. A Removable Disk Backup Set stores data on a single file per logical volume using supported "devices." In version of Retrospect before 5.1, only actual removable devices (Zip, Jaz, MO, etc) could be seen in the Devices dialog. Current version allow for any attached FireWire or USB drive to be treated as if it were a removable device. This allows for unlimited growth of the Backup Set, as long as you have unlimited hard drives to feed in. There is a limit to the size of each member of a Removable Disk Backup Set, although I don't recall what it is. A RAID 5 with a large single logical volume housing your File Backup Set would likely be the easiest method. If this is your primary backup, I'd suggest that your available space be more then twice the amount of data you have, to allow for recycling without being without a backup (you create a second File Backup Set before you recycle the first, then swap them back and forth on a regular schedule). Quote Link to comment Share on other sites More sharing options...
philrichardson Posted January 4, 2008 Author Report Share Posted January 4, 2008 Thankyou! I would rather stick with Macs though. But can you tell me what you mean by windows being able to handle bigger backup sets? This may be an issue. TA! Quote Link to comment Share on other sites More sharing options...
philrichardson Posted January 4, 2008 Author Report Share Posted January 4, 2008 Hello CallmeDave Thanks for the response. This is undoubtably a File backup scenario. Do you know of any limitations of Retrospect's file size handling for snapshots, sets etc? I shall also plug away at EMC Support. Thanks P Quote Link to comment Share on other sites More sharing options...
Mayoff Posted January 4, 2008 Report Share Posted January 4, 2008 File backup set is limited to the maximum file size in the file system, which I believe is 1 TB in size. Also, going over the network can be a problem when using large files. Bigger the file, the more chance for corruption. Older version of AFP will not allow a file larger then 2 GB to be modified over the network. Retrospect for Windows uses disk backup sets, which uses small segment sizes and the total backup set can be 1000 TB in size if you really want (I don't recommend it). Quote Link to comment Share on other sites More sharing options...
philrichardson Posted January 4, 2008 Author Report Share Posted January 4, 2008 Hello Mayoff and thank-you again for the help. The maximum permissible file size under MAC 0S X 10.4 HFS PLUS is now 8 EBytes - which seems quite generous!!! Or, have i missunderstood you and you are referring to internal file size handling in Retrospect? http://docs.info.apple.com/article.html?artnum=25557 So in theory, i should be onto more of a winner? The various volumes and MACs are connected through a 4Gb Fibre Network, although i realise that the underlying processes remain similar to ethernet. Thanks for help Phil Quote Link to comment Share on other sites More sharing options...
Mayoff Posted January 4, 2008 Report Share Posted January 4, 2008 A file backup set has either a 1 or 2 TB limit, I can not remember off hand which number it is. I believe it is 1 TB. It is a Retrospect limit for sure. The KB article from Apple is interesting. I wonder if the OS has the ability to limit the file size? Just because the file system supports it, doesn't mean the OS does. Quote Link to comment Share on other sites More sharing options...
rhwalker Posted January 4, 2008 Report Share Posted January 4, 2008 Remember, there are different APIs being discussed here (Carbon and Cocoa). I believe that Retrospect's limit is because of its code base / API. Quote Link to comment Share on other sites More sharing options...
CallMeDave Posted January 4, 2008 Report Share Posted January 4, 2008 Quote: A file backup set has either a 1 or 2 TB limit... This is something I'd like to know more about. One of the biggest things to come with Retrospect 6.0 was the breaking of The Terabyte Limit for Backup Sets. See the 6.0 ReadMe: New Features Since Retrospect 5.1 Backup sets can now store more than 1 terabyte of data Now I realize on re-reading this that the limit was for _all_ backup sets, meaning that even a tape collection could not contain more then 1 TB of stored data (Russ must have suffered with this over the years). But I have not been able to find notes on the limit to a size of a File Backup Set in 6.x. The only other KB entry I found states: Retrospect 6.0 for Macintosh and later has a limit of 1000TB per backup set. Dave Quote Link to comment Share on other sites More sharing options...
philrichardson Posted January 7, 2008 Author Report Share Posted January 7, 2008 Hello Folks! Okay, service update... I kicked off a large backupset backup over the weekend of a project on our system, which was 2.2TB in size. It's Monday morning and the backup was mostly succesful except for 12 execution errors. I shall look into these after i have carried out a restore test to a new folder. I shall report back on the success of all of this... WTS Phil Quote Link to comment Share on other sites More sharing options...
CallMeDave Posted January 7, 2008 Report Share Posted January 7, 2008 OK, so this confirms File Backup Sets larger then 2 TB. Goodie. Still, if it's possible to segment your Sources into multiple File Backup Sets (which can all be stored on the same volume) you'll decrease the chances of any issue with one large single file (bad sector on the hard drive, IO error during the copy, etc) effecting the entirety of your backup. Dave Quote Link to comment Share on other sites More sharing options...
philrichardson Posted January 9, 2008 Author Report Share Posted January 9, 2008 Indeed, a good idea and i think that would be useful. I did get 12 execution errors during the process and so i would like to avoide those in the future. Although to be honest with you, the tapes had been used before, so i am not too astonished. This was a test though, not serious paid material. Also got this from Mr Anderson at Tech Support: Quote: You can backup to a direct attached drive or a SAN drive, though if it is a XSAN that is not fully supported and in the readme for 6.1 describes the type of issues we have with XSAN units. There really is no size limitation for Retrospect though several terabytes is pretty big it’s the number of files that will eventually cause problems. Once a backup set reaches around 2 million Retrospect will start to run into out of memory errors as well as slow backups and restores. How many files are you backing up? Which is all very interesting! Thankyou all for the help Phil Quote Link to comment Share on other sites More sharing options...
jdpal Posted January 13, 2008 Report Share Posted January 13, 2008 As a side note, the 2 million limit that Mr. Anderson mentions is very important. Retrospect will not stop you from backing up more than 2 million files but you will run into problems trying to restore (happened to us). Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.