Craigo5000 Posted November 16, 2015 Report Share Posted November 16, 2015 I'm currently trialing Multi-server 45-day trial version 10.5.0.110 (latest as of lat month) with a Overland Neo 24 slot /LTO6 library. It work great apart from a couple of issues I'd love some help with. 1. Scheduling backups. I have an issue where a scheduled nightly backup wants to backup files which are already safely backed up to LTO. The files haven't been modified and so shouldn't be required to be backup up again. This has the potential to waste LTO media as the library will simply choose a blank LTO and and in this instance, another 749GB of data will be unnecessarily used up. I've manually scanned the backup set and even gone to the point of restoring the files the catalog thinks need backing up again. How I can ensure this doesn't happen? 2. LTO tape usage isn't being used to capacity. I've noticed there are some tape members within a backup setup with hardly any usage on them until Retrospect deemed it full for whatever reason and then chose a new tape to continue it's backup. For instance... backup set A Tape 1 - 85GB used of 1.5TB Tape 2 - 1100GB used of 1.5TB Tape 3 - 1450GB used of 1.5TB. So why wasn't the capacity of Tape 1 and 2 utilised? The files within the backup are relatively small ranging from a few KB to possibly 50GB in size (a mix of documents and media files for example). If anyone could help, that would be great. Quote Link to comment Share on other sites More sharing options...
Lennart_T Posted November 16, 2015 Report Share Posted November 16, 2015 1. It's impossible to say with your limited information. What is the source, for instance? A local drive? A network share? 2. Here is some information: http://www.retrospec...pe-capacity-faq Key facts from the FAQ: Retrospect writes to a tape until its full, regardless of claimed capacity. Some file compresses well, other not at all. If data isn't provided to the tape drive fast enough, there will be blank blocks on the tape. If there is a write error, it will show in log. Quote Link to comment Share on other sites More sharing options...
Craigo5000 Posted November 16, 2015 Author Report Share Posted November 16, 2015 Hi Lennart, The sources are varied but are always network shares. I've witnessed this on backup sets belonging to ethernet network attached storage systems (aggregated links so network transfer speeds of around 100Mb/s). Other sources are fibre attached storage arrays capable of 4GB/s. I understand LTO is capable of very fast spool speeds and will slow down or even stop in some instances during a backup before then restarting. I still don't understand when I preview my next scheduled backup, it wants to back up 749GB of old media already contained within the existing backup set of which is also searchable via the catalog for restore? Thanks Quote Link to comment Share on other sites More sharing options...
Lennart_T Posted November 16, 2015 Report Share Posted November 16, 2015 It could be something in the file metadata that changed. For instance, a co-worker of mine had three hard drives in his PC, named WinXP, Vista and Data. It was a dual boot computer. When booted in Windows XP, the drives WinXP and Data was backed up. When booted in Windows Vista, the drives Vista and Data was backed up. You would think that "Data" was already backed up, right? WRONG. EVERY file was backed up again since XP and Vista didn't have the same metadata. I have seen the same thing happening when someone had a local USB drive that was backed up. When that same drive instead was connected to NAS, every file was backed up again. So what have you done? Updated the software for the storage arrays that would affect the metadata? Changed the way of mounting the volumes? Quote Link to comment Share on other sites More sharing options...
Scillonian Posted November 16, 2015 Report Share Posted November 16, 2015 It could be something in the file metadata that changed. To expand on Lennarts's thought it could be something in the security metadata for the files and folders that is being modified. To see if this is the case try disabling the backup of file and folder security data for the backup script. In the backup script's properties dialog click Options the More Choices. The under Windows > Security uncheck options as appropriate. If disabling file and folder security backup stops the repeated backup of apparently unmodified files you will then need to try and discover what is causing the security metadata to be changed. 1 Quote Link to comment Share on other sites More sharing options...
Craigo5000 Posted November 17, 2015 Author Report Share Posted November 17, 2015 Thanks, I'll give that a try and report back. I've come in this morning to find that one of the shared NAS folders which is scheduled for nightly backups has wasted 2 x LTO6 tapes. It's added two of them as members to the backup set and only used 515.8GB on tape 3 and is at 240.7GB for tape 4. I've configured backup sets to not use hardware or software compression. I'd really like to be confident in using at least 2TB per tape or else this is a very expensive way of backing up when it's £25 per tape! Quote Link to comment Share on other sites More sharing options...
Lennart_T Posted November 17, 2015 Report Share Posted November 17, 2015 How did you manage to turn off hardware compression in the tape drive? And, more importantly, why on earth did you do that? Back in the early 90's a 40 MB (yes, megabyte) tape cost about £25. How much is that in today's values? Quote Link to comment Share on other sites More sharing options...
Craigo5000 Posted November 17, 2015 Author Report Share Posted November 17, 2015 I've added those settings changes. Also, I've analysed the log and can see the following error code at the point where the current tape was swapped by the loader with a partial backup: Trouble writing media: "3-ProductionAssets 2015" error -100 (device rejected command) Looking this up, it appears "file sharing" could be an issue or SCSI termination? The LTO6/autoloader drive is SAS connected. Quote Link to comment Share on other sites More sharing options...
Craigo5000 Posted November 17, 2015 Author Report Share Posted November 17, 2015 Cross posting sorry! When creating a new tape backup set at the naming stage there is an option only available when "tape" is chosen which allows you to toggle on/off "allow hardware data compression". The reason I did this is because LTO6 is native 2.5TB uncompressed and I ran a speed test for a day over the same backup data with and without hardware data compression. Turning it off displayed an backup speed increase of around 30% which is more desirable than the compression feature. (Where already compressed files don't re-compress very well). In 2015, post production budgets are shrunk beyond those of the early 90's. If I allow this backup set to continually waste available media (using 500GB of 2.5TB available!) on a tape then I get it in the neck from my Finance Director when I need to unnecessarily make another bulk order for 50 x LTO6 tapes (@ £1250). I need to get this figured out. Quote Link to comment Share on other sites More sharing options...
Scillonian Posted November 17, 2015 Report Share Posted November 17, 2015 Have you considered trying disk-to-disk-to-tape? Below is the link to a Retrospect white paper on this: http://download.retrospect.com/partners/sales_tools/white_papers/retrospect_dda_white_paper.pdf Quote Link to comment Share on other sites More sharing options...
Lennart_T Posted November 17, 2015 Report Share Posted November 17, 2015 Looking this up, it appears "file sharing" could be an issue or SCSI termination? Most likely, yes. In the FAQ i mentioned it says that retrospect fills the tape until the end-of-tape marker OR until an unrecoverable error occurs. A SCSI error is usually unrecoverable. Quote Link to comment Share on other sites More sharing options...
Craigo5000 Posted November 17, 2015 Author Report Share Posted November 17, 2015 Thanks for the link. I guess it's something to consider... Out of interest, I'm in limbo now with this backup set. It contains 4 tape members and tapes 3 and 4 I know I can erase and set them to "forget" as the media is definitely duplicate media that exists on tapes 1 and 2. How would one go about completely removing the members tapes 3 and 4 from the backup set so that on the next scheduled run, it picks an erased tape and automatically calls it tape 3 and not tape 5? I can't rebuild the backup set as I need two LTO6 drives to perform this task. Will a catalog repair do this job? Thanks Quote Link to comment Share on other sites More sharing options...
Craigo5000 Posted November 17, 2015 Author Report Share Posted November 17, 2015 How would one ensure SCSI errors never happen? This is a brand new autoloader and tape drive running on a vanilla version of Windows 7 on a very capable HP Z600 far exceeding the minimum system requirements. Sorry to pester - I just need this to work reliably asap. Quote Link to comment Share on other sites More sharing options...
Lennart_T Posted November 17, 2015 Report Share Posted November 17, 2015 Thanks for the link. I guess it's something to consider... Out of interest, I'm in limbo now with this backup set. It contains 4 tape members and tapes 3 and 4 I know I can erase and set them to "forget" as the media is definitely duplicate media that exists on tapes 1 and 2. How would one go about completely removing the members tapes 3 and 4 from the backup set so that on the next scheduled run, it picks an erased tape and automatically calls it tape 3 and not tape 5? I can't rebuild the backup set as I need two LTO6 drives to perform this task. Will a catalog repair do this job? Thanks That is a problem. I can think of three options. 1. You could mark tapes 3 and 4 as "lost" and erase them. But Retrospect would still ask for a blank tape for member 5. 2. You could perform a recatalog and tell Retrospect that tape 2 is the last tape. Meaning, enter tape 2 when asked for the last tape and answer "no" to the question about additional tapes. 3. If you have a USB-3 port on the server, buy a cheap 4 TB USB-3 drive and transfer specific snapshots to a temporary disk media set on the new drive. Then transfer them back to a new tape backup set (just for good measure. When everything is working fine, "forget" the current set and reuse the tapes.) Quote Link to comment Share on other sites More sharing options...
Lennart_T Posted November 17, 2015 Report Share Posted November 17, 2015 How would one ensure SCSI errors never happen? This is a brand new autoloader and tape drive running on a vanilla version of Windows 7 on a very capable HP Z600 far exceeding the minimum system requirements. Sorry to pester - I just need this to work reliably asap. That's the million dollar question. Even brand new cables and contacts can have defects, but that's not likely. Check that all cables are properly seated in the contacts. Check the drivers for the SAS card in the server. Check the firmware on the SAS card in the server. Check the firmware in the tape loader (and its SAS card, if applicable). Check the firmware in the tape drive. No worries about pestering. I know how it can be, struggling to something working that just SHOULD work. Quote Link to comment Share on other sites More sharing options...
Craigo5000 Posted November 17, 2015 Author Report Share Posted November 17, 2015 That is a problem. I can think of three options. 1. You could mark tapes 3 and 4 as "lost" and erase them. But Retrospect would still ask for a blank tape for member 5. 2. You could perform a recatalog and tell Retrospect that tape 2 is the last tape. Meaning, enter tape 2 when asked for the last tape and answer "no" to the question about additional tapes. 3. If you have a USB-3 port on the server, buy a cheap 4 TB USB-3 drive and transfer specific snapshots to a temporary disk media set on the new drive. Then transfer them back to a new tape backup set (just for good measure. When everything is working fine, "forget" the current set and reuse the tapes.) That is a problem. I can think of three options. 1. You could mark tapes 3 and 4 as "lost" and erase them. But Retrospect would still ask for a blank tape for member 5. 2. You could perform a recatalog and tell Retrospect that tape 2 is the last tape. Meaning, enter tape 2 when asked for the last tape and answer "no" to the question about additional tapes. 3. If you have a USB-3 port on the server, buy a cheap 4 TB USB-3 drive and transfer specific snapshots to a temporary disk media set on the new drive. Then transfer them back to a new tape backup set (just for good measure. When everything is working fine, "forget" the current set and reuse the tapes.) Thanks Lennart, I've chosen option 2. Having "forgot" tapes 3 and 4, Retrospect prompted me to select the last tape in the set, tape 2. I made a copy of the original .rbc catalog and moved it somewhere safe as Retrospect wanted to overwrite the existing .rbc. Will it not want to also rebuild the catalog using tape 1 too? I need to ensure this rebuilt .rbc references all media in the backup set. Thanks Quote Link to comment Share on other sites More sharing options...
Lennart_T Posted November 17, 2015 Report Share Posted November 17, 2015 There is a thing called "fast catalog rebuild", which is "on" by default. A copy of the catalog file is then placed at the beginning of each tape. So tape 2 "knows" what's on tape 1. (Tape 3 would "know" what's on tape 1 and 2.) Quote Link to comment Share on other sites More sharing options...
Craigo5000 Posted November 17, 2015 Author Report Share Posted November 17, 2015 There is a thing called "fast catalog rebuild", which is "on" by default. A copy of the catalog file is then placed at the beginning of each tape. So tape 2 "knows" what's on tape 1. (Tape 3 would "know" what's on tape 1 and 2.) Ahh. I knew about that, but assumed since I'd effectively erased Tape 4 and set the backup set member to "missing", the fast-build facility would no longer function. I didn't realise it worked in that manner. That's good. Quote Link to comment Share on other sites More sharing options...
Craigo5000 Posted November 18, 2015 Author Report Share Posted November 18, 2015 Just updating to this with success on the RBC rebuild from the latest member. The rescanning of Tape 2 took about 3 hours and then left me with a redefined backup set with only the two associated members. Hopefully with my refined security backup settings, I shouldn't get any further huge duplication and have to repeat the above process. Thanks for the help Lennart and Scillonion. Quote Link to comment Share on other sites More sharing options...
Craigo5000 Posted November 20, 2015 Author Report Share Posted November 20, 2015 Hmm. Not quite. I came in this morning to find the scheduled backup set has only used 500GB on tape 3 (of 2.5TB available) and then picked another member to add to the backup set where the rest of the 25GB was allocated in order to complete the scheduled backup. Here's the log: - 20/11/2015 01:00:00: Copying Productions on assets 20/11/2015 01:02:21: Found: 32,121 files, 4,617 folders, 2,300.0 GB 20/11/2015 01:02:22: Finished matching 20/11/2015 01:02:22: Copying: 158 files (523.6 GB) and 0 hard links E-mail notification failed: error -507 (incorrect password) Trouble writing: "3-ProductionAssets 2015" (3899326464), error -100 (device rejected command) - 20/11/2015 04:01:33: Verifying Productions on assets Trouble writing media: "3-ProductionAssets 2015" error -100 (device rejected command) - 20/11/2015 05:01:31: Copying Productions on assets 20/11/2015 05:09:19: Building Snapshot... 20/11/2015 05:09:21: Copying Snapshot: 1 files (8,513 KB) 20/11/2015 05:09:25: Snapshot stored, 8,513 KB 20/11/2015 05:09:25: 1 execution errors Completed: 158 files, 523.6 GB Performance: 2907.2 MB/minute Duration: 03:13:33 (00:09:07 idle/loading/preparing) - 20/11/2015 05:09:25: Verifying ProductionAssets 2015 20/11/2015 05:13:19: Execution completed successfully Completed: 158 files, 523.6 GB Performance: 9133.7 MB/minute Duration: 00:59:45 (00:01:02 idle/loading/preparing) Quote Link to comment Share on other sites More sharing options...
Lennart_T Posted November 20, 2015 Report Share Posted November 20, 2015 Looks like you still have problems. Most likely a SAS error. Did you check the firmware? Not only against Windows Update, but directly against the vendor's web site. Windows Update may lag years behind sometimes. Quote Link to comment Share on other sites More sharing options...
Scillonian Posted November 20, 2015 Report Share Posted November 20, 2015 Also check in the Windows Event Viewer (available in Administrative Tools from Control Panel) to see if there are any entries related to the hardware and drivers at the time Retrospect logged the error. If there are any entries they may give some indication of where the error occurred. 1 Quote Link to comment Share on other sites More sharing options...
Craigo5000 Posted December 11, 2015 Author Report Share Posted December 11, 2015 Both good points. I did check all firmware. I actually updated the IBM LTO6 drive firmware and autoloader to their latest and greatest before putting this into service. They were both shipped to me one revision out of date. After spending some time rebuilding the catalogue as mentioned earlier, a few days passed by and the loader complained that the drive needed a clean. I ran a cleaning tape through it and it's been fine ever since. I can only conclude from the error console that the "trouble writing" was referring to dirt/dust on the tape head? (which is brand new) Thanks Quote Link to comment Share on other sites More sharing options...
Craigo5000 Posted January 28, 2016 Author Report Share Posted January 28, 2016 *update* I've since had my shared storage LTO backup waste tapes on two occasions now. At around the 400GB and 600GB mark of 2.5TB tapes. I recatalogued off the previous last tape in the set and wiped the wasted tape so it could be used for something else. Since then, I've decided to give Disk-Disk-Tape a go as a method to stop this tape wastage. here is my workflow: Data to be backed up - 2.5TB NAS shared to all clients for general use. I've hooked up a new 40TB fiber DAS which is also available on the ethernet network. I've created a folder on here for the 2.5 TB NAS share to be backed up by Retrospect on a nightly bases (with a 3TB capacity). The LTO backup process then backs up from the DAS Retrospect backup. The DAS backup is all obviously in propriety language but I can search the DAS backup nicely via Retrospect and restore anything required. The problem is, this archive will just grow and grow. How do i cull it, but know that I have EVERYTHING on the LTO archive? Thanks Quote Link to comment Share on other sites More sharing options...
Lennart_T Posted January 28, 2016 Report Share Posted January 28, 2016 I think you should be looking at "grooming", as it is called in Retrospect parlance. Set the DAS backup to keep the (say) last 5 backups. Schedule a groom script to run during non-business hours, such as early Sunday morning. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.