Jump to content


  • Content count

  • Joined

  • Last visited

Everything posted by cmcfarling

  1. cmcfarling

    Open File Backup

    Is the Open File Backup add-on required to backup open files when using Multi Server on Windows? For example, if Multi Server is installed on a Windows 2008 server, is the Open File backup license required in order to create a full back of the local drive that would be useful for disaster recovery?
  2. cmcfarling

    Open File Backup

    So more specifically. Does Multi Server utilize VSS when doing a backup if the Open File Backup add-on is not active? I tested this and a ton of required system files were not backed up due to the "sharing violation" error. Below are a few lines from the log. Wouldn't there be no sharing violation errors if VSS was being used? File "C:\Boot\BCD": can't read, error -1020 (sharing violation) File "C:\Boot\BCD.LOG": can't read, error -1020 (sharing violation) File "C:\System Volume Information\DFSR\FileIDTable_2": can't read, error -1020 (sharing violation) File "C:\System Volume Information\DFSR\SimilarityTable_2": can't read, error -1020 (sharing violation) File "C:\System Volume Information\DFSR\database_D8A8_B3A8_A8B3_8392\dfsr.db": can't read, error -1020 (sharing violation) File "C:\System Volume Information\DFSR\database_D8A8_B3A8_A8B3_8392\fsr.log": can't read, error -1020 (sharing violation) File "C:\System Volume Information\DFSR\database_D8A8_B3A8_A8B3_8392\tmp.edb": can't read, error -1020 (sharing violation) File "C:\Users\admin\NTUSER.DAT": can't read, error -1020 (sharing violation) File "C:\Users\admin\ntuser.dat.LOG1": can't read, error -1020 (sharing violation) File "C:\Users\admin\ntuser.dat.LOG2": can't read, error -1020 (sharing violation)
  3. Due to Retrospect's imperfect grooming feature, disk based backup sets don't always get groomed thoroughly causing disk backup sets to fill up. Once this happens the catalog must be rebuilt in order to perform a successful groom operation. I've had to go through this procedure several times in the last 18 or so months. Normally the rebuild completes with no problems. However, I have a catalog now that will not rebuild. I'm using the Repair Catalog >> Recreate from disks command to recreate the catalog from the disk based backup data. Once Retrospect gets to the point where it begins to read the backup data from the disk, the following error come up: Device trouble: 1-Production2, error -105 (unexpected end of data) The disk is good working order. It is used for backup data for several other backup sets as well, all of which are functioning without problems. I've run chkdsk on the disk also and it checks out fine. Does anyone have any experience with this particular problem? Any suggestions for getting this catalog to rebuild at this point without having to wipe out all of the existing backup data and start over from scratch?
  4. No, there is no reference to any .rdb file.
  5. I changed the logging level to 7 but did not get any additional info. I then tried changing the log level of Engine, Trees and Volumes, Backup Sets and Catalog Files and Networking to 7 as well, which still did not generate any useful info in the log. The only additional entry that appeared was xipSetDevice: can't seek, error -105 (unexpected end of data) Any other suggestions?
  6. As usual, grooming has failed to work as expected. On 7/21 the grooming policy dictated that an automatic grooming should be done on one of my backup sets. As is common, it failed with error -1101 (file/directory not found). I then started the normal procedure of recreating the catalog. After that ran for 13:42:43, I then ran a manual groom operation on the backup set. That proceded to run for 12:41:17 before it failed with this error: Grooming Backup Set Production1 failed, error -2242 (Catalog File duplicated or ambiguous) You must recreate the Backup Set's Catalog File. See the Retrospect User's Guide or online help for details on recreating Catalog Files. Can't compress Catalog File for Backup Set Production1, error -1 (unknown) Sweet!! So here we go again, looks like I'm in for another day of rebuilding/regrooming and keeping my fingers crossed. Meanwhile data hasn't been backed up for 4 days. Are any of these well documented grooming issues addressed in 7.6 I wonder?
  7. cmcfarling

    Grooming failed, again

    Yes, "Keep the Catalog File compressed" is selected for the backup set, and all of my backup sets for that matter. It always has been and I've never seen this error before, through many catalog rebuilds. Storage space is not an issue here, there is plenty. For an excruciatingly detailed overview of my setup see these posts: http://forums.dantz.com/showtopic.php?tid/26204 http://forums.dantz.com/showtopic.php?tid/26354
  8. I've been getting this error message intermittently. Here's a log snippet: 1/31/2008 8:04:27 PM: Copying Web on C on server02 1/31/2008 8:04:27 PM: No files need to be copied 1/31/2008 8:04:35 PM: Snapshot stored, 30 KB 1/31/2008 8:04:46 PM: Execution completed successfully Duration: 00:00:18 (00:00:09 idle/loading/preparing) 1/31/2008 8:04:48 PM: Copying backup on C$ on server01 Scanning incomplete, error -1020 (sharing violation) 1/31/2008 8:04:49 PM: Copying CTuner Setups on C$ on server09 Scanning incomplete, error -1020 (sharing violation) 1/31/2008 8:04:50 PM: Copying ORIS on Output on server09 Scanning incomplete, error -1020 (sharing violation) I'm curious as to why it's failing on the scanning phase. I talked to tech support and their explanation was this... When Retrospect is scanning the folder, one or more of the files in the folder is in use by another process. As a result Retrospect is deciding not to backup the entire folder. He went on to explain how an application can communicate directly with Retrospect to inform it of files that are in use. To the best of my knowledge that may be the case if I were using Open File Backup, but I am not, which I explained to him. On top of that the source folders involved wih these 1020 errors are always SMB shared folders on a different server than the backup server. Even with OFB, if Retrospect is running on server02 and wants to backup an open file via SMB on server01, is there any way for server01 to allow that to happen? His explanation doesn't really make any sense to me. Why would Retrospect skip an entire folder if (potentially) just one file in that folder is in use? Why would it not go ahead and backup the files that it can and skip the file(s) that have an open file lock on them? Was I perhaps misinformed (again) by tech support or is that really how Retrospect works? To note, All servers involved are running Winows 2000 Server. If I go to Configure>>Volumes, I can browse any of the remote folders with no problems. Again, this happens intermittently. Sometimes these source folders backup with no errors. It seems to me that there is something else wrong and that this is not just standard behavior.
  9. I've been posting to another thread some issues I've had with Retrospect Multiserver 7.5.387. http://forums.dantz.com/ubbthreads/showflat.php/Cat/0/Number/104567/an/0/page/ While troubleshooting those problems I've come accross a couple other ones so I started this thread. On page 257 of the UG the following is stated under the Snapshots Tab section: "When you forget a snapshot from a disk backup set with grooming enabled, Retrospect deletes the selected snapshot and it's associated files." page 258: "NOTE: Snapshots with the lock icon are protected from grooming and cannot be groomed until they are unlocked. They can be forgotten." "WARNING: Once you click forget, Retrospect *will* groom the snapshot and it's associated files" So let's say I open the Snapshots tab of a backup set. I then highlight a single locked snapshot and click Forget. A confirmation dialog box pops up and I confirm that I want to forget the snapshot. I then close the backup set window and the following warning message is displayed: "The snapshot(s) you forgot will be groomed from the backup set to reclaim disk space, when do you want to groom?" [Now] [Later] Wait a minute, didn't to UG say that locked snapshots won't be groomed? Well I guess it also said that once you click forget, Retrospect *will* groom the snapshot and it's associated files. It would seem that if a locked snapshot was forgotten there should be no message coming up claiming that it's going to be groomed. I guess that message just comes up no matter what. I think Retrospect should be smart enough to know if it's really supposed to groom or if it's supposed to just forget the snapshot. So back to the message. Since my only options are Now or Later I click Now. At that point nothing happens, which is really what I wanted anyway, I didn't want it to actually groom anything. So now let's say I do want to groom a snapshot using this method. Back in the snapshot tab I highlight a snapshot, this time one that is not locked. I click Forget, the confirmation box comes up and I confirm. I close the backup set window and again see "The snapshot(s) you forgot will be groomed from the backup set to reclaim disk space, when do you want to groom?" [Now] [Later] I click Now but this time nothing happens either. There is no grooming operation initiated, although this time I did want a groom operation to occurr. No Event or History entry either btw. So basically this whole mechanism for kicking off a grooming operation from the snapshots tab has both some UI problems and some funtional problems. On top of that the documentation could probably stand to be a little clearer. On a similar note, one is supposed to be able to initiate a groom operation from the option tab by changing the grooming policy parameters. For example, say the grooming policy is set to Restrospect defined policy. If I change that to a user defined number of snapshots and then close the backup set window, the follow message is displayed: "Grooming preference has changed. Retrospect will retrieve older snapshots to make sure they do not get groomed out" By hiting OK you would think that something would show up in the executing tab showing the progress of the snapshot retrieval. And then maybe an entry in the History and/or the Events tab notifying of failure or success. However none of that happens. Retrospect never does what it said it was going to do. I've seen the same behavior if the number of saved snapshots is changed on the Options tab. For example, say I change the config from storing 10 snapshots to 12 snapshots. Again the "Grooming preference has changed..." window and then nothing happens. It simply just doesn't work.
  10. cmcfarling

    Grooming fails to start in some cases

    If you read through this thread I'm not asking how things are supposed to work. I know how things are supposed to work. I'm asking why things don't work as they are supposed to. It seems to me that for anyone trying to implement a backup system beyond something fairly basic, Retrospect tends to fall on it's face, or at least require a lot of babysitting. There are several threads on this forum that support that statement I think. That's not the kind of backup software most admins want to deal with. By posting my findings here, calling tech spport on the phone, emailing you directly, I was hoping to spark some interest from the support and/or development departments to try to make this software better. I don't know, maybe some of this has filtered through. From my perspective though, I get the impression that support doesn't want to be bothered with these "annoying inquiries by that guy who owns one copy of MultiServer." After all they have better things to do like get v8 out the door, right? Nothing personal, it's just frustrating when you know something should be working better and there is no real recourse to make that happen, especially after investing a great deal of time and money.
  11. cmcfarling

    Grooming fails to start in some cases

    Quote: It is very unusual to have anyone saving 70 snapshots for EVERY hard disk you backup Why? I've had a similar setup for years when backing up to tape. The source volumes are publicly shared volumes. Users on production desktops work directly from these network volumes. Since files are constantly changing, being deleted, added, etc throughout the day, backing up the sources once a day is not adequate for an effective recovery policy. For these production volumes, I'm backing them up 5 times a day at approx 5 hour intervals. By keeping 70 snapshots per volume that allows at least 2 weeks of backed up data. On the other hand, I also have backup sets which backup just once per day and are set to keep 21 snapshots (3 weeks). Here's an overview of my setup BACKUP SET SCHEDULE SNAPSHOTS TO RETAIN SIZE LIMIT ---------------------------------------------------------------------------------- Production1 Sun-Sat 1AM,6AM,11AM,4PM,9PM 70 1800G Production2 Sun-Sat 3AM,8AM,1PM,6PM,11PM 70 3000G Production3 Sun-Sat 5AM,10AM,3PM,8PM,2AM 70 1500G Operations1 Sun-Sat 8PM 21 100G Operations2 Sun-Sat 8PM 21 200G Databases1 Sun-Sat 12AM,12PM 28 100G Quote: If you backup Drive C 71 times and Drive D 71 times, then the unique data that exists in the oldest snapshot will be groomed out. Grooming will only happen when the disk actually fills or if you schedule a grooming operation. Yes I realize that Quote: You mention that Retrospect may not have retrieved all of the snapshots after you made a setting change. Are the "inactive" snapshots all IDENTICALLY named from the same source volume? Yes, they are identically named. And, I didn't mention that Retrospect may not have retrieved all of the snapshots, I mentioned that it didn't retrieve any snapshots. What about all of the other issues with Retrospect NOT doing something it is supposed to do as I've oulined in this thread? I can't help but get the feeling that no one at EMC is going to take 2 seconds to try to reproduce any of these issues. Am I wrong? As I've been beta testing, er um, trobleshooting over the last month or so, the recurring theme from EMC tech support is that all of the problems are with my configuration, my hardware, my backup strategy, etc, etc, etc. If I am pushing Retrospect beyond its limits then how would I know? Where is the documentation stating what the limits are???
  12. cmcfarling

    I am getting tired...

    While I see that most of the posts on this thread are from mid 2007, most of the gripes still apply today I think. Namely the stuff about bugs and the inordinate amount of time that too many admins have to spend babysitting Retrospect. I've had my own issues; grooming problems, constant corruption, UI glitches, etc, etc. Chris
  13. cmcfarling

    Grooming fails to start in some cases

    If you recall from post #105225 - 01/07/08 01:37 PM above, I had changed the snapshot count from 35 to 70 for the Production1 backup set. That's when Retrospect told me it was going to retrieve older snapshots so they wouldn't get groomed out, which of course it did not do. Then this whole progression of trying to force Production1 to groom started with the intent that maybe Retrospect would retrieve those older snapshots as part of the groom process. Well... that didn't work out so good. The backup set went from having 83 snapshots total to just 40 now. In other words, after changing the snapshot count from 35 to 70, the backup set contained 35 active snapshots and 48 inactive snapshots. Retrospect was supposed to retrieve 35 of the 48 inactive snapshots to make the total number of active snapshots 70. After doing so a groom operation would have groomed out the remaining 13 snapshots. Instead those 35 (+ the 13) were never retrieved and were groomed out leaving just the original 35 active snapshots. At this point 5 new snapshots have been created for a total of 40. So not only does grooming fail to start sometimes, you will potentially lose data unintentionally if you can get it to work. Sorry EMC but it may be time to look at other options.
  14. cmcfarling

    Grooming fails to start in some cases

    Update... The Production3 groom operation finished after 16m at 1/8/2008 8:00am. As of 1/8/2008 9:47am the Production1 groom operation that was stuck in the waiting queue is still there. Let's try stopping the proactive backup yet again and see what happens... Finally!! The Production1 groom operation moved from the waiting queue to the executing queue. I don't know what else to say that hasn't been said. There are some serious issues with this software to the point where managing groom enabled disk backup sets is almost undoable.
  15. cmcfarling

    Amazing recatalog speed

    No, single disk. Yes, "all disks" was selected.
  16. Just pointing out a little issue with the UI. Note the performance number in this screenshot: http://www.pacific-plus.com/retrospect/retrospect2.JPG At that rate this catalog operation would have been completed in under 4 minutes. MultiServer 7.5.387
  17. cmcfarling

    Grooming fails to start in some cases

    Update... The Production1 groom operation finished after 2h 23m at 1/7/2008 8:18pm. As of 1/8/2008 7:43am the Production1 groom operation that was stuck in the waiting queue is still there. Let's try stopping the proactive backup again and see what happens... Amazing, after stopping the proactive backup, a groom operation for yet another backup set, Production3, has started executing. Again, where did this mysteriously come from? And of course the Production1 groom operation in the waiting queue is still there. It just gets better and better (or actually worse and worse).
  18. BTW... Quote: You do realize that Retrospect is an SMB product and not an enterprise level product. We have never marketed Retrospect for the enterprise. The argument could be made that Retrospect is being marketed to the enterprise. From the product web page: "EMC® Retrospect® 7.5 for Windows is designed to deliver automated, reliable, cost-effective protection for small and medium businesses (SMBs) and the distributed enterprise."
  19. cmcfarling

    Grooming fails to start in some cases

    Update... Grooming of Operations1 completed successfully after 40 minutes. The waiting groom operation of Production1 didn't budge though, it's still in the waiting queue. I thought I'd stop the Proactive backup again to see if that would trigger it. Well it did trigger a groom of Production1 in fact. However it wasn't the groom operation that was sitting in the waiting queue, that's still there. In the meantime Production1 is being groomed though. The only thing I can think of is that the operation that's executing now stems from one of the times that I tried to initiate a groom from the Action window and nothing happened. As if Retrospect remembered that I wanted to do that and just decided that now was the time to start it because the proactive backup had been stopped. To be honest I can't remember if I had actually tried to manually groom this backup set though. It's hard to keep track of all of the things that have gone wrong.
  20. Don't know of a fix but I can relate to the frustration. Although Retrospect gives you the ability to name tapes, and even the ability to barcode them, it will happily ignore the name and barcode if the tape is blank/erased. If using a tape library, if Retrospect runs out of space it starts searching for any blank media. So for example let's say the current backup is going to the backup set "Backup_A". In your 7 tape library you have tapes 1-Backup_A 2-Backup_A 3-Backup_A 4-Backup_A 5-Backup_A 6-Backup_A 1-Backup_B The 1-Backup_B tape is erased but you manually named it when you erased it with Retrospect. So the unattended backup moves along and fills up 1-6-BackupA and now it needs a new tape. Even though 1-Backup_B doesn't follow the same naming convention, Retrospect will grab it, rename it 7-Backup_A and carry on with the backup. The other problem with this behavior is if some of the tapes are blank. So let's say this time you fill your library with 7 tapes. 1-Backup_A has already been written to by Retrospect causing it's name to be embedded on the tape. You go ahead and apply stickers to the tapes so you know which one is which when they're out of the library. The tapes are loaded as follows: 1-Backup_A (slot1) 2-Backup_A (slot2) 3-Backup_A (slot3) 4-Backup_A (slot4) 5-Backup_A (slot5) 6-Backup_A (slot6) 7-Backup_A (slot7) Then you take the time to go though and manually name each tape in Retrospect. At this point going to Configure >> Devices displays the tape library slots. The name Retrospect sees corresponds to the physical label on the tape. So far so good. Now, an unattended backup to Backup_A starts as scheduled. Retrospect will grab the tape in slot 1, see that it is the 1st member of Backup_A and proceed to write to it. Then tape 1 fills up so Retrospect moves onto slot 2. Well, since you named that tape 2-Backup_A Retrospect will surely notice that and proceed with that tape. Wrong. Since the tape is erased Retrospect ignores the name and continues to move through the library looking for a non-empty tape named 2-Backup_A. When it gets to slot 7 it realizes that no such tape exists and they are all erased. As such it decides to use the tape in slot 7 as 2-Backup_A to save the time of skipping back to slot 2. From then on it backs up one slot at a time as each tape fills up. When you come in and check in a couple days the tapes are arranged as follows: 1-Backup_A (slot1) 7-Backup_A (slot2) 6-Backup_A (slot3) 5-Backup_A (slot4) 4-Backup_A (slot5) 3-Backup_A (slot6) 2-Backup_A (slot7) Now each tape is named differently than what it is labeled on the outside, except for the tape in slot 1. Time to peel and reapply those labels. So with that said, it would be nice if the naming feature actually did something meaningful. As it stands you can name tapes all you want, then Retrospect will be happy to override all those names that you spent all that time setting up.
  21. Update... The recatlog finished without error after 43 hours, 6 minutes. At that point the catalog stats were: 4543G for 292541 files, 975 sessions, 105 snapshots, 13432 rdb segments At 1/6/2008 8:15am I started a groom operation (initiated via the Actions window). The groom completed successfully after 22 hours 30 minutes. At that point the catalog stats were: 1043.5G for 87642 files, 444 sessions, 105 snapshots, 4711 rdb segments Yay, I finally got this thing groomed. That seems to only be half the battle though. See more trials and tribulations here: Grooming fails to start in some cases http://forums.dantz.com/ubbthreads/showflat.php/Cat/0/Number/105159/an/0/page/ Note that Retrospect groomed through 13432 segments, and nearly 4.43TB, in 22.5 hours. Mayoff, you mentioned that after 24 hours your 7000 segment, 3TB groom operation was not even half way done. Wonder why? This machine is no speed demon. It's a Dell Poweredge 1650 with 2.5GB RAM. Chris
  22. cmcfarling

    Grooming fails to start in some cases

    And yet another instance of this behavior... On a backup set I changed the "Groom to remove backups older than" setting to 70 from 35. As noted above the "Grooming preference has changed. Retrospect will retrieve older snapshots to make sure they do not get groomed out" dialog box came up. After clicking OK nothing happend. Hmmm, maybe forcing a groom at this point will cause those older snapshots to be retrieved. From the Options tab I clicked the Action button. From the list of potential actions I selected Groom and clicked OK. Now in the past I have done this very thing and it worked. What's supposed to happen is the backup set properties window automatically closes at this point and the groom action shows up in the Executing tab. However, that is not working now. Retrospect just ignores my request like it never happened. Ok, let's go to plan C. I'll create a groom script and force it to groom that way. So I go to Manage Scripts and edit my Groom script that has already been set up for this type of thing. I set it so that there is just a single source, the backup set that I'm attempting to groom, and click OK. From the Run menu I select Groom and click Execute. Finally I'm going to force Retrospect to groom this backup set. Oh wait a minute, not so fast. The groom operation goes to the Waiting queue with a status of "Waiting for Production1" (my backup set name in this case). Why is it waiting???? Production1 is not in use by any other operation. Unfortunately I have seen this behavior before too. This will sit in the Waiting queue indefinitely. One before when this happened and I let it go to see if it would eventually execute. It did not. It happily sat there for a couple days before I cancelled it. In fact this has happened enough for me to develop a workaround. There is a Proactive backup setup in Rerospect too. Note that the Production1 backup set has nothing to do with the Proactive backup. For whatever reason, stopping the Proactive backup has, in the past, triggered the waiting groom operation to start. (that makes perfectly good sense doesn't it?) I bet that if I stop the Proactive backup, the waiting groom operation will execute. Let's try... Hmmm, I guess I was wrong the Production1 groom operation is still waiting. However once I stopped the Proactive backup a grooming operation for a backup set named Operations1 started. Where in the hel* did that come from???? This is completely messed up. Instead of moving toward a set it and forget approach of backing up, Retrospect seems to be going toward the set it, hope it works, reset it, hope some more, then keep resetting it every so often approach. Maybe EMC will give me a free upgrade to v8 for beta testing this software?
  23. cmcfarling

    Snapshots vs. sessions

    After thinking about it some more, I definitely think the best way is to list all snapshots (active and inactive) in one window. That way the user sees all possible snapshots in one window instead of having them split between 2 windows. Here's what my snapshot window would look like if EMC said "hey Chris, if you care about it so much why don't you design it" http://www.pacific-plus.com/retrospect/snapshotwindow.jpg
  24. Is there a need to differentiate between snapshots and sessions? When viewing the properties for a backup set the snapshots tab shows the active snapshots, or, the snapshots that are stored in the catalog file. It's possible that other snapshots are available. To make a snapshot available (active), one clicks Add and then selects from a list of sessions. It would seem that only sessions that actually have a snapshot associated with them should show in this list. Perhaps that is how it is supposed to work but recently I had a situation where this was not the case. I attempted to add a snapshot from a session only to find out that the snapshot didn't exist. Why did it even show up in the list as an available session to load a snapshot from? If that is a bug that needs to be fixed first of all. All of this can be rather confusing though and can be simplified. Why not do away with sessions and just have snapshots? Then, a snapshot is either active (stored in the catalog file) or not active (stored on the media). When someone clicks Add to add a snapshot to the active list, they're presented with a list of inactive snapshots. Or another way to do it, the snapshot list shows every snapshot, active or inactive. Each one would then have some sort of signifier (grayed out, icon, etc) to show what its status is. Either way, Retrospect has to be smart enough to not display snapshots that no longer exist, such as ones that have been groomed out. Chris
  25. At 1529GB into the recatalog process there are no reported errors or warnings yet. 2.25GB of RAM in this machine Can't check the total file count at the moment as the backup set is locked I understand what you're saying about the number of segments. That is actually my intent, to parse this backup set down. But unless I want to wipe it out and start over I have to get a successfull groom completed in order to get rid of some of those segments. If I can make that happen then I was going to set a limit to the amount of space the backup set can consume on the backup volume. As mentionend previously, the backup set is configured to use at most 99% of the volume. I'm going to change that to around 30%, or ~2.3TB. That brings up another question. Let's say I config the backup set in this way so that it is limited to using just 2.3TB of the 7.5TB volume. Also, the grooming policy is set to a defined number of snapshots, 10 just for examples sake. Which one takes precedence? As long as <2.3TB are used, Retrospect should adhere to the grooming policy and keep at minimum 10 snapshots. However what if the backup data reaches the 2.3TB limit and only 8 snapshots have been saved? Will retrospect try to keep going until it saves 10 snapshots even if it takes 3TB? Or will it go against the grooming policy and only save 8 snapshots in order to honor the size limit? I'm curious, how long has your backup set with 7000 segments existed? It seems that with the way it's implemented the segment count could continue to grow indefinitely, even with grooming. As long as one file exists in a segment, that segment will exist. So over time I could see many segments being shrunken down but not completely eliminated. All the while new segments are constantly being created. It seems to me that EMC should be doing more testing on this and come up with some more concrete numbers to give to customers.