Jump to content

DavidHertzberg

Members
  • Content count

    1,219
  • Joined

  • Last visited

  • Days Won

    40

Everything posted by DavidHertzberg

  1. DavidHertzberg

    Scheduling Proactive Backups

    I have never used a Proactive backup script. However I believe I have a basic understanding of them, and I have the capacity to read the last paragraph in this section of the old Wikipedia article on Retrospect and the Retrospect User's Guides. I think you—jelockwood—don't properly understand the concept of a Proactive script, or possibly the idea that you can have multiple schedules for a Proactive script. To quote from the WP article "... Retrospect also has a special Proactive script type that—while it is running—maintains on the "backup server" a queue[my emphasis] of its designated Source volumes in the oldest-first order of their most-recent backup date and time." Let's start by assuming that you could schedule a Proactive script from midday to midday, and that it was scheduled to run on Mondays through Fridays. The week's first execution of the Proactive script would start at noon Monday, and end at noon Tuesday. Then the week's second execution of the Proactive script would start at noon on Tuesday, recreate the same queue that the week's first execution had when it ended, and go merrily on its Proactive way until noon Wednesday. This process of recreating the script's queue at noon would continue until noon Saturday, at which point the Proactive script would cease execution until noon the following Monday. Now let me tell you how to do the same thing with the current facilities of Retrospect, as described in steps 8 and 9 on pages 133 and 134 of the Retrospect Mac 13 User's Guide. To schedule executions of a Proactive script from noon Monday until noon Saturday: First create a schedule for the script for Monday only from noon to midnight. Then add a second schedule for the script for Tuesday through Friday only from midnight to midnight. Finally add a third schedule for the script for Saturday only from midnight to noon. If that sounds a little complicated, jelockwood, you can take an envious look at "Customizing the Schedule" on pages 267 through 269 of the Retrospect Windows 11 UG. Retrospect Mac 6 used to have the same UI and capability, but it got simplified per the first paragraph of this section of the WP article. All that was really lost was the "wrap up" period capability. If you want to know why the Retrospect Windows UI wasn't similarly simplified, read the last paragraph of the section linked-to in the second sentence of this paragraph. P.S.: In case it wasn't obvious from the third paragraph of this post, if you want to switch the backup media (which I assume means the Media Set) for your Proactive script at noon each day, then you must do it by manual action. The script should specify all the noon-to-noon Media Sets to be used during each week. I know this is a little complicated, jelockwood, but it's workable. P.P.S.: Revised last sentence in P.S.; it was snide, and I'm sorry. The implication of that last sentence is that what you, jelockwood, are asking for in your OP should be a Product Suggestion; there is no bug since you can do what you want with Retrospect as it now exists. Yes, if you are using Proactive scripts but also want to alternate between Media Sets on alternate days so you can take one Media Set off-site, there ought to be an easier method of scheduling the Proactive script. However, on page 127 of the Retrospect Mac 13 User's Guide, it says "Copies to the most ideal available [my emphasis] Media Set in the destinations list. Automatic media rotation among multiple available [my emphasis] Media Sets." Note also that page 258 in the Retrospect Windows 11 UG says "Create multiple Backup Sets [the Retrospect Windows term for Media Sets] and use them all as destinations in your Proactive Backup script. Rotate through the sets by inserting different media in the backup device each day. Proactive Backup uses whatever media you inserted." If you are alternating Media Sets at noon, you may want to manually Pause Proactive at noon while you switch tapes/disks. P.P.P.S.: Totally revised the first and second sentences of my P.S.; I forgot that Proactive scripts—unlike regular scripts—cannot specify a different Media Set for each schedule but can specify multiple Media Sets for the script—it's that way in Retrospect Windows too (presumably the EMC Dantz engineers decided far back in the mists of pre-Retrospect-8 that manual switching of what were then called Backup Sets was the way to go for what were then—for Retrospect Mac—called Backup Server scripts). Emphasized "available" in two quotes in my P.P.S., to explain why switching tapes/disks at noon would do what jelockwood wants.
  2. DavidHertzberg

    Scheduling Proactive Backups

    If you think this is a bug that should be fixed by Retrospect Inc., you will have to submit it as a Support Case. For English speakers, that is done by going here, and filling out the form (sorry, I don't know what the equivalent addresses are for non-English speakers, but they can figure it out from their appropriate Retrospect website address). IMHO this is quite reasonable; obliging you to fill out the form provides Retrospect Inc. with useful details about your Retrospect installation that they would otherwise have to query you for. As a result, Retrospect Inc. will pay no attention to your post in this sub-forum. On 12 December 2016, in response to a letter I snail-mailed to Mayoff, I received an e-mail through a Mayoff account that was signed by JG Heithcock, CEO, Retrospect, Inc.. In it he says "From reading your letter, I think the main issue is that you view the forums as a good place to talk to us, Retrospect, Inc. But we view the audience of the forums as restricted to our customers [my emphasis]. The one caveat we have made on that is for feature requests, largely as we would like to see if other customers also agree on the desirability and feature set for these requests." That means that the only audience for "Retrospect bug reports" in this sub-forum will be other administrators of Retrospect Mac. Nevertheless, by posting in this sub-forum you are providing a useful service to us fellow Mac administrator peasants—one that is denied to administrators of Retrospect Windows who are evidently considered too peasanty (insert appropriate smiley here) to benefit from such bug notifications. Thank you. Please be aware that the "description of your issue" in the Support Case form is IME limited to about 2000 characters by the Support Case software. If you go over that limit your "description" will be broken up into a "description" plus one or more "additional notes". The same is true for any additional notes you may later post yourself. I suggest that, to avoid the appearance of choppiness in your Support Case, you create your case in a post in this forum and then copy it paragraph-by-paragraph to your Support Case. If this post sounds formulaic, that's because I intend it to be. I intend to post it in every new thread that appears in this sub-forum, unless the OP indicates that he/she has or will open a Support Case for the bug that the thread reports. Of course, Mayoff could take 5 minutes of his time to post a slightly-more-polite version of this post as a "sticky thread" that will always appear at the top of the forum. I don't intend to hold my breath until that happens (insert appropriate smiley here).
  3. Given the response from Mayoff documented in this post, I have elected to start this thread in "Product Suggestions—Mac OS X". If you people want the Retrospect Inc. engineers to implement the enhancement, you'd better indicate you want it—either in this thread or in the "Product Suggestions—Mac OS X" thread. Remember what President Obama said at the end of his speech tonight, citizens. P.S.: This is post #20 in the thread.
  4. I phoned Mayoff early this afternoon. He says that Retrospect Inc. is not looking for Likes or additional posts in a Product Suggestions thread. Instead he says they have some kind of database for product suggestions, and they check that plus look at the discussions forums. I intend to make sure that any Product Suggestion thread I start in the future has at least one link back to a forum thread discussing the problem the suggestion is designed to solve.
  5. Apparently Retrospect Inc. will pay no attention to anyone's thread in the "Product Suggestions—Mac OS X" sub-forum unless it gets some kind of up-vote . On 12 December 2016, in response to a letter I snail-mailed to Mayoff, I received an e-mail through a Mayoff account that was signed by JG Heithcock, CEO, Retrospect, Inc.. In it he says in a paragraph "From reading your letter, I think the main issue is that you view the forums as a good place to talk to us, Retrospect, Inc. But we view the audience of the forums as restricted to our customers [my emphasis]. The one caveat we have made on that is for feature requests, largely as we would like to see if other customers also agree on the desirability and feature set for these requests [my emphasis]." Mr. Heithcock followed this paragraph with one quoting the forum rules. Given that he did so, and that I have not signed any kind of non-disclosure agreement, I have no hesitation in making his paragraph above public. What does Mr. Heithcock mean about feature requests? Is he looking for Likes? Is he looking for additional posts by other users in request threads, pro or con or in-between? I intend to phone Mayoff on Monday, and see if I can get an answer from him.
  6. On second thought about my post #18 immediately above, maybe we don't need the additional CopyLock and CopyUnlock script types at all, or the additional option for Backup scripts. Instead make the setting of the CopyLock bit in the Source drive's Media Set Catalog File header be standard operating procedure for a Source drive about to be backed up to a particular Media Set. It would also be standard operating procedure for a Copy Media Set or Copy Backup script run to go into a pause state when it encounters a Catalog File header for a Source drive on which the CopyLock bit is still set. The CopyLock bit would—of course—be cleared for a particular drive's header in a Media Set Catalog File as soon the copy phase of a Backup for that drive had been completed and the drive's Catalog File had been updated for the Media Set. As soon as a CopyLock bit had been cleared, an overlapping Copy Media Set or Copy Backup script run that had gone into a pause state because of that CopyLock bit would come out of the pause state and proceed. This would make the CopyLock bit a "drive backup copying in progress" bit. That would be sufficient to ensure that a Copy Media Set execution would run far enough behind any Backup run it is overlapped with that its copying—based on a drive's Catalog File entries—could never occur before that drive's Catalog File entries have been updated by the Backup. Probably it would be a good idea to add an "ignore CopyLock" option to Copy Media Set and Copy Backup scripts, in order to handle weird situations in which a Backup script run for a particular drive had not completed its copy phase successfully—but the administrator nevertheless wanted to run a Copy Media Set or Copy Backup execution using the Media Set's Catalog File entries for that drive as they existed previously. Actually even this option might not be needed; a Catalog File Rebuild for a Media Set would automatically clear any CopyLock bit that had been set for a Source drive's header within that Catalog File. So how about it, jweisbin and others? What I have just proposed is a modest enhancement to Retrospect that would enable the overlapping I tested in posts #15 and #16 in this thread to really work as I thought it had, thus increasing the usefulness of a major capability that has been in Retrospect since Mac version 8. Would it be too much to ask you to show a little support? P.S.: This is post #19 in the thread.
  7. OK, where do we go from here in regards to Copy Media Set runs overlapped with Backup runs, considering my post #17 immediately above? It is now evident that the existing version of Retrospect can't do what I was proposing in posts #15 and #16. Last night, on my way to see the movie "Hidden Figures" (IMHO very good), I conceived an enhancement to Retrospect that would enable it to do so. In its simplest form, it would add a CopyLock script type—referencing an existing Media Set. A CopyLock script run would simply access the Catalog File header for each Source drive in the Media Set, setting a bit that would cause a Copy Media Set or Copy Backup run with that Media Set as the Source to go into a pause state when it tried to use the Catalog File to copy a Source drive whose bit was still set. That bit in the Media Set's Catalog File header for a Source drive would be cleared when a Backup script completed the Copying phase for that drive and updated the Catalog File for it, enabling the Copy Media Set or Copy Backup run to come out of the pause state and proceed to copy that drive. This would ensure that a Copy Media Set execution would run far enough behind any Backup run it is overlapped with that its copying—based on a drive's Catalog File entries—could never occur before that drive's Catalog File entries have been updated. To cover the practical possibility of scripts never getting run or crashing, the enhancement would also need to include a CopyUnlock script. This would simply do the opposite of a CopyLock script, accessing the Catalog File header for each Source drive in the Media Se and un-setting the bit that would cause a Copy Media Set or Copy Backup run with that Media Set as the Source to go into a pause state. Ideally we might also want the enhancement to include a new option for Backup scripts that would do the equivalent of a CopyLock bit-setting just before starting the actual Backup process for a particular Source drive, ensuring that that Source drive could not have a Copy Media Set or Copy Backup done on it via the same Media Set before its Backup copying had been subsequently completed and its Catalog File header updated. That second enhancement would take care of the situation in which a drive is dynamically cabled to a Source computer after a CopyUnlock script run has completed and a Backup script run has started, but before the Backup script run gets to processing that Source computer. My question to all of you forum readers is: do you want me to bother to post a request for such an enhancement in the "Product Suggestions—Mac OS X" sub-forum? My OP in this thread reports that the CEO of Retrospect Inc. and his cabal will not consider a feature request unless there is evidence of support for it from other Retrospect customers. I'll find out Monday exactly what kind of evidence of support he requires, but I won't proceed further until I have prior indications of support from you. Frankly, although I had fun doing the tests for Copy Media Set runs overlapped with Backup runs, I'm getting more than a bit tired of doing work for other people who don't seem to appreciate it. I'm afraid that appreciation must now go beyond simply bumping the view counts for threads I contribute to; I don't consider myself to be in the entertainment business (do not insert appropriate smiley here—I'm dead serious). I'll even give you a couple of arguments against a request for such an enhancement. First, maybe nobody—including me—except jweisbin needs the overlapping. With many people (not including me) installing SSDs in their computers, Backup script runs for a string of "clients" may be taking such an increasingly shorter time that overlapping of Copy Media Set script runs with Backup script runs may no longer be necessary. Second, maybe everybody except jweisbin is satisfied doing a non-overlapped Copy Media Set to a Tape or Cloud Media Set over the weekend—even if that means an off-site backup once a week instead of every workday. So let's see some posts from someone besides myself in this thread, pro or con! P.S.: This is post #18 in the thread.
  8. Stop, jweisbin and others. Do not attempt to overlap executions of Backup scripts and Copy Media Set scripts until you have read the rest of this post—which will probably convince you not to attempt it at all. I phoned Mayoff this afternoon by calling Tech Support at extension 3 at the Retrospect Inc. number (I had been obliged to leave a message for him yesterday at x806; he is "very busy"—make of that what you will). I meant to quickly give him a heads-up about this thread and my experimental results, and to ask a couple of related questions. Fortunately the first related question I asked was why overlapping didn't work with a Recycle Backup run per post #14 in this thread. He said it's because Copy Media Set uses the Catalog File, and a Recycle wipes out every entry in the Catalog file and does not create new entries until the Updating Catalog File phase(s) of the Backup run. I immediately realized that—on a Copy Media Set run that is overlapped—only the files that are already entered in the Catalog File are copied. That means that an overlapped Copy Media Set run will not copy files that were backed up for the first time in the overlapped Backup run, and will not copy the latest versions of files that were updated since the previous Backup run. This explains something that has been bothering me since yesterday, which is that my statement in my fifth sentence in the first paragraph of post #16 is not quite correct. The total number of files copied from both drives in both Backup script runs is 1236, but the total number of files copied in the Copy Media Set run is only 1235. Checking—now that I know what to look for—in Documents, I see that I modified a file (actually the LibreOffice version of this post) at 9:05 a.m. on 4 January, which means that the modified file would have been backed up in the 5 January Backup run. That is the file whose latest version must not have been copied in the overlapped Copy Media Set run on 5 January, because its latest version was not yet in the Catalog File. I executed an extra "Sun.-Fri. Backup" script run this afternoon, backing up to "Media Set White" (which will be recycled tomorrow morning), and plugging in the USB drive "OS X File Transfers" to get a second Source drive on my MacBook Pro. I observed that the "updating catalog" phase for each drive, although it is not shown in the Log, occurs after the Copying phase for each drive backed up. This means that a Copy Media Set execution must run far enough behind any Backup run it is overlapped with that its copying—based on a drive's Catalog File entries—never occurs before that drive's Catalog File entries have been updated. Given how much faster an execution that involves only drives local to the "backup server" runs than an execution that involves drives attached to a "client", I don't think that can ever be guaranteed for a Copy Media Set execution that overlaps with a Backup of "client" drives . P.S.: This is post #17 in the thread.
  9. Here's the further test, showing a Copy Media Set script can copy the backups from two or more days of daily Backup script runs. The logs are at the bottom of this post—in reverse chronological order—to prevent eye fatigue, but I'll give you the highlights up top. First, note that the two Backup script runs each also copied a second drive on David’s MacBook Pro—the USB drive "OS X File Transfers" which was left plugged in. Second, note that again the Copy Media Set script started running 5 minutes after the second-day Backup script, and that it was thereafter running simultaneously—in Activity Thread 2—with the Backup script running in Activity Thread 1. Third, note again that the Copy Media Set script copied the same number and total GB of files scripts as the total copied in the two Backup scripts. Fourth, note that the speed of the "Macintosh HD" copy phase for the second Backup script run was 501.6 MB/min. vs. 686.3 MB/min. for the Copy Media Set script run, but that the speed of the "Macintosh HD" compare phase for the second Backup script run was 1,848 MB/min. vs. 3,722 MB/min. for the Copy Media Set script run's verify phase. That shows the Copy Media Set run was—correctly—being somewhat slowed so as not to overrun the progress of the second Backup run's copy phase. Log for Copy Media Set script run Thurs. 5 January: Copy Media Set Log 5 January 2017: + Executing Copy Media Set - Media Set Red at 1/5/17, 3:05:00 AM (Activity Thread 2) To Backup Set Media Set Pink... - 1/5/17 3:05:16 AM: Transferring from Media Set Red 1/5/17 3:23:34 AM: Execution completed successfully Completed: 1,235 files, 11.8 GB Performance: 686.3 MB/minute Duration: 00:18:17 (00:00:44 idle/loading/preparing) - 1/5/17 3:23:34 AM: Verifying Media Set Pink 1/5/17 3:27:13 AM: Execution completed successfully Completed: 1,235 files, 11.8 GB Performance: 3,722 MB/minute Duration: 00:03:38 (00:00:24 idle/loading/preparing) Log for Backup script run Thurs. 5 January: + Normal backup using Sun.-Fri. Backup at 1/5/17, 3:00:00 AM (Activity Thread 1) To Backup Set Media Set Red... 1/5/17 3:00:02 AM: Connected to David’s MacBook Pro * Resolved container David’s MacBook Pro to 2 volumes: Macintosh HD on David’s MacBook Pro OS X File Transfers on David’s MacBook Pro - 1/5/17 3:00:00 AM: Copying Macintosh HD on David’s MacBook Pro Using Instant Scan 1/5/17 3:00:41 AM: Found: 680462 files, 152327 folders, 49.9 GB 1/5/17 3:01:50 AM: Finished matching 1/5/17 3:02:15 AM: Copying: 396 files (5.7 GB) and 0 hard links 1/5/17 3:13:50 AM: Building Snapshot... 1/5/17 3:13:50 AM: Checking 152,327 folders for ACLs or extended attributes 1/5/17 3:15:30 AM: Finished copying 8,380 folders with ACLs or extended attributes 1/5/17 3:15:57 AM: Copying Snapshot: 2 files (217.5 MB) 1/5/17 3:16:13 AM: Snapshot stored, 217.5 MB 1/5/17 3:16:13 AM: Comparing Macintosh HD on David’s MacBook Pro *File "Macintosh HD/retroclient.log": didn't compare .... *File "Macintosh HD/Users/davidhertzberg/Library/IdentityServices/ids.db-shm": didn't compare 1/5/17 3:19:28 AM: Execution completed successfully Completed: 396 files, 5.7 GB Performance: 789 MB/minute (501.6 copy, 1,848 compare) Duration: 00:19:27 (00:04:36 idle/loading/preparing) ****** Warning: volume "OS X File Transfers on David’s MacBook Pro" has the "Ignore ownership" setting enabled. ****** - 1/5/17 3:19:29 AM: Copying OS X File Transfers on David’s MacBook Pro 1/5/17 3:19:41 AM: Found: 396 files, 163 folders, 282.7 MB 1/5/17 3:20:49 AM: Finished matching 1/5/17 3:20:50 AM: Copying: 0 files (0 and 0 hard links 1/5/17 3:20:53 AM: Building Snapshot... 1/5/17 3:20:54 AM: Checking 163 folders for ACLs or extended attributes 1/5/17 3:20:54 AM: Finished copying 1 folders with ACLs or extended attributes 1/5/17 3:20:54 AM: Copying Snapshot: 2 files (282 KB) 1/5/17 3:21:00 AM: Snapshot stored, 282 KB 1/5/17 3:21:00 AM: Comparing OS X File Transfers on David’s MacBook Pro 1/5/17 3:21:02 AM: Execution completed successfully Duration: 00:01:33 (00:01:29 idle/loading/preparing) 1/5/17 3:21:02 AM: Execution completed successfully Total performance: 783.7 MB/minute Total duration: 00:21:01 (00:06:05 idle/loading/preparing) Log for Backup script run Wed. 4 January: + Normal backup using Sun.-Fri. Backup at 1/4/17, 9:10:50 AM To Backup Set Media Set Red... 1/4/17 9:10:57 AM: Connected to David’s MacBook Pro * Resolved container David’s MacBook Pro to 2 volumes: Macintosh HD on David’s MacBook Pro OS X File Transfers on David’s MacBook Pro - 1/4/17 9:10:50 AM: Copying Macintosh HD on David’s MacBook Pro Using Instant Scan 1/4/17 9:11:49 AM: Found: 646509 files, 152251 folders, 49.5 GB 1/4/17 9:13:06 AM: Finished matching 1/4/17 9:13:40 AM: Copying: 530 files (5.9 GB) and 0 hard links 1/4/17 9:24:30 AM: Building Snapshot... 1/4/17 9:24:30 AM: Checking 152,251 folders for ACLs or extended attributes 1/4/17 9:26:13 AM: Finished copying 8,379 folders with ACLs or extended attributes 1/4/17 9:26:36 AM: Copying Snapshot: 2 files (207.6 MB) 1/4/17 9:26:45 AM: Snapshot stored, 207.6 MB 1/4/17 9:26:45 AM: Comparing Macintosh HD on David’s MacBook Pro *File "Macintosh HD/retroclient.log": didn't compare .... *File "Macintosh HD/Users/davidhertzberg/Library/IdentityServices/ids.db-shm": didn't compare 1/4/17 9:29:40 AM: Execution completed successfully Completed: 530 files, 5.9 GB Performance: 875 MB/minute (552 copy, 2,108.7 compare) Duration: 00:18:50 (00:05:00 idle/loading/preparing) ****** Warning: volume "OS X File Transfers on David’s MacBook Pro" has the "Ignore ownership" setting enabled. ****** - 1/4/17 9:29:40 AM: Copying OS X File Transfers on David’s MacBook Pro 1/4/17 9:29:49 AM: Found: 396 files, 163 folders, 282.7 MB 1/4/17 9:30:16 AM: Finished matching 1/4/17 9:30:17 AM: Copying: 310 files (137.2 MB) and 0 hard links 1/4/17 9:30:37 AM: Building Snapshot... 1/4/17 9:30:38 AM: Checking 163 folders for ACLs or extended attributes 1/4/17 9:30:38 AM: Finished copying 1 folders with ACLs or extended attributes 1/4/17 9:30:38 AM: Copying Snapshot: 2 files (282 KB) 1/4/17 9:30:44 AM: Snapshot stored, 282 KB 1/4/17 9:30:44 AM: Comparing OS X File Transfers on David’s MacBook Pro 1/4/17 9:30:54 AM: Execution completed successfully Completed: 310 files, 137.2 MB Performance: 548.9 MB/minute (411.7 copy, 914.8 compare) Duration: 00:01:13 (00:00:43 idle/loading/preparing) 1/4/17 9:30:55 AM: Execution completed successfully Total performance: 861.6 MB/minute Total duration: 00:20:04 (00:05:43 idle/loading/preparing) P.S.: This is post #16 in the thread.
  10. Last night I thought of a variant on the simultaneous-backups solution that is so much more promising than my post #13 in this thread that it appears to be the definitive solution for jweisbin and others with the same need. It keeps the near-simultaneous running of a Copy Media Set script along with a Backup script, but eliminates the need for that Copy Media Set script to do a from-scratch copy of the entire Media Set—a copy that can take hours. It does so by having both the "Match source files against the Media Set" and the "Don’t add duplicate files to the Media Set" options checked. These options are mentioned on page 162 of the Retrospect Macintosh 13 User's Guide, but are only discussed under the "In the Matching category, there are the following options:" sub-section of the "Backup Script Options" section on pages 116-117 of that UG. There it says that both options are key components of Retrospect’s Smart Incremental backups. And that's what jweisbin and others want for backing up to two media sets at the same time; to have the Copy Media Set script copy to the second Media Set only those backups that the Backup script has just added—doing a Smart Incremental backup—to the first Media Set. Naturally I ran a test early this morning (after booting my Mac Pro Desktop Edition "backup server" very early so I could reset the "Allow n Activity Threads" Preferences dropdown to 4); the logs are at the bottom of this post to prevent eye fatigue, but I'll give you the highlights up top. First, note that the Copy Media Set script started running 5 minutes after the Backup script, and that it was thereafter running simultaneously—in Activity Thread 2—with the Backup script running in Activity Thread 1. Second, note that both scripts copied precisely the same number and total GB of files. Third, note that the speed of the copy phase for the Backup script run was 524.3 MB/min. vs. 402.6 MB/min. for the Copy Media Set script run, but that the speed of the compare phase for the Backup script run was 2,061.5 MB/min. vs. 3,607.7 MB/min. for the Copy Media Set script run's verify phase. That shows the Copy Media Set run was—correctly—being slowed so as not to overrun the progress of the Backup run. I intend to rerun this test early Thursday or Friday morning, to confirm that with these options the Copy Media Set script can copy the backups from two or more days of daily Backup script runs. That means that jweisbin and others could run their overlapping Copy Media Sets alternately onto two or more different Tape Media Sets, e.g. "Tape Media Set MonWedFri" and "Tape Media Set TueThuSat", keeping yesterday's Tape Media Set off-site while bringing the day-before-yesterday's Tape Media Set on-site to be updated. Log for Backup script run: + Normal backup using Sun.-Fri. Backup at 1/3/17, 3:00:00 AM (Activity Thread 1) To Backup Set Media Set Red... 1/3/17 3:00:02 AM: Connected to David’s MacBook Pro * Resolved container David’s MacBook Pro to 1 volumes: Macintosh HD on David’s MacBook Pro - 1/3/17 3:00:00 AM: Copying Macintosh HD on David’s MacBook Pro Using Instant Scan 1/3/17 3:00:39 AM: Found: 676463 files, 152266 folders, 50 GB 1/3/17 3:01:48 AM: Finished matching 1/3/17 3:02:13 AM: Copying: 414 files (5.9 GB) and 0 hard links 1/3/17 3:13:37 AM: Building Snapshot... 1/3/17 3:13:37 AM: Checking 152,266 folders for ACLs or extended attributes 1/3/17 3:15:11 AM: Finished copying 8,375 folders with ACLs or extended attributes 1/3/17 3:15:41 AM: Copying Snapshot: 2 files (216.9 MB) 1/3/17 3:16:03 AM: Snapshot stored, 216.9 MB 1/3/17 3:16:03 AM: Comparing Macintosh HD on David’s MacBook Pro *File "Macintosh HD/retroclient.log": didn't compare ... *File "Macintosh HD/Users/davidhertzberg/Library/IdentityServices/ids.db-shm": didn't compare 1/3/17 3:19:02 AM: Execution completed successfully Completed: 414 files, 5.9 GB Performance: 835.1 MB/minute (524.3 copy, 2,061.5 compare) Duration: 00:19:02 (00:04:38 idle/loading/preparing) Log for Copy Media Set script run: + Executing Copy Media Set - Media Set Red at 1/3/17, 3:05:00 AM (Activity Thread 2) To Backup Set Media Set Pink... - 1/3/17 3:05:05 AM: Transferring from Media Set Red 1/3/17 3:20:25 AM: Execution completed successfully Completed: 414 files, 5.9 GB Performance: 402.6 MB/minute Duration: 00:15:19 (00:00:22 idle/loading/preparing) - 1/3/17 3:20:25 AM: Verifying Media Set Pink 1/3/17 3:22:21 AM: Execution completed successfully Completed: 414 files, 5.9 GB Performance: 3,607.7 MB/minute Duration: 00:01:56 (00:00:16 idle/loading/preparing) P.S.: This is post #15 in the thread.
  11. DavidHertzberg

    Sources mess up selected drives for backups

    If you think this is a bug that should be fixed by Retrospect Inc., you will have to submit it as a Support Case. For English speakers, that is done by going here, and filling out the form (sorry, I don't know what the equivalent addresses are for non-English speakers, but they can figure it out from their appropriate Retrospect website address). IMHO this is quite reasonable; obliging you to fill out the form provides Retrospect Inc. with useful details about your Retrospect installation that they would otherwise have to query you for. As a result, Retrospect Inc. will pay no attention to your post in this sub-forum. On 12 December 2016, in response to a letter I snail-mailed to Mayoff, I received an e-mail through a Mayoff account that was signed by JG Heithcock, CEO, Retrospect, Inc.. In it he says "From reading your letter, I think the main issue is that you view the forums as a good place to talk to us, Retrospect, Inc. But we view the audience of the forums as restricted to our customers [my emphasis]. The one caveat we have made on that is for feature requests, largely as we would like to see if other customers also agree on the desirability and feature set for these requests." That means that the only audience for "Retrospect bug reports" in this sub-forum will be other administrators of Retrospect Mac. Nevertheless, by posting in this sub-forum you are providing a useful service to us fellow Mac administrator peasants—one that is denied to administrators of Retrospect Windows who are evidently considered too peasanty (insert appropriate smiley here) to benefit from such bug notifications. Thank you. Please be aware that the "description of your issue" in the Support Case form is IME limited to about 2000 characters by the Support Case software. If you go over that limit your "description" will be broken up into a "description" plus one or more "additional notes". The same is true for any "additional notes" you may later post yourself. I suggest that, to avoid the appearance of choppiness in your Support Case, you create your case in a post in this forum and then copy it paragraph-by-paragraph to your Support Case. If this post sounds formulaic, that's because I intend it to be. I intend to post it in every new thread that appears in this sub-forum, unless the OP indicates that he/she has or will open a Support Case for the bug that the thread reports. Of course, Mayoff could take 5 minutes of his time to post a slightly-more-polite version of this post as a "sticky thread" that will always appear at the top of the forum. I don't intend to hold my breath until that happens (insert appropriate smiley here).
  12. Early Saturday I tried a high-volume rerun of the test in post #13 of this thread, to simulate what would happen if jweisbin or someone else ran a Copy Media Set script simultaneously with a Backup script for a number of "client" and local drives. It didn't work the way it was supposed to, but the further test didn't invalidate my concept. My failures may be instructive to others. My idea was to overlap my scheduled "Sat. Backup" Recycle Media Set script run onto "Media Set Red" ("Media Set Blue" having gone to the bank safe deposit box Friday afternoon) with a Copy Media Set script run copying "Media Set Red" onto a new "Media Set Pink" on my spare FireWire 800 portable drive (after Removing "Media Set Violet" from that drive). Because "Sat. Backup" takes about 10.5 hours, I booted my Mac Pro Desktop Edition "backup server" very early Saturday morning and reset the "Allow n Activity Threads" Preferences dropdown to 4, and scheduled a Copy Media Set script that had as its Source "Media Set Red" and its Destination "Media Set Pink" with its Activity Thread set = 2—with that script scheduled to start 5 minutes after my "Sat. Backup" script. I then left both my Mac Pro "backup server" and my MacBook Pro "client" awake and went to bed, expecting that because of BPH I would awaken before 8 a.m. Saturday to boot my Digital Audio G4 "client" before "Sat. Backup" had finished 5 hours of backing up and verifying the "Macintosh HD" on my MBP (the G4 is from 2001, and I prefer not to leave it running any more than necessary). I unexpectedly slept straight through until 9:45 a.m. Saturday, and found that my "Sat. Backup" script—instead of skipping the shut-down G4 and going on to backup my Mac Pro's local drives—was now backing up a second drive attached to the MBP. This second drive was my other spare FireWire 800 portable drive, which I had mistakenly left on and cabled to the MBP after successfully booted the MBP from it Friday evening. That second drive now contains a Restored copy of my MBP's "Macintosh HD"—the Retrospect 12 Client had mistakenly picked up the second drive per jelockwood's "Retrospect bug reports" thread, and I certainly didn't want to have "Sat. Backup" continue to backup and verify it for another 3.5 hours, so I stopped the "Sat. Backup" execution. Meanwhile the Copy Media Set script had never started executing; it had been waiting for nearly 7 hours while the "Sat. Backup" script ran. Until just now I thought the Copy Media Set script was waiting because I had somehow messed up the threads, even though the log shows the "Sat. Backup" script ran in Activity Thread 1 and the Copy Media Set script finally started executing—after I had stopped the "Sat. Backup" script execution—in Activity Thread 2. I see now from page 289 of the Retrospect Windows 11 User's Guide why it was waiting: "Certain operations require exclusive access to the Backup Set (e.g. updating a Catalog File, grooming, recycling [my emphasis], verifying media). When one of these operations is using a disk Backup Set, no other operations can use that Backup Set at the same time." The "Sat. Backup" script is always scheduled to do a Recycle Media Set of its Destination. I stopped execution of the Copy Media Set script and, after shutting down my MacBook Pro, ran my "Sat. Backup Incremental" script with "Media Set Red" as the Destination. "Sat. Backup Incremental" is actually a duplicate of "Sat. Backup"; it is never scheduled, so when run it always does No Media Action backups of my 6 Source drives. It ran fine, although it gave an error because it couldn't access the MBP. However, when I overlapped it with another execution of the Copy Media Set script, that execution gave a -2249 (could not find session) error. Early this morning, after returning from a movie (it was Jim Jarmusch's "Paterson"; fairly good, but an art-house movie), I did a Catalog Rebuild of "Media Set Red"—thinking that my stopping the "Sat. Backup" execution had messed up the Media Set—before leaving the MBP and Mac Pro on to run "Sun.-Fri. Backup" to "Media Set Red" as scheduled. I see now, from this thread, that my instinct to do a Catalog Rebuild of "Media Set Red" was correct. To rerun this test next Saturday morning, I'll have to first do a manual Recycle of "Media Set White" and then—in place of the scheduled "Sat. Backup" script—schedule a run of the "Sat. Backup Incremental" script with "Media Set White" as the Destination followed by an overlapping Copy Media Set script run with "Media Set White" as the Source. P.S.: This is post #14 in the thread.
  13. Hold on there, jweisbin and others, yesterday I thought of a variant on the simultaneous-backups solution that is much more promising. I realized that the Retrospect Windows 11 UG excerpts I quoted in post #7 in this thread mean that you can simultaneously use the same Disk Media Set as both a Destination for a backup and a Source for a Copy Media Set! That, of course, assumes that what applies to Retrospect Windows also applies to Retrospect Mac—which as we all know has the same underlying (but occasionally undocumented) capabilities because the Retrospect Inc. engineers built it that way. How could I possibly resist running a test of this? Early this morning, before my "Sun-Fri Backup" script was scheduled to make its last MacBook Pro backup to "Media Set Blue" before going to the bank safe deposit box, I booted my Mac Pro Desktop Edition "backup server" and reset the "Allow n Activity Threads" Preferences dropdown to 4. I then changed my Copy Media Set script that had as its Source "Media Set Blue" and its Destination a "Media Set Violet", with its single member on a spare FireWire 800 portable drive, so that its Activity Thread was set = 2 and it was scheduled to start 5 minutes after my "Sun-Fri Backup" script. (Because I run the Instant Scan process on my MBP, I know that a Backup script will begin actual copying in less than 5 minutes after the script starts running.) I then booted my MBP and went back to sleep. When I awoke an hour after "Sun-Fri Backup" was scheduled to run, I found that that script had completed execution in Activity Thread 1 in only two more minutes (22 minutes instead of 20 minutes) than it usually does but that the Copy Media Set script was still executing merrily in Activity Thread 2. The Copy Media Set script eventually completed execution in about 3:25 hours (remember that it is copying the Recycle backups of all my 6 drives, plus 6 more incremental backups of my MBP), which is around 10 minutes more than it took per post #9 in this thread. This evening I ran a Restore from "Media Set Violet" of the 7 cumulative backups of my MBP onto another spare FireWire 800 drive, and then successfully booted the MBP from that drive. Why, you may well ask, did using "Media Set Blue" simultaneously as both a Destination for the "Sun-Fri Backup" script and a Source for the Copy Media Set script not slow the backup appreciably? The reason is that, at least when the Source for a Backup is a hard disk drive, the gating factor is primarily speed of directory traversal on the Source drive—not LAN speed or Destination drive speed (because the Destination drive is not traversing directories). That may not hold true if the Source is an SSD, but I'm not about to spend money upgrading an Early 2011 MacBook Pro which will become officially "obsolete" when the ball falls at midnight tonight in Times Square. So, jweisbin, on your 16GB-RAM Mac mini Server you could follow each daily "client" backup-to-Disk-Media-Set script with an near-simultaneous Copy Media Set to Tape script. Because the Copy Media Set would be copying the entire cumulative Disk Media Set, it would probably finish hours after the last "client" had been incrementally backed up—but that wouldn't matter much because the Copy Media Set script would only be using drives on your "backup server" machine. Your "client" machines would already be freed up for daytime non-Retrospect activities, and you could grab your tape at the end of the day and take it off-site. Alternatively you could run a near-simultaneous Copy Backup script instead of the Copy Media Set script, but that would require bringing an up-to-date backup tape back on-site before each backup script run—and the Copy Backup script would fall behind when first started because of the need for it to space down to the previous end of the tape. I personally think a pair of alternating Copy Media Set scripts running near-simultaneously with the Backup script would be simpler. P.S.: This is post #13 in the thread.
  14. To be fair to Mayoff, the disk-to-disk-to-tape solution suggested by Lennart Thelander in post #2 of this thread would be the standard one under most circumstances. It would be the only solution if jweisbin wants to make a tape copy of an already-existing Disk Media Set. However the simultaneous-backups solution would be worth trying under the following circumstances: 1) jweisbin is starting a new backup procedure from scratch. 2) He/she wants to take the Tape Media Set off-site every time it is updated, and bring it back on-site for the next Backup run. In those circumstances simultaneous Backups might run faster than a Backup to a Disk Media Set followed by a Copy Backup to tape, but only if the Source(s) for the Backup runs is/are disks locally-attached to the "backup server" or clients whose disks are SSDs. My experience is that, doing a Recycle backup of all my drives—client and local—every Saturday, the copy-specific speed for my Early 2011 MacBook Pro client—which has a HDD instead of an SSD— is around 17GB/hour vs. 125GB/hour for a local disk—and the gating factor for the MBP is not the 36GB/hour effective copying speed (allowing for 20% overhead) of my 100Base-T LAN. Note that the copy-specific speed of my Copy Media Set tests was also about 125GB/hour. P.S.: Revised next-to-last sentence in second paragraph to use copy-specific speeds; verify should be excluded because—per the verify paragraph in this section of the old Wikipedia article—verify of client backups is done using MD5 checksums stored on the "backup server" during copy, but verify of local-drive backups is done using byte-by-byte comparison. P.P.S.: The second paragraph in this post does not mean that simultaneously-running scripts would not be useful in many cases. It just means that simultaneously-running Backups would not be useful in most cases. To be usefully run simultaneously, the scripts would have to only use locally-attached drives as Sources; thus the simultaneously-run Copy Media Set scripts I tested per the previous posts in the thread—or the equivalent kinds of Utility scripts—could be useful. P.P.P.S.: This is post #12 in the thread.
  15. I posted the contents of my last three posts in this thread as Support Case #52765. Today I received the following reply from support@Retrospect .com (with Mayoff's name on it): "Agent Response: The original forum thread makes reference to a very simple question and it looks like Lennard Thelander already answered the question. All server editions of Retrospect support the ability to back up using multiple executions at the same time. If a user wants to write to 2 different tape drives at the same time, you must use the Advanced Tape Support add-on license." To which I replied in the support case—which is now marked Closed: "In his/her OP in the thread, jweisbin said 'I thought I would set up a disk backup set and a tape backup set that would be perfectly in sync.' Based on my tests, there's no reason he can't do that with simultaneously-running scripts. I actually have an old HP DAT drive that worked in June 2015, when I tested Retrospect 6.1 on my Digital Audio G4. I would rerun the test Copy Media Set scripts simultaneously using it for one Destination Media Set, but unfortunately my Mac Pro doesn't have a SCSI card. So I guess we'll just have to wait for jweisbin to try it, which I'll urge him/her to do. If it works, the Retrospect Inc. documentation committee will have a little problem. Will they want the knowledge of this capability of Retrospect Mac 13 to be continue to be disseminated only as a "secret" on the Forums, or will they want to officially document it—either in the User's Guide or as a KnowledgeBase article? That's why this Support Case is a Documentation request." So try it, jweisbin, and please let us know if it works. I believe you have a Server Edition, in which the Retrospect Preferences "Allow n Activity Threads" dropdown should default to 4. So all you should need to do is create two backup scripts, one to a Disk Media and one to a Tape Media Set, and in the Summary pane for the scripts set the "Use" dropdown to different Activity Threads. You can schedule them for the same time, and they should run simultaneously. P.S.: This is post #11 in the thread.
  16. I was curious what would happen if I submitted the two scripts from yesterday's test before resetting the "Allow n Activity Threads" Preferences dropdown to 4 from 1, so I tried it late this morning. One thing I learned is that, in the Summary tabbed view of the Console detail for the script, the dropdown with "Use" to its left and "Activity Thread 1" showing is grayed-out so long as the "Allow n Activity Threads" Preferences dropdown is set to 1. So the Retrospect Inc. engineers did what the Marketing grinch forced them to do in a prevent-administrator-errors fashion. I re-submitted the Copy Media Set script that had had its Activity Thread set to 2, and it immediately started executing without the log showing an Activity Thread. I then re-submitted the other Copy Media Set script that had had its Activity Thread set to 1; it was made to wait. I then killed that waiting activity, set the "Allow n Activity Threads" Preferences dropdown to 4 from 1, and re-submitted the other Copy Media Set script with its Activity Thread set to 2.; it started executing immediately. The conclusion I draw from this test is that, if you want to execute multiple scheduleable (as opposed to Proactive) scripts simultaneously in Retrospect Desktop Edition, you should first set the "Allow n Activity Threads" Preferences dropdown to a number greater than 1 and then set the scripts' Activity Threads. If you previously set the scripts' Activity Threads but then subsequently re-started the Engine, I wouldn't rely on those scripts' Activity Threads being remembered without resetting them. P.S.: This is post #10 in the thread.
  17. More than a fighting chance; I did it yesterday. I shouldn't be able to do that in my lowly Desktop Edition of Retrospect Mac 12.5, but this thread shows that the only problem is that every restart of the Engine in Desktop Edition resets the "Allow n Activity Threads" dropdown in the General tab of Retrospect Preferences to 1. That was obviously forced on the Retrospect Inc. engineers by a grinch in Marketing per the quote from the WP article, but the engineers didn't do more than the minimum; once the Engine had been started by rebooting my Mac Pro I was able to reset the "Allow n Activity Threads" dropdown to 4. I then simultaneously ran two Copy Media Set scripts as described here and my immediately-following post in that thread. In both cases the Source was my "Media Set Blue", the "Media Set of the week"—"G-Drive White" containing "Media Set White" now being ensconced in my bank safe deposit box. The first script, with its Activity Thread set = 1, had as its destination a "Media Set Violet" with its single member on a spare FireWire 800 portable drive; it ran in 3:15 hours including Verify. The second script, with its Activity Thread set = 2, had as its destination a "Media Set Purple" with its first member in the available space on USB3 drive "G-Drive Red" and its second member in the available space on USB3 drive "G-Drive Blue"; it ran longer including Verify, because the Source for both scripts is also on USB3 drive "G-Drive Blue". This meant that, in the latter half of the second script's Copy (as opposed to Verify) stage, two sets of reads and one set of writes were being interspersed on "G-Drive Blue"—speaking of "serializing conflicting executions" per post #7 in this thread! P.S.: This is post #9 in the threaad.
  18. DavidHertzberg

    Saving backups to OneDrive for Business (Retrospect 7.7)

    I have now discovered that Mayoff was factually correct in post #8, but there's no way I could have known that. The apology and explanation are here, because it's a Wikipedia question.
  19. DavidHertzberg

    Major enhancement of Wikipedia article on Retrospect

    Mayoff made this post in the "Saving backups to OneDrive for Business (Retrospect 7.7)" thread: I pooh-poohed the statement in a subsequent post in that thread, but it now looks as if Mayoff was factually correct for Retrospect Windows 7.7. In researching a couple of threads for the "Retrospect 9 or higher for Macintosh" forum, I discovered over the last few days that Retrospect Windows 7.5 was apparently retrofitted with most of the non-UI features of Retrospect Mac 8. However there was no way I could have known that, because the Retrospect Windows 7.5 User's Guide did not have a "What's New" chapter and there seems not to have been any press release for Retrospect Windows 7.5. I was getting my information from this section of the (old) Wikipedia article, which DovidBenAvraham wrote from EMC's Retrospect Windows 7.0 press release. In writing for a WP article, any factual statement DBA—or anyone else—makes must be referenced to a source outside of Wikipedia. In fact DBA is skating on thin ice in the last sentence of the linked-to-above section, because he doesn't actually show the YouTube video as a reference (which he didn't do because it would have been considered another "primary source"). "Original research" is not allowed, which IMHO—and that of DBA—would include any systematic comparison of the Retrospect Windows 7.5 UG with the Mac 8 UG that I—and DBA—have done. So it looks like the linked-to-above section of the WP article will have to remain as is, accurate for Retrospect Windows 7.0 features but inaccurate for Retrospect Windows 7.5-7.7 features. The explanation for why the then-current release of Retrospect Windows retained the name "Retrospect Windows 7..." for 8 years despite major enhancements, and for why there was no press release for any of the major enhancements, probably lies in the last paragraph of this section of the WP article. I am making this post for belated accuracy, but also to be instructive to any budding WP editors who may be reading this thread.
  20. A friendly soul has just done a major enhancement of the Wikipedia article on Retrospect, bringing it up from around 2006 to the present. In doing so he made some statements in the section "Retrospect Macintosh 10 and Retrospect Windows 8" about the updating of Retrospect Windows 8 that may be open to question. He doesn't know for sure, because he uses Retrospect Mac. He would appreciate any replies with corrective information, and will try to incorporate these in the article. Of course, this being Wikipedia, there is nothing stopping anyone from editing the article himself/herself. Just remember to: A ) Sign up as a WP editor to prevent your IP from being made public. B ) Spend a few minutes learning to Edit Source; it's not that difficult. C ) You must provide references for factual statements; use the Templates dropdown in the editor and fill in the minimum necessary fields—you'll get automated warnings if you don't. D ) Keep a Neutral Point of View in what you write; WP requires it. Thanks!
  21. DovidBenAvraham has just catapulted my brain into the 21st Century, by reminding me that—as he has now clarified here in the "Powerful new engine" item—Retrospect Mac 8 and above "is capable of simultaneously performing multiple Backup, Restore, and Copy operations in separate threads". So, since jweisbin's previous posts show that he/she is running a Server Edition of Retrospect, he/she should be able to simultaneously run two backup scripts by designating that they run in separate threads. And, based on what I quoted from the Retrospect Windows 11 User's Guide in my immediately-preceding post in this thread, there is a fighting chance that the two scripts could simultaneously backup from the same Disk Source in Retrospect Mac 13. How, you may well ask, can jweisbin designate that the scripts run in separate threads? I had a vague recollection that this is possible, but there is absolutely no mention of this capability in the Retrospect Mac 13 User's Guide. So I thought of another undocumented capability for scripts that derek500 brought to my attention last year, which is the capability of rearranging the sequence in which Sources are processed by a script. That is done by dragging Sources in the Details pane of the Summary tabbed view of the Console detail for the script. So I looked in the Summary view, and—sure enough—at the bottom of the left-hand Overview pane is a dropdown with "Use" to its left and "Activity Thread 1" showing! However jweisbin must also ensure that his/her copy of Retrospect Mac 13 can run multiple simultaneous activity threads. Page 184 of the Retrospect Mac 13 UG says that this is done with the "Allow n Activity Threads" dropdown in the General tab of Retrospect Preferences, which are accessed from the Retrospect Menu. Page 184 says this preference defaults to 4, but in my Desktop Edition copy of Retrospect Mac 12.5 it defaults to 1—which DBA says in the "Retrospect 8.0 Desktop 3-User" item towards the bottom of the WP article section linked to in the first sentence of this post is because it "thus simulated the operating limitations of Retrospect Macintosh 6, and was priced accordingly." The same paragraph on page 184 of the UG says "In general, you should have one Gigabyte of free RAM for each activity thread you will run." If the two scripts would be backing up more than one Source, I would also advise jweisbin to vary the Sources' sequence in the two scripts (per the method mentioned in the paragraph above this) in order to avoid having both scripts compete for positioning of the same Disk read heads. So why is the capability of designating which activity thread a script runs in not documented? My favorite hypothesis is that various enhancements to the Summary tabbed view of the Scripts detail were done by a programmer who then left Retrospect Inc. without reminding his/her boss to include the enhancements in the next edition of the UG. A more macabre hypothesis is that the body of that programmer will eventually be found buried near where the body of Hans Reiser's wife Nina was found, which is actually not far from Walnut Creek CA (insert appropriate smiley here). However the most likely hypothesis does not involve the programmer leaving, but merely the famous Retrospect Inc. documentation committee getting tripped up by a terminology difference; an "activity thread" in Retrospect Mac turns out to be an "execution unit" (it took me some time to figure this out) in Retrospect Windows (we shouldn't expect those ignorant Windows administrators—insert appropriate smiley here— to know what a "thread" is), and setting it for a script is documented up the wazoo in the Retrospect Windows 11 UG (and, FWIW, in the Retrospect Windows 7.5 UG). Naturally I intend to belatedly file a Support Case requesting Retrospect Mac UG documentation of both Console Scripts category Detail Summary enhancements, which will constitute a Christmas/Hanukkah/Kwanzaa present for Mayoff. P.S.: In final paragraph, a Retrospect Mac "activity thread" is a Retrospect Windows "execution unit"; its specification for a script is thoroughly documented in the Retrospect Windows UG. P.P.S.: It turns out that the capability of rearranging the sequence in which Sources are processed by a script—mentioned in the second paragraph of this post—has also been thoroughly documented from the Retrospect Windows 7.5 UG onward (I just had to search for "drag"). So, insofar as the non-documentation in the Retrospect Mac UG of this capability as well as the one discussed in this thread is concerned, it seems as though a variant of my "most likely hypothesis" is the correct one. Because rearranging the sequences in which Sources are processed has no terminology difference between Windows and Mac, the variant is that the committee was not tripped up—but instead ran afoul of the Retrospect Mac 8-and-after UG non-expansion imperative hypothesized by DBA in the first sentence of the last paragraph of this section in the WP article. P.P.P.S.: This is post #8 in the thread.
  22. Thanks for the hardware and software particulars, jethro. I apologize for implying that you might have penny-pinching tendencies. I had a hunch that you didn't buy such a fairly fancy Mac mini Server just to run Retrospect, so I would be interested in knowing what "the typical server software" is—aside from OS X Server itself. That software may be what is causing your Retrospect slowness and crashing. In regard to that, I strongly urge you to contact Retrospect Inc. Tech Support. First phone one of the numbers I listed in post #6 in this thread. However don't stop there; IME A. (if he still works there) isn't the brightest bulb in the chandelier. Insist on speaking to Mayoff (dial again and if necessary phone again and dial x806 to get him), and file a Support Case by going to www.Retrospect.com and clicking the pane with the telephone icon in it at the upper right. As far as the "long startup times" is concerned, "syncing catalogs" (which presumably you see at the upper left of the Retrospect main window) is shown while the Retrospect Console is obtaining information from the particular Retrospect Engine(s) you are running. If you don't understand that, DovidBenAvraham had a brief but pithy explanation here in the "Retrospect Macintosh 8" section, starting with the last sentence in the "Powerful new engine" indented item-bulleted paragraph. Upon booting my Mac Pro "backup server", my Startup Items Retrospect Console app's "syncing catalogs" takes at most a second with my single RetrospectEngine process (I have Retrospect Mac 12.5 Desktop Edition running under OS X 10.10.5); if it is hanging for several minutes on your "backup server", you definitely have a problem that should be discussed with Retrospect Inc. Tech Support. As far as 5.6TB turning out to be over 6TB, you are almost certainly the victim of a decades-old traditional difference in measurement units between programmers and storage-drive-makers. I ran afoul of it over a year ago, and put an explanation into this post. For example Retrospect says my three USB3 G-Tech G-Drive Slims each have a capacity of 465GB, but HGST marketed them as 500GB drives. When I multiply 1024 by itself 3 successive times to get 1TB binary and then multiply that product by 5.6, I get nearly 6.2TB decimal using my handy-dandy Apple Calculator app. What we have here is what I have termed a "conceptual bug", and I don't think we have a prayer of getting Retrospect Inc.—a spin-off founded by programmers—to fix it. "Tradition, tradition ..." (about a year ago I took my now-ex-girlfriend, at her request, to a staging of "Fiddler On the Roof").
  23. I see from Wikipedia and his first quoted paragraph that jethro must have the Mid-2011 MacMini(5,3). Therefore he has as many CPU cores and twice as much RAM as my Mac Pro "backup server", which runs Copy Media Set much faster than jethro. Until now I had forgotten about jethro's original post in the 2 March thread, which reported he was having problems opening Retrospect and keeping it running. From jethro's first quoted paragraph he is still having the problems, so I am wondering along with Mayoff whether jethro has a hardware/software issue not related specifically to Copy Media Set. For instance, jethro has double the official maximum RAM for the MacMini(5,3). Unless jethro just happened to pick up a bargain on a previously-custom-built machine with quadruple the amount of RAM that he should normally need for Retrospect, I am wondering what other software he is running on his MacMini(5,3). Could it be that he is running Mac OS X Server? If so, he must be aware that—according to the ever-helpful DovidBenAvraham's WP article—he should be running a Retrospect Edition that supports OS X Server, not merely the Desktop Edition. Is that why jethro did seem and does seem reluctant to contact Retrospect Support about his other Retrospect problems (insert appropriate smiley here)—even last March when he ws still entitled to free telephone support for Retrospect Mac 13? Or, considering hardware only, is jethro just avoiding the possibility that his machine has a flakey internal hard disk drive—which would cost some money to replace (insert appropriate smiley here)? Before I saw jethro's quoted post #8 in this thread, I did have the additional idea that jethro's presumed leaving "Don’t add duplicate files to the Media Set" checked might be causing cumulative RAM-grabbing by the RetrospectEngine process while his Copy Media Set script is running. So this afternoon I reran the same test as in post #7 in this thread, except with that option checked. Intermittently looking at Real Memory (which was almost as unexciting as watching paint dry) in Activity Monitor, I noticed sudden temporary increases during Updating Catalog File phases. Each temporary Real Memory increase was greater than the increase after the previous Backup was copied, so it is possible that—since jethro's Copy Media Set run must be copying hundreds of Backups of the same Source—his temporary increases might be eventually exceeding the available Real Memory and causing what used to be known as "virtual memory thrashing". I was considering converting the posts I have made in this thread into a Support Case. However I no longer feel I can do that until jethro posts further particulars on what version of OS X and other software he is running on his Mac mini Server, his Edition of Retrospect Mac 13, and whether he has tested his internal hard disk drive. In any case—even if jethro fixes his Copy Media Set problem so that it runs as fast as mine does, his proposed weekly run cannot run within a single working day unless he starts a new Media Set every 8-10 months (depending on how long his workday is) . Do the arithmetic: 6TB/5 years-saved = (1200GB/year-saved) / 100GB/hour = 12 hours/year-saved. IMHO jethro should instead follow the procedure I suggested in this post, which uses Copy Backup on a weekly basis after an initial Copy Media Set. P.S.: The test described in my third paragraph ran in roughly the same amount of time as my previous tests, allowing for the additional No Media Action run of my "Sun.-Fri. Backup" script to "Media Set White" I had made before I ran the latest test. So leaving "Don’t add duplicate files to the Media Set" checked didn't seem to increase the running time. OTOH "Don’t add duplicate files to the Media Set" was checked during all the three daily runs of the "Sun.-Fri. Backup" script to "Media Set White", so I wouldn't expect there would be any duplicate files in the Source of the Copy Media Set. P.P.S: Revised last paragraph; jethro's last quoted paragraph says he's going to start start a new Media Set at the beginning of 2017.
  24. Prompted by what Mayoff said on the phone this morning, I started to think about overtaxed resources other than CPU speed—namely RAM. So early this afternoon I reran the same test as in post #5 in this thread, but with Activity Monitor also running. A year ago, when I first starting benchmarking the "backup server" on my Mac Pro for my thread on Ars Technica, I added a Real Memory column to Activity Monitor because I—being an old fuddy-duddy—do not completely believe in the reality of paging (it's virtual memory, after all). So, in between folding socks out of the dryer in the bedroom where my Mac Pro sits, I looked at the Real Memory used during the Copy Media Set run. I noticed that the biggest user of Real Memory was the RetrospectInstantScan process, not the RetrospectEngine process. And the Real Memory used by RetrospectInstantScan went up from about 720MB to about 820MB. It didn't keep climbing from there, however, as I thought it might. However 820MB is a fair amount of RAM by any standards. It didn't make any difference on my Mac Pro "backup server", because I upped its RAM from 3GB to 7GB when I inherited it in late spring 2015. But it might make a difference on Jethro's Mac mini Server, which only comes with 4GB RAM if it's the model I linked to from the P.S. of post #4. Therefore I recommend that jethro, and anyone else running a Copy Media Set script, first go to System Preferences->Retrospect and turn off Instant Scan for the duration of the script run. I also recommend that jethro make an investment in additional RAM; my additional 4GB cost me US$43 from—IIRC—Other World Computing. P.S.: Look at this thread. Problem sound familiar? Note that, if you click the link for item #11 in the Low End Mac page linked to in the P.S. of post #4 in this thread, that link goes to a Low End Mac page that says in the next-to-last paragraph that OWC has found you can upgrade that model to either 8GB or 16GB—and links to the OWC page.
  25. As soon as Retrospect Inc. opened this morning, I phoned Mayoff. He suggests that jethro phone Support, (888) 376-1078 or (925) 476-1030. He says that he's seen situations in which slowdowns like this turn out to be hardware or overtaxed-resources problems. Good luck jethro, and please post what Support tells you to this thread.
×