Jump to content

DavidHertzberg

Members
  • Content count

    1,190
  • Joined

  • Last visited

  • Days Won

    40

Everything posted by DavidHertzberg

  1. Apparently Retrospect Inc. will pay no attention to anyone's thread in the "Product Suggestions—Mac OS X" sub-forum unless it gets some kind of up-vote . On 12 December 2016, in response to a letter I snail-mailed to Mayoff, I received an e-mail through a Mayoff account that was signed by JG Heithcock, CEO, Retrospect, Inc.. In it he says in a paragraph "From reading your letter, I think the main issue is that you view the forums as a good place to talk to us, Retrospect, Inc. But we view the audience of the forums as restricted to our customers [my emphasis]. The one caveat we have made on that is for feature requests, largely as we would like to see if other customers also agree on the desirability and feature set for these requests [my emphasis]." Mr. Heithcock followed this paragraph with one quoting the forum rules. Given that he did so, and that I have not signed any kind of non-disclosure agreement, I have no hesitation in making his paragraph above public. What does Mr. Heithcock mean about feature requests? Is he looking for Likes? Is he looking for additional posts by other users in request threads, pro or con or in-between? I intend to phone Mayoff on Monday, and see if I can get an answer from him.
  2. On second thought about my post #18 immediately above, maybe we don't need the additional CopyLock and CopyUnlock script types at all, or the additional option for Backup scripts. Instead make the setting of the CopyLock bit in the Source drive's Media Set Catalog File header be standard operating procedure for a Source drive about to be backed up to a particular Media Set. It would also be standard operating procedure for a Copy Media Set or Copy Backup script run to go into a pause state when it encounters a Catalog File header for a Source drive on which the CopyLock bit is still set. The CopyLock bit would—of course—be cleared for a particular drive's header in a Media Set Catalog File as soon the copy phase of a Backup for that drive had been completed and the drive's Catalog File had been updated for the Media Set. As soon as a CopyLock bit had been cleared, an overlapping Copy Media Set or Copy Backup script run that had gone into a pause state because of that CopyLock bit would come out of the pause state and proceed. This would make the CopyLock bit a "drive backup copying in progress" bit. That would be sufficient to ensure that a Copy Media Set execution would run far enough behind any Backup run it is overlapped with that its copying—based on a drive's Catalog File entries—could never occur before that drive's Catalog File entries have been updated by the Backup. Probably it would be a good idea to add an "ignore CopyLock" option to Copy Media Set and Copy Backup scripts, in order to handle weird situations in which a Backup script run for a particular drive had not completed its copy phase successfully—but the administrator nevertheless wanted to run a Copy Media Set or Copy Backup execution using the Media Set's Catalog File entries for that drive as they existed previously. Actually even this option might not be needed; a Catalog File Rebuild for a Media Set would automatically clear any CopyLock bit that had been set for a Source drive's header within that Catalog File. So how about it, jweisbin and others? What I have just proposed is a modest enhancement to Retrospect that would enable the overlapping I tested in posts #15 and #16 in this thread to really work as I thought it had, thus increasing the usefulness of a major capability that has been in Retrospect since Mac version 8. Would it be too much to ask you to show a little support? P.S.: This is post #19 in the thread.
  3. OK, where do we go from here in regards to Copy Media Set runs overlapped with Backup runs, considering my post #17 immediately above? It is now evident that the existing version of Retrospect can't do what I was proposing in posts #15 and #16. Last night, on my way to see the movie "Hidden Figures" (IMHO very good), I conceived an enhancement to Retrospect that would enable it to do so. In its simplest form, it would add a CopyLock script type—referencing an existing Media Set. A CopyLock script run would simply access the Catalog File header for each Source drive in the Media Set, setting a bit that would cause a Copy Media Set or Copy Backup run with that Media Set as the Source to go into a pause state when it tried to use the Catalog File to copy a Source drive whose bit was still set. That bit in the Media Set's Catalog File header for a Source drive would be cleared when a Backup script completed the Copying phase for that drive and updated the Catalog File for it, enabling the Copy Media Set or Copy Backup run to come out of the pause state and proceed to copy that drive. This would ensure that a Copy Media Set execution would run far enough behind any Backup run it is overlapped with that its copying—based on a drive's Catalog File entries—could never occur before that drive's Catalog File entries have been updated. To cover the practical possibility of scripts never getting run or crashing, the enhancement would also need to include a CopyUnlock script. This would simply do the opposite of a CopyLock script, accessing the Catalog File header for each Source drive in the Media Se and un-setting the bit that would cause a Copy Media Set or Copy Backup run with that Media Set as the Source to go into a pause state. Ideally we might also want the enhancement to include a new option for Backup scripts that would do the equivalent of a CopyLock bit-setting just before starting the actual Backup process for a particular Source drive, ensuring that that Source drive could not have a Copy Media Set or Copy Backup done on it via the same Media Set before its Backup copying had been subsequently completed and its Catalog File header updated. That second enhancement would take care of the situation in which a drive is dynamically cabled to a Source computer after a CopyUnlock script run has completed and a Backup script run has started, but before the Backup script run gets to processing that Source computer. My question to all of you forum readers is: do you want me to bother to post a request for such an enhancement in the "Product Suggestions—Mac OS X" sub-forum? My OP in this thread reports that the CEO of Retrospect Inc. and his cabal will not consider a feature request unless there is evidence of support for it from other Retrospect customers. I'll find out Monday exactly what kind of evidence of support he requires, but I won't proceed further until I have prior indications of support from you. Frankly, although I had fun doing the tests for Copy Media Set runs overlapped with Backup runs, I'm getting more than a bit tired of doing work for other people who don't seem to appreciate it. I'm afraid that appreciation must now go beyond simply bumping the view counts for threads I contribute to; I don't consider myself to be in the entertainment business (do not insert appropriate smiley here—I'm dead serious). I'll even give you a couple of arguments against a request for such an enhancement. First, maybe nobody—including me—except jweisbin needs the overlapping. With many people (not including me) installing SSDs in their computers, Backup script runs for a string of "clients" may be taking such an increasingly shorter time that overlapping of Copy Media Set script runs with Backup script runs may no longer be necessary. Second, maybe everybody except jweisbin is satisfied doing a non-overlapped Copy Media Set to a Tape or Cloud Media Set over the weekend—even if that means an off-site backup once a week instead of every workday. So let's see some posts from someone besides myself in this thread, pro or con! P.S.: This is post #18 in the thread.
  4. Stop, jweisbin and others. Do not attempt to overlap executions of Backup scripts and Copy Media Set scripts until you have read the rest of this post—which will probably convince you not to attempt it at all. I phoned Mayoff this afternoon by calling Tech Support at extension 3 at the Retrospect Inc. number (I had been obliged to leave a message for him yesterday at x806; he is "very busy"—make of that what you will). I meant to quickly give him a heads-up about this thread and my experimental results, and to ask a couple of related questions. Fortunately the first related question I asked was why overlapping didn't work with a Recycle Backup run per post #14 in this thread. He said it's because Copy Media Set uses the Catalog File, and a Recycle wipes out every entry in the Catalog file and does not create new entries until the Updating Catalog File phase(s) of the Backup run. I immediately realized that—on a Copy Media Set run that is overlapped—only the files that are already entered in the Catalog File are copied. That means that an overlapped Copy Media Set run will not copy files that were backed up for the first time in the overlapped Backup run, and will not copy the latest versions of files that were updated since the previous Backup run. This explains something that has been bothering me since yesterday, which is that my statement in my fifth sentence in the first paragraph of post #16 is not quite correct. The total number of files copied from both drives in both Backup script runs is 1236, but the total number of files copied in the Copy Media Set run is only 1235. Checking—now that I know what to look for—in Documents, I see that I modified a file (actually the LibreOffice version of this post) at 9:05 a.m. on 4 January, which means that the modified file would have been backed up in the 5 January Backup run. That is the file whose latest version must not have been copied in the overlapped Copy Media Set run on 5 January, because its latest version was not yet in the Catalog File. I executed an extra "Sun.-Fri. Backup" script run this afternoon, backing up to "Media Set White" (which will be recycled tomorrow morning), and plugging in the USB drive "OS X File Transfers" to get a second Source drive on my MacBook Pro. I observed that the "updating catalog" phase for each drive, although it is not shown in the Log, occurs after the Copying phase for each drive backed up. This means that a Copy Media Set execution must run far enough behind any Backup run it is overlapped with that its copying—based on a drive's Catalog File entries—never occurs before that drive's Catalog File entries have been updated. Given how much faster an execution that involves only drives local to the "backup server" runs than an execution that involves drives attached to a "client", I don't think that can ever be guaranteed for a Copy Media Set execution that overlaps with a Backup of "client" drives . P.S.: This is post #17 in the thread.
  5. Here's the further test, showing a Copy Media Set script can copy the backups from two or more days of daily Backup script runs. The logs are at the bottom of this post—in reverse chronological order—to prevent eye fatigue, but I'll give you the highlights up top. First, note that the two Backup script runs each also copied a second drive on David’s MacBook Pro—the USB drive "OS X File Transfers" which was left plugged in. Second, note that again the Copy Media Set script started running 5 minutes after the second-day Backup script, and that it was thereafter running simultaneously—in Activity Thread 2—with the Backup script running in Activity Thread 1. Third, note again that the Copy Media Set script copied the same number and total GB of files scripts as the total copied in the two Backup scripts. Fourth, note that the speed of the "Macintosh HD" copy phase for the second Backup script run was 501.6 MB/min. vs. 686.3 MB/min. for the Copy Media Set script run, but that the speed of the "Macintosh HD" compare phase for the second Backup script run was 1,848 MB/min. vs. 3,722 MB/min. for the Copy Media Set script run's verify phase. That shows the Copy Media Set run was—correctly—being somewhat slowed so as not to overrun the progress of the second Backup run's copy phase. Log for Copy Media Set script run Thurs. 5 January: Copy Media Set Log 5 January 2017: + Executing Copy Media Set - Media Set Red at 1/5/17, 3:05:00 AM (Activity Thread 2) To Backup Set Media Set Pink... - 1/5/17 3:05:16 AM: Transferring from Media Set Red 1/5/17 3:23:34 AM: Execution completed successfully Completed: 1,235 files, 11.8 GB Performance: 686.3 MB/minute Duration: 00:18:17 (00:00:44 idle/loading/preparing) - 1/5/17 3:23:34 AM: Verifying Media Set Pink 1/5/17 3:27:13 AM: Execution completed successfully Completed: 1,235 files, 11.8 GB Performance: 3,722 MB/minute Duration: 00:03:38 (00:00:24 idle/loading/preparing) Log for Backup script run Thurs. 5 January: + Normal backup using Sun.-Fri. Backup at 1/5/17, 3:00:00 AM (Activity Thread 1) To Backup Set Media Set Red... 1/5/17 3:00:02 AM: Connected to David’s MacBook Pro * Resolved container David’s MacBook Pro to 2 volumes: Macintosh HD on David’s MacBook Pro OS X File Transfers on David’s MacBook Pro - 1/5/17 3:00:00 AM: Copying Macintosh HD on David’s MacBook Pro Using Instant Scan 1/5/17 3:00:41 AM: Found: 680462 files, 152327 folders, 49.9 GB 1/5/17 3:01:50 AM: Finished matching 1/5/17 3:02:15 AM: Copying: 396 files (5.7 GB) and 0 hard links 1/5/17 3:13:50 AM: Building Snapshot... 1/5/17 3:13:50 AM: Checking 152,327 folders for ACLs or extended attributes 1/5/17 3:15:30 AM: Finished copying 8,380 folders with ACLs or extended attributes 1/5/17 3:15:57 AM: Copying Snapshot: 2 files (217.5 MB) 1/5/17 3:16:13 AM: Snapshot stored, 217.5 MB 1/5/17 3:16:13 AM: Comparing Macintosh HD on David’s MacBook Pro *File "Macintosh HD/retroclient.log": didn't compare .... *File "Macintosh HD/Users/davidhertzberg/Library/IdentityServices/ids.db-shm": didn't compare 1/5/17 3:19:28 AM: Execution completed successfully Completed: 396 files, 5.7 GB Performance: 789 MB/minute (501.6 copy, 1,848 compare) Duration: 00:19:27 (00:04:36 idle/loading/preparing) ****** Warning: volume "OS X File Transfers on David’s MacBook Pro" has the "Ignore ownership" setting enabled. ****** - 1/5/17 3:19:29 AM: Copying OS X File Transfers on David’s MacBook Pro 1/5/17 3:19:41 AM: Found: 396 files, 163 folders, 282.7 MB 1/5/17 3:20:49 AM: Finished matching 1/5/17 3:20:50 AM: Copying: 0 files (0 and 0 hard links 1/5/17 3:20:53 AM: Building Snapshot... 1/5/17 3:20:54 AM: Checking 163 folders for ACLs or extended attributes 1/5/17 3:20:54 AM: Finished copying 1 folders with ACLs or extended attributes 1/5/17 3:20:54 AM: Copying Snapshot: 2 files (282 KB) 1/5/17 3:21:00 AM: Snapshot stored, 282 KB 1/5/17 3:21:00 AM: Comparing OS X File Transfers on David’s MacBook Pro 1/5/17 3:21:02 AM: Execution completed successfully Duration: 00:01:33 (00:01:29 idle/loading/preparing) 1/5/17 3:21:02 AM: Execution completed successfully Total performance: 783.7 MB/minute Total duration: 00:21:01 (00:06:05 idle/loading/preparing) Log for Backup script run Wed. 4 January: + Normal backup using Sun.-Fri. Backup at 1/4/17, 9:10:50 AM To Backup Set Media Set Red... 1/4/17 9:10:57 AM: Connected to David’s MacBook Pro * Resolved container David’s MacBook Pro to 2 volumes: Macintosh HD on David’s MacBook Pro OS X File Transfers on David’s MacBook Pro - 1/4/17 9:10:50 AM: Copying Macintosh HD on David’s MacBook Pro Using Instant Scan 1/4/17 9:11:49 AM: Found: 646509 files, 152251 folders, 49.5 GB 1/4/17 9:13:06 AM: Finished matching 1/4/17 9:13:40 AM: Copying: 530 files (5.9 GB) and 0 hard links 1/4/17 9:24:30 AM: Building Snapshot... 1/4/17 9:24:30 AM: Checking 152,251 folders for ACLs or extended attributes 1/4/17 9:26:13 AM: Finished copying 8,379 folders with ACLs or extended attributes 1/4/17 9:26:36 AM: Copying Snapshot: 2 files (207.6 MB) 1/4/17 9:26:45 AM: Snapshot stored, 207.6 MB 1/4/17 9:26:45 AM: Comparing Macintosh HD on David’s MacBook Pro *File "Macintosh HD/retroclient.log": didn't compare .... *File "Macintosh HD/Users/davidhertzberg/Library/IdentityServices/ids.db-shm": didn't compare 1/4/17 9:29:40 AM: Execution completed successfully Completed: 530 files, 5.9 GB Performance: 875 MB/minute (552 copy, 2,108.7 compare) Duration: 00:18:50 (00:05:00 idle/loading/preparing) ****** Warning: volume "OS X File Transfers on David’s MacBook Pro" has the "Ignore ownership" setting enabled. ****** - 1/4/17 9:29:40 AM: Copying OS X File Transfers on David’s MacBook Pro 1/4/17 9:29:49 AM: Found: 396 files, 163 folders, 282.7 MB 1/4/17 9:30:16 AM: Finished matching 1/4/17 9:30:17 AM: Copying: 310 files (137.2 MB) and 0 hard links 1/4/17 9:30:37 AM: Building Snapshot... 1/4/17 9:30:38 AM: Checking 163 folders for ACLs or extended attributes 1/4/17 9:30:38 AM: Finished copying 1 folders with ACLs or extended attributes 1/4/17 9:30:38 AM: Copying Snapshot: 2 files (282 KB) 1/4/17 9:30:44 AM: Snapshot stored, 282 KB 1/4/17 9:30:44 AM: Comparing OS X File Transfers on David’s MacBook Pro 1/4/17 9:30:54 AM: Execution completed successfully Completed: 310 files, 137.2 MB Performance: 548.9 MB/minute (411.7 copy, 914.8 compare) Duration: 00:01:13 (00:00:43 idle/loading/preparing) 1/4/17 9:30:55 AM: Execution completed successfully Total performance: 861.6 MB/minute Total duration: 00:20:04 (00:05:43 idle/loading/preparing) P.S.: This is post #16 in the thread.
  6. Last night I thought of a variant on the simultaneous-backups solution that is so much more promising than my post #13 in this thread that it appears to be the definitive solution for jweisbin and others with the same need. It keeps the near-simultaneous running of a Copy Media Set script along with a Backup script, but eliminates the need for that Copy Media Set script to do a from-scratch copy of the entire Media Set—a copy that can take hours. It does so by having both the "Match source files against the Media Set" and the "Don’t add duplicate files to the Media Set" options checked. These options are mentioned on page 162 of the Retrospect Macintosh 13 User's Guide, but are only discussed under the "In the Matching category, there are the following options:" sub-section of the "Backup Script Options" section on pages 116-117 of that UG. There it says that both options are key components of Retrospect’s Smart Incremental backups. And that's what jweisbin and others want for backing up to two media sets at the same time; to have the Copy Media Set script copy to the second Media Set only those backups that the Backup script has just added—doing a Smart Incremental backup—to the first Media Set. Naturally I ran a test early this morning (after booting my Mac Pro Desktop Edition "backup server" very early so I could reset the "Allow n Activity Threads" Preferences dropdown to 4); the logs are at the bottom of this post to prevent eye fatigue, but I'll give you the highlights up top. First, note that the Copy Media Set script started running 5 minutes after the Backup script, and that it was thereafter running simultaneously—in Activity Thread 2—with the Backup script running in Activity Thread 1. Second, note that both scripts copied precisely the same number and total GB of files. Third, note that the speed of the copy phase for the Backup script run was 524.3 MB/min. vs. 402.6 MB/min. for the Copy Media Set script run, but that the speed of the compare phase for the Backup script run was 2,061.5 MB/min. vs. 3,607.7 MB/min. for the Copy Media Set script run's verify phase. That shows the Copy Media Set run was—correctly—being slowed so as not to overrun the progress of the Backup run. I intend to rerun this test early Thursday or Friday morning, to confirm that with these options the Copy Media Set script can copy the backups from two or more days of daily Backup script runs. That means that jweisbin and others could run their overlapping Copy Media Sets alternately onto two or more different Tape Media Sets, e.g. "Tape Media Set MonWedFri" and "Tape Media Set TueThuSat", keeping yesterday's Tape Media Set off-site while bringing the day-before-yesterday's Tape Media Set on-site to be updated. Log for Backup script run: + Normal backup using Sun.-Fri. Backup at 1/3/17, 3:00:00 AM (Activity Thread 1) To Backup Set Media Set Red... 1/3/17 3:00:02 AM: Connected to David’s MacBook Pro * Resolved container David’s MacBook Pro to 1 volumes: Macintosh HD on David’s MacBook Pro - 1/3/17 3:00:00 AM: Copying Macintosh HD on David’s MacBook Pro Using Instant Scan 1/3/17 3:00:39 AM: Found: 676463 files, 152266 folders, 50 GB 1/3/17 3:01:48 AM: Finished matching 1/3/17 3:02:13 AM: Copying: 414 files (5.9 GB) and 0 hard links 1/3/17 3:13:37 AM: Building Snapshot... 1/3/17 3:13:37 AM: Checking 152,266 folders for ACLs or extended attributes 1/3/17 3:15:11 AM: Finished copying 8,375 folders with ACLs or extended attributes 1/3/17 3:15:41 AM: Copying Snapshot: 2 files (216.9 MB) 1/3/17 3:16:03 AM: Snapshot stored, 216.9 MB 1/3/17 3:16:03 AM: Comparing Macintosh HD on David’s MacBook Pro *File "Macintosh HD/retroclient.log": didn't compare ... *File "Macintosh HD/Users/davidhertzberg/Library/IdentityServices/ids.db-shm": didn't compare 1/3/17 3:19:02 AM: Execution completed successfully Completed: 414 files, 5.9 GB Performance: 835.1 MB/minute (524.3 copy, 2,061.5 compare) Duration: 00:19:02 (00:04:38 idle/loading/preparing) Log for Copy Media Set script run: + Executing Copy Media Set - Media Set Red at 1/3/17, 3:05:00 AM (Activity Thread 2) To Backup Set Media Set Pink... - 1/3/17 3:05:05 AM: Transferring from Media Set Red 1/3/17 3:20:25 AM: Execution completed successfully Completed: 414 files, 5.9 GB Performance: 402.6 MB/minute Duration: 00:15:19 (00:00:22 idle/loading/preparing) - 1/3/17 3:20:25 AM: Verifying Media Set Pink 1/3/17 3:22:21 AM: Execution completed successfully Completed: 414 files, 5.9 GB Performance: 3,607.7 MB/minute Duration: 00:01:56 (00:00:16 idle/loading/preparing) P.S.: This is post #15 in the thread.
  7. DavidHertzberg

    Sources mess up selected drives for backups

    If you think this is a bug that should be fixed by Retrospect Inc., you will have to submit it as a Support Case. For English speakers, that is done by going here, and filling out the form (sorry, I don't know what the equivalent addresses are for non-English speakers, but they can figure it out from their appropriate Retrospect website address). IMHO this is quite reasonable; obliging you to fill out the form provides Retrospect Inc. with useful details about your Retrospect installation that they would otherwise have to query you for. As a result, Retrospect Inc. will pay no attention to your post in this sub-forum. On 12 December 2016, in response to a letter I snail-mailed to Mayoff, I received an e-mail through a Mayoff account that was signed by JG Heithcock, CEO, Retrospect, Inc.. In it he says "From reading your letter, I think the main issue is that you view the forums as a good place to talk to us, Retrospect, Inc. But we view the audience of the forums as restricted to our customers [my emphasis]. The one caveat we have made on that is for feature requests, largely as we would like to see if other customers also agree on the desirability and feature set for these requests." That means that the only audience for "Retrospect bug reports" in this sub-forum will be other administrators of Retrospect Mac. Nevertheless, by posting in this sub-forum you are providing a useful service to us fellow Mac administrator peasants—one that is denied to administrators of Retrospect Windows who are evidently considered too peasanty (insert appropriate smiley here) to benefit from such bug notifications. Thank you. Please be aware that the "description of your issue" in the Support Case form is IME limited to about 2000 characters by the Support Case software. If you go over that limit your "description" will be broken up into a "description" plus one or more "additional notes". The same is true for any "additional notes" you may later post yourself. I suggest that, to avoid the appearance of choppiness in your Support Case, you create your case in a post in this forum and then copy it paragraph-by-paragraph to your Support Case. If this post sounds formulaic, that's because I intend it to be. I intend to post it in every new thread that appears in this sub-forum, unless the OP indicates that he/she has or will open a Support Case for the bug that the thread reports. Of course, Mayoff could take 5 minutes of his time to post a slightly-more-polite version of this post as a "sticky thread" that will always appear at the top of the forum. I don't intend to hold my breath until that happens (insert appropriate smiley here).
  8. Early Saturday I tried a high-volume rerun of the test in post #13 of this thread, to simulate what would happen if jweisbin or someone else ran a Copy Media Set script simultaneously with a Backup script for a number of "client" and local drives. It didn't work the way it was supposed to, but the further test didn't invalidate my concept. My failures may be instructive to others. My idea was to overlap my scheduled "Sat. Backup" Recycle Media Set script run onto "Media Set Red" ("Media Set Blue" having gone to the bank safe deposit box Friday afternoon) with a Copy Media Set script run copying "Media Set Red" onto a new "Media Set Pink" on my spare FireWire 800 portable drive (after Removing "Media Set Violet" from that drive). Because "Sat. Backup" takes about 10.5 hours, I booted my Mac Pro Desktop Edition "backup server" very early Saturday morning and reset the "Allow n Activity Threads" Preferences dropdown to 4, and scheduled a Copy Media Set script that had as its Source "Media Set Red" and its Destination "Media Set Pink" with its Activity Thread set = 2—with that script scheduled to start 5 minutes after my "Sat. Backup" script. I then left both my Mac Pro "backup server" and my MacBook Pro "client" awake and went to bed, expecting that because of BPH I would awaken before 8 a.m. Saturday to boot my Digital Audio G4 "client" before "Sat. Backup" had finished 5 hours of backing up and verifying the "Macintosh HD" on my MBP (the G4 is from 2001, and I prefer not to leave it running any more than necessary). I unexpectedly slept straight through until 9:45 a.m. Saturday, and found that my "Sat. Backup" script—instead of skipping the shut-down G4 and going on to backup my Mac Pro's local drives—was now backing up a second drive attached to the MBP. This second drive was my other spare FireWire 800 portable drive, which I had mistakenly left on and cabled to the MBP after successfully booted the MBP from it Friday evening. That second drive now contains a Restored copy of my MBP's "Macintosh HD"—the Retrospect 12 Client had mistakenly picked up the second drive per jelockwood's "Retrospect bug reports" thread, and I certainly didn't want to have "Sat. Backup" continue to backup and verify it for another 3.5 hours, so I stopped the "Sat. Backup" execution. Meanwhile the Copy Media Set script had never started executing; it had been waiting for nearly 7 hours while the "Sat. Backup" script ran. Until just now I thought the Copy Media Set script was waiting because I had somehow messed up the threads, even though the log shows the "Sat. Backup" script ran in Activity Thread 1 and the Copy Media Set script finally started executing—after I had stopped the "Sat. Backup" script execution—in Activity Thread 2. I see now from page 289 of the Retrospect Windows 11 User's Guide why it was waiting: "Certain operations require exclusive access to the Backup Set (e.g. updating a Catalog File, grooming, recycling [my emphasis], verifying media). When one of these operations is using a disk Backup Set, no other operations can use that Backup Set at the same time." The "Sat. Backup" script is always scheduled to do a Recycle Media Set of its Destination. I stopped execution of the Copy Media Set script and, after shutting down my MacBook Pro, ran my "Sat. Backup Incremental" script with "Media Set Red" as the Destination. "Sat. Backup Incremental" is actually a duplicate of "Sat. Backup"; it is never scheduled, so when run it always does No Media Action backups of my 6 Source drives. It ran fine, although it gave an error because it couldn't access the MBP. However, when I overlapped it with another execution of the Copy Media Set script, that execution gave a -2249 (could not find session) error. Early this morning, after returning from a movie (it was Jim Jarmusch's "Paterson"; fairly good, but an art-house movie), I did a Catalog Rebuild of "Media Set Red"—thinking that my stopping the "Sat. Backup" execution had messed up the Media Set—before leaving the MBP and Mac Pro on to run "Sun.-Fri. Backup" to "Media Set Red" as scheduled. I see now, from this thread, that my instinct to do a Catalog Rebuild of "Media Set Red" was correct. To rerun this test next Saturday morning, I'll have to first do a manual Recycle of "Media Set White" and then—in place of the scheduled "Sat. Backup" script—schedule a run of the "Sat. Backup Incremental" script with "Media Set White" as the Destination followed by an overlapping Copy Media Set script run with "Media Set White" as the Source. P.S.: This is post #14 in the thread.
  9. Hold on there, jweisbin and others, yesterday I thought of a variant on the simultaneous-backups solution that is much more promising. I realized that the Retrospect Windows 11 UG excerpts I quoted in post #7 in this thread mean that you can simultaneously use the same Disk Media Set as both a Destination for a backup and a Source for a Copy Media Set! That, of course, assumes that what applies to Retrospect Windows also applies to Retrospect Mac—which as we all know has the same underlying (but occasionally undocumented) capabilities because the Retrospect Inc. engineers built it that way. How could I possibly resist running a test of this? Early this morning, before my "Sun-Fri Backup" script was scheduled to make its last MacBook Pro backup to "Media Set Blue" before going to the bank safe deposit box, I booted my Mac Pro Desktop Edition "backup server" and reset the "Allow n Activity Threads" Preferences dropdown to 4. I then changed my Copy Media Set script that had as its Source "Media Set Blue" and its Destination a "Media Set Violet", with its single member on a spare FireWire 800 portable drive, so that its Activity Thread was set = 2 and it was scheduled to start 5 minutes after my "Sun-Fri Backup" script. (Because I run the Instant Scan process on my MBP, I know that a Backup script will begin actual copying in less than 5 minutes after the script starts running.) I then booted my MBP and went back to sleep. When I awoke an hour after "Sun-Fri Backup" was scheduled to run, I found that that script had completed execution in Activity Thread 1 in only two more minutes (22 minutes instead of 20 minutes) than it usually does but that the Copy Media Set script was still executing merrily in Activity Thread 2. The Copy Media Set script eventually completed execution in about 3:25 hours (remember that it is copying the Recycle backups of all my 6 drives, plus 6 more incremental backups of my MBP), which is around 10 minutes more than it took per post #9 in this thread. This evening I ran a Restore from "Media Set Violet" of the 7 cumulative backups of my MBP onto another spare FireWire 800 drive, and then successfully booted the MBP from that drive. Why, you may well ask, did using "Media Set Blue" simultaneously as both a Destination for the "Sun-Fri Backup" script and a Source for the Copy Media Set script not slow the backup appreciably? The reason is that, at least when the Source for a Backup is a hard disk drive, the gating factor is primarily speed of directory traversal on the Source drive—not LAN speed or Destination drive speed (because the Destination drive is not traversing directories). That may not hold true if the Source is an SSD, but I'm not about to spend money upgrading an Early 2011 MacBook Pro which will become officially "obsolete" when the ball falls at midnight tonight in Times Square. So, jweisbin, on your 16GB-RAM Mac mini Server you could follow each daily "client" backup-to-Disk-Media-Set script with an near-simultaneous Copy Media Set to Tape script. Because the Copy Media Set would be copying the entire cumulative Disk Media Set, it would probably finish hours after the last "client" had been incrementally backed up—but that wouldn't matter much because the Copy Media Set script would only be using drives on your "backup server" machine. Your "client" machines would already be freed up for daytime non-Retrospect activities, and you could grab your tape at the end of the day and take it off-site. Alternatively you could run a near-simultaneous Copy Backup script instead of the Copy Media Set script, but that would require bringing an up-to-date backup tape back on-site before each backup script run—and the Copy Backup script would fall behind when first started because of the need for it to space down to the previous end of the tape. I personally think a pair of alternating Copy Media Set scripts running near-simultaneously with the Backup script would be simpler. P.S.: This is post #13 in the thread.
  10. To be fair to Mayoff, the disk-to-disk-to-tape solution suggested by Lennart Thelander in post #2 of this thread would be the standard one under most circumstances. It would be the only solution if jweisbin wants to make a tape copy of an already-existing Disk Media Set. However the simultaneous-backups solution would be worth trying under the following circumstances: 1) jweisbin is starting a new backup procedure from scratch. 2) He/she wants to take the Tape Media Set off-site every time it is updated, and bring it back on-site for the next Backup run. In those circumstances simultaneous Backups might run faster than a Backup to a Disk Media Set followed by a Copy Backup to tape, but only if the Source(s) for the Backup runs is/are disks locally-attached to the "backup server" or clients whose disks are SSDs. My experience is that, doing a Recycle backup of all my drives—client and local—every Saturday, the copy-specific speed for my Early 2011 MacBook Pro client—which has a HDD instead of an SSD— is around 17GB/hour vs. 125GB/hour for a local disk—and the gating factor for the MBP is not the 36GB/hour effective copying speed (allowing for 20% overhead) of my 100Base-T LAN. Note that the copy-specific speed of my Copy Media Set tests was also about 125GB/hour. P.S.: Revised next-to-last sentence in second paragraph to use copy-specific speeds; verify should be excluded because—per the verify paragraph in this section of the old Wikipedia article—verify of client backups is done using MD5 checksums stored on the "backup server" during copy, but verify of local-drive backups is done using byte-by-byte comparison. P.P.S.: The second paragraph in this post does not mean that simultaneously-running scripts would not be useful in many cases. It just means that simultaneously-running Backups would not be useful in most cases. To be usefully run simultaneously, the scripts would have to only use locally-attached drives as Sources; thus the simultaneously-run Copy Media Set scripts I tested per the previous posts in the thread—or the equivalent kinds of Utility scripts—could be useful. P.P.P.S.: This is post #12 in the thread.
  11. I posted the contents of my last three posts in this thread as Support Case #52765. Today I received the following reply from support@Retrospect .com (with Mayoff's name on it): "Agent Response: The original forum thread makes reference to a very simple question and it looks like Lennard Thelander already answered the question. All server editions of Retrospect support the ability to back up using multiple executions at the same time. If a user wants to write to 2 different tape drives at the same time, you must use the Advanced Tape Support add-on license." To which I replied in the support case—which is now marked Closed: "In his/her OP in the thread, jweisbin said 'I thought I would set up a disk backup set and a tape backup set that would be perfectly in sync.' Based on my tests, there's no reason he can't do that with simultaneously-running scripts. I actually have an old HP DAT drive that worked in June 2015, when I tested Retrospect 6.1 on my Digital Audio G4. I would rerun the test Copy Media Set scripts simultaneously using it for one Destination Media Set, but unfortunately my Mac Pro doesn't have a SCSI card. So I guess we'll just have to wait for jweisbin to try it, which I'll urge him/her to do. If it works, the Retrospect Inc. documentation committee will have a little problem. Will they want the knowledge of this capability of Retrospect Mac 13 to be continue to be disseminated only as a "secret" on the Forums, or will they want to officially document it—either in the User's Guide or as a KnowledgeBase article? That's why this Support Case is a Documentation request." So try it, jweisbin, and please let us know if it works. I believe you have a Server Edition, in which the Retrospect Preferences "Allow n Activity Threads" dropdown should default to 4. So all you should need to do is create two backup scripts, one to a Disk Media and one to a Tape Media Set, and in the Summary pane for the scripts set the "Use" dropdown to different Activity Threads. You can schedule them for the same time, and they should run simultaneously. P.S.: This is post #11 in the thread.
  12. I was curious what would happen if I submitted the two scripts from yesterday's test before resetting the "Allow n Activity Threads" Preferences dropdown to 4 from 1, so I tried it late this morning. One thing I learned is that, in the Summary tabbed view of the Console detail for the script, the dropdown with "Use" to its left and "Activity Thread 1" showing is grayed-out so long as the "Allow n Activity Threads" Preferences dropdown is set to 1. So the Retrospect Inc. engineers did what the Marketing grinch forced them to do in a prevent-administrator-errors fashion. I re-submitted the Copy Media Set script that had had its Activity Thread set to 2, and it immediately started executing without the log showing an Activity Thread. I then re-submitted the other Copy Media Set script that had had its Activity Thread set to 1; it was made to wait. I then killed that waiting activity, set the "Allow n Activity Threads" Preferences dropdown to 4 from 1, and re-submitted the other Copy Media Set script with its Activity Thread set to 2.; it started executing immediately. The conclusion I draw from this test is that, if you want to execute multiple scheduleable (as opposed to Proactive) scripts simultaneously in Retrospect Desktop Edition, you should first set the "Allow n Activity Threads" Preferences dropdown to a number greater than 1 and then set the scripts' Activity Threads. If you previously set the scripts' Activity Threads but then subsequently re-started the Engine, I wouldn't rely on those scripts' Activity Threads being remembered without resetting them. P.S.: This is post #10 in the thread.
  13. More than a fighting chance; I did it yesterday. I shouldn't be able to do that in my lowly Desktop Edition of Retrospect Mac 12.5, but this thread shows that the only problem is that every restart of the Engine in Desktop Edition resets the "Allow n Activity Threads" dropdown in the General tab of Retrospect Preferences to 1. That was obviously forced on the Retrospect Inc. engineers by a grinch in Marketing per the quote from the WP article, but the engineers didn't do more than the minimum; once the Engine had been started by rebooting my Mac Pro I was able to reset the "Allow n Activity Threads" dropdown to 4. I then simultaneously ran two Copy Media Set scripts as described here and my immediately-following post in that thread. In both cases the Source was my "Media Set Blue", the "Media Set of the week"—"G-Drive White" containing "Media Set White" now being ensconced in my bank safe deposit box. The first script, with its Activity Thread set = 1, had as its destination a "Media Set Violet" with its single member on a spare FireWire 800 portable drive; it ran in 3:15 hours including Verify. The second script, with its Activity Thread set = 2, had as its destination a "Media Set Purple" with its first member in the available space on USB3 drive "G-Drive Red" and its second member in the available space on USB3 drive "G-Drive Blue"; it ran longer including Verify, because the Source for both scripts is also on USB3 drive "G-Drive Blue". This meant that, in the latter half of the second script's Copy (as opposed to Verify) stage, two sets of reads and one set of writes were being interspersed on "G-Drive Blue"—speaking of "serializing conflicting executions" per post #7 in this thread! P.S.: This is post #9 in the threaad.
  14. DavidHertzberg

    Saving backups to OneDrive for Business (Retrospect 7.7)

    I have now discovered that Mayoff was factually correct in post #8, but there's no way I could have known that. The apology and explanation are here, because it's a Wikipedia question.
  15. DavidHertzberg

    Major enhancement of Wikipedia article on Retrospect

    Mayoff made this post in the "Saving backups to OneDrive for Business (Retrospect 7.7)" thread: I pooh-poohed the statement in a subsequent post in that thread, but it now looks as if Mayoff was factually correct for Retrospect Windows 7.7. In researching a couple of threads for the "Retrospect 9 or higher for Macintosh" forum, I discovered over the last few days that Retrospect Windows 7.5 was apparently retrofitted with most of the non-UI features of Retrospect Mac 8. However there was no way I could have known that, because the Retrospect Windows 7.5 User's Guide did not have a "What's New" chapter and there seems not to have been any press release for Retrospect Windows 7.5. I was getting my information from this section of the (old) Wikipedia article, which DovidBenAvraham wrote from EMC's Retrospect Windows 7.0 press release. In writing for a WP article, any factual statement DBA—or anyone else—makes must be referenced to a source outside of Wikipedia. In fact DBA is skating on thin ice in the last sentence of the linked-to-above section, because he doesn't actually show the YouTube video as a reference (which he didn't do because it would have been considered another "primary source"). "Original research" is not allowed, which IMHO—and that of DBA—would include any systematic comparison of the Retrospect Windows 7.5 UG with the Mac 8 UG that I—and DBA—have done. So it looks like the linked-to-above section of the WP article will have to remain as is, accurate for Retrospect Windows 7.0 features but inaccurate for Retrospect Windows 7.5-7.7 features. The explanation for why the then-current release of Retrospect Windows retained the name "Retrospect Windows 7..." for 8 years despite major enhancements, and for why there was no press release for any of the major enhancements, probably lies in the last paragraph of this section of the WP article. I am making this post for belated accuracy, but also to be instructive to any budding WP editors who may be reading this thread.
  16. A friendly soul has just done a major enhancement of the Wikipedia article on Retrospect, bringing it up from around 2006 to the present. In doing so he made some statements in the section "Retrospect Macintosh 10 and Retrospect Windows 8" about the updating of Retrospect Windows 8 that may be open to question. He doesn't know for sure, because he uses Retrospect Mac. He would appreciate any replies with corrective information, and will try to incorporate these in the article. Of course, this being Wikipedia, there is nothing stopping anyone from editing the article himself/herself. Just remember to: A ) Sign up as a WP editor to prevent your IP from being made public. B ) Spend a few minutes learning to Edit Source; it's not that difficult. C ) You must provide references for factual statements; use the Templates dropdown in the editor and fill in the minimum necessary fields—you'll get automated warnings if you don't. D ) Keep a Neutral Point of View in what you write; WP requires it. Thanks!
  17. DovidBenAvraham has just catapulted my brain into the 21st Century, by reminding me that—as he has now clarified here in the "Powerful new engine" item—Retrospect Mac 8 and above "is capable of simultaneously performing multiple Backup, Restore, and Copy operations in separate threads". So, since jweisbin's previous posts show that he/she is running a Server Edition of Retrospect, he/she should be able to simultaneously run two backup scripts by designating that they run in separate threads. And, based on what I quoted from the Retrospect Windows 11 User's Guide in my immediately-preceding post in this thread, there is a fighting chance that the two scripts could simultaneously backup from the same Disk Source in Retrospect Mac 13. How, you may well ask, can jweisbin designate that the scripts run in separate threads? I had a vague recollection that this is possible, but there is absolutely no mention of this capability in the Retrospect Mac 13 User's Guide. So I thought of another undocumented capability for scripts that derek500 brought to my attention last year, which is the capability of rearranging the sequence in which Sources are processed by a script. That is done by dragging Sources in the Details pane of the Summary tabbed view of the Console detail for the script. So I looked in the Summary view, and—sure enough—at the bottom of the left-hand Overview pane is a dropdown with "Use" to its left and "Activity Thread 1" showing! However jweisbin must also ensure that his/her copy of Retrospect Mac 13 can run multiple simultaneous activity threads. Page 184 of the Retrospect Mac 13 UG says that this is done with the "Allow n Activity Threads" dropdown in the General tab of Retrospect Preferences, which are accessed from the Retrospect Menu. Page 184 says this preference defaults to 4, but in my Desktop Edition copy of Retrospect Mac 12.5 it defaults to 1—which DBA says in the "Retrospect 8.0 Desktop 3-User" item towards the bottom of the WP article section linked to in the first sentence of this post is because it "thus simulated the operating limitations of Retrospect Macintosh 6, and was priced accordingly." The same paragraph on page 184 of the UG says "In general, you should have one Gigabyte of free RAM for each activity thread you will run." If the two scripts would be backing up more than one Source, I would also advise jweisbin to vary the Sources' sequence in the two scripts (per the method mentioned in the paragraph above this) in order to avoid having both scripts compete for positioning of the same Disk read heads. So why is the capability of designating which activity thread a script runs in not documented? My favorite hypothesis is that various enhancements to the Summary tabbed view of the Scripts detail were done by a programmer who then left Retrospect Inc. without reminding his/her boss to include the enhancements in the next edition of the UG. A more macabre hypothesis is that the body of that programmer will eventually be found buried near where the body of Hans Reiser's wife Nina was found, which is actually not far from Walnut Creek CA (insert appropriate smiley here). However the most likely hypothesis does not involve the programmer leaving, but merely the famous Retrospect Inc. documentation committee getting tripped up by a terminology difference; an "activity thread" in Retrospect Mac turns out to be an "execution unit" (it took me some time to figure this out) in Retrospect Windows (we shouldn't expect those ignorant Windows administrators—insert appropriate smiley here— to know what a "thread" is), and setting it for a script is documented up the wazoo in the Retrospect Windows 11 UG (and, FWIW, in the Retrospect Windows 7.5 UG). Naturally I intend to belatedly file a Support Case requesting Retrospect Mac UG documentation of both Console Scripts category Detail Summary enhancements, which will constitute a Christmas/Hanukkah/Kwanzaa present for Mayoff. P.S.: In final paragraph, a Retrospect Mac "activity thread" is a Retrospect Windows "execution unit"; its specification for a script is thoroughly documented in the Retrospect Windows UG. P.P.S.: It turns out that the capability of rearranging the sequence in which Sources are processed by a script—mentioned in the second paragraph of this post—has also been thoroughly documented from the Retrospect Windows 7.5 UG onward (I just had to search for "drag"). So, insofar as the non-documentation in the Retrospect Mac UG of this capability as well as the one discussed in this thread is concerned, it seems as though a variant of my "most likely hypothesis" is the correct one. Because rearranging the sequences in which Sources are processed has no terminology difference between Windows and Mac, the variant is that the committee was not tripped up—but instead ran afoul of the Retrospect Mac 8-and-after UG non-expansion imperative hypothesized by DBA in the first sentence of the last paragraph of this section in the WP article. P.P.P.S.: This is post #8 in the thread.
  18. Thanks for the hardware and software particulars, jethro. I apologize for implying that you might have penny-pinching tendencies. I had a hunch that you didn't buy such a fairly fancy Mac mini Server just to run Retrospect, so I would be interested in knowing what "the typical server software" is—aside from OS X Server itself. That software may be what is causing your Retrospect slowness and crashing. In regard to that, I strongly urge you to contact Retrospect Inc. Tech Support. First phone one of the numbers I listed in post #6 in this thread. However don't stop there; IME A. (if he still works there) isn't the brightest bulb in the chandelier. Insist on speaking to Mayoff (dial again and if necessary phone again and dial x806 to get him), and file a Support Case by going to www.Retrospect.com and clicking the pane with the telephone icon in it at the upper right. As far as the "long startup times" is concerned, "syncing catalogs" (which presumably you see at the upper left of the Retrospect main window) is shown while the Retrospect Console is obtaining information from the particular Retrospect Engine(s) you are running. If you don't understand that, DovidBenAvraham had a brief but pithy explanation here in the "Retrospect Macintosh 8" section, starting with the last sentence in the "Powerful new engine" indented item-bulleted paragraph. Upon booting my Mac Pro "backup server", my Startup Items Retrospect Console app's "syncing catalogs" takes at most a second with my single RetrospectEngine process (I have Retrospect Mac 12.5 Desktop Edition running under OS X 10.10.5); if it is hanging for several minutes on your "backup server", you definitely have a problem that should be discussed with Retrospect Inc. Tech Support. As far as 5.6TB turning out to be over 6TB, you are almost certainly the victim of a decades-old traditional difference in measurement units between programmers and storage-drive-makers. I ran afoul of it over a year ago, and put an explanation into this post. For example Retrospect says my three USB3 G-Tech G-Drive Slims each have a capacity of 465GB, but HGST marketed them as 500GB drives. When I multiply 1024 by itself 3 successive times to get 1TB binary and then multiply that product by 5.6, I get nearly 6.2TB decimal using my handy-dandy Apple Calculator app. What we have here is what I have termed a "conceptual bug", and I don't think we have a prayer of getting Retrospect Inc.—a spin-off founded by programmers—to fix it. "Tradition, tradition ..." (about a year ago I took my now-ex-girlfriend, at her request, to a staging of "Fiddler On the Roof").
  19. I see from Wikipedia and his first quoted paragraph that jethro must have the Mid-2011 MacMini(5,3). Therefore he has as many CPU cores and twice as much RAM as my Mac Pro "backup server", which runs Copy Media Set much faster than jethro. Until now I had forgotten about jethro's original post in the 2 March thread, which reported he was having problems opening Retrospect and keeping it running. From jethro's first quoted paragraph he is still having the problems, so I am wondering along with Mayoff whether jethro has a hardware/software issue not related specifically to Copy Media Set. For instance, jethro has double the official maximum RAM for the MacMini(5,3). Unless jethro just happened to pick up a bargain on a previously-custom-built machine with quadruple the amount of RAM that he should normally need for Retrospect, I am wondering what other software he is running on his MacMini(5,3). Could it be that he is running Mac OS X Server? If so, he must be aware that—according to the ever-helpful DovidBenAvraham's WP article—he should be running a Retrospect Edition that supports OS X Server, not merely the Desktop Edition. Is that why jethro did seem and does seem reluctant to contact Retrospect Support about his other Retrospect problems (insert appropriate smiley here)—even last March when he ws still entitled to free telephone support for Retrospect Mac 13? Or, considering hardware only, is jethro just avoiding the possibility that his machine has a flakey internal hard disk drive—which would cost some money to replace (insert appropriate smiley here)? Before I saw jethro's quoted post #8 in this thread, I did have the additional idea that jethro's presumed leaving "Don’t add duplicate files to the Media Set" checked might be causing cumulative RAM-grabbing by the RetrospectEngine process while his Copy Media Set script is running. So this afternoon I reran the same test as in post #7 in this thread, except with that option checked. Intermittently looking at Real Memory (which was almost as unexciting as watching paint dry) in Activity Monitor, I noticed sudden temporary increases during Updating Catalog File phases. Each temporary Real Memory increase was greater than the increase after the previous Backup was copied, so it is possible that—since jethro's Copy Media Set run must be copying hundreds of Backups of the same Source—his temporary increases might be eventually exceeding the available Real Memory and causing what used to be known as "virtual memory thrashing". I was considering converting the posts I have made in this thread into a Support Case. However I no longer feel I can do that until jethro posts further particulars on what version of OS X and other software he is running on his Mac mini Server, his Edition of Retrospect Mac 13, and whether he has tested his internal hard disk drive. In any case—even if jethro fixes his Copy Media Set problem so that it runs as fast as mine does, his proposed weekly run cannot run within a single working day unless he starts a new Media Set every 8-10 months (depending on how long his workday is) . Do the arithmetic: 6TB/5 years-saved = (1200GB/year-saved) / 100GB/hour = 12 hours/year-saved. IMHO jethro should instead follow the procedure I suggested in this post, which uses Copy Backup on a weekly basis after an initial Copy Media Set. P.S.: The test described in my third paragraph ran in roughly the same amount of time as my previous tests, allowing for the additional No Media Action run of my "Sun.-Fri. Backup" script to "Media Set White" I had made before I ran the latest test. So leaving "Don’t add duplicate files to the Media Set" checked didn't seem to increase the running time. OTOH "Don’t add duplicate files to the Media Set" was checked during all the three daily runs of the "Sun.-Fri. Backup" script to "Media Set White", so I wouldn't expect there would be any duplicate files in the Source of the Copy Media Set. P.P.S: Revised last paragraph; jethro's last quoted paragraph says he's going to start start a new Media Set at the beginning of 2017.
  20. Prompted by what Mayoff said on the phone this morning, I started to think about overtaxed resources other than CPU speed—namely RAM. So early this afternoon I reran the same test as in post #5 in this thread, but with Activity Monitor also running. A year ago, when I first starting benchmarking the "backup server" on my Mac Pro for my thread on Ars Technica, I added a Real Memory column to Activity Monitor because I—being an old fuddy-duddy—do not completely believe in the reality of paging (it's virtual memory, after all). So, in between folding socks out of the dryer in the bedroom where my Mac Pro sits, I looked at the Real Memory used during the Copy Media Set run. I noticed that the biggest user of Real Memory was the RetrospectInstantScan process, not the RetrospectEngine process. And the Real Memory used by RetrospectInstantScan went up from about 720MB to about 820MB. It didn't keep climbing from there, however, as I thought it might. However 820MB is a fair amount of RAM by any standards. It didn't make any difference on my Mac Pro "backup server", because I upped its RAM from 3GB to 7GB when I inherited it in late spring 2015. But it might make a difference on Jethro's Mac mini Server, which only comes with 4GB RAM if it's the model I linked to from the P.S. of post #4. Therefore I recommend that jethro, and anyone else running a Copy Media Set script, first go to System Preferences->Retrospect and turn off Instant Scan for the duration of the script run. I also recommend that jethro make an investment in additional RAM; my additional 4GB cost me US$43 from—IIRC—Other World Computing. P.S.: Look at this thread. Problem sound familiar? Note that, if you click the link for item #11 in the Low End Mac page linked to in the P.S. of post #4 in this thread, that link goes to a Low End Mac page that says in the next-to-last paragraph that OWC has found you can upgrade that model to either 8GB or 16GB—and links to the OWC page.
  21. As soon as Retrospect Inc. opened this morning, I phoned Mayoff. He suggests that jethro phone Support, (888) 376-1078 or (925) 476-1030. He says that he's seen situations in which slowdowns like this turn out to be hardware or overtaxed-resources problems. Good luck jethro, and please post what Support tells you to this thread.
  22. This afternoon I've just rerun the test I ran last night, only doing the Copy Media Set to a single FireWire 800 drive destination—which I remembered this morning had available space—instead of onto destination members on two separate USB3 drives (one of which was also the source drive). The time is about the same; 2.7 hours for 261GB—6 GB more than yesterday because I ran my usual "Sun.-Fri. Backup" No Media Action script for my MacBook Pro onto "Media Set White" early this morning. The "Files remaining" (what is that?) reduce the total actually copied by about 20GB for each run. So why did my Copy Media Set runs get approximately 100GB/hour, whereas jethro's run got approximately 60GB/hour for his first drive and is getting approximately 20GB/hour for his second drive? The three variables that differ between my runs and jethro's runs are: 1) I had "Don’t add duplicate files to the Media Set" unchecked, while he probably has it checked. 2) My "backup server" machine has an intrinsically faster CPU than jethro's "backup server" machine, although both of our machines probably have a 4-core CPU. 3) My Copy Media Set runs copied less data than on jethro's first and second volumes, although the speed drop-off seems less than linear for his first volume and more than linear for his second volume—and of course the question remains why the amount of data copied should cause any speed drop-off at all. I'm going to make a phonecall to Mayoff, to see if he can provide a judgement as to the comparative importance of the three variables. P.S.: Added third variable that differs—the amount of data copied—as a final sentence in the second paragraph.
  23. Being a "guy at home"—with the setup described in this post—and responsible only to myself, I can run tests. So I ran one this evening, doing a Copy Media Set of my 235GB "Media Set White"—recreated from a Recycle backup of 6 drives today—into a brand-new "Media Set Violet". I only have spare space available on two USB3 drives, one of which is "G-Drive Blue"—last backed up to a week ago—and the other of which is "G-Drive White"—which already contains the "Media Set White" the test is copying from (I put "G-Drive Red" into my bank safe-deposit box this morning, and I wasn't prepared to dig out my non-existent diamond drill and nitroglycerine just to facilitate this test). I ran out of spare space on "G-Drive Blue", and—after spending 10 minutes in another room before I noticed the Add Member message—had to put the second member of "Media Set Violet" onto the spare space on "G-Drive White". So we have to allow for some inefficiency, given that the second half of the Copy Media Set run was copying onto the same drive it was copying from. Nevertheless, the Copy Media Set run took 2.5 hours—not counting the Add Member wait time—to copy 255GB. At that rate the Copy Media Set for jethro's first drive should have taken well under 10 hours, not 17 hours. Of course my "backup server" is an inherited 2010 Mac Pro (5,1) with the cheapest available 4-core processor, which is still more powerful than Jethro's "backup server"—although it's probably only using one core because I'm a destitute old fogey with Retrospect 12.5 Desktop Edition. Still, if I were jethro, I'd kill the Copy Backup Set run he has going and start again—with both the options "Match Source Media Set to destination Media Set" and "Don’t add duplicate files to the Media Set" unchecked. P.S.: Slightly revised fourth sentence in second paragraph; if jethro has the Mac mini Server (Mid 2011) that is item #11 on this page, he has just as many cores as my 2010 Mac Pro (5,1)—even though his processor is only 2.0GHz vs. my 2.6GHz.
  24. If what Lennart Thelander says in the first paragraph of the post immediately above is correct, then jethro should have followed the instruction fragment "with the option Match Source Media Set to destination Media Set unchecked ..." in step C2) of this post. But also, as it doesn't say in that linked-to post, he also should have had the option "Don’t add duplicate files to the Media Set" unchecked (IMHO the fact that this option defaults to checked is either a careless holdover from Backup scripts—where it forms "the other key component of Retrospect’s Smart Incremental backups"—or a stupid attempt to save space on the destination). And if those two things don't speed up Copy Backup Set, then the whole idea of creating an off-site copy of an existing on-site Media Set—whether for "seeding"/upload to the cloud or for physical off-site storage—using Copy Media Set is impractical! BTW, jethro, did you either manage to repair or to bypass the bad disk member of your on-site Media Set, which you discussed in this thread? P.S.: Inserted second sentence, about leaving the option "Don’t add duplicate files to the Media Set" unchecked, to the first paragraph. Based on my having setup a test which is now finished running, that option—unlike Match Source Media Set to destination Media Set—defaults to checked. P.P.S.: jethro, you should wait and read my post below this, which I have updated now that my test is finished. Based on the results, I think you should kill your current Copy Media Set script and restart it from scratch, with the setup per my first paragraph.
  25. DavidHertzberg

    Where's the search function for the forum?

    In the upper-right corner, on the same line where it says "IPS Community" on the left. I understand people's colors used to vary, making it difficult for some of them to see the search box. Heavens to Betsy, clicking the little gear icon to the right of the green-background (on my monitor for either Firefox or Safari on a Mac) magnifying-glass icon now gives Advanced Search! I don't think Advanced Search was there before. I don't have time to test out its capabilities much now, but it found all my posts (for brevity, I specified "as a topic list") going back to 2006. Oh IPS Community forums software, will thy wonders never cease (but I still don't get a toolbar with buttons above the box when I click Edit on Firefox)?
×