Jump to content


  • Content count

  • Joined

  • Last visited

  • Days Won


Everything posted by DavidHertzberg

  1. Hold on there, jweisbin and others, yesterday I thought of a variant on the simultaneous-backups solution that is much more promising. I realized that the Retrospect Windows 11 UG excerpts I quoted in post #7 in this thread mean that you can simultaneously use the same Disk Media Set as both a Destination for a backup and a Source for a Copy Media Set! That, of course, assumes that what applies to Retrospect Windows also applies to Retrospect Mac—which as we all know has the same underlying (but occasionally undocumented) capabilities because the Retrospect Inc. engineers built it that way. How could I possibly resist running a test of this? Early this morning, before my "Sun-Fri Backup" script was scheduled to make its last MacBook Pro backup to "Media Set Blue" before going to the bank safe deposit box, I booted my Mac Pro Desktop Edition "backup server" and reset the "Allow n Activity Threads" Preferences dropdown to 4. I then changed my Copy Media Set script that had as its Source "Media Set Blue" and its Destination a "Media Set Violet", with its single member on a spare FireWire 800 portable drive, so that its Activity Thread was set = 2 and it was scheduled to start 5 minutes after my "Sun-Fri Backup" script. (Because I run the Instant Scan process on my MBP, I know that a Backup script will begin actual copying in less than 5 minutes after the script starts running.) I then booted my MBP and went back to sleep. When I awoke an hour after "Sun-Fri Backup" was scheduled to run, I found that that script had completed execution in Activity Thread 1 in only two more minutes (22 minutes instead of 20 minutes) than it usually does but that the Copy Media Set script was still executing merrily in Activity Thread 2. The Copy Media Set script eventually completed execution in about 3:25 hours (remember that it is copying the Recycle backups of all my 6 drives, plus 6 more incremental backups of my MBP), which is around 10 minutes more than it took per post #9 in this thread. This evening I ran a Restore from "Media Set Violet" of the 7 cumulative backups of my MBP onto another spare FireWire 800 drive, and then successfully booted the MBP from that drive. Why, you may well ask, did using "Media Set Blue" simultaneously as both a Destination for the "Sun-Fri Backup" script and a Source for the Copy Media Set script not slow the backup appreciably? The reason is that, at least when the Source for a Backup is a hard disk drive, the gating factor is primarily speed of directory traversal on the Source drive—not LAN speed or Destination drive speed (because the Destination drive is not traversing directories). That may not hold true if the Source is an SSD, but I'm not about to spend money upgrading an Early 2011 MacBook Pro which will become officially "obsolete" when the ball falls at midnight tonight in Times Square. So, jweisbin, on your 16GB-RAM Mac mini Server you could follow each daily "client" backup-to-Disk-Media-Set script with an near-simultaneous Copy Media Set to Tape script. Because the Copy Media Set would be copying the entire cumulative Disk Media Set, it would probably finish hours after the last "client" had been incrementally backed up—but that wouldn't matter much because the Copy Media Set script would only be using drives on your "backup server" machine. Your "client" machines would already be freed up for daytime non-Retrospect activities, and you could grab your tape at the end of the day and take it off-site. Alternatively you could run a near-simultaneous Copy Backup script instead of the Copy Media Set script, but that would require bringing an up-to-date backup tape back on-site before each backup script run—and the Copy Backup script would fall behind when first started because of the need for it to space down to the previous end of the tape. I personally think a pair of alternating Copy Media Set scripts running near-simultaneously with the Backup script would be simpler. P.S.: This is post #13 in the thread.
  2. To be fair to Mayoff, the disk-to-disk-to-tape solution suggested by Lennart Thelander in post #2 of this thread would be the standard one under most circumstances. It would be the only solution if jweisbin wants to make a tape copy of an already-existing Disk Media Set. However the simultaneous-backups solution would be worth trying under the following circumstances: 1) jweisbin is starting a new backup procedure from scratch. 2) He/she wants to take the Tape Media Set off-site every time it is updated, and bring it back on-site for the next Backup run. In those circumstances simultaneous Backups might run faster than a Backup to a Disk Media Set followed by a Copy Backup to tape, but only if the Source(s) for the Backup runs is/are disks locally-attached to the "backup server" or clients whose disks are SSDs. My experience is that, doing a Recycle backup of all my drives—client and local—every Saturday, the copy-specific speed for my Early 2011 MacBook Pro client—which has a HDD instead of an SSD— is around 17GB/hour vs. 125GB/hour for a local disk—and the gating factor for the MBP is not the 36GB/hour effective copying speed (allowing for 20% overhead) of my 100Base-T LAN. Note that the copy-specific speed of my Copy Media Set tests was also about 125GB/hour. P.S.: Revised next-to-last sentence in second paragraph to use copy-specific speeds; verify should be excluded because—per the verify paragraph in this section of the old Wikipedia article—verify of client backups is done using MD5 checksums stored on the "backup server" during copy, but verify of local-drive backups is done using byte-by-byte comparison. P.P.S.: The second paragraph in this post does not mean that simultaneously-running scripts would not be useful in many cases. It just means that simultaneously-running Backups would not be useful in most cases. To be usefully run simultaneously, the scripts would have to only use locally-attached drives as Sources; thus the simultaneously-run Copy Media Set scripts I tested per the previous posts in the thread—or the equivalent kinds of Utility scripts—could be useful. P.P.P.S.: This is post #12 in the thread.
  3. I posted the contents of my last three posts in this thread as Support Case #52765. Today I received the following reply from support@Retrospect .com (with Mayoff's name on it): "Agent Response: The original forum thread makes reference to a very simple question and it looks like Lennard Thelander already answered the question. All server editions of Retrospect support the ability to back up using multiple executions at the same time. If a user wants to write to 2 different tape drives at the same time, you must use the Advanced Tape Support add-on license." To which I replied in the support case—which is now marked Closed: "In his/her OP in the thread, jweisbin said 'I thought I would set up a disk backup set and a tape backup set that would be perfectly in sync.' Based on my tests, there's no reason he can't do that with simultaneously-running scripts. I actually have an old HP DAT drive that worked in June 2015, when I tested Retrospect 6.1 on my Digital Audio G4. I would rerun the test Copy Media Set scripts simultaneously using it for one Destination Media Set, but unfortunately my Mac Pro doesn't have a SCSI card. So I guess we'll just have to wait for jweisbin to try it, which I'll urge him/her to do. If it works, the Retrospect Inc. documentation committee will have a little problem. Will they want the knowledge of this capability of Retrospect Mac 13 to be continue to be disseminated only as a "secret" on the Forums, or will they want to officially document it—either in the User's Guide or as a KnowledgeBase article? That's why this Support Case is a Documentation request." So try it, jweisbin, and please let us know if it works. I believe you have a Server Edition, in which the Retrospect Preferences "Allow n Activity Threads" dropdown should default to 4. So all you should need to do is create two backup scripts, one to a Disk Media and one to a Tape Media Set, and in the Summary pane for the scripts set the "Use" dropdown to different Activity Threads. You can schedule them for the same time, and they should run simultaneously. P.S.: This is post #11 in the thread.
  4. I was curious what would happen if I submitted the two scripts from yesterday's test before resetting the "Allow n Activity Threads" Preferences dropdown to 4 from 1, so I tried it late this morning. One thing I learned is that, in the Summary tabbed view of the Console detail for the script, the dropdown with "Use" to its left and "Activity Thread 1" showing is grayed-out so long as the "Allow n Activity Threads" Preferences dropdown is set to 1. So the Retrospect Inc. engineers did what the Marketing grinch forced them to do in a prevent-administrator-errors fashion. I re-submitted the Copy Media Set script that had had its Activity Thread set to 2, and it immediately started executing without the log showing an Activity Thread. I then re-submitted the other Copy Media Set script that had had its Activity Thread set to 1; it was made to wait. I then killed that waiting activity, set the "Allow n Activity Threads" Preferences dropdown to 4 from 1, and re-submitted the other Copy Media Set script with its Activity Thread set to 2.; it started executing immediately. The conclusion I draw from this test is that, if you want to execute multiple scheduleable (as opposed to Proactive) scripts simultaneously in Retrospect Desktop Edition, you should first set the "Allow n Activity Threads" Preferences dropdown to a number greater than 1 and then set the scripts' Activity Threads. If you previously set the scripts' Activity Threads but then subsequently re-started the Engine, I wouldn't rely on those scripts' Activity Threads being remembered without resetting them. P.S.: This is post #10 in the thread.
  5. More than a fighting chance; I did it yesterday. I shouldn't be able to do that in my lowly Desktop Edition of Retrospect Mac 12.5, but this thread shows that the only problem is that every restart of the Engine in Desktop Edition resets the "Allow n Activity Threads" dropdown in the General tab of Retrospect Preferences to 1. That was obviously forced on the Retrospect Inc. engineers by a grinch in Marketing per the quote from the WP article, but the engineers didn't do more than the minimum; once the Engine had been started by rebooting my Mac Pro I was able to reset the "Allow n Activity Threads" dropdown to 4. I then simultaneously ran two Copy Media Set scripts as described here and my immediately-following post in that thread. In both cases the Source was my "Media Set Blue", the "Media Set of the week"—"G-Drive White" containing "Media Set White" now being ensconced in my bank safe deposit box. The first script, with its Activity Thread set = 1, had as its destination a "Media Set Violet" with its single member on a spare FireWire 800 portable drive; it ran in 3:15 hours including Verify. The second script, with its Activity Thread set = 2, had as its destination a "Media Set Purple" with its first member in the available space on USB3 drive "G-Drive Red" and its second member in the available space on USB3 drive "G-Drive Blue"; it ran longer including Verify, because the Source for both scripts is also on USB3 drive "G-Drive Blue". This meant that, in the latter half of the second script's Copy (as opposed to Verify) stage, two sets of reads and one set of writes were being interspersed on "G-Drive Blue"—speaking of "serializing conflicting executions" per post #7 in this thread! P.S.: This is post #9 in the threaad.
  6. DavidHertzberg

    Saving backups to OneDrive for Business (Retrospect 7.7)

    I have now discovered that Mayoff was factually correct in post #8, but there's no way I could have known that. The apology and explanation are here, because it's a Wikipedia question.
  7. A friendly soul has just done a major enhancement of the Wikipedia article on Retrospect, bringing it up from around 2006 to the present. In doing so he made some statements in the section "Retrospect Macintosh 10 and Retrospect Windows 8" about the updating of Retrospect Windows 8 that may be open to question. He doesn't know for sure, because he uses Retrospect Mac. He would appreciate any replies with corrective information, and will try to incorporate these in the article. Of course, this being Wikipedia, there is nothing stopping anyone from editing the article himself/herself. Just remember to: A ) Sign up as a WP editor to prevent your IP from being made public. B ) Spend a few minutes learning to Edit Source; it's not that difficult. C ) You must provide references for factual statements; use the Templates dropdown in the editor and fill in the minimum necessary fields—you'll get automated warnings if you don't. D ) Keep a Neutral Point of View in what you write; WP requires it. Thanks!
  8. DavidHertzberg

    Major enhancement of Wikipedia article on Retrospect

    Mayoff made this post in the "Saving backups to OneDrive for Business (Retrospect 7.7)" thread: I pooh-poohed the statement in a subsequent post in that thread, but it now looks as if Mayoff was factually correct for Retrospect Windows 7.7. In researching a couple of threads for the "Retrospect 9 or higher for Macintosh" forum, I discovered over the last few days that Retrospect Windows 7.5 was apparently retrofitted with most of the non-UI features of Retrospect Mac 8. However there was no way I could have known that, because the Retrospect Windows 7.5 User's Guide did not have a "What's New" chapter and there seems not to have been any press release for Retrospect Windows 7.5. I was getting my information from this section of the (old) Wikipedia article, which DovidBenAvraham wrote from EMC's Retrospect Windows 7.0 press release. In writing for a WP article, any factual statement DBA—or anyone else—makes must be referenced to a source outside of Wikipedia. In fact DBA is skating on thin ice in the last sentence of the linked-to-above section, because he doesn't actually show the YouTube video as a reference (which he didn't do because it would have been considered another "primary source"). "Original research" is not allowed, which IMHO—and that of DBA—would include any systematic comparison of the Retrospect Windows 7.5 UG with the Mac 8 UG that I—and DBA—have done. So it looks like the linked-to-above section of the WP article will have to remain as is, accurate for Retrospect Windows 7.0 features but inaccurate for Retrospect Windows 7.5-7.7 features. The explanation for why the then-current release of Retrospect Windows retained the name "Retrospect Windows 7..." for 8 years despite major enhancements, and for why there was no press release for any of the major enhancements, probably lies in the last paragraph of this section of the WP article. I am making this post for belated accuracy, but also to be instructive to any budding WP editors who may be reading this thread.
  9. DovidBenAvraham has just catapulted my brain into the 21st Century, by reminding me that—as he has now clarified here in the "Powerful new engine" item—Retrospect Mac 8 and above "is capable of simultaneously performing multiple Backup, Restore, and Copy operations in separate threads". So, since jweisbin's previous posts show that he/she is running a Server Edition of Retrospect, he/she should be able to simultaneously run two backup scripts by designating that they run in separate threads. And, based on what I quoted from the Retrospect Windows 11 User's Guide in my immediately-preceding post in this thread, there is a fighting chance that the two scripts could simultaneously backup from the same Disk Source in Retrospect Mac 13. How, you may well ask, can jweisbin designate that the scripts run in separate threads? I had a vague recollection that this is possible, but there is absolutely no mention of this capability in the Retrospect Mac 13 User's Guide. So I thought of another undocumented capability for scripts that derek500 brought to my attention last year, which is the capability of rearranging the sequence in which Sources are processed by a script. That is done by dragging Sources in the Details pane of the Summary tabbed view of the Console detail for the script. So I looked in the Summary view, and—sure enough—at the bottom of the left-hand Overview pane is a dropdown with "Use" to its left and "Activity Thread 1" showing! However jweisbin must also ensure that his/her copy of Retrospect Mac 13 can run multiple simultaneous activity threads. Page 184 of the Retrospect Mac 13 UG says that this is done with the "Allow n Activity Threads" dropdown in the General tab of Retrospect Preferences, which are accessed from the Retrospect Menu. Page 184 says this preference defaults to 4, but in my Desktop Edition copy of Retrospect Mac 12.5 it defaults to 1—which DBA says in the "Retrospect 8.0 Desktop 3-User" item towards the bottom of the WP article section linked to in the first sentence of this post is because it "thus simulated the operating limitations of Retrospect Macintosh 6, and was priced accordingly." The same paragraph on page 184 of the UG says "In general, you should have one Gigabyte of free RAM for each activity thread you will run." If the two scripts would be backing up more than one Source, I would also advise jweisbin to vary the Sources' sequence in the two scripts (per the method mentioned in the paragraph above this) in order to avoid having both scripts compete for positioning of the same Disk read heads. So why is the capability of designating which activity thread a script runs in not documented? My favorite hypothesis is that various enhancements to the Summary tabbed view of the Scripts detail were done by a programmer who then left Retrospect Inc. without reminding his/her boss to include the enhancements in the next edition of the UG. A more macabre hypothesis is that the body of that programmer will eventually be found buried near where the body of Hans Reiser's wife Nina was found, which is actually not far from Walnut Creek CA (insert appropriate smiley here). However the most likely hypothesis does not involve the programmer leaving, but merely the famous Retrospect Inc. documentation committee getting tripped up by a terminology difference; an "activity thread" in Retrospect Mac turns out to be an "execution unit" (it took me some time to figure this out) in Retrospect Windows (we shouldn't expect those ignorant Windows administrators—🤣— to know what a "thread" is), and setting it for a script is documented up the wazoo in the Retrospect Windows 11 UG (and, FWIW, in the Retrospect Windows 7.5 UG). Naturally I intend to belatedly file a Support Case requesting Retrospect Mac UG documentation of both Console Scripts category Detail Summary enhancements, which will constitute a Christmas/Hanukkah/Kwanzaa present for Mayoff. P.S.: In final paragraph, a Retrospect Mac "activity thread" is a Retrospect Windows "execution unit"; its specification for a script is thoroughly documented in the Retrospect Windows UG. P.P.S.: It turns out that the capability of rearranging the sequence in which Sources are processed by a script—mentioned in the second paragraph of this post—has also been thoroughly documented from the Retrospect Windows 7.5 UG onward (I just had to search for "drag"). So, insofar as the non-documentation in the Retrospect Mac UG of this capability as well as the one discussed in this thread is concerned, it seems as though a variant of my "most likely hypothesis" is the correct one. Because rearranging the sequences in which Sources are processed has no terminology difference between Windows and Mac, the variant is that the committee was not tripped up—but instead ran afoul of the Retrospect Mac 8-and-after UG non-expansion imperative hypothesized by DBA in the first sentence of the last paragraph of this section in the WP article. P.P.P.S.: This is post #8 in the thread.
  10. Thanks for the hardware and software particulars, jethro. I apologize for implying that you might have penny-pinching tendencies. I had a hunch that you didn't buy such a fairly fancy Mac mini Server just to run Retrospect, so I would be interested in knowing what "the typical server software" is—aside from OS X Server itself. That software may be what is causing your Retrospect slowness and crashing. In regard to that, I strongly urge you to contact Retrospect Inc. Tech Support. First phone one of the numbers I listed in post #6 in this thread. However don't stop there; IME A. (if he still works there) isn't the brightest bulb in the chandelier. Insist on speaking to Mayoff (dial again and if necessary phone again and dial x806 to get him), and file a Support Case by going to www.Retrospect.com and clicking the pane with the telephone icon in it at the upper right. As far as the "long startup times" is concerned, "syncing catalogs" (which presumably you see at the upper left of the Retrospect main window) is shown while the Retrospect Console is obtaining information from the particular Retrospect Engine(s) you are running. If you don't understand that, DovidBenAvraham had a brief but pithy explanation here in the "Retrospect Macintosh 8" section, starting with the last sentence in the "Powerful new engine" indented item-bulleted paragraph. Upon booting my Mac Pro "backup server", my Startup Items Retrospect Console app's "syncing catalogs" takes at most a second with my single RetrospectEngine process (I have Retrospect Mac 12.5 Desktop Edition running under OS X 10.10.5); if it is hanging for several minutes on your "backup server", you definitely have a problem that should be discussed with Retrospect Inc. Tech Support. As far as 5.6TB turning out to be over 6TB, you are almost certainly the victim of a decades-old traditional difference in measurement units between programmers and storage-drive-makers. I ran afoul of it over a year ago, and put an explanation into this post. For example Retrospect says my three USB3 G-Tech G-Drive Slims each have a capacity of 465GB, but HGST marketed them as 500GB drives. When I multiply 1024 by itself 3 successive times to get 1TB binary and then multiply that product by 5.6, I get nearly 6.2TB decimal using my handy-dandy Apple Calculator app. What we have here is what I have termed a "conceptual bug", and I don't think we have a prayer of getting Retrospect Inc.—a spin-off founded by programmers—to fix it. "Tradition, tradition ..." (about a year ago I took my now-ex-girlfriend, at her request, to a staging of "Fiddler On the Roof").
  11. I see from Wikipedia and his first quoted paragraph that jethro must have the Mid-2011 MacMini(5,3). Therefore he has as many CPU cores and twice as much RAM as my Mac Pro "backup server", which runs Copy Media Set much faster than jethro. Until now I had forgotten about jethro's original post in the 2 March thread, which reported he was having problems opening Retrospect and keeping it running. From jethro's first quoted paragraph he is still having the problems, so I am wondering along with Mayoff whether jethro has a hardware/software issue not related specifically to Copy Media Set. For instance, jethro has double the official maximum RAM for the MacMini(5,3). Unless jethro just happened to pick up a bargain on a previously-custom-built machine with quadruple the amount of RAM that he should normally need for Retrospect, I am wondering what other software he is running on his MacMini(5,3). Could it be that he is running Mac OS X Server? If so, he must be aware that—according to the ever-helpful DovidBenAvraham's WP article—he should be running a Retrospect Edition that supports OS X Server, not merely the Desktop Edition. Is that why jethro did seem and does seem reluctant to contact Retrospect Support about his other Retrospect problems (insert appropriate smiley here)—even last March when he ws still entitled to free telephone support for Retrospect Mac 13? Or, considering hardware only, is jethro just avoiding the possibility that his machine has a flakey internal hard disk drive—which would cost some money to replace (insert appropriate smiley here)? Before I saw jethro's quoted post #8 in this thread, I did have the additional idea that jethro's presumed leaving "Don’t add duplicate files to the Media Set" checked might be causing cumulative RAM-grabbing by the RetrospectEngine process while his Copy Media Set script is running. So this afternoon I reran the same test as in post #7 in this thread, except with that option checked. Intermittently looking at Real Memory (which was almost as unexciting as watching paint dry) in Activity Monitor, I noticed sudden temporary increases during Updating Catalog File phases. Each temporary Real Memory increase was greater than the increase after the previous Backup was copied, so it is possible that—since jethro's Copy Media Set run must be copying hundreds of Backups of the same Source—his temporary increases might be eventually exceeding the available Real Memory and causing what used to be known as "virtual memory thrashing". I was considering converting the posts I have made in this thread into a Support Case. However I no longer feel I can do that until jethro posts further particulars on what version of OS X and other software he is running on his Mac mini Server, his Edition of Retrospect Mac 13, and whether he has tested his internal hard disk drive. In any case—even if jethro fixes his Copy Media Set problem so that it runs as fast as mine does, his proposed weekly run cannot run within a single working day unless he starts a new Media Set every 8-10 months (depending on how long his workday is) . Do the arithmetic: 6TB/5 years-saved = (1200GB/year-saved) / 100GB/hour = 12 hours/year-saved. IMHO jethro should instead follow the procedure I suggested in this post, which uses Copy Backup on a weekly basis after an initial Copy Media Set. P.S.: The test described in my third paragraph ran in roughly the same amount of time as my previous tests, allowing for the additional No Media Action run of my "Sun.-Fri. Backup" script to "Media Set White" I had made before I ran the latest test. So leaving "Don’t add duplicate files to the Media Set" checked didn't seem to increase the running time. OTOH "Don’t add duplicate files to the Media Set" was checked during all the three daily runs of the "Sun.-Fri. Backup" script to "Media Set White", so I wouldn't expect there would be any duplicate files in the Source of the Copy Media Set. P.P.S: Revised last paragraph; jethro's last quoted paragraph says he's going to start start a new Media Set at the beginning of 2017.
  12. Prompted by what Mayoff said on the phone this morning, I started to think about overtaxed resources other than CPU speed—namely RAM. So early this afternoon I reran the same test as in post #5 in this thread, but with Activity Monitor also running. A year ago, when I first starting benchmarking the "backup server" on my Mac Pro for my thread on Ars Technica, I added a Real Memory column to Activity Monitor because I—being an old fuddy-duddy—do not completely believe in the reality of paging (it's virtual memory, after all). So, in between folding socks out of the dryer in the bedroom where my Mac Pro sits, I looked at the Real Memory used during the Copy Media Set run. I noticed that the biggest user of Real Memory was the RetrospectInstantScan process, not the RetrospectEngine process. And the Real Memory used by RetrospectInstantScan went up from about 720MB to about 820MB. It didn't keep climbing from there, however, as I thought it might. However 820MB is a fair amount of RAM by any standards. It didn't make any difference on my Mac Pro "backup server", because I upped its RAM from 3GB to 7GB when I inherited it in late spring 2015. But it might make a difference on Jethro's Mac mini Server, which only comes with 4GB RAM if it's the model I linked to from the P.S. of post #4. Therefore I recommend that jethro, and anyone else running a Copy Media Set script, first go to System Preferences->Retrospect and turn off Instant Scan for the duration of the script run. I also recommend that jethro make an investment in additional RAM; my additional 4GB cost me US$43 from—IIRC—Other World Computing. P.S.: Look at this thread. Problem sound familiar? Note that, if you click the link for item #11 in the Low End Mac page linked to in the P.S. of post #4 in this thread, that link goes to a Low End Mac page that says in the next-to-last paragraph that OWC has found you can upgrade that model to either 8GB or 16GB—and links to the OWC page.
  13. As soon as Retrospect Inc. opened this morning, I phoned Mayoff. He suggests that jethro phone Support, (888) 376-1078 or (925) 476-1030. He says that he's seen situations in which slowdowns like this turn out to be hardware or overtaxed-resources problems. Good luck jethro, and please post what Support tells you to this thread.
  14. This afternoon I've just rerun the test I ran last night, only doing the Copy Media Set to a single FireWire 800 drive destination—which I remembered this morning had available space—instead of onto destination members on two separate USB3 drives (one of which was also the source drive). The time is about the same; 2.7 hours for 261GB—6 GB more than yesterday because I ran my usual "Sun.-Fri. Backup" No Media Action script for my MacBook Pro onto "Media Set White" early this morning. The "Files remaining" (what is that?) reduce the total actually copied by about 20GB for each run. So why did my Copy Media Set runs get approximately 100GB/hour, whereas jethro's run got approximately 60GB/hour for his first drive and is getting approximately 20GB/hour for his second drive? The three variables that differ between my runs and jethro's runs are: 1) I had "Don’t add duplicate files to the Media Set" unchecked, while he probably has it checked. 2) My "backup server" machine has an intrinsically faster CPU than jethro's "backup server" machine, although both of our machines probably have a 4-core CPU. 3) My Copy Media Set runs copied less data than on jethro's first and second volumes, although the speed drop-off seems less than linear for his first volume and more than linear for his second volume—and of course the question remains why the amount of data copied should cause any speed drop-off at all. I'm going to make a phonecall to Mayoff, to see if he can provide a judgement as to the comparative importance of the three variables. P.S.: Added third variable that differs—the amount of data copied—as a final sentence in the second paragraph.
  15. Being a "guy at home"—with the setup described in this post—and responsible only to myself, I can run tests. So I ran one this evening, doing a Copy Media Set of my 235GB "Media Set White"—recreated from a Recycle backup of 6 drives today—into a brand-new "Media Set Violet". I only have spare space available on two USB3 drives, one of which is "G-Drive Blue"—last backed up to a week ago—and the other of which is "G-Drive White"—which already contains the "Media Set White" the test is copying from (I put "G-Drive Red" into my bank safe-deposit box this morning, and I wasn't prepared to dig out my non-existent diamond drill and nitroglycerine just to facilitate this test). I ran out of spare space on "G-Drive Blue", and—after spending 10 minutes in another room before I noticed the Add Member message—had to put the second member of "Media Set Violet" onto the spare space on "G-Drive White". So we have to allow for some inefficiency, given that the second half of the Copy Media Set run was copying onto the same drive it was copying from. Nevertheless, the Copy Media Set run took 2.5 hours—not counting the Add Member wait time—to copy 255GB. At that rate the Copy Media Set for jethro's first drive should have taken well under 10 hours, not 17 hours. Of course my "backup server" is an inherited 2010 Mac Pro (5,1) with the cheapest available 4-core processor, which is still more powerful than Jethro's "backup server"—although it's probably only using one core because I'm a destitute old fogey with Retrospect 12.5 Desktop Edition. Still, if I were jethro, I'd kill the Copy Backup Set run he has going and start again—with both the options "Match Source Media Set to destination Media Set" and "Don’t add duplicate files to the Media Set" unchecked. P.S.: Slightly revised fourth sentence in second paragraph; if jethro has the Mac mini Server (Mid 2011) that is item #11 on this page, he has just as many cores as my 2010 Mac Pro (5,1)—even though his processor is only 2.0GHz vs. my 2.6GHz.
  16. If what Lennart Thelander says in the first paragraph of the post immediately above is correct, then jethro should have followed the instruction fragment "with the option Match Source Media Set to destination Media Set unchecked ..." in step C2) of this post. But also, as it doesn't say in that linked-to post, he also should have had the option "Don’t add duplicate files to the Media Set" unchecked (IMHO the fact that this option defaults to checked is either a careless holdover from Backup scripts—where it forms "the other key component of Retrospect’s Smart Incremental backups"—or a stupid attempt to save space on the destination). And if those two things don't speed up Copy Backup Set, then the whole idea of creating an off-site copy of an existing on-site Media Set—whether for "seeding"/upload to the cloud or for physical off-site storage—using Copy Media Set is impractical! BTW, jethro, did you either manage to repair or to bypass the bad disk member of your on-site Media Set, which you discussed in this thread? P.S.: Inserted second sentence, about leaving the option "Don’t add duplicate files to the Media Set" unchecked, to the first paragraph. Based on my having setup a test which is now finished running, that option—unlike Match Source Media Set to destination Media Set—defaults to checked. P.P.S.: jethro, you should wait and read my post below this, which I have updated now that my test is finished. Based on the results, I think you should kill your current Copy Media Set script and restart it from scratch, with the setup per my first paragraph.
  17. DavidHertzberg

    Where's the search function for the forum?

    In the upper-right corner, on the same line where it says "IPS Community" on the left. I understand people's colors used to vary, making it difficult for some of them to see the search box. Heavens to Betsy, clicking the little gear icon to the right of the green-background (on my monitor for either Firefox or Safari on a Mac) magnifying-glass icon now gives Advanced Search! I don't think Advanced Search was there before. I don't have time to test out its capabilities much now, but it found all my posts (for brevity, I specified "as a topic list") going back to 2006. Oh IPS Community forums software, will thy wonders never cease (but I still don't get a toolbar with buttons above the box when I click Edit on Firefox)?
  18. (Converted this into a separate post, for more visibility) It turns out that the first sentence of the final quoted paragraph is technically correct, at least for Windows server-class editions of Retrospect—but probably also for Macintosh server-class editions of Retrospect. Page 288 of the Retrospect Windows 11 User's Guide says "Retrospect allows you to: ... handle resource conflicts (including serializing conflicting executions) ...." Page 289 says "Retrospect server-class editions are unique in that they support a single write operation and multiple read operations simultaneously using the same disk Backup Set .... Read operations include: ... • Transferring from the Backup Set." It also says "The only limitation, other than execution units, is that none of the concurrent operations can require ... the same non-disk [my emphasis] Backup Set", but that means you can't do two concurrent operations to the same tape Backup Set (think about the read-write head shuttling back and forth, on a device that has no positioning index). However page 290 says "Retrospect Desktop supports only one execution unit and therefore cannot take advantage of disk Backup Sets’ single write/multiple read capabilities." These sentences were in the Retrospect Windows 7.5 User's Guide—meaning before the great expansion of that document for Retrospect Windows 8, but not in any available edition of the Retrospect Mac User's Guide. I have only a lowly Desktop Edition of Retrospect Mac 12.5, but I think my second and third quoted paragraphs would still apply. P.S.: This is post #7 in the thread.
  19. But jweisbin isn't necessarily too experienced. In any case, I think Mayoff—who always understandably tries to make his video tutorials as short as possible (sometimes too short, IME)—was tacitly taking advantage of two Retrospect scheduling tricks. The first trick is that, if you have two scripts scheduled for the same time on the same "backup server", IIRC the script with the name that alphanumerically sorts earlier starts running first. Thus Mayoff's script named My Backup Schedule will start running before his script named My Offsite Backup Schedule. The second trick is that, if the My Offsite Backup Schedule script uses as a Source a Media Set that is the Destination for the My Backup Schedule script and the My Backup Schedule script starts running even a second earlier, the My Offsite Backup Schedule script will not run until the My Backup Schedule script is finished running. I've personally used a variant of this trick, although I don't remember whether I did so intentionally or accidentally. It would be nice if Mayoff had explained these two tricks at the end of the video. But Mayoff does not seem to believe it's wise to include lengthy explanations in videos, probably because he's afraid of losing the viewer. Given Mayoff's experience, and the long-standing problem with getting Retrospect administrators to read the User's Guides, I can't say that I totally blame him. Come to think of it, I may be wrong; the Retrospect "backup server" software may be designed to do a source-destination conflict analysis of all scripts that are scheduled to run at the same time. But the result will be the same; the My Offsite Backup Schedule script will not begin running until the My Backup Schedule script is finished running. P.S.: (Converted this into a separate post, for more visibility) P.P.S.: The "first trick", covered in my second paragraph, is mentioned on page 253 of the Retrospect Windows 11 User's Guide. It says "When you open several run documents at once, the scripts associated with them will run in alphabetical order by script name, regardless of the run document file names". Don't worry about what a "run document" is; it's a Retrospect Windows thing. P.P.P.S.: This is post #6 in the thread.
  20. jweisbin, there is a YouTube video on "staged backup" for Macintosh that is linked-to from the Retrospect:Tutorials web page. Lennart Thelander, how could you have failed to mention this? 🤣 Mayoff would be so hurt! P.S.: This is post #4 in the thread
  21. DavidHertzberg

    Compare Snapshot so as to Restore purely incremental files?

    Please remember that, per this thread, FrugalOptimist is using Retrospect Windows 7.7.
  22. DavidHertzberg

    Saving backups to OneDrive for Business (Retrospect 7.7)

    I don't think that the nuances of by what percentage the code changed from Retrospect Windows 7.7 to Retrospect Windows 8 "address the original question". My first point in the thread, which does "address the original question", was that Retrospect Windows 7.7 does not support any kind of cloud service but that Retrospect Windows 8 does support WebDAV-compatible cloud services. My second point, which also does "address the original question", was that the Microsoft OneDrive service is not compatible with WebDAV. My third point, which again does "address the original question", was that—even if the Microsoft OneDrive service were compatible with WebDAV—it might not be suitable for Retrospect backups, which is a plausible reason why—despite FrugalOptimist's complaints—Retrospect Inc. does not support it. Only the second and third paragraphs of my post #10 in this thread address the "nuances of a Wikipedia article that really doesn't address" the question that Mayoff himself raised in his post #8 in this thread. I would have been happy, on DovidBenAvraham's behalf, to direct those paragraphs directly to Mayoff without bothering my "fellow forum users". However I don't know of a way to do that. Mayoff's former direct e-mail address no longer works, and he says "I do not accept private messages in the forum." So I did impose on my "fellow forum users" for two paragraphs, but got no help from Mayoff—only a snarky "apology" that probably did not reach FrugalOptimist (who had already posted "OK, bored now. Retrospect 7.7 lives!"). Should I instead have turned those two paragraphs into a Support Case? Is there a "past history of Retrospect and its publicity consequences" Support Case category into which they would have fit? Mayoff has said my "fellow forum users are really not interested in the nuances of a wikipedia article that really doesn't address the original question". But the article does now "address the original question" to the extent that Mayoff's post #8 comment addresses it, because DovidBenAvraham yesterday incorporated the "What's New" revelations of the Retrospect Windows 7 press release (which seems to have recently reappeared on the Web because Dell has insisted on the re-posting of the press releases from EMC's old subsidiaries). And my "fellow forum users" in fact do seem to be interested. The Pageviews Analysis (click "View history" at the top of the article, then click "Page view statistics" at the right just above the second gray horizontal bar from the top) for the article jumped from 32 to 41 yesterday—which brings the daily page views from approximately double what they averaged before DBA began expanding the article on October 5th to nearly triple what they averaged. Either people who are not my "fellow forum users" are through their psychic abilities feeling the urge to take a look whenever DBA does further edits—which should make Mayoff as a Retrospect Inc. employee very happy, or my "fellow forum users" are keeping a watch for changes in the article—which means they really are interested. Maybe Mayoff should, instead of apologizing to FrugalOptimist, arrange to give him/her a cut-rate price on an upgrade to Retrospect Windows 8? Or the oldest version of Retrospect Windows that is compatible with Windows 10—which I see now from the Release Notes would be Retrospect Windows 10.5? P.S.: In last sentence of last paragraph, added earliest version of Retrospect Windows that would be compatible with Windows 10.
  23. DavidHertzberg

    Saving backups to OneDrive for Business (Retrospect 7.7)

    DovidBenAvraham, who did the major update of the Wikipedia article on Retrospect, stands corrected. Unfortunately the earliest version of the Retrospect Windows User's Guide that is available on the Retrospect.com website is for version 7.5. It does not have a "What's New" chapter, let alone a "What's New" chapter that covers Retrospect Windows 7.0, so DBA would have had to search through 389 pages of text (which BTW includes a duplicate of page 48 at the very end) to see which features that are listed as "new" in the press release for Retrospect Macintosh 8 are actually also in Retrospect Windows 7—even though they're not in the newly-found (see P.S. below) Retrospect Windows 7 press release. However, considering that Retrospect Windows 8.0 includes "Instant Scan technology" and "Improved disaster recovery process" and "Support for [WebDAV] cloud storage" along with other new features and "User-initiated backups" (inherited from Retrospect Mac 9 and obviously requiring some new "backup server" code), it seems that the "backup server" code is more than 1% new "under the hood". Of course DBA will try to update the Wikipedia article per what Mayoff says; to do otherwise would violate his principles as a WP editor. He now surmises that there must have been a Retrospect Macintosh 7, un-released and maybe not completely coded, that implemented many of the features that are listed as "new" in the Retrospect Macintosh 8 press release. These features obviously didn't include "powerful new engine" and "all-new, customizable [administrator] interface", but they probably included those features that are apparently new in Retrospect Windows 7. Did those features include the modern Disk type of Backup Set (as distinct from the old Disk type of Backup Set—which has been renamed Removable Disk in Retrospect Windows; see this post for what happens when you try to define a modern RDX disk as a Removable Disk) whose features are not mentioned in the Retrospect Windows 7.5 UG? Did they include "Complete backup of server clients", which seems not to have been not so complete in the Retrospect Windows 7.5 UG? Please provide answers to these questions, Mayoff, if you want DBA to attempt updating the WP article. Even armed with the answers to those questions, DBA will still have two problems updating the WP article. The first problem is that he must provide checkable references for the revised facts, and a forum post from Mayoff may not qualify as checkable. (To understand what a problem this can be, look at item (3) in this section of the Talk page accompanying DBA's WP article on his dead friend Ronny Lee; Lee had never mentioned that he was drafted too late to be sent overseas by the end of WWII to any of the magazine editors who wrote "About the Author" boxes accompanying Lee's articles—even though Lee had mentioned it to his friends, so DBA couldn't state that as a fact in the article.) The second problem is that another Wikipedia editor has already tagged the Retrospect article as "This article relies too much on references to primary sources." Primary sources, in the case of an article on software, means references to documents provided by the software manufacturer. DBA has already, with difficulty, made 50% of the references be from secondary sources—meaning reviews. However the only review of Retrospect Windows 7 that DBA has been able to find is this short one, which complains about the lack of change to the User Interface instead of describing many of the new features. P.S.: Never mind, Mayoff; DovidBenAvraham found a Retrospect Windows 7 press release among the bunch that Dell/EMC now has posted—we don't think it was there previously. Since the WP article's only reference to the Retrospect Macintosh 8 UG is to show that it wasn't copyrighted until 2011, maybe DBA can figure a way around that ref. and substitute the Dell/EMC press release (with its feature list) for it—thus keeping the number of primary-source references the same. P.P.S.: Revised 4th sentence in second paragraph; DovidBenAvraham has now made it clear in the WP article that the Disk type of Backup Set introduced in Retrospect Mac 8 and Retrospect Windows 10 is not the same as the Disk type of Backup Set that was in Retrospect Windows 7 and Retrospect Mac 6—it has new features. P.P.P.S.: DovidBenAvraham has now found a YouTube video, uploaded 28 April 2010, that shows—at minute 1:50 and afterwards—a Backup Media dialog that gives both Disk and Removable Disk as choices. This implies that, by the end of its long life, Retrospect Windows 7 had been retrofitted with the same modern Disk type of Backup Set that was introduced in Retrospect Mac 8. However this retrofitting was too late to be mentioned in the otherwise-so-useful Retrospect Windows 7 press release, and DBA does not consider it worth adding to the WP article another controversial primary source—the 2010 video was audibly narrated by Mayoff—just to make a historical point that was not mentioned in the 2011 Retrospect Windows 7.7 UG Addendum. P.P.P.P.S.: In last sentence of first paragraph. added "User-initiated backups"; "User-initiated restores" were in Retrospect Windows 7 as an Add-On, but not "User-initiated backups". P.P.P.P.P.S: Further enhanced parenthesized part of 4th sentence in second paragraph, to link to ProFromGrover's example of problems with use of Removable Disk instead of Disk to define a modern RDX disk cartridge. P.P.P.P.P.P.S: Added "under the hood" at the end of last sentence in first paragraph, thus clarifying the difference between the code I'm talking about and the code Mayoff is talking about (for British-English-trained readers, the "hood" in American car terminology is the "bonnet" in British car terminology).
  24. DavidHertzberg

    Saving backups to OneDrive for Business (Retrospect 7.7)

    The kindest way I can put this is "Don't judge a book by its cover." You should start by reading this section in the old version of the Wikipedia article, and then reading forward at least through the end of the "Retrospect Macintosh 10 and Retrospect Windows 8" section. What you'll learn is that Retrospect Windows 8 and following are totally different "under the hood" than Retrospect Windows 7.7. The reason the Retrospect Windows User Interface looks the same is that the Retrospect engineers went to a great deal of trouble to make it look the same, for reasons alluded to in the last paragraph of the "Retrospect Macintosh 8" section. I do not dare to be more specific on these forums for fear of upsetting Mayoff, but you can get a better idea of the 2009-2010 situation by following the first reference in the last sentence of that last paragraph in the section and reading the second post in the linked-to TheRegister.co.uk comment thread. The writer of that post does not have atypical views; when in June 2015 I mentioned that I was going to install Retrospect 12, I got over a page of strongly negative views from experienced Ars Technica posters who had been burned by Retrospect Macintosh 8. However I persevered; Retrospect Macintosh 12 has been good for me. To my knowledge there have been at least 3 recent complaints that Retrospect Windows 11.5 does not support some non-Amazon-S3-protocol-compliant cloud storage provider. In addition to the fact that the Retrospect engineers (unlike me; I'm a forum volunteer) have a budget and priorities, you fail to consider the possibility that some of these providers would not be suitable for cloud backup use. Please remember that many of these providers (Microsoft and Google, for example) provide cloud storage for use by "minnows" in conjunction with an office suite; what is sufficient capacity and speed for storing office documents may not be enough for doing Retrospect backups. As an example, a friend set up a free Dropbox account for me around two years ago; I stopped using it for inter-machine transfers after a not-very-large file took 2 hours to show up on the destination machine—whereas a sneakernet transfer of a similar file via USB "key fob" took less than 5 minutes including manual copying. So now it's time to ask a few questions, so we can delve into the wonders of 8th-grade arithmetic. What is your raw or effective upload speed (you can supply it in raw bits/sec., or you can pre-convert it to effective bytes/min. by applying the factors 60 secs./min. and 10 bits/byte—which allows for TCP/IP overhead)? How many bytes will you be backing up in a Normal Backup run, and how many minutes—using the effective bytes/min. calculated per the first question—will that take? Considering that none of these non-Amazon-S3-protocol-compliant cloud storage providers offers "seeding", how many bytes do you need stored in the cloud before you start doing your Normal Backup runs, and how long will it take—using the effective bytes/min. calculated per the first question—to store them? Finally, having calculated your effective download speed the same way you calculated your effective upload speed for the first question, how long will it take you to download a typical and near-bare-metal restore? I suspect you'll discover that your most-cost-effective approach is to continue (I hope you are already doing this) doing an off-site backup by periodically carrying a (preferably rotated) $80 USB 3 hard drive to your bank safe deposit box or to somebody's home. It takes a bit of discipline to do this, but you (I hope) have learned to brush and floss your teeth daily (insert appropriate smiley here). If you must have a practical test, you could download a trial version of Retrospect Windows 11.5, and also a trial version of ExpanDrive (I think you can get a free trial version). YMMV, since I have not used either program.
  25. DavidHertzberg

    Saving backups to OneDrive for Business (Retrospect 7.7)

    Aside from what Mayoff has said, one problem is that AFAIK Retrospect Windows did not support any kind of cloud backup until Retrospect Windows 8. At that point, because it picked up the underlying facilities of Retrospect Mac 10, Retrospect Windows 8 and beyond support the WebDAV protocol—which is the standard protocol for mapping cloud drives to a local computer. The fact that any mention of WebDAV, other than in a release note, is absent from the subsequent Retrospect Windows User's Guides is the fault of a Retrospect Inc. group that Mayoff has previously said he is not part of. Another problem is that Microsoft OneDrive does not officially support the WebDAV protocol. You can take up that question with Satya Nadella. There may be hacks for getting Retrospect 8 and later to support OneDrive, but I don't believe they will work with Retrospect 7.7. Thus, as Mayoff said, your first step would be to upgrade your version of Retrospect. The Retrospect Inc. engineers have made substantial improvements to "that otherwise identical in broad terms software" since 2011. As far as "inbuilt cloud support only for two or three such services", there are four cloud services (Amazon, Google Cloud, DreamHost, and Dropbox) in North America alone that support the Amazon AWS protocol that Retrospect Windows 11.5 supports. There are also cheaper cloud services that support the WebDAV protocol; the one that I am aware of is Box.com. I have expressed myself in the third sentence of the third paragraph of this post about what you want, but (trigger warning) that post is not fit for your frugal optimistic (insert appropriate smiley here) ears.