Jump to content

Leaderboard


Popular Content

Showing content with the highest reputation since 02/21/2019 in all areas

  1. 1 point
    More news after some more testing and waiting for instant scan to catch up. The CPU usage of RetrospectInstantScan seems to be in the low single digits percentage every time I look at them. That's amazing. It used to be above a 100% of a core when it didn't actually work. I have a backup running right now on this machine (iMac 2014 i7) and while copying files, RetrospectClient uses less than 10% CPU in Activity monitor (up to 30% in iStat Menus, though). I've seen it peak to around 60-70% before, but maybe that was before the index was properly built (even though I suspect the index has nothing to do with speed during upload, but who knows). If, in fact, there is an index (see below). The scanning period has not gotten faster than in the tests I did earlier, so I'm unsure if there's actually an index from instant scan. I suspect there isn't one. When looking in /Library/Application Support/Retrospect/RtrISAExec.dir, there's nothing there and I think that's where the index data is supposed to be. There is, however, a retroISA_log.utx file modified at the latest two days ago, with a size of 9.6 MB. Inside, the last entries are from March 5, so I think it stopped updating when I installed the 16.0 client. This leads me to think instant scan is no more, but that full scan has been made so much more effective that it doesn't matter much. Or, maybe the instant scan is saving data elsewhere. Why there's still an instant scan daemon mystifies me, though. Unless it's the old daemon they forgot to stop and delete. Another great bonus: there's a new feature in "sources", namely "automatic update" of the client, which seems to work, at least it updated one of the machines for me. That will be a huge relief, since it means we won't have to run around updating machines, or fiddle with the remote update thing, which I always found obnoxious. It seems to just do it on its own. Latest logs for the two machines, "after8" is a 2013 Macbook Air over wifi (with around 150 Mb/s) and "razr" is the 2014 i7 iMac on wired Gb. All backups to a Synology 1517+. Backup server is a 2019 Mac Mini i7. When looking at the log for the Macbook Air, you'll see it slower than the iMac, but a bit faster than it was before. It's not as fast as I'd like it to be, but it's within the range of acceptable. Arq runs for 13-15 minutes on the same machine, but that's without validation. Arq is also set to only backup the home folder, while Retrospect is set to backup the entire disk (with some exceptions). Now, interestingly, on the iMac, Arq is really slow at times, taking more than two hours to do a backup of the home folder in some cases and 20 minutes in other cases. I really don't know why. So right now, Retrospect rules on the fast machine, while Arq rules on the slower laptop. In conclusion: I think, for the first time, I can set MacOS machines to hourly proactive backups without completely hogging the machines (maybe 2 hour intervals would be more reasonable). The performance now is on par with Arq, and probably close to what can be done on the platform. Arq is still somewhat quicker, but that is maybe at least in part because it defers validation to a separate and more infrequent session, while Retrospect does it every time (if set to verify). But Arq also has slowdowns that I don't know the reason for. Remains the question why "Instant Scan" is still a setting in the client. Weird. Another side note: it looks as if backups have started working for sleeping machines, albeit very slowly. But I haven't looked into that systematically. + Normal backup using Razr at 2019-03-09, 12:52:07 (Activity Thread 1) 2019-03-09 12:52:08: Finished scanning backup set data files To Backup Set Razr... - 2019-03-09 12:52:07: Copying Macintosh HD on razr 2019-03-09 13:04:02: Found: 3797143 files, 864033 folders, 596.3 GB 2019-03-09 13:04:52: Finished matching 2019-03-09 13:08:23: Selector "Martin's machines" was used to select 3 681 955 files out of 3 797 143. 2019-03-09 13:08:46: Copying: 2406 files (2.5 GB) and 0 hard links 2019-03-09 13:11:20: Building Snapshot... 2019-03-09 13:11:20: Checking 864 033 folders for ACLs or extended attributes 2019-03-09 13:14:22: Finished copying 84 113 folders with ACLs or extended attributes 2019-03-09 13:14:35: Copying Snapshot: 2 files (1.2 GB) 2019-03-09 13:15:14: Snapshot stored, 1.2 GB 2019-03-09 13:15:14: Comparing Macintosh HD on razr *File "Macintosh HD/usr/local/var/mongodb/diagnostic.data/metrics.2019-03-05T19-37-16Z-00000": different data size (set: 991 232, vol: 995 328) *File "Macintosh HD/usr/local/var/mongodb/diagnostic.data/metrics.interim": different data size (set: 2 948, vol: 3 345) 2019-03-09 13:17:14: Execution completed successfully Completed: 2 406 files, 2.5 GB Performance: 1 108.6 MB/minute (943.9 copy, 1 354.4 compare) Duration: 00:25:06 (00:20:24 idle/loading/preparing) + Normal backup using After8 at 2019-03-09, 14:04:31 (Activity Thread 1) 2019-03-09 14:04:31: Finished scanning backup set data files To Backup Set After8 Martin... - 2019-03-09 14:04:31: Copying Macintosh HD on after8 (2) 2019-03-09 14:20:40: Found: 2412938 files, 532691 folders, 340.7 GB 2019-03-09 14:20:59: Finished matching 2019-03-09 14:22:12: Selector "Martin's machines" was used to select 2 344 605 files out of 2 412 938. 2019-03-09 14:25:48: Copying: 322 files (715.7 MB) and 0 hard links 2019-03-09 14:27:49: Building Snapshot... 2019-03-09 14:27:49: Checking 532 691 folders for ACLs or extended attributes 2019-03-09 15:01:46: Finished copying 358 898 folders with ACLs or extended attributes 2019-03-09 15:01:54: Copying Snapshot: 2 files (255.3 MB) 2019-03-09 15:02:04: Snapshot stored, 255.3 MB 2019-03-09 15:02:04: Comparing Macintosh HD on after8 (2) *File "Macintosh HD/Applications/Junos Pulse.app/Contents/Plugins/JUNS/accessRecovery.ini": didn't compare *File "Macintosh HD/usr/local/var/postgres/server.log": different data size (set: 123 352 400, vol: 123 367 500) 2019-03-09 15:02:34: Execution completed successfully Completed: 320 files, 712.1 MB, with 52% compression Performance: 662.4 MB/minute (431.5 copy, 1 473.2 compare) Duration: 00:58:03 (00:55:54 idle/loading/preparing)
  2. 1 point
    Just ran a backup on a 2014 iMac with wired Gb network (the Macbook Air is on wifi), and things are decidedly looking more useable than before Retrospect 16. The scanning took just 2 minutes 11 minutes or so and the entire process 20 minutes. Retrospect is back in the race now. I still can't make out if it's using instant scan or not, but with this speed it must be. /Martin + Normal backup using Razr at 2019-03-06, 18:19:20 (Activity Thread 1) 2019-03-06 18:19:20: Finished scanning backup set data files To Backup Set Razr... - 2019-03-06 18:19:20: Copying Macintosh HD on razr 2019-03-06 18:30:18: Found: 3796060 files, 863974 folders, 596.4 GB 2019-03-06 18:30:50: Finished matching 2019-03-06 18:32:18: Selector "Martin's machines" was used to select 3 681 740 files out of 3 796 060. 2019-03-06 18:32:30: Copying: 2330 files (3.9 GB) and 0 hard links 2019-03-06 18:35:25: Building Snapshot... 2019-03-06 18:35:25: Checking 863 974 folders for ACLs or extended attributes 2019-03-06 18:36:42: Finished copying 83 900 folders with ACLs or extended attributes 2019-03-06 18:36:55: Copying Snapshot: 2 files (1.2 GB) 2019-03-06 18:37:28: Snapshot stored, 1.2 GB 2019-03-06 18:37:28: Comparing Macintosh HD on razr *File "Macintosh HD/Users/mw/Pictures/Photos Library.photoslibrary/private/com.apple.mediaanalysisd/MediaAnalysis/mediaanalysis.db-shm": didn't compare *File "Macintosh HD/Users/mw/Pictures/Photos Library.photoslibrary/private/com.apple.mediaanalysisd/MediaAnalysis/mediaanalysis.db-wal": different data size (set: 32 992, vol: 45 352) *File "Macintosh HD/usr/local/var/mongodb/diagnostic.data/metrics.2019-03-05T19-37-16Z-00000": different data size (set: 360 448, vol: 368 640) *File "Macintosh HD/usr/local/var/mongodb/diagnostic.data/metrics.interim": different data size (set: 3 457, vol: 2 913) 2019-03-06 18:39:34: Execution completed successfully Completed: 2 330 files, 3.9 GB Performance: 1 595 MB/minute (1 332.2 copy, 2 003.9 compare) Duration: 00:20:13 (00:15:14 idle/loading/preparing)
  3. 1 point
    I installed 16.0 yesterday and am test running it now on my 2013 MacBook Air with MacOS 10.14.3, APFS. There is a process "Retrospectinstantscan" running so that's good news. Retrospect did a full scan of almost 2,200,000 files, over 300 GB, in 20 minutes. No indication that it was using any index from instant scan, but I don't think you can expect that so early after installing it. Will have to see over the next few days if it starts using an index. The preference pane for the Retrospect client does say that Instant Scan is enabled. (But so it did for the 15.6 as well, even when it wasn't.) After 23 minutes, it started copying files. The total running time, including copying of almost 8 GB of files, was an hour and 17 minutes, as compared to over three hours in my September post (see above). The scanning phase went down from about 2.5 hours to 20 minutes. Yay! The client clearly uses less CPU now. It also backs off if something else uses significant CPU, so the machine is running much cooler, without the usual jet engine sounds. Nice. I'll be back in a couple of days to see if Retrospect actually catches up to the performance of competing-product-that-shall-not-be-named-here. It still has a way to go, but it's entirely possible that a few days of indexing can fix that. /Martin + Normal backup using After8 at 2019-03-06, 11:52:18 (Activity Thread 2) 2019-03-06 11:52:19 : Finished scanning backup set data files To Backup Set After8 Martin... - 2019-03-06 11:52:18 : Copying Macintosh HD on after8 (2) 2019-03-06 12:10:57 : Found: 2414406 files, 532724 folders, 337,8 GB 2019-03-06 12:11:14 : Finished matching 2019-03-06 12:12:16 : Selector "Martin's machines" was used to select 2 344 893 files out of 2 414 406. 2019-03-06 12:15:24 : Copying: 4216 files (7,9 GB) and 2 hard links 2019-03-06 12:32:42 : Building Snapshot... 2019-03-06 12:32:44 : Checking 532 724 folders for ACLs or extended attributes 2019-03-06 13:04:33 : Finished copying 359 066 folders with ACLs or extended attributes 2019-03-06 13:04:38 : Copying Snapshot: 2 files (255,4 MB) 2019-03-06 13:04:47 : Snapshot stored, 255,4 MB 2019-03-06 13:04:47 : Comparing Macintosh HD on after8 (2) *File "Macintosh HD/Applications/Junos Pulse.app/Contents/Plugins/JUNS/accessRecovery.ini": didn't compare *File "Macintosh HD/usr/local/var/postgres/server.log": different data size (set: 121 455 000, vol: 121 471 000) 2019-03-06 13:09:21 : Execution completed successfully Completed: 4 217 files, 7,9 GB, with 25% compression Performance: 762,5 MB/minute (483,3 copy, 1 811,4 compare) Duration: 01:17:03 (00:55:45 idle/loading/preparing)
  4. 1 point
    David, your condescending comments aren't needed here. Martin (and others, myself included) are frustrated at this slowness, and a little venting is not out of order. I don't understand why you feel the need to carry water for Dantz - if you don't have anything positive to contribute, you aren't required to post.
  5. 1 point
    So how about updating the Gender in your Forums Profile, myhrik, from "Not saying" to "Male"—as I did years ago? In general I don't try to guess people's genders from their "handles", although I make an exception for those "handles" that are obviously male or female. OTOH I can't help but be aware that many backup administrators are women, and that a lot of those try to conceal that fact for fear that they won't be taken seriously on the Forums. Unfortunately myhrik's ancestors, when they invaded Britain, contributed a number of words to the English language—but not gender-neutral third-person pronouns. So I write "him/her" and "he/she" when the Forums Profile doesn't specify Gender and I can't guess. My other tactic is to address a poster with the second-person pronoun, which is gender-neutral in English (and has no difference between singular and plural, unless you're either from the American South—where they use "y'all" for the plural—or an old-time Quaker—who might use "thou" in talking to someone besides the Almighty). That's why so many of my posts begin with my naming the poster(s) I am responding to, myhrik.
×