alche Posted November 25, 2009 Report Share Posted November 25, 2009 I've spent the last week developing a Filemaker 10 based log analysis program for Retro8. It works pretty well, and I've prettified it and made it suitable for public consumption. Can be downloaded from here: my website (lintburger.com) The idea was to get a more systematic idea of what's going on with my backups, so summaries, running totals, and counts of errors and throughput are the goal here. There's even a chart function. Free to use, free to modify. Screenshots and more details are on software page. Hope you find it useful. Cheers, Paul Fabris Quote Link to comment Share on other sites More sharing options...
rhwalker Posted November 25, 2009 Report Share Posted November 25, 2009 That's really nice, Paul. Thanks. Russ Quote Link to comment Share on other sites More sharing options...
alche Posted November 25, 2009 Author Report Share Posted November 25, 2009 Whoops. Another double post. Can an admin delete the other post of this topic, please. Quote Link to comment Share on other sites More sharing options...
rhwalker Posted November 25, 2009 Report Share Posted November 25, 2009 If you made the post / topic, you should be able to delete it. Just click the Edit icon, then click Delete Post near the top navigation bar. Russ Quote Link to comment Share on other sites More sharing options...
Mayoff Posted November 25, 2009 Report Share Posted November 25, 2009 This thing is really cool. I will show it to the team on Monday Quote Link to comment Share on other sites More sharing options...
alche Posted November 25, 2009 Author Report Share Posted November 25, 2009 Thanks for the delete tip Russ. Quote Link to comment Share on other sites More sharing options...
alche Posted November 26, 2009 Author Report Share Posted November 26, 2009 (edited) I've updated Retro8LogAnalyser to version 1.1 The new version allows you to mindlessly paste data from the Retrospect Log window without having to worry if it overlaps data you previously pasted in. Should make life a little easier. It will automatically keep track of the last timestamp in the log data. The next time you paste in data, it will clear the pasted data up to that last timestamp, leaving only the new log lines to process. I've also added an update engine that allows you to upgrade to future versions without having to reprocess all your data again. This won't work with the data in 1.0, but will work when I release future versions. A few other bug fixes. Can be downloaded from the Lintburger Software Page Cheers, Paul Edited November 26, 2009 by Guest Quote Link to comment Share on other sites More sharing options...
cpguru21 Posted December 3, 2009 Report Share Posted December 3, 2009 This looks like a great tool. Any chance there is anything out there for someone who does not use file filemaker? Quote Link to comment Share on other sites More sharing options...
rhwalker Posted December 3, 2009 Report Share Posted December 3, 2009 Paul's website includes a runtime version with the FileMaker player. Russ Quote Link to comment Share on other sites More sharing options...
cpguru21 Posted December 3, 2009 Report Share Posted December 3, 2009 Thanks Russ! I seemed to have overlooked that. Quote Link to comment Share on other sites More sharing options...
alche Posted December 4, 2009 Author Report Share Posted December 4, 2009 New in Version 1.2 (Dec 4 2009) Charting: Added Machine Charting. Allows you to chart the growth of backups over time for each machine, by Media Set and Date Range. Charting: Option-Click "Generate" button to automatically update the chart to the current date and move the ceiling if necessary. Shift-Click to Generate charts for a found set of records with the same parameters. Option-Shift-Click to update a Found Set of Charts. Charting: Improvements allow for proper printing of charts Added List Views for Exec Events and Charts Improvements in automatic log line deduplication; a dialog now shows you where the log line processing will start after deduplicating Other minor bug fixes and enhancements You can import data from Version 1.1. Download the new version, launch it, pull down the "Scripts" menu and choose the "Import from an Older Copy of Retro8LogAnalyser". Download from Lintburger Software Page Machine Charting Screenshot: Quote Link to comment Share on other sites More sharing options...
Giobbi Posted December 11, 2009 Report Share Posted December 11, 2009 Absolutly lovely! Great work. Only wish would be to make an export to file instead of copy paste from Retro. Or to take directly from server-log. But really nice work MCI - impressive. Quote Link to comment Share on other sites More sharing options...
alche Posted December 16, 2009 Author Report Share Posted December 16, 2009 Thanks! As for taking directly from the server log file, I did try, but the operations_log.utx is much less readable to the layperson. It contains what appears to be variable placeholders and some raw code that I can't quite grasp. If a future version of Retro8 wrote the readable log file out to a directory somewhere (which would be specifiable in a preference setting), then I could easily just suck in the new data into the Analyser without forcing anyone to copy/paste, and without needing to do any date parsing to figure out where the log overlap was. I would simply keep track of the last line processed and only import data from there. The copy paste method isn't the end of the world, but it can introduce problems. A few weeks ago I deleted several SVN directories from my web server, apparently mid-backup. The Retro Log promptly filled up with around 20,000 lines of file errors. The next morning, i did my usual copy paste and it buggered up my log processor. I have my Log Display set to 2000 lines and so it wasn't making it anywere near the start of the previous nights backups, which kinda screwed everything up. (Luckily, there's the "Delete from last Import" script to deal with these problems). I had to change my Retro Log pref to view 30,000 lines, copy/paste, then swich my pref back to the usual 2000 lines. If I we were looking at a static file instead, I wouldn't have had this problem, and I could make the Analyser process the logs unattended on a timed basis, and even send emails. (Of course, this would introduce the problem of having a log file of unlimited size eating up drive space. Mind you, I haven't seen operations_log.utx rotate, or clip early entries, so this may already be the case.) Until then it's CMD-L, CMD-A, CMD-C, CMD-TAB, CMD-1, CMD-V. Actually, a bigger problem is data inaccuracy in the log file. The CMD-L reports data sizes in GB/MB/KB as needed. This introduces rounding errors when compounding lots of log lines and trying to add them up--not a huge problem but there might be up to a GB of error after a lot of executions. Worse than that is when Retrospect crashes mid-execution. I had this happen over the weekend. Nothing from the execution made it into the operations_log.utx file at all. But my backup disk had 15GB more than the day before. The catalog had the data, but the log buffer didn't get written out before the crash occurred. So what would be really nice would be a way to extract raw columnar data from the catalogue(s), on a timed basis. Or if Retro wrote out the columnar data for each catalogue as it was updated. Anyway, I'm working on making the charts more flexible for version 1.3, plus a few other odds and ends. Quote Link to comment Share on other sites More sharing options...
hmseiden Posted December 18, 2009 Report Share Posted December 18, 2009 Both the database program, and the fonts are real sweet! Great util Paul. Quote Link to comment Share on other sites More sharing options...
hmseiden Posted December 18, 2009 Report Share Posted December 18, 2009 Absolutly lovely! Great work. Only wish would be to make an export to file instead of copy paste from Retro. Or to take directly from server-log. But really nice work MCI - impressive. Ditto. I have yet to add more log data (cutting and pasting doesn't seem too bad for me), even though it would be nice to simply have it pick up the new data. For some reason the machine list doesn't populate like the Media Set do (drop down) and must enter the machine name to chart by new record as warning indicates. But after that, it great! Quote Link to comment Share on other sites More sharing options...
alche Posted December 22, 2009 Author Report Share Posted December 22, 2009 Try the "Generate Machine Records" script under the Scripts menu. Will automatically generate Chart records for all the Machines it can find in the Exec Events table. Otherwise, yes, you have to create records manually for Machines. Figured some people with large lists of machines might not want automatic here. And, agreed about the copy/paste, but sadly can't do anything about it at this point (see my post above). Cheers, PF Quote Link to comment Share on other sites More sharing options...
francishampden Posted January 5, 2010 Report Share Posted January 5, 2010 Hi Paul, Like the DB. Very useful. I've notced that it is reading my log file for 5th Janaury 2010 (1/5/2010) as May 1, likewise 3 January as March. I've got FMProAdv so can go and amend myself if it is an easy change that you'd recommend. Cheers, Frank Quote Link to comment Share on other sites More sharing options...
alche Posted January 7, 2010 Author Report Share Posted January 7, 2010 That's strange. Can you tell me which screen you are seeing this in? My logs are being correctly interpreted, as far as I can see. Might also be useful to know what Date & Time settings you are using in System Preferences>>International>>Formats. Quote Link to comment Share on other sites More sharing options...
leopod Posted January 13, 2010 Report Share Posted January 13, 2010 (edited) I have the same issue with dates...and no FMpro to amend...International Format is set to Canada...I changed to US to see if it changes something to the log but it doesn't seem to. edit : the change to US M/D/Y actually worked. Nevertheless, thank you for the nice piece of work! Edited January 13, 2010 by Guest Quote Link to comment Share on other sites More sharing options...
alche Posted January 15, 2010 Author Report Share Posted January 15, 2010 Version 1.2.1 is out. It addresses the issue of when the International System Preference is not set to use US formats by embedding the US format in the Filemaker Pro database file (effectively ignoring your International setting). This is necessary due to the fact that Retro Log Data uses dates that are always mm/dd/yyyy. I believe this should address the issues posted above without requiring user intervention. Please let me know. There's a couple of other minor tweaks and fixes as well. Download from Lintburger Software Page Cheers, Paul Quote Link to comment Share on other sites More sharing options...
sdmactech Posted January 22, 2010 Report Share Posted January 22, 2010 It appears the Runtime link is dead? 1 Quote Link to comment Share on other sites More sharing options...
alche Posted January 22, 2010 Author Report Share Posted January 22, 2010 Sorry about that. Typo. It's fixed now. Quote Link to comment Share on other sites More sharing options...
jel Posted March 22, 2010 Report Share Posted March 22, 2010 I've updated Retro8LogAnalyser to version 1.1 The new version allows you to mindlessly paste data from the Retrospect Log window without having to worry if it overlaps data you previously pasted in. Should make life a little easier. It will automatically keep track of the last timestamp in the log data. The next time you paste in data, it will clear the pasted data up to that last timestamp, leaving only the new log lines to process. I've also added an update engine that allows you to upgrade to future versions without having to reprocess all your data again. This won't work with the data in 1.0, but will work when I release future versions. A few other bug fixes. Can be downloaded from the Lintburger Software Page Cheers, Paul FileMaker 11 (now out) adds a new repeating import facility. Would this be useful for your database so it can automatically import new log data from Retrospect? (I have Retrospect 11.) Quote Link to comment Share on other sites More sharing options...
alche Posted March 22, 2010 Author Report Share Posted March 22, 2010 FileMaker 11 (now out) adds a new repeating import facility. Would this be useful for your database so it can automatically import new log data from Retrospect? (I have Retrospect 11.) Hi John, The problem isn't FileMaker's ability to import on a recurring basis (that can be done in a number of different ways). The problem is that the RetroLog.utx file, where all the log data is stored, is incomprehensible to me. It's filled with what appear to be variables that only make sense to the RetroConsole App, in conjunction with a Server configuration. I can't parse this file. I have to rely on the "interpreted" output of this file, which is currently only available by using the Show Log command from within the RetroConsole app. 2 ways around this: 1. Retro Engineers do something that allows RetroServer to output a plain text log file using the built in scheduling engine, preferably in a coherent delimited format. (Currently, I have to read the log data almost as if it were a story.) 2. You could write an Applescript that does the following: Launch Retro Console wait issue a CMD-L wait CMD-A CMD-C Launch RetroLogAnalyser wait issue a CMD-1 wait CMD-V ENTER wait ENTER Tie it to a crontab on a computer that's running all the time (without screensaver) and you've got Automation. Quote Link to comment Share on other sites More sharing options...
macjoy Posted April 27, 2010 Report Share Posted April 27, 2010 I just wanted to say that this is a very useful utility and just say thank you for the work you obviously put into it. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.