Scanning incomplete, error -108 (not enough memory)

[ccpi]

I have Retrospect 6.1.126 running on a G4 with 768MB of RAM with OS X 10.2.8.

On a MacPro and a G5 with OS X 10.4.9, I have Retrospect Remote 6.1.130.

 

On the client machines there is an Adobe Lightroom database. The database keeps its previews in a package named "Lightroom Previews.lrdata". Inside this package, there are 48,695 directories, which contain a total of 86,636 files.

 

The files are mostly JPEGs of varying sizes with long filenames like this:

 

FFF5B4AD-FF41-11DA-8BC5-011124CC99EA-19a240b5dcdd7d1ccc6c7819491bfc96.lr-preview.noindex

 

When the backup for this client runs, during scanning (not matching) it gets the following error:

 

Scanning incomplete, error -108 (not enough memory)

 

If I copy this package/directory to the machine running Retrospect so it can scan it locally, it does not get this error.

 

Also, if I temporarily remove 6 of the subdirectories from the package on the remote machine, so the number of directories is reduced to 30,404 and the number of files is reduced to 54,894, the error does not occur.

 

Retrospect Remote seems to have no trouble with a Mac OS X installation which has over 80,000 folders and over 350,000 files, but Retrospect gets the not enough memory error with this particular package/directory. I've also tried duplicating it to a normal directory but get the same error.

 

Has anyone run into a similar problem? Any ideas how to fix it? Is it the length of filenames which causes this problem?

 

The error occurs with Retrospect Driver Update, version 6.1.9.102 and version 6.1.10.100. I also tried installing Retrospect 6.1.126 on a G5 with 2.5gb of RAM with OS X 10.4.9, and had the same problem, but strangely, a different error message:

 

Error!!! Scanning incomplete, error -116 (unknown)

[rhwalker]

Quote:

---

On a MacPro and a G5 with OS X 10.4.9, I have Retrospect Remote 6.1.130.

 

---

There is no longer any product "Retrospect Remote" and has not been for some time. I suspect that you are running some flavor of Retrospect (desktop, workgroup, server). See this product comparison chart:

Retrospect product comparison chart

 

These machines should be running Retrospect 6.1.126. The only use for Retrospect 6.1.130 is for an INTEL Mac running Mac OS (server or non-server) 10.4.7 or 10.4.8 with a volume that has ACLs turned on. See:

Retrospect ACL hack

 

Your post is a little unclear as to what is happening with what versions. Could you please clarify?

 

Russ

[ccpi]

Russ,

Quote:

---

Quote:

---

On a MacPro and a G5 with OS X 10.4.9, I have Retrospect Remote 6.1.130.

 

---

There is no longer any product "Retrospect Remote" and has not been for some time. I suspect that you are running some flavor of Retrospect (desktop, workgroup, server).

 

---

 

I meant to say Retrospect Client 6.1.130.

It is Retrospect Workgroup 6.1.126 running on the G4 accessing the Clients.

 

Quote:

---

 

Your post is a little unclear as to what is happening with what versions. Could you please clarify?

 

 

---

 

I tested with two different versions of the Retrospect Driver Update, and the result was the same.

 

Retrospect Workgroup 6.1.126 on G4 (OS X 10.2.8) scanning on a local drive = no problem

 

Retrospect Workgroup 6.1.126 on G4 (OS X 10.2.8) scanning via Retrospect Client 6.1.130 on a remote machine = Scanning incomplete, error -108 (not enough memory)

 

Retrospect Workgroup 6.1.126 on G5 (OS X 10.4.9) scanning via Retrospect Client 6.1.130 on a remote machine = Scanning incomplete, error -116 (unknown)

 

-jr

[rhwalker]

Quote:

---

On the client machines there is an Adobe Lightroom database. The database keeps its previews in a package named "Lightroom Previews.lrdata". Inside this package, there are 48,695 directories, which contain a total of 86,636 files.

 

The files are mostly JPEGs of varying sizes with long filenames like this:

 

FFF5B4AD-FF41-11DA-8BC5-011124CC99EA-19a240b5dcdd7d1ccc6c7819491bfc96.lr-preview.noindex

 

---

I agree, that's not many files. But I didn't write the code; perhaps the memory needed is sensitive to the length of filenames, such that longer filenames require more memory.

 

Quote:

---

Also, if I temporarily remove 6 of the subdirectories from the package on the remote machine, so the number of directories is reduced to 30,404 and the number of files is reduced to 54,894, the error does not occur.

 

---

How much memory is on this remote machine? From your first post, it appears that you have one MacPro running Mac OS X 10.4.9 with (from your second post) Retrospect client 6.1.130. It also appears from your first post that you have a G5 running Mac OS X 10.4.9 with (from your second post) Retrospect client 6.1.130.

 

Retrospect (and client) on an Intel machine run emulated under Rosetta, which requires about 1 GB itself in addition to the OS memory requirements. There is not any Universal Binary for Retrospect or Retrospect client. So, on an Intel machine, you would need at least 2 GB or so.

 

It's unclear from your posts whether you are seeing this same result both on the G5 client running Mac OS X 10.4.9 AND on the Intel MacPro client running Mac OS X 10.4.9.

 

Are you?

 

It may be a simple case of the message being correct, and you just need more memory.

 

Russ

[ccpi]

Quote:

---

How much memory is on this remote machine? From your first post, it appears that you have one MacPro running Mac OS X 10.4.9 with (from your second post) Retrospect client 6.1.130. It also appears from your first post that you have a G5 running Mac OS X 10.4.9 with (from your second post) Retrospect client 6.1.130.

 

---

 

The error occurs on the following machines running Retrospect Client 6.1.130:

 

MacPro 8-core (OS X 10.4.9) with 10 GB RAM

Dual 2.7 GHz G5 (OS X 10.4.9) with 8 GB RAM

G4 (Mac OS X 10.4.9) with 1.75 GB RAM

 

Quote:

---

It's unclear from your posts whether you are seeing this same result both on the G5 client running Mac OS X 10.4.9 AND on the Intel MacPro client running Mac OS X 10.4.9.

 

Are you?

 

It may be a simple case of the message being correct, and you just need more memory.

 

---

 

Yes, same problem on all three machines above. I thought it might be because the older G4 (OS X 10.2.8) machine running Retrospect Workgroup 6.1.126 only had 768 MB RAM, so I installed Retrospect Workgroup 6.1.126 on a dual 2 GHz G5 (OS X 10.4.9) with 2.5 GB RAM. The same problem occurred, but with the "Scanning incomplete, error -116 (unknown)" message instead of "Scanning incomplete, error -108 (not enough memory)"

 

-jr

[CallMeDave]

Sounds like a Client bug. Contacting tech support, paying the money upfront, and then asking for a refund if they are unable to help and/or acknowledge it as a software defect might be one way to go.

 

> Is it the length of filenames which causes this problem?

 

Might very well be.

 

Dave

[abarker]

I'm also getting the same error message in my log when I'm trying to backup:

 

I'm using Retrospect Server v6.1 on an Apple X-Server Dual G4 1.33 GHz with 2GB RAM,

running OS X Server 10.4.9

The main server volume has around 230GB free space.

I'm backup up an 1TB X-Serve Raid (460GB of used data) to an External Western Digital

My Book Firewire 800 1TB drive. All the volumes are HFS+ Journaled.

 

I've scheduled a Normal backup to File on the external but after scanning for ages

it comes up with the above error message.

 

Unfortunately the X-Serve according to Everymac can ONLY take a max of 2GB so if it

is indeed a memory issue then we can't upgrade to cater.

 

Does anyone have any solutions other than ditching Retrospect in favour of something else.

 

Cheers

[CallMeDave]

Quote:

---

I'm also getting the same error message in my log when I'm trying to backup:

 

---

 

A smarter use of the progrom is to "granularize" your Source volumes; define sets of folders as sub-volumes, and use them as a group as your Source. That way, Retrospect can scan fewer files at any one time.

[ccpi]

abarker,

Quote:

---

I've scheduled a Normal backup to File on the external but after scanning for ages

it comes up with the above error message.

 

---

 

Do you know if it's stopping at a particular place? I ended up making multiple sub-volumes and scanning each one to figure out that a particular package was causing the error message. This was a tedious, time consuming process.

 

 

Dave,

Quote:

---

A smarter use of the progrom is to "granularize" your Source volumes; define sets of folders as sub-volumes, and use them as a group as your Source. That way, Retrospect can scan fewer files at any one time.

 

---

 

This workaround has some general problems and I can't think of a way to make it work for the specific problem I am experiencing. In addition to the extra work of making multiple sub-volumes for each client machine, one of the general problems is that users sometimes make additional folders at the root level of the hard drive, which wouldn't be covered by any pre-existing sub-volumes.

 

The directory (actually a package) which causes the problem in my case has nearly 3,000 files and 15 sub-directories (each with many files and subdirectories). The 15 sub-directories of the package could be made into 15 sub-volumes, but that would still leave the 3,000 files in the directory. I don't know of any way to include those files. I'd also have to move the package up a few directory levels so I wouldn't have to make additional sub-volumes for every parallel directory.

 

-jr

[rchavez]

What if you can't granularize. I have an xServe with managed students (2.1GB) in one folder/directory. I would have to select each student as a source. My clients are solely servers with staff and student data.

 

What gets me though is I run several other similar backups of student and staff sources without problems. They vary in size from 1.4GB to 4GB. They run fine (normal/recycle). It's just this one source. I have changed target - different external HDs, even to one solely for this source but same problem, changed times, redonne scripts.

 

They are running in the off hours, so there is nothing else running at the scheduled time. Frustrated!!!!

 

I run Retrospect 6.1.126 on a G5 xServe (5GB memory). The client with the problem resides on another G5 xServe (4GB memory). Other sources are on the same xServe where I am having a problem, but those run fine.

[ccpi]

Here are some things I found out:

Retrospect uses about 30% of available system memory, so on the older G4 with 768mb, that's not too much.

 

If I added more memory to that machine, I might not have the error -108 out of memory error, but in the case of this directory, I'd probably end up getting the error -116 unknown error (which I got on the G5 which has 2.5gb RAM).

 

Using a selector to exclude the directory that the problematic package is in will not work because Retrospect scans everything, then does the exclusion, so it will run out of memory during the scan.

 

The only way to exclude the directory so Retrospect doesn't even look at it is to open Retrospect Client, Click Preferences and check the "Private Files/Folders/Volumes name begins or ends with • (Option-8)" option in the Access Restrictions section. Then rename the directory the package is in, in my case to "Lightroom•". If you do use this option, any other files, folders or volumes which begin or end with the bullet character will be ignored and NOT backed up.

 

I might set up a cron job to create an archive of the directory which includes the package before the backup runs so the data which is excluded will get backed up. It's not the best solution, but it should work.

 

I received no indication that this issue would be addressed in an update. It seems to me to be a serious flaw, so I hope that is not the case.

[abarker]

Dave,

Quote:

---

A smarter use of the progrom is to "granularize" your Source volumes; define sets of folders as sub-volumes, and use them as a group as your Source. That way, Retrospect can scan fewer files at any one time.

 

---

 

I understand what you're saying, but I'll probably have to wait til the summer to move and group home folders.

Currently I have 1137 user folders in my 'Homes' folder, I think the filecount was something like 3 million :-o, so indeed it probably is barfing on the catalogue memory consumtion in this regard.

 

It's a shame Retrospect can't allow you to select a number of folders and define them as part of a sub-volume ?

I've tried using filters, so that say all folders beginning with 'a' would be backuped, but it still barfs. I have a feeling

Retrospect still scans all folders before it filters. A good function to have, would be if you can define a Sub-Volume or set for say 'a' and then tell retrospect to backup on a* folder names and their contents. I've tried all sorts with the filtering, but to no avail, but that's not to say retrospect can't do it, it might just be my inexperience.

 

I've even tried making a folder with a couple of symbolic links to the folders I wanted to backup, but it only backs up the

link and not the folder it's linked too. If that had worked I could have created a script to generate symlinks to new folders.

 

Obviously I could define a sub-volume for 1000+ folders, but that would take ages, and when I create or delete a user account and their home folder I'd have to manually add it into the backup set which kind of defeats automation.

 

I even tried a filecopy rather than backup to File/Disk/etc, and it still failed with a error -108 so still has to scan all files.

 

I wonder if it's possible for Retrospect to catalogue in chunks ? So that it would use so much RAM, then swap it out to disk to do another chunk, then at the end reassemble it into one catalogue ? I guess you'd still have the problem of reading it back into memory for restore, but still chunks could work ?

 

Has anyone got any further on this matter, rather than us speculating ?

[rhwalker]

Quote:

---

I have a feeling Retrospect still scans all folders before it filters.

 

---

Yep.

[CallMeDave]

Quote:

---

It's a shame Retrospect can't allow you to select a number of folders and define them as part of a sub-volume

 

---

 

Certainly having a more efficient way of defining folders as subvolumes would be welcome. The current method uses the old SFPut API (SF=Standard File) from Classic Mac OS days. Coming up with something more efficient would require an entirely new paradigm; there may or may not even be APIs available to do this. Note that the folder definition interface for Apple's own Workgroup Manager still requires the user to click through file trees, even though it uses a different style of interface.

 

But no matter; it's unlikely that Retrospect will ever get the sort of radical interface changes that you're hoping for in this regard.