Jump to content
Sign in to follow this  
TooTall

Memory Problem, any possible solutions??

Recommended Posts

I am currently getting this error when trying to back up a volume with a large number of files inside. Ex. the volume contains a folder for every day and then the files for that day are inside. Does any one know of a way I can get past this memory problem by suing selectors or subvolumes?? I could use subvolumes but then I would have to modify the script everyday since a new folder is created every day. If I reduce the volume selection with selectors will that help with the memory problem or not?

 

Tech support said they know they have a problem with voules containing over 1.4 million files, I believe this volume on my server has about 1.8 million.

 

ANy ideas???

 

Here is the error:

 

 

- 8/2/2003 11:33:17 AM: Copying New Volume (F:) on FILESERVER

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(361.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(353.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(353.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(353.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(353.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(353.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(353.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(353.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(353.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(349.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(349.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(349.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(349.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(349.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(349.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(349.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(349.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(347.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(347.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(347.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(347.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(347.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(347.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

TMemory::mhalloc: VirtualAlloc(346.0 M, MEM_RESERVE) failed, error 8

Scanning incomplete, error -625 (not enough memory)

8/2/2003 12:02:54 PM: Execution incomplete

 

 

Thanks alot.

 

TooTall

 

Share this post


Link to post
Share on other sites

Sure:

 

 

The server running Retrospect: Dell PowerVault 725N Dual 2.0ghz 512MB Ram, WindowsPowered OS

 

The server being backed up: Gateway 7400 Dual Xeon 800's 4 gb ram, Windows 2000 Server

 

The volume retrospect can't back up is structured like the following:

 

Root of drive-->Img-->(in here is one folder for every day--over 2.5years worth)-->(inside here is 8 subfolders representing sorting areaas)-->in each of the 8 subfolders are the files being backed up

 

So essentially inside the "Img" folder is about 1.8 million files (they are archive images from image based processing) and is about 90gb in total size.

 

Retrospect tech support told me that Retrospect does not utilize more than 512 mb ram so increasing ram won't help.

 

My real question is if I can break the volume up somehow but still allow me to include the new folder that is created daily automatically without having to modify the backup script daily.

 

It is too bad I can't create a virtual volume that includes multiple subfolders then I could break the volume that way.

 

Thnks for any help you all can give.

 

TooTall

 

Share this post


Link to post
Share on other sites

Actually, the 6.5.319 version is the one I am having problems with.

 

I hope you make some progress with this memory issue soon, as it is a very big hindrance in my network.

 

That along with another issue of mine with roff.sys still not working on my two database servers, that have 8gb ram. I was told by tech support that roff was fixed in 6.5.319 but it still blue screens at boot up after installing the client. (The roff.sys that installs with teh 6.5.319 download is versioned at 6.5.107 and has a date of 3/17/2003, if anyone has a newer version to try let me know.)

 

I was very hopeful that Retrospect would be the solution to my backup needs but in light of these two issues I am afraid I will be forced to try something else.

 

Thanks,

TooTall

 

 

Share this post


Link to post
Share on other sites

Actually, Retrospect 6.5 will use around 1GB of RAM on the backup server for scanning and Matching.

 

Keep in mind that Retrospect also counts out the folders with the files, so we are probably going above the 1.8 million you mentioned.

 

I have given your report to our engineers as another "case study".

 

Thanks

Share this post


Link to post
Share on other sites

Do you have virtual memory configured to 2 GB? You should also make sure that there are no other executions going on at the same time. Are you using the backup machine for any thing else that could also be using a lot of memory?

 

1.8 million files is very close to what we can handle currently, so it is possible that all of this might not help.

 

Is this a System volume? If not, depending on your directory structure, you can set up subvolumes to be backed up instead of the entire volume.

 

 

Share this post


Link to post
Share on other sites

I have only one execution running at a time, I will try setting the virtual memory to 2 gb but since this approaching the limit, and considering the number of files grows 15,000 - 20,000 a day it may not help alot.

 

I thought of subvoluming, but I believe the only option for that would be to have a subvolume for each day (over 700 of them) due to the folder layout. Is there anyway to automatically include any new folders that are created (one per day) in the subvoluming so I don't need to modify the script daily??

 

TooTall

Share this post


Link to post
Share on other sites

I, too, am bitten occasionally by the -625 out of memory bug. I'm seeing this on a system that has ~150,000 files. But, my backup server is not very powerful. I'm going to try running the backup with a larger page file.

 

But I'd like to investigate breaking up this one large volume into subvolumes. I've read the manual, and several other postings, but I don't see how to do this. What I want is to have a backup solution that will automatically notice and backup anything new. How do I use subvolumes in this environment? For instance, this one large volume has ~10 subdirectories. If I create a separate subvolume for each directory (which, in itself, is cumbersome), then what happens if the user creates another, new directory at the top level? The backups wouldn't notice it and back it up.

 

I'm thinking what I need to do is to create a subvolume for, say, one large subdirectory, but have another volume for everything else. But I can't figure out how to include some subdirectories in one subvolume.

 

Thoughts on this?

 

Thanks

tl

 

p.s. Are there plans to raise the maximum number of files per volume? As other postings have said 1-2 million is not really a lot of files these days.

Share this post


Link to post
Share on other sites

Hi

 

What OS is your backup machine running? If it is NT4 then you will run into memory limitations much sooner than with windows 2000/XP.

 

You are correct- if you are backing up all of the folders on the root of the disk as subvolumes, Retrospect will not notice that another folder has been created at the root. You would have to specify it manually in order for it to be seen.

 

You can't exclude subdirectories from a volume/ subvolume scan either which sort of leaves you stuck.

 

Honestly though you should not be getting these errors untill you hit a million files at least (with NT). I'd take a look at the system running Retrospect first.

 

Nate

 

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×