lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 30421] - Merging indexes leads to out-of-memory condition
Date Sat, 31 Jul 2004 16:23:43 GMT
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG 
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=30421>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND 
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=30421

Merging indexes leads to out-of-memory condition





------- Additional Comments From ddreon1@yahoo.com  2004-07-31 16:23 -------
>>> You: You said 'any size'.  Are you saying that if you have 2 indices with 10
documents each, and you try to merge them, you will run out of memory?

>>> Me: should read an indexes "of size." I didn't use the words small, medium, 
or large because this is relative. The merge operation fails on what I consider 
to be small indexes. Yes, 2 x 10 works fine. But 20 x 5GB (read small) fails 
every time. I just had one fail again this morning.

>>> You: Have you tried giving your JVM more memory with -Xmx (e.g. java 
-Xmx777m)?

>>> Me: Yes my message states that I have worked with larger VMs than this. 
Suffice it to say, any developer trying to allocate ever increasing blocks of 
memory to perform a merge is playing a losing game. Without investigating the 
internals, one can speculate that memory-based merge operations need to be 
shifted to disk.

>>> You: Can you provide a working, self-enclosed code that shows this? I 
included a code sample with my post.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message