lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 32847] - IndexWriter.addIndexes results in java.lang.OutOfMemoryError
Date Mon, 31 Jan 2005 17:59:55 GMT
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=32847>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=32847





------- Additional Comments From cutting@apache.org  2005-01-31 18:59 -------
Are the 100 fields all indexed, or are some only stored?  If indexed, 100 is a
very large number of indexed fields.  7M documents with 100 indexed fields could
require 700MB when searching, since one byte per searched field per document of
RAM is used to cache the norms for each field.  But that RAM is not required
when indexing.  Merging an index, where you're having troubles, should not
require much RAM.

Since your problem only requires a 5-line program to demonstrate, and it only
requires the Lucene jar file, please create such a 5-line java program as a
single file that depends only on the Lucene jar and demonstrate the problem with
something like:

  javac -classpath lucene.jar Test.java
  java -classpath lucene.jar Test index1 index2 index3 ...

Test.java should look something like:

import org.apache.lucene.index.IndexWriter;
public class Test {
  public static void main(String[] paths) throws Exception {
    Directory[] sources = new Directory[paths.length];
    for (int i = 0; i < paths.length) {
      sources[i] = FSDirectory.getDirectory(paths[i], false);
    }
    IndexWriter writer = 
      new IndexWriter("dest", new StandardAnalyzer(), true);   
    writer.addIndexes(sources);
    writer.close();
  }
}

Once you have replicated the bug with such a program, please attach the program
to this bug report.  This way we can be certain that there is nothing involved
but Lucene.

Note that this code does not explicitly try to catch exceptions but rather lets
the JVM print a final stack trace if it exits in an exception.  That may work
better.

If the problem still appears and we still don't get a stack trace then we can
try putting in log statements in SegmentMerger.java.

Thanks for your patience.

-- 
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message