lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Created: (LUCENE-1566) Large Lucene index can hit false OOM due to Sun JRE issue
Date Tue, 17 Mar 2009 12:10:50 GMT
Large Lucene index can hit false OOM due to Sun JRE issue
---------------------------------------------------------

                 Key: LUCENE-1566
                 URL: https://issues.apache.org/jira/browse/LUCENE-1566
             Project: Lucene - Java
          Issue Type: Bug
          Components: Index
    Affects Versions: 2.4.1
            Reporter: Michael McCandless
            Priority: Minor


This is not a Lucene issue, but I want to open this so future google
diggers can more easily find it.

There's this nasty bug in Sun's JRE:

  http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6478546

The gist seems to be, if you try to read a large (eg 200 MB) number of
bytes during a single RandomAccessFile.read call, you can incorrectly
hit OOM.  Lucene does this, with norms, since we read in one byte per
doc per field with norms, as a contiguous array of length maxDoc().

The workaround was a custom patch to do large file reads as several
smaller reads.

Background here:

  http://www.nabble.com/problems-with-large-Lucene-index-td22347854.html


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message