lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andy Hind (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-415) Merge error during add to index (IndexOutOfBoundsException)
Date Tue, 21 Mar 2006 19:05:01 GMT
    [ http://issues.apache.org/jira/browse/LUCENE-415?page=comments#action_12371284 ] 

Andy Hind commented on LUCENE-415:
----------------------------------

We have tested the above solution pretty heavily since 18/11/2005 and would regard it as stable
in 1.4.3.

Looking at the 1.9 code stream the issue is likely to be present, unless there is some other
code that checks if an index segment file already exists when not expected, or the next segement
is generated based in the segments that actually exist in the directory.

In 1.4.3 ..... in FSDirectory

......
final class FSOutputStream extends OutputStream
{
    RandomAccessFile file = null;

    public FSOutputStream(File path) throws IOException
    {
        file = new RandomAccessFile(path, "rw");
        file.setLength(0);
        file.getChannel();
    }
......

will sort this issue and some other file handle issues I have seen under XP

Something similar is likely to be required in FSIndexOutput  in the 1.9 code line



> Merge error during add to index (IndexOutOfBoundsException)
> -----------------------------------------------------------
>
>          Key: LUCENE-415
>          URL: http://issues.apache.org/jira/browse/LUCENE-415
>      Project: Lucene - Java
>         Type: Bug
>   Components: Index
>     Versions: 1.4
>  Environment: Operating System: Linux
> Platform: Other
>     Reporter: Daniel Quaroni
>     Assignee: Lucene Developers

>
> I've been batch-building indexes, and I've build a couple hundred indexes with 
> a total of around 150 million records.  This only happened once, so it's 
> probably impossible to reproduce, but anyway... I was building an index with 
> around 9.6 million records, and towards the end I got this:
> java.lang.IndexOutOfBoundsException: Index: 54, Size: 24
>         at java.util.ArrayList.RangeCheck(ArrayList.java:547)
>         at java.util.ArrayList.get(ArrayList.java:322)
>         at org.apache.lucene.index.FieldInfos.fieldInfo(FieldInfos.java:155)
>         at org.apache.lucene.index.FieldInfos.fieldName(FieldInfos.java:151)
>         at org.apache.lucene.index.SegmentTermEnum.readTerm(SegmentTermEnum.java
> :149)
>         at org.apache.lucene.index.SegmentTermEnum.next
> (SegmentTermEnum.java:115)
>         at org.apache.lucene.index.SegmentMergeInfo.next
> (SegmentMergeInfo.java:52)
>         at org.apache.lucene.index.SegmentMerger.mergeTermInfos
> (SegmentMerger.java:294)
>         at org.apache.lucene.index.SegmentMerger.mergeTerms
> (SegmentMerger.java:254)
>         at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:93)
>         at org.apache.lucene.index.IndexWriter.mergeSegments
> (IndexWriter.java:487)
>         at org.apache.lucene.index.IndexWriter.maybeMergeSegments
> (IndexWriter.java:458)
>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:310)
>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:294)

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message