lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kyle L. (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (LUCENE-2553) IOException: read past EOF
Date Mon, 02 Aug 2010 14:55:17 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kyle L. resolved LUCENE-2553.
-----------------------------

    Resolution: Not A Problem

Custom {{Collector}} was using the re-based id to load the document from the {{SegmentReader}}.
When I switched over to using the raw id, the issue has not resurfaced. It would be nice to
have a more descriptive error message though.

> IOException: read past EOF
> --------------------------
>
>                 Key: LUCENE-2553
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2553
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Search
>    Affects Versions: 3.0.2
>            Reporter: Kyle L.
>
> We have been getting an {{IOException}} with the following stack trace:
> \\
> \\
> {noformat}
> java.io.IOException: read past EOF
> 	at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:154)
> 	at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
> 	at org.apache.lucene.store.IndexInput.readInt(IndexInput.java:69)
> 	at org.apache.lucene.store.IndexInput.readLong(IndexInput.java:92)
> 	at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:218)
> 	at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:901)
> 	at com.cargurus.search.IndexManager$AllHitsUnsortedCollector.collect(IndexManager.java:520)
> 	at org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:275)
> 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:212)
> 	at org.apache.lucene.search.Searcher.search(Searcher.java:67)
>         ...
> {noformat}
> \\
> \\
> We have implemented a basic custom collector that collects all hits in an unordered manner:
> {code}
>     private class AllHitsUnsortedCollector extends Collector {
>         private Log logger = LogFactory.getLog(AllHitsUnsortedCollector.class); 
>         private IndexReader reader;
>         private int baselineDocumentId;
>         private List<Document> matchingDocuments = new ArrayList<Document>();
>         
>         @Override
>         public boolean acceptsDocsOutOfOrder() {
>             return true;
>         }
>         @Override
>         public void collect(int docId) throws IOException {
>             int documentId = baselineDocumentId + docId;
>             Document document = reader.document(documentId, getFieldSelector());
>             
>             if (document == null) {
>                 logger.info("Null document from search results!");
>             } else {
>                 matchingDocuments.add(document);
>             }
>         }
>         @Override
>         public void setNextReader(IndexReader segmentReader, int baseDocId) throws IOException
{
>             this.reader = segmentReader;
>             this.baselineDocumentId = baseDocId;
>         }
>         @Override
>         public void setScorer(Scorer scorer) throws IOException {
>             // do nothing
>         }
>         public List<Document> getMatchingDocuments() {
>             return matchingDocuments;
>         }
>     }
> {code}
> The exception arises when users perform searches while indexing/optimization is occurring.
Our {{IndexReader}} is read-only. From the documentation I have read, a read-only {{IndexReader}}
instance should be immune from any uncommitted index changes and should return consistent
results during indexing and optimization. As this exception occurs during indexing/optimization,
it seems to me that the read-only {{IndexReader}} is somehow stumbling upon the uncommitted
content? 
> The problem is difficult to replicate as it is sporadic in nature and so far has only
occurred in Production.
> We have rebuilt the indexes a number of times, but that does not seem to alleviate the
issue.
> Any other information I can provide that will help isolate the issue? 
> The most likely other possibility is that the {{Collector}} we have written is doing
something it shouldn't. Any pointers?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message