lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1409) read past EOF
Date Wed, 01 Oct 2008 17:05:44 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636098#action_12636098
] 

Michael McCandless commented on LUCENE-1409:
--------------------------------------------


I would still like to get to the root cause here, in case it still exists in 2.4.

Adam, around how many added objects (using your test posted in the above forum) do you see
the error happen.  I've run several times so far and haven't gotten the error to happen.

Also, the test seems to slow down drastically over time.  At first it indexes at ~ 10 objects/
sec; after 4000 object this slows down to ~ 3 objects/sec.  Do you know why that is?  Lucene
by itself doesn't exhibit this slowdown (well, it does a little, as merges become necessary,
but it's much less of a slowdown).  It seems like compass is periodically waking up and optimizing
the index, fairly frequently.  That could explain this slowdown.

> read past EOF
> -------------
>
>                 Key: LUCENE-1409
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1409
>             Project: Lucene - Java
>          Issue Type: Bug
>    Affects Versions: 2.3.2
>         Environment: jdk 1.5.0_08
>            Reporter: Adam Łączyński
>
> I create index with a lot of documents (~500 000). During add documents to "read past
EOF" error occured. It occure after random number of indexed documents. I used lucene with
compass framework but I think that is not important. It is a link to compass forum where that
problem was reporeted http://forum.compass-project.org/thread.jspa?threadID=215641&tstart=0
> java.io.IOException: read past EOF
> 	at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:146)
> 	at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
> 	at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:76)
> 	at org.apache.lucene.index.FieldInfos.read(FieldInfos.java:304)
> 	at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:59)
> 	at org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:298)
> 	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:262)
> 	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:197)
> 	at org.apache.lucene.index.MultiSegmentReader.<init>(MultiSegmentReader.java:109)
> 	at org.apache.lucene.index.MultiSegmentReader.doReopen(MultiSegmentReader.java:203)
> 	at org.apache.lucene.index.DirectoryIndexReader$2.doBody(DirectoryIndexReader.java:98)
> 	at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:636)
> 	at org.apache.lucene.index.DirectoryIndexReader.reopen(DirectoryIndexReader.java:92)
> 	at org.compass.core.lucene.engine.manager.DefaultLuceneSearchEngineIndexManager.internalRefreshCache(DefaultLuceneSearchEngineIndexManager.java:368)
> 	at org.compass.core.lucene.engine.manager.DefaultLuceneSearchEngineIndexManager.refreshCache(DefaultLuceneSearchEngineIndexManager.java:358)
> 	at org.compass.core.lucene.engine.transaction.readcommitted.ReadCommittedTransaction$CommitCallable.call(ReadCommittedTransaction.java:422)
> 	at org.compass.core.transaction.context.TransactionalCallable$1.doInTransaction(TransactionalCallable.java:44)
> 	at org.compass.core.impl.DefaultCompass$CompassTransactionContext.execute(DefaultCompass.java:342)
> 	at org.compass.core.transaction.context.TransactionalCallable.call(TransactionalCallable.java:41)
> 	at org.compass.core.executor.DefaultExecutorManager.invokeAllWithLimit(DefaultExecutorManager.java:104)
> 	at org.compass.core.executor.DefaultExecutorManager.invokeAllWithLimitBailOnException(DefaultExecutorManager.java:73)
> 	at org.compass.core.lucene.engine.transaction.readcommitted.ReadCommittedTransaction.doCommit(ReadCommittedTransaction.java:142)
> 	at org.compass.core.lucene.engine.transaction.AbstractTransaction.commit(AbstractTransaction.java:98)
> 	at org.compass.core.lucene.engine.LuceneSearchEngine.commit(LuceneSearchEngine.java:172)
> 	at org.compass.core.transaction.LocalTransaction.doCommit(LocalTransaction.java:97)
> 	at org.compass.core.transaction.AbstractTransaction.commit(AbstractTransaction.java:46)
> 	at org.compass.core.CompassTemplate.execute(CompassTemplate.java:131)
> 	at org.compass.core.CompassTemplate.execute(CompassTemplate.java:112)
> 	at asl.simplesearch.compass.CompassService.createCall(Unknown Source)
> 	at asl.util.IndexCreator.createIndex(Unknown Source)
> 	at asl.util.IndexCreator.start(Unknown Source)
> 	at asl.util.IndexCreatorTestCase.main(IndexCreatorTestCase.java:20)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message