lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: "read past EOF" when merge
Date Sat, 03 Nov 2012 13:15:25 GMT
Can you file a JIRA Markus? This is probably related to the new code that uses Directory for
replication.

- Mark

On Nov 2, 2012, at 6:53 AM, Markus Jelsma <markus.jelsma@openindex.io> wrote:

> Hi,
> 
> For what it's worth, we have seen similar issues with Lucene/Solr from this week's trunk.
The issue manifests itself when it want to replicate. The servers have not been taken offline
and did not crash when this happenend. 
> 
> 2012-10-30 16:12:51,061 WARN [solr.handler.ReplicationHandler] - [http-8080-exec
> -3] - : Exception while writing response for params: file=_p_Lucene41_0.doc&comm
> and=filecontent&checksum=true&generation=6&qt=/replication&wt=filestream
> java.io.EOFException: read past EOF: MMapIndexInput(path="/opt/solr/cores/openindex_h/data/index.20121030152234973/_p_Lucene41_0.doc")
>        at org.apache.lucene.store.ByteBufferIndexInput.readBytes(ByteBufferIndexInput.java:100)
>        at org.apache.solr.handler.ReplicationHandler$DirectoryFileStream.write(ReplicationHandler.java:1065)
>        at org.apache.solr.handler.ReplicationHandler$3.write(ReplicationHandler.java:932)
> 
> 
> Markus
> 
> -----Original message-----
>> From:Michael McCandless <lucene@mikemccandless.com>
>> Sent: Fri 02-Nov-2012 11:46
>> To: java-user@lucene.apache.org
>> Subject: Re: &quot;read past EOF&quot; when merge
>> 
>> Are you able to reproduce the corruption?
>> 
>> If at any time you accidentally had two writers open on the same
>> index, it could have created this corruption.
>> 
>> Writing to an index over NFS ought to be OK, however, it's not well
>> tested.  You should use SimpleFSLockFactory (not the default
>> NativeFSLockFactory).
>> 
>> The more "typical" way people use NFS is to write to an index on a
>> local disk, and then other machines read from that index using NFS.
>> 
>> In any event performance is usually much worse than using local disks ...
>> 
>> Mike McCandless
>> 
>> http://blog.mikemccandless.com
>> 
>> On Thu, Nov 1, 2012 at 10:32 PM, superruiye <superruiye@gmail.com> wrote:
>>> oh ,thx,I don't know CheckIndex before...and I use to fix my error index,it
>>> is OK...
>>> I use NFS to share my index,and no change to the LogFactory.
>>> How could I avoid this problem,and not only fix after it was broken
>>> suddenly?
>>> 
>>> 
>>> 
>>> --
>>> View this message in context: http://lucene.472066.n3.nabble.com/read-past-EOF-when-merge-tp4017179p4017734.html
>>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>> 
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message