lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ralf Heyde" <ralf.he...@gmx.de>
Subject Solr adding Documents / Commit in different Threads
Date Tue, 14 Aug 2012 11:45:36 GMT
Hello,

we currently facing a problem which may lost updates for some documents during adding / comitting.


The infrastructure: we have a main solr, which gets documents and distribute them to a lot
of slaves. 

The situation:  we have a Job, which runs scheduled every minute (no run, if a previous job
is not ended), looks for new / modified documents and adds, deletes and finally commits them
to the Main Solr Server. They are added due SolrServer.add(Collection<SolrInputDocument>).
This Collection contains up to 10.000 documents - so the request could take some time.

Additionally we have a second job, which only runs commits every minute.

Today i found a stacktrace in our logs. Can someone tell me, whether this is a problem? Is
it possible that these two processes conflict each other (e.g. lost document updates).

Regards,

Ralf

---------- Stracktrace -----------

2012-08-14 10:05:50,141  WARN ... org.apache.solr.handler.ReplicationHandler$FileStream.write(ReplicationHandler.java:1066):
Exception while writing response for params: checksum=true&wt=filestream&indexversion=1330453083759&file=_eah.frq&command=filecontent
ClientAbortException:  java.io.IOException
        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:373)
        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:356)
        at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:396)
        at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:385)
        at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:93)
        at org.apache.solr.common.util.FastOutputStream.write(FastOutputStream.java:87)
        at org.apache.solr.handler.ReplicationHandler$FileStream.write(ReplicationHandler.java:1054)
        at org.apache.solr.handler.ReplicationHandler$3.write(ReplicationHandler.java:914)
        at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:336)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:265)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
        at de.zalando.solr.shared.experimentalfeature.ExperimentalFeatureFilter.doFilter(ExperimentalFeatureFilter.java:28)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:405)
        at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:257)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:515)
        at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1764)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException
        at org.apache.coyote.http11.InternalAprOutputBuffer.flushBuffer(InternalAprOutputBuffer.java:208)
        at org.apache.coyote.http11.InternalAprOutputBuffer$SocketOutputBuffer.doWrite(InternalAprOutputBuffer.java:238)
        at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:119)
        at org.apache.coyote.http11.AbstractOutputBuffer.doWrite(AbstractOutputBuffer.java:190)
        at org.apache.coyote.Response.doWrite(Response.java:533)
        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:368)
        ... 27 more





---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message