lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SOLR-6776) Data lost when use SoftCommit and TLog
Date Tue, 02 Dec 2014 16:38:13 GMT

     [ https://issues.apache.org/jira/browse/SOLR-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yonik Seeley reassigned SOLR-6776:
----------------------------------

    Assignee: Yonik Seeley

> Data lost when use SoftCommit and TLog
> --------------------------------------
>
>                 Key: SOLR-6776
>                 URL: https://issues.apache.org/jira/browse/SOLR-6776
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 4.10
>            Reporter: yuanyun.cn
>            Assignee: Yonik Seeley
>              Labels: softCommit, updateLog
>             Fix For: 4.10.3
>
>
> We enabled update log and change autoCommit to some bigger value 10 mins.
> After restart, we push one doc with softCommit=true
> http://localhost:8983/solr/update?stream.body=<add><doc><field name="id">id1</field></doc></add>&softCommit=true
> Then we kill the java process after a min. 
> After restart, Tlog failed to replay with following exception, and there is no data in
solr.
> 6245 [coreLoadExecutor-5-thread-1] ERROR org.apache.solr.update.UpdateLog  รป Failure
to open existing log file (non fatal) E:\jeffery\src\apache\solr\4.10.2\solr-4.10.2\example\solr\collection1\data\t
> log\tlog.0000000000000000000:org.apache.solr.common.SolrException: java.io.EOFException
>         at org.apache.solr.update.TransactionLog.<init>(TransactionLog.java:181)
>         at org.apache.solr.update.UpdateLog.init(UpdateLog.java:261)
>         at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:134)
>         at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:94)
>         at org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:100)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
>         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
>         at java.lang.reflect.Constructor.newInstance(Unknown Source)
>         at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:550)
>         at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:620)
>         at org.apache.solr.core.SolrCore.<init>(SolrCore.java:835)
>         at org.apache.solr.core.SolrCore.<init>(SolrCore.java:646)
>         at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
>         at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
>         at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
>         at java.util.concurrent.FutureTask.run(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.EOFException
>         at org.apache.solr.common.util.FastInputStream.readUnsignedByte(FastInputStream.java:73)
>         at org.apache.solr.common.util.FastInputStream.readInt(FastInputStream.java:216)
>         at org.apache.solr.update.TransactionLog.readHeader(TransactionLog.java:268)
>         at org.apache.solr.update.TransactionLog.<init>(TransactionLog.java:159)
>         ... 19 more
> Check the code: seems this is related with: org.apache.solr.update.processor.RunUpdateProcessor,
in processCommit, it sets changesSinceCommit=false(even we are using softCommit)
> So in finish, updateLog.finish will not be called.
>   public void finish() throws IOException {
>     if (changesSinceCommit && updateHandler.getUpdateLog() != null) {
>       updateHandler.getUpdateLog().finish(null);
>     }
>     super.finish();
>   }
> To fix this issue: I have to change RunUpdateProcessor.processCommit like below:
>     if (!cmd.softCommit) {
>       changesSinceCommit = false;
>     }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message