lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Janit Anjaria (Tech-IT)" <anjaria.ja...@flipkart.com>
Subject Re: [ISSUE] After restoring data to a Solrcloud instance
Date Wed, 02 Mar 2016 13:06:01 GMT
Hi,

PFA the results of the CheckIndex on the snapshot and the restored
snapshot. Regarding the case of deleting the snapshot index and again
attempting to restore - the problem persists and it gets corrupted again.

Regards,
Janit

On Wed, Mar 2, 2016 at 4:59 PM, Varun Thacker <varunthacker1989@gmail.com>
wrote:

> Could you post the full output of the CheckIndex command on the restored
> snapshot? Also what happens if you delete the snapshot indexes and attempt
> to restore again? Does it get corrupted again or is it a one off scenario?
>
> On Wed, Mar 2, 2016 at 3:44 PM, Janit Anjaria (Tech-IT) <
> anjaria.janit@flipkart.com> wrote:
>
>> Hi,
>>
>> Varun, we actually ran the test for our restored data snapshot and it
>> threw an error saying "Broken segment".
>>
>> How is it possible that the same test gives success on the snapshot, but
>> not on the restored snapshot? Can you please throw some light on this, so
>> we can proceed and fix this issue.
>>
>> Regards,
>> Janit
>>
>> On Tue, Mar 1, 2016 at 12:05 PM, Varun Thacker <
>> varunthacker1989@gmail.com> wrote:
>>
>>> Hi Janit,
>>>
>>> Please ask these questions the solr-user mailing list. There is no point
>>> in double posting to the dev list as well.
>>>
>>> Looking at the stacktrace it looks like the index files are corrupted.
>>> On the backed up index can you run the CheckIndex
>>> command to see what it says. The command would look something like this
>>> - :
>>> java -cp SOLR_HOME/example/webapps/WEB-INF/lib/lucene-core-[version].jar
>>> -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex
>>> /path/to/backup/data/index
>>>
>>> If the CheckIndex command says its corrupted then there is something
>>> wrong with the backup. If the CheckIndex
>>>  says the index is fine then there might be something wrong with the
>>> restore process in which case we'll need to dig deeper
>>>
>>> On Tue, Mar 1, 2016 at 11:21 AM, Janit Anjaria (Tech-IT) <
>>> anjaria.janit@flipkart.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please find below the console logs for the above issue we have been
>>>> facing:
>>>>
>>>> org.apache.solr.common.SolrException: Error opening new searcher
>>>> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:824)
>>>> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:665)
>>>> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:746)
>>>> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:466)
>>>> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:457)
>>>> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:232)
>>>> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>> 	at java.lang.Thread.run(Thread.java:744)
>>>> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
>>>> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1667)
>>>> 	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1778)
>>>> 	at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:920)
>>>> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:797)
>>>> 	... 9 more
>>>> Caused by: org.apache.lucene.index.CorruptIndexException: codec footer mismatch
(file truncated?): actual footer=438025 vs expected footer=-1071082520 (resource=MMapIndexInput(path="/var/solr/data/fk-ekl-mmi-mapsearch_shard1_replica1/data/restore.snapshot.20160204065051901/_1tdyo.cfs"))
>>>> 	at org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:416)
>>>> 	at org.apache.lucene.codecs.CodecUtil.retrieveChecksum(CodecUtil.java:401)
>>>> 	at org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.<init>(Lucene50CompoundReader.java:79)
>>>> 	at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:71)
>>>> 	at org.apache.lucene.index.IndexWriter.readFieldInfos(IndexWriter.java:1016)
>>>> 	at org.apache.lucene.index.IndexWriter.getFieldNumberMap(IndexWriter.java:1033)
>>>> 	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
>>>> 	at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:79)
>>>> 	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:66)
>>>> 	at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:235)
>>>> 	at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:109)
>>>> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1636)
>>>> 	... 12 more
>>>>
>>>> 3/1/2016, 11:19:44 AM ERROR null CoreContainer Error waiting for
>>>> SolrCore to be created
>>>>
>>>> java.util.concurrent.ExecutionException: org.apache.solr.common.SolrException:
Unable to create core [fk-ekl-mmi-mapsearch_shard1_replica1]
>>>> 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>>>> 	at java.util.concurrent.FutureTask.get(FutureTask.java:188)
>>>> 	at org.apache.solr.core.CoreContainer$2.run(CoreContainer.java:495)
>>>> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:232)
>>>> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>> 	at java.lang.Thread.run(Thread.java:744)
>>>> Caused by: org.apache.solr.common.SolrException: Unable to create core [fk-ekl-mmi-mapsearch_shard1_replica1]
>>>> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:760)
>>>> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:466)
>>>> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:457)
>>>> 	... 5 more
>>>> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
>>>> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:824)
>>>> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:665)
>>>> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:746)
>>>> 	... 7 more
>>>> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
>>>> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1667)
>>>> 	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1778)
>>>> 	at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:920)
>>>> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:797)
>>>> 	... 9 more
>>>> Caused by: org.apache.lucene.index.CorruptIndexException: codec footer mismatch
(file truncated?): actual footer=438025 vs expected footer=-1071082520 (resource=MMapIndexInput(path="/var/solr/data/fk-ekl-mmi-mapsearch_shard1_replica1/data/restore.snapshot.20160204065051901/_1tdyo.cfs"))
>>>> 	at org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:416)
>>>> 	at org.apache.lucene.codecs.CodecUtil.retrieveChecksum(CodecUtil.java:401)
>>>> 	at org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.<init>(Lucene50CompoundReader.java:79)
>>>> 	at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:71)
>>>> 	at org.apache.lucene.index.IndexWriter.readFieldInfos(IndexWriter.java:1016)
>>>> 	at org.apache.lucene.index.IndexWriter.getFieldNumberMap(IndexWriter.java:1033)
>>>> 	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
>>>> 	at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:79)
>>>> 	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:66)
>>>> 	at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:235)
>>>> 	at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:109)
>>>> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1636)
>>>> 	... 12 more
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Janit
>>>>
>>>>
>>>> On Tue, Mar 1, 2016 at 11:02 AM, Janit Anjaria (Tech-IT) <
>>>> anjaria.janit@flipkart.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> We have been facing the problem of corrupted solr index, after
>>>>> restoring our data. When we restore the data, the status shows 'success'
>>>>> after some time. But later, when we make some config changes and then
try
>>>>> to stop the instance and restart it with the new configurations - the
index
>>>>> is corrupted.
>>>>>
>>>>> Can someone throw light on the reason that might lead to this problem,
>>>>> or is there something we are missing in the restore process ? Some help
>>>>> would be greatly appreciated as this is a high priority task for us.
>>>>>
>>>>> Hope to receive a positive and prompt reply.
>>>>>
>>>>> Regards,
>>>>> Janit
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>>
>>> Regards,
>>> Varun Thacker
>>>
>>
>>
>
>
> --
>
>
> Regards,
> Varun Thacker
>

Mime
View raw message