lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: ArrayIndexOutOfBoundsException: -65536 during full-import from old index
Date Fri, 01 Sep 2017 13:00:31 GMT
Is it possible that exception is thrown when trying to index an extremely
large document?

Mike McCandless

http://blog.mikemccandless.com

On Fri, Sep 1, 2017 at 12:07 AM, bebe1437 <edc0421@gmail.com> wrote:

> My solr version is 5.5.4,
> I set docValues="true" to some old fields,
> and I use dataimport to reindex,
> but it keep throwing the exception: Caused by:
> java.lang.ArrayIndexOutOfBoundsException: -65536
>         In org. apache. Lucene. index. TermsHashPerField.
> writeByte(TermsHashPerField.java:197)
>
> I checked the issue: https://issues.apache.org/jira/browse/LUCENE-1995
> and the other article:
> http://lucene.472066.n3.nabble.com/ArrayIndexOutOfBoundsException
> -65536-td3661945.html
>
> I got 1M docs but it's only cost around 3-400mb, I keep the ramBufferSizeMB
> as default. I'm not sure what's the problem, because if update those old
> documents with API, it's working, but it fails with dataimport.
>
> and here is the exception:
>
> org.apache.solr.common.SolrException: Exception writing document id
> uw9G000007301O_T00000Ox01t_E000005L09u_E000005L09u to the index; possible
> analysis error.
>         at
> org.apache.solr.update.DirectUpdateHandler2.addDoc(
> DirectUpdateHandler2.java:180)
>         at
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(
> RunUpdateProcessorFactory.java:68)
>         at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(
> UpdateRequestProcessor.java:48)
>         at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(
> DistributedUpdateProcessor.java:934)
>         at
> org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(
> DistributedUpdateProcessor.java:1089)
>         at
> org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(
> DistributedUpdateProcessor.java:712)
>         at
> org.apache.solr.update.processor.LogUpdateProcessorFactory$
> LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
>         at org.apache.solr.handler.dataimport.SolrWriter.upload(
> SolrWriter.java:74)
>         at
> org.apache.solr.handler.dataimport.DataImportHandler$
> 1.upload(DataImportHandler.java:260)
>         at
> org.apache.solr.handler.dataimport.DocBuilder.
> buildDocument(DocBuilder.java:524)
>         at
> org.apache.solr.handler.dataimport.DocBuilder.
> buildDocument(DocBuilder.java:414)
>         at
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:
> 329)
>         at
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
>         at
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.
> java:416)
>         at
> org.apache.solr.handler.dataimport.DataImporter.
> runCmd(DataImporter.java:480)
>         at
> org.apache.solr.handler.dataimport.DataImporter$1.run(
> DataImporter.java:461)
> Caused by: org.apache.lucene.store.AlreadyClosedException: this
> IndexWriter
> is closed
>         at org.apache.lucene.index.IndexWriter.ensureOpen(
> IndexWriter.java:720)
>         at org.apache.lucene.index.IndexWriter.ensureOpen(
> IndexWriter.java:734)
>         at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
>         at
> org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(
> DirectUpdateHandler2.java:282)
>         at
> org.apache.solr.update.DirectUpdateHandler2.addDoc0(
> DirectUpdateHandler2.java:214)
>         at
> org.apache.solr.update.DirectUpdateHandler2.addDoc(
> DirectUpdateHandler2.java:169)
>         ... 15 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: -65536
>         In org. apache. Lucene. index. TermsHashPerField.
> writeByte(TermsHashPerField.java:197)
>         at
> org.apache.lucene.index.TermsHashPerField.writeVInt(
> TermsHashPerField.java:223)
>         at
> org.apache.lucene.index.FreqProxTermsWriterPerField.writeProx(
> FreqProxTermsWriterPerField.java:82)
>         at
> org.apache.lucene.index.FreqProxTermsWriterPerField.newTerm(
> FreqProxTermsWriterPerField.java:122)
>         at
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:177)
>         at
> org.apache.lucene.index.DefaultIndexingChain$PerField.
> invert(DefaultIndexingChain.java:682)
>         at
> org.apache.lucene.index.DefaultIndexingChain.processField(
> DefaultIndexingChain.java:365)
>         at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(
> DefaultIndexingChain.java:321)
>         at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(
> DocumentsWriterPerThread.java:234)
>         at
> org.apache.lucene.index.DocumentsWriter.updateDocument(
> DocumentsWriter.java:450)
>         at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477)
>         ... 18 more
>
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Lucene-Java-Users-
> f532864.html
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message