lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: may be lucene bug
Date Thu, 28 Dec 2017 21:31:05 GMT
I think there's a bug in your code: this line:

     doc.doc <= leaf.docBase + leaf.reader().maxDoc())

should be < not <=.

Mike McCandless

http://blog.mikemccandless.com

On Thu, Dec 28, 2017 at 6:15 AM, 291699763 <291699763@qq.com> wrote:

> Lucene version:6.6.0
>
> when Index
> document.add(new BinaryDocValuesField("CBID.CCID", new
> BytesRef(myValue)));
>
> and when search
>
>
> int totalHits = indexSearcher.count(SpanNearQuery);
>         int from = 0;
>         int size = 10000;
>         int pageTime = 0;
>         int loadTime = 0;
>         Set<String> fieldsToLoad = new HashSet<>();
>         fieldsToLoad.add("CBID.CCID");
>         List<LeafReaderContext> leaves = indexSearcher.getIndexReader()
> .leaves();
>         while (from < totalHits) {
>             if (from > 0) {
>                 //翻页
>                 Stopwatch started = Stopwatch.createStarted();
>                 TopDocs search = indexSearcher.search(content, from);
>                 ScoreDoc scoreDoc = search.scoreDocs[search.scoreDocs.length
> - 1];
>                 TopDocs topDocs = indexSearcher.searchAfter(scoreDoc,
> content, size);
>                 pageTime += started.elapsed(TimeUnit.MILLISECONDS);
>
>                 started = Stopwatch.createStarted();
>                 ScoreDoc[] scoreDocs = topDocs.scoreDocs;
>                 for (ScoreDoc doc : scoreDocs) {
>                     for (LeafReaderContext leaf : leaves) {
>                         BinaryDocValues binary = DocValues.getBinary(leaf.reader(),
> "CBID.CCID");
>                         if (doc.doc >= leaf.docBase && doc.doc <=
> leaf.docBase + leaf.reader().maxDoc()) {
>                             BytesRef bytesRef = binary.get(doc.doc -
> leaf.docBase);
>                             keyValue.add(bytesRef.utf8ToString());
>                         }
>                     }
>                 }
>                 loadTime += started.elapsed(TimeUnit.MILLISECONDS);
>             } else {
>                 //不翻页
>                 Stopwatch started = Stopwatch.createStarted();
>                 TopDocs search = indexSearcher.search(content, size);
>                 pageTime += started.elapsed(TimeUnit.MILLISECONDS);
>                 started = Stopwatch.createStarted();
>                 ScoreDoc[] scoreDocs = search.scoreDocs;
>                 for (ScoreDoc doc : scoreDocs) {
>                     for (LeafReaderContext leaf : leaves) {
>                         BinaryDocValues binary = DocValues.getBinary(leaf.reader(),
> "CBID.CCID");
>                         if (doc.doc >= leaf.docBase && doc.doc <=
> leaf.docBase + leaf.reader().maxDoc()) {
>                             BytesRef bytesRef = binary.get(doc.doc -
> leaf.docBase);
>                             keyValue.add(bytesRef.utf8ToString());
>                         }
>                     }
>                 }
>                 loadTime += started.elapsed(TimeUnit.MILLISECONDS);
>             }
>             from += size;
>         }
>
>
> but throw exception
> Exception in thread "main" java.lang.RuntimeException:
> java.io.EOFException: read past EOF: MMapIndexInput(path="/data/
> home/p_wxuwang/index_withDocValues/_26.cfs") [slice=_26_Lucene54_0.dvd]
> [slice=var-binary]
>         at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$6.
> get(Lucene54DocValuesProducer.java:740)
>         at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$
> LongBinaryDocValues.get(Lucene54DocValuesProducer.java:1197)
>         at com.yuewen.nrzx.keyword.Main2.getWithDocValues(Main2.java:111)
>         at com.yuewen.nrzx.keyword.Main2.main(Main2.java:187)
> Caused by: java.io.EOFException: read past EOF: MMapIndexInput(path="/data/
> home/p_wxuwang/index_withDocValues/_26.cfs") [slice=_26_Lucene54_0.dvd]
> [slice=var-binary]
>         at org.apache.lucene.store.ByteBufferIndexInput.readBytes(
> ByteBufferIndexInput.java:98)
>         at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$6.
> get(Lucene54DocValuesProducer.java:736)
>
>
> I don't know why??
>
>
>
>
>
>
>
> 王旭   技术部/数据支持
> 18302118258|291699763
> 上海市浦东新区碧波路690号6号楼(201203)
> www.yuewen.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message