lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrien Grand <jpou...@gmail.com>
Subject Re: merge problems
Date Mon, 10 Oct 2016 14:32:05 GMT
It looks like the field infos of your index went out of sync with data
stored in the files about points.

Can you run CheckIndex on your index (potentially with the `-fast` option
so that it only verifies checksums)? It could be that one of these two
parts of the index got corrupted.

Since you were able to modify the way add(IndexableField) is implemented,
I'm wondering if you are running a fork of Lucene? If yes, maybe you did
some changes that triggered this bug?

Otherwise is your application:
 - using IndexWriter.addIndexes?
 - customizing merging in some way, eg. by wrapping the merge readers?

Le mar. 4 oct. 2016 à 16:40, Hans Lund <ha.lund@gmail.com> a écrit :

> After upgrading to 6.2 we are having problems during merges (after running
> for a while).
>
> When the problem occurs its always complaining about the same field - and
> throws:
>
> java.lang.IllegalArgumentException: field="id" did not index point values
>     at
>
> org.apache.lucene.codecs.lucene60.Lucene60PointsReader.getBKDReader(Lucene60PointsReader.java:126)
>     at
>
> org.apache.lucene.codecs.lucene60.Lucene60PointsReader.size(Lucene60PointsReader.java:224)
>     at
>
> org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:169)
>     at
> org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:173)
>     at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:122)
>     at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4312)
>     at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3889)
>
>
> To figure out where we messed up - I have added some ugly logging to
> Document:
>
> public final void add(IndexableField field) {
>         if ("id".equals(field.name()) &&
>                 field.fieldType().pointDimensionCount()
>                         != 0) {
>             System.err.println("Point value detected");
>             for (IndexableField i : fields) {
>                 System.err.println(i);
>             }
>         }
>         fields.add(field);
>   }
>
> In hope to intercept the document we messed up.
>
> But to my surprise toString on the suspected field just says (contains a
> URN):
>
> indexed,omitNorms,indexOptions=DOCS<id:urn:wiki:doc:YEL:57028#1-1>
>
> So any hints as to why field.fieldType().pointDimensionCount() != 0
>
> and any suggestions what might cause this?
>
> Regards
> Hans Lund
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message