lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Smith <>
Subject Highlighting and InvalidTokenOffsetsException in Lucene 4.0
Date Thu, 01 Nov 2012 00:36:40 GMT
I'm migrating code from Lucene 3.5 to 4.0.  I have the following code which is supposed to
highlight text.  I get the exception InvalidTokenOffsetsException.  I have no idea what that
means.  I am using a custom analyzer which seems to work for searching/indexing, so I assume
it will work here (even though it took a couple of "minor" changes to get it to compile in
4.0  This code used to work in 3.5.

Anyone have any ideas?


Code fragment:

            ctf = new CachingTokenFilter(myCustomAnalyzer
                    .tokenStream(MyFieldName, new StringReader(myText)));
        catch (IOException e1)
            s_oLog.error("Search:markCommon: Exception creating CachingTokenFilter: " +
            return null;

        String markedString;
        SimpleHTMLFormatter formatter;
            formatter = new SimpleHTMLFormatter(_zBeginHighlight,
            Scorer score = new QueryScorer(q);
            ht = new Highlighter(formatter, score);
            ht.setTextFragmenter(new NullFragmenter());
            markedString = ht.getBestFragment(ctf, myText);
        catch (IOException e)
            s_oLog.error("Search:markCommon: Unable to highlight string: "
                    + e.getMessage());
            return null;
        catch(InvalidTokenOffsetsException e2)
            s_oLog.error("Search:markCommon: Unable to highlight string2: "
                    + e2.getMessage());
            return null;

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message