lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Smith <ssm...@mainstreamdata.com>
Subject Highlighting and InvalidTokenOffsetsException in Lucene 4.0
Date Thu, 01 Nov 2012 00:36:40 GMT
I'm migrating code from Lucene 3.5 to 4.0.  I have the following code which is supposed to
highlight text.  I get the exception InvalidTokenOffsetsException.  I have no idea what that
means.  I am using a custom analyzer which seems to work for searching/indexing, so I assume
it will work here (even though it took a couple of "minor" changes to get it to compile in
4.0  This code used to work in 3.5.

Anyone have any ideas?

Scott

Code fragment:

        try
        {
            ctf = new CachingTokenFilter(myCustomAnalyzer
                    .tokenStream(MyFieldName, new StringReader(myText)));
        }
        catch (IOException e1)
        {
            s_oLog.error("Search:markCommon: Exception creating CachingTokenFilter: " +
                    e1.getMessage());
            return null;

        }
        String markedString;
        SimpleHTMLFormatter formatter;
        try
        {
            formatter = new SimpleHTMLFormatter(_zBeginHighlight,
                    _zEndHighlight);
            Scorer score = new QueryScorer(q);
            ht = new Highlighter(formatter, score);
            ht.setTextFragmenter(new NullFragmenter());
            markedString = ht.getBestFragment(ctf, myText);
        }
        catch (IOException e)
        {
            s_oLog.error("Search:markCommon: Unable to highlight string: "
                    + e.getMessage());
            return null;
        }
        catch(InvalidTokenOffsetsException e2)
        {
            s_oLog.error("Search:markCommon: Unable to highlight string2: "
                    + e2.getMessage());
            return null;
        }


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message