lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andy Liu (JIRA)" <>
Subject [jira] Commented: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries
Date Thu, 13 Sep 2007 15:13:33 GMT


Andy Liu commented on LUCENE-794:

Ah, I wasn't crazy.  I had the test data wrong.  Here's the code I'm using to produce the
failing result:

        String text = "y z x y z a b";

        Analyzer analyzer = new StandardAnalyzer();
        QueryParser parser = new QueryParser("body", analyzer);
        Query query = parser.parse("\"x y z\"");
        CachingTokenFilter tokenStream = new CachingTokenFilter(analyzer.tokenStream("body",
new StringReader(text)));
        Highlighter highlighter = new Highlighter(new SpanScorer(query, "body", tokenStream));
        highlighter.setTextFragmenter(new NullFragmenter());

        String result = highlighter.getBestFragments(tokenStream, text, 1, "...");

This produces:

<B>y</B> <B>z</B> <B>x</B> <B>y</B> <B>z</B>
a b

The beginning y and z shouldn't be highlighted.

If I change the the beginning y and z to x and y, I get the correct result:

"x y x y z a b" => x y <B>x</B> <B>y</B> <B>z</B> a

Here's a couple other failing results:

"z x y z a b" => <B>z</B> <B>x</B> <B>y</B> <B>z</B>
a b
"z a x y z a b" => <B>z</B> a <B>x</B> <B>y</B> <B>z</B>
a b

FYI, I'm using the latest version of Lucene.

> Extend contrib Highlighter to properly support phrase queries and span queries
> ------------------------------------------------------------------------------
>                 Key: LUCENE-794
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Other
>            Reporter: Mark Miller
>            Priority: Minor
>         Attachments:,,,,,,,,,,,,,,,,,,,,, spanhighlighter.patch,
spanhighlighter10.patch, spanhighlighter2.patch, spanhighlighter3.patch, spanhighlighter5.patch,
spanhighlighter6.patch, spanhighlighter7.patch, spanhighlighter8.patch, spanhighlighter9.patch,,,,,,
> This patch adds a new Scorer class (SpanQueryScorer) to the Highlighter package that
scores just like QueryScorer, but scores a 0 for Terms that did not cause the Query hit. This
gives 'actual' hit highlighting for the range of SpanQuerys and PhraseQuery. There is also
a new Fragmenter that attempts to fragment without breaking up Spans.
> See for some background.
> There is a dependency on MemoryIndex.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message