Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Thu, 12 May 2011 15:46:47 +0000 (UTC)
From: "Michael McCandless (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: 
 <186943239.7047.1305215207897.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1040365464.3003.1305121667325.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (LUCENE-3087) highlighting exact phrase with
 overlapping tokens fails.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/LUCENE-3087?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D130=
32468#comment-13032468 ]=20

Michael McCandless commented on LUCENE-3087:
--------------------------------------------

Ahh, interesting.

So... maybe we should fix the code so that if in fact positions were includ=
ed in the TVs, we use them?  Else, we fallback to the offset check to guess=
 at the posIncr?  Could that work?

> highlighting exact phrase with overlapping tokens fails.
> --------------------------------------------------------
>
>                 Key: LUCENE-3087
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3087
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: contrib/highlighter
>    Affects Versions: 2.9.4, 3.1
>            Reporter: Pierre Goss=C3=A9
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 3.2, 4.0
>
>         Attachments: LUCENE-3087.patch
>
>
> Fields with overlapping token are not highlighted in search results when =
searching exact phrases, when using TermVector.WITH_OFFSET.
> The document builded in MemoryIndex for highlight does not preserve posit=
ions of tokens in this case. Overlapping tokens get "flattened" (position i=
ncrement always set to 1), the spanquery used for searching relevant fragme=
nt will fail to identify the correct token sequence because the position sh=
ift.
> I corrected this by adding a position increment calculation in sub class =
StoredTokenStream. I added junit test covering this case.
> I used the eclipse codestyle from trunk, but style add quite a few format=
 differences between repository and working copy files. I tried to reduce t=
hem, but some linewrapping rules still doesn't match.
> Correction patch joined

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org