Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of wunder@chegg.com designates
 216.32.180.30 as permitted sender)
Received-SPF: softfail (mail199-va3: transitioning domain of chegg.com does
 not designate 157.56.244.213 as permitted sender) client-ip=157.56.244.213;
 envelope-from=wunder@chegg.com;
 helo=CH1PRD0510HT005.namprd05.prod.outlook.com ;.outlook.com ;
From: Walter Underwood <wunder@chegg.com>
To: "solr-user@lucene.apache.org" <solr-user@lucene.apache.org>
Subject: EdgeNgramTokenFilter and positions
Thread-Topic: EdgeNgramTokenFilter and positions
Thread-Index: AQHNi48GyEvmX3/DyEm85QZMmoBPCA==
Date: Wed, 5 Sep 2012 17:51:06 +0000
Message-ID: <222D872F-4B04-48AE-A044-9FB70851E70F@chegg.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: multipart/alternative;
	boundary="_000_222D872F4B0448AEA0449FB70851E70Fcheggcom_"
MIME-Version: 1.0

--_000_222D872F4B0448AEA0449FB70851E70Fcheggcom_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

In the analysis page, the n-grams produced by EdgeNgramTokenFilter are at s=
equential positions. This seems wrong, because an n-gram is associated with=
 a source token at a specific position. It also really messes up phrase mat=
ches.

With the source text "fleen", these positions and tokens are generated:

1,fl
2,fle
3,flee
4,fleen

Is this a known bug? Fixed? I'm running 3.3.

wunder
--
Walter Underwood
Search Guy
wunder@chegg.com<mailto:wunder@chegg.com>


--_000_222D872F4B0448AEA0449FB70851E70Fcheggcom_--