Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
From: Michael McCandless <lucene@mikemccandless.com>
Date: Mon, 9 Apr 2012 15:44:35 -0400
Message-ID: 
 <CAL8PwkZgOvXM5AGfUDDxrJbQYV+2Yv=cWmWwv+8asx_sGP8TQQ@mail.gmail.com>
Subject: Re: svn commit: r1311373 - in /lucene/dev/branches/lucene3969:
 lucene/test-framework/src/java/org/apache/lucene/analysis/
 modules/analysis/common/src/java/org/apache/lucene/analysis/shingle/
 modules/analysis/common/src/test/org/apache/lucene/analysis
To: dev@lucene.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Mon, Apr 9, 2012 at 3:41 PM, Steven A Rowe <sarowe@syr.edu> wrote:
> On 4/9/2012 at 3:06 PM, mikemccand@apache.org wrote:
>> LUCENE-3969: [...] tenatively add posLen to ShingleFilter
>> [...]
>> +++ lucene/dev/branches/lucene3969/modules/analysis/common/src/java/org/
>> +++ apache/lucene/analysis/shingle/ShingleFilter.java Mon Apr =A09 =A019=
:05:47 2012
>> [...]
>> @@ -319,6 +321,8 @@ public final class ShingleFilter extends
>> =A0 =A0 =A0 =A0 =A0 =A0noShingleOutput =3D false;
>> =A0 =A0 =A0 =A0 =A0}
>> =A0 =A0 =A0 =A0 =A0offsetAtt.setOffset(offsetAtt.startOffset(), nextToke=
n.offsetAtt.endOffset());
>> + =A0 =A0 =A0 =A0// nocommit is this right!? =A0i'm just guessing...
>> + =A0 =A0 =A0 =A0posLenAtt.setPositionLength(builtGramSize);
>> =A0 =A0 =A0 =A0 =A0isOutputHere =3D true;
>> =A0 =A0 =A0 =A0 =A0gramSize.advance();
>> =A0 =A0 =A0 =A0 =A0tokenAvailable =3D true;
>
> +1 - looks right to me.
>
> builtGramSize is the position length of the output shingle - missing posi=
tions (e.g. from stop words) are represented as "filler" tokens.

OK thanks Steve!  This fixed the one failure I was working on at the time :=
)

I'll remove the nocommit...

Mike McCandless

http://blog.mikemccandless.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org