lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stephane vaucher <vauc...@LUB.UMontreal.CA>
Subject Re: Should Token be immutable?
Date Mon, 06 Jan 2003 22:26:58 GMT
I've noticed that there is a method public void setPositionIncrement(int 
positionIncrement) that would probably have to disappear for Token to be 
immutable. The CHANGES.txt doc seems to mention some good reasons why it 
was added, but there is no code in CVS that seems to depend on it.

 13. Added new method Token.setPositionIncrement().

     This permits, for the purpose of phrase searching, placing
     multiple terms in a single position.  This is useful with
     stemmers that produce multiple possible stems for a word.

     This also permits the introduction of gaps between terms, so that
     terms which are adjacent in a token stream will not be matched by
     and exact phrase query.  This makes it possible, e.g., to build
     an analyzer where phrases are not matched over stop words which
     have been removed.

     Finally, repeating a token with an increment of zero can also be
     used to boost scores of matches on that token.  (cutting)

Any comments? With an immutable Token, does the positionIncrement still 
have a reason for being there? If not, then I'll remove 
getPositionIncrement as well.


Doug Cutting wrote:

> stephane vaucher wrote:
>> 1) Does anyone mind? Will it break anything?
> It shouldn't break anything.
>> 2) Are there units tests for this? (particularly PorterStemFilter). 
>> The changes are obviously not spectacular, but I prefer not to screw 
>> everyone up...
> I don't know of any unit tests specifically for this.  Mostly this 
> change will affect compilation.  In general though, if you don't see 
> unit tests for things that you think you might break, then it never 
> hurts to write more unit tests.
>> 3) I've checked-out the latest version of lucene, is there anything 
>> special I need to do if I get the go ahead to check my stuff in (like 
>> a dev list review)?
> If you're not a regular committer then please send diffs to lucene-dev 
> before comitting and give folks a few days to consider the changes.
> Doug
> -- 
> To unsubscribe, e-mail:   
> <>
> For additional commands, e-mail: 
> <>

To unsubscribe, e-mail:   <>
For additional commands, e-mail: <>

View raw message