lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DM Smith <dmsmith...@gmail.com>
Subject Re: Token implementation
Date Tue, 20 May 2008 09:12:04 GMT

On May 20, 2008, at 12:50 AM, Hiroaki Kawai wrote:

>
>
> "Michael McCandless" <lucene@mikemccandless.com> wrote:

....
>> More responses below:
>> DM Smith <dmsmith555@gmail.com> wrote:


>
>
>>> But, in TokenFilter, next() should be deprecated, IMHO.
>>
>> I think this is a good idea.  After all if people don't want to  
>> bother
>> using the passed in Token, they are still allowed to return a new
>> one.
>
> I could not see what you meant. Can I ask you to let me know the  
> reason
> why it should be deprecated?

The purpose of the char[] rather than a String is to promote reuse of  
a mutable buffer. Reusing a Token minimizes the number of  
constructions. Each TokenFilter has both
Token next()
and
Token next(Token)

If next() is implemented then he TokenFilter does not have access to a  
shared Token. While next(Token) supplies a shared token by the caller.

The proper way to use next(Token) is as follows

t = input.next(t);

next(Token)
can be implemented as
Token next(Token result) {
   Token t = new Token();
   ...
   return t;
}

This would be the trivial equivalent to next(). Just ignore the  
argument and do as before.

-- DM

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message