lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Taylor <paul_t...@fastmail.fm>
Subject Re: I need to implement a TokenFilter to break season07
Date Tue, 15 Dec 2009 12:25:29 GMT
Uwe Schindler wrote:
> And if you do it yourself, don't forget to call clearAttributes() whenever
> you produce new tokens (else you may have bugs in the token increments). In
> the old token api its Token.clear()... Just a warning!

This comment has worried me, is this ok or am i meant to call 
clearAttributes() somewhere


public class StripLeadingZeroFilter extends TokenFilter {
    /**
     * Construct filtering <i>in</i>.
     */
    public StripLeadingZeroFilter(TokenStream in) {
        super(in);
        termAtt = (TermAttribute) addAttribute(TermAttribute.class);
    }

    private TermAttribute termAtt;

    /**
     *
     * <p>Removes zeroes if first char in token
     */
    public final boolean incrementToken() throws java.io.IOException {
        if (!input.incrementToken()) {
            return false;
        }

        char[] buffer = termAtt.termBuffer();
        final int bufferLength = termAtt.termLength();

        if (buffer[0] == '0') {
            for (int i = 1; i < bufferLength; i++) {
                char c = buffer[i];
                buffer[i - 1] = c;
            }
            termAtt.setTermLength(bufferLength - 1);
            return true;
        } else {
            return true;
        }
    }

}


thanks Paul

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message