lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@apache.org>
Subject Re: TokenStream and Token APIs
Date Wed, 15 Oct 2008 14:41:39 GMT

On Oct 11, 2008, at 7:33 PM, Michael Busch wrote:
> If a consumer is e. g. only interested in the positionIncrement and  
> Payload, it can consume the tokens like this:
>
>  public class Consumer {
>    public void consumeTokens(TokenStream ts) throws IOException {
>      Token token = ts.prototypeToken();
>
>      PayloadAttribute payloadSource = (PayloadAttribute) token;
>      PositionIncrementAttribute positionSource =
>                    (PositionIncrementAttribute) token;
>
>      while (ts.nextToken()) {
>        char[] term = token.termBuffer();
>        int termLength = token.termLength();
>        int positionIncrement = positionSource.getPositionIncrement();
>        Payload payload = payloadSource.getPayload();
>
>        // do something with the term, positionIncrement and payload
>      }
>    }
>  }
>

Bear with me, b/c I'm not sure I'm following, but looking at https://issues.apache.org/jira/browse/LUCENE-1422

, I see at least 5 different implemented Attributes.

So, let's say I add a 5 more attributes and now have a total of 10  
attributes.  Are you saying that I then would have, potentially, 10  
different variables that all point to the token as in the code snippet  
above where the casting takes place?  Or would I just create a single  
"Super" attribute that folds in all of my new attributes, plus any  
other existing ones?  Or, maybe, what I would do is create the 5 new  
attributes and then 1 new attribute that extends all 10, thus allowing  
me to use them individually, but saving me from having to do a whole  
ton of casting in my Consumer.

I like where this is going, but am still a bit fuzzy on how it's going  
to work in practice.

-Grant

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message