lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <>
Subject RE: TokenStream API, Quick Question.
Date Thu, 03 Sep 2009 19:04:04 GMT
The indexer only call getAttribute/addAttribute one time after initializing
(see docs). It will never call it later. If you cache tokens, you always
have to restore the state into the TokenStream's attributes.

Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen

> -----Original Message-----
> From: Daniel Shane [mailto:shaned@LEXUM.UMontreal.CA]
> Sent: Thursday, September 03, 2009 8:55 PM
> To:
> Subject: TokenStream API, Quick Question.
> Does a TokenStream have to return always the same number of attributes
> with the same underlying classes for all the tokens it generates?
> I mean, during the tokenization phase, can the first "token" have a Term
> and Offset Attribute and the second "token" only a Type Attribute or
> does this mean that the first token has to have an empty Type attribute
> as well?
> I'm just not sure,
> Daniel Shane
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message