lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: TokenStream API, Quick Question.
Date Thu, 03 Sep 2009 19:04:04 GMT
The indexer only call getAttribute/addAttribute one time after initializing
(see docs). It will never call it later. If you cache tokens, you always
have to restore the state into the TokenStream's attributes.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Daniel Shane [mailto:shaned@LEXUM.UMontreal.CA]
> Sent: Thursday, September 03, 2009 8:55 PM
> To: java-user@lucene.apache.org
> Subject: TokenStream API, Quick Question.
> 
> Does a TokenStream have to return always the same number of attributes
> with the same underlying classes for all the tokens it generates?
> 
> I mean, during the tokenization phase, can the first "token" have a Term
> and Offset Attribute and the second "token" only a Type Attribute or
> does this mean that the first token has to have an empty Type attribute
> as well?
> 
> I'm just not sure,
> Daniel Shane
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message