lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Shane <sha...@LEXUM.UMontreal.CA>
Subject TokenStream API, Quick Question.
Date Thu, 03 Sep 2009 18:55:00 GMT
Does a TokenStream have to return always the same number of attributes 
with the same underlying classes for all the tokens it generates?

I mean, during the tokenization phase, can the first "token" have a Term 
and Offset Attribute and the second "token" only a Type Attribute or 
does this mean that the first token has to have an empty Type attribute 
as well?

I'm just not sure,
Daniel Shane

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message