lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Beady Geraghty <beadygerag...@gmail.com>
Subject standardTokenizer - how to terminate at End of Stream
Date Wed, 21 Sep 2005 15:02:28 GMT
Could someone tell me how to use the StandardTokenizer properly ?
 I thought that if the tokenizer.getNextToken() returns null, then it is
the end of stream. I have a loop that tries to get the next token until
it is null. But the loop doesn't terminate.
I tried to termintae the loop by t.kind == 0, and it seems to have stopped
upon the end of stream. I am not sure what t.kind really is.
The code mentioned that it is defined in Constants.java, and I looked
that up, but it is apparent not the right file. Maybe I am pointing to
a wrong directory.


StandardTokenizer tokenizer = new StandardTokenizer( r ); // r is a reader
int count = 0;
Token t = tokenizer .getNextToken();

while (t != null) {
count++;
//if (t.kind == 0)
// break;
System.out.println( t );
t = tokenizer.getNextToken() ;
System.out.println(count);
}
System.out.println( "done");
 Thank you for any input.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message