lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From markharw...@yahoo.co.uk
Subject Highlighter package updated with overlapping token support
Date Mon, 26 Jul 2004 20:58:29 GMT
I have updated the Highlighter code in CVS to support tokenizers that generate overlapping
tokens.

The Junit test rig has a new example test that uses a "SynonymTokenizer" which generates multiple
tokens 
in the same position for the same input token eg (the token "football" is expanded into tokens
"soccer","footie" and "football"). 
The Formatter interface had to be changed to take a new "TokenGroup" object instead of a single
token but
I doubt any code changes in clients are required because most people use the default Formatter
implementation and haven't
created their own  implementations.

Cheers
Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message