lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Highlighter package updated with overlapping token support
Date Mon, 26 Jul 2004 20:58:29 GMT
I have updated the Highlighter code in CVS to support tokenizers that generate overlapping

The Junit test rig has a new example test that uses a "SynonymTokenizer" which generates multiple
in the same position for the same input token eg (the token "football" is expanded into tokens
"soccer","footie" and "football"). 
The Formatter interface had to be changed to take a new "TokenGroup" object instead of a single
token but
I doubt any code changes in clients are required because most people use the default Formatter
implementation and haven't
created their own  implementations.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message