lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From markharw00d <markharw...@yahoo.co.uk>
Subject Re: Highlighter apply to Japanese
Date Tue, 06 Sep 2005 06:37:18 GMT
I don't know the behaviour of the Japanese Analyzer you are using.
Can you add to your example diagnosis the Token.getPositionIncrement, 
Token.startOffset and Token.endOffset for each of the tokens?

The highlighter groups tokens with overlapping start and end offsets 
into a single TokenGroup for the purposes of highlighting. This allows 
TokenStreams which produce multiple synonyms for the same source token 
to work. This behaviour was also required to get the CJKAnalyzer to 
work. It could be that the Analyzer you are using is producing a stream 
of tokens which *all* overlap?

Cheers
Mark


		
___________________________________________________________ 
To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre.
http://uk.security.yahoo.com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message