lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Koji Sekiguchi" <koji.sekigu...@m4.dion.ne.jp>
Subject RE: Highlighter apply to Japanese
Date Tue, 06 Sep 2005 11:18:33 GMT
Hi Mark,

With the change, the problem was completely solved!

Sample: (JapaneseAnalyzer)
Text: AMeetingWillBeHeldInTheCityHall

TokenStream:
[A][Meeting][Will][Be][Held][In][The][City][Hall]

Query Text: Meeting
Output: A<B>Meeting</B>WillBeHeldInTheCityHall

Query Text: CityHall
Output: AMeetingWillBeHeldInThe<B>City</B><B>Hall</B>

Although if I use CJKAnalyzer, which is producing a stream
of tokens which all overlap, the problem sill occurs,
but with JapaneseAnalyzer, the highlighter works fine. 

Thank you very much,

Koji


> -----Original Message-----
> From: mark harwood [mailto:markharw00d@yahoo.co.uk] 
> Sent: Tuesday, September 06, 2005 7:22 PM
> To: java-user@lucene.apache.org
> Subject: RE: Highlighter apply to Japanese
> 
> 
> Try change TokenGroup.isDistinct();
> 
> Maybe the offset test code should be >= rather than >
> ie
> 
> 	boolean isDistinct(Token token)
> 	{
> 		return token.startOffset()>=endOffset;
> 	}
> 
> I've just tried the change with the Junit test and all
> seems well still with the non CJK tests.
> 
> 
> 
> 		
> ___________________________________________________________ 
> To help you stay safe and secure online, we've developed the 
> all new Yahoo! Security Centre. http://uk.security.yahoo.com
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message