lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@lucene.com>
Subject Re: [contrib]: StandardTokenizer with sigram based CJK Support
Date Tue, 27 Aug 2002 16:26:11 GMT
+1

Che Dong wrote:
>>Attached  StandardTokenizer.jj with Sigram Based east
>>asia language support:
>>tested under Windows and GNU/Linux
>>
>>Just treat different UnicodeBlock with different word
>>segment method. 
>>
>>Hope in the future released we can add more language
>>support in StandardTokenizer.jj step by step and keep
>>it fit for most i18n environment.
>>Some common app, like Jive, can use it as default
>>Analyser.
>>Use localized Analyzier for advanced usage.
>>
>>Thank you.
>>
>>Che, Dong



--
To unsubscribe, e-mail:   <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>


Mime
View raw message