Return-Path: Mailing-List: contact lucene-dev-help@jakarta.apache.org; run by ezmlm Delivered-To: mailing list lucene-dev@jakarta.apache.org Received: (qmail 25220 invoked from network); 28 Sep 2003 06:51:21 -0000 Received: from unknown (HELO hotmail.com) (64.4.46.194) by daedalus.apache.org with SMTP; 28 Sep 2003 06:51:21 -0000 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Sat, 27 Sep 2003 23:51:33 -0700 Received: from 61.51.200.186 by bay9-dav59.bay9.hotmail.com with DAV; Sun, 28 Sep 2003 06:51:33 +0000 X-Originating-IP: [61.51.200.186] X-Originating-Email: [chedong@hotmail.com] From: "Che Dong" To: "Lucene Developers List" References: <60AB0074-F13D-11D7-863D-000393A564E6@ehatchersolutions.com> Subject: Re: StandardTokenizer CJK Support Date: Sun, 28 Sep 2003 14:51:09 +0800 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0092_01C385CF.F4478C40" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Message-ID: X-OriginalArrivalTime: 28 Sep 2003 06:51:33.0965 (UTC) FILETIME=[F473C7D0:01C3858C] X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N ------=_NextPart_000_0092_01C385CF.F4478C40 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: base64 QXR0YWNoZWQgd2l0aCBDSksgc2lncmFtIHN1cHBvcnQ6DQoNCg0KQ2hlLCBEb25nDQotLS0tLSBP cmlnaW5hbCBNZXNzYWdlIC0tLS0tIA0KRnJvbTogIkVyaWsgSGF0Y2hlciIgPGVyaWtAZWhhdGNo ZXJzb2x1dGlvbnMuY29tPg0KVG86ICJMdWNlbmUgRGV2ZWxvcGVycyBMaXN0IiA8bHVjZW5lLWRl dkBqYWthcnRhLmFwYWNoZS5vcmc+DQpTZW50OiBTdW5kYXksIFNlcHRlbWJlciAyOCwgMjAwMyA2 OjUzIEFNDQpTdWJqZWN0OiBSZTogU3RhbmRhcmRUb2tlbml6ZXIgQ0pLIFN1cHBvcnQNCg0KDQo+ IElmIERvdWcgb3Igc29tZW9uZSBlbHNlIGdpdmVzIGEgZnJlc2ggKzEgdG8gdGhpcyBJJ2xsIGFw cGx5IGl0IGlmIHRoZXkgDQo+IGRvbid0IGZpcnN0LiAgSSBuZWVkIHBhdGNoZXMgYXMgZmlsZSBh dHRhY2htZW50cyBzbyB0aGV5IGRvbid0IGdldCANCj4gbWFuZ2xlZCBpbiBlLW1haWwgZm9ybWF0 dGluZyB0aG91Z2ggLSBiZXN0IHRocm91Z2ggQnVnemlsbGEgc28gaXQgDQo+IGRvZXNuJ3QgZ2V0 IGxvc3QgaW4gdGhlIHNodWZmbGUgYWdhaW4gdG9vLg0KPiANCj4gRXJpaw0KPiANCj4gT24gU2F0 dXJkYXksIFNlcHRlbWJlciAyNywgMjAwMywgYXQgMDE6NTcgIFBNLCBDaGUgRG9uZyB3cm90ZToN Cj4gDQo+ID4gSSB0aGluayBhdCBsZWFzdCB0aGUgc2lncmFtIGJhc2UgdG9rZW4gY291bGQgYmUg c3VwcG9ydGVkIGJ5IA0KPiA+IFN0YW5kYXJkVG9rZW5pemVyLg0KPiA+IEkgYWxzbyB0cnkgdG8g aW1wbGVtZW50IENKS1Rva2VuaXplciB2aWEgU3RhbmRhcmRUb2tlbml6ZXIod2l0aCBzaWdyYW0g DQo+ID4gc3VwcG9ydCkgKyBCaWdyYW1GaWx0ZXIuDQo+ID4NCj4gPiBIZXJlIGlzIG15IENKSyBz aWdyYW0gcGF0Y2ggZm9yIFN0YW5kYXJkVG9rZW5pemVyOg0KPiA+IDU3LDU5YzU3LDU5DQo+ID4g PCAvL0lHTk9SRV9DQVNFID0gdHJ1ZTsNCj4gPiA8IC8vQlVJTERfUEFSU0VSID0gZmFsc2U7DQo+ ID4gPCAvL1VOSUNPREVfSU5QVVQgPSB0cnVlOw0KPiA+IC0tLQ0KPiA+PiAgICAgLy9JR05PUkVf Q0FTRSA9IHRydWU7DQo+ID4+ICAgICAvL0JVSUxEX1BBUlNFUiA9IGZhbHNlOw0KPiA+PiAgICAg VU5JQ09ERV9JTlBVVCA9IHRydWU7DQo+ID4gNjJjNjINCj4gPiA8IC8vREVCVUdfVE9LRU5fTUFO QUdFUiA9IHRydWU7DQo+ID4gLS0tDQo+ID4+ICAgICAvL0RFQlVHX1RPS0VOX01BTkFHRVIgPSB0 cnVlOw0KPiA+IDkyYzkyDQo+ID4gPCAgIDxBTFBIQU5VTTogKDxMRVRURVI+fDxESUdJVD4pKyA+ DQo+ID4gLS0tDQo+ID4+IDxBTFBIQU5VTTogKDxMRVRURVI+fDxESUdJVD4pKyA+DQo+ID4gMTIw YTEyMQ0KPiA+PiB8IDxTSUdSQU06ICg8Q0pLPikgPg0KPiA+IDEyOWMxMzANCj4gPiA8IHwgPCAj TEVUVEVSOiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC8vIHVuaWNvZGUgbGV0 dGVycw0KPiA+IC0tLQ0KPiA+PiB8IDwgI0xFVFRFUjogICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAvLyBhbHBoYWJldHMNCj4gPiAxMzZjMTM3LDE0MQ0KPiA+IDwgICAgICAgICJc dTAxMDAiLSJcdTFmZmYiLA0KPiA+IC0tLQ0KPiA+PiAgICAgICAgICJcdTAxMDAiLSJcdTFmZmYi DQo+ID4+ICAgICBdDQo+ID4+Pg0KPiA+PiB8ICA8ICNDSks6ICAgICAgIC8vIG5vbi1hbHBoYWJl dHMNCj4gPj4gICAgICAgWw0KPiA+IDE2NmMxNzENCj4gPiA8ICA8Tk9JU0U6IH5bXSA+DQo+ID4g LS0tDQo+ID4+IDxOT0lTRTogfltdID4NCj4gPiAxODRhMTkwDQo+ID4+ICAgICAgICAgdG9rZW4g PSA8U0lHUkFNPiB8DQo+ID4NCj4gPg0KPiA+IFJlZ2FyZHMNCj4gPg0KPiA+IENoZSwgRG9uZw0K PiA+DQo+ID4gLS0tLS0gT3JpZ2luYWwgTWVzc2FnZSAtLS0tLQ0KPiA+IEZyb206ICJFcmlrIEhh dGNoZXIiIDxlcmlrQGVoYXRjaGVyc29sdXRpb25zLmNvbT4NCj4gPiBUbzogIkx1Y2VuZSBEZXZl bG9wZXJzIExpc3QiIDxsdWNlbmUtZGV2QGpha2FydGEuYXBhY2hlLm9yZz4NCj4gPiBTZW50OiBT YXR1cmRheSwgU2VwdGVtYmVyIDI3LCAyMDAzIDg6MzggUE0NCj4gPiBTdWJqZWN0OiBSZTogU3Rh bmRhcmRUb2tlbml6ZXIgQ0pLIFN1cHBvcnQNCj4gPg0KPiA+DQo+ID4+IENvdWxkIHlvdSBhZGQg dGhlIHBhdGNoIHRvIGEgQnVnemlsbGEgaXNzdWUgZm9yIGVhc2llciBhY2Nlc3M/ICAgSQ0KPiA+ PiBkb24ndCBtaW5kIGFwcGx5aW5nIGl0IGlmIGl0IGhhcyBEb3VnJ3MgKzENCj4gPj4NCj4gPj4g RXJpaw0KPiA+Pg0KPiA+Pg0KPiA+PiBPbiBGcmlkYXksIFNlcHRlbWJlciAyNiwgMjAwMywgYXQg MTA6MzkgIEFNLCBkYW5yYXBwQGNvbWNhc3QubmV0IA0KPiA+PiB3cm90ZToNCj4gPj4NCj4gPj4+ IEluIEF1Z3VzdCBvZiAyMDAyLCBDaGUsIERvbmcgc3VnZ2VzdGVkIGEgY2hhbmdlIHRvDQo+ID4+ PiBTdGFuZGFyZFRva2VuaXplci5qaiB0aGF0DQo+ID4+PiB3b3VsZCBzdXBwbHkgc29tZSBiYXNp YyBzdXBwb3J0IGZvciBDSksuIChtc2dObzoyMTY0KSBBIGRheSBsYXRlciwNCj4gPj4+IERvdWcg Z2F2ZSBpdA0KPiA+Pj4gKzEuIFRoZSBzdWdnZXN0ZWQgY2hhbmdlIHdhcyBub3QgYWRkZWQgdG8g Q1ZTIG5vciB3YXMgdGhlcmUgYW55IA0KPiA+Pj4gZnVydGhlcg0KPiA+Pj4gZGlzY3Vzc2lvbiBv biB0aGUgbWFpbGluZyBsaXN0Lg0KPiA+Pj4NCj4gPj4+IEknbSB3b3JraW5nIHdpdGggYW4gYXBw bGljYXRpb24gaW4gd2hpY2ggY2VydGFpbiBmaWVsZHMgYXJlIG1peGVkDQo+ID4+PiBsYW5ndWFn ZSBhbmQNCj4gPj4+IHRoaXMgY2hhbmdlIGlzIHZlcnkgdXNlZnVsLiBJcyB0aGVyZSBhIHRlY2hu aWNhbCByZWFzb24gd2h5IHRoaXMNCj4gPj4+IGNoYW5nZSB3YXMgbm90DQo+ID4+PiBtYWRlPw0K PiA+Pj4NCj4gPj4+IFJlZ2FyZHMsDQo+ID4+Pg0KPiA+Pj4gLS1EYW4gUmFwcA0KPiA+Pj4NCj4g Pj4+IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLQ0KPiA+Pj4gVG8gdW5zdWJzY3JpYmUsIGUtbWFpbDogbHVjZW5lLWRl di11bnN1YnNjcmliZUBqYWthcnRhLmFwYWNoZS5vcmcNCj4gPj4+IEZvciBhZGRpdGlvbmFsIGNv bW1hbmRzLCBlLW1haWw6IGx1Y2VuZS1kZXYtaGVscEBqYWthcnRhLmFwYWNoZS5vcmcNCj4gPj4N Cj4gPj4NCj4gPj4gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQo+ID4+IFRvIHVuc3Vic2NyaWJlLCBlLW1haWw6IGx1 Y2VuZS1kZXYtdW5zdWJzY3JpYmVAamFrYXJ0YS5hcGFjaGUub3JnDQo+ID4+IEZvciBhZGRpdGlv bmFsIGNvbW1hbmRzLCBlLW1haWw6IGx1Y2VuZS1kZXYtaGVscEBqYWthcnRhLmFwYWNoZS5vcmcN Cj4gPj4NCj4gDQo+IA0KPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4gVG8gdW5zdWJzY3JpYmUsIGUtbWFpbDog bHVjZW5lLWRldi11bnN1YnNjcmliZUBqYWthcnRhLmFwYWNoZS5vcmcNCj4gRm9yIGFkZGl0aW9u YWwgY29tbWFuZHMsIGUtbWFpbDogbHVjZW5lLWRldi1oZWxwQGpha2FydGEuYXBhY2hlLm9yZw0K PiANCj4g ------=_NextPart_000_0092_01C385CF.F4478C40--