Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of fancyerii@gmail.com designates
 209.85.213.47 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CACGP497FYQquOrVCQhVyWSu+spsQuMCVC7y23VyZ+7UrU=svoQ@mail.gmail.com>
References: 
 <CACGP4962GOTw2-X4E4BWFf6NckPuwaDv+mHB-nf5QEqSMpFWKA@mail.gmail.com>
	<CAFAd71UD0=ec0U0Gj29xoVttswnTqebiL3HW9pGe+3RWPBsa8A@mail.gmail.com>
	<CACGP497FYQquOrVCQhVyWSu+spsQuMCVC7y23VyZ+7UrU=svoQ@mail.gmail.com>
Date: Thu, 28 Jun 2012 14:43:45 +0800
Message-ID: 
 <CAFAd71XmsqcvQrfoQjaHqAJBz-Z5tZ_tEbnEwUy5hGqwRdKzPA@mail.gmail.com>
Subject: Re: Question about chinese and WildcardQuery
From: Li Li <fancyerii@gmail.com>
To: java-user@lucene.apache.org
Content-Type: text/plain; charset=GB2312
Content-Transfer-Encoding: quoted-printable

in Chinese, there isn't word boundary between words. it writes like:
Iamok. you should tokenize it to I am ok
if you want to search *amo*, you should view I am ok as one token. In
Chinese, fuzzy search is not very useful. even use Standard Analyzer,
it's ok to use boolean query. because "Iamok" is tokenized as I a m o
k. if search boolean query +a +m +o, it's fine. Chinese has many
letters(commonly used more than 3000). and words are very short(most
words has only 2 letters).


On Thu, Jun 28, 2012 at 2:31 PM, Paco Avila <monkiki@gmail.com> wrote:
> Thank, using Whitespace Analyzer works, but I don't understand why
> StandardAnalyzer does not work if according with the ChineseAnalyzer
> deprecation I should use StandardAnalyzer:
>
> @deprecated Use {@link StandardAnalyzer} instead, which has the same
> functionality.
>
> Is very annoying.
>
> 2012/6/27 Li Li <fancyerii@gmail.com>
>
>> standard analyzer will segment each character into a token, you should u=
se
>> whitespace analyzer or your own analyzer that can tokenize it as one tok=
en
>> for wildcard search
>> =D4=DA 2012-6-27 =B0=F8=CD=ED6:20=A3=AC"Paco Avila" <monkiki@gmail.com>=
=D0=B4=B5=C0=A3=BA
>>
>> > Hi there,
>> >
>> > I have to index chinese content and I don't get the expected results w=
hen
>> > searching. It seems that the WildcardQuery does not work properly with
>> the
>> > chinese characters. See attached sample code.
>> >
>> > I store the string "=D7=A8=CF=EE=D0=C5=CF=A2=B9=DC=C0=ED.doc" using th=
e StandardAnalyzer and after that
>> > search for "=D7=A8=CF=EE=D0=C5*" and no result is given. AFAIK, it sho=
uld match the
>> > "=D7=A8=CF=EE=D0=C5=CF=A2=B9=DC=C0=ED.doc" string but it doesn't :(
>> >
>> > NOTE: Use Lucene 3.1.0
>> >
>> > Regards.
>> > --
>> > http://www.openkm.com
>> > http://www.guia-ubuntu.org
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>> >
>>
>
>
>
> --
> OpenKM
> http://www.openkm.com
> http://www.guia-ubuntu.org

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org