Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of rcmuir@gmail.com designates
 209.85.210.197 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=ltjCaDDQqs5XnUS+YbpLpfG175iW0H1ljcdjMTbj+dv+1iBHo5SVMnZi0lje8L3wWB
         TwM0jN7uSVGD/SGox/QUfAZ62W0w41R5djzLZ0tcT/DTZ3FM2kcIFxe7ERI8e43FEDM2
         T1OeT8S7IAjk8VswlI2swYYThOLDo8bTtzP6g=
MIME-Version: 1.0
In-Reply-To: <BAY119-DS54ED70BB70E398670D5AA8E270@phx.gbl>
References: <BAY119-DS54ED70BB70E398670D5AA8E270@phx.gbl>
Date: Fri, 10 Jul 2009 18:13:20 -0400
Message-ID: <8f0ad1f30907101513n40ec9e71v3d25c0fab3e345f6@mail.gmail.com>
Subject: Re: Hindi, diacritics and search results
From: Robert Muir <rcmuir@gmail.com>
To: java-user@lucene.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Which analyzer in particular are you using?

Its probably not doing what you want for hindi. These "diacritics" are
important (vowels, etc).


On Fri, Jul 10, 2009 at 3:10 PM, OBender<osya_bender@hotmail.com> wrote:
> Hi All,
>
>
>
> I'm using the default setup of lucene (no custom analyzers configured) an=
d
> came across the following issue:
>
> In Hindi if there is a letter with a diacritic in a phrase lucene will fi=
nd
> the phrase with this letter even if the search string is for the letter
> without a diacritics.
>
> Is this an expected behavior? Maybe this is standard for all languages wi=
th
> letters that have diacritics?
>
>
>
> From pure byte standpoint I can see the logic, the letter with diacritics
> takes 6 bytes (E0 A4 95 E0 A5 87) and the single letter takes =C2=A03 (E0=
 A4 95)
> so if I search for *some_letter* where some letter has code (E0 A4 95)
> lucene finds the "phrase" (E0 A4 95 E0 A5 87) that includes that letter.
>
>
>
> Any comments much appreciated.
>
>
>
> Thanks.
>
>
>
>


--=20
Robert Muir
rcmuir@gmail.com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org