Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Mon, 27 May 2013 12:12:20 +0000 (UTC)
From: "Karl Wettin (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: <JIRA.12648931.1369242245256.26021.1369656740727@arcas>
In-Reply-To: <JIRA.12648931.1369242245256@arcas>
References: <JIRA.12648931.1369242245256@arcas>
Subject: [jira] [Updated] (LUCENE-5013)
 ScandinavianInterintelligableASCIIFoldingFilter
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


     [ https://issues.apache.org/jira/browse/LUCENE-5013?page=3Dcom.atlassi=
an.jira.plugin.system.issuetabpanels:all-tabpanel ]

Karl Wettin updated LUCENE-5013:
--------------------------------

    Attachment: LUCENE-5013.patch

Patch blessed with ASL2
               =20
> ScandinavianInterintelligableASCIIFoldingFilter
> -----------------------------------------------
>
>                 Key: LUCENE-5013
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5013
>             Project: Lucene - Core
>          Issue Type: New Feature
>          Components: modules/analysis
>    Affects Versions: 4.3
>            Reporter: Karl Wettin
>            Priority: Trivial
>         Attachments: LUCENE-5013-2.txt, LUCENE-5013-3.txt, LUCENE-5013-4.=
txt, LUCENE-5013-5.txt, LUCENE-5013-6.txt, LUCENE-5013.patch, LUCENE-5013.t=
xt
>
>
> This filter is an augmentation of output from ASCIIFoldingFilter,
> it discriminate against double vowels aa, ae, ao, oe and oo, leaving just=
 the first one.
> bl=C3=A5b=C3=A6rsyltet=C3=B8j =3D=3D bl=C3=A5b=C3=A4rsyltet=C3=B6j =3D=3D=
 blaabaarsyltetoej =3D=3D blabarsyltetoj
> r=C3=A4ksm=C3=B6rg=C3=A5s =3D=3D r=C3=A6ksm=C3=B8rg=C3=A5s =3D=3D r=C3=A6=
ksm=C3=B6rgaos =3D=3D raeksmoergaas =3D=3D raksmorgas
> Caveats:
> Since this is a filtering on top of ASCIIFoldingFilter =C3=A4=C3=B6=C3=A5=
=C3=B8=C3=A6 already has been folded down to aoaoae when handled by this fi=
lter it will cause effects such as:
> b=C3=B8en -> boen -> bon
> =C3=A5ene -> aene -> ane
> I find this to be a trivial problem compared to not finding anything at a=
ll.
> Background:
> Swedish =C3=A5=C3=A4=C3=B6 is in fact the same letters as Norwegian and D=
anish =C3=A5=C3=A6=C3=B8 and thus interchangeable in when used between thes=
e languages. They are however folded differently when people type them on a=
 keyboard lacking these characters and ASCIIFoldingFilter handle =C3=A4 and=
 =C3=A6 differently.
> When a Swedish person is lacking umlauted characters on the keyboard they=
 consistently type a, a, o instead of =C3=A5, =C3=A4, =C3=B6. Foreigners al=
so tend to use a, a, o.
> In Norway people tend to type aa, ae and oe instead of =C3=A5, =C3=A6 and=
 =C3=B8. Some use a, a, o. I've also seen oo, ao, etc. And permutations. No=
t sure about Denmark but the pattern is probably the same.
> This filter solves that problem, but might also cause new.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrato=
rs
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org