lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-5013) ScandinavianFoldingFilterFactory and ScandinavianNormalizationFilterFactory
Date Wed, 03 Jul 2013 15:08:22 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-5013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13699062#comment-13699062
] 

ASF subversion and git services commented on LUCENE-5013:
---------------------------------------------------------

Commit 1499437 from janhoy@apache.org
[ https://svn.apache.org/r1499437 ]

LUCENE-5013: ScandinavianFoldingFilterFactory and ScandinavianNormalizationFilterFactory (backport)
                
> ScandinavianFoldingFilterFactory and ScandinavianNormalizationFilterFactory
> ---------------------------------------------------------------------------
>
>                 Key: LUCENE-5013
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5013
>             Project: Lucene - Core
>          Issue Type: New Feature
>          Components: modules/analysis
>    Affects Versions: 4.3
>            Reporter: Karl Wettin
>            Assignee: Jan Høydahl
>            Priority: Trivial
>             Fix For: 5.0, 4.4
>
>         Attachments: LUCENE-5013-2.txt, LUCENE-5013-3.txt, LUCENE-5013-4.txt, LUCENE-5013-5.txt,
LUCENE-5013-6.txt, LUCENE-5013.patch, LUCENE-5013.txt
>
>
> This filter is an augmentation of output from ASCIIFoldingFilter,
> it discriminate against double vowels aa, ae, ao, oe and oo, leaving just the first one.
> blåbærsyltetøj == blåbärsyltetöj == blaabaarsyltetoej == blabarsyltetoj
> räksmörgås == ræksmørgås == ræksmörgaos == raeksmoergaas == raksmorgas
> Caveats:
> Since this is a filtering on top of ASCIIFoldingFilter äöåøæ already has been folded
down to aoaoae when handled by this filter it will cause effects such as:
> bøen -> boen -> bon
> åene -> aene -> ane
> I find this to be a trivial problem compared to not finding anything at all.
> Background:
> Swedish åäö is in fact the same letters as Norwegian and Danish åæø and thus interchangeable
in when used between these languages. They are however folded differently when people type
them on a keyboard lacking these characters and ASCIIFoldingFilter handle ä and æ differently.
> When a Swedish person is lacking umlauted characters on the keyboard they consistently
type a, a, o instead of å, ä, ö. Foreigners also tend to use a, a, o.
> In Norway people tend to type aa, ae and oe instead of å, æ and ø. Some use a, a,
o. I've also seen oo, ao, etc. And permutations. Not sure about Denmark but the pattern is
probably the same.
> This filter solves that problem, but might also cause new.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message