lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Koji Sekiguchi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-2673) CJKAnalyzer not matching mutlibyte character followed by non-multibyte character
Date Mon, 27 Sep 2010 14:45:33 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915323#action_12915323
] 

Koji Sekiguchi commented on LUCENE-2673:
----------------------------------------

I think CJKAnalyzer works as expected.

{quote}
QES様 -> not retrievable
SOA基 -> not retrievable
{quote}

Because CJK chars are tokenized 2-gram, "様" and "基" are not token.

{quote}
日経B -> not retrievable
{quote}

Because non CJK chars are tokenized at white space, "B" is not token.

> CJKAnalyzer not matching mutlibyte character followed by non-multibyte character
> --------------------------------------------------------------------------------
>
>                 Key: LUCENE-2673
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2673
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: contrib/analyzers
>    Affects Versions: 3.0.1
>            Reporter: Kevin Hayen
>
> Here is a listing of text indexed in a field, followed by various search terms that did
or did not match the document.
> [QES様文字化けテスト]
> QES -> retrievable
> QES様 -> not retrievable
> QES様文字化けテスト -> retrievable
> [SOA基盤]
> SOA ->retrievable
> SOA基 -> not retrievable
> SOA基盤 -> retrievable
> [日経BP]
> 日経 -> retrievable
> 日経B -> not retrievable
> 日経BP -> retrievable

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message