lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Torao Takami (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-1227) NGramTokenizer to handle more than 1024 chars
Date Mon, 25 Jul 2011 04:07:09 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070309#comment-13070309
] 

Torao Takami commented on LUCENE-1227:
--------------------------------------

I avoided this problem by putting NGramFilterFactory and WhitespaceTokenizerFactory together
in schema.

<tokenizer class="solr.NGramTokenizerFactory" maxGramSize="2" minGramSize="2" />

is written to:

<filter class="solr.NGramFilterFactory" minGramSize="2" maxGramSize="2"/>
<tokenizer class="solr.WhitespaceTokenizerFactory"/>


> NGramTokenizer to handle more than 1024 chars
> ---------------------------------------------
>
>                 Key: LUCENE-1227
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1227
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: modules/analysis
>            Reporter: Hiroaki Kawai
>            Assignee: Grant Ingersoll
>            Priority: Minor
>         Attachments: LUCENE-1227.patch, NGramTokenizer.patch, NGramTokenizer.patch
>
>
> Current NGramTokenizer can't handle character stream that is longer than 1024. This is
too short for non-whitespace-separated languages.
> I created a patch for this issues.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message