Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Message-ID: <1743210637.1205860886472.JavaMail.jira@brutus>
Date: Tue, 18 Mar 2008 10:21:26 -0700 (PDT)
From: "Steven Rowe (JIRA)" <jira@apache.org>
To: java-dev@lucene.apache.org
Subject: [jira] Commented: (LUCENE-400) NGramFilter -- construct n-grams
 from a TokenStream
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/LUCENE-400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579933#action_12579933 ] 

Steven Rowe commented on LUCENE-400:
------------------------------------

re-ping, Otis, do you still plan to commit?

> NGramFilter -- construct n-grams from a TokenStream
> ---------------------------------------------------
>
>                 Key: LUCENE-400
>                 URL: https://issues.apache.org/jira/browse/LUCENE-400
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Analysis
>    Affects Versions: unspecified
>         Environment: Operating System: All
> Platform: All
>            Reporter: Sebastian Kirsch
>            Assignee: Otis Gospodnetic
>            Priority: Minor
>             Fix For: 2.4
>
>         Attachments: LUCENE-400.patch, NGramAnalyzerWrapper.java, NGramAnalyzerWrapperTest.java, NGramFilter.java, NGramFilterTest.java
>
>
> This filter constructs n-grams (token combinations up to a fixed size, sometimes
> called "shingles") from a token stream.
> The filter sets start offsets, end offsets and position increments, so
> highlighting and phrase queries should work.
> Position increments > 1 in the input stream are replaced by filler tokens
> (tokens with termText "_" and endOffset - startOffset = 0) in the output
> n-grams. (Position increments > 1 in the input stream are usually caused by
> removing some tokens, eg. stopwords, from a stream.)
> The filter uses CircularFifoBuffer and UnboundedFifoBuffer from Apache
> Commons-Collections.
> Filter, test case and an analyzer are attached.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org