Mailing-List: contact general-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of davidshen84@gmail.com
 designates 209.85.220.179 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <A0B1F09E-0CD4-46BB-B3B4-44D491CAAAFF@gmail.com>
References: 
 <CANO68EOTk3tPLKZTRw7cop0VrtMUnNNZAv8VTEd0QDAE4pj3fQ@mail.gmail.com>
	<CANO68EMg8JnxT-sieHPVsBzHmCzb5QPM-wXb0J4AES=PCJSbEw@mail.gmail.com>
	<A0B1F09E-0CD4-46BB-B3B4-44D491CAAAFF@gmail.com>
Date: Fri, 21 Dec 2012 15:46:48 +0800
Message-ID: 
 <CANO68EMWdrtK8B0J917e+TDOYThXqJXFuJt6kf_Xs4=K14r1CQ@mail.gmail.com>
Subject: Re: Which token filter can combine 2 terms into 1?
From: Xi Shen <davidshen84@gmail.com>
To: general@lucene.apache.org
Content-Type: multipart/alternative; boundary=bcaec502d4c8c6c24404d1580c1c

--bcaec502d4c8c6c24404d1580c1c
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi Steve,

This is a language dependent case. Basically, I will use white space token
filter to process the input. But some of the inputs should be one term,
instead of split into 2 terms. I think am thinking developing a special
filter to fix these terms.


On Fri, Dec 21, 2012 at 3:34 PM, Steve Rowe <sarowe@gmail.com> wrote:

> Hi David,
>
> Not very many people read this mailing list - I suggest you switch to the
> java-user list - see <http://lucene.apache.org/core/discussion.html>.
>
> SingleFilter and CommonGramsFilter combine terms, though the conditions
> under which they do so don't appear to be the same as what you want.
>
> Why are only the second two terms combined?
>
> Steve
>
> On Dec 21, 2012, at 2:27 AM, Xi Shen <davidshen84@gmail.com> wrote:
>
> > Hi,
> >
> > I am looking for a token filter that can combine 2 terms into 1? E.g.
> >
> > the input has been tokenized by white space:
> >
> > t1 t2 t2a t3
> >
> > I want a filter that output:
> >
> > t1 t2t2a t3
> >
> > I know it is a very special case, and I am thinking about develop a
> filter
> > of my own. But I cannot figure out which API I should use to look for
> terms
> > in a Token Stream.
> >
> >
> > --
> > Regards=EF=BC=8C
> > David Shen
> >
> > http://about.me/davidshen
> > https://twitter.com/#!/davidshen84
>
>


--=20
Regards=EF=BC=8C
David Shen

http://about.me/davidshen
https://twitter.com/#!/davidshen84

--bcaec502d4c8c6c24404d1580c1c--