Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of davidshen84@gmail.com
 designates 209.85.220.178 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CABYvkPTA-D=a7MCyW01wzefVSxEkzKYRk=MqK6AGp5sVqtSifQ@mail.gmail.com>
References: 
 <CANO68EPGM6EPxoVdXT=9SDNMCPOdLPBhd=HqLfO3gXZMEscpVw@mail.gmail.com>
	<CABYvkPTA-D=a7MCyW01wzefVSxEkzKYRk=MqK6AGp5sVqtSifQ@mail.gmail.com>
Date: Fri, 21 Dec 2012 16:42:38 +0800
Message-ID: 
 <CANO68ENu3wWUTjyaviQphQOhd2iNc2o+aBUvfg86qmt=8BJhRA@mail.gmail.com>
Subject: Re: Which token filter can combine 2 terms into 1?
From: Xi Shen <davidshen84@gmail.com>
To: java-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=f46d043d64b377b40504d158d401

--f46d043d64b377b40504d158d401
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

I have to use the white space and word delimiter to process the input
first. I tried many combination, and it seems to me that it is inevitable
the term will be split into two :(

I think developing my own filter is the only resolution...but I just cannot
find a guide to help me understand what I need to do to implement a
TokenFilter.


On Fri, Dec 21, 2012 at 4:03 PM, Danil =C5=A2ORIN <torindan@gmail.com> wrot=
e:

> Easiest way would be to pre-process your input and join those 2 tokens
> before splitting them by white space.
>
> But from given context I might miss some details...still worth a shot.
>
> On Fri, Dec 21, 2012 at 9:50 AM, Xi Shen <davidshen84@gmail.com> wrote:
>
> > Hi,
> >
> > I am looking for a token filter that can combine 2 terms into 1? E.g.
> >
> > the input has been tokenized by white space:
> >
> > t1 t2 t2a t3
> >
> > I want a filter that output:
> >
> > t1 t2t2a t3
> >
> > I know it is a very special case, and I am thinking about develop a
> filter
> > of my own. But I cannot figure out which API I should use to look for
> terms
> > in a Token Stream.
> >
> > --
> > Regards=EF=BC=8C
> > David Shen
> >
> > http://about.me/davidshen
> > https://twitter.com/#!/davidshen84
> >
>


--=20
Regards=EF=BC=8C
David Shen

http://about.me/davidshen
https://twitter.com/#!/davidshen84

--f46d043d64b377b40504d158d401--