lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yonik Seeley <ysee...@gmail.com>
Subject Re: StandardTokenizer
Date Tue, 27 Sep 2005 14:05:17 GMT
I'd write a TokenFilter for that... much easier.

-Yonik
Now hiring -- http://tinyurl.com/7m67g

On 9/27/05, Lorenzo Viscanti <lorenzo.viscanti@gmail.com> wrote:
>
> Hi, I'm trying to modify the StandardTokenizer, in order to get to get a
> good tokenization for my needs.
> Basically I would like to separate two tokens when I find an apostrophe. I
> think I should modify the StandardTokenizer.jj file to do that, but I'm in
> trouble while changing the grammar. Can someone help me with the right
> syntax?
> Thanks,
> Lorenzo
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message