Mailing-List: contact opennlp-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: opennlp-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of kottmann@gmail.com designates
 209.85.214.47 as permitted sender)
Message-ID: <4E4CEA35.1040103@gmail.com>
Date: Thu, 18 Aug 2011 12:32:21 +0200
From: =?UTF-8?B?SsO2cm4gS290dG1hbm4=?= <kottmann@gmail.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6;
 rv:6.0) Gecko/20110812 Thunderbird/6.0
MIME-Version: 1.0
To: opennlp-dev@incubator.apache.org
Subject: Re: Stemmer
References: <4E4CE74A.7020000@gmail.com>
 <CAFvE7K6f0DB0=j6M_yZZrZrE8MwPEknBLrKssW7TA9tevfJT-Q@mail.gmail.com>
In-Reply-To: 
 <CAFvE7K6f0DB0=j6M_yZZrZrE8MwPEknBLrKssW7TA9tevfJT-Q@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 8/18/11 12:24 PM, Olivier Grisel wrote:
> Is this better or cover more languages than what's already provided by
> Apache Lucene? Maybe it should better be contributed to the Lucene
> project and make it easy to use the generic, battle tested Lucene
> analyzers / tokenizers infrastructure to generate features in OpenNLP.

The OpenNLP APIs are all not designed to work on token streams, instead
a user usually has to provide an entire sentence at once, so that does not
make a nice fit.

And since we are an NLP library I believe it is absolutly fine to implement
our own stemming here.

Jörn