Mailing-List: contact users-help@opennlp.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@opennlp.apache.org
Received-SPF: pass (nike.apache.org: domain of kottmann@gmail.com designates
 209.85.214.49 as permitted sender)
Message-ID: <4F7C3EB2.5020908@gmail.com>
Date: Wed, 04 Apr 2012 14:29:38 +0200
From: =?ISO-8859-1?Q?J=F6rn_Kottmann?= <kottmann@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:11.0) Gecko/20120310 Thunderbird/11.0
MIME-Version: 1.0
To: users@opennlp.apache.org
Subject: Re: will version 1.5.3 be announced here?
References: <4F7871B5.80504@gmail.com> <4F788FC5.1010400@gmail.com>
 <4F79963C.6060308@gmail.com> <4F7ACAEB.3040501@gmail.com>
 <4F7AD2F1.3050509@gmail.com> <4F7AD3C9.9050800@gmail.com>
 <4F7ADE17.9020205@gmail.com>
 <CALV1_=JBb5TWQKm8J+f=sc69W9AWEo0m9q-QNyYzt46jGS4atQ@mail.gmail.com>
 <4F7C2691.8010104@gmail.com>
 <CALV1_=K5bxpoE4aK4YCNRz9-9V0FeOinxrdpztp9jVyCZZ1TDw@mail.gmail.com>
 <4F7C364C.5020905@gmail.com> <4F7C370D.2000509@gmail.com>
 <4F7C3CD9.70703@gmail.com>
In-Reply-To: <4F7C3CD9.70703@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

On 04/04/2012 02:21 PM, Jim - FooBar(); wrote:
> Hmm...it is not that simple! what about the maxent model? Let's say you
>
> --create a class that extends Evaluator  (as all evaluators do)
> --allow the constructor of that class to take variable number of 
> arguments (of type TokenNameFinder) so we can use them later on
> --pretty much copy the code from TokenNameFinderEvaluator and paste it 
> in the new class
> --make sure the code asks both (or however many) name-finders before 
> it classifies a prediction as right or wrong.
>
> Up to here everything is quite straight forward...now the problems 
> begin!!!
> The Dictionary has to be evaluated on separate data than the 
> model...That is because the Dictionary can only deal with  the 
> "default tag" and nothing else! A quick workaround would be to retrain 
> the maxent model with "default" tags so i can evaluate it on the 
> dictionary's test-set but then what about multiple types??? 

We could pass a tag to the DictionaryNameFinder it should output, then 
it works with the data you give to the
evaluator.
You would run your tests twice, once with the maxent name finder and 
once with the dictionary name finder.

Why do you want to extend the evaluators? It should work out of the box.

I would do it like this:
ObejctStream<NameSample> samples = ...;

// create NameFinderME
NameFinderME maxentNameFinder = ...;

TokenNameFinderEvaluator maxentEval = new 
TokenNameFinderEvaluator(maxentNameFinder);
maxentEval.evaluate(samples);
maxentEval.getFMeasure(); // here are you maxent results

samples.reset();

DictionaryNameFinder dictNameFinder = ...;
TokenNameFinderEvaluator dictEval = new 
TokenNameFinderEvaluator(dictNameFinder);
dictEval.evaluate(samples);
dictEval.getFMeasure(); // here are you dictionary name finder results

Doesn't that more or less like this work?

J�rn