opennlp-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joern Kottmann <kottm...@gmail.com>
Subject Re: [VOTE] Apache OpenNLP 1.8.0 Release Candidate 2
Date Mon, 15 May 2017 08:39:23 GMT
Hello Richard,

thanks for reporting this. For 1.8.0 we replaced a Heap with a SortedSet
[1]. In this commit there is one loop [2] which iterates through the parses
which will be advanced. The order of the Parsers in the Heap was not so
well defined, therefore we decided to sort them by probability.
We also noticed that this change is changing the output of the parser with
the existing models in our SourceForge model eval test [3].

After running the evaluation on the OntoNotes4 data set I only got  very
small change and decided it is ok to do this. I am not aware of how big the
change is but is was less than the delta in test case [4] of 0.001.

What do you think? Should this be rolled back?

Anyway, that said, about the parser, I still need to understand what
happened with the lemmatizer.

Jörn

[1]
https://github.com/apache/opennlp/commit/3df659b9bfb02084e782f1e8b6ec716f56e0611c
[2]
https://github.com/apache/opennlp/blob/3df659b9bfb02084e782f1e8b6ec716f56e0611c/opennlp-tools/src/main/java/opennlp/tools/parser/AbstractBottomUpParser.java#L285
[3]
https://github.com/apache/opennlp/commit/3df659b9bfb02084e782f1e8b6ec716f56e0611c#diff-a5834f32b8a41b76a336126e4b13d4f7L349
[4]
https://github.com/apache/opennlp/blob/3df659b9bfb02084e782f1e8b6ec716f56e0611c/opennlp-tools/src/test/java/opennlp/tools/eval/OntoNotes4ParserEval.java#L70

On Sat, May 13, 2017 at 10:35 PM, Richard Eckart de Castilho <rec@apache.org
> wrote:

> Hi all,
>
> > On 11.05.2017, at 18:37, Joern Kottmann <kottmann@gmail.com> wrote:
> >
> > The Apache OpenNLP PMC would like to call for a Vote on Apache OpenNLP
> > 1.8.0 Release Candidate 2.
>
> Should OpenNLP 1.8.0 yield identical results as 1.7.2 when the same
> models are used during classification?
>
> E.g. the English parser model seems to create different POS tags now
> for the sentence "We need a very complicated example sentence ,
> which contains as many constituents and dependencies as possible .".
> "a" is now wrongly tagged as "," whereas 1.7.2 tagged it correctly as "DT".
>
> Should OpenNLP 1.8.0 yield identical results as 1.7.2 when the same
> training data is used during training?
>
> I have a test that trains a lemmatizer model on GUM 3.0.0. With 1.7.2,
> this model reached an f-score of ~0.96. With 1.8.0, I only get ~0.84.
>
> Cheers,
>
> -- Richard
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message