Mailing-List: contact dev-help@opennlp.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@opennlp.apache.org
MIME-Version: 1.0
In-Reply-To: <9DA5AA0B-3F01-4232-9DC1-1D90E8B8CB0D@apache.org>
References: <1494520661.11460.5.camel@gmail.com> <9DA5AA0B-3F01-4232-9DC1-1D90E8B8CB0D@apache.org>
From: Joern Kottmann <kottmann@gmail.com>
Date: Mon, 15 May 2017 10:39:23 +0200
Message-ID: <CA+V=Wqi2F2AgWpx1K5zz17U1AeSOWpg4FQpaB4YBfZyvvyiokA@mail.gmail.com>
Subject: Re: [VOTE] Apache OpenNLP 1.8.0 Release Candidate 2
To: "dev@opennlp.apache.org" <dev@opennlp.apache.org>
Content-Type: multipart/alternative; boundary="001a113d3430f93940054f8bfd57"
archived-at: Mon, 15 May 2017 08:39:29 -0000

--001a113d3430f93940054f8bfd57
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hello Richard,

thanks for reporting this. For 1.8.0 we replaced a Heap with a SortedSet
[1]. In this commit there is one loop [2] which iterates through the parses
which will be advanced. The order of the Parsers in the Heap was not so
well defined, therefore we decided to sort them by probability.
We also noticed that this change is changing the output of the parser with
the existing models in our SourceForge model eval test [3].

After running the evaluation on the OntoNotes4 data set I only got  very
small change and decided it is ok to do this. I am not aware of how big the
change is but is was less than the delta in test case [4] of 0.001.

What do you think? Should this be rolled back?

Anyway, that said, about the parser, I still need to understand what
happened with the lemmatizer.

J=C3=B6rn

[1]
https://github.com/apache/opennlp/commit/3df659b9bfb02084e782f1e8b6ec716f56=
e0611c
[2]
https://github.com/apache/opennlp/blob/3df659b9bfb02084e782f1e8b6ec716f56e0=
611c/opennlp-tools/src/main/java/opennlp/tools/parser/AbstractBottomUpParse=
r.java#L285
[3]
https://github.com/apache/opennlp/commit/3df659b9bfb02084e782f1e8b6ec716f56=
e0611c#diff-a5834f32b8a41b76a336126e4b13d4f7L349
[4]
https://github.com/apache/opennlp/blob/3df659b9bfb02084e782f1e8b6ec716f56e0=
611c/opennlp-tools/src/test/java/opennlp/tools/eval/OntoNotes4ParserEval.ja=
va#L70

On Sat, May 13, 2017 at 10:35 PM, Richard Eckart de Castilho <rec@apache.or=
g
> wrote:

> Hi all,
>
> > On 11.05.2017, at 18:37, Joern Kottmann <kottmann@gmail.com> wrote:
> >
> > The Apache OpenNLP PMC would like to call for a Vote on Apache OpenNLP
> > 1.8.0 Release Candidate 2.
>
> Should OpenNLP 1.8.0 yield identical results as 1.7.2 when the same
> models are used during classification?
>
> E.g. the English parser model seems to create different POS tags now
> for the sentence "We need a very complicated example sentence ,
> which contains as many constituents and dependencies as possible .".
> "a" is now wrongly tagged as "," whereas 1.7.2 tagged it correctly as "DT=
".
>
> Should OpenNLP 1.8.0 yield identical results as 1.7.2 when the same
> training data is used during training?
>
> I have a test that trains a lemmatizer model on GUM 3.0.0. With 1.7.2,
> this model reached an f-score of ~0.96. With 1.8.0, I only get ~0.84.
>
> Cheers,
>
> -- Richard
>
>
>

--001a113d3430f93940054f8bfd57--