opennlp-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damiano Porta <damianopo...@gmail.com>
Subject Re: Why can i not serialize a Dictionary ?
Date Mon, 31 Oct 2016 12:49:08 GMT
hmm ok i load the postagger in the constructor of my custom
FeatureGenerator, this is an example:

https://gist.github.com/anonymous/542275c2fdb817c02c6275eb227c467d

what do you think?

2016-10-31 13:16 GMT+01:00 William Colen <william.colen@gmail.com>:

> Unfortunately I don't think the API supports this. You will need a hack.
>
> 2016-10-30 12:59 GMT-02:00 Damiano Porta <damianoporta@gmail.com>:
>
> > Jorn
> > what suffix should i use if i need a postagger model in a
> FeatureGenerator?
> >
> > For dictionary i use mydictionary.dictionary as you told me. What about
> > postagger .bin?
> >
> > Thanks
> > Damiano
> >
> > Il 29/Ott/2016 14:27, "Damiano Porta" <damianoporta@gmail.com> ha
> scritto:
> >
> > > ok! thank you Jorn!
> > >
> > > 2016-10-29 13:54 GMT+02:00 Joern Kottmann <kottmann@gmail.com>:
> > >
> > >> The class has to be on your classpath otherwise it can't be loaded.
> > >>
> > >> Jörn
> > >>
> > >> On Fri, 2016-10-28 at 22:59 +0200, Damiano Porta wrote:
> > >> > Jorn,
> > >> > as I wrote i have created the ner model, but when i try to use it
> via
> > >> > "opennlp TokenNameFinder" tool it can't locate my custom
> > >> > FeatureGenerator.
> > >> > This is the output:
> > >> >
> > >> > bash opennlp TokenNameFinder /home/damiano/custom-ner-model.bin
> > >> > Loading Token Name Finder model ... done (5,948s)
> > >> > Exception in thread "main"
> > >> > opennlp.tools.util.ext.ExtensionNotLoadedException: Unable to find
> > >> > implementation for
> > >> > opennlp.tools.util.featuregen.AdaptiveFeatureGenerator,
> > >> > the class or service
> > >> > com.damiano.parser.generator.SpanWindowFeatureGenerator could not
> be
> > >> > located!
> > >> > at
> > >> > opennlp.tools.util.ext.ExtensionLoader.
> instantiateExtension(Extension
> > >> > Loader.java:122)
> > >> > at
> > >> > opennlp.tools.util.featuregen.GeneratorFactory$
> CustomFeatureGenerator
> > >> > Factory.create(GeneratorFactory.java:582)
> > >> > at
> > >> > opennlp.tools.util.featuregen.GeneratorFactory.
> createGenerator(Genera
> > >> > torFactory.java:661)
> > >> > at
> > >> > opennlp.tools.util.featuregen.GeneratorFactory$
> AggregatedFeatureGener
> > >> > atorFactory.create(GeneratorFactory.java:129)
> > >> > at
> > >> > opennlp.tools.util.featuregen.GeneratorFactory.
> createGenerator(Genera
> > >> > torFactory.java:661)
> > >> > at
> > >> > opennlp.tools.util.featuregen.GeneratorFactory$
> CachedFeatureGenerator
> > >> > Factory.create(GeneratorFactory.java:171)
> > >> > at
> > >> > opennlp.tools.util.featuregen.GeneratorFactory.
> createGenerator(Genera
> > >> > torFactory.java:661)
> > >> > at
> > >> > opennlp.tools.util.featuregen.GeneratorFactory$
> AggregatedFeatureGener
> > >> > atorFactory.create(GeneratorFactory.java:129)
> > >> > at
> > >> > opennlp.tools.util.featuregen.GeneratorFactory.
> createGenerator(Genera
> > >> > torFactory.java:661)
> > >> > at
> > >> > opennlp.tools.util.featuregen.GeneratorFactory.create(
> GeneratorFactor
> > >> > y.java:711)
> > >> > at
> > >> > opennlp.tools.namefind.TokenNameFinderFactory.
> createFeatureGenerators
> > >> > (TokenNameFinderFactory.java:153)
> > >> > at
> > >> > opennlp.tools.namefind.TokenNameFinderFactory.
> createContextGenerator(
> > >> > TokenNameFinderFactory.java:118)
> > >> > at opennlp.tools.namefind.NameFinderME.<init>(NameFinderME.java:90)
> > >> > at
> > >> > opennlp.tools.cmdline.namefind.TokenNameFinderTool.
> run(TokenNameFinde
> > >> > rTool.java:59)
> > >> > at opennlp.tools.cmdline.CLI.main(CLI.java:227)
> > >> >
> > >> > Why does the custom generator is not included on the model?
> > >> > This is the my xml descriptor:
> > >> >
> > >> > <generators>
> > >> >   <cache>
> > >> >     <generators>
> > >> >       <window prevLength="6" nextLength="3">
> > >> >         <tokenclass/>
> > >> >       </window>
> > >> >       <window prevLength="6" nextLength="3">
> > >> >         <token/>
> > >> >       </window>
> > >> >       <definition/>
> > >> >       <prevmap/>
> > >> >       <bigram/>
> > >> >       <sentence begin="true" end="false"/>
> > >> >       <dictionary prefix="name" dict="names.dictionary"/>
> > >> >       <custom
> > >> > class="com.damiano.parser.generator.SpanWindowFeatureGenerator"
> > >> > leftTokens="6" rightTokens="3"/>
> > >> >     </generators>
> > >> >   </cache>
> > >> > </generators>
> > >> >
> > >> >
> > >> > Damiano
> > >> >
> > >> > 2016-10-28 14:00 GMT+02:00 Damiano Porta <damianoporta@gmail.com>:
> > >> >
> > >> > >
> > >> > > Pardon, my wrong, i forgot to change <dictionary prefix="city"
> > >> > > dict="damiano"/>  into <dictionary prefix="city"
> > >> > > dict="damiano.dictionary"/>    in my train.xml
> > >> > >
> > >> > > now it is working well! and the .bin has my dictionary too
> > >> > >
> > >> > > 2016-10-28 13:51 GMT+02:00 Damiano Porta <damianoporta@gmail.com
> >:
> > >> > >
> > >> > > >
> > >> > > > Jorn
> > >> > > > i change the code as you told me, this exactly:
> > >> > > > https://gist.github.com/anonymous/8877b09d441d2e64c181fa9b5a
> > >> > > > de4550#file-test-java-L15
> > >> > > >
> > >> > > > but i get this error:
> > >> > > >
> > >> > > > opennlp.tools.namefind.TokenNameFinderModel$
> FeatureGeneratorCreat
> > >> > > > ionError:
> > >> > > > opennlp.tools.util.InvalidFormatException: No dictionary
> resource
> > >> > > > for
> > >> > > > key: damiano
> > >> > > > at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureG
> > >> > > > enerators(TokenNameFinderFactory.java:176)
> > >> > > > at opennlp.tools.namefind.TokenNameFinderFactory.createContextG
> > >> > > > enerator(TokenNameFinderFactory.java:118)
> > >> > > > at
> > >> > > > opennlp.tools.namefind.NameFinderME.train(
> NameFinderME.java:333)
> > >> > > > at com.damiano.parser.trainer.NER.compileNER(NER.java:185)
> > >> > > > at com.damiano.parser.trainer.NER.main(NER.java:155)
> > >> > > >
> > >> > > > Caused by: opennlp.tools.util.InvalidFormatException: No
> > >> > > > dictionary
> > >> > > > resource for key: damiano
> > >> > > > at opennlp.tools.util.featuregen.GeneratorFactory$DictionaryFea
> > >> > > > tureGeneratorFactory.create(GeneratorFactory.java:251)
> > >> > > > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat
> > >> > > > or(GeneratorFactory.java:661)
> > >> > > > at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFea
> > >> > > > tureGeneratorFactory.create(GeneratorFactory.java:129)
> > >> > > > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat
> > >> > > > or(GeneratorFactory.java:661)
> > >> > > > at opennlp.tools.util.featuregen.GeneratorFactory$CachedFeature
> > >> > > > GeneratorFactory.create(GeneratorFactory.java:171)
> > >> > > > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat
> > >> > > > or(GeneratorFactory.java:661)
> > >> > > > at opennlp.tools.util.featuregen.GeneratorFactory$AggregatedFea
> > >> > > > tureGeneratorFactory.create(GeneratorFactory.java:129)
> > >> > > > at opennlp.tools.util.featuregen.GeneratorFactory.createGenerat
> > >> > > > or(GeneratorFactory.java:661)
> > >> > > > at opennlp.tools.util.featuregen.GeneratorFactory.create(Genera
> > >> > > > torFactory.java:711)
> > >> > > > at opennlp.tools.namefind.TokenNameFinderFactory.createFeatureG
> > >> > > > enerators(TokenNameFinderFactory.java:153)
> > >> > > > ... 4 more
> > >> > > >
> > >> > > > 2016-10-28 12:55 GMT+02:00 Joern Kottmann <kottmann@gmail.com>:
> > >> > > >
> > >> > > > >
> > >> > > > > Try to rename the dictionary key to xyz.dictionary
then the
> > >> > > > > serializer
> > >> > > > > will
> > >> > > > > be mapped correctly.
> > >> > > > >
> > >> > > > > Jörn
> > >> > > > >
> > >> > > > > On Thu, Oct 27, 2016 at 11:14 PM, Damiano Porta
> <damianoporta@g
> > >> > > > > mail.com>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > >
> > >> > > > > > Jorn i add the Dictionary here:
> > >> > > > > > https://gist.github.com/anonymous/
> bc822fb0520c4c42b75748bf414
> > >> > > > > > 7da
> > >> > > > > > 34#file-train-java-L15
> > >> > > > > >
> > >> > > > > > And unfortunately i only see this error:
> > >> > > > > >
> > >> > > > > > java.lang.IllegalStateException: Missing serializer
for
> > >> > > > > > damiano
> > >> > > > > > at
> > >> > > > > > opennlp.tools.util.model.BaseModel.serialize(BaseModel.
> java:6
> > >> > > > > > 10)
> > >> > > > > >
> > >> > > > > > I do not have other info.
> > >> > > > > > Do i have to create a custom Serializer too?
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > 2016-10-27 22:04 GMT+02:00 Joern Kottmann <
> kottmann@gmail.com
> > >> > > > > > >:
> > >> > > > > >
> > >> > > > > > >
> > >> > > > > > > On Thu, 2016-10-27 at 21:18 +0200, Joern
Kottmann wrote:
> > >> > > > > > > >
> > >> > > > > > > > On Tue, 2016-10-25 at 18:49 +0200, Damiano
Porta wrote:
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > i am getting a strange error during
the compiling of a
> > >> > > > > > > > > NER model.
> > >> > > > > > > > > Basically, the end of the build
output is:
> > >> > > > > > > > >
> > >> > > > > > > > >  98:  ... loglikelihood=-13340.018762351776
> > >> > > > > > > > > 0.999005934601099
> > >> > > > > > > > >  99:  ... loglikelihood=-13258.358751926637
> > >> > > > > > > > > 0.9990120681028991
> > >> > > > > > > > > 100:  ... loglikelihood=-13178.039964721707
> > >> > > > > > > > > 0.9990177634974279
> > >> > > > > > > > > Exception in thread "main"
> > >> > > > > > > > > java.lang.IllegalStateException:
> > >> > > > > Missing
> > >> > > > > >
> > >> > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > serializer for *mydictionary*
> > >> > > > > > > > > at
> > >> > > > > > > > > opennlp.tools.util.model.
> BaseModel.serialize(BaseModel.
> > >> > > > > > > > > java:
> > >> > > > > 610)
> > >> > > > > >
> > >> > > > > > >
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > Can you please post the full exception
stack trace?
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > And what is the name of they key you used
for the
> > >> > > > > > > dictionary?
> > >> > > > > > > The dictionary serializers are only mapped
by extension.
> > >> > > > > > >
> > >> > > > > > > Jörn
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message