lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ahmed El-dawy <aseld...@gmail.com>
Subject Re: Using the new tokenizer API from a jar file
Date Mon, 04 Jan 2010 21:32:32 GMT
Sorry for this delay. I was having a silly problem compiling solr but I
figured it out.
I tested it and it worked correctly. Thanks

On Wed, Dec 30, 2009 at 8:31 PM, Uwe Schindler <uwe@thetaphi.de> wrote:

> That would be good, if you could test it!
>
> Please checkout Lucene 2.9 branch from svn
> (http://svn.apache.org/repos/asf/lucene/java/branches/lucene_2_9), compile
> the whole package (at least lucene-core.jar) and then replace the lucene
> jar
> files in solr's lib folder.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> > -----Original Message-----
> > From: Ahmed El-dawy [mailto:aseldawy@gmail.com]
> > Sent: Wednesday, December 30, 2009 11:56 AM
> > To: java-user@lucene.apache.org
> > Cc: solr-user@lucene.apache.org
> > Subject: Re: Using the new tokenizer API from a jar file
> >
> > Thanks all for your interest, especially Uwe. I asked this question on
> > solr-user at the beginning but I got no reply. That's why I re-asked the
> > question at java-user.
> > Thanks for your efforts. I will try it now.
> >
> > On Mon, Dec 28, 2009 at 12:02 PM, Uwe Schindler <uwe@thetaphi.de> wrote:
> >
> > > I opened https://issues.apache.org/jira/browse/LUCENE-2182 about this
> > > problem and already have a fix.
> > >
> > > This is really a bug. The solution is simple because you have to load
> > the
> > > IMPL class using the same classloader as the passed in interface. The
> > > default for Class.forName is the classloader of AttributeSource.class,
> > > which
> > > is the wrong one.
> > >
> > > -----
> > > Uwe Schindler
> > > H.-H.-Meier-Allee 63, D-28213 Bremen
> > > http://www.thetaphi.de
> > > eMail: uwe@thetaphi.de
> > >
> > > > -----Original Message-----
> > > > From: Uwe Schindler [mailto:uwe@thetaphi.de]
> > > > Sent: Monday, December 28, 2009 9:20 AM
> > > > To: java-user@lucene.apache.org
> > > > Cc: solr-user@lucene.apache.org
> > > > Subject: RE: Using the new tokenizer API from a jar file
> > > >
> > > > The question on this list was ok,as it shows a minor problem of using
> > the
> > > > new TokenStream API with Solr.
> > > >
> > > > His plugin was loaded correctly, because if Lucene says, that it
> > cannot
> > > > find
> > > > the *Impl class, it was able to load the interface class before ->
> the
> > > JAR
> > > > file is "visible" to the JVM.
> > > >
> > > > The problem is the following and has to do with classloaders:
> > > >
> > > > 1. We have different class loaders for different places in Solr. Solr
> > > uses
> > > > for plugins a SolrResourceLoader that searches for JAR files in the
> > local
> > > > lib folder before handling over to the webapp's classloader.
> > > >
> > > > 2. Initially, the lucene JAR is loaded by the Webapp's class loader
> > > >
> > > > 3. If a AttributeImpl is placed into a jar file e.g. in the plugin
> > folder
> > > > of
> > > > solr (the lib folder where solr loads all resources, stop words,...),
> > the
> > > > loading mechanism inside AttributeSource.DEFAULT_ATTRIBUTE_FACTORY is
> > > > unable
> > > > to locate the class file, because Class.forName() always uses the
> > class'
> > > > classloader and not the global/thread one's. So AttributeSource will
> > only
> > > > find the class file if it is in the *same* directory as the lucene-
> > > > core.jar
> > > > file (WEB-INF/lib) and so accessible by the webapp's class loader.
> > > >
> > > > A good introduction about the problem is this one:
> > > >
> > >
> >
> http://www.theserverside.com/tt/articles/content/dm_classForname/DynLoad.p
> > > > df
> > > >
> > > > The problem is here described for the JVM extensions folder but also
> > > > applies
> > > > to solr, because it has another classloader for plugins.
> > > >
> > > > A solution to fix this would be in lucene to use the thread's context
> > > > class
> > > > loader in AttributeSource.DEFAULT_ATTRIBUTE_FACTORY, but I strongly
> > > > discourage this, as it would break the whole AttributeSource
> > > functionality
> > > > if you add two different attributes with same class names from
> > different
> > > > class loaders to the AttributeSource.
> > > >
> > > > The only solution to the problem is placing the JAR file inside the
> > > > WEB-INF/lib folder where lucene-core.jar is. Plugins in Solr cannot
> > > define
> > > > own attribute implementations. Alternatively he could try to force
> > > preload
> > > > the class by calling Class.forName in his plugin initialization code
> > on
> > > > the
> > > > Impl class. But I am not sure if this works (as Java handles classes
> > from
> > > > different classloaders different).
> > > >
> > > > Uwe
> > > >
> > > > -----
> > > > Uwe Schindler
> > > > H.-H.-Meier-Allee 63, D-28213 Bremen
> > > > http://www.thetaphi.de
> > > > eMail: uwe@thetaphi.de
> > > >
> > > > > -----Original Message-----
> > > > > From: Chris Hostetter [mailto:hossman_lucene@fucit.org]
> > > > > Sent: Monday, December 28, 2009 4:27 AM
> > > > > To: java-user@lucene.apache.org
> > > > > Subject: Re: Using the new tokenizer API from a jar file
> > > > >
> > > > >
> > > > > : I tried to use it with solr and the problems began. It's always
> > > > telling
> > > > > me
> > > > > : that it cannot find the class GlossAttributeImpl. I think the
> > problem
> > > > is
> > > > > : that my jar file is added to the class path at run time not from
> > the
> > > > > command
> > > > > : line. Do you have a good solution or workaround?
> > > > >
> > > > > You're likely to get mmore helpful answers from other people in the
> > > Solr
> > > > > User community (solr-user@lucene.a.o)
> > > > >
> > > > > As long as you put your jar in the "lib" directory under your solr
> > home
> > > > > (or refrence it using a <lib/> directive in your solrconfig.xml)
> > Solr's
> > > > > plugin loader will take care of hte classloading for you.
> > > > >
> > > > > if you are confident you have your jar in the correct place, please
> > > > email
> > > > > solr-user with the ClassNotFound stack trace from your solr logs,
> as
> > > > well
> > > > > as hierarchy  of files from your solr home (ie: the output of "find
> > .")
> > > > >
> > > > >
> > > > > -Hoss
> > > > >
> > > > >
> > > > >
> --------------------------------------------------------------------
> > -
> > > > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >
> > >
> >
> >
> > --
> > regards,
> > Ahmed Saad
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


-- 
regards,
Ahmed Saad

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message