lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: Using the new tokenizer API from a jar file
Date Mon, 04 Jan 2010 21:43:46 GMT
Thanks for checking this out!

So my research was fine and I fixed it the intuitive and in my opinion
"correct way" (not with such hacks like using the thread's class loader you
see so often in the internet, but which are contraprodutive because they
often break the Java security model or break class re-loading on webapp
refresh in various app servers).

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Ahmed El-dawy [mailto:aseldawy@gmail.com]
> Sent: Monday, January 04, 2010 10:33 PM
> To: java-user@lucene.apache.org
> Subject: Re: Using the new tokenizer API from a jar file
> 
> Sorry for this delay. I was having a silly problem compiling solr but I
> figured it out.
> I tested it and it worked correctly. Thanks
> 
> On Wed, Dec 30, 2009 at 8:31 PM, Uwe Schindler <uwe@thetaphi.de> wrote:
> 
> > That would be good, if you could test it!
> >
> > Please checkout Lucene 2.9 branch from svn
> > (http://svn.apache.org/repos/asf/lucene/java/branches/lucene_2_9),
> compile
> > the whole package (at least lucene-core.jar) and then replace the lucene
> > jar
> > files in solr's lib folder.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: uwe@thetaphi.de
> >
> >
> > > -----Original Message-----
> > > From: Ahmed El-dawy [mailto:aseldawy@gmail.com]
> > > Sent: Wednesday, December 30, 2009 11:56 AM
> > > To: java-user@lucene.apache.org
> > > Cc: solr-user@lucene.apache.org
> > > Subject: Re: Using the new tokenizer API from a jar file
> > >
> > > Thanks all for your interest, especially Uwe. I asked this question on
> > > solr-user at the beginning but I got no reply. That's why I re-asked
> the
> > > question at java-user.
> > > Thanks for your efforts. I will try it now.
> > >
> > > On Mon, Dec 28, 2009 at 12:02 PM, Uwe Schindler <uwe@thetaphi.de>
> wrote:
> > >
> > > > I opened https://issues.apache.org/jira/browse/LUCENE-2182 about
> this
> > > > problem and already have a fix.
> > > >
> > > > This is really a bug. The solution is simple because you have to
> load
> > > the
> > > > IMPL class using the same classloader as the passed in interface.
> The
> > > > default for Class.forName is the classloader of
> AttributeSource.class,
> > > > which
> > > > is the wrong one.
> > > >
> > > > -----
> > > > Uwe Schindler
> > > > H.-H.-Meier-Allee 63, D-28213 Bremen
> > > > http://www.thetaphi.de
> > > > eMail: uwe@thetaphi.de
> > > >
> > > > > -----Original Message-----
> > > > > From: Uwe Schindler [mailto:uwe@thetaphi.de]
> > > > > Sent: Monday, December 28, 2009 9:20 AM
> > > > > To: java-user@lucene.apache.org
> > > > > Cc: solr-user@lucene.apache.org
> > > > > Subject: RE: Using the new tokenizer API from a jar file
> > > > >
> > > > > The question on this list was ok,as it shows a minor problem of
> using
> > > the
> > > > > new TokenStream API with Solr.
> > > > >
> > > > > His plugin was loaded correctly, because if Lucene says, that it
> > > cannot
> > > > > find
> > > > > the *Impl class, it was able to load the interface class before ->
> > the
> > > > JAR
> > > > > file is "visible" to the JVM.
> > > > >
> > > > > The problem is the following and has to do with classloaders:
> > > > >
> > > > > 1. We have different class loaders for different places in Solr.
> Solr
> > > > uses
> > > > > for plugins a SolrResourceLoader that searches for JAR files in
> the
> > > local
> > > > > lib folder before handling over to the webapp's classloader.
> > > > >
> > > > > 2. Initially, the lucene JAR is loaded by the Webapp's class
> loader
> > > > >
> > > > > 3. If a AttributeImpl is placed into a jar file e.g. in the plugin
> > > folder
> > > > > of
> > > > > solr (the lib folder where solr loads all resources, stop
> words,...),
> > > the
> > > > > loading mechanism inside AttributeSource.DEFAULT_ATTRIBUTE_FACTORY
> is
> > > > > unable
> > > > > to locate the class file, because Class.forName() always uses the
> > > class'
> > > > > classloader and not the global/thread one's. So AttributeSource
> will
> > > only
> > > > > find the class file if it is in the *same* directory as the
> lucene-
> > > > > core.jar
> > > > > file (WEB-INF/lib) and so accessible by the webapp's class loader.
> > > > >
> > > > > A good introduction about the problem is this one:
> > > > >
> > > >
> > >
> >
> http://www.theserverside.com/tt/articles/content/dm_classForname/DynLoad.p
> > > > > df
> > > > >
> > > > > The problem is here described for the JVM extensions folder but
> also
> > > > > applies
> > > > > to solr, because it has another classloader for plugins.
> > > > >
> > > > > A solution to fix this would be in lucene to use the thread's
> context
> > > > > class
> > > > > loader in AttributeSource.DEFAULT_ATTRIBUTE_FACTORY, but I
> strongly
> > > > > discourage this, as it would break the whole AttributeSource
> > > > functionality
> > > > > if you add two different attributes with same class names from
> > > different
> > > > > class loaders to the AttributeSource.
> > > > >
> > > > > The only solution to the problem is placing the JAR file inside
> the
> > > > > WEB-INF/lib folder where lucene-core.jar is. Plugins in Solr
> cannot
> > > > define
> > > > > own attribute implementations. Alternatively he could try to force
> > > > preload
> > > > > the class by calling Class.forName in his plugin initialization
> code
> > > on
> > > > > the
> > > > > Impl class. But I am not sure if this works (as Java handles
> classes
> > > from
> > > > > different classloaders different).
> > > > >
> > > > > Uwe
> > > > >
> > > > > -----
> > > > > Uwe Schindler
> > > > > H.-H.-Meier-Allee 63, D-28213 Bremen
> > > > > http://www.thetaphi.de
> > > > > eMail: uwe@thetaphi.de
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Chris Hostetter [mailto:hossman_lucene@fucit.org]
> > > > > > Sent: Monday, December 28, 2009 4:27 AM
> > > > > > To: java-user@lucene.apache.org
> > > > > > Subject: Re: Using the new tokenizer API from a jar file
> > > > > >
> > > > > >
> > > > > > : I tried to use it with solr and the problems began. It's
> always
> > > > > telling
> > > > > > me
> > > > > > : that it cannot find the class GlossAttributeImpl. I think
the
> > > problem
> > > > > is
> > > > > > : that my jar file is added to the class path at run time not
> from
> > > the
> > > > > > command
> > > > > > : line. Do you have a good solution or workaround?
> > > > > >
> > > > > > You're likely to get mmore helpful answers from other people
in
> the
> > > > Solr
> > > > > > User community (solr-user@lucene.a.o)
> > > > > >
> > > > > > As long as you put your jar in the "lib" directory under your
> solr
> > > home
> > > > > > (or refrence it using a <lib/> directive in your solrconfig.xml)
> > > Solr's
> > > > > > plugin loader will take care of hte classloading for you.
> > > > > >
> > > > > > if you are confident you have your jar in the correct place,
> please
> > > > > email
> > > > > > solr-user with the ClassNotFound stack trace from your solr
> logs,
> > as
> > > > > well
> > > > > > as hierarchy  of files from your solr home (ie: the output of
> "find
> > > .")
> > > > > >
> > > > > >
> > > > > > -Hoss
> > > > > >
> > > > > >
> > > > > >
> > --------------------------------------------------------------------
> > > -
> > > > > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > > > > For additional commands, e-mail: java-user-
> help@lucene.apache.org
> > > > >
> > > > >
> > > > >
> > > > > ------------------------------------------------------------------
> ---
> > > > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > > >
> > > >
> > > >
> > > > --------------------------------------------------------------------
> -
> > > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > > >
> > > >
> > >
> > >
> > > --
> > > regards,
> > > Ahmed Saad
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> 
> 
> --
> regards,
> Ahmed Saad


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message