accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dlmar...@comcast.net
Subject Re: java.lang.NoClassDefFoundError with fields of custom Filter
Date Thu, 07 Jul 2016 18:54:57 GMT
+1 good catch Josh 

----- Original Message -----

From: "Josh Elser" <josh.elser@gmail.com> 
To: user@accumulo.apache.org 
Sent: Thursday, July 7, 2016 2:12:42 PM 
Subject: Re: java.lang.NoClassDefFoundError with fields of custom Filter 

Beware using the HDFS classloader in any Accumulo release that does not 
contain commons-vfs-2.1 as a dependency. 

Commons-vfs-2.0 has multiple known issues which prevent it from being 
usable in the more basic sense. 

Presently, there is only one release which contains the fix already: 
Accumulo 1.7.2. The upcoming 1.6.6 and 1.8.0 releases will also have the 
updated dependency. 

Massimilian Mattetti wrote: 
> Hi Jim, 
> 
> the approach of using namespace from HDFS looks promising. I need to 
> investigate a little on how it works but I guess I will take your advice. 
> Thank you. 
> 
> Cheers, 
> Massimiliano 
> 
> 
> 
> 
> From: James Hughes <jnh5y@virginia.edu> 
> To: user@accumulo.apache.org 
> Date: 07/07/2016 08:28 PM 
> Subject: Re: java.lang.NoClassDefFoundError with fields of custom Filter 
> ------------------------------------------------------------------------ 
> 
> 
> 
> Hi Massimiliano, 
> 
> I'm a fan of producing uber jars for this kind of thing; we do that for 
> GeoMesa. There is one gotcha which can come up: if you have several uber 
> jars in lib/ext, they can collide in rather unexpected ways. 
> 
> There are two options to call out: 
> 
> First, Accumulo has support for loading jars from HDFS into namespaces. 
> With that, you could have various namespaces for different versions or 
> different collections of iterator projects. If you are sharing a dev 
> cloud with other projects or co-workers working on the same project that 
> can be helpful since it would avoid restarts, etc. Big thumbs-up for 
> this approach! 
> 
> Second, rather than having an uber jar, you could build up zip files 
> with the various jars you need for your iterators and unzip them in 
> lib/ext. If you did that for multiple competing iterator projects, you'd 
> avoid duplication of code inside uber-jars. Also, you'd be able to see 
> if there are 8 versions of Log4J and Guava in lib/ext...;) It wouldn't 
> be as powerful as the namespace, but there's something nice about having 
> a low-tech approach. 
> 
> Others will likely have varied experiences; I'm not sure if there's an 
> established 'best practice' here. 
> 
> Cheers, 
> 
> Jim 
> 
> On Thu, Jul 7, 2016 at 12:56 PM, Massimilian Mattetti 
> <_MASSIMIL@il.ibm.com_ <mailto:MASSIMIL@il.ibm.com>> wrote: 
> Thanks for your prompt response, you are right Jim. There is a static 
> dependency to Log4J in my lexicoder. Adding the Log4J Jar to the 
> classpath solved the problem. 
> Would you suggest to use an Uber Jar to avoid this kind of problems? 
> 
> Regards, 
> Massimiliano 
> 
> 
> 
> 
> From: James Hughes <_jnh5y@virginia.edu_ <mailto:jnh5y@virginia.edu>> 
> To: _user@accumulo.apache.org_ <mailto:user@accumulo.apache.org> 
> Date: 07/07/2016 06:25 PM 
> Subject: Re: java.lang.NoClassDefFoundError with fields of custom Filter 
> ------------------------------------------------------------------------ 
> 
> 
> 
> 
> Hi Massimilian, 
> 
> As a quick note, your error says that it could not initialize class 
> accumulo.lexicoders.MyLexicoder. Did you provide all the dependencies 
> for your class on Accumulo's classpath? 
> 
> That exception (or similar) can occur if there is a static block in your 
> MyLexicoder class which can't run properly. 
> 
> Cheers, 
> 
> Jim 
> 
> 
> On Thu, Jul 7, 2016 at 11:19 AM, Massimilian Mattetti 
> <_MASSIMIL@il.ibm.com_ <mailto:MASSIMIL@il.ibm.com>> wrote: 
> Hi, 
> 
> I have implemented a custom filter and a custom lexicoder. Both this two 
> classes are packed in the same jar that has been deployed under the 
> directory $ACCUMULO_HOME/*lib*/*ext*of my Accumulo servers (version 
> 1.7.1). The lexicoder is used by the filter to get the real object from 
> the accumulo value and test some conditions on it. When I tried to scan 
> the table applying this filter I got the following exception: 
> 
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class 
> accumulo.lexicoders.MyLexicoder 
> at accumulo.filters.MyFilter.<init>(MyFilter.java:24) 
> at sun.reflect.GeneratedConstructorAccessor9.newInstance(Unknown Source) 
> at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 
> at java.lang.Class.newInstance(Class.java:442) 
> at 
> org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:261)

> at 
> org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:237)

> at 
> org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:218)

> at 
> org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:205)

> at 
> org.apache.accumulo.tserver.tablet.ScanDataSource.createIterator(ScanDataSource.java:193)

> at 
> org.apache.accumulo.tserver.tablet.ScanDataSource.iterator(ScanDataSource.java:127) 
> at 
> org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.seek(SourceSwitchingIterator.java:180)

> at org.apache.accumulo.tserver.tablet.Tablet.nextBatch(Tablet.java:880) 
> at org.apache.accumulo.tserver.tablet.Scanner.read(Scanner.java:98) 
> at org.apache.accumulo.tserver.scan.NextBatchTask.run(NextBatchTask.java:69) 
> at org.apache.htrace.wrappers.TraceRunnable.run(TraceRunnable.java:57) 
> ... 4 more 
> 
> 
> I do not undestand how it is possible that the class loader is able to 
> find the filter and no the lexicoder. Am I missing something? 
> 
> Thanks, 
> Massimiliano 
> 
> 
> 
> 
> 
> 
> 


Mime
View raw message