accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: java.lang.NoClassDefFoundError with fields of custom Filter
Date Thu, 07 Jul 2016 18:12:42 GMT
Beware using the HDFS classloader in any Accumulo release that does not 
contain commons-vfs-2.1 as a dependency.

Commons-vfs-2.0 has multiple known issues which prevent it from being 
usable in the more basic sense.

Presently, there is only one release which contains the fix already: 
Accumulo 1.7.2. The upcoming 1.6.6 and 1.8.0 releases will also have the 
updated dependency.

Massimilian Mattetti wrote:
> Hi Jim,
>
> the approach of using namespace from HDFS looks promising. I need to
> investigate a little on how it works but I guess I will take your advice.
> Thank you.
>
> Cheers,
> Massimiliano
>
>
>
>
> From: James Hughes <jnh5y@virginia.edu>
> To: user@accumulo.apache.org
> Date: 07/07/2016 08:28 PM
> Subject: Re: java.lang.NoClassDefFoundError with fields of custom Filter
> ------------------------------------------------------------------------
>
>
>
> Hi Massimiliano,
>
> I'm a fan of producing uber jars for this kind of thing; we do that for
> GeoMesa. There is one gotcha which can come up: if you have several uber
> jars in lib/ext, they can collide in rather unexpected ways.
>
> There are two options to call out:
>
> First, Accumulo has support for loading jars from HDFS into namespaces.
> With that, you could have various namespaces for different versions or
> different collections of iterator projects. If you are sharing a dev
> cloud with other projects or co-workers working on the same project that
> can be helpful since it would avoid restarts, etc. Big thumbs-up for
> this approach!
>
> Second, rather than having an uber jar, you could build up zip files
> with the various jars you need for your iterators and unzip them in
> lib/ext. If you did that for multiple competing iterator projects, you'd
> avoid duplication of code inside uber-jars. Also, you'd be able to see
> if there are 8 versions of Log4J and Guava in lib/ext...;) It wouldn't
> be as powerful as the namespace, but there's something nice about having
> a low-tech approach.
>
> Others will likely have varied experiences; I'm not sure if there's an
> established 'best practice' here.
>
> Cheers,
>
> Jim
>
> On Thu, Jul 7, 2016 at 12:56 PM, Massimilian Mattetti
> <_MASSIMIL@il.ibm.com_ <mailto:MASSIMIL@il.ibm.com>> wrote:
> Thanks for your prompt response, you are right Jim. There is a static
> dependency to Log4J in my lexicoder. Adding the Log4J Jar to the
> classpath solved the problem.
> Would you suggest to use an Uber Jar to avoid this kind of problems?
>
> Regards,
> Massimiliano
>
>
>
>
> From: James Hughes <_jnh5y@virginia.edu_ <mailto:jnh5y@virginia.edu>>
> To: _user@accumulo.apache.org_ <mailto:user@accumulo.apache.org>
> Date: 07/07/2016 06:25 PM
> Subject: Re: java.lang.NoClassDefFoundError with fields of custom Filter
> ------------------------------------------------------------------------
>
>
>
>
> Hi Massimilian,
>
> As a quick note, your error says that it could not initialize class
> accumulo.lexicoders.MyLexicoder. Did you provide all the dependencies
> for your class on Accumulo's classpath?
>
> That exception (or similar) can occur if there is a static block in your
> MyLexicoder class which can't run properly.
>
> Cheers,
>
> Jim
>
>
> On Thu, Jul 7, 2016 at 11:19 AM, Massimilian Mattetti
> <_MASSIMIL@il.ibm.com_ <mailto:MASSIMIL@il.ibm.com>> wrote:
> Hi,
>
> I have implemented a custom filter and a custom lexicoder. Both this two
> classes are packed in the same jar that has been deployed under the
> directory $ACCUMULO_HOME/*lib*/*ext*of my Accumulo servers (version
> 1.7.1). The lexicoder is used by the filter to get the real object from
> the accumulo value and test some conditions on it. When I tried to scan
> the table applying this filter I got the following exception:
>
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> accumulo.lexicoders.MyLexicoder
> at accumulo.filters.MyFilter.<init>(MyFilter.java:24)
> at sun.reflect.GeneratedConstructorAccessor9.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at java.lang.Class.newInstance(Class.java:442)
> at
> org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:261)
> at
> org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:237)
> at
> org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:218)
> at
> org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:205)
> at
> org.apache.accumulo.tserver.tablet.ScanDataSource.createIterator(ScanDataSource.java:193)
> at
> org.apache.accumulo.tserver.tablet.ScanDataSource.iterator(ScanDataSource.java:127)
> at
> org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.seek(SourceSwitchingIterator.java:180)
> at org.apache.accumulo.tserver.tablet.Tablet.nextBatch(Tablet.java:880)
> at org.apache.accumulo.tserver.tablet.Scanner.read(Scanner.java:98)
> at org.apache.accumulo.tserver.scan.NextBatchTask.run(NextBatchTask.java:69)
> at org.apache.htrace.wrappers.TraceRunnable.run(TraceRunnable.java:57)
> ... 4 more
>
>
> I do not undestand how it is possible that the class loader is able to
> find the filter and no the lexicoder. Am I missing something?
>
> Thanks,
> Massimiliano
>
>
>
>
>
>
>

Mime
View raw message