accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From William Slacum <wsla...@gmail.com>
Subject Re: java.lang.NoClassDefFoundError with fields of custom Filter
Date Thu, 07 Jul 2016 18:19:46 GMT
You could also shade/relocate dependency classes within the uber/fat jar.
It has pitfalls but it is very easy to set up.

On Thursday, July 7, 2016, Massimilian Mattetti <MASSIMIL@il.ibm.com> wrote:

> Hi Jim,
>
> the approach of using namespace from HDFS looks promising. I need to
> investigate a little on how it works but I guess I will take your advice.
> Thank you.
>
> Cheers,
> Massimiliano
>
>
>
>
> From:        James Hughes <jnh5y@virginia.edu
> <javascript:_e(%7B%7D,'cvml','jnh5y@virginia.edu');>>
> To:        user@accumulo.apache.org
> <javascript:_e(%7B%7D,'cvml','user@accumulo.apache.org');>
> Date:        07/07/2016 08:28 PM
> Subject:        Re: java.lang.NoClassDefFoundError with fields of custom
> Filter
> ------------------------------
>
>
>
> Hi Massimiliano,
>
> I'm a fan of producing uber jars for this kind of thing; we do that for
> GeoMesa.  There is one gotcha which can come up:  if you have several uber
> jars in lib/ext, they can collide in rather unexpected ways.
>
> There are two options to call out:
>
> First, Accumulo has support for loading jars from HDFS into namespaces.
> With that, you could have various namespaces for different versions or
> different collections of iterator projects.  If you are sharing a dev cloud
> with other projects or co-workers working on the same project that can be
> helpful since it would avoid restarts, etc.  Big thumbs-up for this
> approach!
>
> Second, rather than having an uber jar, you could build up zip files with
> the various jars you need for your iterators and unzip them in lib/ext.  If
> you did that for multiple competing iterator projects, you'd avoid
> duplication of code inside uber-jars.  Also, you'd be able to see if there
> are 8 versions of Log4J and Guava in lib/ext...;)  It wouldn't be as
> powerful as the namespace, but there's something nice about having a
> low-tech approach.
>
> Others will likely have varied experiences; I'm not sure if there's an
> established 'best practice' here.
>
> Cheers,
>
> Jim
>
> On Thu, Jul 7, 2016 at 12:56 PM, Massimilian Mattetti <
> *MASSIMIL@il.ibm.com*
> <javascript:_e(%7B%7D,'cvml','MASSIMIL@il.ibm.com');>> wrote:
> Thanks for your prompt response, you are right Jim. There is a static
> dependency to Log4J in my lexicoder. Adding the Log4J Jar to the classpath
> solved the problem.
> Would you suggest to use an Uber Jar to avoid this kind of problems?
>
> Regards,
> Massimiliano
>
>
>
>
> From:        James Hughes <*jnh5y@virginia.edu*
> <javascript:_e(%7B%7D,'cvml','jnh5y@virginia.edu');>>
> To:        *user@accumulo.apache.org*
> <javascript:_e(%7B%7D,'cvml','user@accumulo.apache.org');>
> Date:        07/07/2016 06:25 PM
> Subject:        Re: java.lang.NoClassDefFoundError with fields of custom
> Filter
> ------------------------------
>
>
>
>
> Hi Massimilian,
>
> As a quick note, your error says that it could not initialize class
> accumulo.lexicoders.MyLexicoder.  Did you provide all the dependencies for
> your class on Accumulo's classpath?
>
> That exception (or similar) can occur if there is a static block in your
> MyLexicoder class which can't run properly.
>
> Cheers,
>
> Jim
>
>
> On Thu, Jul 7, 2016 at 11:19 AM, Massimilian Mattetti <
> *MASSIMIL@il.ibm.com*
> <javascript:_e(%7B%7D,'cvml','MASSIMIL@il.ibm.com');>> wrote:
> Hi,
>
> I have implemented a custom filter and a custom lexicoder. Both this two
> classes are packed in the same jar that has been deployed under the
> directory $ACCUMULO_HOME/*lib*/*ext*of my Accumulo servers (version
> 1.7.1). The lexicoder is used by the filter to get the real object from the
> accumulo value and test some conditions on it. When I tried to scan the
> table applying this filter I got the following exception:
>
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> accumulo.lexicoders.MyLexicoder
>         at accumulo.filters.MyFilter.<init>(MyFilter.java:24)
>         at sun.reflect.GeneratedConstructorAccessor9.newInstance(Unknown
> Source)
>         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
>         at java.lang.Class.newInstance(Class.java:442)
>         at
> org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:261)
>         at
> org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:237)
>         at
> org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:218)
>         at
> org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:205)
>         at
> org.apache.accumulo.tserver.tablet.ScanDataSource.createIterator(ScanDataSource.java:193)
>         at
> org.apache.accumulo.tserver.tablet.ScanDataSource.iterator(ScanDataSource.java:127)
>         at
> org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.seek(SourceSwitchingIterator.java:180)
>         at
> org.apache.accumulo.tserver.tablet.Tablet.nextBatch(Tablet.java:880)
>         at org.apache.accumulo.tserver.tablet.Scanner.read(Scanner.java:98)
>         at
> org.apache.accumulo.tserver.scan.NextBatchTask.run(NextBatchTask.java:69)
>         at
> org.apache.htrace.wrappers.TraceRunnable.run(TraceRunnable.java:57)
>         ... 4 more
>
>
>  I do not undestand how it is possible that the class loader is able to
> find the filter and no the lexicoder. Am I missing something?
>
> Thanks,
> Massimiliano
>
>
>
>
>
>
>
>

Mime
View raw message