Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 97F2511C84 for ; Mon, 16 Jun 2014 15:37:06 +0000 (UTC) Received: (qmail 70642 invoked by uid 500); 16 Jun 2014 15:37:01 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 70597 invoked by uid 500); 16 Jun 2014 15:37:01 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 70587 invoked by uid 99); 16 Jun 2014 15:37:01 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 Jun 2014 15:37:01 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of vicky.kak@gmail.com designates 209.85.213.51 as permitted sender) Received: from [209.85.213.51] (HELO mail-yh0-f51.google.com) (209.85.213.51) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 Jun 2014 15:36:56 +0000 Received: by mail-yh0-f51.google.com with SMTP id f10so4413410yha.38 for ; Mon, 16 Jun 2014 08:36:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=D5uke+AtrhWKKF5csfFYZbPX4SyP6i8ZitD9B1EfKXg=; b=nnRpxfqoaxdxOjMDWPuIUp91Vcn3/o4HHSrwkjty2ixLIyIEYZA+hPQ+vri4TCGwIM QE6iznwZxgyrSyDRJdaGdyBqcKLxXwz7vfvJi7ozs3dBm47hO58/p6/gSKd06J9JQjb9 pPaYEsRU1pZi4cZr0syXA0z4R/J/kWgLLKlLFCskTtwU6pynOkfWZvVxx/G8k9i51gO+ A4rKqxFVAIcUCW61aoU5k/t7J4LCm1N9sjOzA6YSj2O6rgpjdfVvrR0/5TliaLxMYVei SunA2DFHCzKKq0kGVcq+oKwPxVI62dKQ//W7S1wyxRP73eRI8uhEeIUIjemQu9osyKI9 Zn3A== MIME-Version: 1.0 X-Received: by 10.236.229.133 with SMTP id h5mr35231465yhq.64.1402932996124; Mon, 16 Jun 2014 08:36:36 -0700 (PDT) Received: by 10.170.135.147 with HTTP; Mon, 16 Jun 2014 08:36:36 -0700 (PDT) In-Reply-To: References: Date: Mon, 16 Jun 2014 21:06:36 +0530 Message-ID: Subject: Re: Need help. Error: java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.encodeBase64String From: Vicky Kak To: user@accumulo.apache.org Content-Type: multipart/alternative; boundary=089e0160c36ae0b6a504fbf5ca3c X-Virus-Checked: Checked by ClamAV on apache.org --089e0160c36ae0b6a504fbf5ca3c Content-Type: text/plain; charset=UTF-8 >> at com.paypal.rtgraph.demo.MapReduceWriter.main(MapReduceWriter.scala) Can you add the following piece of code in MapReduceWriter, System.out.println(" codec location "+org.apache.commons.codec. binary.Base64.class.getProtectionDomain().getCodeSource()); This should give the jar location of the apache coder librabry, the error that you get in the info says method not found in the class possibly due to version mismatch. On Mon, Jun 16, 2014 at 2:49 PM, Jianshi Huang wrote: > Hi, > > I'm trying to use Accumulo with Spark writing to AccumuloOutputFormat. I > got the following errors in my spark app log: > > 14/06/16 02:01:44 INFO cluster.YarnClientClusterScheduler: > YarnClientClusterScheduler.postStartHook done > Exception in thread "main" java.lang.NoSuchMethodError: > org.apache.commons.codec.binary.Base64.encodeBase64String([B)Ljava/lang/String; > at > org.apache.accumulo.core.client.mapreduce.lib.impl.ConfiguratorBase.setConnectorInfo(ConfiguratorBase.java:127) > at > org.apache.accumulo.core.client.mapreduce.AccumuloOutputFormat.setConnectorInfo(AccumuloOutputFormat.java:92) > at > com.paypal.rtgraph.demo.MapReduceWriter$.main(MapReduceWriter.scala:44) > at > com.paypal.rtgraph.demo.MapReduceWriter.main(MapReduceWriter.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:292) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:55) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > > Anyone knows what's wrong with my code or settings? I've added all jars to > spark's classpath. > > And here's my spark-submit cmd: > > spark-submit --class com.paypal.rtgraph.demo.MapReduceWriter --master > yarn-client --jars `find lib -type f | tr '\n' ','` --driver-memory 2G > --executor-memory 20G --executor-cores 8 --num-executors 2 --verbose > rtgraph.jar > > If anyone has Accumulo+Spark examples, it would be great if they could be > shared. :) > > > Cheers, > > -- > Jianshi Huang > > LinkedIn: jianshi > Twitter: @jshuang > Github & Blog: http://huangjs.github.com/ > --089e0160c36ae0b6a504fbf5ca3c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 at com.paypal.rtg= raph.demo.MapReduceWriter.main(MapReduceWriter.scala)

Can= you add the following piece of code in MapReduceWriter,

System.out.= println(" codec location "+org.apache.commons.codec.
binary.Base64.class.getProtectionDomain().getCodeSource());

This should give the jar location of the apache coder librabry, the erro= r that you get in the info says method not found in the class possibly due = to version mismatch.



On Mon, Jun 16, 2014 at 2:49 PM, Jianshi Huang &= lt;jianshi.hua= ng@gmail.com> wrote:
Hi,

I'= m trying to use Accumulo with Spark writing to AccumuloOutputFormat. I got = the following errors in my spark app log:

14/06/16 02:01:44= INFO cluster.YarnClientClusterScheduler: YarnClientClusterScheduler.postSt= artHook done
Exception in thread "main" java.lang.NoSuchMethodError: org.= apache.commons.codec.binary.Base64.encodeBase64String([B)Ljava/lang/String;=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.accumulo.core.client.m= apreduce.lib.impl.ConfiguratorBase.setConnectorInfo(ConfiguratorBase.java:1= 27)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.accumulo.core.client.mapredu= ce.AccumuloOutputFormat.setConnectorInfo(AccumuloOutputFormat.java:92)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at com.paypal.rtgraph.demo.MapReduceWrite= r$.main(MapReduceWriter.scala:44)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at com.paypal.rtgraph.demo.MapReduceWriter= .main(MapReduceWriter.scala)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at sun.r= eflect.NativeMethodAccessorImpl.invoke0(Native Method)
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMeth= odAccessorImpl.java:57)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at sun.reflect.DelegatingMethodAccessorImp= l.invoke(DelegatingMethodAccessorImpl.java:43)
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 at java.lang.reflect.Method.invoke(Method.java:606)
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.spark.deploy.SparkSubmit$.launch(= SparkSubmit.scala:292)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.spark.deploy.SparkSubmit$.ma= in(SparkSubmit.scala:55)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apach= e.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Anyone knows what's wrong with my code or settings? I've added all= jars to spark's classpath.

And here's my spark-submit cmd:

spark-submit --class com.paypal.rtgraph.demo.MapReduceWriter --master= yarn-client --jars `find lib -type f | tr '\n' ','` --driv= er-memory 2G --executor-memory 20G --executor-cores 8 --num-executors 2 --v= erbose rtgraph.jar

If anyone has Accumulo+Spark examples, it would b= e great if they could be shared. :)


Cheers,
--
Jianshi Huang

LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/

--089e0160c36ae0b6a504fbf5ca3c--