Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 09821C3F6 for ; Tue, 22 May 2012 16:43:32 +0000 (UTC) Received: (qmail 37053 invoked by uid 500); 22 May 2012 16:43:31 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 37028 invoked by uid 500); 22 May 2012 16:43:31 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 37018 invoked by uid 99); 22 May 2012 16:43:31 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 May 2012 16:43:31 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of Bob.Thorman@l-3com.com designates 166.20.51.112 as permitted sender) Received: from [166.20.51.112] (HELO smtp2out.l-3com.com) (166.20.51.112) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 May 2012 16:43:22 +0000 X-filenames: X-filesizes: None X-filetypes: X-IronPort-AV: E=Sophos;i="4.75,639,1330905600"; d="scan'208,217";a="79947463" From: Bob.Thorman@l-3com.com Received: from host-128-170-184-25.l-3com.com (HELO mx1.Comcept.L-3Com.com) ([128.170.184.25]) by smtp2out.l-3com.com with ESMTP; 22 May 2012 16:43:00 +0000 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01CD3839.F300760D" X-MimeOLE: Produced By Microsoft Exchange V6.5 Subject: RE: AcculumoFileOutputFormat class cannot be found by child jvm Date: Tue, 22 May 2012 11:41:10 -0500 Message-ID: <97EB0FF1279CC5428640A3FB61B10BD602B61F6C@mx1.Comcept.L-3Com.com> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: AcculumoFileOutputFormat class cannot be found by child jvm Thread-Index: Ac04K1Pz4L3ORkCaSd2AUhHn7kiZhQADYCYw References: <97EB0FF1279CC5428640A3FB61B10BD602B61ED9@mx1.Comcept.L-3Com.com> <4FBBA763.1070808@ccri.com> <1317250116.150171.1337698501679.JavaMail.root@linzimmb04o.imo.intelink.gov> To: This is a multi-part message in MIME format. ------_=_NextPart_001_01CD3839.F300760D Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Yep. Here's the script I'm using...everything is happy until the job executes under the configuration that uses AccumuloFileOutputFormat class... =20 HADOOP_BIN=3D/cloudbase/hadoop-0.20.2/bin ACCUMULO_BIN=3D/cloudbase/accumulo-1.4.0/bin =20 INGESTER_JAR=3D/mnt/hgfs/CSI.Cloudbase/Java/CloudbaseServices/out/artifac= t s/CloudbaseIngesters/CloudbaseIngesters.jar PLACEMARK_CLASS=3Dcom.comcept.cloudbase.ingesters.placemarks.PlacemarkIng= e ster CONFIG=3D/mnt/hgfs/CSI.Cloudbase/Java/CloudbaseServices/out/artifacts/Clo= u dbaseIngesters/placemark-config.xml =20 KXML_JAR=3D/usr/lib/ncct/kxml2-2.3.0.jar XMLPULL_JAR=3D/usr/lib/ncct/xmlpull-1.1.3.1.jar XSTREAM_JAR=3D/usr/lib/ncct/xstream-1.4.1.jar =20 INGESTER_LIBS=3D$KXML_JAR,$XMLPULL_JAR,$XSTREAM_JAR =20 $HADOOP_BIN/hadoop dfs -ls / $HADOOP_BIN/hadoop dfs -rmr /output $HADOOP_BIN/hadoop dfs -rmr /input $HADOOP_BIN/hadoop dfs -mkdir /input $HADOOP_BIN/hadoop dfs -mkdir /output $HADOOP_BIN/hadoop dfs -mkdir /output/pfailures $HADOOP_BIN/hadoop dfs -mkdir /output/gfailures $HADOOP_BIN/hadoop dfs -mkdir /output/efailures $HADOOP_BIN/hadoop dfs -mkdir /output/tfailures $HADOOP_BIN/hadoop dfs -put ./*.kml /input =20 $ACCUMULO_BIN/tool.sh $INGESTER_JAR $PLACEMARK_CLASS -libjars $INGESTER_LIBS -c $CONFIG =20 =20 Here is the code that initializes the first job in the chain... =20 conf.set(_sVisTag, ic.getVisibility()); =20 Job job =3D new Job(conf, "NCCT Placemark Ingester"); job.setJarByClass(this.getClass()); job.setInputFormatClass(TextInputFormat.class); job.setMapperClass(PlacemarkMapClass.class); job.setMapOutputKeyClass(Text.class); job.setMapOutputValueClass(Text.class); job.setReducerClass(PlacemarkReduceClass.class); =20 job.setOutputFormatClass(AccumuloFileOutputFormat.class); =20 AccumuloFileOutputFormat.setZooKeeperInstance(conf, ic.getInstance(), ic.getZooKeeper()); Instance instance =3D new ZooKeeperInstance(ic.getInstance(), ic.getZooKeeper()); Connector connector =3D instance.getConnector(ic.getUserName(), password); TextInputFormat.setInputPaths(job,new Path(ic.getHdfsInput())); AccumuloFileOutputFormat.setOutputPath(job, new Path(ic.getHdfsOutput() + "/pfiles")); =20 job.waitForCompletion(true); =20 connector.tableOperations().importDirectory(ic.getMetaTable(), ic.getHdfsOutput() + "/pfiles", ic.getHdfsOutput() + "/pfailures", false); =20 From: John Vines [mailto:john.w.vines@ugov.gov]=20 Sent: Tuesday, May 22, 2012 09:57 To: user@accumulo.apache.org Subject: Re: AcculumoFileOutputFormat class cannot be found by child jvm =20 Does your script utilize $ACCUMULO_HOME/bin/tool.sh to kick off the mapreduce? That script is similar to hadoop jar, but it will libjar the accumulo libraries for you. John On Tue, May 22, 2012 at 10:55 AM, wrote: Right now I'm using stand-alone mode, but is there another place I need to put the jar file? -----Original Message----- From: John Armstrong [mailto:jrja@ccri.com] Sent: Tuesday, May 22, 2012 09:49 To: user@accumulo.apache.org Subject: Re: AcculumoFileOutputFormat class cannot be found by child jvm On 05/22/2012 10:40 AM, Bob.Thorman@l-3com.com wrote: > I upgrade to accumulo-1.4.0 and updated my map/reduce jobs and now > they don't run. The parent class path has the accumulo-core-1.4.0.jar > file included. Do the accumulo jar files have to be manually put on a > distribute cache? Any help is appreciated. Just to check: did you replace the Accumulo JAR files on all the cluster nodes? =20 ------_=_NextPart_001_01CD3839.F300760D Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Yep.  Here’s the script I’m using…everything = is happy until the job executes under the configuration that uses = AccumuloFileOutputFormat class…

 

HADOOP_BIN=3D/cloudbase/hadoop-0.20.2/bin

ACCUMULO_BIN=3D/cloudbase/accumulo-1.4.0/bin

 

INGESTER_JAR=3D/mnt/hgfs/CSI.Cloudbase/Java/CloudbaseServices/out/arti= facts/CloudbaseIngesters/CloudbaseIngesters.jar

PLACEMARK_CLASS=3Dcom.comcept.cloudbase.ingesters.placemarks.Placemark= Ingester

CONFIG=3D/mnt/hgfs/CSI.Cloudbase/Java/CloudbaseServices/out/artifacts/= CloudbaseIngesters/placemark-config.xml

 

KXML_JAR=3D/usr/lib/ncct/kxml2-2.3.0.jar

XMLPULL_JAR=3D/usr/lib/ncct/xmlpull-1.1.3.1.jar

<= p class=3DMsoNormal>XSTREAM_JAR=3D/usr/lib/ncct/xstream-1.4.1.jar

 

INGESTER_LIBS=3D$KXML_JAR,$XMLPULL_JAR,$XSTREAM_JAR<= /p>

 

$HADOOP_BIN/hadoop dfs -ls /

$HADOOP_BIN/hadoop dfs -rmr /output

$HADOOP_BIN/hadoop dfs -rmr /input

$HADOOP_BIN/hadoop dfs -mkdir /input

$HADOOP_BIN/hadoop dfs -mkdir /output

$HADOOP_BIN/hadoop dfs -mkdir = /output/pfailures

$HADOOP_BIN/hadoop dfs -mkdir = /output/gfailures

$HADOOP_BIN/hadoop dfs -mkdir = /output/efailures

$HADOOP_BIN/hadoop dfs -mkdir = /output/tfailures

$HADOOP_BIN/hadoop dfs -put ./*.kml /input

 

$ACCUMULO_BIN/tool.sh $INGESTER_JAR $PLACEMARK_CLASS -libjars = $INGESTER_LIBS -c $CONFIG

 

 

Here is the code that initializes the first job in the = chain…

 

            =     conf.set(_sVisTag, = ic.getVisibility());

 

Job job =3D new Job(conf, "NCCT Placemark = Ingester");

job.setJarByClass(this.getClass());

           &nbs= p;    = job.setInputFormatClass(TextInputFormat.class);

           &nbs= p;    = job.setMapperClass(PlacemarkMapClass.class);

           &nbs= p;    = job.setMapOutputKeyClass(Text.class);

           &nbs= p;    = job.setMapOutputValueClass(Text.class);

           &nbs= p;    = job.setReducerClass(PlacemarkReduceClass.class);

           &nbs= p;    = job.setOutputFormatClass(AccumuloFileOutputFormat.class);

 

AccumuloFileOutputFormat.setZooKeeperInstance(conf, ic.getInstance(), = ic.getZooKeeper());

           &nbs= p;    Instance instance =3D new = ZooKeeperInstance(ic.getInstance(), = ic.getZooKeeper());

           &nbs= p;    Connector connector =3D = instance.getConnector(ic.getUserName(), = password);

           &nbs= p;    TextInputFormat.setInputPaths(job,new = Path(ic.getHdfsInput()));

           &nbs= p;    AccumuloFileOutputFormat.setOutputPath(job, new = Path(ic.getHdfsOutput() + "/pfiles"));

 

           &nbs= p;    = job.waitForCompletion(true);

           &nbs= p;    = connector.tableOperations().importDirectory(ic.getMetaTable(), = ic.getHdfsOutput() + "/pfiles", ic.getHdfsOutput() + = "/pfailures", false);

 

From:= = John Vines [mailto:john.w.vines@ugov.gov]
Sent: Tuesday, May = 22, 2012 09:57
To: user@accumulo.apache.org
Subject: = Re: AcculumoFileOutputFormat class cannot be found by child = jvm

 

Does your script utilize = $ACCUMULO_HOME/bin/tool.sh to kick off the mapreduce? That script is = similar to hadoop jar, but it will libjar the accumulo libraries for = you.

John

On Tue, May 22, = 2012 at 10:55 AM, <Bob.Thorman@l-3com.com> wrote:

Right now I'm using stand-alone mode, but is there = another place I need
to put the jar file?


-----Original = Message-----
From: John Armstrong [mailto:jrja@ccri.com]
Sent: Tuesday, May = 22, 2012 09:49
To: user@accumulo.apache.org
= Subject: Re: AcculumoFileOutputFormat class cannot be found by child = jvm

On 05/22/2012 10:40 AM, Bob.Thorman@l-3com.com = wrote:
> I upgrade to accumulo-1.4.0 and updated my map/reduce = jobs and now
> they don't run.  The parent class path has the = accumulo-core-1.4.0.jar

> file included.  Do the accumulo = jar files have to be manually put on a

> distribute cache? =  Any help is appreciated.

Just to check: did you replace the = Accumulo JAR files on all the = cluster
nodes?

 

------_=_NextPart_001_01CD3839.F300760D--