accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: AcculumoFileOutputFormat class cannot be found by child jvm
Date Tue, 22 May 2012 18:03:59 GMT
I think it should be in the job.xml file.  The way the class path for
the java program that starts map reduce is configured is different
from how the classpath for the remote task are configured.   So your
program that starts the map reduce job has AccumuloFileOutputFormat on
the class path because you use a static method on it and it runs.  So
I am thinking its not in --libjars for some reason or maybe you are
not using Hadoop Tool?

On Tue, May 22, 2012 at 1:54 PM,  <Bob.Thorman@l-3com.com> wrote:
> No, it's not in the job.xml file.  A reference to AccumuloFileOutputFormat is but not
the accumulo-core-1.4.0.jar.  The Job*.jar file is referenced there.  I was under the impression
that the child jvm inherits the parent jvm attributes which makes the job*.jar file.  Is
that not correct?
>
> -----Original Message-----
> From: Keith Turner [mailto:keith@deenlo.com]
> Sent: Tuesday, May 22, 2012 12:07
> To: user@accumulo.apache.org
> Subject: Re: AcculumoFileOutputFormat class cannot be found by child jvm
>
> If you look at Job.xml, do you see accumulo-core there?  There should be a link to this
file on the jobs page on the tasktracker.
>
> On Tue, May 22, 2012 at 10:40 AM,  <Bob.Thorman@l-3com.com> wrote:
>> I upgrade to accumulo-1.4.0 and updated my map/reduce jobs and now
>> they don't run.  The parent class path has the accumulo-core-1.4.0.jar
>> file included.  Do the accumulo jar files have to be manually put on a
>> distribute cache?  Any help is appreciated.
>>
>> [hadoop@redhat-cloudbase1 placemarks]$ ./runPlacemarkIngester.sh Found
>> 5 items drwxrwxr-x   - hadoop hadoop          0 2012-05-21 14:13
>> /accumulo drwxrwxr-x   - hadoop hadoop          0 2012-05-21 15:06
>> /data drwxr-xr-x   - hadoop hadoop          0 2012-05-22 08:58 /input
>> drwxr-xr-x   - hadoop hadoop          0 2012-05-22 08:58 /output
>> drwxrwxr-x   - hadoop hadoop          0 2012-05-21 14:34 /usr Deleted
>> hdfs://redhat-cloudbase1:9000/output
>> Deleted hdfs://redhat-cloudbase1:9000/input
>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: Invoking
>> ToolRunner.run
>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: zooKeeper is
>> redhat-cloudbase1:2181
>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: instanceName is
>> NCCT-Cloudbase
>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: timeTableName is
>> NCCTServicesTimes
>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: geoTableName is
>> NCCTServicesGeos
>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: metadataTableName
>> is NCCTServicesMetadata
>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: edgeTableName is
>> NCCTEdgeTable
>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: userName is
>> NCCT.Services.Client
>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: password is
>> *********
>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: visibility is
>> public,BD2,UNCLASSIFIED-NO_COMPARTMENT-UNRESTRICTED
>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: inputDir is
>> /input
>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: outputDir is
>> /output
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48
>> GMT
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:host.name=redhat-cloudbase1
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:java.version=1.6.0_32
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:java.vendor=Sun Microsystems Inc.
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:java.home=/usr/java/jdk1.6.0_32/jre
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:java.class.path=/cloudbase/hadoop-0.20.2/bin/../conf:/usr/
>> ja
>> va/jdk1.6.0_32/lib/tools.jar:/cloudbase/hadoop-0.20.2/bin/..:/cloudbas
>> e/
>> hadoop-0.20.2/bin/../hadoop-0.20.2-core.jar:/cloudbase/hadoop-0.20.2/b
>> in
>> /../lib/commons-cli-1.2.jar:/cloudbase/hadoop-0.20.2/bin/../lib/common
>> s-
>> codec-1.3.jar:/cloudbase/hadoop-0.20.2/bin/../lib/commons-el-1.0.jar:/
>> cl
>> oudbase/hadoop-0.20.2/bin/../lib/commons-httpclient-3.0.1.jar:/cloudba
>> se
>> /hadoop-0.20.2/bin/../lib/commons-logging-1.0.4.jar:/cloudbase/hadoop-0.
>> 20.2/bin/../lib/commons-logging-api-1.0.4.jar:/cloudbase/hadoop-0.20.2
>> /b
>> in/../lib/commons-net-1.4.1.jar:/cloudbase/hadoop-0.20.2/bin/../lib/co
>> re
>> -3.1.1.jar:/cloudbase/hadoop-0.20.2/bin/../lib/hadoop-0.20.2-ant.jar:/
>> cl
>> oudbase/hadoop-0.20.2/bin/../lib/hadoop-0.20.2-core.jar:/cloudbase/had
>> oo
>> p-0.20.2/bin/../lib/hadoop-0.20.2-examples.jar:/cloudbase/hadoop-0.20.
>> 2/
>> bin/../lib/hadoop-0.20.2-test.jar:/cloudbase/hadoop-0.20.2/bin/../lib/
>> ha
>> doop-0.20.2-tools.jar:/cloudbase/hadoop-0.20.2/bin/../lib/hsqldb-1.8.0
>> .1
>> 0.jar:/cloudbase/hadoop-0.20.2/bin/../lib/jasper-compiler-5.5.12.jar:/
>> cl
>> oudbase/hadoop-0.20.2/bin/../lib/jasper-runtime-5.5.12.jar:/cloudbase/
>> ha
>> doop-0.20.2/bin/../lib/jets3t-0.6.1.jar:/cloudbase/hadoop-0.20.2/bin/.
>> ./
>> lib/jetty-6.1.14.jar:/cloudbase/hadoop-0.20.2/bin/../lib/jetty-util-6.1.
>> 14.jar:/cloudbase/hadoop-0.20.2/bin/../lib/junit-3.8.1.jar:/cloudbase/
>> ha
>> doop-0.20.2/bin/../lib/kfs-0.2.2.jar:/cloudbase/hadoop-0.20.2/bin/../l
>> ib
>> /log4j-1.2.15.jar:/cloudbase/hadoop-0.20.2/bin/../lib/mockito-all-1.8.0.
>> jar:/cloudbase/hadoop-0.20.2/bin/../lib/oro-2.0.8.jar:/cloudbase/hadoo
>> p-
>> 0.20.2/bin/../lib/servlet-api-2.5-6.1.14.jar:/cloudbase/hadoop-0.20.2/
>> bi
>> n/../lib/slf4j-api-1.4.3.jar:/cloudbase/hadoop-0.20.2/bin/../lib/slf4j
>> -l
>> og4j12-1.4.3.jar:/cloudbase/hadoop-0.20.2/bin/../lib/xmlenc-0.52.jar:/
>> cl
>> oudbase/hadoop-0.20.2/bin/../lib/jsp-2.1/jsp-2.1.jar:/cloudbase/hadoop
>> -0
>> .20.2/bin/../lib/jsp-2.1/jsp-api-2.1.jar:/cloudbase/accumulo-1.4.0/lib
>> /l
>> ibthrift-0.6.1.jar:/cloudbase/accumulo-1.4.0/lib/accumulo-core-1.4.0.j
>> ar
>> :/cloudbase/zookeeper-3.4.3/zookeeper-3.4.3.jar:/cloudbase/accumulo-1.4.
>> 0/lib/cloudtrace-1.4.0.jar:/usr/lib/ncct/kxml2-2.3.0.jar:/usr/lib/ncct
>> /x
>> mlpull-1.1.3.1.jar:/usr/lib/ncct/xstream-1.4.1.jar:/cloudbase/accumulo
>> -1
>> .4.0/lib/commons-collections-3.2.jar:/cloudbase/accumulo-1.4.0/lib/com
>> mo
>> ns-configuration-1.5.jar:/cloudbase/accumulo-1.4.0/lib/commons-io-1.4.
>> ja
>> r:/cloudbase/accumulo-1.4.0/lib/commons-jci-core-1.0.jar:/cloudbase/ac
>> cu
>> mulo-1.4.0/lib/commons-jci-fam-1.0.jar:/cloudbase/accumulo-1.4.0/lib/c
>> om
>> mons-lang-2.4.jar:/cloudbase/accumulo-1.4.0/lib/commons-logging-1.0.4.
>> ja
>> r:/cloudbase/accumulo-1.4.0/lib/commons-logging-api-1.0.4.jar:
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:java.library.path=/cloudbase/hadoop-0.20.2/bin/../lib/nati
>> ve
>> /Linux-amd64-64
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:java.io.tmpdir=/tmp
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:java.compiler=<NA>
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:os.name=Linux
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:os.arch=amd64
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:os.version=2.6.32-131.0.15.el6.x86_64
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:user.name=hadoop
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:user.home=/home/hadoop
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
>> environment:user.dir=/mnt/hgfs/CSI.Cloudbase/Java/CloudbaseServices/Cl
>> ou dbaseIngesters/src/com/comcept/cloudbase/ingesters/placemarks
>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Initiating client
>> connection, connectString=redhat-cloudbase1:2181 sessionTimeout=30000
>> watcher=org.apache.accumulo.core.zookeeper.ZooSession$AccumuloWatcher@
>> 4f
>> 4db0e3
>> 12/05/22 09:05:29 INFO zookeeper.ClientCnxn: Opening socket connection
>> to server /192.168.136.2:2181
>> 12/05/22 09:05:29 WARN client.ZooKeeperSaslClient: SecurityException:
>> java.lang.SecurityException: Unable to locate a login configuration
>> occurred when trying to find JAAS configuration.
>> 12/05/22 09:05:29 INFO client.ZooKeeperSaslClient: Client will not
>> SASL-authenticate because the default JAAS configuration section
>> 'Client' could not be found. If you are not using SASL, you may ignore
>> this. On the other hand, if you expected SASL to work, please fix your
>> JAAS configuration.
>> 12/05/22 09:05:29 INFO zookeeper.ClientCnxn: Socket connection
>> established to redhat-cloudbase1/192.168.136.2:2181, initiating
>> session
>> 12/05/22 09:05:29 INFO zookeeper.ClientCnxn: Session establishment
>> complete on server redhat-cloudbase1/192.168.136.2:2181, sessionid =
>> 0x1377101615a323f, negotiated timeout = 30000
>> 12/05/22 09:05:30 INFO input.FileInputFormat: Total input paths to
>> process : 3
>> 12/05/22 09:05:30 INFO mapred.JobClient: Running job:
>> job_201205211505_0012
>> 12/05/22 09:05:31 INFO mapred.JobClient:  map 0% reduce 0%
>> 12/05/22 09:05:38 INFO mapred.JobClient: Task Id :
>> attempt_201205211505_0012_m_000004_0, Status : FAILED
>> java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> org.apache.accumulo.core.client.mapreduce.AccumuloFileOutputFormat
>>        at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:809)
>>        at
>> org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext
>> .j
>> ava:193)
>>        at org.apache.hadoop.mapred.Task.initialize(Task.java:413)
>>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:288)
>>        at org.apache.hadoop.mapred.Child.main(Child.java:170)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.accumulo.core.client.mapreduce.AccumuloFileOutputFormat
>>        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>        at java.security.AccessController.doPrivileged(Native Method)
>>        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>        at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>        at java.lang.Class.forName0(Native Method)
>>        at java.lang.Class.forName(Class.java:247)
>>        at
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java
>> :7
>> 62)
>>        at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
>>        ... 4 more
>>
>> Bob Thorman
>> Engineering Fellow
>> L-3 Communications, ComCept
>> 1700 Science Place
>> Rockwall, TX 75032
>> (972) 772-7501 work
>> Bob.Thorman@ncct.af.smil.mil
>> rdthorm@nsa.ic.gov
>>
>>

Mime
View raw message