accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Vines <john.w.vi...@ugov.gov>
Subject Re: AcculumoFileOutputFormat class cannot be found by child jvm
Date Tue, 22 May 2012 18:19:34 GMT
I'm wondering if our tool.sh plays nice with an explicit -libjars.

John

On Tue, May 22, 2012 at 2:07 PM, William Slacum <wslacum@gmail.com> wrote:

> Make sure that the paths you pass to `-libjars` are correct. I have
> noticed behavior with Hadoop where it will silently drop the end of a
> libjars string if the paths do not exist.
>
> On Tue, May 22, 2012 at 11:03 AM, Keith Turner <keith@deenlo.com> wrote:
> > I think it should be in the job.xml file.  The way the class path for
> > the java program that starts map reduce is configured is different
> > from how the classpath for the remote task are configured.   So your
> > program that starts the map reduce job has AccumuloFileOutputFormat on
> > the class path because you use a static method on it and it runs.  So
> > I am thinking its not in --libjars for some reason or maybe you are
> > not using Hadoop Tool?
> >
> > On Tue, May 22, 2012 at 1:54 PM,  <Bob.Thorman@l-3com.com> wrote:
> >> No, it's not in the job.xml file.  A reference to
> AccumuloFileOutputFormat is but not the accumulo-core-1.4.0.jar.  The
> Job*.jar file is referenced there.  I was under the impression that the
> child jvm inherits the parent jvm attributes which makes the job*.jar file.
>  Is that not correct?
> >>
> >> -----Original Message-----
> >> From: Keith Turner [mailto:keith@deenlo.com]
> >> Sent: Tuesday, May 22, 2012 12:07
> >> To: user@accumulo.apache.org
> >> Subject: Re: AcculumoFileOutputFormat class cannot be found by child jvm
> >>
> >> If you look at Job.xml, do you see accumulo-core there?  There should
> be a link to this file on the jobs page on the tasktracker.
> >>
> >> On Tue, May 22, 2012 at 10:40 AM,  <Bob.Thorman@l-3com.com> wrote:
> >>> I upgrade to accumulo-1.4.0 and updated my map/reduce jobs and now
> >>> they don't run.  The parent class path has the accumulo-core-1.4.0.jar
> >>> file included.  Do the accumulo jar files have to be manually put on a
> >>> distribute cache?  Any help is appreciated.
> >>>
> >>> [hadoop@redhat-cloudbase1 placemarks]$ ./runPlacemarkIngester.sh Found
> >>> 5 items drwxrwxr-x   - hadoop hadoop          0 2012-05-21 14:13
> >>> /accumulo drwxrwxr-x   - hadoop hadoop          0 2012-05-21 15:06
> >>> /data drwxr-xr-x   - hadoop hadoop          0 2012-05-22 08:58 /input
> >>> drwxr-xr-x   - hadoop hadoop          0 2012-05-22 08:58 /output
> >>> drwxrwxr-x   - hadoop hadoop          0 2012-05-21 14:34 /usr Deleted
> >>> hdfs://redhat-cloudbase1:9000/output
> >>> Deleted hdfs://redhat-cloudbase1:9000/input
> >>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: Invoking
> >>> ToolRunner.run
> >>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: zooKeeper is
> >>> redhat-cloudbase1:2181
> >>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: instanceName is
> >>> NCCT-Cloudbase
> >>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: timeTableName is
> >>> NCCTServicesTimes
> >>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: geoTableName is
> >>> NCCTServicesGeos
> >>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: metadataTableName
> >>> is NCCTServicesMetadata
> >>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: edgeTableName is
> >>> NCCTEdgeTable
> >>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: userName is
> >>> NCCT.Services.Client
> >>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: password is
> >>> *********
> >>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: visibility is
> >>> public,BD2,UNCLASSIFIED-NO_COMPARTMENT-UNRESTRICTED
> >>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: inputDir is
> >>> /input
> >>> 12/05/22 09:05:29 INFO placemarks.PlacemarkIngester: outputDir is
> >>> /output
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48
> >>> GMT
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:host.name=redhat-cloudbase1
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:java.version=1.6.0_32
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:java.vendor=Sun Microsystems Inc.
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:java.home=/usr/java/jdk1.6.0_32/jre
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:java.class.path=/cloudbase/hadoop-0.20.2/bin/../conf:/usr/
> >>> ja
> >>> va/jdk1.6.0_32/lib/tools.jar:/cloudbase/hadoop-0.20.2/bin/..:/cloudbas
> >>> e/
> >>> hadoop-0.20.2/bin/../hadoop-0.20.2-core.jar:/cloudbase/hadoop-0.20.2/b
> >>> in
> >>> /../lib/commons-cli-1.2.jar:/cloudbase/hadoop-0.20.2/bin/../lib/common
> >>> s-
> >>> codec-1.3.jar:/cloudbase/hadoop-0.20.2/bin/../lib/commons-el-1.0.jar:/
> >>> cl
> >>> oudbase/hadoop-0.20.2/bin/../lib/commons-httpclient-3.0.1.jar:/cloudba
> >>> se
> >>>
> /hadoop-0.20.2/bin/../lib/commons-logging-1.0.4.jar:/cloudbase/hadoop-0.
> >>> 20.2/bin/../lib/commons-logging-api-1.0.4.jar:/cloudbase/hadoop-0.20.2
> >>> /b
> >>> in/../lib/commons-net-1.4.1.jar:/cloudbase/hadoop-0.20.2/bin/../lib/co
> >>> re
> >>> -3.1.1.jar:/cloudbase/hadoop-0.20.2/bin/../lib/hadoop-0.20.2-ant.jar:/
> >>> cl
> >>> oudbase/hadoop-0.20.2/bin/../lib/hadoop-0.20.2-core.jar:/cloudbase/had
> >>> oo
> >>> p-0.20.2/bin/../lib/hadoop-0.20.2-examples.jar:/cloudbase/hadoop-0.20.
> >>> 2/
> >>> bin/../lib/hadoop-0.20.2-test.jar:/cloudbase/hadoop-0.20.2/bin/../lib/
> >>> ha
> >>> doop-0.20.2-tools.jar:/cloudbase/hadoop-0.20.2/bin/../lib/hsqldb-1.8.0
> >>> .1
> >>> 0.jar:/cloudbase/hadoop-0.20.2/bin/../lib/jasper-compiler-5.5.12.jar:/
> >>> cl
> >>> oudbase/hadoop-0.20.2/bin/../lib/jasper-runtime-5.5.12.jar:/cloudbase/
> >>> ha
> >>> doop-0.20.2/bin/../lib/jets3t-0.6.1.jar:/cloudbase/hadoop-0.20.2/bin/.
> >>> ./
> >>>
> lib/jetty-6.1.14.jar:/cloudbase/hadoop-0.20.2/bin/../lib/jetty-util-6.1.
> >>> 14.jar:/cloudbase/hadoop-0.20.2/bin/../lib/junit-3.8.1.jar:/cloudbase/
> >>> ha
> >>> doop-0.20.2/bin/../lib/kfs-0.2.2.jar:/cloudbase/hadoop-0.20.2/bin/../l
> >>> ib
> >>>
> /log4j-1.2.15.jar:/cloudbase/hadoop-0.20.2/bin/../lib/mockito-all-1.8.0.
> >>> jar:/cloudbase/hadoop-0.20.2/bin/../lib/oro-2.0.8.jar:/cloudbase/hadoo
> >>> p-
> >>> 0.20.2/bin/../lib/servlet-api-2.5-6.1.14.jar:/cloudbase/hadoop-0.20.2/
> >>> bi
> >>> n/../lib/slf4j-api-1.4.3.jar:/cloudbase/hadoop-0.20.2/bin/../lib/slf4j
> >>> -l
> >>> og4j12-1.4.3.jar:/cloudbase/hadoop-0.20.2/bin/../lib/xmlenc-0.52.jar:/
> >>> cl
> >>> oudbase/hadoop-0.20.2/bin/../lib/jsp-2.1/jsp-2.1.jar:/cloudbase/hadoop
> >>> -0
> >>> .20.2/bin/../lib/jsp-2.1/jsp-api-2.1.jar:/cloudbase/accumulo-1.4.0/lib
> >>> /l
> >>> ibthrift-0.6.1.jar:/cloudbase/accumulo-1.4.0/lib/accumulo-core-1.4.0.j
> >>> ar
> >>>
> :/cloudbase/zookeeper-3.4.3/zookeeper-3.4.3.jar:/cloudbase/accumulo-1.4.
> >>> 0/lib/cloudtrace-1.4.0.jar:/usr/lib/ncct/kxml2-2.3.0.jar:/usr/lib/ncct
> >>> /x
> >>> mlpull-1.1.3.1.jar:/usr/lib/ncct/xstream-1.4.1.jar:/cloudbase/accumulo
> >>> -1
> >>> .4.0/lib/commons-collections-3.2.jar:/cloudbase/accumulo-1.4.0/lib/com
> >>> mo
> >>> ns-configuration-1.5.jar:/cloudbase/accumulo-1.4.0/lib/commons-io-1.4.
> >>> ja
> >>> r:/cloudbase/accumulo-1.4.0/lib/commons-jci-core-1.0.jar:/cloudbase/ac
> >>> cu
> >>> mulo-1.4.0/lib/commons-jci-fam-1.0.jar:/cloudbase/accumulo-1.4.0/lib/c
> >>> om
> >>> mons-lang-2.4.jar:/cloudbase/accumulo-1.4.0/lib/commons-logging-1.0.4.
> >>> ja
> >>> r:/cloudbase/accumulo-1.4.0/lib/commons-logging-api-1.0.4.jar:
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:java.library.path=/cloudbase/hadoop-0.20.2/bin/../lib/nati
> >>> ve
> >>> /Linux-amd64-64
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:java.io.tmpdir=/tmp
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:java.compiler=<NA>
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:os.name=Linux
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:os.arch=amd64
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:os.version=2.6.32-131.0.15.el6.x86_64
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:user.name=hadoop
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:user.home=/home/hadoop
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Client
> >>> environment:user.dir=/mnt/hgfs/CSI.Cloudbase/Java/CloudbaseServices/Cl
> >>> ou dbaseIngesters/src/com/comcept/cloudbase/ingesters/placemarks
> >>> 12/05/22 09:05:29 INFO zookeeper.ZooKeeper: Initiating client
> >>> connection, connectString=redhat-cloudbase1:2181 sessionTimeout=30000
> >>> watcher=org.apache.accumulo.core.zookeeper.ZooSession$AccumuloWatcher@
> >>> 4f
> >>> 4db0e3
> >>> 12/05/22 09:05:29 INFO zookeeper.ClientCnxn: Opening socket connection
> >>> to server /192.168.136.2:2181
> >>> 12/05/22 09:05:29 WARN client.ZooKeeperSaslClient: SecurityException:
> >>> java.lang.SecurityException: Unable to locate a login configuration
> >>> occurred when trying to find JAAS configuration.
> >>> 12/05/22 09:05:29 INFO client.ZooKeeperSaslClient: Client will not
> >>> SASL-authenticate because the default JAAS configuration section
> >>> 'Client' could not be found. If you are not using SASL, you may ignore
> >>> this. On the other hand, if you expected SASL to work, please fix your
> >>> JAAS configuration.
> >>> 12/05/22 09:05:29 INFO zookeeper.ClientCnxn: Socket connection
> >>> established to redhat-cloudbase1/192.168.136.2:2181, initiating
> >>> session
> >>> 12/05/22 09:05:29 INFO zookeeper.ClientCnxn: Session establishment
> >>> complete on server redhat-cloudbase1/192.168.136.2:2181, sessionid =
> >>> 0x1377101615a323f, negotiated timeout = 30000
> >>> 12/05/22 09:05:30 INFO input.FileInputFormat: Total input paths to
> >>> process : 3
> >>> 12/05/22 09:05:30 INFO mapred.JobClient: Running job:
> >>> job_201205211505_0012
> >>> 12/05/22 09:05:31 INFO mapred.JobClient:  map 0% reduce 0%
> >>> 12/05/22 09:05:38 INFO mapred.JobClient: Task Id :
> >>> attempt_201205211505_0012_m_000004_0, Status : FAILED
> >>> java.lang.RuntimeException: java.lang.ClassNotFoundException:
> >>> org.apache.accumulo.core.client.mapreduce.AccumuloFileOutputFormat
> >>>        at
> >>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:809)
> >>>        at
> >>> org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext
> >>> .j
> >>> ava:193)
> >>>        at org.apache.hadoop.mapred.Task.initialize(Task.java:413)
> >>>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:288)
> >>>        at org.apache.hadoop.mapred.Child.main(Child.java:170)
> >>> Caused by: java.lang.ClassNotFoundException:
> >>> org.apache.accumulo.core.client.mapreduce.AccumuloFileOutputFormat
> >>>        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> >>>        at java.security.AccessController.doPrivileged(Native Method)
> >>>        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> >>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> >>>        at
> >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> >>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> >>>        at java.lang.Class.forName0(Native Method)
> >>>        at java.lang.Class.forName(Class.java:247)
> >>>        at
> >>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java
> >>> :7
> >>> 62)
> >>>        at
> >>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
> >>>        ... 4 more
> >>>
> >>> Bob Thorman
> >>> Engineering Fellow
> >>> L-3 Communications, ComCept
> >>> 1700 Science Place
> >>> Rockwall, TX 75032
> >>> (972) 772-7501 work
> >>> Bob.Thorman@ncct.af.smil.mil
> >>> rdthorm@nsa.ic.gov
> >>>
> >>>
>

Mime
View raw message