Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 26733C437 for ; Sun, 10 Jun 2012 18:31:41 +0000 (UTC) Received: (qmail 23498 invoked by uid 500); 10 Jun 2012 18:31:39 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 23457 invoked by uid 500); 10 Jun 2012 18:31:39 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 23449 invoked by uid 99); 10 Jun 2012 18:31:39 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 10 Jun 2012 18:31:39 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FSL_RCVD_USER,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates 209.85.210.41 as permitted sender) Received: from [209.85.210.41] (HELO mail-pz0-f41.google.com) (209.85.210.41) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 10 Jun 2012 18:31:34 +0000 Received: by dakp5 with SMTP id p5so5457208dak.14 for ; Sun, 10 Jun 2012 11:31:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding:x-gm-message-state; bh=A0ctKxao1A4N0R8TcmPh0wvkLjG3P8UzI0Fd2tzn03A=; b=IMCONTKVEprjsmaA4w6qVEgg2yKm0O7HD3jtySNSkg96VlikeAXnaTZN007Fx5U8Et yYjpwNoziGyc6VYRC3x0/BZg2JI/FEgP4sp/dvOmusih8gSbF8mKNKkZs6ttZiYOoxYd nyhQXqvN8zyyG3+NIK1Ylu6Bp3bnlN02ZgB+co6n55eky+vI2zbCSJx9F5VP9eF/riW4 +ZP4DUKqg6jhyLuzyApE7InY8qrq7jg5IWvH8m34Flyrz1vGjldnzZO0DWGF12Fyji2+ 2TW8WHQ/6zo6OsJl5G+jDD+CUkO6UPbq35/jjW+dp7uY1PwzK6VxnUSbnKWy/Gkc5I+q pLGw== Received: by 10.68.194.201 with SMTP id hy9mr18381376pbc.69.1339353073991; Sun, 10 Jun 2012 11:31:13 -0700 (PDT) MIME-Version: 1.0 Received: by 10.68.130.201 with HTTP; Sun, 10 Jun 2012 11:30:53 -0700 (PDT) In-Reply-To: References: From: Harsh J Date: Mon, 11 Jun 2012 00:00:53 +0530 Message-ID: Subject: Re: Row Counter rpoblem To: user@hbase.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Gm-Message-State: ALoCoQlL17A347YmwZJQlAlZrWDmNzWYWB2enBlwjt+wRtyOdNvyjpBe3h4akTfUrdQEVRNqtnyL X-Virus-Checked: Checked by ClamAV on apache.org Hi Ryan, The issue doesn't seem to be ACLs if those are the only changed properties. I think the issue may be either this: "Error: Could not initialize class org.apache.log4j.LogManager" or better detailed at http://datanode003.cluster.local:50060/tasklog?plaintext=3Dtrue&attemptid= =3Dattempt_201206101609_0001_m_000098_0 I've not tried the combo of the two versions you're using yet, but its either a bad log4j lib mixup or some other class-loading-related trouble. Can you tell us what the latter link shows, if anything at all, for one failed task attempt? On Sun, Jun 10, 2012 at 9:09 PM, Ryan Smith w= rote: >> >> Hello, >> >> I am having problems using the rowcounter mapreduce job on my hbase tabl= e. >> =A0The table exists, and rowcoutner works in the hbase shell, However, w= hen I >> try it from the command below using the mapreduce jar, I get this error = at >> the bottom.. >> >> # HADOOP_CLASSPATH=3D`/opt/hbase/bin/hbase classpath` /opt/hadoop/bin/ha= doop >> jar /opt/hbase/hbase-0.92.0.jar rowcounter mytable >> log4j:ERROR Could not find value for key log4j.appender.NullAppender >> log4j:ERROR Could not instantiate appender named "NullAppender". >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client >> environment:zookeeper.version=3D3.4.2-1221870, built on 12/21/2011 20:46= GMT >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client environment:host.name >> =3Dnamenodeone.cluster.local >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client >> environment:java.version=3D1.6.0_29 >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client >> environment:java.vendor=3DSun Microsystems Inc. >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client >> environment:java.home=3D/usr/java/jdk1.6.0_29/jre >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client >> environment:java.class.path=3D/opt/hadoop-1.0.2/libexec/../conf:/usr/jav= a/default//lib/tools.jar:/opt/hadoop-1.0.2/libexec/..:/opt/hadoop-1.0.2/lib= exec/../hadoop-core-1.0.2.jar:/opt/hadoop-1.0.2/libexec/../lib/asm-3.2.jar:= /opt/hadoop-1.0.2/libexec/../lib/aspectjrt-1.6.5.jar:/opt/hadoop-1.0.2/libe= xec/../lib/aspectjtools-1.6.5.jar:/opt/hadoop-1.0.2/libexec/../lib/commons-= beanutils-1.7.0.jar:/opt/hadoop-1.0.2/libexec/../lib/commons-beanutils-core= -1.8.0.jar:/opt/hadoop-1.0.2/libexec/../lib/commons-cli-1.2.jar:/opt/hadoop= -1.0.2/libexec/../lib/commons-codec-1.4.jar:/opt/hadoop-1.0.2/libexec/../li= b/commons-collections-3.2.1.jar:/opt/hadoop-1.0.2/libexec/../lib/commons-co= nfiguration-1.6.jar:/opt/hadoop-1.0.2/libexec/../lib/commons-daemon-1.0.1.j= ar:/opt/hadoop-1.0.2/libexec/../lib/commons-digester-1.8.jar:/opt/hadoop-1.= 0.2/libexec/../lib/commons-el-1.0.jar:/opt/hadoop-1.0.2/libexec/../lib/comm= ons-httpclient-3.0.1.jar:/opt/hadoop-1.0.2/libexec/../lib/commons-lang-2.4.= jar:/opt/hadoop-1.0.2/libexec/../lib/commons-logging-1.1.1.jar:/opt/hadoop-= 1.0.2/libexec/../lib/commons-logging-api-1.0.4.jar:/opt/hadoop-1.0.2/libexe= c/../lib/commons-math-2.1.jar:/opt/hadoop-1.0.2/libexec/../lib/commons-net-= 1.4.1.jar:/opt/hadoop-1.0.2/libexec/../lib/core-3.1.1.jar:/opt/hadoop-1.0.2= /libexec/../lib/guava-11.0.2.jar:/opt/hadoop-1.0.2/libexec/../lib/hadoop-ca= pacity-scheduler-1.0.2.jar:/opt/hadoop-1.0.2/libexec/../lib/hadoop-fairsche= duler-1.0.2.jar:/opt/hadoop-1.0.2/libexec/../lib/hadoop-thriftfs-1.0.2.jar:= /opt/hadoop-1.0.2/libexec/../lib/hsqldb-1.8.0.10.jar:/opt/hadoop-1.0.2/libe= xec/../lib/jackson-core-asl-1.8.8.jar:/opt/hadoop-1.0.2/libexec/../lib/jack= son-mapper-asl-1.8.8.jar:/opt/hadoop-1.0.2/libexec/../lib/jasper-compiler-5= .5.12.jar:/opt/hadoop-1.0.2/libexec/../lib/jasper-runtime-5.5.12.jar:/opt/h= adoop-1.0.2/libexec/../lib/jdeb-0.8.jar:/opt/hadoop-1.0.2/libexec/../lib/je= rsey-core-1.8.jar:/opt/hadoop-1.0.2/libexec/../lib/jersey-json-1.8.jar:/opt= /hadoop-1.0.2/libexec/../lib/jersey-server-1.8.jar:/opt/hadoop-1.0.2/libexe= c/../lib/jets3t-0.6.1.jar:/opt/hadoop-1.0.2/libexec/../lib/jetty-6.1.26.jar= :/opt/hadoop-1.0.2/libexec/../lib/jetty-util-6.1.26.jar:/opt/hadoop-1.0.2/l= ibexec/../lib/jsch-0.1.42.jar:/opt/hadoop-1.0.2/libexec/../lib/junit-4.5.ja= r:/opt/hadoop-1.0.2/libexec/../lib/kfs-0.2.2.jar:/opt/hadoop-1.0.2/libexec/= ../lib/log4j-1.2.15.jar:/opt/hadoop-1.0.2/libexec/../lib/log4j-1.2.16.jar:/= opt/hadoop-1.0.2/libexec/../lib/mockito-all-1.8.5.jar:/opt/hadoop-1.0.2/lib= exec/../lib/oro-2.0.8.jar:/opt/hadoop-1.0.2/libexec/../lib/servlet-api-2.5-= 20081211.jar:/opt/hadoop-1.0.2/libexec/../lib/slf4j-api-1.4.3.jar:/opt/hado= op-1.0.2/libexec/../lib/slf4j-log4j12-1.4.3.jar:/opt/hadoop-1.0.2/libexec/.= ./lib/xmlenc-0.52.jar:/opt/hadoop-1.0.2/libexec/../lib/jsp-2.1/jsp-2.1.jar:= /opt/hadoop-1.0.2/libexec/../lib/jsp-2.1/jsp-api-2.1.jar:/opt/hbase-0.92.0/= lib/zookeeper-3.4.2.jar:/opt/hbase/bin/../conf:/usr/java/default/lib/tools.= jar:/opt/hbase/bin/..:/opt/hbase/bin/../hbase-0.92.0.jar:/opt/hbase/bin/../= hbase-0.92.0-tests.jar:/opt/hbase/bin/../lib/activation-1.1.jar:/opt/hbase/= bin/../lib/asm-3.1.jar:/opt/hbase/bin/../lib/avro-1.5.3.jar:/opt/hbase/bin/= ../lib/avro-ipc-1.5.3.jar:/opt/hbase/bin/../lib/commons-beanutils-1.7.0.jar= :/opt/hbase/bin/../lib/commons-beanutils-core-1.8.0.jar:/opt/hbase/bin/../l= ib/commons-cli-1.2.jar:/opt/hbase/bin/../lib/commons-codec-1.4.jar:/opt/hba= se/bin/../lib/commons-collections-3.2.1.jar:/opt/hbase/bin/../lib/commons-c= onfiguration-1.6.jar:/opt/hbase/bin/../lib/commons-digester-1.8.jar:/opt/hb= ase/bin/../lib/commons-el-1.0.jar:/opt/hbase/bin/../lib/commons-httpclient-= 3.1.jar:/opt/hbase/bin/../lib/commons-lang-2.5.jar:/opt/hbase/bin/../lib/co= mmons-logging-1.1.1.jar:/opt/hbase/bin/../lib/commons-math-2.1.jar:/opt/hba= se/bin/../lib/commons-net-1.4.1.jar:/opt/hbase/bin/../lib/core-3.1.1.jar:/o= pt/hbase/bin/../lib/guava-r09.jar:/opt/hbase/bin/../lib/hadoop-core-1.0.0.j= ar:/opt/hbase/bin/../lib/high-scale-lib-1.1.1.jar:/opt/hbase/bin/../lib/htt= pclient-4.0.1.jar:/opt/hbase/bin/../lib/httpcore-4.0.1.jar:/opt/hbase/bin/.= ./lib/jackson-core-asl-1.5.5.jar:/opt/hbase/bin/../lib/jackson-jaxrs-1.5.5.= jar:/opt/hbase/bin/../lib/jackson-mapper-asl-1.5.5.jar:/opt/hbase/bin/../li= b/jackson-xc-1.5.5.jar:/opt/hbase/bin/../lib/jamon-runtime-2.3.1.jar:/opt/h= base/bin/../lib/jasper-compiler-5.5.23.jar:/opt/hbase/bin/../lib/jasper-run= time-5.5.23.jar:/opt/hbase/bin/../lib/jaxb-api-2.1.jar:/opt/hbase/bin/../li= b/jaxb-impl-2.1.12.jar:/opt/hbase/bin/../lib/jersey-core-1.4.jar:/opt/hbase= /bin/../lib/jersey-json-1.4.jar:/opt/hbase/bin/../lib/jersey-server-1.4.jar= :/opt/hbase/bin/../lib/jettison-1.1.jar:/opt/hbase/bin/../lib/jetty-6.1.26.= jar:/opt/hbase/bin/../lib/jetty-util-6.1.26.jar:/opt/hbase/bin/../lib/jruby= -complete-1.6.5.jar:/opt/hbase/bin/../lib/jsp-2.1-6.1.14.jar:/opt/hbase/bin= /../lib/jsp-api-2.1-6.1.14.jar:/opt/hbase/bin/../lib/libthrift-0.7.0.jar:/o= pt/hbase/bin/../lib/log4j-1.2.16.jar:/opt/hbase/bin/../lib/netty-3.2.4.Fina= l.jar:/opt/hbase/bin/../lib/protobuf-java-2.4.0a.jar:/opt/hbase/bin/../lib/= servlet-api-2.5-6.1.14.jar:/opt/hbase/bin/../lib/servlet-api-2.5.jar:/opt/h= base/bin/../lib/slf4j-api-1.5.8.jar:/opt/hbase/bin/../lib/slf4j-log4j12-1.5= .8.jar:/opt/hbase/bin/../lib/snappy-java-1.0.3.2.jar:/opt/hbase/bin/../lib/= stax-api-1.0.1.jar:/opt/hbase/bin/../lib/velocity-1.7.jar:/opt/hbase/bin/..= /lib/xmlenc-0.52.jar:/opt/hbase/bin/../lib/zookeeper-3.4.2.jar >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client >> environment:java.library.path=3D/opt/hadoop-1.0.2/libexec/../lib/native/= Linux-amd64-64 >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client >> environment:java.io.tmpdir=3D/tmp >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client >> environment:java.compiler=3D >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client environment:os.name >> =3DLinux >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client >> environment:os.arch=3Damd64 >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client >> environment:os.version=3D2.6.32-220.7.1.el6.x86_64 >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client environment:user.name >> =3Droot >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client >> environment:user.home=3D/root >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Client >> environment:user.dir=3D/opt/hadoop-1.0.2 >> 12/06/10 16:12:09 INFO zookeeper.ZooKeeper: Initiating client connection= , >> connectString=3D188.94.23.3:2181 sessionTimeout=3D180000 watcher=3Dhconn= ection >> 12/06/10 16:12:09 INFO zookeeper.ClientCnxn: Opening socket connection t= o >> server /188.94.23.3:2181 >> 12/06/10 16:12:09 INFO zookeeper.RecoverableZooKeeper: The identifier of >> this process is 16267@namenodeone >> 12/06/10 16:12:09 INFO zookeeper.ClientCnxn: Socket connection establish= ed >> to namenodeone.cluster.local/188.94.23.3:2181, initiating session >> 12/06/10 16:12:09 INFO zookeeper.ClientCnxn: Session establishment >> complete on server namenodeone.cluster.local/188.94.23.3:2181, sessionid >> =3D 0x1377407b6930099, negotiated timeout =3D 180000 >> 12/06/10 16:12:09 INFO hdfs.DFSClient: Exception in >> createBlockOutputStream 188.94.23.13:50010 java.io.IOException: Bad >> connect ack with firstBadLink as 188.94.23.11:50010 >> 12/06/10 16:12:09 INFO hdfs.DFSClient: Abandoning block >> blk_5041391389848324461_85581 >> 12/06/10 16:12:09 INFO hdfs.DFSClient: Excluding datanode >> 188.94.23.11:50010 >> 12/06/10 16:12:16 INFO mapred.JobClient: Running job: job_201206101609_0= 001 >> 12/06/10 16:12:17 INFO mapred.JobClient: =A0map 0% reduce 0% >> 12/06/10 16:12:26 INFO mapred.JobClient: Task Id : >> attempt_201206101609_0001_m_000098_0, Status : FAILED >> Error: Could not initialize class org.apache.log4j.LogManager >> 12/06/10 16:12:27 WARN mapred.JobClient: Error reading task >> outputhttp://datanode003.cluster.local:50060/tasklog?plaintext=3Dtrue&at= temptid=3Dattempt_201206101609_0001_m_000098_0&filter=3Dstdout >> 12/06/10 16:12:27 WARN mapred.JobClient: Error reading task >> outputhttp://datanode003.cluster.local:50060/tasklog?plaintext=3Dtrue&at= temptid=3Dattempt_201206101609_0001_m_000098_0&filter=3Dstderr >> 12/06/10 16:12:30 INFO mapred.JobClient: Task Id : >> attempt_201206101609_0001_r_000001_0, Status : FAILED >> Error: Could not initialize class org.apache.log4j.LogManager >> 12/06/10 16:12:30 WARN mapred.JobClient: Error reading task >> outputhttp://datanode003.cluster.local:50060/tasklog?plaintext=3Dtrue&at= temptid=3Dattempt_201206101609_0001_r_000001_0&filter=3Dstdout >> 12/06/10 16:12:30 WARN mapred.JobClient: Error reading task >> outputhttp://datanode003.cluster.local:50060/tasklog?plaintext=3Dtrue&at= temptid=3Dattempt_201206101609_0001_r_000001_0&filter=3Dstderr >> >> >> I will get the errors you see above on all the nodes until the job ends = in >> failure. =A0 =A0So next I edited the mapred-queue-acls.xml to look like = this >> then restarted TaskTrackers and jobTracker: >> >> >> =A0 mapred.queue.default.acl-submit-job >> =A0 * >> >> >> >> =A0 mapred.queue.default.acl-administer-jobs >> =A0 * >> >> >> >> And I got the same errors. =A0Any ideas what I am doing wrong? >> >> Thank you, >> -Ryan >> >> --=20 Harsh J