Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B47F8F656 for ; Wed, 1 May 2013 13:56:16 +0000 (UTC) Received: (qmail 35134 invoked by uid 500); 1 May 2013 13:43:04 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 34540 invoked by uid 500); 1 May 2013 13:43:01 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 34502 invoked by uid 99); 1 May 2013 13:43:00 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 May 2013 13:43:00 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of rkevinburton@charter.net designates 216.33.127.80 as permitted sender) Received: from [216.33.127.80] (HELO mta11.charter.net) (216.33.127.80) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 May 2013 13:42:49 +0000 Received: from imp11 ([10.20.200.11]) by mta11.charter.net (InterMail vM.8.01.05.09 201-2260-151-124-20120717) with ESMTP id <20130501134227.RNWK6169.mta11.charter.net@imp11>; Wed, 1 May 2013 09:42:27 -0400 Received: from wt08 ([209.225.8.58]) by imp11 with smtp.charter.net id WdiT1l00E1F7hH505diT3J; Wed, 01 May 2013 09:42:27 -0400 X-Authority-Analysis: v=2.0 cv=dIr+A5lb c=1 sm=1 a=os3phs8/4a+HRgZgvItj1g==:17 a=3dz4mxVvhEEA:10 a=cWQ9uGxeeyIA:10 a=yUnIBFQkZM0A:10 a=hOpmn2quAAAA:8 a=3s95SbjHtX8A:10 a=Cb6IxKRDAAAA:8 a=vV5ofkA0AAAA:8 a=pGLkceISAAAA:8 a=lVBABN-2AAAA:8 a=COfzQ7OkAAAA:8 a=TeE1_d55AAAA:8 a=mV9VRH-2AAAA:8 a=6quYA1peAAAA:8 a=YjOBGq4wAAAA:8 a=aAbPJnMWAAAA:8 a=o5ydADn6DARmVW8c6k4A:9 a=QEXdDO2ut3YA:10 a=IxbdWj-SGzYA:10 a=t0WRet2RL3kA:10 a=hUswqBWy9Q8A:10 a=MSl-tDqOz04A:10 a=qiWi7Edwir4A:10 a=gA6IeH5FQcgA:10 a=NWVoK91CQyQA:10 a=w_8BRHzJBgyhXUsT:21 a=p9NnyEJNSiVRV5T_:21 a=xKaAtcs7f8-lfG5a:21 a=r9bmuvJpeH5zwn7xTK8A:9 a=Sf_gFPzhefAA:10 a=tXsnliwV7b4A:10 a=-mxWGvpHCjNf_myO:21 a=0XaZZ8OhYQc3p552:21 a=eSvQN9dc54mnvB1Y:21 a=1tjqaQL0AM0UPNcVwqj5HA==:117 Received: from [173.227.207.246] by enhanced.charter.net with HTTP; Wed, 1 May 2013 09:42:27 -0400 Date: Wed, 1 May 2013 09:42:27 -0400 (EDT) From: rkevinburton@charter.net To: Mohammad Tariq Cc: user@hadoop.apache.org Message-ID: <4e995401.609f75.13e60547306.Webtop.48@charter.net> Subject: Re: Can't initialize cluster MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_22869123_9723910.1367415747176" User-Agent: Laszlo Mail 3 X-SID: 48 X-Originating-IP: [173.227.207.246] X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_22869123_9723910.1367415747176 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline I made these changes. The log files in /var/log/hadoop-0.20-mapreduce=20 do not reflect todays date so they are not updated. The datanode log in=20 /var/log/hadoop-hdfs only shows this warning: 2013-05-01 08:16:12,678 WARN=20 org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to=20 server: devUbuntu05/172.16.26.68:9000 The entries in the log after that (and there are many) all are INFO=20 messages. Not more warnings or errors. The namenode log is just INFO messages. The secondarynamenode log (on the node that is acting as the secondary=20 namenode) contains the following error exception: 2013-05-01 08:34:16,431 ERROR=20 org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Exception in=20 doCheckpoint java.io.IOException: Inconsistent checkpoint fields. LV =3D -40 namespaceID =3D 1097063765 cTime =3D 0 ; clusterId =3D=20 CID-23b9f9c7-2c25-411f-8bd2-4d5c9d7c25a1 ; blockpoolId =3D=20 BP-1306349046-172.16.26.68-1367256199559. Expecting respectively: -40; 1666002936; 0;=20 CID-e3f6b811-c1b4-4778-a31e-14dea8b2cca8;=20 BP-1964515668-172.16.26.68-1366403021016. at=20 org.apache.hadoop.hdfs.server.namenode.CheckpointSignature.validateStorageI= nfo(CheckpointSignature.java:133) at=20 org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doCheckpoint(Secon= daryNameNode.java:475) The logs for the any of the nodes serving as the task tracker are out of=20 date. When I try to attach to the master with a browser on port 50070 to look=20 at the HDFS the page responds but when I try to connect to the master at=20 port 50030 nothing is listening so it seems the JobTracker is down. So it seems from what I can gather the reason that the JobTracker=20 doesn't come up is because of the error on the secondary name node. Do=20 you understand how I can get rid of this error? Kevin On Tue, Apr 30, 2013 at 8:55 PM, Mohammad Tariq wrote: Are you sure about the case in=C2=A0 devuBuntu06 devuBuntu07 devuBuntu08 Is 'B' actually in upper case? Also,=C2=A0 mapred.local.dir=C2=A0 belongs to mapred-site.xml and=C2=A0 had= oop.tmp.dir=20 belongs to core-site.xml file. Warm Regards, Tariq https://mtariq.jux.com/ cloudfront.blogspot.com On Wed, May 1, 2013 at 7:09 AM, Kevin Burton < rkevinburton@charter.net=20 =20 > wrote: Here are what I consider the most applicable conf files.=20 =C2=A0=20 Again thank you for your help.=20 =C2=A0=20 Kevin=20 =C2=A0=20 From: Mohammad Tariq [mailto: dontariq@gmail.com=20 ] Sent: Tuesday, April 30, 2013 6:43 PM=20 To: Kevin Burton Subject: Re: Can't initialize cluster=20 =C2=A0 Sorry for the late response Kevin. I was away for a while. I saw the=20 logs. There seems to be some problem with your configuration or name=20 resolution. Could you please show me your configuration files?=20 Warm Regards,=20 Tariq=20 https://mtariq.jux.com/ =20 cloudfront.blogspot.com =20 =C2=A0 On Wed, May 1, 2013 at 4:27 AM, Kevin Burton < rkevinburton@charter.net=20 =20 > wrote:=20 The JobTracker logs don=E2=80=99t seem to be updated:=20 =C2=A0=20 kevin@devUbuntu05:/etc/hadoop/conf$ cd /var/log/hadoop-0.20-mapreduce/=20 kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$=20 =20 ls -l=20 total 3028=20 -rw-rw-r-- 1 mapred mapred=C2=A0 383838 Apr 30 15:11=20 hadoop-hadoop-jobtracker-devUbuntu05.log=20 =C2=A0=20 But I have attached the last lines of the JT log along with the TT logs.=20 =C2=A0=20 Kevin=20 =C2=A0=20 From: Mohammad Tariq [mailto: dontariq@gmail.com=20 ] Sent: Tuesday, April 30, 2013 5:38 PM=20 To: Kevin Burton Subject: Re: Can't initialize cluster=20 =C2=A0 sorry for the typo in the last mail. it was 50070 and not=20 9000..anyways..=20 it's correct..it shows only the slaves..can you show me the JT and TT=20 logs?=20 Warm Regards,=20 Tariq=20 https://mtariq.jux.com/ =20 cloudfront.blogspot.com =20 =C2=A0 On Wed, May 1, 2013 at 4:00 AM, Kevin Burton < rkevinburton@charter.net=20 =20 > wrote:=20 There is something running. Here is the master:=20 =C2=A0=20 kevin@devUbuntu05:~$ ps aux | grep hadoop=20 =C2=A0=20 hdfs=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 2004=C2=A0 0.3=C2=A0 4.4 1977848 178396 = ?=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Sl=C2=A0=C2=A0 16:59=C2=A0=C2=A0 0:07=20 /usr/lib/jvm/java-7-openjdk-amd64//bin/java -Dproc_namenode -Xmx1000m=20 -Dhadoop.log.dir=3D/var/log/hadoop-hdfs=20 -Dhadoop.log.file=3Dhadoop-hdfs-namenode-devUbuntu05.log=20 -Dhadoop.home.dir=3D/usr/lib/hadoop -Dhadoop.id.str=3Dhdfs=20 -Dhadoop.root.logger=3DINFO,RFA=20 -Djava.library.path=3D/usr/lib/hadoop/lib/native=20 -Dhadoop.policy.file=3Dhadoop-policy.xml -Djava.net.preferIPv4Stack=3Dtrue= =20 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote=20 -Dcom.sun.management.jmxremote -Dhadoop.security.logger=3DINFO,RFAS=20 org.apache.hadoop.hdfs.server.namenode.NameNode=20 kevin=C2=A0=C2=A0=C2=A0=C2=A0 2743=C2=A0 0.0=C2=A0 0.0=C2=A0=C2=A0 9384=C2= =A0=C2=A0 924 pts/0=C2=A0=C2=A0=C2=A0 S+=C2=A0=C2=A0 17:28=C2=A0=C2=A0 0:00= grep=20 --color=3Dauto hadoop=20 =C2=A0=20 From: Mohammad Tariq [mailto: dontariq@gmail.com=20 ] Sent: Tuesday, April 30, 2013 5:21 PM=20 To: Kevin Burton Subject: Re: Can't initialize cluster=20 =C2=A0 if jps is showing just "jps" as output that means nothing is running.=20 you have one more way to cross check. point your web browser to=C2=A0=20 "namenode_machine:9000" and see if the cluster is behaving properly.=20 =C2=A0 if you don't mind, can we switch to chat??=20 Warm Regards,=20 Tariq=20 https://mtariq.jux.com/ =20 cloudfront.blogspot.com =20 =C2=A0 On Wed, May 1, 2013 at 3:48 AM, Kevin Burton < rkevinburton@charter.net=20 =20 > wrote:=20 There must be some option to run jps because running jps on all machines=20 give something like:=20 =C2=A0=20 kevin@devUbuntu05:~/WordCount$ jps=20 2398 Jps=20 =C2=A0=20 From: Mohammad Tariq [mailto: dontariq@gmail.com=20 ] Sent: Tuesday, April 30, 2013 5:11 PM=20 To: Kevin Burton Subject: Re: Can't initialize cluster=20 =C2=A0 Do jps and see which process is not running. Show me the corresponding=20 log. Warm Regards,=20 Tariq=20 https://mtariq.jux.com/ =20 cloudfront.blogspot.com =20 =C2=A0 On Wed, May 1, 2013 at 3:36 AM, Kevin Burton < rkevinburton@charter.net=20 =20 > wrote:=20 The task tracker and job tracker did give any indication of success or=20 failure when I started them. The namenode, secondarynamenode, datanode=20 daemons all gave back a message the they were started.=20 =C2=A0=20 =C2=A0=20 =C2=A0=20 From: Mohammad Tariq [mailto: dontariq@gmail.com=20 ] Sent: Tuesday, April 30, 2013 4:45 PM=20 To: Kevin Burton Subject: Re: Can't initialize cluster=20 =C2=A0 Ok..things are getting complex..we'll make sure that you cluster is=20 running fine=C2=A0first. stop everything and restart all 5 daemons and let = me=20 know if you are able to do it properly.=20 Warm Regards,=20 Tariq=20 https://mtariq.jux.com/ =20 cloudfront.blogspot.com =20 =C2=A0 On Wed, May 1, 2013 at 3:10 AM, Kevin Burton < rkevinburton@charter.net=20 =20 > wrote:=20 This didn=E2=80=99t seem to help. I get about 10 retries then (running the = job):=20 =C2=A0=20 kevin@devUbuntu05:~/WordCount$ hadoop jar WordCount.jar=20 file:///home/kevin/WordCount/inputfile:///home/kevin/WordCount/output . . . . . Exception in thread "main" java.net.ConnectException: Call From=20 devUbuntu05/ 172.16.26.68 to devUbuntu05:9001=20 failed on connection exception: java.net.ConnectException: Connection=20 refused; For more details see:=C2=A0=20 http://wiki.apache.org/hadoop/ConnectionRefused=20 =20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at sun.reflect.NativeConstructor= AccessorImpl.newInstance0(Native=20 Method) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAcce= ssorImpl.java:57)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstru= ctorAccessorImpl.java:45)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 java.lang.reflect.Constructor.newInstance(Constructor.java:525)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:779)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:726)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.ipc.Client.= call(Client.java:1229)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEng=20 =C2=A0 But still no indication why the JobTracker will not start.=20 =C2=A0 From: Mohammad Tariq [mailto: dontariq@gmail.com=20 ] Sent: Tuesday, April 30, 2013 4:28 PM=20 To: Kevin Burton Subject: Re: Can't initialize cluster=20 =C2=A0 Do this :=20 hadoop dfsadmin -safemode leave=20 =C2=A0 Then restart the daemons and re-run the job.=20 Warm Regards,=20 Tariq=20 https://mtariq.jux.com/ =20 cloudfront.blogspot.com =20 =C2=A0 On Wed, May 1, 2013 at 2:50 AM, Kevin Burton < rkevinburton@charter.net=20 =20 > wrote:=20 Now it looks like the JobTracker isn=E2=80=99t start at all and there doesn= =E2=80=99t=20 seem to be a log entry in /var/log/hadoop-0.20-mapreduce to indicate the=20 reason. I looked in /var/log/hadoop-hdfs and for the namenode I found=20 what looked like an exception :=20 =C2=A0=20 The reported blocks 0 needs additional 4 blocks to reach the threshold=20 0.9990 of total blocks 4. Safe mode will be turned off automatically.=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 org.apache.hadoop.hdfs.server.namenode.FSNamesystem.rollEditLog(FSNamesyste= m.java:4568)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.rollEditLog(NameNo= deRpcServer.java:742)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolServerSideTranslatorPB.ro= llEditLog(NamenodeProtocolServerSideTranslatorPB.java:129)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 org.apache.hadoop.hdfs.protocol.proto.NamenodeProtocolProtos$NamenodeProtoc= olService$2.callBlockingMethod(NamenodeProtocolProtos.java:8762)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(Prot= obufRpcEngine.java:453)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.ipc.RPC$Ser= ver.call(RPC.java:1002)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.ipc.Server$= Handler$1.run(Server.java:1695)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.ipc.Server$= Handler$1.run(Server.java:1691)=20 =C2=A0=20 Then I looked in the datanode log and I didn=E2=80=99t see what looked like= an=20 exception or error. The secondary name node log looked like:=20 =C2=A0=20 2013-04-29 14:11:25,465 ERROR=20 org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Exception in=20 doCheckpoint=20 java.net.ConnectException: Call From devUbuntu05/ 172.16.26.68=20 to devUbuntu05:9000 failed on connection=20 exception: java.net.ConnectException: Connection refused; For more=20 details see:=C2=A0 http://wiki.apache.org/hadoop/ConnectionRefused=20 =20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at sun.reflect.GeneratedConstruc= torAccessor4.newInstance(Unknown=20 Source) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstru= ctorAccessorImpl.java:45)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 java.lang.reflect.Constructor.newInstance(Constructor.java:525)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:779)=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at=20 org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:726)=20 =C2=A0 =C2=A0 =C2=A0 From: Mohammad Tariq [mailto: dontariq@gmail.com=20 ] Sent: Tuesday, April 30, 2013 3:41 PM=20 To: Kevin Burton Subject: Re: Can't initialize cluster=20 =C2=A0 add the libraries to your class path and re-run the job. if it still=20 doesn't work, add all the libraries individually on the command line=20 while running the job.=20 Warm Regards,=20 Tariq=20 https://mtariq.jux.com/ =20 cloudfront.blogspot.com =20 =C2=A0 On Wed, May 1, 2013 at 2:01 AM, Kevin Burton < rkevinburton@charter.net=20 =20 > wrote:=20 Eliminating it then takes the first file in $HADOOP_HOME/*.jar as the=20 jar containing =E2=80=98main=E2=80=99 and I get another error:=20 =C2=A0=20 kevin@devUbuntu05:~/WordCount$ hadoop jar=20 $HADOOP_HOME/*.jar:$HADOOP_HOME/lib/* WordCount.jar=20 file:///home/kevin/WordCount/inputfile:///home/kevin/WordCount/output Exception in thread "main" java.io.IOException: Error opening job jar:=20 /usr/lib/hadoop/*.jar:/usr/lib/hadoop/lib/* =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.util.RunJar= .main(RunJar.java:135) Caused by: java.io.FileNotFoundException:=20 /usr/lib/hadoop/*.jar:/usr/lib/hadoop/lib/* (No such file or directory) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.zip.ZipFile.open(Na= tive Method) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.zip.ZipFile.(= ZipFile.java:214) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.zip.ZipFile.(= ZipFile.java:144) =C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0at java.util.jar.JarFile.(= JarFile.java:153) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.jar.JarFile.(= JarFile.java:90) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.util.RunJar= .main(RunJar.java:133) =C2=A0 From: Mohammad Tariq [mailto: dontariq@gmail.com=20 ] Sent: Tuesday, April 30, 2013 3:27 PM=20 To: Kevin Burton Subject: Re: Can't initialize cluster=20 =C2=A0 it was a typo..eliminate it=20 Warm Regards,=20 Tariq=20 https://mtariq.jux.com/ =20 cloudfront.blogspot.com =20 =C2=A0 On Wed, May 1, 2013 at 1:55 AM, Kevin Burton < rkevinburton@charter.net=20 =20 > wrote:=20 Stilll the =E2=80=98-=E2=80=98 is not accepted on my end.=20 =C2=A0=20 kevin@devUbuntu05:~/WordCount$ hadoop jar -=20 $HADOOP_HOME/*.jar:$HADOOP_HOME/lib/* WordCount.jar=20 file:///home/kevin/WordCount/inputfile:///home/kevin/WordCount/output Exception in thread "main" java.io.IOException : Error opening job jar:=20 - =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.util.RunJar= .main(RunJar.java:135) Caused by: java.io.FileNotFoundException: - (No such file or directory) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.zip.ZipFile.open(Na= tive Method) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.zip.ZipFile.(= ZipFile.java:214) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.zip.ZipFile.(= ZipFile.java:144) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.jar.JarFile.(= JarFile.java:153) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.jar.JarFile.(= JarFile.java:90) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.util.RunJar= .main(RunJar.java:133) =C2=A0 From: Mohammad Tariq [mailto: dontariq@gmail.com=20 ] Sent: Tuesday, April 30, 2013 3:23 PM=20 To: Kevin Burton Subject: Re: Can't initialize cluster=20 =C2=A0 Sorry. My bad. I forgot that you are using CDH. use this :=20 =C2=A0 hadoop jar - $HADOOP_HOME/*.jar:$HADOOP_HOME/lib/*=C2=A0WordCount.jar=C2=A0= =20 file:///home/kevin/WordCount/inputfile:///home/kevin/WordCount/output Warm Regards, Tariq https://mtariq.jux.com/ =20 cloudfront.blogspot.com =20 =C2=A0 On Wed, May 1, 2013 at 1:47 AM, Kevin Burton < rkevinburton@charter.net=20 =20 > wrote:=20 I look at what could be HADOOP_HOME. It looks like:=20 =C2=A0=20 kevin@devUbuntu05:~$ ls /usr/lib/hadoop/=20 bin=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0hadoop-au= th.jar=20 client=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hadoop-common-2.0.0-cdh4.2= .1.jar=20 client-0.20=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=20 hadoop-common-2.0.0-cdh4.2.1-tests.jar=20 cloudera=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hadoop-common.jar=20 etc=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0lib=20 hadoop-annotations-2.0.0-cdh4.2.1.jar=C2=A0 libexec=20 hadoop-annotations.jar=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 sbin=20 hadoop-auth-2.0.0-cdh4.2.1.jar=20 =C2=A0=20 But notice there isn=E2=80=99t a hadoop-core-*.jar. Have I got the right fo= lder?=20 =C2=A0=20 Also when I run the command as you suggested I immediately get an error=20 that =E2=80=98-=E2=80=98 doesn=E2=80=99t exist. If I eliminate the =E2=80= =98-=E2=80=98 then there has to be=20 someway to pass in the what looks like you are forming a CLASSPATH.=20 =C2=A0=20 From: Mohammad Tariq [mailto: dontariq@gmail.com=20 ] Sent: Tuesday, April 30, 2013 2:51 PM=20 To: Kevin Burton Subject: Re: Can't initialize cluster=20 =C2=A0 "HADOOP_HOME is depricated" is not a problem. Just a warning. Add this=20 line "export HADOOP_HOME_WARN_SUPPRESS=3Dtrue" into your hadoop-env.sh=20 file if you want to get rid of it. It's the path where your hadoop=20 folder, which contain all the binaries, is present.=20 =C2=A0 Use this command and see if it helps :=20 hadoop jar -=20 $HADOOP_HOME/hadoop-core-*.jar:$HADOOP_HOME/lib/*=C2=A0WordCount.jar=C2=A0= =20 file:///home/kevin/WordCount/inputfile:///home/kevin/WordCount/output =C2=A0 Assuming you are already inside=C2=A0 /home/kevin/WordCount/ Warm Regards, Tariq https://mtariq.jux.com/ =20 cloudfront.blogspot.com =20 =C2=A0 On Wed, May 1, 2013 at 1:01 AM, Kevin Burton < rkevinburton@charter.net=20 =20 > wrote:=20 The path to the jar is /home/kevin/WordCount/WorCount.jar=20 I have not set HADOOP_HOME because on an earlier installation (the=20 Apache release not the CDH distro) I kept getting a message that=20 HADOOP_HOME was deprecated. =C2=A0Should it be set? If so should its path b= e=20 the same as HADOOP_MAPRED_HOME?=20 The command that throws the exception is hadoop jar WordCount.jar=20 file:///home/kevin/WordCount/inputfile:///home/kevin/WordCount/output =C2=A0 Thanks again. =C2=A0 From: Mohammad Tariq [mailto: dontariq@gmail.com=20 ] Sent: Tuesday, April 30, 2013 2:19 PM=20 To: Kevin Burton Subject: Re: Can't initialize cluster=20 =C2=A0 Could you please answer these 3 questions :=20 1-Where is your jar located?(I mean the directory path)=20 2-Have you set=C2=A0=C2=A0HADOOP_HOME? If yes then what is the path?=20 3-What is the command which you are using?=20 Warm Regards,=20 Tariq=20 https://mtariq.jux.com/ =20 cloudfront.blogspot.com =20 =C2=A0 On Wed, May 1, 2013 at 12:42 AM, Kevin Burton < rkevinburton@charter.net=20 =20 > wrote:=20 Tariq,=20 =C2=A0=20 Thank you. I tried this and the summary of the map reduce job looks=20 like:=20 =C2=A0=20 13/04/30 14:02:35 INFO mapred.JobClient: Job complete:=20 job_201304301251_0004=20 13/04/30 14:02:35 INFO mapred.JobClient: Counters: 7=20 13/04/30 14:02:35 INFO mapred.JobClient:=C2=A0=C2=A0 Job Counters=20 13/04/30 14:02:35 INFO mapred.JobClient:=C2=A0=C2=A0=C2=A0=C2=A0 Failed map= tasks=3D1=20 13/04/30 14:02:35 INFO mapred.JobClient:=C2=A0=C2=A0=C2=A0=C2=A0 Launched m= ap tasks=3D27=20 13/04/30 14:02:35 INFO mapred.JobClient:=C2=A0=C2=A0=C2=A0=C2=A0 Rack-local= map tasks=3D27=20 13/04/30 14:02:35 INFO mapred.JobClient:=C2=A0=C2=A0=C2=A0=C2=A0 Total time= spent by all=20 maps in occupied slots (ms)=3D151904=20 13/04/30 14:02:35 INFO mapred.JobClient:=C2=A0=C2=A0=C2=A0=C2=A0 Total time= spent by all=20 reduces in occupied slots (ms)=3D0=20 13/04/30 14:02:35 INFO mapred.JobClient:=C2=A0=C2=A0=C2=A0=C2=A0 Total time= spent by all=20 maps waiting after reserving slots (ms)=3D0=20 13/04/30 14:02:35 INFO mapred.JobClient:=C2=A0=C2=A0=C2=A0=C2=A0 Total time= spent by all=20 reduces waiting after reserving slots (ms)=3D0=20 =C2=A0=20 But there were a number of exceptions thrown and it seemed to take=20 longer than just running it standalone (I should have at least 4=20 machines working on this). The exceptions are my main concern now:=20 =C2=A0=20 (there were quite a few)=20 . . . . .=20 13/04/30 14:02:27 INFO mapred.JobClient: Task Id :=20 attempt_201304301251_0004_m_000005_1, Status : FAILED=20 java.io.FileNotFoundException: File=20 file:/home/kevin/WordCount/input/hadoop-core-2.0.0-mr1-cdh4.2.1.jar does=20 not exist=20 . . . .=20 13/04/30 14:02:28 INFO mapred.JobClient: Task Id :=20 attempt_201304301251_0004_m_000006_1, Status : FAILED=20 java.io.FileNotFoundException: File=20 file:/home/kevin/WordCount/input/guava-11.0.2.jar does not exist=20 . . . .=20 13/04/30 14:02:28 INFO mapred.JobClient: Task Id :=20 attempt_201304301251_0004_m_000008_0, Status : FAILED=20 java.io.FileNotFoundException: File=20 file:/home/kevin/WordCount/input/zookeeper-3.4.5-cdh4.2.1.jar does not=20 exist=20 . . . . .=20 13/04/30 14:02:28 INFO mapred.JobClient: Task Id :=20 attempt_201304301251_0004_m_000001_2, Status : FAILED=20 java.io.FileNotFoundException: File=20 file:/home/kevin/WordCount/input/tools.jar does not exist=20 . . . . .=20 13/04/30 14:02:28 INFO mapred.JobClient: Task Id :=20 attempt_201304301251_0004_m_000000_2, Status : FAILED=20 java.io.FileNotFoundException: File=20 file:/home/kevin/WordCount/input/Websters.txt does not exist=20 . . . .=20 13/04/30 14:02:33 INFO mapred.JobClient: Task Id :=20 attempt_201304301251_0004_m_000002_2, Status : FAILED=20 java.io.FileNotFoundException: File=20 file:/home/kevin/WordCount/input/hadoop-hdfs-2.0.0-cdh4.2.1.jar does not=20 exist=20 . . . .=20 13/04/30 14:02:33 INFO mapred.JobClient: Task Id :=20 attempt_201304301251_0004_m_000004_2, Status : FAILED=20 java.io.FileNotFoundException: File=20 file:/home/kevin/WordCount/input/hadoop-common-2.0.0-cdh4.2.1.jar does=20 not exist=20 .=C2=A0 . . .=20 13/04/30 14:02:33 INFO mapred.JobClient: Task Id :=20 attempt_201304301251_0004_m_000003_2, Status : FAILED=20 java.io.FileNotFoundException: File=20 file:/home/kevin/WordCount/input/core-3.1.1.jar does not exist=20 =C2=A0=20 No output folder was created (probably because of the numerous errors).=20 =C2=A0=20 Kevin=20 =C2=A0=20 From: Mohammad Tariq [mailto: dontariq@gmail.com=20 ] Sent: Tuesday, April 30, 2013 1:32 PM To: Kevin Burton=20 Subject: Re: Can't initialize cluster=20 =C2=A0 Hello again Kevin,=20 =C2=A0 =C2=A0 =C2=A0 =C2=A0Good that you are making progress. This is happening be= cause when=20 you are running it as a hadoop job, it looks for the the files in HDFS=20 and when you run it as a job program it looks into the local FS. Use=20 this as your input in your code and see if it helps :=C2=A0=20 =C2=A0 file:///home/kevin/input =C2=A0 Warm Regards, Tariq https://mtariq.jux.com/ =20 cloudfront.blogspot.com =20 =C2=A0 On Tue, Apr 30, 2013 at 11:36 PM, Kevin Burton <=20 rkevinburton@charter.net=20 =20 > wrote:=20 We/I are/am making progress. Now I get the error:=20 =C2=A0=20 13/04/30 12:59:40 WARN mapred.JobClient: Use GenericOptionsParser for=20 parsing the arguments. Applications should implement Tool for the same.=20 13/04/30 12:59:40 INFO mapred.JobClient: Cleaning up the staging area=20 hdfs://devubuntu05:9000/data/hadoop/tmp/hadoop-mapred/mapred/staging/kevin/= .staging/job_201304301251_0003=20 13/04/30 12:59:40 ERROR security.UserGroupInformation:=20 PriviledgedActionException as:kevin (auth:SIMPLE)=20 cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input=20 path does not exist: hdfs://devubuntu05:9000/user/kevin/input=20 Exception in thread "main"=20 org.apache.hadoop.mapreduce.lib.input.InvalidInputException : Input path=20 does not exist: hdfs://devubuntu05:9000/user/kevin/input=20 =C2=A0=20 When I run it with java =E2=80=93jar the input and output is the local fold= er.=20 When running it with hadoop jar it seems to be expecting the folders=20 (input and output) to be on the HDFS file system. I am not sure why=20 these two methods of invocation don=E2=80=99t make the same file system=20 assumptions.=20 =C2=A0=20 It is=20 =C2=A0=20 hadoop jar WordCount.jar input output (which gives the above exception)=20 =C2=A0=20 versus=20 =C2=A0=20 java =E2=80=93jar WordCount.jar input output (which outputs the wordcount= =20 statistics to the output folder)=20 =C2=A0=20 This is run in the local /home/kevin/WordCount folder.=20 =C2=A0=20 Kevin=20 =C2=A0=20 From: Mohammad Tariq [mailto: dontariq@gmail.com=20 ] Sent: Tuesday, April 30, 2013 12:33 PM To: user@hadoop.apache.org=20 Subject: Re: Can't initialize cluster=20 =C2=A0=20 Set " HADOOP_MAPRED_HOME " in your hadoop-env.sh file and re-run the=20 job. See if it helps.=20 Warm Regards,=20 Tariq=20 https://mtariq.jux.com/ =20 cloudfront.blogspot.com =20 =C2=A0 On Tue, Apr 30, 2013 at 10:10 PM, Kevin Burton <=20 rkevinburton@charter.net=20 =20 > wrote:=20 To be clear when this code is run with =E2=80=98java =E2=80=93jar=E2=80=99 = it runs without=20 exception. The exception occurs when I run with =E2=80=98hadoop jar=E2=80= =99.=20 =C2=A0=20 From: Kevin Burton [mailto: rkevinburton@charter.net=20 =20 ] Sent: Tuesday, April 30, 2013 11:36 AM To: user@hadoop.apache.org=20 Subject: Can't initialize cluster=20 =C2=A0=20 I have a simple MapReduce job that I am trying to get to run on my=20 cluster. When I run it I get:=20 =C2=A0=20 13/04/30 11:27:45 INFO mapreduce.Cluster: Failed to use=20 org.apache.hadoop.mapred.LocalClientProtocolProvider due to error:=20 Invalid "mapreduce.jobtracker.address" configuration value for=20 LocalJobRunner : "devubuntu05:9001"=20 13/04/30 11:27:45 ERROR security.UserGroupInformation:=20 PriviledgedActionException as:kevin (auth:SIMPLE)=20 cause:java.io.IOException: Cannot initialize Cluster . Please check your=20 configuration for mapreduce.framework.name=20 and the correspond server addresses.=20 Exception in thread "main" java.io.IOException: Cannot initialize=20 Cluster. Please check your configuration for mapreduce.framework.name=20 and the correspond server addresses.=20 =C2=A0 My core-site.xml looks like: =C2=A0 =C2=A0 fs.default.name =20 =C2=A0 hdfs://devubuntu05:9000 =C2=A0 The name of the default file system. A URI whose scheme= =20 and authority determine the FileSystem implementation. =20 =C2=A0 So I am unclear as to why it is looking at devubuntu05:9001?=20 =C2=A0 Here is the code: =C2=A0 =C2=A0=C2=A0=C2=A0 public static void WordCount( String[] args )=C2=A0 thro= ws Exception {=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Configuration conf =3D new Confi= guration();=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 String[] otherArgs =3D new Gener= icOptionsParser(conf,=20 args).getRemainingArgs(); =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (otherArgs.length !=3D 2) { = =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 System.e= rr.println("Usage: wordcount ");=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 System.e= xit(2); =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Job job =3D new Job(conf, "word = count"); =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 job.setJarByClass(WordCount.clas= s); =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 job.setMapperClass(WordCount.Tok= enizerMapper.class);=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 job.setCombinerClass(WordCount.I= ntSumReducer.class);=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 job.setReducerClass(WordCount.In= tSumReducer.class);=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 job.setOutputKeyClass(Text.class= ); =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 job.setOutputValueClass(IntWrita= ble.class);=20 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=20 org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(job,=20 new Path(otherArgs[0])); =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=20 org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.setOutputPath(job,= =20 new Path(otherArgs[1])); =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 System.exit(job.waitForCompletio= n(true) ? 0 : 1);=20 =C2=A0 Ideas? =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ------=_Part_22869123_9723910.1367415747176 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
 I made these changes. The log files in= /var/log/hadoop-0.20-mapreduce do not reflect todays date so they are not = updated. The datanode log in /var/log/hadoop-hdfs only shows this warning:<= /FONT>

2013-05-01= 08:16:12,678 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem= connecting to server: devUbuntu05/172.16.26.68:9000

The entries in the log after tha= t (and there are many) all are INFO messages. Not more warnings or errors.<= /FONT>

The nameno= de log is just INFO messages.

The secondarynamenode log (on the node that is acting = as the secondary namenode) contains the following error exception:
2013-05-01 08:34= :16,431 ERROR org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Exc= eption in doCheckpoint
java.io.I= OException: Inconsistent checkpoint fields.
LV =3D -40 namespaceID =3D 1097063765 cTime =3D 0 ; clusterId = =3D CID-23b9f9c7-2c25-411f-8bd2-4d5c9d7c25a1 ; blockpoolId =3D BP-130634904= 6-172.16.26.68-1367256199559.
Ex= pecting respectively: -40; 1666002936; 0; CID-e3f6b811-c1b4-4778-a31e-14dea= 8b2cca8; BP-1964515668-172.16.26.68-1366403021016.
        at org.apac= he.hadoop.hdfs.server.namenode.CheckpointSignature.validateStorageInfo(Chec= kpointSignature.java:133)
 =        at org.apache.hadoop.hdfs.server.namen= ode.SecondaryNameNode.doCheckpoint(SecondaryNameNode.java:475)
=
The logs for the any o= f the nodes serving as the task tracker are out of date.

When I try to attach to th= e master with a browser on port 50070 to look at the HDFS the page responds= but when I try to connect to the master at port 50030 nothing is listening= so it seems the JobTracker is down.


So it seems from what I ca= n gather the reason that the JobTracker doesn't come up is because of the e= rror on the secondary name node. Do you understand how I can get rid of thi= s error?

Kevin

On = Tue, Apr 30, 2013 at 8:55 PM, Mohammad Tariq wrote:

 Are you s= ure about the case in=C2=A0 devuBuntu06
devuBuntu07
devuBuntu08


Is '= B' actually in upper case?


Also,=C2= =A0 mapred.local.dir=C2=A0 belongs to mapred-site.xml and=C2=A0 h= adoop.tmp.dir belongs to core-site.xml file.

Warm Regard= s, Tariq


On Wed, = May 1, 2013 at 7:09 AM, Kevin Burton < rkevinburton@charter.net> wrote:
Here are what I consider the most= applicable conf files. <= /A>
Again thank you for your help.
=C2=A0
Kevin
=C2=A0
From: Mohammad Tariq [mailto: = dontariq@gmail.com]
Sent: Tuesday, April 30, 2013 6:43 PM

To: Kevin Burton
Subject: Re: Can't initialize cluster
<= FONT FACE=3D"Verdana" SIZE=3D"3" COLOR=3D"#3333FF" LETTERSPACING=3D"0" KERN= ING=3D"0">
=C2=A0
Sorry for the late response Kevin. I was = away for a while. I saw the logs. There seems to be some problem with your = configuration or name resolution. Could you please show me your configurati= on files?
Warm Regards,
Tariq <= /FONT>
=C2=A0
On Wed, May 1, 2= 013 at 4:27 AM, Kevin Burton < rkevinburton@charter.net> wrote:
The JobTracker logs don=E2=80=99t seem to be updated: =
<= DIV ALIGN=3D"LEFT">=C2=A0
kevin@de= vUbuntu05:/etc/hadoop/conf$ cd /var/log/hadoop-0.20-mapreduce/
total 3028
-rw-rw-r= -- 1 mapred mapred=C2=A0 383838 Apr 30 15:11 hadoop-hadoop-jobtracker-devUb= untu05.log
=C2=A0 <= FONT COLOR=3D"#3333FF"><= /A>
But I have attached t= he last lines of the JT log along with the TT logs.
=C2=A0
Kevin
=C2=A0
Fro= m: Mohammad Tariq [mailto: = dontariq@gmail.com]
Sent: Tuesday, April 30, 2013 = 5:38 PM
To: Kevin Burt= on
Subject: Re: Can't in= itialize cluster
=C2=A0
sor= ry for the typo in the last mail. it was 50070 and not 9000..anyways..
it's correct..it shows only the slaves..can you show m= e the JT and TT logs?

Warm Regards, =
Tariq <= /A>
=C2=A0
On= Wed, May 1, 2013 at 4:00 AM, Kevin Burton < rkevinburton@charter.net> wrote= :
There is something running. Here is the mas= ter:
=C2=A0
kevin@devUbuntu05:~$ ps aux | grep hadoop
=C2=A0
hdfs=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 20= 04=C2=A0 0.3=C2=A0 4.4 1977848 178396 ?=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Sl=C2= =A0=C2=A0 16:59=C2=A0=C2=A0 0:07 /usr/lib/jvm/java-7-openjdk-amd64//bin/jav= a -Dproc_namenode -Xmx1000m -Dhadoop.log.dir=3D/var/log/hadoop-hdfs -Dhadoo= p.log.file=3Dhadoop-hdfs-namenode-devUbuntu05.log -Dhadoop.home.dir=3D/usr/= lib/hadoop -Dhadoop.id.str=3Dhdfs -Dhadoop.root.logger=3DINFO,RFA -Djava.li= brary.path=3D/usr/lib/hadoop/lib/native -Dhadoop.policy.file=3Dhadoop-polic= y.xml -Djava.net.preferIPv4Stack=3Dtrue -Dcom.sun.management.jmxremote -Dco= m.sun.management.jmxremote -Dcom.sun.management.jmxremote -Dhadoop.security= .logger=3DINFO,RFAS org.apache.hadoop.hdfs.server.namenode.NameNode
kevin=C2=A0=C2=A0=C2=A0=C2=A0 2743=C2=A0 0.0=C2=A0 0= .0=C2=A0=C2=A0 9384=C2=A0=C2=A0 924 pts/0=C2=A0=C2=A0=C2=A0 S+=C2=A0=C2=A0 = 17:28=C2=A0=C2=A0 0:00 grep --color=3Dauto hadoop <= A HREF=3D"javascript:parent.wgMail.openComposeWindow('rkevinburton@charter.= net')" TARGET=3D"_blank">
=C2=A0 =
From: Mohammad Tariq [ma= ilto: dontariq@gmail.com= ]
Sent: Tuesday, April 30, 2013 5:21 PM

To: Kevin Burton
Subject: Re: Can't initialize cluster
=C2=A0
if jps is showing just "= ;jps" as output that means nothing is running. you have one more way t= o cross check. point your web browser to=C2=A0
&quo= t;namenode_machine:9000" and see if the cluster is behaving properly. =
=C2=A0 =
if you don't mind, c= an we switch to chat??

Warm Regards,=
Tariq =
=C2=A0
On= Wed, May 1, 2013 at 3:48 AM, Kevin Burton < rkevinburton@charter.net> wrote= :
There must be some option to run jps becaus= e running jps on all machines give something like: =
=C2=A0 =
kevin@devUbuntu05:~/WordCount$ = jps
2398 Jps
=C2=A0
From: Mohammad Tariq [mailto: = dontariq@gmail.com]
Sent: Tuesday, April 30, 2013 5:11 PM

To: Kevin Burton
Subject: Re: Can't initialize cluster
<= FONT FACE=3D"Verdana" SIZE=3D"3" COLOR=3D"#000000" LETTERSPACING=3D"0" KERN= ING=3D"0">=C2=A0
Do jps and see which process is no= t running. Show me the corresponding log.

Warm Regards,
Tariq
=C2=A0
On Wed, May 1, 2013 at 3:36 AM, Kevin Burton < rkevinburton@charter.net<= /A>> wrote:
=C2=A0
From: Mohammad Tariq [mailto: dontariq@gmail.com]=
Sent: Tue= sday, April 30, 2013 4:45 PM

To: Kevin Burton
=C2=A0
Ok..things are getting complex..we'll make sure that y= ou cluster is running fine=C2=A0first. stop everything and restart all 5 da= emons and let me know if you are able to do it properly.

Warm Regards,
Tariq
cloudfront.blogspot.com<= A HREF=3D"http://cloudfront.blogspot.com" TARGET=3D"_blank">
=C2=A0
<= DIV ALIGN=3D"LEFT">On Wed, May 1, 2013 at 3:10 AM, Kevin Burton = < rkevinburton@char= ter.net> wrote:
This didn= =E2=80=99t seem to help. I get about 10 retries then (running the job):
=C2=A0
kevin@devUb= untu05:~/WordCount$ hadoop jar WordCount.jar fil= e:///home/kevin/WordCount/inputfile:///home/kevin/WordCount/output
. . . . .
Exception in thr= ead "main" java.net.ConnectException: Call From devUbuntu05/ 17= 2.16.26.68to devUbuntu05:9001 failed on connection exception= : java.net.ConnectException: Connection refused; For more details see:=C2= =A0 http://wiki.apache.org/hadoop/ConnectionR= efused
=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at sun.reflect.NativeConstructorAcc= essorImpl.newInstance0(Native Method)
=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 at sun.reflect.NativeConstructorAccessorImpl.newInstance(Na= tiveConstructorAccessorImpl.java:57) =
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(D= elegatingConstructorAccessorImpl.java:45) =
=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at java.lang.reflect.Constructor.newInstance(Constructor= .java:525)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.= hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:779)
=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.net.NetUtils.wrapExcept= ion(NetUtils.java:726)
<= DIV ALIGN=3D"LEFT">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at= org.apache.hadoop.ipc.Client.call(Client.java:1229)
=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.ipc.WritableRpcEngine$Invo= ker.invoke(WritableRpcEng
=C2=A0
But still no indication why = the JobTracker will not start.
=C2=A0
Sent: Tuesday, April 30, 2013 4:28 PM

<= /FONT>
To: Kevin Burton
<= DIV ALIGN=3D"LEFT">Subject: Re: Can't initialize cluster =
=C2=A0 =
Do this :
hadoop dfsadmin -safemode leave
=C2=A0
Then restart the daemons and re-run the jo= b.

Warm Regards,
Tariq =
=C2=A0 <= /A>
Now it looks like the JobTracker isn=E2=80=99t start at all an= d there doesn=E2=80=99t seem to be a log entry in /var/log/hadoop-0.20-mapr= educe to indicate the reason. I looked in /var/log/hadoop-hdfs and for the = namenode I found what looked like an exception :
=C2=A0
The reported blocks 0 needs additi= onal 4 blocks to reach the threshold 0.9990 of total blocks 4. Safe mode wi= ll be turned off automatically.
=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hdfs.server.namenode.FS= Namesystem.rollEditLog(FSNamesystem.java:4568)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.hdfs.serv= er.namenode.NameNodeRpcServer.rollEditLog(NameNodeRpcServer.java:742)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.= apache.hadoop.hdfs.protocolPB.NamenodeProtocolServerSideTranslatorPB.rollEd= itLog(NamenodeProtocolServerSideTranslatorPB.java:129)
<= FONT FACE=3D"Verdana" SIZE=3D"3" COLOR=3D"#000000" LETTERSPACING=3D"0" KERN= ING=3D"0">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.h= dfs.protocol.proto.NamenodeProtocolProtos$NamenodeProtocolService$2.callBlo= ckingMethod(NamenodeProtocolProtos.java:8762)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.ipc.Protob= ufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org= .apache.hadoop.ipc.RPC$Server.call(RPC.java:1002) <= A HREF=3D"javascript:parent.wgMail.openComposeWindow('rkevinburton@charter.= net')" TARGET=3D"_blank">
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.ipc.= Server$Handler$1.run(Server.java:1695)
=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.ipc.Server$Handle= r$1.run(Server.java:1691)
=C2=A0
Then I looked in the datanode log and I didn=E2=80=99t= see what looked like an exception or error. The secondary name node log lo= oked like:
=C2=A0
2013-04-29 14:11:25,465 ERROR org.apache.hadoop.hdfs.server.namenode.Sec= ondaryNameNode: Exception in doCheckpoint
java.net.ConnectException: Call From devUbuntu05/ = 172.16.26.68to devUbuntu05:9000 failed on connection exception: java.net.ConnectEx= ception: Connection refused; For more details see:=C2=A0 http://wiki.apache.org/hadoop/ConnectionRefused=
=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at sun.reflect.GeneratedConstructorAccessor4.newInstance= (Unknown Source)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at sun.r= eflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorA= ccessorImpl.java:45)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at j= ava.lang.reflect.Constructor.newInstance(Constructor.java:525)
=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.net.NetUtils.w= rapWithMessage(NetUtils.java:779)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:726= )
=C2=A0
=C2=A0 =
=C2=A0 =
From: Mohammad Ta= riq [mailto: dontariq@gmail.= com]
Sent: Tuesday, April 30, 2013 3:41 PM

<= /FONT>
To: Kevin Burton
<= DIV ALIGN=3D"LEFT">Subject: Re: Can't initialize cluster =
=C2=A0 =
add the libraries to= your class path and re-run the job. if it still doesn't work, add all the = libraries individually on the command line while running the job.

<= /FONT>
<= DIV ALIGN=3D"LEFT">Warm Regards,
Tariq <= FONT COLOR=3D"#3333FF">
=C2=A0
On Wed, May 1, 2013 at 2:01 AM, Kevi= n Burton < rkevinbu= rton@charter.net> wrote:
Eliminating it then takes the first file in $HADOOP_HOME/*.jar as the jar = containing =E2=80=98main=E2=80=99 and I get another error:
=C2=A0 =
kevin@devUbuntu05:~/Word= Count$ hadoop jar $HADOOP_HOME/*.jar:$HADOOP_HOME/lib/* WordCount.jar file:///home/kevin/WordCount/inputfile:///home/kevin/= WordCount/output
Exce= ption in thread "main" java.io.IOException: Error opening job jar= : /usr/lib/hadoop/*.jar:/usr/lib/hadoop/lib/*
=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 at org.apache.hadoop.util.RunJar.main(RunJar.java:135)
Caus= ed by: java.io.FileNotFoundException: /usr/lib/hadoop/*.jar:/usr/lib/hadoop= /lib/* (No such file or directory)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 at java.util.zip.ZipFile.open(Native Method)
=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at java.util.zip.ZipFile.<init>(ZipFile.java:214) =
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.zip.ZipFile.<in= it>(ZipFile.java:144)
=C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0at java= .util.jar.JarFile.<init>(JarFile.java:153)
=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at java.util.jar.JarFile.<init>(JarFile.java:90) <= FONT COLOR=3D"#3333FF">
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.util.RunJa= r.main(RunJar.java:133)
=C2=A0
=
From: Mohammad Tariq [mailto: dontariq@gmail.com]
Sent: Tuesday, April 30, 2013 3:27 PM

To: Kevin Burton
<= FONT FACE=3D"Verdana" SIZE=3D"3" COLOR=3D"#000000" LETTERSPACING=3D"0" KERN= ING=3D"0">Subject: Re: Can't initialize cluster
=C2=A0
it was a typo..eliminate it

Warm Regards,
Tariq <= FONT COLOR=3D"#3333FF">
=C2=A0
On Wed, May 1, 2013 at 1:55 AM, Kevi= n Burton < rkevinbu= rton@charter.net> wrote:
Stilll the =E2=80=98-=E2=80=98 is not accepted on my end.
=C2=A0 =
kevin@devUbuntu05:~/Word= Count$ hadoop jar - $HADOOP_HOME/*.jar:$HADOOP_HOME/lib/* WordCount.jar file:///home/kevin/WordCount/inputfile:///home/kevi= n/WordCount/output
Ex= ception in thread "main" java.io.IOException : Error opening j= ob jar: -
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hado= op.util.RunJar.main(RunJar.java:135) =
Caused by: java.io.FileNotFoundExcepti= on: - (No such file or directory)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 at java.util.zip.ZipFile.open(Native Method)
=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at java.util.zip.ZipFile.<init>(ZipFile.java:214) =
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.util.zip.ZipFile.<in= it>(ZipFile.java:144)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java= .util.jar.JarFile.<init>(JarFile.java:153)
=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 at java.util.jar.JarFile.<init>(JarFile.java:90) <= FONT COLOR=3D"#3333FF">
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.util.RunJa= r.main(RunJar.java:133)
=C2=A0
=
From: Mohammad Tariq [mailto: dontariq@gmail.com]
Sent: Tuesday, April 30, 2013 3:23 PM

To: Kevin Burton
<= FONT FACE=3D"Verdana" SIZE=3D"3" COLOR=3D"#000000" LETTERSPACING=3D"0" KERN= ING=3D"0">Subject: Re: Can't initialize cluster
=C2=A0
Sorry. My bad. I forgot that you are usi= ng CDH. use this : <= /FONT>
=C2=A0 <= A HREF=3D"javascript:parent.wgMail.openComposeWindow('dontariq@gmail.com')"= TARGET=3D"_blank">
hadoop jar - $HADOOP_HOME/*.jar:$HADOOP_HOME/lib/*=C2=A0WordCount.jar=C2= =A0 file:///home/kevin/WordCount/inputfile:///ho= me/kevin/WordCount/output


Warm Regards,
Tariq
https://mtariq.jux.com/=
=C2=A0
= On Wed, May 1, 2013 at 1:47 AM, Kevin Burton < rkevinburton@charter.net> wrote:
I look at what could be HADO= OP_HOME. It looks like: <= /A>
kevin@devUbuntu05:~$ ls /usr/lib/hadoop/
bin=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0hadoop-auth.jar
client=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 hadoop-common-2.0.0-cdh4.2.1.jar
client-0.20=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hadoop-common-2.0.0-cdh4.2.1-tests.jar=
=
cloudera=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hadoo= p-common.jar <= /FONT>
etc=C2=A0=C2=A0=C2=A0 =C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0lib
hadoop-a= nnotations-2.0.0-cdh4.2.1.jar=C2=A0 libexec
hadoop-annotations.jar=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 sbin =
hadoop-auth-2.0.0-cdh4.2.1.jar
=C2=A0 =
<= DIV ALIGN=3D"LEFT">But notice there isn=E2=80=99t a hadoop-core-= *.jar. Have I got the right folder?
=C2=A0 <= FONT COLOR=3D"#3333FF">
Also when I run the command as you suggested I= immediately get an error that =E2=80=98-=E2=80=98 doesn=E2=80=99t exist. I= f I eliminate the =E2=80=98-=E2=80=98 then there has to be someway to pass = in the what looks like you are forming a CLASSPATH.
=C2=A0 =
From: Mohammad Tariq [ma= ilto: dontariq@gmail.com= ]
Sent: Tuesday, April 30, 2013 2:51 PM

To: Kevin Burton
Subject: Re: Can't initialize cluster
=C2=A0
"HADOOP_HOME is deprica= ted" is not a problem. Just a warning. Add this line "export HADO= OP_HOME_WARN_SUPPRESS=3Dtrue" into your hadoop-env.sh file if you want= to get rid of it. It's the path where your hadoop folder, which contain al= l the binaries, is present.
Use this command and see if it helps :
hado= op jar - $HADOOP_HOME/hadoop-core-*.jar:$HADOOP_HOME/lib/*=C2=A0WordCount.j= ar=C2=A0 file:///home/kevin/WordCount/inputfile:= ///home/kevin/WordCount/output
<= FONT FACE=3D"Verdana" SIZE=3D"3" COLOR=3D"#000000" LETTERSPACING=3D"0" KERN= ING=3D"0">=C2=A0
Assuming you are already inside=C2=A0 /home/kevin/WordCoun= t/


Warm Regards,
Tariq
https://mtariq.jux.com/=
=C2=A0
= On Wed, May 1, 2013 at 1:01 AM, Kevin Burton < rkevinburton@charter.net> wrote:
The path to the jar is /home= /kevin/WordCount/WorCount.jar
I have not set = HADOOP_HOME because on an earlier installation (the Apache release not the = CDH distro) I kept getting a message that HADOOP_HOME was deprecated. =C2= =A0Should it be set? If so should its path be the same as HADOOP_MAPRED_HOM= E?
The command that throws the exception is h= adoop jar WordCount.jar file:///home/kevin/WordC= ount/inputfile:///home/kevin/WordCount/output
=C2=A0
Thanks again.
=C2=A0
From: Mohammad= Tariq [mailto: dontariq@gma= il.com]
Sent: Tuesday, April 30, 2013 2:19 PM
To: Kevin Burton
Subject: Re: Can't initialize cluste= r
=C2=A0
Could you please= answer these 3 questions :
2-Have you set= =C2=A0=C2=A0HADOOP_HOME? If yes then what is the path?
3-What is the command which you are using?

= Warm Regards,
Tariq
=C2=A0
On Wed, May 1, 2013 at 12:42 AM, Kevin Burton < rkevinburton@charter.net<= /U>> wrote:
Tariq,
=C2=A0
Thank you. I tri= ed this and the summary of the map reduce job looks like:
=C2=A0 <= /FONT>
13/04/30 14:02:35 INFO ma= pred.JobClient: Job complete: job_201304301251_0004
13/04/30 14:02:35 INFO mapred.JobClient: Counters: 7
13/04/30 14:02:35 INFO mapred.JobClient:=C2=A0=C2=A0 Job Cou= nters <= /DIV>
13/04/30 14:02:35 INFO mapred.JobClient= :=C2=A0=C2=A0=C2=A0=C2=A0 Failed map tasks=3D1
13/04/30 14:02:35 INFO mapred.JobClient:=C2=A0=C2=A0=C2=A0=C2=A0 Launched= map tasks=3D27
13/04/30 14:02:35 INFO mapre= d.JobClient:=C2=A0=C2=A0=C2=A0=C2=A0 Rack-local map tasks=3D27
13/04/30 14:02:35 INFO mapred.JobClient:=C2=A0=C2=A0= =C2=A0=C2=A0 Total time spent by all maps in occupied slots (ms)=3D151904 <= FONT COLOR=3D"#3333FF">
13/04/30 14:02:35 INFO mapred.JobClient:=C2=A0= =C2=A0=C2=A0=C2=A0 Total time spent by all reduces in occupied slots (ms)= =3D0
13/04/30 14:02:35 INFO mapred.JobClient:= =C2=A0=C2=A0=C2=A0=C2=A0 Total time spent by all maps waiting after reservi= ng slots (ms)=3D0
13/04/30 14:02:35 INFO mapr= ed.JobClient:=C2=A0=C2=A0=C2=A0=C2=A0 Total time spent by all reduces waiti= ng after reserving slots (ms)=3D0
=C2=A0
But there were a number of exceptions thrown and = it seemed to take longer than just running it standalone (I should have at = least 4 machines working on this). The exceptions are my main concern now: =
<= DIV ALIGN=3D"LEFT">=C2=A0
(there w= ere quite a few)
. . . . .
13/04/30 14:02:27 INFO mapred.JobClient: Task Id : attempt_2013= 04301251_0004_m_000005_1, Status : FAILED
java.io.FileNotFoundException: File file:/home/kevin/WordCount/input/hadoo= p-core-2.0.0-mr1-cdh4.2.1.jar does not exist
. . . .
13/04/30 14:02:28 INFO mapred.JobCli= ent: Task Id : attempt_201304301251_0004_m_000006_1, Status : FAILED
java.io.FileNotFoundException: File file:/home/kevi= n/WordCount/input/guava-11.0.2.jar does not exist <= A HREF=3D"javascript:parent.wgMail.openComposeWindow('rkevinburton@charter.= net')" TARGET=3D"_blank">
. . . .
13/04/30 14:02:28 INFO mapre= d.JobClient: Task Id : attempt_201304301251_0004_m_000008_0, Status : FAILE= D
java.io.FileNotFoundException: File file:/h= ome/kevin/WordCount/input/zookeeper-3.4.5-cdh4.2.1.jar does not exist
. . . . .
13/04/30= 14:02:28 INFO mapred.JobClient: Task Id : attempt_201304301251_0004_m_0000= 01_2, Status : FAILED
. . . . .
13/04/30= 14:02:28 INFO mapred.JobClient: Task Id : attempt_201304301251_0004_m_0000= 00_2, Status : FAILED
. . . .
13/04/30= 14:02:33 INFO mapred.JobClient: Task Id : attempt_201304301251_0004_m_0000= 02_2, Status : FAILED
. . . .
<= FONT FACE=3D"Verdana" SIZE=3D"3" COLOR=3D"#000000" LETTERSPACING=3D"0" KERN= ING=3D"0">13/04/30 14:02:33 INFO mapred.JobClient: Task Id : attempt_201304= 301251_0004_m_000004_2, Status : FAILED
java.= io.FileNotFoundException: File file:/home/kevin/WordCount/input/hadoop-comm= on-2.0.0-cdh4.2.1.jar does not exist
.=C2=A0 = . . . <= /DIV>
13/04/30 14:02:33 INFO mapred.JobClient= : Task Id : attempt_201304301251_0004_m_000003_2, Status : FAILED
java.io.FileNotFoundException: File file:/home/kevin/W= ordCount/input/core-3.1.1.jar does not exist
=C2=A0
No output folder was created (probabl= y because of the numerous errors).
=C2=A0 <= FONT COLOR=3D"#3333FF">
Kevin
=C2=A0 <= FONT COLOR=3D"#3333FF">
From: Mohammad Tariq [mailto: dontariq@gmail.com]
Sent: Tuesday, April 30, 2013 1:32 PM
To: Kevin Burton

Subject: Re: Can't initialize cluster
=C2=A0
Hello again Kevin, =
=C2=A0
=C2=A0 =C2=A0 =C2=A0Good that you are makin= g progress. This is happening because when you are running it as a hadoop j= ob, it looks for the the files in HDFS and when you run it as a job program= it looks into the local FS. Use this as your input in your code and see if= it helps :=C2=A0
=C2=A0
file:///home/kevin/input
= =C2=A0


Warm Regards,
Tariq
https://mtariq.jux.com= /<= /DIV>
=C2=A0
On Tue, Apr 30, 2013 at 11:36 PM, Kevin Burton < rkevinburton@charter.net<= /FONT>> wrote:
We/I are/am making progress= . Now I get the error:
13/04/30 12:59:40 WARN mapred.JobClient: Use GenericOptionsP= arser for parsing the arguments. Applications should implement Tool for the= same. =
13/04/30 12:59:40 INFO mapred.JobClien= t: Cleaning up the staging area hdfs://devubuntu05:9000/data/hadoop/tmp/had= oop-mapred/mapred/staging/kevin/.staging/job_201304301251_0003
13/04/30 12:59:40 ERROR security.UserGroupInformation:= PriviledgedActionException as:kevin (auth:SIMPLE) cause:org.apache.hadoop.= mapreduce.lib.input.InvalidInputException: Input path does not exist: hd= fs://devubuntu05:9000/user/kevin/input
Exception in thread "main" org.apache.hadoop.mapreduce.lib.input= .InvalidInputException : Input path does not exist: hdfs://devubuntu05:9= 000/user/kevin/input =
=C2=A0
When I run it with java =E2=80=93jar the input and out= put is the local folder. When running it with hadoop jar it seems to be exp= ecting the folders (input and output) to be on the HDFS file system. I am n= ot sure why these two methods of invocation don=E2=80=99t make the same fil= e system assumptions.
It is <= /FONT>
=C2=A0
hadoop jar WordCount.jar input output (which gives the above ex= ception)
=C2=A0
versus
=C2=A0
java =E2=80=93jar WordCount.jar input output (which outputs the wordcount = statistics to the output folder)
=C2=A0
This is run in the local /home/kevin/WordCount fol= der.
=C2=A0
Kevin =
=C2=A0
From: Mohammad Tariq [mailto: dontariq@gmail.com]
Sent: Tuesday, April = 30, 2013 12:33 PM
= Subject: Re: Can't initialize cluster
=C2=A0 =
Set " HADOOP_MAPRED_HOME &= quot; in your hadoop-env.sh file and re-run the job. See if it helps.

=
Warm Regards,
= Tariq
=C2=A0
On Tue, Apr = 30, 2013 at 10:10 PM, Kevin Burton < rkevinburton@charter.net> wrote:
To be clear when this code is run with =E2=80=98jav= a =E2=80=93jar=E2=80=99 it runs without exception. The exception occurs whe= n I run with =E2=80=98hadoop jar=E2=80=99.
=C2=A0
From: Kevin Burton [mailto: rkevinburton@charter.net= ]
Sent: Tuesday, April 30, 2013 11:36 AM
Subject: = Can't initialize cluster
I have a simple MapReduce job that I am trying to get to run o= n my cluster. When I run it I get:
=C2=A0
13/04/30 11:27:45 INFO mapreduce.Cluster: Failed to = use org.apache.hadoop.mapred.LocalClientProtocolProvider due to error: Inva= lid "mapreduce.jobtracker.address" configuration value for LocalJ= obRunner : "devubuntu05:9001"
13/04/3= 0 11:27:45 ERROR security.UserGroupInformation: PriviledgedActionException = as:kevin (auth:SIMPLE) cause:java.io.IOException: Cannot initialize Clus= ter . Please check your configuration for mapreduce.frame= work.nameand the correspond server addresses.
=C2=A0
My core-site.xml looks like:
=C2=A0
<property> =
=C2=A0 <name> = fs.default.name</name>
=
=C2=A0 <value>hdfs://devubuntu05:9000&= lt;/value>
=C2= =A0 <description>The name of the default file system. A URI whose sch= eme and authority determine the FileSystem implementation. </description= >
</property= >
=C2=A0 =
So I am unclear as to wh= y it is looking at devubuntu05:9001?
=C2=A0
<= FONT FACE=3D"Verdana" SIZE=3D"3" COLOR=3D"#000000" LETTERSPACING=3D"0" KERN= ING=3D"0">Here is the code:
<= FONT FACE=3D"Verdana" SIZE=3D"3" COLOR=3D"#000000" LETTERSPACING=3D"0" KERN= ING=3D"0">=C2=A0
=C2=A0=C2=A0=C2=A0 public static void WordCount( String[] args )=C2=A0 thr= ows Exception {
= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Configuration conf =3D new Confi= guration();
=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 String[] otherArgs =3D new GenericOpti= onsParser(conf, args).getRemainingArgs();
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (ot= herArgs.length !=3D 2) {
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 S= ystem.err.println("Usage: wordcount <in> <out>"); <= /A>
=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 }
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Job job =3D new Job(conf, "= ;word count");
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 job.setJarByClass(WordCount.cl= ass);
=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 job.setMapperClass(WordCount.TokenizerMapper= .class);
=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 job.setCombinerClass(WordCount.IntSumR= educer.class);
=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 job.setReducerClass(WordCount.IntSu= mReducer.class);
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 job.setOutputKeyClass(Text.clas= s);
=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 job.setOutputValueClass(IntWritable.class); =
=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 org.apache.hadoop.mapreduce.lib.input.FileInput= Format.addInputPath(job, new Path(otherArgs[0])); <= A HREF=3D"http://fs.default.name" TARGET=3D"_blank">
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.setOutputPath(job, = new Path(otherArgs[1]));
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 System.exit(job.waitForCo= mpletion(true) ? 0 : 1);
=C2=A0
Idea= s?
=C2=A0
=C2=A0
=C2=A0
=C2=A0
=C2=A0
=C2=A0
=C2= =A0
=C2=A0 <= /FONT>
=C2=A0
=C2=A0
=C2=A0
=C2=A0
=C2=A0

------=_Part_22869123_9723910.1367415747176--