Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 14A3E1000B for ; Wed, 28 Aug 2013 06:48:14 +0000 (UTC) Received: (qmail 22208 invoked by uid 500); 28 Aug 2013 06:48:08 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 21864 invoked by uid 500); 28 Aug 2013 06:47:57 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 21856 invoked by uid 99); 28 Aug 2013 06:47:55 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Aug 2013 06:47:55 +0000 X-ASF-Spam-Status: No, hits=1.8 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of hadoop.guy0303@gmail.com designates 209.85.216.182 as permitted sender) Received: from [209.85.216.182] (HELO mail-qc0-f182.google.com) (209.85.216.182) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Aug 2013 06:47:48 +0000 Received: by mail-qc0-f182.google.com with SMTP id k18so856835qcv.13 for ; Tue, 27 Aug 2013 23:47:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=9bW1K6mOTZf5m+7Sfp/H7je+/Y3F1Soy70qbKlcdSvs=; b=MXAoMedhvC9qtS4HleXrBTG2yNDPhCTJDk3cqrRuDNavSqDD3WFwaz03pxQBsrHOiS Y41ZheU8VW9yOkm0SPpT9J6rOWRGrR0cLUF0S4+OLR9NY7i0A0lT9M31qDAMZpWmEZNY 2tTTtiL/3qoWzIMyCBHPtuofui9nlXlFbNQKPuW4IQ3gGG8PQkAvKxan7npSh6m0j/Em hMun4ItSmimchtoO0gvJOKcQLD1wiZ/X6pklmJL53DgDvsDnoI2aiWGmUH2Tp7MPrwZR FGTYNqv9o3+2calcMXOtXH1zZ5zqAWZokHtKQQYikbTEjxgX+zSwjDFRWDv38taS/I7u Wxdg== MIME-Version: 1.0 X-Received: by 10.49.39.39 with SMTP id m7mr28348403qek.60.1377672447368; Tue, 27 Aug 2013 23:47:27 -0700 (PDT) Received: by 10.224.152.5 with HTTP; Tue, 27 Aug 2013 23:47:27 -0700 (PDT) In-Reply-To: References: Date: Wed, 28 Aug 2013 12:17:27 +0530 Message-ID: Subject: Re: Creation of symlink from failed From: Pradeep Singh To: Hadoop user mailing list Content-Type: multipart/alternative; boundary=047d7bdc1564d7a3bd04e4fc5c3d X-Virus-Checked: Checked by ClamAV on apache.org --047d7bdc1564d7a3bd04e4fc5c3d Content-Type: text/plain; charset=ISO-8859-1 Hi All , I Guess the problem might has something to do with the hadoop setting and permissions related to Symlink creation . *Creation of symlink from /windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0 to /tmp/hadoop-root/mapred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0 failed\.* Can anybody throw some light on this . Regards Pradeep Singh On Wed, Aug 28, 2013 at 12:37 AM, Pradeep Singh wrote: > Hi All , > I am trying to run a simple steaming command as mentioned below . > > bin/hadoop jar > /windows/Hadoop/hadoop-1.2.1/contrib/streaming/hadoop-streaming-1.2.1.jar > -input /usr/pradeep/input/'Good words' -output /usr/pradeep/output mapper > /bin/cat reducer /bin/wc -w > > I am getting below as message in the error file > > packageJobJar: [/tmp/hadoop-root/hadoop-unjar3249316336279467583/] [] > /tmp/streamjob1400561611088699684.jar tmpDir=null > 13/08/27 23:55:54 INFO util.NativeCodeLoader: Loaded the native-hadoop > library > 13/08/27 23:55:54 WARN snappy.LoadSnappy: Snappy native library not loaded > 13/08/27 23:55:54 INFO mapred.FileInputFormat: Total input paths to > process : 1 > 13/08/27 23:55:55 INFO streaming.StreamJob: getLocalDirs(): > [/tmp/hadoop-root/mapred/local] > 13/08/27 23:55:55 INFO streaming.StreamJob: Running job: > job_201308272335_0004 > 13/08/27 23:55:55 INFO streaming.StreamJob: To kill this job, run: > 13/08/27 23:55:55 INFO streaming.StreamJob: > /windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job > -Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004 > 13/08/27 23:55:55 INFO streaming.StreamJob: Tracking URL: > http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004 > 13/08/27 23:55:56 INFO streaming.StreamJob: map 0% reduce 0% > 13/08/27 23:56:21 INFO streaming.StreamJob: map 100% reduce 100% > 13/08/27 23:56:21 INFO streaming.StreamJob: To kill this job, run: > 13/08/27 23:56:21 INFO streaming.StreamJob: > /windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job > -Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004 > 13/08/27 23:56:21 INFO streaming.StreamJob: Tracking URL: > http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004 > 13/08/27 23:56:21 ERROR streaming.StreamJob: Job not successful. Error: > JobCleanup Task Failure, Task: task_201308272335_0004_m_000002 > 13/08/27 23:56:21 INFO streaming.StreamJob: killJob... > Streaming Command Failed! > > I checked the log and this is what i found : > MapAttempt TASK_TYPE="CLEANUP" TASKID="task_201308272335_0004_m_000002" > TASK_ATTEMPT_ID="attempt_201308272335_0004_m_000002_0" TASK_STATUS="FAILED" > FINISH_TIME="1377627971824" HOSTNAME="pradeep-pc" > ERROR="java\.lang\.Throwable: Child Error > at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:271) > Caused by: java\.io\.IOException: Creation of symlink from > /windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0 > to > /tmp/hadoop-root/mapred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0 > failed\. > at > org\.apache\.hadoop\.mapred\.TaskLog\.createTaskAttemptLogDir(TaskLog\.java:110) > at > org\.apache\.hadoop\.mapred\.DefaultTaskController\.createLogDir(DefaultTaskController\.java:71) > at > org\.apache\.hadoop\.mapred\.TaskRunner\.prepareLogFiles(TaskRunner\.java:316) > at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:228) > > Cant understand the reason for such error .Everything seems to be fine > .Would appreicate any help in this respect . > > Regards > Pradeep Singh > --047d7bdc1564d7a3bd04e4fc5c3d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
H= i All ,
I=A0 Guess the problem might has something to do with the hadoop= setting and permissions related to Symlink creation .
Creation of sy= mlink from /windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_20= 1308272335_0004/attempt_201308272335_0004_m_000002_0 to /tmp/hadoop-root/ma= pred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_00000= 2_0 failed\.

Can anybody throw some light on this .

Regards
Pradeep Singh

<= br>
On Wed, Aug 28, 2013 at 12:37 AM, Pradeep Sin= gh <hadoop.guy0303@gmail.com> wrote:
Hi All ,
I am trying to run a simple steam= ing command as mentioned below .

=A0bin/hadoop jar /windows/Hadoop/hadoop-1.2.1/contrib/streaming/hadoop= -streaming-1.2.1.jar -input /usr/pradeep/input/'Good words' -output= /usr/pradeep/output mapper /bin/cat reducer /bin/wc -w

I am getting below as messag= e in the error file

packageJobJar: [/tmp/hadoop-root/hadoop-unjar32= 49316336279467583/] [] /tmp/streamjob1400561611088699684.jar tmpDir=3Dnull<= br> 13/08/27 23:55:54 INFO util.NativeCodeLoader: Loaded the native-hadoop libr= ary
13/08/27 23:55:54 WARN snappy.LoadSnappy: Snappy native library not = loaded
13/08/27 23:55:54 INFO mapred.FileInputFormat: Total input paths = to process : 1
13/08/27 23:55:55 INFO streaming.StreamJob: getLocalDirs(): [/tmp/hadoop-ro= ot/mapred/local]
13/08/27 23:55:55 INFO streaming.StreamJob: Running job= : job_201308272335_0004
13/08/27 23:55:55 INFO streaming.StreamJob: To k= ill this job, run:
13/08/27 23:55:55 INFO streaming.StreamJob: /windows/Hadoop/hadoop-1.2.1/li= bexec/../bin/hadoop job=A0 -Dmapred.job.tracker=3Dlocalhost:9001 -kill job_= 201308272335_0004
13/08/27 23:55:55 INFO streaming.StreamJob: Tracking U= RL: http://localhost:50030/jobdetails.jsp?jobid=3Djo= b_201308272335_0004
13/08/27 23:55:56 INFO streaming.StreamJob:=A0 map 0%=A0 reduce 0%
13/08= /27 23:56:21 INFO streaming.StreamJob:=A0 map 100%=A0 reduce 100%
13/08/= 27 23:56:21 INFO streaming.StreamJob: To kill this job, run:
13/08/27 23= :56:21 INFO streaming.StreamJob: /windows/Hadoop/hadoop-1.2.1/libexec/../bi= n/hadoop job=A0 -Dmapred.job.tracker=3Dlocalhost:9001 -kill job_20130827233= 5_0004
13/08/27 23:56:21 INFO streaming.StreamJob: Tracking URL: http://localhost:50030/jobdetails.jsp?jobid=3Djob_201308272335_0004
13/08/27 23:56:21 ERROR streaming.StreamJob: Job not successful. Error: Job= Cleanup Task Failure, Task: task_201308272335_0004_m_000002
13/08/27 23:56:21 INFO streaming.StreamJob: killJob...
Streaming Command= Failed!

I checked the log and this is what i found :
MapAttempt TASK_TYPE=3D&qu= ot;CLEANUP" TASKID=3D"task_201308272335_0004_m_000002" TASK_= ATTEMPT_ID=3D"attempt_201308272335_0004_m_000002_0" TASK_STATUS= =3D"FAILED" FINISH_TIME=3D"1377627971824" HOSTNAME=3D&q= uot;pradeep-pc" ERROR=3D"java\.lang\.Throwable: Child Error
=A0=A0=A0 at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:= 271)
Caused by: java\.io\.IOException: Creation of symlink from /windows= /Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/att= empt_201308272335_0004_m_000002_0 to /tmp/hadoop-root/mapred/local/userlogs= /job_201308272335_0004/attempt_201308272335_0004_m_000002_0 failed\.
=A0=A0=A0 at org\.apache\.hadoop\.mapred\.TaskLog\.createTaskAttemptLogDir(= TaskLog\.java:110)
=A0=A0=A0 at org\.apache\.hadoop\.mapred\.DefaultTask= Controller\.createLogDir(DefaultTaskController\.java:71)
=A0=A0=A0 at or= g\.apache\.hadoop\.mapred\.TaskRunner\.prepareLogFiles(TaskRunner\.java:316= )
=A0=A0=A0 at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:= 228)

Can= t understand the reason for such error .Everything seems to be fine .Would = appreicate any help in this respect .

Regards
Pradeep Singh

--047d7bdc1564d7a3bd04e4fc5c3d--