Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 22A3410B91 for ; Thu, 7 Nov 2013 16:29:41 +0000 (UTC) Received: (qmail 26505 invoked by uid 500); 7 Nov 2013 16:29:34 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 26364 invoked by uid 500); 7 Nov 2013 16:29:31 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 26355 invoked by uid 99); 7 Nov 2013 16:29:29 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Nov 2013 16:29:29 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of indrashish@ufl.edu designates 128.227.74.220 as permitted sender) Received: from [128.227.74.220] (HELO smtp.ufl.edu) (128.227.74.220) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Nov 2013 16:29:21 +0000 X-UFL-GatorLink-Authenticated: authenticated as glwm-service () with LOGIN from 10.241.70.47 Received: from webmail.ufl.edu (webmail-prod05.osg.ufl.edu [10.241.70.47]) (authenticated bits=0) by smtp.ufl.edu (8.13.8/8.13.8/3.0.0) with ESMTP id rA7GSv9A024938 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Thu, 7 Nov 2013 11:28:58 -0500 Received: from [10.227.56.25] by webmail.ufl.edu with HTTP (HTTP/1.1 POST); Thu, 07 Nov 2013 11:28:57 -0500 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=_ab772b3ab72b6ed6ec1c9291df371b4d" Date: Thu, 07 Nov 2013 11:28:57 -0500 From: "Basu,Indrashish" To: Subject: Re: Error while running Hadoop Source Code In-Reply-To: References: <2dc251abf0d49b28fcec6fedf5ba55ca@ufl.edu> <5d91d5bc1ed9e19b20a91c68ec239087@ufl.edu> <24AA4657-6F46-4E58-AC5E-CB5535E90280@hortonworks.com> <8f8276f069db32d3adec91724a984344@ufl.edu> <282a6c98d3066587a19b6bc26bafb3ed@ufl.edu> Message-ID: <2291c55b0e828b9c4133e2a2646a258d@ufl.edu> X-Sender: indrashish@ufl.edu (indrashish) User-Agent: Roundcube Webmail/0.5.2 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8794,1.0.431,0.0.0000 definitions=2013-11-07_05:2013-11-07,2013-11-07,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=9 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1305240000 definitions=main-1311070116 X-Spam-Level: * X-UFL-Spam-Level: * X-Virus-Checked: Checked by ClamAV on apache.org --=_ab772b3ab72b6ed6ec1c9291df371b4d Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=UTF-8 Hi Vinod, Please find the below details requested: A) ALL OF THE TASKTRACKER LOG : Same what I have provided in the earlier email. B) THE TASK-LOGS : SYSLOG : 2013-11-06 16:37:21,331 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId= 2013-11-06 16:37:22,245 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 2 2013-11-06 16:37:22,301 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 100 2013-11-06 16:37:22,832 INFO org.apache.hadoop.mapred.MapTask: data buffer = 79691776/99614720 2013-11-06 16:37:22,833 INFO org.apache.hadoop.mapred.MapTask: record buffer = 262144/327680 2013-11-06 16:37:23,157 ERROR org.apache.hadoop.mapred.pipes.BinaryProtocol: java.io.EOFException at java.io.DataInputStream.readByte(DataInputStream.java:267) at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:298) at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:319) at org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:114) 2013-11-06 16:37:23,157 INFO org.apache.hadoop.mapred.pipes.Application: Aborting because of java.net.SocketException: Broken pipe at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113) at java.net.SocketOutputStream.write(SocketOutputStream.java:159) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126) at java.io.DataOutputStream.write(DataOutputStream.java:107) at org.apache.hadoop.mapred.pipes.BinaryProtocol.writeObject(BinaryProtocol.java:333) at org.apache.hadoop.mapred.pipes.BinaryProtocol.mapItem(BinaryProtocol.java:286) at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:92) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:363) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at org.apache.hadoop.mapred.Child.main(Child.java:170) 2013-11-06 16:37:23,284 WARN org.apache.hadoop.mapred.TaskTracker: Error running child java.io.IOException: pipe child exception at org.apache.hadoop.mapred.pipes.Application.abort(Application.java:191) at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:103) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:363) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by: java.net.SocketException: Broken pipe at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113) at java.net.SocketOutputStream.write(SocketOutputStream.java:159) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126) at java.io.DataOutputStream.write(DataOutputStream.java:107) at org.apache.hadoop.mapred.pipes.BinaryProtocol.writeObject(BinaryProtocol.java:333) at org.apache.hadoop.mapred.pipes.BinaryProtocol.mapItem(BinaryProtocol.java:286) at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:92) ... 3 more STDERR : Command:0received before authentication. Exiting.. STDOUT : cmd: [bash, -c, exec '/app/hadoop/tmp/mapred/local/taskTracker/archive/10.227.56.195bin/cpu-kmeans2D/cpu-kmeans2D' '0' < /dev/null 1>> /usr/lo cal/hadoop/hadoop-gpu-0.20.1/bin/../logs/userlogs/attempt_201311061630_0001_m_000001_0/stdout 2>> /usr/local/hadoop/hadoop-gpu-0.20.1/bin/../log s/userlogs/attempt_201311061630_0001_m_000001_0/stderr] deviceID: C) SPECIFIC TASKATTEMPT'S TASKATTEMTPID THAT IS FAILING : attempt_201311061630_0001_m_000000_0 attempt_201311061630_0001_m_000001_0 attempt_201311061630_0001_m_000002_0 attempt_201311061630_0001_m_000000_1 attempt_201311061630_0001_m_000001_1 attempt_201311061630_0001_m_000003_0 attempt_201311061630_0001_m_000000_2 attempt_201311061630_0001_m_000001_2 attempt_201311061630_0001_m_000000_3 attempt_201311061630_0001_m_000001_3 Kindly let me know in case of any more details required. Thanks again for your time and patience regarding this. Regards, Indrashish. On Wed, 6 Nov 2013 16:58:53 -0800, Vinod Kumar Vavilapalli wrote: > Don't see anything in the logs that you pasted. > Can you paste the following in say pastebin? > - All of the TaskTracker log > - The task-logs. These are syslog, stderr, stdout files for a specific TaskAttempt. > - And specific TaskAttempt's TaskAttemtpID that is failing. > > Thanks, > +Vinod > > On Nov 6, 2013, at 2:50 PM, Basu,Indrashish wrote: > >> Can anyone please assist regarding this ? >> >> Thanks in advance >> >> Regards, >> >> Indra >> >> On Wed, 06 Nov 2013 09:50:02 -0500, Basu,Indrashish wrote: >> >>> Hi Vinod, >>> >>> Thanks for your help regarding this. I checked the task logs, this is what it is giving as output. >>> >>> 2013-11-06 06:40:05,541 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201311060636_0001_m_1100862588 >>> >>> 2013-11-06 06:40:05,553 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201311060636_0001_m_1100862588 spawned. >>> >>> 2013-11-06 06:40:05,650 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201311060636_0001_m_-1960039766 >>> >>> 2013-11-06 06:40:05,651 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201311060636_0001_m_-1960039766 spawned. >>> >>> 2013-11-06 06:40:07,496 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201311060636_0001_m_1100862588 given task: attempt_201311060 >>> >>> 636_0001_m_000001_3 >>> >>> 2013-11-06 06:40:07,618 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201311060636_0001_m_-1960039766 given task: attempt_20131106 >>> >>> 0636_0001_m_000000_3 >>> >>> 2013-11-06 06:40:08,013 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu xx from status: false >>> >>> 2013-11-06 06:40:08,014 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false >>> >>> 2013-11-06 06:40:08,015 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu xx from status: false >>> >>> 2013-11-06 06:40:08,015 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false >>> >>> 2013-11-06 06:40:09,361 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311060636_0001_m_000001_3 0.0% >>> >>> 2013-11-06 06:40:09,735 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311060636_0001_m_000000_3 0.0% >>> >>> 2013-11-06 06:40:10,018 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201311060636_0001_m_1100862588 exited. Number of tasks it ran: 0 >>> >>> 2013-11-06 06:40:11,021 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201311060636_0001_m_-1960039766 exited. Number of tasks it ran: 0 >>> >>> 2013-11-06 06:40:11,442 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu xx from status: false >>> >>> 2013-11-06 06:40:11,442 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false >>> >>> 2013-11-06 06:40:11,443 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu xx from status: false >>> >>> 2013-11-06 06:40:11,443 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false >>> >>> 2013-11-06 06:40:13,021 INFO org.apache.hadoop.mapred.TaskRunner: attempt_201311060636_0001_m_000001_3 done; removing files. >>> >>> 2013-11-06 06:40:13,037 INFO org.apache.hadoop.mapred.TaskTracker: addCPUFreeSlot : current free slots : 3 >>> >>> 2013-11-06 06:40:14,025 INFO org.apache.hadoop.mapred.TaskRunner: attempt_201311060636_0001_m_000000_3 done; removing files. >>> >>> 2013-11-06 06:40:14,028 INFO org.apache.hadoop.mapred.TaskTracker: addCPUFreeSlot : current free slots : 4 >>> >>> 2013-11-06 06:40:14,476 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu xx from status: false >>> >>> 2013-11-06 06:40:14,477 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false >>> >>> 2013-11-06 06:40:14,477 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu xx from status: false >>> >>> 2013-11-06 06:40:14,477 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false >>> >>> 2013-11-06 06:40:14,894 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu : false >>> >>> 2013-11-06 06:40:14,900 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201311060636_0001_m_000002_0 task's >>> >>> state:UNASSIGNED >>> >>> 2013-11-06 06:40:14,902 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201311060636_0001_m_000002_0 on CPU >>> >>> 2013-11-06 06:40:14,904 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free CPU slots : 4 and trying to launch attempt_2013 >>> >>> 11060636_0001_m_000002_0 >>> >>> 2013-11-06 06:40:14,909 INFO org.apache.hadoop.mapred.TaskTracker: Received KillTaskAction for task: attempt_201311060636_0001_m_000000_3 >>> >>> 2013-11-06 06:40:14,920 INFO org.apache.hadoop.mapred.TaskTracker: Received KillTaskAction for task: attempt_201311060636_0001_m_000001_3 >>> >>> 2013-11-06 06:40:15,161 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201311060636_0001_m_164532908 >>> >>> 2013-11-06 06:40:15,162 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201311060636_0001_m_164532908 spawned. >>> >>> 2013-11-06 06:40:17,216 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201311060636_0001_m_164532908 given task: attempt_2013110606 >>> >>> 36_0001_m_000002_0 >>> >>> Regards, >>> >>> Indrashish >>> >>> On Tue, 5 Nov 2013 10:09:36 -0800, Vinod Kumar Vavilapalli wrote: >>> >>>> It seems like your pipes mapper is exiting before consuming all the input. Did you check the task-logs on the web UI? >>>> >>>> Thanks, >>>> +Vinod >>>> >>>> On Nov 5, 2013, at 7:25 AM, Basu,Indrashish wrote: >>>> >>>>> Hi, >>>>> >>>>> Can anyone kindly assist on this ? >>>>> >>>>> Regards, >>>>> Indrashish >>>>> >>>>> On Mon, 04 Nov 2013 10:23:23 -0500, Basu,Indrashish wrote: >>>>> >>>>>> Hi All, >>>>> >>>>>> Any update on the below post ? >>>>> solution as " The *nopipe* example needs more documentation. It is run with the InputFormat from >>>>> x; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;">src/test/org/apache/*hadoop*/mapred/*p >>>>> >>>>>> : 5px; width: 100%;">*WordCountInputFormat*.java >>>>> very specific input split probably be recoded *to* use libhdfs *too*, but t >>>>> blockquote> if you have nfs or something working across the cluster. " >>>>> blockquote style="padding-left: 5px; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;" >>>>> >>>>>> ding-left: 5px; border-left: #1010ff 2px solid; margin-l >>>>> th: 100%;">could elaborate a bit about the same as what needs to be done exactly. >>>>> >>>>>>> To mention, I am trying to run a sample KMeans algorithm on a GPU >>>>> blockquote style="padding-left: 5px; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;">usin >>>>> >>>>>> eft: 5px; width: 100%;">Thanks in advance. >>>>> g-left: 5px; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;">Regards, ndrashish. >>>>> eft: #1010ff 2px solid; margin-left: 5px; width: 100%;">On Thu, 31 Oct 2013 20:00:10 -0400, Basu,Indrashish wrote: : 100%;"> >>>>> id; margin-left: 5px; width: 100%;">Hi, I am trying to run a sample Hadoop GPU source >>>>> algorithm) M processor and getting the below error. Can anyone please >>>>> ckquote> 2px solid; marg >>>>> width: 100%;">rmr: cannot remove output: No such file or directory. uote sty >>>>> eft: 5px; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;">13/10/31 13:43:12 WARN mapred. >>>>> >>>>>> px; border- >>>>> 2px solid; margin-left: 5px; width: 100%;"> 13/10/31 13:43:12 INFO mapred.FileInputFormat: Total input paths to eft: 5px; width: 100%;"> >>>>>> >>>>>>> process : 1 >>>>> 5px; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;"> 13/10/31 13:43:13 INFO mapred.JobClient: >>>>> >>>>>> er-left: #1010ff 2px solid; margin-left: 5px; width: 100%;"> 13/10/31 13:43:14 >>>>> JobClient: map 0% reduce 0% attempt_201310311320_0001_m_000000_0, Status : FAILED java.io.IOException: pipe child exception >>>>> olid; margin-left: 5px; width: 100%;"> at org.apache.hadoop.mapred.pipes.Application.abort(Application.java:191) idth: 100%;"> >>>>>> >>>>>>> at >>>>> border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;"> org.apach >>>>> >>>>>> style="padding-left: 5px; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;"> >>>>> 5px; width: 100%;"> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:363) ;at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) >>>>> d; margin-left: 5px; width: 100%;"> at org.apac >>>>> >>>>>> 5px; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;"> Caused by >>>>> cketException: Broken pipe at java.net.SocketOutputStream.socketWrite(Soc >>>>> >>>>>> #1010ff 2px solid; margin-left: 5px; width: 100%;"> at java.net.Socke >>>>> .write(SocketOutputStream.java:159) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126) 100%;"> at java.io.DataOutputStream.write(DataO >>>>> ava:107) >>>>> ">org.apache.hadoop.mapred.pipes.BinaryProtocol.writeObject(BinaryProtocol.java:333) >>>>> >>>>>> at >>>>> padding-left: 5px; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;">org.apache.hadoop.mapred.pipes.BinaryProtocol.mapItem(BinaryProtocol.java:286) >>>>> >>>>>>> >>>>>> >>>>>>> at >>>>> olid; margin-left: 5px; width: 100%;"> org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:92) e="padding-left: 5px; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;"> ... 3 more >>>>> ote style="padding-left: 5px; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;"> attempt_201310311320_0001_m_000000_0: cmd: [bash, -c, exec '/app/hadoop/tmp/mapred/local/taskTracker/archive/10.227.56.195bin/cpu-kmeans2D/cpu-kmeans >>>>> te> < /dev/null 1>> >>>>> 1010ff 2px solid; margin-left: 5px; width: 100%;">/usr/local/hadoop/hadoop-gpu-0.20.1/bin/../logs/userlogs/attempt_201310311320_0001_m_000000_0/stdout 2>> /usr/local/hadoop/hadoop-gpu-0.20.1/bin/../logs/userlogs/ >>>>> px; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;"> Regards, >>>>> -- >>>>> Indrashish Basu >>>>> Graduate Student >>>>> Department o >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>> >>>> CONFIDENTIALITY NOTICE >>>> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. >>> >>> -- >>> >>> Indrashish Basu >>> Graduate Student >>> Department of Electrical and Computer Engineering >>> University of Florida >> >> -- >> >> Indrashish Basu >> Graduate Student >> Department of Electrical and Computer Engineering >> University of Florida > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Indrashish Basu Graduate Student Department of Electrical and Computer Engineering University of Florida --=_ab772b3ab72b6ed6ec1c9291df371b4d Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=UTF-8

 

Hi Vinod,

Please find the below details requested:

a) All of the TaskTracker Log= :

Same what I have provided in the earlier email.

b) The task-logs :

 syslog :

2013-11-06 16:37:21,331 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: I= nitializing JVM Metrics with processName=3DMAP, sessionId=3D
2013-11-0= 6 16:37:22,245 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 2
2013-11-06 16:37:22,301 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb= =3D 100
2013-11-06 16:37:22,832 INFO org.apache.hadoop.mapred.MapTask= : data buffer =3D 79691776/99614720
2013-11-06 16:37:22,833 INFO org= =2Eapache.hadoop.mapred.MapTask: record buffer =3D 262144/327680
2013-= 11-06 16:37:23,157 ERROR org.apache.hadoop.mapred.pipes.BinaryProtocol: jav= a.io.EOFException
    at java.io.DataInputStream.readBy= te(DataInputStream.java:267)
    at org.apache.hadoop= =2Eio.WritableUtils.readVLong(WritableUtils.java:298)
   &nb= sp;at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:319)    at org.apache.hadoop.mapred.pipes.BinaryProtocol$Upl= inkReaderThread.run(BinaryProtocol.java:114)

2013-11-06 16:37:23= ,157 INFO org.apache.hadoop.mapred.pipes.Application: Aborting because of j= ava.net.SocketException: Broken pipe
    at java.net.So= cketOutputStream.socketWrite0(Native Method)
    at jav= a.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
&nbs= p;   at java.net.SocketOutputStream.write(SocketOutputStream.java= :159)
    at java.io.BufferedOutputStream.flushBuffer(B= ufferedOutputStream.java:82)
    at java.io.BufferedOut= putStream.write(BufferedOutputStream.java:126)
    at j= ava.io.DataOutputStream.write(DataOutputStream.java:107)
   =  at org.apache.hadoop.mapred.pipes.BinaryProtocol.writeObject(BinaryPr= otocol.java:333)
    at org.apache.hadoop.mapred.pipes= =2EBinaryProtocol.mapItem(BinaryProtocol.java:286)
    = at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:92= )
    at org.apache.hadoop.mapred.MapTask.runOldMapper(= MapTask.java:363)
    at org.apache.hadoop.mapred.MapTa= sk.run(MapTask.java:307)
    at org.apache.hadoop.mapre= d.Child.main(Child.java:170)

2013-11-06 16:37:23,284 WARN org.ap= ache.hadoop.mapred.TaskTracker: Error running child
java.io.IOExceptio= n: pipe child exception
    at org.apache.hadoop.mapred= =2Epipes.Application.abort(Application.java:191)
    at= org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:103)=
    at org.apache.hadoop.mapred.MapTask.runOldMapper(M= apTask.java:363)
    at org.apache.hadoop.mapred.MapTas= k.run(MapTask.java:307)
    at org.apache.hadoop.mapred= =2EChild.main(Child.java:170)
Caused by: java.net.SocketException: Bro= ken pipe
    at java.net.SocketOutputStream.socketWrite= 0(Native Method)
    at java.net.SocketOutputStream.soc= ketWrite(SocketOutputStream.java:113)
    at java.net= =2ESocketOutputStream.write(SocketOutputStream.java:159)
   =  at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java= :82)
    at java.io.BufferedOutputStream.write(Buffered= OutputStream.java:126)
    at java.io.DataOutputStream= =2Ewrite(DataOutputStream.java:107)
    at org.apache= =2Ehadoop.mapred.pipes.BinaryProtocol.writeObject(BinaryProtocol.java:333)<= br />    at org.apache.hadoop.mapred.pipes.BinaryProtocol.ma= pItem(BinaryProtocol.java:286)
    at org.apache.hadoop= =2Emapred.pipes.PipesMapRunner.run(PipesMapRunner.java:92)
  = ;  ... 3 more

  stderr :

  Command:0received before authentication. Exiting..

 

  stdout :

  cmd: [bash, -c, exec '/app/hadoop/tmp/mapred/local/taskTracker/ar= chive/10.227.56.195bin/cpu-kmeans2D/cpu-kmeans2D' '0'  < /dev/null&= nbsp; 1>> /usr/lo
  cal/hadoop/hadoop-gpu-0.20.1/bin/../log= s/userlogs/attempt_201311061630_0001_m_000001_0/stdout 2>> /usr/local= /hadoop/hadoop-gpu-0.20.1/bin/../log
  s/userlogs/attempt_2013110= 61630_0001_m_000001_0/stderr]
  deviceID:

c) Specific TaskAttempt's Tas= kAttemtpID that is failing :

  attempt_201311061630_0001_m_000000_0

  attempt_201311061630_0001_m_000001_0 

  attempt_201311061630_0001_m_000002_0 

  attempt_201311061630_0001_m_000000_1 

  attempt_201311061630_0001_m_000001_1 

  attempt_201311061630_0001_m_000003_0

  attempt_201311061630_0001_m_000000_2 

  attempt_201311061630_0001_m_000001_2 

  attempt_201311061630_0001_m_000000_3 

  attempt_201311061630_0001_m_000001_3

 

Kindly let me know in case of any more details required.

Thanks again for your time and patience regarding this.

Regards,

Indrashish.

 

 

On Wed, 6 Nov 2013 16:58:53 -0800, Vinod Kumar Vavilapalli wrote:

Don't see anything in the logs that you pasted.
Can you paste the following in say pastebin?
 - All of the TaskTracker log
 - The task-logs. These are syslog, stderr, stdout files for a sp= ecific TaskAttempt.
 - And specific TaskAttempt's TaskAttemtpID that is failing.

Thanks,
+Vinod

On Nov 6, 2013, at 2:50 PM, Basu,Indrashish wrote:

 

Can anyone please assist regarding this ?

Thanks in advance

Regards,

Indra

 
 

On Wed, 06 Nov 2013 09:50:02 -0500, Basu,Indrashish wrote:

 

Hi Vinod,

Thanks for your help regarding this. I checked the task logs, this is wh= at it is giving as output.

 

2013-11-06 06:40:05,541 INFO org.apache.ha= doop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201311060636_0= 001_m_1100862588

2013-11-06 06:40:05,553 INFO org.apache.ha= doop.mapred.JvmManager: JVM Runner jvm_201311060636_0001_m_1100862588 spawn= ed.

2013-11-06 06:40:05,650 INFO org.apache.ha= doop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201311060636_0= 001_m_-1960039766

2013-11-06 06:40:05,651 INFO org.apache.ha= doop.mapred.JvmManager: JVM Runner jvm_201311060636_0001_m_-1960039766 spaw= ned.

2013-11-06 06:40:07,496 INFO org.apache.ha= doop.mapred.TaskTracker: JVM with ID: jvm_201311060636_0001_m_1100862588 gi= ven task: attempt_201311060

636_0001_m_000001_3

2013-11-06 06:40:07,618 INFO org.apache.ha= doop.mapred.TaskTracker: JVM with ID: jvm_201311060636_0001_m_-1960039766 g= iven task: attempt_20131106

0636_0001_m_000000_3

2013-11-06 06:40:08,013 INFO org.apache.ha= doop.mapred.TaskTracker: running on gpu xx from status: false

2013-11-06 06:40:08,014 INFO org.apache.ha= doop.mapred.TaskTracker: running on gpu xx from task: false

2013-11-06 06:40:08,015 INFO org.apache.ha= doop.mapred.TaskTracker: running on gpu xx from status: false

2013-11-06 06:40:08,015 INFO org.apache.ha= doop.mapred.TaskTracker: running on gpu xx from task: false

2013-11-06 06:40:09,361 INFO org.apache.ha= doop.mapred.TaskTracker: attempt_201311060636_0001_m_000001_3 0.0%

2013-11-06 06:40:09,735 INFO org.apache.ha= doop.mapred.TaskTracker: attempt_201311060636_0001_m_000000_3 0.0%

2013-11-06 06:40:10,018 INFO org.apache.ha= doop.mapred.JvmManager: JVM : jvm_201311060636_0001_m_1100862588 exited. Nu= mber of tasks it ran: 0

2013-11-06 06:40:11,021 INFO org.apache.ha= doop.mapred.JvmManager: JVM : jvm_201311060636_0001_m_-1960039766 exited. N= umber of tasks it ran: 0

2013-11-06 06:40:11,442 INFO org.apache.ha= doop.mapred.TaskTracker: running on gpu xx from status: false

2013-11-06 06:40:11,442 INFO org.apache.ha= doop.mapred.TaskTracker: running on gpu xx from task: false

2013-11-06 06:40:11,443 INFO org.apache.ha= doop.mapred.TaskTracker: running on gpu xx from status: false

2013-11-06 06:40:11,443 INFO org.apache.ha= doop.mapred.TaskTracker: running on gpu xx from task: false

2013-11-06 06:40:13,021 INFO org.apache.ha= doop.mapred.TaskRunner: attempt_201311060636_0001_m_000001_3 done; removing= files.

2013-11-06 06:40:13,037 INFO org.apache.ha= doop.mapred.TaskTracker: addCPUFreeSlot : current free slots : 3

2013-11-06 06:40:14,025 INFO org.apache.ha= doop.mapred.TaskRunner: attempt_201311060636_0001_m_000000_3 done; removing= files.

2013-11-06 06:40:14,028 INFO org.apache.ha= doop.mapred.TaskTracker: addCPUFreeSlot : current free slots : 4

2013-11-06 06:40:14,476 INFO org.apache.ha= doop.mapred.TaskTracker: running on gpu xx from status: false

2013-11-06 06:40:14,477 INFO org.apache.ha= doop.mapred.TaskTracker: running on gpu xx from task: false

2013-11-06 06:40:14,477 INFO org.apache.ha= doop.mapred.TaskTracker: running on gpu xx from status: false

2013-11-06 06:40:14,477 INFO org.apache.ha= doop.mapred.TaskTracker: running on gpu xx from task: false

2013-11-06 06:40:14,894 INFO org.apache.ha= doop.mapred.TaskTracker: running on gpu : false

2013-11-06 06:40:14,900 INFO org.apache.ha= doop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201311060= 636_0001_m_000002_0 task's

state:UNASSIGNED

2013-11-06 06:40:14,902 INFO org.apache.ha= doop.mapred.TaskTracker: Trying to launch : attempt_201311060636_0001_m_000= 002_0 on CPU

2013-11-06 06:40:14,904 INFO org.apache.ha= doop.mapred.TaskTracker: In TaskLauncher, current free CPU slots : 4 and tr= ying to launch attempt_2013

11060636_0001_m_000002_0

2013-11-06 06:40:14,909 INFO org.apache.ha= doop.mapred.TaskTracker: Received KillTaskAction for task: attempt_20131106= 0636_0001_m_000000_3

2013-11-06 06:40:14,920 INFO org.apache.ha= doop.mapred.TaskTracker: Received KillTaskAction for task: attempt_20131106= 0636_0001_m_000001_3

2013-11-06 06:40:15,161 INFO org.apache.ha= doop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201311060636_0= 001_m_164532908

2013-11-06 06:40:15,162 INFO org.apache.ha= doop.mapred.JvmManager: JVM Runner jvm_201311060636_0001_m_164532908 spawne= d.

2013-11-06 06:40:17,216 INFO org.apache.ha= doop.mapred.TaskTracker: JVM with ID: jvm_201311060636_0001_m_164532908 giv= en task: attempt_2013110606

36_0001_m_000002_0

 

Regards,

Indrashish

 

On Tue, 5 Nov 2013 10:09:36 -0800, Vinod Kumar Vavilapalli wrote:

It seems like your pipes mapper is exiting= before consuming all the input. Did you check the task-logs on the web UI?

Thanks,
+Vinod

On Nov 5, 2013, at 7:25 AM, Basu,Indrashish wrote:


Hi,

Can anyone kindly assist on this ?

Reg= ards,
Indrashish


On Mon, 04 Nov 2013 10:23:23 -0500, = Basu,Indrashish wrote:
Hi All,
Any update on the below post ?
I came across some old post regarding the = same issue. It explains the
solution as " The *nopipe* example needs m= ore documentation.  It
assumes that it is run with the InputForma= t from
src/test/org/apache/*hadoop*/mapred/*pipes= */
*WordCountInputFormat*.java, which has a v= ery specific input split
format. By running with a TextInputFormat,= it will send binary bytes
as the input split and won't work right. T= he *nopipe* example should
probably be recoded *to* use libhdfs *too*= , but that is more
complicated *to* get running as a unit tes= t. Also note that since the
C++ example is using local file reads, it = will only work on a cluster
if you have nfs or something working acros= s the cluster. "
I would need some more light on the above = explanation , so if anyone
could elaborate a bit about the same as wh= at needs to be done exactly.
To mention, I am trying to run a sample KM= eans algorithm on a GPU
using Hadoop.
Thanks in advance.
Regards,
Indrashish.
On Thu, 31 Oct 2013 20:00:10 -0400, Basu,I= ndrashish wrote:
Hi,
I am trying to run a sample Hadoop GPU sou= rce code (kmeans algorithm)
on an ARM processor and getting the below = error. Can anyone please
throw some light on this ?
rmr: cannot remove output: No such file or= directory.
13/10/31 13:43:12 WARN mapred.JobClient: N= o job jar file set.  User
classes may not be found. See JobConf(Clas= s) or
JobConf#setJar(String).
13/10/31 13:43:12 INFO mapred.FileInputFor= mat: Total input paths to
process : 1
13/10/31 13:43:13 INFO mapred.JobClient: R= unning job: job_201310311320_0001
13/10/31 13:43:14 INFO mapred.JobClient: &= nbsp;map 0% reduce 0%
13/10/31 13:43:39 INFO mapred.JobClient: T= ask Id :
attempt_201310311320_0001_m_000000_0, Stat= us : FAILED
java.io.IOException: pipe child exception<= /blockquote>
   at org.apache.hadoop.map= red.pipes.Application.abort(Application.java:191)
   at
org.apache.hadoop.mapred.pipes.PipesMapRun= ner.run(PipesMapRunner.java:103)
   at org.apache.hadoop.map= red.MapTask.runOldMapper(MapTask.java:363)
   at org.apache.hadoop.map= red.MapTask.run(MapTask.java:307)
   at org.apache.hadoop.map= red.Child.main(Child.java:170)
Caused by: java.net.SocketException: Broke= n pipe
   at java.net.SocketOutput= Stream.socketWrite0(Native Method)
   at java.net.SocketOutput= Stream.socketWrite(SocketOutputStream.java:113)
   at java.net.SocketOutput= Stream.write(SocketOutputStream.java:159)
   at java.io.BufferedOutpu= tStream.flushBuffer(BufferedOutputStream.java:82)
   at java.io.BufferedOutpu= tStream.write(BufferedOutputStream.java:126)
   at java.io.DataOutputStr= eam.write(DataOutputStream.java:107)
   at
org.apache.hadoop.mapred.pipes.BinaryProto= col.writeObject(BinaryProtocol.java:333)
   at
org.apache.hadoop.mapred.pipes.BinaryProto= col.mapItem(BinaryProtocol.java:286)
   at
org.apache.hadoop.mapred.pipes.PipesMapRun= ner.run(PipesMapRunner.java:92)
   ... 3 more
attempt_201310311320_0001_m_000000_0: cmd:= [bash, -c, exec
'/app/hadoop/tmp/mapred/local/taskTracker/= archive/10.227.56.195bin/cpu-kmeans2D/cpu-kmeans2D'
'0'  < /dev/null  1>>
/usr/local/hadoop/hadoop-gpu-0.20.1/bin/= =2E./logs/userlogs/attempt_201310311320_0001_m_000000_0/stdout
2>> /usr/local/hadoop/hadoop-gpu-0= =2E20.1/bin/../logs/userlogs/
Regards,

--
Indrashish Basu
Graduate Student
Department of Ele= ctrical and Computer Engineering
University of Florida

= CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the indivi= dual or entity to which it is addressed and may contain information that is= confidential, privileged and exempt from disclosure under applicable law= =2E If the reader of this message is not the intended recipient, you are he= reby notified that any printing, copying, dissemination, distribution, disc= losure or forwarding of this communication is strictly prohibited. If you h= ave received this communication in error, please contact the sender immedia= tely and delete it from your system. Thank You.
 
-- 
Indrashish Basu 
Graduate Student 
Department of El=
ectrical and Computer Engineering 
University of Florida

 
-- 
Indrashish Basu 
Graduate Student 
Department of Elec=
trical and Computer Engineering 
University of Florida


CONFIDENTIALITY NOTICE
NOTICE: This message is intended for = the use of the individual or entity to which it is addressed and may contai= n information that is confidential, privileged and exempt from disclosure u= nder applicable law. If the reader of this message is not the intended reci= pient, you are hereby notified that any printing, copying, dissemination, d= istribution, disclosure or forwarding of this communication is strictly pro= hibited. If you have received this communication in error, please contact t= he sender immediately and delete it from your system. Thank You.

 

-- 
Indrashish Basu 
Graduate Student 
Department of =
Electrical and Computer Engineering 
University of Florida

=
--=_ab772b3ab72b6ed6ec1c9291df371b4d--