hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4264) DFSIO is failing on 500 nodes cluster
Date Thu, 25 Sep 2008 07:09:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634392#action_12634392
] 

Amareshwari Sriramadasu commented on HADOOP-4264:
-------------------------------------------------

TestDFSIO is creating files in DATA_DIR, which is not output directory for the mapreduce job.
If speculative execution is on for the test, two attempts of the task are trying to create
same file. This test should have speculative execution off. 

> DFSIO is failing on 500 nodes cluster
> -------------------------------------
>
>                 Key: HADOOP-4264
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4264
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: io, test
>    Affects Versions: 0.19.0
>            Reporter: Suman Sehgal
>
> On executing following command : 
> bin/hadoop jar ~/hadoop/hadoop-0.19.0-test.jar TestDFSIO -write -nrFiles 990 -fileSize
320     
> This error occurs:
> 08/09/24 06:15:03 INFO mapred.JobClient:  map 98% reduce 32%
> java.io.IOException: Job failed!
> 	at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1201)
> 	at org.apache.hadoop.fs.TestDFSIO.runIOTest(TestDFSIO.java:236)
> 	at org.apache.hadoop.fs.TestDFSIO.writeTest(TestDFSIO.java:218)
> 	at org.apache.hadoop.fs.TestDFSIO.main(TestDFSIO.java:354)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
> 	at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
> 	at org.apache.hadoop.test.AllTestDriver.main(AllTestDriver.java:77)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.util.RunJar.main(RunJar.java:165)
> 	at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> 	at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
> On looking at hadoop logs, It seems that file names are clashing
> 2008-09-24 06:21:41,618 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task
'attempt_200809240600_0005_m_000802_2_1222236048515' from 'tracker_xxxx/client x.x.x.x:xxxxx'
> 2008-09-24 06:21:41,627 INFO org.apache.hadoop.mapred.JobTracker: Adding task 'attempt_200809240600_0005_m_000802_4_1222236048515'
to tip task_200809240600_0005_m_000802, for tracker 'tracker_xxxx/client x.x.x.x:xxxxx'
> 2008-09-24 06:21:41,627 INFO org.apache.hadoop.mapred.JobInProgress: Choosing rack-local
task task_200809240600_0005_m_000802
> 2008-09-24 06:21:41,724 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_200809240600_0005_m_000900_2_1222236048515:
org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException:
failed to create file /benchmarks/TestDFSIO/io_data/test_io_20 for DFSClient_attempt_200809240600_0005_m_000900_2_1222236048515
on client client x.x.x.x, because this file is already being created by DFSClient_attempt_200809240600_0005_m_000900_0_1222236048515
on client x.x.x.x

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message