hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Seigel <ja...@tynt.com>
Subject Re: so many failures on reducers.
Date Mon, 02 May 2011 11:32:53 GMT
What are your permissions on your hadoop.tmp.dir ?

James

Sent from my mobile. Please excuse the typos.

On 2011-05-02, at 1:26 AM, Jun Young Kim <juneng603@gmail.com> wrote:

> hi, all.
>
> I got so many failures on a reducing step.
>
> see this error.
>
> java.io.IOException: Failed to delete earlier output of task: attempt_201105021341_0021_r_000001_0
>    at org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:157)
>    at org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:173)
>    at org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:173)
>    at org.apache.hadoop.mapred.FileOutputCommitter.commitTask(FileOutputCommitter.java:133)
>    at org.apache.hadoop.mapred.OutputCommitter.commitTask(OutputCommitter.java:233)
>    at org.apache.hadoop.mapred.Task.commit(Task.java:962)
>    at org.apache.hadoop.mapred.Task.done(Task.java:824)
>    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
>    at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
>    at java.security.AccessController.doPrivileged(Native Method)
>    at javax.security.auth.Subject.doAs(Subject.java:396)
>    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:742)
>    at org.apache.hadoop.mapred.C
>
>
> this error was happened after adopting MultipleTextOutputFormat class in my job.
> the job is producing thousands of different output files on a HDFS.
>
> anybody can guess reasons?
>
> thanks.
>
> --
> Junyoung Kim (juneng603@gmail.com)
>

Mime
View raw message