hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris K Wensel (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (HADOOP-5524) DiskChecker$DiskErrorException when 'reduce > reduce'
Date Sun, 05 Apr 2009 07:35:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695817#action_12695817
] 

Chris K Wensel edited comment on HADOOP-5524 at 4/5/09 12:34 AM:
-----------------------------------------------------------------

Was this issue resolved? Looks related to HADOOP-4963.

      was (Author: cwensel):
    Was this issue resolved?
  
> DiskChecker$DiskErrorException when 'reduce > reduce'
> -----------------------------------------------------
>
>                 Key: HADOOP-5524
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5524
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.1
>         Environment: hadoop-0.19.1; CentOS 5
>            Reporter: Leitao Guo
>
> We have  9900 maptasks and 60 reducetasks in the job. When all the other 59 reducetasks
have finished, the last reducetask runs so slow and finally finished after throwing out a
lot of DiskErrorExceptions.
> The following is the tasktracker log on which the reducetask is running. 
> 2009-03-18 14:39:52,025 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:39:57,028 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:00,695 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200903171026_0091_r_000030_0
0.977961% reduce > reduce
> 2009-03-18 14:40:02,032 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:07,036 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:12,040 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:17,045 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:22,050 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:27,054 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:32,058 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:37,062 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:42,066 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:47,136 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:52,140 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:57,143 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:41:02,147 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:41:06,760 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200903171026_0091_r_000030_0
0.9788374% reduce > reduce
> 2009-03-18 14:41:07,152 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:41:09,762 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200903171026_0091_r_000030_0
0.9788374% reduce > reduce
> 2009-03-18 14:41:12,158 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:41:17,162 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:41:22,168 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:41:27,172 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message