hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5146) LocalDirAllocator misses files on the local filesystem
Date Wed, 04 Feb 2009 18:17:59 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tsz Wo (Nicholas), SZE updated HADOOP-5146:
-------------------------------------------

    Attachment: 5146_20090204job.output.txt

> Nicholas, is this problem reproducible? From the code, it seems that the exception you
pointed out should not cause tasks/job to fail. 

Yes, this can be reporduced deterministically in Windows.  It seems that this is not a problem
in Linux.

>  Is there a possibility of anything else happening here, something like a disk failure?

No, the disk is working fine.


5146_20090204job.output.txt: the output for running PiEstimator.

> LocalDirAllocator misses files on the local filesystem
> ------------------------------------------------------
>
>                 Key: HADOOP-5146
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5146
>             Project: Hadoop Core
>          Issue Type: Bug
>    Affects Versions: 0.20.0
>            Reporter: Arun C Murthy
>            Assignee: Devaraj Das
>            Priority: Blocker
>             Fix For: 0.20.0
>
>         Attachments: 5146_20090204job.output.txt, localdirallocator.patch, localdirallocator.patch
>
>
> For some reason the LocalDirAllocator.getLocaPathToRead doesn't find files which are
present, extra logging shows:
> {noformat}
> 2009-01-30 06:43:32,312 INFO org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext:
in ifExists, /grid/2/arunc/mapred-local/taskTracker/archive/xxx.yyy.com/tera/in/_partition.lst
exists
> 2009-01-30 06:43:32,389 WARN org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext:
in getLocalPathToRead, taskTracker/archive/xxx.yyy.com/tera/in/_partition.lst doesn't exist
> 2009-01-30 06:43:32,390 WARN org.apache.hadoop.mapred.TaskRunner: attempt_200901300512_0007_m_000055_0
Child Error
>  org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/archive/xx.yyy.com/tera/in/_partition.lst
in any of the configured local directories
>          at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:388)
>          at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
>          at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:172)
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message