hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-2227) wrong usage of mapred.local.dir.minspacestart
Date Sun, 16 Dec 2007 21:25:43 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Devaraj Das updated HADOOP-2227:

    Status: Open  (was: Patch Available)

Some comments:
1) In LocalDirAllocator.java, 
  1.1)  isExists should probably be renamed to ifExists
  1.2) The implementation of isExists returns 'false' when IOException is encountered. I think
it should just catch the exception and continue looking.

2) In ReduceTask.java, shouldn't workDir be using the LocalDirAllocator.getLocalPathToRead
(like jobCacheDir)?

3) DistributedCache.java is a public class and so the public APIs that were there earlier
should still be there (maybe redirect the implementation to your new APIs).

4) In TaskTracker.java, getLocalPathForWrite is invoked for both jobfile and the jar file.
Is the second one required since you already account for the sizes in the first call and you
should probably use the same path that the first call returns.

> wrong usage of mapred.local.dir.minspacestart 
> ----------------------------------------------
>                 Key: HADOOP-2227
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2227
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.15.1
>            Reporter: Christian Kunz
>            Assignee: Amareshwari Sri Ramadasu
>            Priority: Critical
>             Fix For: 0.16.0
>         Attachments: patch-2227.txt, patch-2227.txt
> As I understand it, mapred.local.dir.minspacestart should be used to specify that a map
or reduce task requires a minimum disk space to be executed. When several disks are available,
and mapred.local.dir lists all of them, then the free space should be added up. But TaskTracker.enoughFreeSpace
returns false whenever one of the disks listed in mapred.local.dir has less than the space
specified by mapred.local.dir.minspacestart. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message