hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Milind Bhandarkar (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-5914) JobClient should use multiple volumes as hadoop.tmp.dir
Date Tue, 26 May 2009 17:45:45 GMT
JobClient should use multiple volumes as hadoop.tmp.dir
-------------------------------------------------------

                 Key: HADOOP-5914
                 URL: https://issues.apache.org/jira/browse/HADOOP-5914
             Project: Hadoop Core
          Issue Type: Improvement
          Components: mapred
    Affects Versions: 0.20.0
         Environment: All
            Reporter: Milind Bhandarkar


Currently, hadoop.tmp.dir configuration variable allows specification of only a single directory
to be used as scratch space. In particular, on the job launcher nodes with multiple volumes,
this fails the entire job if the tmp.dir is somehow unusable. When the job launcher nodes
have multiple volumes, the tmp space availability can be improved by using multiple volumes
(either randomly or in round-robin.) The code for choosing a volume from a comma-separated
list of multiple volumes is already there for mapred.local.dir etc. That needs to be used
by job client as well.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message