hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-4565) MultiFileInputSplit can use data locality information to create splits
Date Fri, 31 Oct 2008 22:09:44 GMT
MultiFileInputSplit can use data locality information to create splits
----------------------------------------------------------------------

                 Key: HADOOP-4565
                 URL: https://issues.apache.org/jira/browse/HADOOP-4565
             Project: Hadoop Core
          Issue Type: Improvement
          Components: mapred
            Reporter: dhruba borthakur


The MultiFileInputFormat takes a set of paths and creates splits based on file sizes. Each
splits contains a few files an each split are roughly equal in size. It would be efficient
if we can extend this InputFormat to create splits such each all the blocks in one split and
either node-local or rack-local.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message