hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-451) Add a Split interface
Date Fri, 15 Dec 2006 23:21:23 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-451?page=all ]

Owen O'Malley updated HADOOP-451:
---------------------------------

    Attachment: input-split.patch

This patch makes the relevant changes:
  1. Introduces InputSplit
  2. Changes InputFormat to return InputSplit[] from getSplits
  3. Remove the FileSystem parameters to InputFormats since I was changing the interfaces
anyway.
  4. Add getProgress to RecordReaders to track progress.
  5. Change the InputSplit areValidInputDirectories to validateInput.
  6. Fixes javadoc warnings that have been introduced recently.

> Add a Split interface
> ---------------------
>
>                 Key: HADOOP-451
>                 URL: http://issues.apache.org/jira/browse/HADOOP-451
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.2
>            Reporter: Doug Cutting
>         Assigned To: Owen O'Malley
>             Fix For: 0.10.0
>
>         Attachments: input-split.patch
>
>
> The InputFormat interface has a method:
> FileSplit[] getSplits();
> This should change to:
> Split[] getSplits();
> The Split interface would look like:
> public interface Split extends Writable {
>   /** Returns a list of hosts that contain this split.
>        This is only used to optimize task placement, so this may be empty. */
>   String[] getLocations(FileSystem fs);
>   /** The relative, estimated cost of operating on this.  Typically the size of the data
in the split.
>        Used to prioritize tasks in a job (high-cost tasks are run first).  */
>    long getCost();
> }

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message