hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5478) TeraInputFormat unnecessarily defines its own FileSplit subclass
Date Sat, 24 Aug 2013 11:01:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749357#comment-13749357

Hudson commented on MAPREDUCE-5478:

FAILURE: Integrated in Hadoop-Yarn-trunk #311 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/311/])
MAPREDUCE-5478. TeraInputFormat unnecessarily defines its own FileSplit subclass (Sandy Ryza)
(sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517046)
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java

> TeraInputFormat unnecessarily defines its own FileSplit subclass
> ----------------------------------------------------------------
>                 Key: MAPREDUCE-5478
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5478
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: examples
>    Affects Versions: 2.1.0-beta
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>            Priority: Minor
>             Fix For: 2.1.1-beta
>         Attachments: MAPREDUCE-5478.patch
> TeraInputFormat defines its own TeraFileSplit subclass of FileSplit that adds a locations
field, which is already included in FileSplit.
> This is causing MR2 TeraSort to fail on MR1, which, for a System.arraycopy, requires
splits to be of the FileSplit class.  While nobody is promising that everything that runs
on MR2 should run on MR1, fixing this would be easy and make it possible to compare MR2 TeraSort
performance between MR1 and MR2.
> We should just get rid of TeraFileSplit and use FileSplit.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message