hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5069) add concrete common implementations of CombineFileInputFormat
Date Tue, 19 Mar 2013 05:21:16 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13606070#comment-13606070
] 

Sangjin Lee commented on MAPREDUCE-5069:
----------------------------------------

Hmm... The build is clean, and all the tests that I have added are green. However, it seems
that somehow more unit tests are run than before, many of which are failing. Prior to the
patch 210 unit tests run. After the patch, 464 unit tests run. I believe all the failing tests
are contained in that difference.

https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3431/testReport/
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3432/testReport/

I'm unsure why this patch causes more tests to run? Is hadoop-mapreduce-client-jobclient not
the right place to put the unit tests for combine file input format tests?
                
> add concrete common implementations of CombineFileInputFormat
> -------------------------------------------------------------
>
>                 Key: MAPREDUCE-5069
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5069
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv1, mrv2
>    Affects Versions: 2.0.3-alpha
>            Reporter: Sangjin Lee
>            Priority: Minor
>         Attachments: MAPREDUCE-5069-1.patch, MAPREDUCE-5069-2.patch, MAPREDUCE-5069.patch
>
>
> CombineFileInputFormat is abstract, and its specific equivalents to TextInputFormat,
SequenceFileInputFormat, etc. are currently not in the hadoop code base.
> These sound like very common need wherever CombineFileInputFormat is used, and different
folks would write the same code over and over to achieve the same goal. It sounds very natural
for hadoop to provide at least the text and sequence file implementations of the CombineFileInputFormat
class.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message