hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-830) Providing BZip2 splitting support for Text data
Date Fri, 11 Sep 2009 00:53:57 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753930#action_12753930

Hadoop QA commented on MAPREDUCE-830:

+1 overall.  Here are the results of testing the latest attachment 
  against trunk revision 813585.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 6 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit

    +1 core tests.  The patch passed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/59/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/59/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/59/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/59/console

This message is automatically generated.

> Providing BZip2 splitting support for Text data
> -----------------------------------------------
>                 Key: MAPREDUCE-830
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-830
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 0.21.0
>            Reporter: Abdul Qadeer
>            Assignee: Abdul Qadeer
>             Fix For: 0.21.0
>         Attachments: M830-2.patch, M830-3.patch, M830-4.patch, M830-4.patch, MapReduce-830-version1.patch
> HADOOP-4012 (https://issues.apache.org/jira/browse/HADOOP-4012) is providing support
to handle BZip2 compressed data such that the input compressed file is split at arbitrary
points.  This JIRA uses that functionality in LineRecordReader.  The benefit of this work
is that, if user provides compressed BZip2 Text data, it will be split by Hadoop and hence
will be processed by multiple mappers.  So BZip2 compressed data will be able to fully utilize
the cluster power.  Currently BZip2 compressed Text file goes to one mapper and is not split.
 So the enhancement in this JIRA provides splitting support  and a considerable performance

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message