hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Kling (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1752) Implement getFileBlockLocations in HarFilesystem
Date Mon, 22 Nov 2010 22:24:24 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12934635#action_12934635

Patrick Kling commented on MAPREDUCE-1752:

Mahadev/Nicholas, could one of you please have a look at this patch?

ant test-patch results:
     [exec] +1 overall.  
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec]     +1 tests included.  The patch appears to include 6 new or modified tests.
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9)
     [exec]     +1 release audit.  The applied patch does not increase the total number of
release audit warnings.
     [exec]     +1 system test framework.  The patch passed system test framework compile.

> Implement getFileBlockLocations in HarFilesystem
> ------------------------------------------------
>                 Key: MAPREDUCE-1752
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1752
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: harchive
>            Reporter: Dmytro Molkov
>            Assignee: Dmytro Molkov
>             Fix For: 0.22.0
>         Attachments: MAPREDUCE-1752.2.patch, MAPREDUCE-1752.3.patch, MR-1752.patch
> To efficiently run map reduce on the data that has been HAR'ed it will be great to actually
implement getFileBlockLocations for a given filename.
> This way the JobTracker will have information about data locality and will schedule tasks
> I believe the overhead introduced by doing lookups in the index files can be smaller
than that of copying data over the wire.
> Will upload the patch shortly, but would love to get some feedback on this. And any ideas
on how to test it are very welcome.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message